From auriocus at gmx.de Wed Feb 1 01:37:03 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 1 Feb 2017 07:37:03 +0100 Subject: Python3.6 tkinter bug? In-Reply-To: References: Message-ID: Am 01.02.17 um 00:02 schrieb MRAB: > On 2017-01-31 22:34, Christian Gollwitzer wrote: >>> .!frame.!checkbutton >>> .!frame.!checkbutton2 >>> .!frame2.!checkbutton >>> .!frame2.!checkbutton2 >> >> > Perhaps someone who knows Tcl and tk can tell me, but I notice that in > the first example, the second part of the widget names are unique, > whereas in the second example, the second part of the widget names are > the reused (both "!checkbutton" and "!checkbutton2" occur twice). It is indeed the reason, but it has some strange legacy cause: the default name for the checkbutton-linked variable is the name of the button inside the parent. Therefore creating a checkbutton has the side effect of creating a variable with the button's name. In this case, the first buttons in the frames are linked to a variable called "!checkbutton" and the other two are linked to "!checkbutton2". (a variable name in Tcl can be anything apart from the empty string). This can also be demonstrated by this Tcl script: package require Tk pack [frame .f1] pack [frame .f2] pack [checkbutton .f1.c1 -text "A" ] pack [checkbutton .f1.c2 -text "B" ] pack [checkbutton .f2.c1 -text "C" ] pack [checkbutton .f2.c2 -text "D" ] which is equivalent to the Python code above. Note that this surprising behaviour was corrected for the (modern) ttk widgets, so if "checkbutton" is replaced by "ttk::checkbutton", they are not any longer linked. In Python, that would be from tkinter import ttk ... w = ttk.Checkbutton() (Personally, I'm not using the legacy widgets any longer) > Do the names need to be: > > .!frame.!checkbutton > .!frame.!checkbutton2 > .!frame2.!checkbutton3 > .!frame2.!checkbutton4 Good question. Maybe there should be unique variable names? I.e., if the script is changed into package require Tk pack [frame .f1] pack [frame .f2] pack [checkbutton .f1.c1 -text "A" -variable v1] pack [checkbutton .f1.c2 -text "B" -variable v2] pack [checkbutton .f2.c1 -text "C" -variable v3] pack [checkbutton .f2.c2 -text "D" -variable v4] then they are also not linked. Christian From tmv at redhat.com Wed Feb 1 04:59:01 2017 From: tmv at redhat.com (Tiago M. Vieira) Date: Wed, 1 Feb 2017 09:59:01 +0000 Subject: The use of sys.exc_info for catching exceptions Message-ID: <20170201095901.pygsb6bwnvchcsmj@dev.localdomain> Hi, I've came to a problem where I want to keep backwards and forwards compatibility with an exception syntax. And I mean by backwards going further down to Python 2.5. I was pointed to this option from a stack overflow answer[1] that works forward and backwards, I rewrite the solution here: import sys try: pr.update() except (ConfigurationException,): e = sys.exc_info()[1] returnString = "%s %s" % (e.line, e.errormsg) I understand that there has been a discussion about deprecating sys.exc_info() on [1] and in section "Possible Future Compatible Changes" on [2]. Aside of deprecating this function, what could be other possible issues if I use the solution above? Performance overhead or any known case that could cause problems? [1] - http://stackoverflow.com/a/2513890? [2] - https://www.python.org/dev/peps/pep-3110/#open-issues [3] - https://www.python.org/dev/peps/pep-3134/ Thanks! -- Tiago -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 880 bytes Desc: not available URL: From josemsuarezsierra at gmail.com Wed Feb 1 05:54:46 2017 From: josemsuarezsierra at gmail.com (=?UTF-8?Q?Jos=C3=A9_Manuel_Su=C3=A1rez_Sierra?=) Date: Wed, 1 Feb 2017 02:54:46 -0800 (PST) Subject: doubt loading pages Message-ID: hello everyone, Im trying to make a program that takes an archive from pdb (for instance this link http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=5HXY after reading it I want it to save in a list only this part of the archive: MGSSHHHHHHSSGLVPRGSHMASMTGGQQMGRGSMPAETNEYLSRFVEYMTGERKSRYTIKEYRFLVDQFLSFMNKKPDEITPMDIERYKNFLAVKKRYSKTSQYLAIKAVKLFYKALDLRVPINLTPPKRPSHMPVYLSEDEAKRLIEAASSDTRMYAIVSVLAYTGVRVGELCNLKISDVDLQESIINVRSGKGDKDRIVIMAEECVKALGSYLDLRLSMDTDNDYLFVSNRRVRFDTSTIERMIRDLGKKAGIQKKVTPHVLRHTFATSVLRNGGDIRFIQQILGHASVATTQIYTHLNDSALREMYTQHRPRY I have written this: import urllib2 seq=raw_input("Introduce pdb code \n") seq = urllib2.urlopen("http://www.rcsb.org/pdb/files/fasta.txt?structureIdList="+seq) print seq.read() seq.close() My question is, how do I save this into a python list? Thank you! From __peter__ at web.de Wed Feb 1 06:37:02 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Feb 2017 12:37:02 +0100 Subject: doubt loading pages References: Message-ID: Jos? Manuel Su?rez Sierra wrote: > hello everyone, > Im trying to make a program that takes an archive from pdb (for instance > this link http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=5HXY > > after reading it I want it to save in a list only this part of the > archive: > > MGSSHHHHHHSSGLVPRGSHMASMTGGQQ...IRFIQQILGHASVATTQIYTHLNDSALREMYTQHRPRY > > I have written this: > > import urllib2 > > > seq=raw_input("Introduce pdb code \n") > > > > seq = > urllib2.urlopen( > "http://www.rcsb.org/pdb/files/fasta.txt?structureIdList="+seq) > print seq.read() > > > seq.close() > > > My question is, how do I save this into a python list? While you could cook up something yourself it's probably better to use an existing library like biopython. $ cat retrieve_fasta.py import urllib2 import Bio.SeqIO seq = raw_input("Introduce pdb code \n") seq = urllib2.urlopen( "http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=" + seq ) for record in Bio.SeqIO.parse(seq, "fasta"): seq_list = list(record.seq.tostring()) break # stop after the first iteration print seq_list $ python retrieve_fasta.py Introduce pdb code 5HXY ['M', 'G', 'S', 'S', 'H', 'H', 'H', 'H', 'H', 'H', 'S', 'S', 'G', 'L', 'V', ... 'R', 'Y'] See . From tjreedy at udel.edu Wed Feb 1 06:55:11 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 1 Feb 2017 06:55:11 -0500 Subject: Python3.6 tkinter bug? In-Reply-To: References: Message-ID: <0780ab80-b8de-90cb-4f78-7e1ee5611af7@udel.edu> On 2/1/2017 1:37 AM, Christian Gollwitzer wrote: > Am 01.02.17 um 00:02 schrieb MRAB: >> On 2017-01-31 22:34, Christian Gollwitzer wrote: >>>> .!frame.!checkbutton >>>> .!frame.!checkbutton2 >>>> .!frame2.!checkbutton >>>> .!frame2.!checkbutton2 >>> >>> >> Perhaps someone who knows Tcl and tk can tell me, but I notice that in >> the first example, the second part of the widget names are unique, >> whereas in the second example, the second part of the widget names are >> the reused (both "!checkbutton" and "!checkbutton2" occur twice). > > It is indeed the reason, but it has some strange legacy cause: the > default name for the checkbutton-linked variable is the name of the > button inside the parent. Therefore creating a checkbutton has the side > effect of creating a variable with the button's name. > > In this case, the first buttons in the frames are linked to a variable > called "!checkbutton" and the other two are linked to "!checkbutton2". > (a variable name in Tcl can be anything apart from the empty string). > This can also be demonstrated by this Tcl script: > > package require Tk > > pack [frame .f1] > pack [frame .f2] > > pack [checkbutton .f1.c1 -text "A" ] > pack [checkbutton .f1.c2 -text "B" ] > > pack [checkbutton .f2.c1 -text "C" ] > pack [checkbutton .f2.c2 -text "D" ] > > which is equivalent to the Python code above. > > Note that this surprising behaviour was corrected for the (modern) ttk > widgets, so if "checkbutton" is replaced by "ttk::checkbutton", they are > not any longer linked. In Python, that would be > > from tkinter import ttk > ... > w = ttk.Checkbutton() > > (Personally, I'm not using the legacy widgets any longer) Christian, could you repeat any relevant parts of your comments on the tracker, especially any ideas on how we might fix tkinter? https://bugs.python.org/issue29402 >> Do the names need to be: >> >> .!frame.!checkbutton >> .!frame.!checkbutton2 >> .!frame2.!checkbutton3 >> .!frame2.!checkbutton4 Serhiy considered that but, not knowing that this would cause a regression, we both liked numbering within parent better. There is a similar issue with radiobuttons on ttk.OptionMenus that existed *before* the 3.6 name changes. https://bugs.python.org/issue25684 So there seems to be a systematic issue with tk or how we are (mis)using it. > Good question. Maybe there should be unique variable names? I.e., if the > script is changed into package require Tk > > pack [frame .f1] > pack [frame .f2] > > pack [checkbutton .f1.c1 -text "A" -variable v1] > pack [checkbutton .f1.c2 -text "B" -variable v2] > > pack [checkbutton .f2.c1 -text "C" -variable v3] > pack [checkbutton .f2.c2 -text "D" -variable v4] > > then they are also not linked. -- Terry Jan Reedy From josemsuarezsierra at gmail.com Wed Feb 1 07:07:53 2017 From: josemsuarezsierra at gmail.com (=?UTF-8?Q?Jos=C3=A9_Manuel_Su=C3=A1rez_Sierra?=) Date: Wed, 1 Feb 2017 04:07:53 -0800 (PST) Subject: doubt loading pages In-Reply-To: References: Message-ID: El mi?rcoles, 1 de febrero de 2017, 11:55:11 (UTC+1), Jos? Manuel Su?rez Sierra escribi?: > hello everyone, > Im trying to make a program that takes an archive from pdb (for instance this link http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=5HXY > > after reading it I want it to save in a list only this part of the archive: > > MGSSHHHHHHSSGLVPRGSHMASMTGGQQMGRGSMPAETNEYLSRFVEYMTGERKSRYTIKEYRFLVDQFLSFMNKKPDEITPMDIERYKNFLAVKKRYSKTSQYLAIKAVKLFYKALDLRVPINLTPPKRPSHMPVYLSEDEAKRLIEAASSDTRMYAIVSVLAYTGVRVGELCNLKISDVDLQESIINVRSGKGDKDRIVIMAEECVKALGSYLDLRLSMDTDNDYLFVSNRRVRFDTSTIERMIRDLGKKAGIQKKVTPHVLRHTFATSVLRNGGDIRFIQQILGHASVATTQIYTHLNDSALREMYTQHRPRY > > I have written this: > > import urllib2 > > > seq=raw_input("Introduce pdb code \n") > > > > seq = urllib2.urlopen("http://www.rcsb.org/pdb/files/fasta.txt?structureIdList="+seq) > print seq.read() > > > seq.close() > > > My question is, how do I save this into a python list? > > Thank you! What I need is to convert the whole archive into a list, how could I do it? From __peter__ at web.de Wed Feb 1 07:29:26 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Feb 2017 13:29:26 +0100 Subject: doubt loading pages References: Message-ID: Jos? Manuel Su?rez Sierra wrote: > El mi?rcoles, 1 de febrero de 2017, 11:55:11 (UTC+1), Jos? Manuel Su?rez > Sierra escribi?: >> hello everyone, >> Im trying to make a program that takes an archive from pdb (for instance >> this link http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=5HXY >> >> after reading it I want it to save in a list only this part of the >> archive: >> >> MGSSHHHHHHSSG... >> >> I have written this: >> >> import urllib2 >> >> >> seq=raw_input("Introduce pdb code \n") >> >> >> >> seq = >> urllib2.urlopen("http://www.rcsb.org/pdb/files/fasta.txt?structureIdList="+seq) >> print seq.read() >> >> >> seq.close() >> >> >> My question is, how do I save this into a python list? >> >> Thank you! > > What I need is to convert the whole archive into a list, how could I do > it? Dunno. Perhaps $ cat retrieve_fasta2.py import urllib2 import Bio.SeqIO seq = raw_input("Introduce pdb code \n") seq = urllib2.urlopen( "http://www.rcsb.org/pdb/files/fasta.txt?structureIdList=" + seq ) seqs = [record.seq.tostring() for record in Bio.SeqIO.parse(seq, "fasta")] print seqs $ python retrieve_fasta2.py Introduce pdb code 5HXY ['MGS...PRY', 'MGS...PRY', 'MGS...PRY', 'MGS...PRY', 'MGS...PRY', 'MGS...PRY'] From me.on.nzt at gmail.com Wed Feb 1 10:10:39 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 07:10:39 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data Message-ID: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> # Give user the file requested url = "http://superhost.gr/data/files/%s" % realfile user, password = 'user', 'passwd' r = requests.get( url, auth = (user, password) ) # send auth unconditionally r.raise_for_status() ========================== How can i ASK the user for http auth data and store them isntead of giving them to the script? From ivo.bellinsalarin at gmail.com Wed Feb 1 10:11:09 2017 From: ivo.bellinsalarin at gmail.com (Ivo Bellin Salarin) Date: Wed, 01 Feb 2017 15:11:09 +0000 Subject: Python doesn't catch exceptions ? Message-ID: Hi all, I have a curious problem with Python exceptions. The following code doesn't catch HttpError: ``` from server.libs.googleapiclient.errors import HttpError [..] try: OAuth.backoffExec(request) return True except HttpError as e: return e.resp.status == 404 except Exception as e: import inspect import os logging.error("caught exception: {}, defined in {}. we are in {}".format( e.__class__.__name__, inspect.getfile(e.__class__), os.getcwd() )) logging.error("processed exception: {}, defined in {}.".format( HttpError.__name__, inspect.getfile(HttpError) )) logging.error('e is an HttpError? {}'.format(isinstance(e, HttpError))) ``` This code generates instead the messages: ``` ERROR 47.135[bigquery.py.create_table:362] caught exception: HttpError, defined in server/libs/googleapiclient/errors.pyc. we are in /Users/nilleb/dev/gae-sample-project/app ERROR 47.135[bigquery.py.create_table:366] processed exception: HttpError, defined in /Users/nilleb/dev/gae-sample-project/app/server/libs/googleapiclient/errors.pyc. ERROR 47.136[bigquery.py.create_table:368] e is an HttpError? False ``` I haven't joined the paths in the messages above, but they are identical if I do that manually: ``` /Users/nilleb/dev/gae-sample-project/app/server/libs/googleapiclient/errors.pyc /Users/nilleb/dev/gae-sample-project/app/server/libs/googleapiclient/errors.pyc ``` Any ideas about how to diagnostic what's going wrong? Thanks in advance, nilleb From stephane at wirtel.be Wed Feb 1 10:57:36 2017 From: stephane at wirtel.be (Stephane Wirtel) Date: Wed, 1 Feb 2017 16:57:36 +0100 Subject: Call for Volunteers at PythonFOSDEM 2017 Message-ID: <20170201155736.lfh57kqi3b27uoq5@sg1> # Introduction The Python Community will be represented during FOSDEM 2017 with the Python Devrooms. This year, we will have two devrooms, the first one for 150 people on Saturday and the second one for 450 people on Sunday, it's really cool because we had accepted 24 talks instead of 16. This is the official call for sessions for the [Python devroom](https://www.python-fosdem.org) at [FOSDEM 2017](https://fosdem.org/2017) from **4th** to **5th** February 2017. FOSDEM is the Free and Open source Software Developer's European Meeting, a free and non-commercial two-day week-end that offers open source contributors a place to meet, share ideas and collaborate. It's the biggest event in Europe with +5000 hackers, +400 speakers. For this edition, Python will be represented by its Community. If you want to discuss with a lot of Python Users, it's the place to be! But we have an issue, we need some volunteers because on the week-end, we will only be 4 volunteers. If you think you can help us, please fill this [Google Form](https://goo.gl/forms/IOmGbEgFLVnmUGjO2) Thank you so much for your help Stephane -- St?phane Wirtel - http://wirtel.be - @matrixise From cl at isbd.net Wed Feb 1 12:12:26 2017 From: cl at isbd.net (Chris Green) Date: Wed, 1 Feb 2017 17:12:26 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? Message-ID: I'm often hitting this problem, how does one find out what package to install to provide what a give import needs? Currently I'm modifying some code which has 'import gtk', I want to migrate from Python 2 to Python 3 if I can but at the moment the import fails in Python 3. There are dozens of packages in the Ubuntu repositories which *might* provide what I need I don't want to try them all! So, is there an easy way to find out? ... and while I'm here, can someone tell me what package I need? -- Chris Green ? From __peter__ at web.de Wed Feb 1 12:12:41 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 01 Feb 2017 18:12:41 +0100 Subject: Python doesn't catch exceptions ? References: Message-ID: Ivo Bellin Salarin wrote: > Hi all, > > I have a curious problem with Python exceptions. > > The following code doesn't catch HttpError: > ``` > from server.libs.googleapiclient.errors import HttpError > [..] > try: > OAuth.backoffExec(request) > return True > except HttpError as e: > return e.resp.status == 404 > except Exception as e: > import inspect > import os > logging.error("caught exception: {}, defined in {}. we are in > {}".format( > e.__class__.__name__, > inspect.getfile(e.__class__), > os.getcwd() > )) > logging.error("processed exception: {}, defined in {}.".format( > HttpError.__name__, > inspect.getfile(HttpError) > )) > logging.error('e is an HttpError? {}'.format(isinstance(e, > HttpError))) > ``` > > This code generates instead the messages: > ``` > ERROR 47.135[bigquery.py.create_table:362] caught exception: HttpError, > defined in server/libs/googleapiclient/errors.pyc. we are in > /Users/nilleb/dev/gae-sample-project/app > ERROR 47.135[bigquery.py.create_table:366] processed exception: HttpError, > defined in > /Users/nilleb/dev/gae-sample- project/app/server/libs/googleapiclient/errors.pyc. > ERROR 47.136[bigquery.py.create_table:368] e is an HttpError? False > ``` > > I haven't joined the paths in the messages above, but they are identical > if I do that manually: > > ``` > /Users/nilleb/dev/gae-sample- project/app/server/libs/googleapiclient/errors.pyc > /Users/nilleb/dev/gae-sample- project/app/server/libs/googleapiclient/errors.pyc > ``` > > Any ideas about how to diagnostic what's going wrong? Make sure your program starts from scratch. There may be a "helpful" reload() that leaves the server in an inconsistent state. Here's a demo: $ cat reload_demo.py import binhex from binhex import Error for message in ["first", "second"]: try: raise Error(message) except binhex.Error as err: print "caught", err reload(binhex) # this creates a new Error class which will not be caught $ python reload_demo.py caught first Traceback (most recent call last): File "reload_demo.py", line 6, in raise Error(message) binhex.Error: second From ian.g.kelly at gmail.com Wed Feb 1 12:29:06 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 1 Feb 2017 10:29:06 -0700 Subject: Python doesn't catch exceptions ? In-Reply-To: References: Message-ID: On Wed, Feb 1, 2017 at 8:11 AM, Ivo Bellin Salarin wrote: > This code generates instead the messages: > ``` > ERROR 47.135[bigquery.py.create_table:362] caught exception: HttpError, > defined in server/libs/googleapiclient/errors.pyc. we are in > /Users/nilleb/dev/gae-sample-project/app > ERROR 47.135[bigquery.py.create_table:366] processed exception: HttpError, > defined in > /Users/nilleb/dev/gae-sample-project/app/server/libs/googleapiclient/errors.pyc. > ERROR 47.136[bigquery.py.create_table:368] e is an HttpError? False > ``` > > I haven't joined the paths in the messages above, but they are identical if > I do that manually: The fact that the paths are two different strings to begin with although you're using the same function to produce them suggests that the module is being imported twice. Make sure that this directory isn't being included more than once in your sys.path. In particular note that Python adds the directory containing the script to the path (or '' if interactive) and since the working directory is a parent directory of the file in question I'd take a close look at that. From best_lay at yahoo.com Wed Feb 1 12:41:56 2017 From: best_lay at yahoo.com (Wildman) Date: Wed, 01 Feb 2017 11:41:56 -0600 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: Message-ID: On Wed, 01 Feb 2017 17:12:26 +0000, Chris Green wrote: > I'm often hitting this problem, how does one find out what package to > install to provide what a give import needs? > > Currently I'm modifying some code which has 'import gtk', I want to > migrate from Python 2 to Python 3 if I can but at the moment the > import fails in Python 3. > > There are dozens of packages in the Ubuntu repositories which *might* > provide what I need I don't want to try them all! So, is there an > easy way to find out? > > ... and while I'm here, can someone tell me what package I need? Try this: import gi gi.require_version('Gtk', '3.0') from gi.repository import Gtk -- GNU/Linux user #557453 The cow died so I don't need your bull! From pkpearson at nowhere.invalid Wed Feb 1 14:11:33 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 1 Feb 2017 19:11:33 GMT Subject: Python3 using requests to grab HTTP Auth Data References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> Message-ID: On Wed, 1 Feb 2017 07:10:39 -0800 (PST), ????? ?????? wrote: > # Give user the file requested > url = "http://superhost.gr/data/files/%s" % realfile > > user, password = 'user', 'passwd' > > r = requests.get( url, auth = (user, password) ) # send auth unconditionally > r.raise_for_status() >========================== > > How can i ASK the user for http auth data and store them isntead of > giving them to the script? Maybe like this? user = raw_input("User: ") password = raw_input("Password: ") -- To email me, substitute nowhere->runbox, invalid->com. From cl at isbd.net Wed Feb 1 14:15:13 2017 From: cl at isbd.net (Chris Green) Date: Wed, 1 Feb 2017 19:15:13 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: Message-ID: <1no9md-fbb.ln1@esprimo.zbmc.eu> Wildman wrote: > On Wed, 01 Feb 2017 17:12:26 +0000, Chris Green wrote: > > > I'm often hitting this problem, how does one find out what package to > > install to provide what a give import needs? > > > > Currently I'm modifying some code which has 'import gtk', I want to > > migrate from Python 2 to Python 3 if I can but at the moment the > > import fails in Python 3. > > > > There are dozens of packages in the Ubuntu repositories which *might* > > provide what I need I don't want to try them all! So, is there an > > easy way to find out? > > > > ... and while I'm here, can someone tell me what package I need? > > Try this: > > import gi > gi.require_version('Gtk', '3.0') > from gi.repository import Gtk > That works but it's a workaround rather than the proper way to do it isn't it? ... and doesn't it need an internet connection? -- Chris Green ? From rosuav at gmail.com Wed Feb 1 14:22:32 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Feb 2017 06:22:32 +1100 Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> Message-ID: On Thu, Feb 2, 2017 at 2:10 AM, ????? ?????? wrote: > # Give user the file requested > url = "http://superhost.gr/data/files/%s" % realfile > > user, password = 'user', 'passwd' > > r = requests.get( url, auth = (user, password) ) # send auth unconditionally > r.raise_for_status() > ========================== > > How can i ASK the user for http auth data and store them isntead of giving them to the script? You should use the input() function (called raw_input() in Python 2) for a user name, and the getpass module for the password: https://docs.python.org/3/library/getpass.html ChrisA From eeppeliteloop at gmail.com Wed Feb 1 14:28:27 2017 From: eeppeliteloop at gmail.com (Philippe Proulx) Date: Wed, 1 Feb 2017 14:28:27 -0500 Subject: Why doesn't module finalization delete names as expected? Message-ID: I'm developing a C module with the help of SWIG. My library manages objects with reference counting, much like Python, except that it's deterministic: there's no GC. I'm using Python 3.5.2. I create two Python objects like this: bakery = Bakery() bread = bakery.create_bread() Behind the scenes, the situation looks like this: +--------------------+ | UserBread obj (Py) | +----------^---+-----+ | : | : | : +------------------+ +---------+---V---+ | Bakery obj (lib) <----------------+ Bread obj (lib) | +--------^---+-----+ +--------^--------+ | : | | : | +--------+---V----+ +--------+-------+ | Bakery obj (Py) | | Bread obj (Py) | +---------^-------+ +------^---------+ | | | | + + bakery bread A pipe link means one "strong" reference and a colon link means one borrowed/weak reference. I have some ownership inversion magic for the Bakery lib. and Python objects to always coexist. So here it's pretty clear what can happen. I don't know which reference gets deleted first, but let's assume it's `bakery`. Then the situation looks like this: +--------------------+ | UserBread obj (Py) | +----------^---+-----+ | : | : | : +------------------+ +---------+---V---+ | Bakery obj (lib) <----------------+ Bread obj (lib) | +--------^---+-----+ +--------^--------+ : | | : | | +--------+---V----+ +--------+-------+ | Bakery obj (Py) | | Bread obj (Py) | +-----------------+ +------^---------+ | | + bread The Bakery Python object's __del__() drops the reference to its library object, but first its reference count is incremented during this call (so it's not really destroyed) and it's marked as only owned by the library object from now on. When `bread` gets deleted: 1. The Bread Python object's __del__() method gets called: the reference to its library object is dropped. 2. The Bread library object's destroy function drops its reference to the Bakery library object. 3. The Bakery library object's destroy function drops its reference to the Bakery Python object. 4. The Bakery Python object's __del__() method does nothing this time, since the object is marked as only owned by its library object (inverted ownership). 5. The Bread library object's destroy function then drops its reference to the UserBread Python object. In the end, everything is correctly destroyed and released. This also works if `bread` is deleted before `bakery`. My problem is that this works as expected when used like this: def func(): bakery = Bakery() bread = bakery.create_bread() if __name__ == '__main__': func() but NOTHING is destroyed when used like this: bakery = Bakery() bread = bakery.create_bread() That is, directly during the module's initialization. It works, however, if I delete `bread` manually: bakery = Bakery() bread = bakery.create_bread() del bread It also works with `bakery` only: bakery = Bakery() My question is: what could explain this? My guess is that my logic is correct since it works fine in the function call situation. It feels like `bread` is never deleted in the module initialization situation, but I don't know why: the only reference to the Bread Python object is this `bread` name in the module... what could prevent this object's __del__() method to be called? It works when I call `del bread` manually: I would expect that the module finalization does the exact same thing? Am I missing anything? Thanks, Phil From best_lay at yahoo.com Wed Feb 1 15:03:59 2017 From: best_lay at yahoo.com (Wildman) Date: Wed, 01 Feb 2017 14:03:59 -0600 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: On Wed, 01 Feb 2017 19:15:13 +0000, Chris Green wrote: > Wildman wrote: >> On Wed, 01 Feb 2017 17:12:26 +0000, Chris Green wrote: >> >> > I'm often hitting this problem, how does one find out what package to >> > install to provide what a give import needs? >> > >> > Currently I'm modifying some code which has 'import gtk', I want to >> > migrate from Python 2 to Python 3 if I can but at the moment the >> > import fails in Python 3. >> > >> > There are dozens of packages in the Ubuntu repositories which *might* >> > provide what I need I don't want to try them all! So, is there an >> > easy way to find out? >> > >> > ... and while I'm here, can someone tell me what package I need? >> >> Try this: >> >> import gi >> gi.require_version('Gtk', '3.0') >> from gi.repository import Gtk >> > That works but it's a workaround rather than the proper way to do it > isn't it? It is the proper way. This page helps explain it. http://askubuntu.com/questions/784068/what-is-gi-repository-in-python > ... and doesn't it need an internet connection? No. -- GNU/Linux user #557453 The cow died so I don't need your bull! From gordon at panix.com Wed Feb 1 15:48:54 2017 From: gordon at panix.com (John Gordon) Date: Wed, 1 Feb 2017 20:48:54 +0000 (UTC) Subject: Python3 using requests to grab HTTP Auth Data References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> Message-ID: In Peter Pearson writes: > > How can i ASK the user for http auth data and store them isntead of > > giving them to the script? > Maybe like this? > user = raw_input("User: ") > password = raw_input("Password: ") If it doesn't need to be interactive, you could require that the user supply a file in the current directory containing the username and password. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From me.on.nzt at gmail.com Wed Feb 1 15:48:57 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 12:48:57 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> Message-ID: ?? ???????, 1 ??????????? 2017 - 9:22:46 ?.?. UTC+2, ? ??????? Chris > You should use the input() function (called raw_input() in Python 2) > for a user name, and the getpass module for the password: i have just tried: # Give user the file requested url = "http://superhost.gr/data/files/%s" % realfile username = getpass.getuser() password = getpass.getpass() r = requests.get( url, auth = (username, password) )# ask user for authentication data r.raise_for_status() From me.on.nzt at gmail.com Wed Feb 1 15:51:52 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 12:51:52 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> Message-ID: <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> ?? ???????, 1 ??????????? 2017 - 9:22:46 ?.?. UTC+2, ? ??????? Chris Angelico ??????: > You should use the input() function (called raw_input() in Python 2) > for a user name, and the getpass module for the password: I have just tried =============================== # Give user the file requested url = "http://superhost.gr/data/files/%s" % realfile username = getpass.getuser() password = getpass.getpass() r = requests.get( url, auth = (username, password) ) # ask user for authentication data r.raise_for_status() =============================== as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window. Any idea why? From cl at isbd.net Wed Feb 1 16:29:00 2017 From: cl at isbd.net (Chris Green) Date: Wed, 1 Feb 2017 21:29:00 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: Wildman wrote: > On Wed, 01 Feb 2017 19:15:13 +0000, Chris Green wrote: > > > Wildman wrote: > >> On Wed, 01 Feb 2017 17:12:26 +0000, Chris Green wrote: > >> > >> > I'm often hitting this problem, how does one find out what package to > >> > install to provide what a give import needs? > >> > > >> > Currently I'm modifying some code which has 'import gtk', I want to > >> > migrate from Python 2 to Python 3 if I can but at the moment the > >> > import fails in Python 3. > >> > > >> > There are dozens of packages in the Ubuntu repositories which *might* > >> > provide what I need I don't want to try them all! So, is there an > >> > easy way to find out? > >> > > >> > ... and while I'm here, can someone tell me what package I need? > >> > >> Try this: > >> > >> import gi > >> gi.require_version('Gtk', '3.0') > >> from gi.repository import Gtk > >> > > That works but it's a workaround rather than the proper way to do it > > isn't it? > > It is the proper way. This page helps explain it. > > http://askubuntu.com/questions/784068/what-is-gi-repository-in-python > OK, thank you, what a strange way to do it. > > ... and doesn't it need an internet connection? > > No. > OK, no problem, but isn't it very non-portable? -- Chris Green ? From torriem at gmail.com Wed Feb 1 16:41:05 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 1 Feb 2017 14:41:05 -0700 Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: On 02/01/2017 01:51 PM, ????? ?????? wrote: > as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window. > > Any idea why? What you're describing is not something you can do with an interactive Python script. HTTP-level authentication is requested of your browser by the web server itself. On Apache there are numerous methods you can use. Individual users can use .htaccess directives to add authentication to a directory, for example. You'll need to learn about it: https://www.google.com/search?q=apache+http+authentication If you're using a framework like Django, there are mechanisms for checking the username and password against a Python method. Again, google for http authentication and whatever framework you're using. I once used a special python script that was called by an Apache module to verify users against a customized LDAP filter. Again, that involves server cooperation though a module. In general, the browser pops up the username and password box in response to a request from the web server. It's not something your CGI script can just do without some cooperation from the web server. From torriem at gmail.com Wed Feb 1 16:46:20 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 1 Feb 2017 14:46:20 -0700 Subject: How to know what to install (Ubuntu/Debian) for a given import? In-Reply-To: References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: On 02/01/2017 01:03 PM, Wildman via Python-list wrote: > > It is the proper way. This page helps explain it. > > http://askubuntu.com/questions/784068/what-is-gi-repository-in-python > >> ... and doesn't it need an internet connection? > > No. However the gi module provides access to GTK+3, and it's quite likely Chris's project requires GTK+2, and would probably take some work to port from GTK+2 to GTK+3, though in the long run it's worth the effort. Anyway the GTK+2 module for python is usually called pygtk in the repos. From me.on.nzt at gmail.com Wed Feb 1 16:51:38 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 13:51:38 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: ?? ???????, 1 ??????????? 2017 - 11:41:28 ?.?. UTC+2, ? ??????? Michael Torrie ??????: > On 02/01/2017 01:51 PM, ????? ?????? wrote: > > as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window. > > > > Any idea why? > > What you're describing is not something you can do with an interactive > Python script. HTTP-level authentication is requested of your browser > by the web server itself. On Apache there are numerous methods you can > use. Individual users can use .htaccess directives to add > authentication to a directory, for example. You'll need to learn about it: > https://www.google.com/search?q=apache+http+authentication > > If you're using a framework like Django, there are mechanisms for > checking the username and password against a Python method. Again, > google for http authentication and whatever framework you're using. > > I once used a special python script that was called by an Apache module > to verify users against a customized LDAP filter. Again, that involves > server cooperation though a module. > > In general, the browser pops up the username and password box in > response to a request from the web server. It's not something your CGI > script can just do without some cooperation from the web server. I used to have this workaround solution for triggering the web server to pop-up the HTTP Auth window print '''''' % file_requested and i have tried to read the the login auth name that user entered by using authuser = os.environ.get( 'REMOTE_USER', '????????' ) unfortunately it always failes to receive it that's why i'm trying to do the trick with the requests module. From best_lay at yahoo.com Wed Feb 1 18:20:09 2017 From: best_lay at yahoo.com (Wildman) Date: Wed, 01 Feb 2017 17:20:09 -0600 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: On Wed, 01 Feb 2017 21:29:00 +0000, Chris Green wrote: > Wildman wrote: >> On Wed, 01 Feb 2017 19:15:13 +0000, Chris Green wrote: >> >> > Wildman wrote: >> >> On Wed, 01 Feb 2017 17:12:26 +0000, Chris Green wrote: >> >> >> >> > I'm often hitting this problem, how does one find out what package to >> >> > install to provide what a give import needs? >> >> > >> >> > Currently I'm modifying some code which has 'import gtk', I want to >> >> > migrate from Python 2 to Python 3 if I can but at the moment the >> >> > import fails in Python 3. >> >> > >> >> > There are dozens of packages in the Ubuntu repositories which *might* >> >> > provide what I need I don't want to try them all! So, is there an >> >> > easy way to find out? >> >> > >> >> > ... and while I'm here, can someone tell me what package I need? >> >> >> >> Try this: >> >> >> >> import gi >> >> gi.require_version('Gtk', '3.0') >> >> from gi.repository import Gtk >> >> >> > That works but it's a workaround rather than the proper way to do it >> > isn't it? >> >> It is the proper way. This page helps explain it. >> >> http://askubuntu.com/questions/784068/what-is-gi-repository-in-python >> > OK, thank you, what a strange way to do it. I am sure there is a reason and I believe it has something to do with the fact that GTK3 is a recent addition. Things might change in the future. >> > ... and doesn't it need an internet connection? >> >> No. >> > OK, no problem, but isn't it very non-portable? I don't see why not. It should work on any system that has Python3 installed, at least that is my understanding. I'm sure someone will correct me if I'm wrong. OTOH if you want in insure 100% portability with any script, you can use pyinstaller. To install for Python2: pip install pyinstaller For Python3: pip3 install pyinstaller http://www.pyinstaller.org/ -- GNU/Linux user #557453 The cow died so I don't need your bull! From python at lucidity.plus.com Wed Feb 1 18:49:47 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 1 Feb 2017 23:49:47 +0000 Subject: How coding in Python is bad for you In-Reply-To: <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <355c105c-d51a-4c66-be66-dde41cba20c8@googlegroups.com> <87ab542f-0473-40c1-8a56-b41b397fb8b2@googlegroups.com> <588dff48$0$22142$c3e8da3$5496439d@news.astraweb.com> <00e3f05e-86b4-ea26-7211-39bebc6b4afe@lucidity.plus.com> <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> On 30/01/17 02:14, Steve D'Aprano wrote: > On Mon, 30 Jan 2017 10:52 am, Erik wrote: >> It would be even better if it was "else if not break:" to make the >> meaning clearer. > > break is not the only way to exit the for loop Fine - "else if not break or raise or return:", then ;) [that is not a serious suggestion] You're conflating two types of exit though. One is a way of exiting the innermost loop construct and nothing more. The others are ways of exiting a separate, outer construct (be that a try block or a function/method). I'm specifically talking about the interaction between 'break' and 'else'. The other things are at a different level. >> I would agree that it would be even better than that if >> it was "then if not break:" (apart from needing the new keyword ;)), as >> then the conditional aspect is explicit. > > But it isn't conditional. Yes it is. When one reads the code, the statements in the "else:" (or your theoretical "then:") block will be executed only if a "break" was not executed in the loop statements the "then:" is associated with. How is that NOT conditional? > Your syntax implies that the interpreter keeps > some sort of flag did_we_reach_the_end_of_the_loop_without_break or > something, and then it checks the state of that flag. There is no such > flag. You are correct and I never said there was - you're now arguing against a point that you have made and I didn't! I have written more than my fair share of assembly code in the past and can identify the "for/while" vs "else" construct as a loop with two exit targets that are jumped to unconditionally. In fact, that was one of the "oh, nice!" moments when I first learned Python - I'd never seen a high level language do that before (even C doesn't have it!). All I have said is that the *spelling* of "else:" could be "else if not break:" or - because you mentioned it and I think it actually reads better - "then if not break:". > They hoped to use the same flag the for-loop used. Well, _logically_ there is a flag (in as much as it could be thought of like that to make it easy to understand - and in C, that's pretty much what you have to actually do unless you really want to use 'goto'). > Not a single condition to be seen, anywhere. Its all unconditional jumps. > > To anticipate a possible objection: it is possible that the FOR_ITER > bytecode is implemented with a conditional test, but even so, all that > tests for is whether to enter the main body of the for-loop (10 > STORE_NAME ...) or jump to (18 LOAD_NAME ...). Again, you're arguing against something I didn't say. I never suggested "if not break" (whether following "else" or "then") should generate code that dynamically tests some sort of flag. It's just a different way of spelling exactly the same construct we have today, generating exactly the same bytecode. I can see what you're trying to say, but a naked "then:" really doesn't do it for me. while 1: break then: print ("Yes!") while 1: break then if not break: print ("Yes!") It would be interesting to know what novices thought those meant. Maybe "if not break:" is not the answer either. I just don't think "then:" is ;) E. From ian.g.kelly at gmail.com Wed Feb 1 18:50:50 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 1 Feb 2017 16:50:50 -0700 Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: On Wed, Feb 1, 2017 at 2:51 PM, ????? ?????? wrote: > ?? ???????, 1 ??????????? 2017 - 11:41:28 ?.?. UTC+2, ? ??????? Michael Torrie ??????: >> On 02/01/2017 01:51 PM, ????? ?????? wrote: >> > as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window. >> > >> > Any idea why? >> >> What you're describing is not something you can do with an interactive >> Python script. HTTP-level authentication is requested of your browser >> by the web server itself. On Apache there are numerous methods you can >> use. Individual users can use .htaccess directives to add >> authentication to a directory, for example. You'll need to learn about it: >> https://www.google.com/search?q=apache+http+authentication >> >> If you're using a framework like Django, there are mechanisms for >> checking the username and password against a Python method. Again, >> google for http authentication and whatever framework you're using. >> >> I once used a special python script that was called by an Apache module >> to verify users against a customized LDAP filter. Again, that involves >> server cooperation though a module. >> >> In general, the browser pops up the username and password box in >> response to a request from the web server. It's not something your CGI >> script can just do without some cooperation from the web server. > > I used to have this workaround solution for triggering the web server to pop-up the HTTP Auth window > > print '''''' % file_requested > > and i have tried to read the the login auth name that user entered by using > > authuser = os.environ.get( 'REMOTE_USER', '????????' ) > > unfortunately it always failes to receive it that's why i'm trying to do the trick with the requests module. Fails how? It doesn't ask the user, or the environment variable is empty? requests is an HTTP client library. It's not very useful server-side unless you're talking to other servers. It is, in any case, nonsensical to send an HTTP request to the browser. From me.on.nzt at gmail.com Wed Feb 1 19:00:33 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 16:00:33 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: <956b9e50-606c-464e-8831-8f551e73b797@googlegroups.com> ?? ??????, 2 ??????????? 2017 - 1:51:52 ?.?. UTC+2, ? ??????? Ian ??????: > On Wed, Feb 1, 2017 at 2:51 PM, ????? ?????? wrote: > > ?? ???????, 1 ??????????? 2017 - 11:41:28 ?.?. UTC+2, ? ??????? Michael Torrie ??????: > >> On 02/01/2017 01:51 PM, ????? ?????? wrote: > >> > as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window. > >> > > >> > Any idea why? > >> > >> What you're describing is not something you can do with an interactive > >> Python script. HTTP-level authentication is requested of your browser > >> by the web server itself. On Apache there are numerous methods you can > >> use. Individual users can use .htaccess directives to add > >> authentication to a directory, for example. You'll need to learn about it: > >> https://www.google.com/search?q=apache+http+authentication > >> > >> If you're using a framework like Django, there are mechanisms for > >> checking the username and password against a Python method. Again, > >> google for http authentication and whatever framework you're using. > >> > >> I once used a special python script that was called by an Apache module > >> to verify users against a customized LDAP filter. Again, that involves > >> server cooperation though a module. > >> > >> In general, the browser pops up the username and password box in > >> response to a request from the web server. It's not something your CGI > >> script can just do without some cooperation from the web server. > > > > I used to have this workaround solution for triggering the web server to pop-up the HTTP Auth window > > > > print '''''' % file_requested > > > > and i have tried to read the the login auth name that user entered by using > > > > authuser = os.environ.get( 'REMOTE_USER', '????????' ) > > > > unfortunately it always failes to receive it that's why i'm trying to do the trick with the requests module. > > Fails how? It doesn't ask the user, or the environment variable is empty? > > requests is an HTTP client library. It's not very useful server-side > unless you're talking to other servers. It is, in any case, > nonsensical to send an HTTP request to the browser. triggers the http auth windows, so the user can enter the auth info, i just cant seem to grab the auth username back. From python at lucidity.plus.com Wed Feb 1 19:14:32 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 2 Feb 2017 00:14:32 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? In-Reply-To: References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: <6c9d35db-9f9d-4469-a18f-442fe0d2a267@lucidity.plus.com> On 01/02/17 23:20, Wildman via Python-list wrote: > On Wed, 01 Feb 2017 21:29:00 +0000, Chris Green wrote: > >> Wildman wrote: >>> On Wed, 01 Feb 2017 19:15:13 +0000, Chris Green wrote: >> OK, no problem, but isn't it very non-portable? > > I don't see why not. It should work on any system > that has Python3 installed, at least that is my > understanding. I'm sure someone will correct me > if I'm wrong. > > OTOH if you want in insure 100% portability with any > script, you can use pyinstaller. > > To install for Python2: > pip install pyinstaller > > For Python3: > > pip3 install pyinstaller Out of interest (as someone who grew up on the great 1.5.7 ;)) - is there a definitive resource that explains all of the various packaging and installation options that exist for Python modules these days (both for an author and a user)? A lot of Linux distributions have Python-related packages (other than the language itself) which can be installed using the system installer. Then there's "pip", which is an installer which is installed using the system installer. Now, apparently, there's "pyinstaller" which can be installed using the "pip" installer! I'd like to understand the differences and how this all fits together. Thanks, E. From steve+python at pearwood.info Wed Feb 1 19:21:23 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 02 Feb 2017 11:21:23 +1100 Subject: Rename file without overwriting existing files References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <58927b85$0$1611$c3e8da3$5496439d@news.astraweb.com> On Tue, 31 Jan 2017 02:56 am, Grant Edwards wrote: > On 2017-01-30, Terry Reedy wrote: >> On 1/30/2017 8:58 AM, Peter Otten wrote: >>> Jussi Piitulainen wrote: >> >>>> It doesn't seem to be documented. >>> >>> For functions with a C equivalent a look into the man page is usually >>> helpful. >> >> Man pages do not exist on Windows. I suspect that there are more >> individual Python programs on Windows than *nix. I am more sure of this >> as applied to beginners, most of whom have no idea what a 'man' page is >> (sexist docs? ;-). > > IMO, beginners shouldn't be using the os module. Do you mean beginners to Python or beginners to programming, or both? [...] > I always found the first sentence to be a bit funny: > > This module provides a portable way of using operating system > dependent functionality. > > I understand whay they're tying to say, but I always found it amusing > to say you're going to provide a portable way to do something > non-portable. Fortunately, as Python has matured as a language, it has moved away from that simplistic "dumb interface to OS specific functions". For example, os.urandom: - calls the getrandom() syscall if available; - fall back on /dev/urandom if not; - calls CryptGenRandom() on Windows; - and getentropy() on BSD. https://docs.python.org/3.6/library/os.html#os.urandom Similarly, quite a few os functions either come in a pair of flavours, one returning a string, the other returning bytes, or they take either a byte argument or a string argument. Some even accept pathlib path objects. Python is a high-level, platform-independent language, and the more platform-independent wrappers around platform-dependent functions, the better. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From me.on.nzt at gmail.com Wed Feb 1 19:22:54 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Wed, 1 Feb 2017 16:22:54 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: ================================ # Give user the file requested print('''''' % realfile) authuser = os.environ.get( 'REMOTE_USER', '????????' ) print( authuser ) ================================ Trying this, feels liek i'm almost there except that when printing the value of authuser variable it default to "????????" meaning not there. is there any other way i can grab what the user gave a auth login info? From steve+python at pearwood.info Wed Feb 1 20:24:22 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 02 Feb 2017 12:24:22 +1100 Subject: Is shutil.get_terminal_size useless? References: <588c505f$0$1583$c3e8da3$5496439d@news.astraweb.com> <588c7728$0$1602$c3e8da3$5496439d@news.astraweb.com> <588cc39f$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <58928a49$0$1612$c3e8da3$5496439d@news.astraweb.com> On Sun, 29 Jan 2017 04:58 am, Chris Angelico wrote: > On Sun, Jan 29, 2017 at 3:15 AM, Steve D'Aprano > wrote: >> On Sat, 28 Jan 2017 10:50 pm, Chris Angelico wrote: >> >>> On Sat, Jan 28, 2017 at 9:49 PM, Steve D'Aprano >>> wrote: >>>> The terminal size doesn't change just because I'm piping output to >>>> another process. Using the terminal size as a proxy for "being piped" >>>> is sheer insanity. >>> >>> In a sense, there _is no_ terminal size when you're being piped to >>> another process. >> >> In which sense, and why do you think it is relevant? >> >> There clearly is a terminal, because that's where I'm running the code. >> Regardless of whether I pipe it to grep or cat or something else, the >> output from *that* process still ends up in the same terminal that I >> typed the command in. > > No, not automatically. I've written plenty of programs that accept > input via a pipe and don't display any of it. Well of course if your program doesn't produce any output, the output doesn't go to stdout. Because there is none. > Just because someone > types "command1 | command2", you can't assume that the terminal > command2 is outputting to is the same size as the one command1 should > be outputting to. I'm not assuming anything of the sort. Obviously command2 can send output where it likes, including /dev/null, or simply not produce any output. And as I've *repeatedly* acknowledged, there are circumstances where there is no terminal at all. And shutil does the wrong thing then too: it returns an arbitrary size, instead of raising. (Of course we should be able to explicitly pass a default to shutil.get_terminal_size() if we wish.) > >>> etc, etc, etc, etc. It's >>> not a proxy for "being piped" - it's that when your output isn't going >>> to a terminal, asking "what is my terminal size" isn't particularly >>> productive. >> >> Then explain why os.get_terminal_size() returns the correct answer. No answer Chris? You've spent a lot of time telling me that asking for the terminal size doesn't even make sense for the exact circumstances where os.get_terminal_size() not only returns a size, but the *correct* size. And, of course, it also correctly raises an exception in those unusual cases where there truly is no terminal, or it is unknowable. >> The output might not be going to a terminal (not directly at least) but >> the question isn't "what's the size of the terminal that output is going >> to". The question is "what's the size of the terminal that this process >> is running in", and that has an answer regardless of where output is >> piped. > > Processes aren't always running "in" terminals, though. That's my > point. A terminal is not a fundamental feature of a process. And I have acknowledged this, oh, about a million times. But when the processes *are* running in terminals, shutil should return the size of the terminal, not some arbitrary made up size. Otherwise it is useless. >>> Would you expect a cronjob to use the terminal size when you >>> most recently edited crontab? No. >> >> Of course not -- the terminal where you edited crontab is not where the >> process is running. Why would it be the least bit relevant? > > So where *is* that process running? What terminal is it in? There isn't one. > Don't you > see how similar this is to the pipe situation No. os.get_terminal_size() is correctly able to return the terminal size in the pipe situation, even in a *double* pipe situation (you need to look at stderr rather than stdin or stout). I'm not sure if it is able to cope with situations where all three of std[in|out|err] are redirected, but that's okay. I'm a realist: if there are scenarios where the terminal size is unknowable, then so be it. But a simple pipe is not one of those cases. > - sure, there might be a > terminal that the program was invoked from, but it's utterly > irrelevant to how the program actually runs. Except that in the pipe situation, "how the program actually runs" *is* in a terminal. > >> You might well be a completely background process. >> >> And if that background process is running in a terminal? What's your >> point? > > Background processes don't have terminal access. Really? I think you are wrong: [steve at ando ~]$ cat bkg.py from os import get_terminal_size print(get_terminal_size(0)) [steve at ando ~]$ python3.5 bkg.py & [1] 13776 [steve at ando ~]$ os.terminal_size(columns=116, lines=29) [1]+ Done python3.5 bkg.py [steve at ando ~]$ Looks like the background process is perfectly able to return the terminal size, and not an arbitrary number either, it has the correct size. > Whether it's a daemon > or something started as "commandname >/dev/null 2>/dev/null &" from bash, it doesn't have access to a terminal. As I've repeatedly said, if there are situations where the terminal size is genuinely unknowable, then so be it. But shutil fails even when the terminal size *is* knowable. How is this useful? > Please don't assume that every program has a controlling UI. I'm not, and I haven't. > I get > enough of that mentality from the people I support on Windows, where > the fundamental assumption of people seems to be that there's a GUI > for everything. There isn't. And there isn't always a terminal either. > Processes in the middle of pipelines *do not have* terminals. Sometimes they do. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Feb 1 20:28:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 02 Feb 2017 12:28:33 +1100 Subject: Is shutil.get_terminal_size useless? References: <588c505f$0$1583$c3e8da3$5496439d@news.astraweb.com> <588c79eb$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <58928b43$0$1612$c3e8da3$5496439d@news.astraweb.com> On Sat, 28 Jan 2017 11:53 pm, Peter Otten wrote: [...] >> I see that as "Hey look, we can fool shutil into returning >> absolute garbage instead of the terminal size!" > > There are valid reasons for temporarily altering the number of columns, > like writing to a file or preparing a code sample. Hmmm... I hadn't thought of that. You're right that there are scenarios where you might wish to tell a process to use a different number of columns (or even lines). Good point. I'm not sure that the given API is a good one, but I accept that the COLUMNS + LINES environment variables could be used to override the actual terminal size. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From torriem at gmail.com Wed Feb 1 20:29:00 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 1 Feb 2017 18:29:00 -0700 Subject: How to know what to install (Ubuntu/Debian) for a given import? In-Reply-To: References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: <05492933-4861-1e25-3e27-0a2a6f0b521d@gmail.com> On 02/01/2017 02:29 PM, Chris Green wrote: > OK, thank you, what a strange way to do it. Why is it strange? Essentially, python bindings for any GObject-based library are now fully automatic via this gi module. No longer do we need custom bindings for each component of a glib-based library. This means even your own custom libraries, if they use gobject, can be bound dynamically to Python with no work on your part. So lots of gnome libraries are accessible this way, as well as GTK3. This does come at a cost though: since the bindings are dynamic and based on gobject types, they aren't quite as pythonic as PyGTK used to be. AT least that's what I've been told. Some of the GObject stuff leaks into the python side of things, whereas PyGTK would custom wrap them and try to wrap some types in more familiar Python versions. > OK, no problem, but isn't it very non-portable? Not inherently, no. Should work on OS X and Windows. From rosuav at gmail.com Wed Feb 1 20:35:43 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Feb 2017 12:35:43 +1100 Subject: Is shutil.get_terminal_size useless? In-Reply-To: <58928a49$0$1612$c3e8da3$5496439d@news.astraweb.com> References: <588c505f$0$1583$c3e8da3$5496439d@news.astraweb.com> <588c7728$0$1602$c3e8da3$5496439d@news.astraweb.com> <588cc39f$0$1609$c3e8da3$5496439d@news.astraweb.com> <58928a49$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 2, 2017 at 12:24 PM, Steve D'Aprano wrote: >>>> etc, etc, etc, etc. It's >>>> not a proxy for "being piped" - it's that when your output isn't going >>>> to a terminal, asking "what is my terminal size" isn't particularly >>>> productive. >>> >>> Then explain why os.get_terminal_size() returns the correct answer. > > No answer Chris? > > You've spent a lot of time telling me that asking for the terminal size > doesn't even make sense for the exact circumstances where > os.get_terminal_size() not only returns a size, but the *correct* size. > Because you've already decided in your own mind that one answer is THE correct one, and nothing I say will possibly change that. In my very first post on this subject, I pointed out that shutil and os were returning *different* answers, *both* of which are useful. You persist in insisting that only one of them is useful and correct, and so there's no point responding. Let me redirect this discussion slightly. How do you calculate the square root of a number? >>> (-1) ** 0.5 (6.123233995736766e-17+1j) >>> math.sqrt(-1) Traceback (most recent call last): File "", line 1, in ValueError: math domain error >>> cmath.sqrt(-1) 1j Why does only one of these give the correct answer? If you absolutely INSIST that one is correct, how can anyone argue as to why the others exist? Or is it possible that there are times when you want ValueError and times when you want a complex result? ChrisA From rosuav at gmail.com Wed Feb 1 20:41:20 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 2 Feb 2017 12:41:20 +1100 Subject: How coding in Python is bad for you In-Reply-To: <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> References: <355c105c-d51a-4c66-be66-dde41cba20c8@googlegroups.com> <87ab542f-0473-40c1-8a56-b41b397fb8b2@googlegroups.com> <588dff48$0$22142$c3e8da3$5496439d@news.astraweb.com> <00e3f05e-86b4-ea26-7211-39bebc6b4afe@lucidity.plus.com> <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> Message-ID: On Thu, Feb 2, 2017 at 10:49 AM, Erik wrote: > Well, _logically_ there is a flag (in as much as it could be thought of like > that to make it easy to understand - and in C, that's pretty much what you > have to actually do unless you really want to use 'goto'). The last time I wanted a for-else in C, I used a goto. And no velociraptor came and ate me. ChrisA From python at mrabarnett.plus.com Wed Feb 1 21:05:42 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 2 Feb 2017 02:05:42 +0000 Subject: How coding in Python is bad for you In-Reply-To: <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> References: <355c105c-d51a-4c66-be66-dde41cba20c8@googlegroups.com> <87ab542f-0473-40c1-8a56-b41b397fb8b2@googlegroups.com> <588dff48$0$22142$c3e8da3$5496439d@news.astraweb.com> <00e3f05e-86b4-ea26-7211-39bebc6b4afe@lucidity.plus.com> <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> Message-ID: <51819207-982c-8d78-ba02-44a27a6f392d@mrabarnett.plus.com> On 2017-02-01 23:49, Erik wrote: > On 30/01/17 02:14, Steve D'Aprano wrote: >> On Mon, 30 Jan 2017 10:52 am, Erik wrote: >>> It would be even better if it was "else if not break:" to make the >>> meaning clearer. >> >> break is not the only way to exit the for loop > > Fine - "else if not break or raise or return:", then ;) [that is not a > serious suggestion] > [snip] Both suggestions are a little long-winded. Couldn't we just abbreviate them to "else:"? :-) From python at lucidity.plus.com Wed Feb 1 21:14:21 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 2 Feb 2017 02:14:21 +0000 Subject: How coding in Python is bad for you In-Reply-To: <51819207-982c-8d78-ba02-44a27a6f392d@mrabarnett.plus.com> References: <355c105c-d51a-4c66-be66-dde41cba20c8@googlegroups.com> <87ab542f-0473-40c1-8a56-b41b397fb8b2@googlegroups.com> <588dff48$0$22142$c3e8da3$5496439d@news.astraweb.com> <00e3f05e-86b4-ea26-7211-39bebc6b4afe@lucidity.plus.com> <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> <51819207-982c-8d78-ba02-44a27a6f392d@mrabarnett.plus.com> Message-ID: On 02/02/17 02:05, MRAB wrote: > Both suggestions are a little long-winded. Couldn't we just abbreviate > them to "else:"? :-) You are not wrong ;) E. From python at lucidity.plus.com Wed Feb 1 21:17:06 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 2 Feb 2017 02:17:06 +0000 Subject: How coding in Python is bad for you In-Reply-To: References: <355c105c-d51a-4c66-be66-dde41cba20c8@googlegroups.com> <87ab542f-0473-40c1-8a56-b41b397fb8b2@googlegroups.com> <588dff48$0$22142$c3e8da3$5496439d@news.astraweb.com> <00e3f05e-86b4-ea26-7211-39bebc6b4afe@lucidity.plus.com> <588ea186$0$22140$c3e8da3$5496439d@news.astraweb.com> <32c1bfbe-b10e-3b42-3b16-27392e87de77@lucidity.plus.com> Message-ID: On 02/02/17 01:41, Chris Angelico wrote: > On Thu, Feb 2, 2017 at 10:49 AM, Erik wrote: >> Well, _logically_ there is a flag (in as much as it could be thought of like >> that to make it easy to understand - and in C, that's pretty much what you >> have to actually do unless you really want to use 'goto'). > > The last time I wanted a for-else in C, I used a goto. > > And no velociraptor came and ate me. Thanks for your input. E. From rustompmody at gmail.com Wed Feb 1 21:23:31 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 1 Feb 2017 18:23:31 -0800 (PST) Subject: How to know what to install (Ubuntu/Debian) for a given import? In-Reply-To: References: <1no9md-fbb.ln1@esprimo.zbmc.eu> <6c9d35db-9f9d-4469-a18f-442fe0d2a267@lucidity.plus.com> Message-ID: <9be19826-c4ea-4286-bb11-d7516038e592@googlegroups.com> On Thursday, February 2, 2017 at 5:44:51 AM UTC+5:30, Erik wrote: > On 01/02/17 23:20, Wildman via Python-list wrote: > > On Wed, 01 Feb 2017 21:29:00 +0000, Chris Green wrote: > > > >> Wildman wrote: > >>> On Wed, 01 Feb 2017 19:15:13 +0000, Chris Green wrote: > >> OK, no problem, but isn't it very non-portable? > > > > I don't see why not. It should work on any system > > that has Python3 installed, at least that is my > > understanding. I'm sure someone will correct me > > if I'm wrong. > > > > OTOH if you want in insure 100% portability with any > > script, you can use pyinstaller. > > > > To install for Python2: > > pip install pyinstaller > > > > For Python3: > > > > pip3 install pyinstaller > > Out of interest (as someone who grew up on the great 1.5.7 ;)) - is > there a definitive resource that explains all of the various packaging > and installation options that exist for Python modules these days (both > for an author and a user)? > > A lot of Linux distributions have Python-related packages (other than > the language itself) which can be installed using the system installer. > > Then there's "pip", which is an installer which is installed using the > system installer. > > Now, apparently, there's "pyinstaller" which can be installed using the > "pip" installer! > > I'd like to understand the differences and how this all fits together. +10 From me for this question My impression is that this question is signally badly addressed because it falls between OS-language stools: ie in the larger Linux-Python ecosystem some things are naturally addressed as Linux docs, some as python docs. This (and such issues) seems to be a buck that loves to be passed around From paul.wolf at yewleaf.com Thu Feb 2 03:50:33 2017 From: paul.wolf at yewleaf.com (paul.wolf at yewleaf.com) Date: Thu, 2 Feb 2017 00:50:33 -0800 (PST) Subject: command line micro wiki written in Python In-Reply-To: References: <8560kusst2.fsf@benfinney.id.au> Message-ID: <1d51357b-7515-4b1f-910a-ee0f522e63d1@googlegroups.com> On Tuesday, 31 January 2017 23:39:41 UTC, Ben Finney wrote: > The Python community has a stronger (?) preference for reStructuredText > format. Can that be the default? > > That is, I want my text files to be named ?foo? (no suffix) or ?foo.txt? > (because they're primarily text), and have the default be to parse them > as reStructuredText. Good point. It should at least be possible to provide an arbitrary default so you can have rst as default for new files. And perhaps as you say also start with rst as the main default. Regarding no suffix, whatever the default is gets used for newly created files, but not for the 'take' command which I use mainly to link to (with --symlink) config files, like: yd take ~/.emacs for instance. From list at qtrac.plus.com Thu Feb 2 04:05:45 2017 From: list at qtrac.plus.com (Mark Summerfield) Date: Thu, 2 Feb 2017 01:05:45 -0800 (PST) Subject: Is there a lint tool that can spot unused classes/methods/attributes/functions/etc? Message-ID: Suppose I have a project with an application .py file and various module .py files all in the same directory, and after lots of refactoring and other changes the modules contain lots of unused stuff. Is there a lint tool that will spot the unused things so that I can get rid of them? I've tried flake and pylint but neither seem to help. Here's the smallest example I can think of: # app.py import Mod a = Mod.A() print(a.quack()) # Mod.py class A: # Used in app.py def __init__(self): self.x = "hello" # Used in quack() which is used in app.py self.y = 7 # Unused @property def z(self): # Unused return 4.5 def quack(self): # Used in app.py return self.x class B: # Unused pass CONST = 999 # Unused def func(): # Unused pass Thanks. From cl at isbd.net Thu Feb 2 04:25:06 2017 From: cl at isbd.net (Chris Green) Date: Thu, 2 Feb 2017 09:25:06 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: <1no9md-fbb.ln1@esprimo.zbmc.eu> <05492933-4861-1e25-3e27-0a2a6f0b521d@gmail.com> Message-ID: Michael Torrie wrote: > On 02/01/2017 02:29 PM, Chris Green wrote: > > OK, thank you, what a strange way to do it. > > Why is it strange? Essentially, python bindings for any GObject-based > library are now fully automatic via this gi module. No longer do we Which is thus different from every other python library! So we have to somehow 'know' that the components of GObject are imported by this special mechanism. Everything else works the old/normal way. > > > OK, no problem, but isn't it very non-portable? > > Not inherently, no. Should work on OS X and Windows. > OK, good, I thought from the original reply that it was Ubuntu specific, but it's not. -- Chris Green ? From cl at isbd.net Thu Feb 2 04:28:30 2017 From: cl at isbd.net (Chris Green) Date: Thu, 2 Feb 2017 09:28:30 +0000 Subject: How to know what to install (Ubuntu/Debian) for a given import? References: <1no9md-fbb.ln1@esprimo.zbmc.eu> Message-ID: Michael Torrie wrote: > On 02/01/2017 01:03 PM, Wildman via Python-list wrote: > > > > It is the proper way. This page helps explain it. > > > > http://askubuntu.com/questions/784068/what-is-gi-repository-in-python > > > >> ... and doesn't it need an internet connection? > > > > No. > > However the gi module provides access to GTK+3, and it's quite likely > Chris's project requires GTK+2, and would probably take some work to > port from GTK+2 to GTK+3, though in the long run it's worth the effort. > > Anyway the GTK+2 module for python is usually called pygtk in the repos. > It would appear that there isn't a GTK+2 for Python 3 in Ubuntu, or at least I can't find one. I've reverted to staying with Python 2 fr the moment. I can't manage changing the code to work with a database instead of LDAP *and* migrating from GTK+2 to GTK+3. :-) When I have it working as I want in python 2 I'll consider the GTK+2 to GTK+3 migration and moving to python 3. -- Chris Green ? From __peter__ at web.de Thu Feb 2 05:03:33 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 02 Feb 2017 11:03:33 +0100 Subject: Is there a lint tool that can spot unused classes/methods/attributes/functions/etc? References: Message-ID: Mark Summerfield wrote: > Suppose I have a project with an application .py file and various module > .py files all in the same directory, and after lots of refactoring and > other changes the modules contain lots of unused stuff. Is there a lint > tool that will spot the unused things so that I can get rid of them? > > I've tried flake and pylint but neither seem to help. I often "find" unused code with coverage.py. However, it cannot detect statements that are executed but have no effect, and thus will report only A.z() and func() in the example below. > Here's the smallest example I can think of: > > # app.py > import Mod > a = Mod.A() > print(a.quack()) > > # Mod.py > class A: # Used in app.py > def __init__(self): > self.x = "hello" # Used in quack() which is used in app.py > self.y = 7 # Unused > @property > def z(self): # Unused > return 4.5 > def quack(self): # Used in app.py > return self.x > class B: # Unused > pass > CONST = 999 # Unused > def func(): # Unused > pass > > Thanks. From dieter at handshake.de Thu Feb 2 05:19:48 2017 From: dieter at handshake.de (dieter) Date: Thu, 02 Feb 2017 11:19:48 +0100 Subject: Why doesn't module finalization delete names as expected? References: Message-ID: <871svggaij.fsf@handshake.de> Philippe Proulx writes: > ... > It feels like `bread` is never deleted in the module initialization > situation, but I don't know why: the only reference to the Bread Python > object is this `bread` name in the module... what could prevent this > object's __del__() method to be called? It works when I call `del bread` > manually: I would expect that the module finalization does the exact > same thing? I expect, you are speaking about "module finalization" as part of "process finalization", i.e. when the whole Python process terminates: a module is not "finalized" at the end of its code execution. Python's behaviour during process finalization is complex (and has changed over time). It may still be prone to surprises. Somewhere, there is a document describing in details how Python performs finalization. When I last read it (may have changed meanwhile), it told that first "_*" variables are unbound, then the module dict is cleared. From bajimicbiga at gmail.com Thu Feb 2 05:43:49 2017 From: bajimicbiga at gmail.com (bajimicbiga at gmail.com) Date: Thu, 2 Feb 2017 02:43:49 -0800 (PST) Subject: How to add months to a date (datetime object)? In-Reply-To: <49bd3ab8$0$510$bed64819@news.gradwell.net> References: <49bd3ab8$0$510$bed64819@news.gradwell.net> Message-ID: <3c0bf486-ab97-45de-8a20-92caf13bf03d@googlegroups.com> for start of month to the beginning of next month from datetime import timedelta from dateutil.relativedelta import relativedelta end_date = start_date + relativedelta(months=delta_period) + timedelta(days=-delta_period) From muetced124 at gmail.com Thu Feb 2 08:38:32 2017 From: muetced124 at gmail.com (muetced124 at gmail.com) Date: Thu, 2 Feb 2017 05:38:32 -0800 (PST) Subject: how i can write command to run...........This is code to convert .json file to .csv......Input file name is sample.json Message-ID: <41da8fac-0d33-41d4-8b36-b351d7a364b8@googlegroups.com> import sys import json import csv ## # Convert to string keeping encoding in mind... ## def to_string(s): try: return str(s) except: #Change the encoding type if needed return s.encode('utf-8') ## # This function converts an item like # { # "item_1":"value_11", # "item_2":"value_12", # "item_3":"value_13", # "item_4":["sub_value_14", "sub_value_15"], # "item_5":{ # "sub_item_1":"sub_item_value_11", # "sub_item_2":["sub_item_value_12", "sub_item_value_13"] # } # } # To # { # "node_item_1":"value_11", # "node_item_2":"value_12", # "node_item_3":"value_13", # "node_item_4_0":"sub_value_14", # "node_item_4_1":"sub_value_15", # "node_item_5_sub_item_1":"sub_item_value_11", # "node_item_5_sub_item_2_0":"sub_item_value_12", # "node_item_5_sub_item_2_0":"sub_item_value_13" # } ## def reduce_item(key, value): global reduced_item #Reduction Condition 1 if type(value) is list: i=0 for sub_item in value: reduce_item(key+'_'+to_string(i), sub_item) i=i+1 #Reduction Condition 2 elif type(value) is dict: sub_keys = value.keys() for sub_key in sub_keys: reduce_item(key+'_'+to_string(sub_key), value[sub_key]) #Base Condition else: reduced_item[to_string(key)] = to_string(value) if __name__ == "__main__": if len(sys.argv) != 4: print "\nUsage: python json_to_csv.py \n" else: #Reading arguments node = sys.argv[1] json_file_path = sys.argv[2] csv_file_path = sys.argv[3] fp = open(json_file_path, 'r') json_value = fp.read() raw_data = json.loads(json_value) try: data_to_be_processed = raw_data[node] except: data_to_be_processed = raw_data processed_data = [] header = [] for item in data_to_be_processed: reduced_item = {} reduce_item(node, item) header += reduced_item.keys() processed_data.append(reduced_item) header = list(set(header)) header.sort() with open(csv_file_path, 'wb+') as f: writer = csv.DictWriter(f, header, quoting=csv.QUOTE_ALL) writer.writeheader() for row in processed_data: writer.writerow(row) print "Just completed writing csv file with %d columns" % len(header) From marco.buttu at gmail.com Thu Feb 2 11:31:18 2017 From: marco.buttu at gmail.com (Marco Buttu) Date: Thu, 02 Feb 2017 17:31:18 +0100 Subject: user+sys time bigger than real time (with threads) Message-ID: <58935ED6.50505@oa-cagliari.inaf.it> Let's have the following example: $ cat foo.py from threading import Thread, Lock result = 0 lock = Lock() def task(): global result for i in range(10**6): lock.acquire() result += 1 lock.release() if __name__ == '__main__': t1, t2 = Thread(target=task), Thread(target=task) t1.start() t2.start() t1.join() t2.join() print('result:', result) When I execute it (Python 3.6), I get a sys+user time bigger than the real time: $ time python foo.py result: 2000000 real 0m7.088s user 0m6.597s sys 0m5.043s I can expect this result when I run some processes in parallel on different CPUs, but this code uses threads, so the GIL prevents the two task() functions to be executed in parallel. What am I missing? -- Marco Buttu INAF-Osservatorio Astronomico di Cagliari Via della Scienza n. 5, 09047 Selargius (CA) Phone: 070 711 80 217 Email: mbuttu at oa-cagliari.inaf.it From cl at isbd.net Thu Feb 2 14:30:31 2017 From: cl at isbd.net (Chris Green) Date: Thu, 2 Feb 2017 19:30:31 +0000 Subject: What is meant by "Duty Cycle" when referring to laser printers? Message-ID: Where is GDK_BUTTON_PRESS defined in gtk2, i.e. how can I use it when testing for button presses? I have tried gtk.GDK_BUTTON_PRESS and gtk.gdk.GDK_BUTTON_PRESS but both result in "object has no attribute 'GDK_BUTTON_PRESS'" errors. -- Chris Green ? From ian.g.kelly at gmail.com Thu Feb 2 14:37:54 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 2 Feb 2017 12:37:54 -0700 Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: On Wed, Feb 1, 2017 at 5:22 PM, ????? ?????? wrote: > ================================ > # Give user the file requested > > print('''''' % realfile) > > authuser = os.environ.get( 'REMOTE_USER', '????????' ) > print( authuser ) > ================================ > > Trying this, feels liek i'm almost there except that when printing the value of authuser variable it default to "????????" meaning not there. > > is there any other way i can grab what the user gave a auth login info? Hold on, are those consecutive lines within the same script? I think you need to better understand the HTTP request cycle. The browser sends a request to your server, the server runs your CGI which builds a response, and then the server sends the response back to the browser. At that point the CGI is done with this request. The tag that you're printing is part of that response. The browser can't do anything with it until it sees it. When it does, it will perform the refresh which creates a second request to the server at the new URL. If the server's response to the second request is a 401 Unauthorized, then the browser shows the username/password dialog and after the user enters those it will make a /third/ request containing that info, also to the new URL. Your script which ran on the first request is trying to get the REMOTE_USER from the authentication data that was passed to that first request, but there wasn't any. The user didn't enter any until the third request, at which point your script was long since finished running. If you want the user to authenticate to your script and not just whatever file you're redirecting them to, then you need to configure the server to require authorization for the script and not just the redirect target. Most likely you would do this with an .htaccess directive as Michael Torrie already suggested. Once that's done, then as soon as your script is invoked you'll be able to get the REMOTE_USER. The tag has nothing to do with requesting auth and you only need it if you want the browser to perform a delayed redirect to that file. From tomuxiong at gmx.com Thu Feb 2 14:41:41 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 2 Feb 2017 14:41:41 -0500 Subject: Is requests[security] required for python 3.5+ ? Message-ID: <7555c237-dfd7-1b31-6d91-a523c3d7859c@gmx.com> Hello, I'm trying to understand whether requests[security] or just plain requests should be installed for python 3.5. I.e. do the following packages need to be installed: pyOpenSSL, cryptography, idna. The reason I'm asking is because I'm moving an application to python 3 and I am testing out which requirements continue to be required in the version. Thanks for any info. Cheers, Thomas From kwpolska at gmail.com Thu Feb 2 14:43:32 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Thu, 2 Feb 2017 20:43:32 +0100 Subject: Is requests[security] required for python 3.5+ ? In-Reply-To: <7555c237-dfd7-1b31-6d91-a523c3d7859c@gmx.com> References: <7555c237-dfd7-1b31-6d91-a523c3d7859c@gmx.com> Message-ID: On 2 February 2017 at 20:41, Thomas Nyberg wrote: > Hello, > > I'm trying to understand whether requests[security] or just plain requests > should be installed for python 3.5. I.e. do the following packages need to > be installed: pyOpenSSL, cryptography, idna. > > The reason I'm asking is because I'm moving an application to python 3 and I > am testing out which requirements continue to be required in the version. No, requests[security] is only needed for old Python 2.7 versions (before 2.7.9). -- Chris Warrick PGP: 5EAAEA16 From python at mrabarnett.plus.com Thu Feb 2 14:48:22 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 2 Feb 2017 19:48:22 +0000 Subject: What is meant by "Duty Cycle" when referring to laser printers? In-Reply-To: References: Message-ID: <969f15de-4268-0a54-6cc1-47a07bef63ad@mrabarnett.plus.com> On 2017-02-02 19:30, Chris Green wrote: > Where is GDK_BUTTON_PRESS defined in gtk2, i.e. how can I use it when > testing for button presses? > > I have tried gtk.GDK_BUTTON_PRESS and gtk.gdk.GDK_BUTTON_PRESS but > both result in "object has no attribute 'GDK_BUTTON_PRESS'" errors. > Does this help? http://faq.pygtk.org/index.py?req=edit&file=faq02.007.htp BTW, why is the subject line what it is? From me.on.nzt at gmail.com Thu Feb 2 15:13:25 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Thu, 2 Feb 2017 12:13:25 -0800 (PST) Subject: Python3 using requests to grab HTTP Auth Data In-Reply-To: References: <37fc3b88-e0c9-419d-8cd2-bce795a19a5d@googlegroups.com> <5756f77f-2b62-4e21-a1c9-07a2ba2287b6@googlegroups.com> Message-ID: <59b679c4-1299-4254-9a36-2085d739dabc@googlegroups.com> ?? ??????, 2 ??????????? 2017 - 9:38:47 ?.?. UTC+2, ? ??????? Ian ??????: > If you want the user to authenticate to your script and not just > whatever file you're redirecting them to, then you need to configure > the server to require authorization for the script and not just the > redirect target. Most likely you would do this with an .htaccess > directive as Michael Torrie already suggested. Once that's done, then > as soon as your script is invoked you'll be able to get the > REMOTE_USER. The tag has nothing to do with requesting auth and > you only need it if you want the browser to perform a delayed redirect > to that file. I have also came up with this idea but the problem with that approach is that in order for 'files.py' script to run an authentice dialog must come first. As i have the script coded the 'files.py' need to run first normally and only after a form submit bases on user selection of a a listed file, onlty then the authentication process to be initiated , not earlier. To see what i eman just visit my website at http://superhost.gr, scroll down on the middle of the page, hit download now image link. A list of files will be presented to you and when you click someone then the tage initiate the authentication. Can this also be done in such a way with .htaccess & .htpasswd ?! From hpj at urpla.net Thu Feb 2 15:15:11 2017 From: hpj at urpla.net (Hans-Peter Jansen) Date: Thu, 02 Feb 2017 21:15:11 +0100 Subject: ANN: psutil 5.1.0 with hardware sensors released In-Reply-To: References: Message-ID: <1696529.Y7S43JNBpq@xrated> On Mittwoch, 1. Februar 2017 21:54:06 Giampaolo Rodola' wrote: > Hello all, > I'm glad to announce the release of psutil 5.1.1: ^ Guess, you meant to say 5.1.0 here, or probably your time machine broke ;) Cheers, Pete From cl at isbd.net Thu Feb 2 15:50:25 2017 From: cl at isbd.net (Chris Green) Date: Thu, 2 Feb 2017 20:50:25 +0000 Subject: What is meant by "Duty Cycle" when referring to laser printers? References: <969f15de-4268-0a54-6cc1-47a07bef63ad@mrabarnett.plus.com> Message-ID: MRAB wrote: > On 2017-02-02 19:30, Chris Green wrote: > > Where is GDK_BUTTON_PRESS defined in gtk2, i.e. how can I use it when > > testing for button presses? > > > > I have tried gtk.GDK_BUTTON_PRESS and gtk.gdk.GDK_BUTTON_PRESS but > > both result in "object has no attribute 'GDK_BUTTON_PRESS'" errors. > > > Does this help? > > http://faq.pygtk.org/index.py?req=edit&file=faq02.007.htp > Yes, thank you, just what I need! :-) > BTW, why is the subject line what it is? > Er? I suspect I didn't change it! :-) -- Chris Green ? From ceh329 at gmail.com Thu Feb 2 18:30:28 2017 From: ceh329 at gmail.com (Charles Heizer) Date: Thu, 2 Feb 2017 15:30:28 -0800 (PST) Subject: PyDictObject to NSDictionary Message-ID: <63332606-7057-4020-b25f-5bfb656aeb96@googlegroups.com> Hello, I'm embedding python in to my Objective-C app and I need to know how do I convert a PyDictObject (PyObject) to a NSDictionary? Thanks! From me.on.nzt at gmail.com Thu Feb 2 18:42:36 2017 From: me.on.nzt at gmail.com (=?UTF-8?B?zp3Or866zr/PgiDOks6tz4HOs86/z4I=?=) Date: Thu, 2 Feb 2017 15:42:36 -0800 (PST) Subject: Downloading and Sending the file to the client's browser Message-ID: <2899f1e8-9762-411b-8879-36483c2d2cb2@googlegroups.com> ========================================== # Give user the file requested url = "http://superhost.gr/data/files/%s" % realfile response = requests.get( url, stream=True ) with open( realfile, 'wb' ) as f: f.write( response.content ) ========================================== The above code will download and save the specific file locally to the server where the python script is running. What alternation does this need to be able to send it to the client's browser? From hemla21 at gmail.com Fri Feb 3 06:34:19 2017 From: hemla21 at gmail.com (Heli) Date: Fri, 3 Feb 2017 03:34:19 -0800 (PST) Subject: Interpolation gives negative values Message-ID: <4c912505-24ba-479e-b5ce-dfbf639f314a@googlegroups.com> Dear all, I have an ASCII file (f1) with 100M lines with columns x,y,z ( new data points) and then I have a second ASCII file (f2) with 2M lines with columns x,y,z and VALUE (v10).(known data points). I need to interpolate f1 over f2, that is I need to have values for v10 for all coordinated in f1. I am using the following script to interpolate. from scipy.interpolate import NearestNDInterpolator #new data points f1=np.loadtxt(os.path.join(dir,coord_source),delimiter=None,skiprows=0) x_coord=f1[:,1] y_coord=f1[:,2] z_coord=f1[:,3] # known data points f2=np.loadtxt(os.path.join(dir,val_source),delimiter=None,skiprows=1) x_val=f2[:,1] y_val=f2[:,2] z_val=f2[:,3] v10=f2[:,10] # Value to be interpolated # my interpolation function myf_v10=NearestNDInterpolator((x_val, y_val,z_val),v10) interpolated_v10=myf_v10(x_coord,y_coord,z_coord) I have the following questions. 1. Considering f1 is 50 times bigger than f2. can I still use the above script? 2. The variable v10 that I need to interpolate should always be >= 0 (positive). Using the above script I am getting negative values for v10. How can I fix this? I would really appreciate your help, Thanks in Advance, From hmmeeranrizvi18 at gmail.com Fri Feb 3 07:04:48 2017 From: hmmeeranrizvi18 at gmail.com (Meeran Rizvi) Date: Fri, 3 Feb 2017 04:04:48 -0800 (PST) Subject: Get Result based on the Top value Message-ID: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> Hello guys, Here i am creating a GUI which will act as a search engine. When you search for a data in search box it will fetch the results from the very first page in browser and it is displayed in a Text widget When the TOP value is 10 it should display only the first 10 results based on numbers.if it is 20 it should display first 20 results based on the number. How should i do that? Here is my script from Tkinter import * import mechanize from bs4 import BeautifulSoup root = Tk() def clear_search():#Reset the search field Search.delete(0,END) Result.delete("1.0",END) def fun(self=0): new = Search.get() url = "https://duckduckgo.com/" br = mechanize.Browser() br.set_handle_robots(False) br.open(url) br.select_form(name="x") br["q"] = str(new) res = br.submit() content = res.read() #print content soup = BeautifulSoup(content,"html.parser") mylink = soup.find_all('a') v = 0 for i in mylink: try: if i.attrs['class'][0] == "result__a": v = v + 1 Result.insert(END,v) Result.insert(END,i.text) Result.insert(END,'\n') elif i.attrs['class'][0] == "result__snippet": Result.insert(END,i.text) Result.insert(END,'\n') Result.insert(END,i.attrs['href']) Result.insert(END,'\n ') Result.insert(END,'--------------------------------------------------------------------------') except KeyError: pass with open("result1.xls", "w") as f: f.write(content) Value = Label(root,text="Value:", font="-weight bold") Value.grid(row=0,column=0,padx=15) Search = Entry(root,width=50) Search.grid(row=0,column=1) Top = Label(root,text="TOP",font="-weight bold") Top.grid(row=1,column=0) var = StringVar(root) var.set('10') Dropdownlist = OptionMenu(root,var,'10','20','50','100') Dropdownlist.grid(row=1,column=1,sticky="w") Go = Button(root,text="GO",width=5,command=fun) Go.bind("",fun) Go.grid(row=1,column=1) Reset = Button(root,text="RESET",width=5,command=clear_search) Reset.grid(row=1,column=1,sticky="e") Result = Text(root,height=20,width=75) Result.grid(row=2,column=1,padx=50,pady=10) Scroll = Scrollbar(root,command=Result.yview) Scroll.grid(row=2,column=2,sticky=N+S) Result.config(yscrollcommand=Scroll.set) root.mainloop() From Antonio_Aquino at msn.com Fri Feb 3 08:10:16 2017 From: Antonio_Aquino at msn.com (Antonio) Date: Fri, 3 Feb 2017 13:10:16 +0000 Subject: Fw: Context In-Reply-To: References: Message-ID: ________________________________ From: Antonio Sent: Friday, February 3, 2017 1:02 PM To: python-list at python.org Subject: Context I have python version 3.6.0 installed into my desktop)windows 7) but the menu/context (file,edit..etc) is missing. How to fix this problem? Thanks Antonio [cid:36574bcd-0958-41f0-a1b3-2c34586b236a] From kw at codebykevin.com Fri Feb 3 08:18:49 2017 From: kw at codebykevin.com (Kevin Walzer) Date: Fri, 3 Feb 2017 08:18:49 -0500 Subject: PyDictObject to NSDictionary In-Reply-To: <63332606-7057-4020-b25f-5bfb656aeb96@googlegroups.com> References: <63332606-7057-4020-b25f-5bfb656aeb96@googlegroups.com> Message-ID: On 2/2/17 6:30 PM, Charles Heizer wrote: > Hello, > I'm embedding python in to my Objective-C app and I need to know how do I convert a PyDictObject (PyObject) to a NSDictionary? > > Thanks! > Does the PyObjC library provide any support for this? It allows integration between the Cocoa API's and Python. --Kevin -- Kevin Walzer Code by Kevin/Mobile Code by Kevin http://www.codebykevin.com http://www.wtmobilesoftware.com From amit at phpandmore.net Fri Feb 3 08:59:55 2017 From: amit at phpandmore.net (Amit Yaron) Date: Fri, 3 Feb 2017 15:59:55 +0200 Subject: Numpy and Scipy for Python3.5 Message-ID: Can I find those modules in the FreeBSD ports? From wrw at mac.com Fri Feb 3 09:57:40 2017 From: wrw at mac.com (William Ray Wing) Date: Fri, 03 Feb 2017 09:57:40 -0500 Subject: Context In-Reply-To: References: Message-ID: <63226E93-BCBC-4D17-8CE1-83E565929ABB@mac.com> > On Feb 3, 2017, at 8:10 AM, Antonio wrote: > ________________________________ > From: Antonio > Sent: Friday, February 3, 2017 1:02 PM > To: python-list at python.org > Subject: Context > > I have python version 3.6.0 installed into my desktop)windows 7) but the menu/context (file,edit..etc) is missing. > > How to fix this problem? > python, is not a GUI program, it doesn?t have a menu of actions. If you want to surround python with a GUI you would have to install any of several IDEs (Integrated Development Environments) which will let you create python programs, edit them, and test them interactively. Not being a Windows user, I shouldn't recommend any particular one. Bill > > Thanks > > > Antonio > > > [cid:36574bcd-0958-41f0-a1b3-2c34586b236a] > -- > https://mail.python.org/mailman/listinfo/python-list From tjreedy at udel.edu Fri Feb 3 11:03:09 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 3 Feb 2017 11:03:09 -0500 Subject: Fw: Context In-Reply-To: References: Message-ID: On 2/3/2017 8:10 AM, Antonio wrote: > I have python version 3.6.0 installed into my desktop)windows 7) but the menu/context (file,edit..etc) is missing. Run IDLE (there should be a Start menu icon) or install or run another IDE. -- Terry Jan Reedy From ndbecker2 at gmail.com Fri Feb 3 11:06:00 2017 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 03 Feb 2017 11:06 -0500 Subject: best way to ensure './' is at beginning of sys.path? Message-ID: I want to make sure any modules I build in the current directory overide any others. To do this, I'd like sys.path to always have './' at the beginning. What's the best way to ensure this is always true whenever I run python3? From python.list at tim.thechases.com Fri Feb 3 13:15:51 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 3 Feb 2017 12:15:51 -0600 Subject: Meta: mailing list, bounces, and SPF? Message-ID: <20170203121551.0e85feee@bigbox.christie.dr> Thanks to an abysmal failure of my hosting service (Site5), I was without email for multiple days, and when it came back up, the SPF record was pointed at the wrong place. I normally get a steady stream of comp.lang.python/python-list@ messages at my email address, so when they/I finally got things sorted out, I expected a backlog of a week's worth of c.l.p email, or at least for the mail to resume flowing. However, despite seeing messages appearing in the online archives, I'm not receiving anything via email. When I send a "subscribe" message to mailman, it responds telling me that I'm already subscribed (and checking the settings on the web interface confirm that everything should be flowing). Is there something I've missed to get the messages flowing again? Thanks, -tkc PS: for obvious reasons, please CC me. From mail at timgolden.me.uk Fri Feb 3 13:25:06 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 3 Feb 2017 18:25:06 +0000 Subject: Meta: mailing list, bounces, and SPF? In-Reply-To: <20170203121551.0e85feee@bigbox.christie.dr> References: <20170203121551.0e85feee@bigbox.christie.dr> Message-ID: On 03/02/2017 18:15, Tim Chase wrote: > However, despite seeing messages appearing in the online archives, > I'm not receiving anything via email. When I send a "subscribe" > message to mailman, it responds telling me that I'm already > subscribed (and checking the settings on the web interface confirm > that everything should be flowing). Your email address had been tagged to receive no mail because of the bounces mailman was getting. (At least, I think that's what it was). I've cleared that now so you should be getting new emails via the list -- including this one! TJG From best_lay at yahoo.com Fri Feb 3 13:58:15 2017 From: best_lay at yahoo.com (Wildman) Date: Fri, 03 Feb 2017 12:58:15 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: On Fri, 03 Feb 2017 11:06:00 -0500, Neal Becker wrote: > I want to make sure any modules I build in the current directory overide any > others. To do this, I'd like sys.path to always have './' at the beginning. > > What's the best way to ensure this is always true whenever I run python3? In python, this method will work but it is only in effect for the running process that calls it while it is running. It is not system wide and it is not permanent. import os os.environ["PATH"] = os.environ["PATH"] + ":./" or os.environ["PATH"] = "./:" + os.environ["PATH"] (assuming Linux platform) To make it permanent for a certain user, add one of these lines to /home/user/.profile and log out/in: PATH="$PATH:./" or PATH="./:$PATH" To make it permanent for all users, add one of these pairs of lines to /etc/rc.local and reboot: export PATH=$PATH:./ exit 0 or export PATH=./:$PATH exit 0 Add 'exit 0' only if it doesn't exist already and it must be the last line. If /etc/rc.local does not exist, create it. -- GNU/Linux user #557453 The cow died so I don't need your bull! From best_lay at yahoo.com Fri Feb 3 14:07:33 2017 From: best_lay at yahoo.com (Wildman) Date: Fri, 03 Feb 2017 13:07:33 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: On Fri, 03 Feb 2017 12:58:15 -0600, Wildman wrote: > On Fri, 03 Feb 2017 11:06:00 -0500, Neal Becker wrote: > >> I want to make sure any modules I build in the current directory overide any >> others. To do this, I'd like sys.path to always have './' at the beginning. >> >> What's the best way to ensure this is always true whenever I run python3? > > In python, this method will work but it is only in effect > for the running process that calls it while it is running. > It is not system wide and it is not permanent. > > import os > os.environ["PATH"] = os.environ["PATH"] + ":./" > or > os.environ["PATH"] = "./:" + os.environ["PATH"] > > (assuming Linux platform) > To make it permanent for a certain user, add one of > these lines to /home/user/.profile and log out/in: > > PATH="$PATH:./" > or > PATH="./:$PATH" > > To make it permanent for all users, add one of > these pairs of lines to /etc/rc.local and reboot: > > export PATH=$PATH:./ > exit 0 > or > export PATH=./:$PATH > exit 0 > > Add 'exit 0' only if it doesn't exist already and it > must be the last line. If /etc/rc.local does not > exist, create it. Sorry, I forgot something important. If you use /etc/rc.local, the execute bit must be set. -- GNU/Linux user #557453 The cow died so I don't need your bull! From torriem at gmail.com Fri Feb 3 15:19:30 2017 From: torriem at gmail.com (Michael Torrie) Date: Fri, 3 Feb 2017 13:19:30 -0700 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: Message-ID: On 02/03/2017 12:07 PM, Wildman via Python-list wrote: > Sorry, I forgot something important. If you use > /etc/rc.local, the execute bit must be set. I don't think this is what Neal Becker was asking about. He's talking about the Python module search path (sys.path) not the operating system PATH variable. Correct me if I'm wrong. From best_lay at yahoo.com Fri Feb 3 15:55:39 2017 From: best_lay at yahoo.com (Wildman) Date: Fri, 03 Feb 2017 14:55:39 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: On Fri, 03 Feb 2017 13:19:30 -0700, Michael Torrie wrote: > On 02/03/2017 12:07 PM, Wildman via Python-list wrote: >> Sorry, I forgot something important. If you use >> /etc/rc.local, the execute bit must be set. > > I don't think this is what Neal Becker was asking about. He's talking > about the Python module search path (sys.path) not the operating system > PATH variable. Correct me if I'm wrong. After re-reading the post I see you are correct. I will endeavor to read a little closer and a little slower in the future. -- GNU/Linux user #557453 "Well, that's quite different. Never mind." -Emily Litella From cs at zip.com.au Fri Feb 3 17:25:42 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 09:25:42 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: Message-ID: <20170203222542.GA63533@cskk.homeip.net> On 03Feb2017 14:55, Wildman wrote: >On Fri, 03 Feb 2017 13:19:30 -0700, Michael Torrie wrote: > >> On 02/03/2017 12:07 PM, Wildman via Python-list wrote: >>> Sorry, I forgot something important. If you use >>> /etc/rc.local, the execute bit must be set. >> >> I don't think this is what Neal Becker was asking about. He's talking >> about the Python module search path (sys.path) not the operating system >> PATH variable. Correct me if I'm wrong. > >After re-reading the post I see you are correct. >I will endeavor to read a little closer and a >little slower in the future. Also, what you describe with rc.local wouldn't work anyway, even if it had ben what was asked. Cheers, Cameron Simpson From info at wingware.com Fri Feb 3 17:34:00 2017 From: info at wingware.com (Wingware) Date: Fri, 03 Feb 2017 17:34:00 -0500 Subject: ANN: Wing Python IDE version 6.0.2 released Message-ID: <58950558.8040802@wingware.com> Hi, We've just released Wing version 6.0.2 which expands and improves support for remote development, adds a drop down of found Python installations, introduces symbol name style refactoring operations, improves multi-selection with the mouse, fixes debugging Jupyter notebooks, and makes many other minor improvements. For details, see the change log at http://wingware.com/pub/wingide/6.0.2/CHANGELOG.txt Wing 6 is the latest major release in Wingware's family of Python IDEs, including Wing Pro, Wing Personal, and Wing 101. Wing 6 adds many new features, introduces a new annual license option, and makes some changes to the product line. New Features * Improved Multiple Selections: Quickly add selections and edit them all at once * Easy Remote Development: Work seamlessly on remote Linux, OS X, and Raspberry Pi systems * Debugging in the Python Shell: Reach breakpoints and exceptions in (and from) the Python Shell * Recursive Debugging: Debug code invoked in the context of stack frames that are already being debugged * PEP 484 and PEP 526 Type Hinting: Inform Wing's static analysis engine of types it cannot infer * Support for Python 3.6 and Stackless 3.4: Use async and other new language features * Optimized debugger: Run faster, particularly in multi-process and multi-threaded code * Support for OS X full screen mode: Zoom to a virtual screen, with auto-hiding menu bar * Added a new One Dark color palette: Enjoy the best dark display style yet * Updated French and German localizations: Thanks to Jean Sanchez, Laurent Fasnacht, and Christoph Heitkamp For a much more detailed overview of new features see the release notice at http://wingware.com/news/2017-02-02 Annual Use License Option Wing 6 adds the option of purchasing a lower-cost expiring annual license for Wing Pro. An annual license includes access to all available Wing Pro versions while it is valid, and then ceases to function if it is now renewed. Pricing for annual licenses is US$ 179/user for Commercial Use and US$ 69/user for Non-Commercial Use. Perpetual licenses for Wing Pro will continue to be available at the same pricing. The cost of extending Support+Upgrades subscriptions on Non-Commercial Use perpetual licenses for Wing Pro has also been dropped from US$ 89 to US$ 39 per user. For details, see https://wingware.com/store/purchase Wing Personal is Free Wing Personal is now free and no longer requires a license to run. It now also includes the Source Browser, PyLint, and OS Commands tools, and supports the scripting API and Perspectives. However, Wing Personal does not include Wing Pro's advanced editing, debugging, testing and code management features, such as remote host access, refactoring, find uses, version control, unit testing, interactive debug probe, multi-process and child process debugging, move program counter, conditional breakpoints, debug watch, framework-specific support (for matplotlib, Django, and others), find symbol in project, and other features. Links Release notice: http://wingware.com/news/2017-02-02 Free trial: http://wingware.com/wingide/trial Downloads: http://wingware.com/downloads Feature list: http://wingware.com/wingide/features Buy: http://wingware.com/store/purchase Upgrade: https://wingware.com/store/upgrade Questions? Don't hesitate to email us at support at wingware.com. Thanks, -- Stephan Deibel Wingware | Python IDE The Intelligent Development Environment for Python Programmers wingware.com From ben+python at benfinney.id.au Fri Feb 3 18:13:50 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 04 Feb 2017 10:13:50 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: <85bmuirhox.fsf@benfinney.id.au> Neal Becker writes: > I want to make sure any modules I build in the current directory > overide any others. To do this, I'd like sys.path to always have './' > at the beginning. The ?sys.path? list is used only for *absolute* imports. Modules in the current directory should not be imported with an absolute path[0]. So, the current directory should not be in ?sys.path?. So, for the past ten years and more, Python supports import of modules from the current directory with an explicit *relative* path:: # Absolute imports, searching ?sys.path?. import datetime from collections import namedtuple # Relative imports, starting from this module's directory. from . import foo from .bar import baz See , in particular . [0]: Why not? See the rationale for forcing absolute import by default , ?[I]t is not clear whether ?import foo? refers to a top-level module or to another module inside the package.[?] It's a particularly difficult problem inside packages because [without explicit relative import syntax] there's no way to specify which module is meant.? -- \ ?True greatness is measured by how much freedom you give to | `\ others, not by how much you can coerce others to do what you | _o__) want.? ?Larry Wall | Ben Finney From best_lay at yahoo.com Fri Feb 3 18:21:08 2017 From: best_lay at yahoo.com (Wildman) Date: Fri, 03 Feb 2017 17:21:08 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: On Sat, 04 Feb 2017 09:25:42 +1100, Cameron Simpson wrote: > On 03Feb2017 14:55, Wildman wrote: >>On Fri, 03 Feb 2017 13:19:30 -0700, Michael Torrie wrote: >> >>> On 02/03/2017 12:07 PM, Wildman via Python-list wrote: >>>> Sorry, I forgot something important. If you use >>>> /etc/rc.local, the execute bit must be set. >>> >>> I don't think this is what Neal Becker was asking about. He's talking >>> about the Python module search path (sys.path) not the operating system >>> PATH variable. Correct me if I'm wrong. >> >>After re-reading the post I see you are correct. >>I will endeavor to read a little closer and a >>little slower in the future. > > Also, what you describe with rc.local wouldn't work anyway, even if it had ben > what was asked. > > Cheers, > Cameron Simpson Of course, you are correct. I don't know where my head was. I think my tongue got in front of my eye teeth and I could not see what I was saying. :-) If anyone is interested the correct way is to add this to /etc/profile (at the bottom): PATH=$PATH:./ export PATH -- GNU/Linux user #557453 The cow died so I don't need your bull! From cs at zip.com.au Fri Feb 3 20:07:44 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 12:07:44 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: Message-ID: <20170204010744.GA66237@cskk.homeip.net> On 03Feb2017 17:21, Wildman wrote: >On Sat, 04 Feb 2017 09:25:42 +1100, Cameron Simpson wrote: >> Also, what you describe with rc.local wouldn't work anyway, even if it had >> ben >> what was asked. > >Of course, you are correct. I don't know where my head >was. I think my tongue got in front of my eye teeth and >I could not see what I was saying. :-) > >If anyone is interested the correct way is to add this to >/etc/profile (at the bottom): > >PATH=$PATH:./ >export PATH Indeed. But this should usually be accompanied by the advice that this isn't a good idea. Having one's commands at the mercy of whatever directory one is standing in is a recipe for being subverted. While the risk is lessened by having "." at the end of the search path, that just means the attacker (== other user of this system) has to name their trojan horse commands after typing errors, such as the ever popular "gerp" search programme. Even with Python I'd feel it is better to not have "." in the sys.path; I'd be happier with a full path to a preferred source of modules. (Yes, I know Python likes to search the current directory for modules, notmy favourite feature.) Cheers, Cameron Simpson From cs at zip.com.au Fri Feb 3 20:10:29 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 12:10:29 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <85bmuirhox.fsf@benfinney.id.au> References: <85bmuirhox.fsf@benfinney.id.au> Message-ID: <20170204011029.GA86595@cskk.homeip.net> On 04Feb2017 10:13, Ben Finney wrote: >Neal Becker writes: >> I want to make sure any modules I build in the current directory >> overide any others. To do this, I'd like sys.path to always have './' >> at the beginning. > >The ?sys.path? list is used only for *absolute* imports. Modules in the >current directory should not be imported with an absolute path[0]. So, the >current directory should not be in ?sys.path?. > >So, for the past ten years and more, Python supports import of modules >from the current directory with an explicit *relative* path:: > > # Absolute imports, searching ?sys.path?. > import datetime > from collections import namedtuple > > # Relative imports, starting from this module's directory. > from . import foo > from .bar import baz > >See , in particular >. I wrote: >Yes, I know Python likes to search >the current directory for modules, notmy favourite feature. I stand corrected. Cheers, Cameron Simpson From steve+python at pearwood.info Fri Feb 3 20:16:21 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 04 Feb 2017 12:16:21 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: <58952b68$0$1595$c3e8da3$5496439d@news.astraweb.com> On Sat, 4 Feb 2017 03:06 am, Neal Becker wrote: > I want to make sure any modules I build in the current directory overide > any > others. To do this, I'd like sys.path to always have './' at the > beginning. > > What's the best way to ensure this is always true whenever I run python3? For some definition of "always"... I don't know about "best", but you can do this: 1. In your .bashrc file, or equivalent, set the environment variable PYTHONPATH: export PYTHONPATH='./;$PYTHONPATH' * * * If you need this to be site-wide, rather than per person, another possibility, untested, would be to get your site administrator to create a sitecustomize.py file. Suppose your Python is installed in /usr/local/lib/python3.5/ Get your site admin to create the following file: /usr/local/lib/python3.5/site-packages/sitecustomize.py containing the following code: import sys if './' not in sys.path: sys.path.insert(0, './') * * * * Alternatively, a per-user solution is to create your own usercustomize.py file, containing the same code as above. By default, the per-user site packages directory will be: # Unix: ~/.local/lib/pythonX.Y/site-packages # Mac: ~/Library/Python/X.Y/lib/python/site-packages # Windows: %APPDATA%\Python\PythonXY\site-packages where X Y are the major and minor version numbers, e.g. 3 5. See the documentation for the site module for more detail: https://docs.python.org/3/library/site.html -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Feb 3 20:46:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 04 Feb 2017 12:46:04 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <85bmuirhox.fsf@benfinney.id.au> Message-ID: <5895325f$0$1585$c3e8da3$5496439d@news.astraweb.com> On Sat, 4 Feb 2017 10:13 am, Ben Finney wrote: > Neal Becker writes: > >> I want to make sure any modules I build in the current directory >> overide any others. To do this, I'd like sys.path to always have './' >> at the beginning. > > The ?sys.path? list is used only for *absolute* imports. Modules in the > current directory should not be imported with an absolute path[0]. So, the > current directory should not be in ?sys.path?. [...] > [0]: Why not? See the rationale for forcing absolute import by default > , > ?[I]t is not clear whether ?import foo? refers to a top-level > module or to another module inside the package.[?] It's a > particularly difficult problem inside packages because [without > explicit relative import syntax] there's no way to specify which > module is meant.? I'm not sure that this reasoning applies to Neal's situation. He doesn't seem to be talking about a single package, as such, more like he has a bunch of unrelated libraries in a single directory. My *guess* is that he probably has something like a ~/python or ~/scripts directory, and he likes to cd into that directory before running one of the scripts. I don't know what problem Neal thinks he is solving by always inserting ./ at the front of sys.path. But the rationale above doesn't seem relevant to the *pure technical question* of inserting ./ at the start of sys.path. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From cs at zip.com.au Fri Feb 3 21:13:27 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 13:13:27 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <58952b68$0$1595$c3e8da3$5496439d@news.astraweb.com> References: <58952b68$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170204021327.GA96623@cskk.homeip.net> On 04Feb2017 12:16, Steve D'Aprano wrote: >On Sat, 4 Feb 2017 03:06 am, Neal Becker wrote: >> I want to make sure any modules I build in the current directory overide >> any >> others. To do this, I'd like sys.path to always have './' at the >> beginning. >> >> What's the best way to ensure this is always true whenever I run python3? > >For some definition of "always"... >I don't know about "best", but you can do this: > >1. In your .bashrc file, or equivalent, set the environment > variable PYTHONPATH: >export PYTHONPATH='./;$PYTHONPATH' You want double quotes (allowing parameter substitution) instead of single quotes here. Or, of course, no quotes at all. And the separator is ":", not ";". Personally, I'm against hacking the $PYTHONPATH this way at all. Far better to invoke the Python script via a shell script that includes the absolute path of the current directory (or the module directory) in the $PYTHONPATH. Cheers, Cameron Simpson That article and its poster have been canceled. - David B. O'Donnell, Sysadmin, America OnLine From cs at zip.com.au Fri Feb 3 21:26:51 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 13:26:51 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <5895325f$0$1585$c3e8da3$5496439d@news.astraweb.com> References: <5895325f$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170204022651.GA11570@cskk.homeip.net> CCed to Neal: could you confirm or refute my suppositions below, or further clarify? Thanks. On 04Feb2017 12:46, Steve D'Aprano wrote: >On Sat, 4 Feb 2017 10:13 am, Ben Finney wrote: >> Neal Becker writes: >>> I want to make sure any modules I build in the current directory >>> overide any others. To do this, I'd like sys.path to always have './' >>> at the beginning. >> >> The ?sys.path? list is used only for *absolute* imports. Modules in the >> current directory should not be imported with an absolute path[0]. So, the >> current directory should not be in ?sys.path?. >[...] >> [0]: Why not? See the rationale for forcing absolute import by default >> >, >> ?[I]t is not clear whether ?import foo? refers to a top-level >> module or to another module inside the package.[?] It's a >> particularly difficult problem inside packages because [without >> explicit relative import syntax] there's no way to specify which >> module is meant.? > >I'm not sure that this reasoning applies to Neal's situation. He doesn't >seem to be talking about a single package, as such, more like he has a >bunch of unrelated libraries in a single directory. My *guess* is that he >probably has something like a ~/python or ~/scripts directory, and he likes >to cd into that directory before running one of the scripts. > >I don't know what problem Neal thinks he is solving by always inserting ./ >at the front of sys.path. But the rationale above doesn't seem relevant to >the *pure technical question* of inserting ./ at the start of sys.path. I think it is relevant. Why? _Because_ we don't have Neal's wider context, which he hasn't provided. Purely technical advice is all very well, and definitiely a necessary component to answers. However, I can't count the number of times I've encountered people requesting a specific technical solution to a problem defined only in terms of the technical task. And when that particular solution is something that is _usually_ a bad idea, one should always accompany the techincal advice with a bit of discussion of related alternatives and the up and downsides. Particularly, why the requested technical solution is often not thought to be a good thing. And ideally with a request for the wider context that _causes_ the OP to request a particular solution. Now, we do actually have a bit of Neal's wider context: I want to make sure any modules I build in the current directory overide any others. His requested _particular_ technical solution is this: To do this, I'd like sys.path to always have './' at the beginning. I think that's probably a poor technical solution for reasons I've alluded to in another post. Ben has provided greater context around that can be done with imports so that Neal can perhaps choose another method with the same outcome in terms of his wider objective. From my own perspective, I often want something that feels like what I think Neal may want: when I'm debugging my own modules I want the "lib/python" subdirectory of my code repository at the start of my Python path in order that imports find my developement modules ahead of their installed/published copies. Now, I have _many_ development directories because I tend to make one for each branch in progress, and just cd to the particular dev tree I'm working on. In this circumstance I've got a personal script called "dev" which prepends the _absolute_ paths of the various lib subdirs to the relevant paths, so it sticks $PWD/lib/python to the front of $PYTHONPATH. And then it executes whatever command I supply: dev python -m cs.venti.store_tests for example. I suspect Neal could do with something like that instead. Neal, could you confirm or refute or clarify how this all fits your situation? Cheers, Cameron Simpson From steve+python at pearwood.info Fri Feb 3 21:59:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 04 Feb 2017 13:59:04 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <58952b68$0$1595$c3e8da3$5496439d@news.astraweb.com> <20170204021327.GA96623@cskk.homeip.net> Message-ID: <58954378$0$1584$c3e8da3$5496439d@news.astraweb.com> On Sat, 4 Feb 2017 01:13 pm, Cameron Simpson wrote: >>1. In your .bashrc file, or equivalent, set the environment >> variable PYTHONPATH: >>export PYTHONPATH='./;$PYTHONPATH' > > You want double quotes (allowing parameter substitution) instead of single > quotes here. Or, of course, no quotes at all. And the separator is ":", > not ";". Thanks for the correction. > Personally, I'm against hacking the $PYTHONPATH this way at all. Oh? Why's that? This is the whole purpose of PYTHONPATH. > Far better to invoke the Python script via a shell script that includes > the absolute path of the current directory (or the module directory) in > the $PYTHONPATH. For some definition of "better" -- that means you have double the cognitive load. For every script.py you want, you need to create both script.py (what you actually want) and script.sh (a wrapper that sets the environment). -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Feb 3 22:16:50 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 04 Feb 2017 14:16:50 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <85bmuirhox.fsf@benfinney.id.au> Message-ID: <589547a3$0$1611$c3e8da3$5496439d@news.astraweb.com> On Sat, 4 Feb 2017 10:13 am, Ben Finney wrote: > So, for the past ten years and more, Python supports import of modules > from the current directory with an explicit *relative* path:: > > # Absolute imports, searching ?sys.path?. > import datetime > from collections import namedtuple > > # Relative imports, starting from this module's directory. > from . import foo > from .bar import baz > > See , in particular > . I think you are conflating the package directory and . the current working directory. Relative imports do not, as far as I can see, have anything to do with the current working directory. I created these files in a directory under my home directory: [steve at ando ~]$ ls test eggs.py spam.py [steve at ando ~]$ cat test/eggs.py pass [steve at ando ~]$ cat test/spam.py from . import eggs print(eggs.__file__) I cd'ed into the test folder, and tried running spam.py: [steve at ando ~]$ cd test [steve at ando test]$ python3.5 spam.py Traceback (most recent call last): File "spam.py", line 1, in from . import eggs SystemError: Parent module '' not loaded, cannot perform relative import So relative imports using . dot have nothing to do with importing from the current directory. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From cs at zip.com.au Fri Feb 3 23:58:06 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 4 Feb 2017 15:58:06 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <58954378$0$1584$c3e8da3$5496439d@news.astraweb.com> References: <58954378$0$1584$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20170204045806.GA18037@cskk.homeip.net> On 04Feb2017 13:59, Steve D'Aprano wrote: >On Sat, 4 Feb 2017 01:13 pm, Cameron Simpson wrote: > >>>1. In your .bashrc file, or equivalent, set the environment >>> variable PYTHONPATH: >>>export PYTHONPATH='./;$PYTHONPATH' >> >> You want double quotes (allowing parameter substitution) instead of single >> quotes here. Or, of course, no quotes at all. And the separator is ":", >> not ";". > >Thanks for the correction. > >> Personally, I'm against hacking the $PYTHONPATH this way at all. > >Oh? Why's that? This is the whole purpose of PYTHONPATH. I'm only against doing it "this way", for values of "this way" including "putting a ., which is totally dependent on my working directory, in my $PYTHONPATH". I'm sorry I wasn't clear. >> Far better to invoke the Python script via a shell script that includes >> the absolute path of the current directory (or the module directory) in >> the $PYTHONPATH. > >For some definition of "better" -- that means you have double the cognitive >load. For every script.py you want, you need to create both script.py (what >you actually want) and script.sh (a wrapper that sets the environment). Sure, but (I hope) Neal's need is a little special purpose. Besides, when i do this with "installed" code, generally all the python is in a module in my normal $PYTHONPATH and the shell script, if it exists (eg to hack the $PYTHONPATH) looks a bit like this: #!/bin/sh set -ue PYTHONPATH=/path/to/extra/libs:$PYTHONPATH export PYTHONPATH exec python -m my.module ${1+"$@"} So you don't really install 2 scripts. You install a python module in your normal place, and the invoker is a shell script which might prehack the environment. My "dev" script is little different (though it mucks with $PATH and $PERL5LIB etc as well as $PYTHONPATH), but its whole purpose is to reduce my cognitive load, which just becomes "I want this directory as the basis for my dev env for this one command". Cheers, Cameron Simpson From ben+python at benfinney.id.au Sat Feb 4 02:02:33 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 04 Feb 2017 18:02:33 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <85bmuirhox.fsf@benfinney.id.au> <589547a3$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <857f56qvzq.fsf@benfinney.id.au> Steve D'Aprano writes: > On Sat, 4 Feb 2017 10:13 am, Ben Finney wrote: > > See , in particular > > . > > I think you are conflating the package directory and . the current > working directory. I was careful not to say ?current working directory?, for that reason. I didn't want to open the more complex topic of Python packages, so I let ?current directory? stand in for ?directory where the current module is located?. I'm glad that distinction did get mentioned, though. -- \ ?Capitalism has destroyed our belief in any effective power but | `\ that of self interest backed by force.? ?George Bernard Shaw | _o__) | Ben Finney From michael at felt.demon.nl Sat Feb 4 03:48:17 2017 From: michael at felt.demon.nl (Michael Felt) Date: Sat, 4 Feb 2017 09:48:17 +0100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <20170204010744.GA66237@cskk.homeip.net> References: <20170204010744.GA66237@cskk.homeip.net> Message-ID: <58959551.2030609@felt.demon.nl> On 04-Feb-17 02:07, Cameron Simpson wrote: > On 03Feb2017 17:21, Wildman wrote: >> On Sat, 04 Feb 2017 09:25:42 +1100, Cameron Simpson wrote: >>> Also, what you describe with rc.local wouldn't work anyway, even if >>> it had ben >>> what was asked. >> >> Of course, you are correct. I don't know where my head >> was. I think my tongue got in front of my eye teeth and >> I could not see what I was saying. :-) >> >> If anyone is interested the correct way is to add this to >> /etc/profile (at the bottom): >> >> PATH=$PATH:./ >> export PATH > > Indeed. But this should usually be accompanied by the advice that this > isn't a good idea. Having one's commands at the mercy of whatever > directory one is standing in is a recipe for being subverted. While > the risk is lessened by having "." at the end of the search path, that > just means the attacker (== other user of this system) has to name > their trojan horse commands after typing errors, such as the ever > popular "gerp" search programme. > > Even with Python I'd feel it is better to not have "." in the > sys.path; I'd be happier with a full path to a preferred source of > modules. (Yes, I know Python likes to search the current directory for > modules, notmy favourite feature.) > This sound like something that could be hidden, read moved, if not removed - to a feature only available if built with something resembling a --dev-build flag (Python3.7 of course, as there may be earlier projects that depend on it - certainly officially deprecate in all other Python3 releases - there never being a Python2.8 a deprecate in Python2.7 is pointless). > Cheers, > Cameron Simpson From jussi.piitulainen at helsinki.fi Sat Feb 4 04:27:01 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sat, 04 Feb 2017 11:27:01 +0200 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: Wildman writes: [snip] > If anyone is interested the correct way is to add this to > /etc/profile (at the bottom): > > PATH=$PATH:./ > export PATH Out of interest, can you think of a corresponding way that a mere user can remove the dot from their $PATH after some presumably well-meaning system administrator has put it there? Is there any simple shell command for it? One that works whether the dot is at the start, in the middle, or at the end, and with or without the slash, and whether it's there more than once or not at all. And I'd like it to be as short and simple as PATH="$PATH:.", please. From ndbecker2 at gmail.com Sat Feb 4 08:10:46 2017 From: ndbecker2 at gmail.com (Neal Becker) Date: Sat, 04 Feb 2017 08:10:46 -0500 Subject: best way to ensure './' is at beginning of sys.path? References: Message-ID: Neal Becker wrote: > I want to make sure any modules I build in the current directory overide > any > others. To do this, I'd like sys.path to always have './' at the > beginning. > > What's the best way to ensure this is always true whenever I run python3? Sorry if I was unclear, let me try to describe the problem more precisely. I have a library of modules I have written using boost::python. They are all in a directory under my home directory called 'sigproc'. In ~/.local/lib/python3.5/site-packages, I have --- sigproc.pth /home/nbecker /home/nbecker/sigproc --- The reason I have 2 here is so I could use either import modA or import sigproc.modA although I almost always just use import modA . Now I have started experimenting with porting to pybind11 to replace boost::python. I am working in a directory called pybind11-test. I built modules there, with the same names as ones in sigproc. What I observed, I believe, is that when I try in that directory, import modA it imported the old one in sigproc, not the new one in "./". This behavior I found surprising. I examined sys.path, and found it did not contain "./". Then I prepended "./" to sys.path and found import modA appeared to correctly import the module in the current directory. I think I want this behavior always, and was asking how to ensure it. Thanks. From best_lay at yahoo.com Sat Feb 4 10:19:38 2017 From: best_lay at yahoo.com (Wildman) Date: Sat, 04 Feb 2017 09:19:38 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: On Sat, 04 Feb 2017 11:27:01 +0200, Jussi Piitulainen wrote: > Wildman writes: > > [snip] > >> If anyone is interested the correct way is to add this to >> /etc/profile (at the bottom): >> >> PATH=$PATH:./ >> export PATH > > Out of interest, can you think of a corresponding way that a mere user > can remove the dot from their $PATH after some presumably well-meaning > system administrator has put it there? > > Is there any simple shell command for it? One that works whether the dot > is at the start, in the middle, or at the end, and with or without the > slash, and whether it's there more than once or not at all. > > And I'd like it to be as short and simple as PATH="$PATH:.", please. No, I do not know. You might try your question in a linux specific group. Personally I don't understand the danger in having the dot in the path. The './' only means the current directory. DOS and Windows has searched the current directory since their beginning. Is that also dangerous? -- GNU/Linux user #557453 The cow died so I don't need your bull! From saxri89 at gmail.com Sat Feb 4 12:21:53 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 4 Feb 2017 09:21:53 -0800 (PST) Subject: image handling(donwloading/display) in django using html and form Message-ID: <4f743e7a-6a67-43ac-ac87-c907ddada6a8@googlegroups.com> i want to do some image processing using Django and now work and on the image handling(donwloading/display) using Django. first on my task i dont want to store that images on my server. but i have some problems. that my code : urls.py from django.conf.urls import url from . import views urlpatterns = [ #url(r'^$',views.index, name='index'), url(r'^$',views.calc, name='calc'), ] file.html Title
{% csrf_token %} {{ form.as_p }}
{{ flow_direction_uri.url }} forms.py from django import forms class ImageUploadForm(forms.Form): """Image upload form.""" image = forms.ImageField() views.py from django.shortcuts import render from django.shortcuts import render_to_response from django.template import RequestContext from pygeoprocessing import routing from django import forms from blog.forms import ImageUploadForm def calc(request): form = ImageUploadForm() if request.method == "POST" and form.is_valid(): image_file = request.FILES['image'] form.save() '''do something image process''' '''and finaly i take that output in standalone python script out of django''' outpu1 = "outpu1.tif" outpu2 = "outpu2.tif" outpu3 = "outpu3.tif" outpu4 = "outpu4.tif" return render_to_response ('blog/calc.html', {'form':form,'output1':output1}, RequestContext(request)) and i take that error : UnboundLocalError at / local variable 'output1' referenced before assignment but i thing so its not that error why if i change to response request to {return render_to_response ('blog/calc.html', {'form':form}, RequestContext(request)) ` dont show me that error i can see in the browser two buttons where i can choose image from my files and submit button but i thing so that do nothing. maybe to use html forms and no Django forms ? From jussi.piitulainen at helsinki.fi Sat Feb 4 12:49:56 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sat, 04 Feb 2017 19:49:56 +0200 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: Wildman writes: > On Sat, 04 Feb 2017 11:27:01 +0200, Jussi Piitulainen wrote: > >> Wildman writes: >> >> [snip] >> >>> If anyone is interested the correct way is to add this to >>> /etc/profile (at the bottom): >>> >>> PATH=$PATH:./ >>> export PATH >> >> Out of interest, can you think of a corresponding way that a mere >> user can remove the dot from their $PATH after some presumably >> well-meaning system administrator has put it there? >> >> Is there any simple shell command for it? One that works whether the >> dot is at the start, in the middle, or at the end, and with or >> without the slash, and whether it's there more than once or not at >> all. >> >> And I'd like it to be as short and simple as PATH="$PATH:.", please. > > No, I do not know. You might try your question in > a linux specific group. Personally I don't understand > the danger in having the dot in the path. The './' > only means the current directory. DOS and Windows > has searched the current directory since their > beginning. Is that also dangerous? I'd just like to be able to decide for myself. (Which I am, of course. In shell it's just more annoying to remove than it is to add, as far as I know.) From alister.ware at ntlworld.com Sat Feb 4 13:17:56 2017 From: alister.ware at ntlworld.com (alister) Date: Sat, 04 Feb 2017 18:17:56 GMT Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: On Sat, 04 Feb 2017 09:19:38 -0600, Wildman wrote: > On Sat, 04 Feb 2017 11:27:01 +0200, Jussi Piitulainen wrote: > >> Wildman writes: >> >> [snip] >> >>> If anyone is interested the correct way is to add this to /etc/profile >>> (at the bottom): >>> >>> PATH=$PATH:./ >>> export PATH >> >> Out of interest, can you think of a corresponding way that a mere user >> can remove the dot from their $PATH after some presumably well-meaning >> system administrator has put it there? >> >> Is there any simple shell command for it? One that works whether the >> dot is at the start, in the middle, or at the end, and with or without >> the slash, and whether it's there more than once or not at all. >> >> And I'd like it to be as short and simple as PATH="$PATH:.", please. > > No, I do not know. You might try your question in a linux specific > group. Personally I don't understand the danger in having the dot in > the path. The './' > only means the current directory. DOS and Windows has searched the > current directory since their beginning. Is that also dangerous? one example of how it can be dangerous is if you accidentally download a program into your current directory with a name that matches a system command, it will get executed instead of the system command. this can be exploited by the nefarious to install a trojan, virus or other nasty. coupled with the average windows user's insistence in running with admin rights & clicking anything is sight is one of the reasons that windows has so much malware -- QOTD: "You want me to put *holes* in my ears and hang things from them? How... tribal." From grant.b.edwards at gmail.com Sat Feb 4 13:25:03 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 4 Feb 2017 18:25:03 +0000 (UTC) Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: On 2017-02-04, Wildman via Python-list wrote: > No, I do not know. You might try your question in a linux specific > group. Personally I don't understand the danger in having the dot > in the path. The './' only means the current directory. It allows a malicous user to put an evil executable someplace public like /tmp and have it executed accidentally. For example, let's say this executable file was named "sl" and placed in /tmp. ------------------------------sl------------------------------ #!/bin/bash rm -rf $HOME -------------------------------------------------------------- The next time you are in the /tmp directory looking for something, can you guess what happens when you mistype "ls" as "sl"? > DOS and Windows has searched the current directory since their > beginning. Is that also dangerous? Yes. -- Grant Edwards grant.b.edwards Yow! I am a traffic light, at and Alan Ginzberg kidnapped gmail.com my laundry in 1927! From best_lay at yahoo.com Sat Feb 4 13:56:58 2017 From: best_lay at yahoo.com (Wildman) Date: Sat, 04 Feb 2017 12:56:58 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> On Sat, 04 Feb 2017 18:25:03 +0000, Grant Edwards wrote: > On 2017-02-04, Wildman via Python-list wrote: > >> No, I do not know. You might try your question in a linux specific >> group. Personally I don't understand the danger in having the dot >> in the path. The './' only means the current directory. > > It allows a malicous user to put an evil executable someplace public > like /tmp and have it executed accidentally. For example, let's say > this executable file was named "sl" and placed in /tmp. > > ------------------------------sl------------------------------ > #!/bin/bash > rm -rf $HOME > -------------------------------------------------------------- > > The next time you are in the /tmp directory looking for something, can > you guess what happens when you mistype "ls" as "sl"? > >> DOS and Windows has searched the current directory since their >> beginning. Is that also dangerous? > > Yes. Your scenario assumes the malicious user has root access to be able to place a file into /tmp. And there would have to be some reason why I would be looking around in /tmp. After 10 years of using Linux, it hasn't happened yet. And last I would have to be a complete idiot. I suppose all that could be a reality, but, how many computers do you know of have been compromised in this manor? -- GNU/Linux user #557453 The cow died so I don't need your bull! From grant.b.edwards at gmail.com Sat Feb 4 14:12:55 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 4 Feb 2017 19:12:55 +0000 (UTC) Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: On 2017-02-04, Wildman via Python-list wrote: >> >> The next time you are in the /tmp directory looking for something, can >> you guess what happens when you mistype "ls" as "sl"? >> >>> DOS and Windows has searched the current directory since their >>> beginning. Is that also dangerous? >> >> Yes. > > Your scenario assumes the malicious user has root access > to be able to place a file into /tmp. Nope. /tmp is world-writable. > And there would have to be some reason why I would be looking around > in /tmp. After 10 years of using Linux, it hasn't happened yet. > And last I would have to be a complete idiot. To have put '.' in your path? Or to have typed 'sl' by mistake? > I suppose all that could be a reality, but, how many computers do > you know of have been compromised in this manor? I've known a few people over the years who've been caught by that trick. The "evil" program was always more of a joke and did no real harm. -- Grant Edwards grant.b.edwards Yow! JAPAN is a WONDERFUL at planet -- I wonder if we'll gmail.com ever reach their level of COMPARATIVE SHOPPING ... From dan at tombstonezero.net Sat Feb 4 14:17:50 2017 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 4 Feb 2017 19:17:50 -0000 (UTC) Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: On Sat, 04 Feb 2017 12:56:58 -0600, Wildman wrote: > On Sat, 04 Feb 2017 18:25:03 +0000, Grant Edwards wrote: >> It allows a malicous user to put an evil executable someplace public >> like /tmp and have it executed accidentally. For example, let's say >> this executable file was named "sl" and placed in /tmp. >> >> ------------------------------sl------------------------------ >> #!/bin/bash >> rm -rf $HOME >> -------------------------------------------------------------- >> >> The next time you are in the /tmp directory looking for something, can >> you guess what happens when you mistype "ls" as "sl"? >> >>> DOS and Windows has searched the current directory since their >>> beginning. Is that also dangerous? >> >> Yes. > Your scenario assumes the malicious user has root access to be able to > place a file into /tmp. And there would have to be some reason why I > would be looking around in /tmp. After 10 years of using Linux, it > hasn't happened yet. And last I would have to be a complete idiot. The malicious user doesn't need root access. My /tmp directory is, by design, writable by everyone. All it takes is a clever tar file that contains all relative paths except for Grant Edwards's "sl" under /tmp/sl. You unpack the archive somewhere under your own home directory, and tar writes "sl" into /tmp without the slightest hesitation. In addition to /tmp/sl, what prevents the malicious user from putting a malicious "ls" command right into . instead of /tmp? Unpack the archive, type ls to see what happened, and, well, it's already too late. From marko at pacujo.net Sat Feb 4 14:19:06 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 04 Feb 2017 21:19:06 +0200 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: <87efzdahn9.fsf@elektro.pacujo.net> Grant Edwards : > It allows a malicous user to put an evil executable someplace public > like /tmp and have it executed accidentally. For example, let's say > this executable file was named "sl" and placed in /tmp. > > ------------------------------sl------------------------------ > #!/bin/bash > rm -rf $HOME > -------------------------------------------------------------- > > The next time you are in the /tmp directory looking for something, can > you guess what happens when you mistype "ls" as "sl"? There's also the reverse problem. Unix/Linux users are accustomed to writing their own handy scripts and placing them under ~/bin, where the default PATH will pick them up. Now, a Linux system has numerous system commands; on mine: $ ls /usr/bin | wc -l 1790 Nobody can know that most of those commands even exist, so there's a good possibility that the user's private script accidentally shadows with one of the system commands. So if a system command were implemented as a shell script (as many of them are) that depended on other commands along PATH, it would start behaving in a random fashion when the user's personal script got launched accidentally. Example: ===Begin /usr/bin/c89=================================================== #!/bin/sh fl="-std=c89" for opt; do case "$opt" in -ansi|-std=c89|-std=iso9899:1990) fl="";; -std=*) echo "`basename $0` called with non ANSI/ISO C option $opt" >&2 exit 1;; esac done exec gcc $fl ${1+"$@"} ===End /usr/bin/c89===================================================== If I were to write a script called "~/bin/gcc", the "c89" command might start working in unexpected ways. A particularly nasty risk is to write a program called "test" and place it along PATH. The "test" system command is used all over the place. Now, that's why the distros are careful to place $HOME/bin as the final entry of PATH; the system commands take precedence over the user's personal ones. However, the user is free to define the PATH any way they like. There's a school of thought that a script should never rely on PATH but it should spell out the complete path of every command it executes (including "mv", "cp", "rm" and the like). The problem with that approach is that different distros have core commands in different directories. Marko From lew.pitcher at digitalfreehold.ca Sat Feb 4 14:20:05 2017 From: lew.pitcher at digitalfreehold.ca (Lew Pitcher) Date: Sat, 04 Feb 2017 14:20:05 -0500 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: On Saturday February 4 2017 13:56, in comp.lang.python, "Wildman" wrote: > On Sat, 04 Feb 2017 18:25:03 +0000, Grant Edwards wrote: > >> On 2017-02-04, Wildman via Python-list wrote: >> >>> No, I do not know. You might try your question in a linux specific >>> group. Personally I don't understand the danger in having the dot >>> in the path. The './' only means the current directory. >> >> It allows a malicous user to put an evil executable someplace public >> like /tmp and have it executed accidentally. For example, let's say >> this executable file was named "sl" and placed in /tmp. >> >> ------------------------------sl------------------------------ >> #!/bin/bash >> rm -rf $HOME >> -------------------------------------------------------------- >> >> The next time you are in the /tmp directory looking for something, can >> you guess what happens when you mistype "ls" as "sl"? >> >>> DOS and Windows has searched the current directory since their >>> beginning. Is that also dangerous? >> >> Yes. > > Your scenario assumes the malicious user has root access > to be able to place a file into /tmp. It doesn't take root access to write a file to /tmp In fact, /tmp is specifically set up to allow /any/ user to create /any/ file or directory in it. Witness: guest at bitsie:~$ whoami guest I'm not the root account guest at bitsie:~$ groups users audio video cdrom plugdev scanner Nor do I have proxy root access (wheel group) guest at bitsie:~$ ls /tmp 58949ba84867f 58949bab2fe41 ksocket-lpitcher/ 58949ba87a976 gpg-15lbqc/ lost+found/ 58949ba87b6d7 kde-lpitcher/ ssh-0DwhGZKgCeHE/ That's what /tmp has in it right now guest at bitsie:~$ cat >/tmp/dothis < #!/bin/bash > echo gotcha > EOF There. I've created a script as a file in /tmp No root access required; no special privileges at all guest at bitsie:~$ chmod a+x /tmp/dothis Hey! I've even made the file executable guest at bitsie:~$ ls -l /tmp total 120 -rw------- 1 lpitcher users 23735 Feb 3 10:03 58949ba84867f -rw------- 1 lpitcher users 23735 Feb 3 10:03 58949ba87a976 -rw------- 1 lpitcher users 23735 Feb 3 10:03 58949ba87b6d7 -rw------- 1 lpitcher users 23735 Feb 3 10:03 58949bab2fe41 -rwxr-xr-x 1 guest users 24 Feb 4 14:05 dothis* drwx------ 2 lpitcher users 4096 Feb 4 10:10 gpg-15lbqc/ drwx------ 2 lpitcher users 4096 Feb 4 13:47 kde-lpitcher/ drwx------ 2 lpitcher users 4096 Feb 4 14:01 ksocket-lpitcher/ drwx------ 2 root root 4096 Jan 10 2012 lost+found/ drwx------ 2 lpitcher users 4096 Feb 4 10:10 ssh-0DwhGZKgCeHE/ See? /tmp/dothis is an executable file, owned by guest guest at bitsie:~$ /tmp/dothis gotcha See? It is executable Now, imagine that guest had, instead, written /tmp/sl as #!/bin/bash cd $HOME rm -rf ./ and made it executable. If /tmp were part of the $PATH, and there were no other executables named "sl" found in PATH directories before the /tmp directory, then anyone making that simple typo would execute guest's /tmp/sl and lose their home directory. > And there would > have to be some reason why I would be looking around in > /tmp. If you've set your $PATH to include /tmp, then /that's/ the "reason why" you'd "be looking around in /tmp". But, this argument can be said for /any/ directory that untrusted users have write access to (including your own $HOME directory); include that directory in your $PATH, and you risk executing a trojan binary. > After 10 years of using Linux, it hasn't happened > yet. And last I would have to be a complete idiot. > > I suppose all that could be a reality, but, how many > computers do you know of have been compromised in this > manor? Probably many, especially in high-use, public or semi-restricted systems like those found in Universities, libraries, and other "public" institutions. Even corporate systems have this exposure, which is why large corporations spend a lot of money on Information Systems security. -- Lew Pitcher "In Skills, We Trust" PGP public key available upon request From dan at tombstonezero.net Sat Feb 4 17:31:33 2017 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 4 Feb 2017 22:31:33 -0000 (UTC) Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <87efzdahn9.fsf@elektro.pacujo.net> Message-ID: On Sat, 04 Feb 2017 21:19:06 +0200, Marko Rauhamaa wrote: > Now, that's why the distros are careful to place $HOME/bin as the > final entry of PATH; the system commands take precedence over the > user's personal ones. However, the user is free to define the PATH any > way they like. I deliberately put $HOME/bin at the beginning of my path so that I can override system commands. > There's a school of thought that a script should never rely on PATH > but it should spell out the complete path of every command it executes > (including "mv", "cp", "rm" and the like) ... It's usually sufficient to reset PATH at the top of a system script, often just to /bin and /usr/bin (and maybe /sbin and /usr/sbin, although the use case for /bin and /sbin has long since been overcome by events, and the trend is towards eliminating them or simply making them symlinks to /usr/bin and /usr/sbin). > ... The problem with that approach is that different distros have core > commands in different directories. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard takes care of that (and, in fact, corroborates what I wrote about the possibility of /bin and /sbin being symlinks). From marko at pacujo.net Sat Feb 4 17:53:35 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 05 Feb 2017 00:53:35 +0200 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <87efzdahn9.fsf@elektro.pacujo.net> Message-ID: <8760kpa7ps.fsf@elektro.pacujo.net> Dan Sommers : > On Sat, 04 Feb 2017 21:19:06 +0200, Marko Rauhamaa wrote: >> Now, that's why the distros are careful to place $HOME/bin as the >> final entry of PATH; the system commands take precedence over the >> user's personal ones. However, the user is free to define the PATH >> any way they like. > > I deliberately put $HOME/bin at the beginning of my path so that I can > override system commands. You have been warned. >> There's a school of thought that a script should never rely on PATH >> but it should spell out the complete path of every command it >> executes (including "mv", "cp", "rm" and the like) ... > > It's usually sufficient to reset PATH at the top of a system script, But they don't. For example, /usr/bin/mkofm (whatever that does...): ======================================================================== #!/bin/sh # Initial definition. For the moment only makes .tfm files. mktextfm "$@" ======================================================================== Marko From cs at zip.com.au Sat Feb 4 18:01:17 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 5 Feb 2017 10:01:17 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: Message-ID: <20170204230117.GA81905@cskk.homeip.net> On 04Feb2017 08:10, Neal Becker wrote: >Neal Becker wrote: >> I want to make sure any modules I build in the current directory overide >> any >> others. To do this, I'd like sys.path to always have './' at the >> beginning. >> What's the best way to ensure this is always true whenever I run python3? [...] >Sorry if I was unclear, let me try to describe the problem more precisely. > >I have a library of modules I have written using boost::python. They are >all in a directory under my home directory called 'sigproc'. > >In ~/.local/lib/python3.5/site-packages, I have > >--- sigproc.pth >/home/nbecker >/home/nbecker/sigproc >--- > >The reason I have 2 here is so I could use either > >import modA >or >import sigproc.modA >although I almost always just use >import modA Is this desirable? For myself, I would go to the effort of always using one convention. In particular, often a name like "modA" will be ambiguous/generic because it lives _inside_ a descriptive namespace (sigproc). By generally using "import modA" you leave yourself open to conflict with some other "modA" if such a thing is ever loaded in an environment in which your code must run. I'd be wanting to either go: import sigproc/modA or from sigproc import modA myself. >Now I have started experimenting with porting to pybind11 to replace >boost::python. I am working in a directory called pybind11-test. >I built modules there, with the same names as ones in sigproc. >What I observed, I believe, is that when I try in that directory, >import modA >it imported the old one in sigproc, not the new one in "./". Because python is using your general purpose "stable" sys.path. >This behavior I found surprising. I examined sys.path, and found it did not >contain "./". >Then I prepended "./" to sys.path and found >import modA >appeared to correctly import the module in the current directory. Might I commend to you my "dev" script suggestion. To my eyes it seems that you want "modA" from your development directory. For myself, when I do this, I just invoke command like this: dev python3 blah blah ... which runs things with the local dev tree at the front of sys.path. >I think I want this behavior always, and was asking how to ensure it. You may want it, but I recommend against it. Having "." in your sys.path, _especially_ at the front, leaves your programs open to subversion/misbehaviour simply if you stand in an unfortunate directory. It is like a little time bomb waiting to go off in your face. If you really want this, modify your $HOME/.profile (or equivalent, depending on your shell) to prepend "." to your $PYTHONPATH, eg: PYTHONPATH=.${PYTHONPATH:+":$PYTHONPATH"} export PYTHONPATH But really, consider making yourself a convenient script to make _temporarily_ using the current directory as a source for packages/modules easy to do. For _testing_ purposes. When your code is stable/releasable, release it into your normal environment, where everything will benefit. Mine is here: https://bitbucket.org/cameron_simpson/css/src/tip/bin/env-dev for comparison. I use a personal shell function to invoke it as "dev", but that is mere finger convenience. Cheers, Cameron Simpson From david.amadi at digital.beis.gov.uk Sat Feb 4 18:39:04 2017 From: david.amadi at digital.beis.gov.uk (david.amadi at digital.beis.gov.uk) Date: Sat, 4 Feb 2017 15:39:04 -0800 (PST) Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? Message-ID: Hello All, I?m a newbie to python programming ? got into it predominately for the purposes of machine learning and data mining and even though I?ve committed several weeks to learning the scripting language, I have struggled to fully grasp how it works. I?m looking to scrape title, video Id, view Count, like Count, dislike Count, comment Count, favourite Count etc off YouTube using a python script I found via an online tutorial. I have installed ?unidecode? and 'google-api-python-client' packages via my terminal. I have also enabled YouTube Data Api V3 and I?m getting the error below each time I run the script. Could anyone please point me in the right direction? Thanks a lot in advance for your help **** Traceback (most recent call last): File "/Users/*Path*/ML with Python/youtube_search.py", line 73, in youtube_search(args) File "/Users/*Path*/ML with Python/youtube_search.py", line 21, in youtube_search youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) File "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapiclient/discovery.py", line 226, in build credentials=credentials) File "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapiclient/discovery.py", line 358, in build_from_document credentials = _auth.default_credentials() File "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapiclient/_auth.py", line 41, in default_credentials return oauth2client.client.GoogleCredentials.get_application_default() File "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2client/client.py", line 1264, in get_application_default return GoogleCredentials._get_implicit_credentials() File "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2client/client.py", line 1254, in _get_implicit_credentials raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information. -- Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. From cs at zip.com.au Sat Feb 4 18:43:20 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 5 Feb 2017 10:43:20 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: Message-ID: <20170204234320.GA45885@cskk.homeip.net> On 04Feb2017 09:19, Wildman wrote: >[...] Personally I don't understand >the danger in having the dot in the path. The './' >only means the current directory. DOS and Windows >has searched the current directory since their >beginning. Is that also dangerous? Citing DOS and Windows as prior art doesn't support your "it seems safe" argument :-) When you run a program its behaviour should be predictable. If your execution path or python module path includes "." (or some relative-path equivalent) in it, the behaviour of the program becomes dependent on where you're standing when you invoke it. Who has not written a shell script called "ls" and left it in their home directory, ready for one's friends to visit and run it by accident? Mine kicked off a shell script that said "removing files now..." and started reciting fictitious "rm" commands. In _their_ home directory. (Since they were invoking it, that information is instantly available from $HOME.) This is why having "." in your path (of whatever flavour), _particularly_ at the front (but really in general), is hazardous. At the least your commands become subject to misbehaviour through picking up the local (wrong) code; at the worst your programs become subverted if somehow malevolent code is sitting around somewhere. Supposing you visit a dev directory with a testing (and _broken_) common module or command. Misadventure ensues. Supposing you're a sysadmin. Nasty! Supposing you've shunted some malware into a directory for examination? CDing _into_ that directoy to do the examination is then _actively dangerous_. The list goes one. This is why having "the current directory", of whatever flavour, in your path is a bad idea in general. Cheers, Cameron Simpson From cs at zip.com.au Sat Feb 4 18:47:40 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 5 Feb 2017 10:47:40 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> References: <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: <20170204234740.GA90690@cskk.homeip.net> On 04Feb2017 12:56, Wildman wrote: >On Sat, 04 Feb 2017 18:25:03 +0000, Grant Edwards wrote: >> The next time you are in the /tmp directory looking for something, can >> you guess what happens when you mistype "ls" as "sl"? [...] >Your scenario assumes the malicious user has root access >to be able to place a file into /tmp. /tmp is _publicly_ writable. _Any_ user can do that. >And there would >have to be some reason why I would be looking around in >/tmp. After 10 years of using Linux, it hasn't happened >yet. Amazing. I was looking around in /tmp in my first days of using UNIX. There's stuff in there. >And last I would have to be a complete idiot. If you've got "." in your $PATH, I am beginning to think that this thesis is supported. >I suppose all that could be a reality, but, how many >computers do you know of have been compromised in this >manor? Hmm. I've compromised my friends (with harmless pranks) in this way. These days that doesn't work so well became having "." in your path is not done. Cheers, Cameron Simpson From python at deborahswanson.net Sat Feb 4 22:16:25 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sat, 4 Feb 2017 19:16:25 -0800 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: Message-ID: <004401d27f5e$3c626360$27b23dae@sambora> david.amadi at digital.beis.gov.uk wrote, on Saturday, February 04, 2017 3:39 PM > > Hello All, > > I'm a newbie to python programming - got into it > predominately for the purposes of machine learning and data > mining and even though I've committed several weeks to > learning the scripting language, I have struggled to fully > grasp how it works. > > I'm looking to scrape title, video Id, view Count, like > Count, dislike Count, comment Count, favourite Count etc off > YouTube using a python script I found via an online tutorial. > > I have installed 'unidecode' and 'google-api-python-client' > packages via my terminal. I have also enabled YouTube Data > Api V3 and I'm getting the error below each time I run the script. > > Could anyone please point me in the right direction? > > Thanks a lot in advance for your help > > **** > Traceback (most recent call last): > File "/Users/*Path*/ML with Python/youtube_search.py", line > 73, in > youtube_search(args) > File "/Users/*Path*/ML with Python/youtube_search.py", line > 21, in youtube_search > youtube = build(YOUTUBE_API_SERVICE_NAME, > YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > nt/_helpers.py", line 133, in positional_wrapper > return wrapped(*args, **kwargs) > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > lient/discovery.py", line 226, in build > credentials=credentials) > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > nt/_helpers.py", line 133, in positional_wrapper > return wrapped(*args, **kwargs) > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > lient/discovery.py", line 358, in build_from_document > credentials = _auth.default_credentials() > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > lient/_auth.py", line 41, in default_credentials > return > oauth2client.client.GoogleCredentials.get_application_default() > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > nt/client.py", line 1264, in get_application_default > return GoogleCredentials._get_implicit_credentials() > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > nt/client.py", line 1254, in _get_implicit_credentials > raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) > oauth2client.client.ApplicationDefaultCredentialsError: The > Application Default Credentials are not available. They are > available if running in Google Compute Engine. Otherwise, the > environment variable GOOGLE_APPLICATION_CREDENTIALS must be > defined pointing to a file defining the credentials. See > https://developers.google.com/accounts/docs/application-defaul t-credentials for more information. -- Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. -- I'm also a relatively new python coder (little over a year), but it seems plain to me that "The Application Default Credentials ... are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credenti als for more information." If you're not running in Google Compute Engine, you need to make a file defining the credentials, and supposedly there are instructions or requirements in the link above. Looks like Anaconda tried to get the credentials by calling its _auth.default_credentials() function: credentials = _auth.default_credentials() from "python3.5/site-packages/googleapiclient/discovery.py", but it failed. I think you need to read that link and make the file it wants to find. Deborah From torriem at gmail.com Sun Feb 5 00:18:24 2017 From: torriem at gmail.com (Michael Torrie) Date: Sat, 4 Feb 2017 22:18:24 -0700 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: <9222f89c-82b8-709d-4684-0eeda1863cde@gmail.com> On 02/04/2017 08:19 AM, Wildman via Python-list wrote: > No, I do not know. You might try your question in > a linux specific group. Personally I don't understand > the danger in having the dot in the path. The './' > only means the current directory. DOS and Windows > has searched the current directory since their > beginning. Is that also dangerous? Because of how the DOS and Windows command-line interpreters work it's slightly less dangerous. That's because a lot of commands are built into the interpreter. Commands like dir, type, etc. So a malicious download can't really override those with local copies. In linux, a lot of critical commands are actual programs in the search path. Commands like ls, cat, etc. So if . is in the path, it's far easier for a malicious download (or script) to place shadow programs in the current directory that will run when you try to use what you think is a system command. If strict user/root separation is maintained, then the damage can be mitigated somewhat. Except for something like a shadow copy of sudo that snags your password, then uses it to execute an arbitrary script as root using the real sudo. Game over. From torriem at gmail.com Sun Feb 5 00:22:18 2017 From: torriem at gmail.com (Michael Torrie) Date: Sat, 4 Feb 2017 22:22:18 -0700 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: <28ec46a0-74c8-ca68-5a23-bd3b90a0ca86@gmail.com> On 02/04/2017 12:20 PM, Lew Pitcher wrote: > It doesn't take root access to write a file to /tmp > In fact, /tmp is specifically set up to allow /any/ user to create /any/ file > or directory in it. > > Witness: > > > guest at bitsie:~$ chmod a+x /tmp/dothis > > Hey! I've even made the file executable If I'm not mistaken I used to make /tmp a different partition when I managed uni lab computers and I would mount it as noexec. I also made sure that removable media was always mounted as noexec. The main reason for the latter was that ownership and permissions bits were blindly honored on mounted file systems and there's no way we would allow someone to bring a setuid command into our lab computers through a floppy, cd, or usb stick (heck it was probably zip drives back then). From eryksun at gmail.com Sun Feb 5 00:26:45 2017 From: eryksun at gmail.com (eryk sun) Date: Sun, 5 Feb 2017 05:26:45 +0000 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: <20170203222542.GA63533@cskk.homeip.net> Message-ID: On Sat, Feb 4, 2017 at 3:19 PM, Wildman via Python-list wrote: > > Personally I don't understand the danger in having the dot in the path. The './' only > means the current directory. DOS and Windows has searched the current directory > since their beginning. Is that also dangerous? On Windows Vista and later this can be disabled by defining the environment variable NoDefaultCurrentDirectoryInExePath. This affects cmd.exe and CreateProcess, and any program that calls NeedCurrentDirectoryForExePath [1]. However, it doesn't override setting "." in PATH, a practice that should be avoided anyway. [1]: https://msdn.microsoft.com/en-us/library/ms684269 From best_lay at yahoo.com Sun Feb 5 03:01:05 2017 From: best_lay at yahoo.com (Wildman) Date: Sun, 05 Feb 2017 02:01:05 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> Message-ID: <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> On Sat, 04 Feb 2017 19:12:55 +0000, Grant Edwards wrote: > On 2017-02-04, Wildman via Python-list wrote: >>> >>> The next time you are in the /tmp directory looking for something, can >>> you guess what happens when you mistype "ls" as "sl"? >>> >>>> DOS and Windows has searched the current directory since their >>>> beginning. Is that also dangerous? >>> >>> Yes. >> >> Your scenario assumes the malicious user has root access >> to be able to place a file into /tmp. > > Nope. /tmp is world-writable. Yea, I realized that right after I clicked post. I was thinking of the fact that /tmp is owned by root. >> And there would have to be some reason why I would be looking around >> in /tmp. After 10 years of using Linux, it hasn't happened yet. >> And last I would have to be a complete idiot. > > To have put '.' in your path? That is something I would never do. Not because I think it is dangerous but because it had never occurred to me. Anything that I run in the current directory, I always prefix it with './' out of habit. Never thought of doing anything else. > Or to have typed 'sl' by mistake? Well, maybe not an idiot but something would have to be going on to misspell a two letter command. >> I suppose all that could be a reality, but, how many computers do >> you know of have been compromised in this manor? > > I've known a few people over the years who've been caught by that > trick. The "evil" program was always more of a joke and did no real > harm. I don't consider that being compromised. Sure, you could trick someone into running a program that could mess with $HOME but that is all. For anyone, like me, that makes regular backups, that is not a big problem. To do any real damage to the system or install a key logger or some other malicious software, root access would be required. As a Linux user you already know that. That is the scenario where idiot would be the correct term. -- GNU/Linux user #557453 The cow died so I don't need your bull! From david.amadi at digital.beis.gov.uk Sun Feb 5 12:37:47 2017 From: david.amadi at digital.beis.gov.uk (Amadi, David) Date: Sun, 5 Feb 2017 17:37:47 +0000 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: <004401d27f5e$3c626360$27b23dae@sambora> References: <004401d27f5e$3c626360$27b23dae@sambora> Message-ID: Hi Deborah, Thank you for stepping in, I appreciate it. Here's the python code I'm working with - http://codegists.com/snippet/python/youtube_searchpy_valeraradu_python To be honest, I don't know how to define the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to a file defining the credentials. In the python script (link in the above line), you could only add the API Key, which I obtained and provided - but the script won't run. I look forward to hearing back from you. Regards, David On 5 February 2017 at 03:16, Deborah Swanson wrote: > david.amadi at digital.beis.gov.uk wrote, on > Saturday, February 04, 2017 3:39 PM > > > > Hello All, > > > > I'm a newbie to python programming - got into it > > predominately for the purposes of machine learning and data > > mining and even though I've committed several weeks to > > learning the scripting language, I have struggled to fully > > grasp how it works. > > > > I'm looking to scrape title, video Id, view Count, like > > Count, dislike Count, comment Count, favourite Count etc off > > YouTube using a python script I found via an online tutorial. > > > > I have installed 'unidecode' and 'google-api-python-client' > > packages via my terminal. I have also enabled YouTube Data > > Api V3 and I'm getting the error below each time I run the script. > > > > Could anyone please point me in the right direction? > > > > Thanks a lot in advance for your help > > > > **** > > Traceback (most recent call last): > > File "/Users/*Path*/ML with Python/youtube_search.py", line > > 73, in > > youtube_search(args) > > File "/Users/*Path*/ML with Python/youtube_search.py", line > > 21, in youtube_search > > youtube = build(YOUTUBE_API_SERVICE_NAME, > > YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/discovery.py", line 226, in build > > credentials=credentials) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/discovery.py", line 358, in build_from_document > > credentials = _auth.default_credentials() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/_auth.py", line 41, in default_credentials > > return > > oauth2client.client.GoogleCredentials.get_application_default() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1264, in get_application_default > > return GoogleCredentials._get_implicit_credentials() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1254, in _get_implicit_credentials > > raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) > > oauth2client.client.ApplicationDefaultCredentialsError: The > > Application Default Credentials are not available. They are > > available if running in Google Compute Engine. Otherwise, the > > environment variable GOOGLE_APPLICATION_CREDENTIALS must be > > defined pointing to a file defining the credentials. See > > https://developers.google.com/accounts/docs/application-defaul > t-credentials for more information. > > -- > Communications with the Department for Business, Innovation and Skills > may > be automatically logged, monitored and/or recorded for legal purposes. > -- > > I'm also a relatively new python coder (little over a year), but it > seems plain to me that "The Application Default Credentials ... are > available if running in Google Compute Engine. Otherwise, the > environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined > pointing to a file defining the credentials. See > https://developers.google.com/accounts/docs/application-default-credenti > als for more information." > > If you're not running in Google Compute Engine, you need to make a file > defining the credentials, and supposedly there are instructions or > requirements in the link above. > > Looks like Anaconda tried to get the credentials by calling its > _auth.default_credentials() function: > > credentials = _auth.default_credentials() > > from "python3.5/site-packages/googleapiclient/discovery.py", but it > failed. I think you need to read that link and make the file it wants to > find. > > Deborah > > -- David Amadi Digital Performance Analyst | Apprenticeships Service Department for Business, Energy and Industrial Strategy david.amadi at digital.beis.gov.uk M. 07904776300 Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. -- Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. From python at deborahswanson.net Sun Feb 5 14:58:09 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sun, 5 Feb 2017 11:58:09 -0800 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: Message-ID: <001001d27fea$2d4f1d00$27b23dae@sambora> OK David, I'm also a Linux newbie and my Linux PC has a dead power supply that I've been too sick to fix. Since it is a choice (given my limited energy resources), I've spent my time learning python on a very old Windows PC instead of reviving the Linux PC, and I've never used Google Apps. However, looking at the links provided, this is how I'd proceed: 1) At https://developers.google.com/identity/protocols/application-default-cre dentials (Google Application Default Credentials | Google Identity Platform), are the insructions you need to follow. Now, I think you make up your own path and filename for the environment variable, the more iffy question is what file it shoud be. And the answer appears to be that it's a file you download. It then tells you to follow several instructions, using the https://console.developers.google.com/projectselector/apis/credentials page, to get a file that automatically downloads to your computer. 2) It then says you have to make the environment variable GOOGLE_APPLICATION_CREDENTIALS, specifying that it points to the file you just downloaded. I don't have a working Linux machine to test this on, at least it sounds like you're using Linux (let me know if you're using Windows and I can test it there), but to permanently set an environment variable in Linux: Edit ~/.bashrc and add the line: export GOOGLE_APPLICATION_CREDENTIALS= (instructions are different for Windows, or Linux using something other than bash for shell scripts) That's what you need to do, and then your youtube_search.py should work (haha, well you can try it and see). Deborah David Amadi wrote, on February 05, 2017 9:38 AM > > Hi Deborah, > > > Thank you for stepping in, I appreciate it. > > > Here's the python code I'm working with - > http://codegists.com/snippet/python/youtube_searchpy_valeraradu_python > > > To be honest, I don't know how to define the environment > variable GOOGLE_APPLICATION_CREDENTIALS to point to a file > defining the credentials. In the python script (link in the > above line), you could only add the API Key, which I obtained > and provided - but the script won't run. > > > I look forward to hearing back from you. > > > Regards, > David > > > On 5 February 2017 at 03:16, Deborah Swanson > wrote: > > david.amadi at digital.beis.gov.uk wrote, on > Saturday, February 04, 2017 3:39 PM > > > > Hello All, > > > > I'm a newbie to python programming - got into it > > > predominately for the purposes of machine learning and data > mining and > > even though I've committed several weeks to learning the scripting > > language, I have struggled to fully grasp how it works. > > > > I'm looking to scrape title, video Id, view Count, like > Count, dislike > > Count, comment Count, favourite Count etc off YouTube using > a python > > script I found via an online tutorial. > > > > I have installed 'unidecode' and 'google-api-python-client' > packages > > via my terminal. I have also enabled YouTube Data Api V3 and I'm > > getting the error below each time I run the script. > > > > Could anyone please point me in the right direction? > > > > Thanks a lot in advance for your help > > > > **** > > Traceback (most recent call last): > > File "/Users/*Path*/ML with Python/youtube_search.py", > line 73, in > > > > youtube_search(args) > > File "/Users/*Path*/ML with Python/youtube_search.py", > line 21, in > > youtube_search > > youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, > > developerKey=DEVELOPER_KEY) > > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/discovery.py", line 226, in build > > credentials=credentials) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > > lient/discovery.py", line 358, in build_from_document > > credentials = _auth.default_credentials() > > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/_auth.py", line 41, in default_credentials > > return > > oauth2client.client.GoogleCredentials.get_application_default() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1264, in get_application_default > > return GoogleCredentials._get_implicit_credentials() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1254, in _get_implicit_credentials > > raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) > > oauth2client.client.ApplicationDefaultCredentialsError: The > > Application Default Credentials are not available. They are > > available if running in Google Compute Engine. Otherwise, the > > environment variable GOOGLE_APPLICATION_CREDENTIALS must be > > defined pointing to a file defining the credentials. See > > https://developers.google.com/accounts/docs/application-defaul > t-credentials for more information. > > -- > Communications with the Department for Business, Innovation > and Skills may be automatically logged, monitored and/or > recorded for legal purposes. > -- > > I'm also a relatively new python coder (little over a year), > but it seems plain to me that "The Application Default > Credentials ... are available if running in Google Compute > Engine. Otherwise, the environment variable > GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a > file defining the credentials. See > https://developers.google.com/accounts/docs/application-defaul t-credenti als for more information." If you're not running in Google Compute Engine, you need to make a file defining the credentials, and supposedly there are instructions or requirements in the link above. Looks like Anaconda tried to get the credentials by calling its _auth.default_credentials() function: credentials = _auth.default_credentials() from "python3.5/site-packages/googleapiclient/discovery.py", but it failed. I think you need to read that link and make the file it wants to find. Deborah -- David Amadi Digital Performance Analyst | Apprenticeships Service Department for Business, Energy and Industrial Strategy david.amadi at digital.beis.gov.uk M. 07904776300 Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. From steve+python at pearwood.info Sun Feb 5 17:07:34 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 06 Feb 2017 09:07:34 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> Message-ID: <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> On Sun, 5 Feb 2017 07:01 pm, Wildman wrote: > Sure, you > could trick someone into running a program that could > mess with $HOME but that is all. ?For anyone, like me, > that makes regular backups, that is not a big problem. > To do any real damage to the system or install a key > logger or some other malicious software, root access > would be required. The complacency of Linux users (and I include myself here) is frightening. Why do you value the OS more than your own personal files? In the worst case, you could re-install the OS is a couple of hours effort. Losing your personal files, your home directory and email, could be irreplaceable. You're also ignoring the possibility of privilege-escalation attacks. As far as "regular backups", well, you're just not thinking deviously enough. If I were to write a ransomware application, running as the regular user, I would have the application encrypt files and emails just a few at a time, over a period of many weeks, gradually increasing the rate. By the time the victim has realised that their files have been encrypted, their backups have been compromised too: you can restore from backup, but you'll be restoring the encrypted version. Obviously this requires tuning. How many files will people be willing to just write-off as lost rather than pay the ransom? How quickly do you accelerate the process of encrypting files to maximize the number of people who will pay? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Feb 5 17:26:40 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 6 Feb 2017 09:26:40 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 6, 2017 at 9:07 AM, Steve D'Aprano wrote: > As far as "regular backups", well, you're just not thinking deviously > enough. If I were to write a ransomware application, running as the regular > user, I would have the application encrypt files and emails just a few at a > time, over a period of many weeks, gradually increasing the rate. By the > time the victim has realised that their files have been encrypted, their > backups have been compromised too: you can restore from backup, but you'll > be restoring the encrypted version. > If you commit everything to git and keep an eye on your diffs before you push, the encryption would have to be _extremely_ sneaky. For starters, it'd have to infect the 'git' command, so it has all the same protections other people have been talking about. It'd need to somehow make the SHA1s match, or else simultaneously infect my system and whereever I'm pushing to (which is sometimes GitHub and sometimes my own server). So the first thing is to infect everyone's git so it accepts the corrupted files as well as the correct ones... AND it has to still be able to show diffs, or I'd notice it very quickly. Is that still frightening complacency? ChrisA From david.amadi at digital.beis.gov.uk Sun Feb 5 17:44:40 2017 From: david.amadi at digital.beis.gov.uk (Amadi, David) Date: Sun, 5 Feb 2017 22:44:40 +0000 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: <001001d27fea$2d4f1d00$27b23dae@sambora> References: <001001d27fea$2d4f1d00$27b23dae@sambora> Message-ID: If I understand your previous email well, I've created the bits I highlighted below and I'm still getting error messages - see below. I'm running the python script on a Macbook Pro. *DEVELOPER_KEY = "AIza*****"* *YOUTUBE_API_SERVICE_NAME = "youtube"* *YOUTUBE_API_VERSION = "v3"* *GOOGLE_APPLICATION_CREDENTIALS = "/Users/*path*/Downloads/Youtube API-67ty5.json"* *def youtube_search(options):* * youtube = build(YOUTUBE_API_SERVICE_NAME, GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY)* *And here's the error message that was printed on my console:* build() takes at most 2 positional arguments (3 given) Traceback (most recent call last): File "/Users/*path*/ML with Python/youtube_search.py", line 71, in youtube_search(args) File "/Users/*path*/ML with Python/youtube_search.py", line 19, in youtube_search youtube = build(YOUTUBE_API_SERVICE_NAME, GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) File "/Users/*path*/anaconda/lib/python3.5/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/*path*/anaconda/lib/python3.5/site-packages/googleapiclient/discovery.py", line 223, in build requested_url, discovery_http, cache_discovery, cache) File "/Users/*path*/anaconda/lib/python3.5/site-packages/googleapiclient/discovery.py", line 270, in _retrieve_discovery_doc resp, content = http.request(actual_url) AttributeError: 'str' object has no attribute 'request' On 5 February 2017 at 19:58, Deborah Swanson wrote: > OK David, > > I'm also a Linux newbie and my Linux PC has a dead power supply that > I've been too sick to fix. Since it is a choice (given my limited energy > resources), I've spent my time learning python on a very old Windows PC > instead of reviving the Linux PC, and I've never used Google Apps. > > However, looking at the links provided, this is how I'd proceed: > > 1) At > https://developers.google.com/identity/protocols/application-default-cre > dentials (Google Application Default Credentials | Google Identity > Platform), are the insructions you need to follow. > > Now, I think you make up your own path and filename for the environment > variable, the more iffy question is what file it shoud be. > > And the answer appears to be that it's a file you download. It then > tells you to follow several instructions, using the > https://console.developers.google.com/projectselector/apis/credentials > page, to get a file that automatically downloads to your computer. > > 2) It then says you have to make the environment variable > GOOGLE_APPLICATION_CREDENTIALS, specifying that it points to the file > you just downloaded. > > I don't have a working Linux machine to test this on, at least it sounds > like you're using Linux (let me know if you're using Windows and I can > test it there), but to permanently set an environment variable in Linux: > > Edit ~/.bashrc and add the line: > > export GOOGLE_APPLICATION_CREDENTIALS= > > (instructions are different for Windows, or Linux using something other > than bash for shell scripts) > > That's what you need to do, and then your youtube_search.py should work > (haha, well you can try it and see). > > Deborah > > David Amadi wrote, on February 05, 2017 9:38 AM > > > > Hi Deborah, > > > > > > Thank you for stepping in, I appreciate it. > > > > > > Here's the python code I'm working with - > > http://codegists.com/snippet/python/youtube_searchpy_valeraradu_python > > > > > > To be honest, I don't know how to define the environment > > variable GOOGLE_APPLICATION_CREDENTIALS to point to a file > > defining the credentials. In the python script (link in the > > above line), you could only add the API Key, which I obtained > > and provided - but the script won't run. > > > > > > I look forward to hearing back from you. > > > > > > Regards, > > David > > > > > > On 5 February 2017 at 03:16, Deborah Swanson > > wrote: > > > > david.amadi at digital.beis.gov.uk wrote, on > > Saturday, February 04, 2017 3:39 PM > > > > > > Hello All, > > > > > > I'm a newbie to python programming - got into it > > > > > predominately for the purposes of machine learning and data > > mining and > > > even though I've committed several weeks to learning the scripting > > > language, I have struggled to fully grasp how it works. > > > > > > I'm looking to scrape title, video Id, view Count, like > > Count, dislike > > > Count, comment Count, favourite Count etc off YouTube using > > a python > > > script I found via an online tutorial. > > > > > > I have installed 'unidecode' and 'google-api-python-client' > > packages > > > via my terminal. I have also enabled YouTube Data Api V3 and I'm > > > getting the error below each time I run the script. > > > > > > Could anyone please point me in the right direction? > > > > > > Thanks a lot in advance for your help > > > > > > **** > > > Traceback (most recent call last): > > > File "/Users/*Path*/ML with Python/youtube_search.py", > > line 73, in > > > > > > youtube_search(args) > > > File "/Users/*Path*/ML with Python/youtube_search.py", > > line 21, in > > > youtube_search > > > youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, > > > developerKey=DEVELOPER_KEY) > > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > > nt/_helpers.py", line 133, in positional_wrapper > > > return wrapped(*args, **kwargs) > > > File > > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > > lient/discovery.py", line 226, in build > > > credentials=credentials) > > > File > > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > > nt/_helpers.py", line 133, in positional_wrapper > > > return wrapped(*args, **kwargs) > > > File > > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > > > > lient/discovery.py", line 358, in build_from_document > > > credentials = _auth.default_credentials() > > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > > lient/_auth.py", line 41, in default_credentials > > > return > > > oauth2client.client.GoogleCredentials.get_application_default() > > > File > > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > > nt/client.py", line 1264, in get_application_default > > > return GoogleCredentials._get_implicit_credentials() > > > File > > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > > nt/client.py", line 1254, in _get_implicit_credentials > > > raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) > > > oauth2client.client.ApplicationDefaultCredentialsError: The > > > Application Default Credentials are not available. They are > > > available if running in Google Compute Engine. Otherwise, the > > > environment variable GOOGLE_APPLICATION_CREDENTIALS must be > > > defined pointing to a file defining the credentials. See > > > https://developers.google.com/accounts/docs/application-defaul > > t-credentials for more information. > > > > -- > > Communications with the Department for Business, Innovation > > and Skills may be automatically logged, monitored and/or > > recorded for legal purposes. > > -- > > > > I'm also a relatively new python coder (little over a year), > > but it seems plain to me that "The Application Default > > Credentials ... are available if running in Google Compute > > Engine. Otherwise, the environment variable > > GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a > > file defining the credentials. See > > https://developers.google.com/accounts/docs/application-defaul > t-credenti > als for more information." > > If you're not running in Google Compute Engine, you need to make a file > defining the credentials, and supposedly there are instructions or > requirements in the link above. > > Looks like Anaconda tried to get the credentials by calling its > _auth.default_credentials() function: > > credentials = _auth.default_credentials() > > from "python3.5/site-packages/googleapiclient/discovery.py", but it > failed. I think you need to read that link and make the file it wants to > find. > > Deborah > > > > > > > > -- > > David Amadi > > Digital Performance Analyst | Apprenticeships Service Department for > Business, Energy and Industrial Strategy > > > david.amadi at digital.beis.gov.uk > > > M. 07904776300 > > Communications with the Department for Business, Innovation and Skills > may be automatically logged, monitored and/or recorded for legal > purposes. > > > Communications with the Department for Business, Innovation and Skills > may be automatically logged, monitored and/or recorded for legal > purposes. > > -- David Amadi Digital Performance Analyst | Apprenticeships Service Department for Business, Energy and Industrial Strategy david.amadi at digital.beis.gov.uk M. 07904776300 Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. -- Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. From phaas3 at u.rochester.edu Sun Feb 5 18:27:08 2017 From: phaas3 at u.rochester.edu (phaas3 at u.rochester.edu) Date: Sun, 5 Feb 2017 15:27:08 -0800 (PST) Subject: Beginner help Message-ID: <6c0e1e1f-185a-48cc-98fd-c9afbc9f708c@googlegroups.com> Hi everyone. I'm new to python and have hit a bit of a wall with an assignment I'm working on. I created a number of classes and instantiated them, now I need to create a list out of them. I am looking for something more elegant than appending each object to the list as I instantiate it. I tried setting up an if loop, but didn't get far with it. Any suggestions? From python at deborahswanson.net Sun Feb 5 19:08:45 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sun, 5 Feb 2017 16:08:45 -0800 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: Message-ID: <003101d2800d$2f33f4b0$27b23dae@sambora> Hi David, Well, I really don't know the first thing about Macs, but it looks like you got the download and environment variable right, at least it's not complaining about that. Looks like the current problem is with the parameters you're passing to the build() function. I don't have the apiclient module installed, so I can't look at it for myself, but here is the critical error: build() takes at most 2 positional arguments (3 given) This means that you've given too many parameters to build(). I'm a bit confused because your youtube_search()code is calling build() with: youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY), which is passing 3 arguments, and you're using youtube = build(YOUTUBE_API_SERVICE_NAME, GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY), which is passing 4 arguments. In either case the error you're getting is complaining that build() only takes 2 arguments, so it's not matching up with either youtube_search's code or your call to build(). It looks like you don't need the GOOGLE_APPLICATION_CREDENTIALS argument, but more serious than that is the build() that youtube_search is calling only takes 2 arguments and the script is calling with 3 arguments. So the script is fundamentally flawed to begin with. If you want to proceed with this script, you need to get a look at the build() function to see what arguments it wants. You can try altering the script to call build with only the arguments it will use, and try that. But I suspect there are deeper problems, most likely that apiclient.discovery version yo have insalled is either newer or older than this youtube_search is expecting, in which case all bets are off on making this combination work. So you have 2 choices that I see, either getting a look at what the build() you have installed uses and try youtube_search with that, or looking for other versions of apiclient and getting a look at their .discovery submodule build() functions. If you find a build() taking 3 arguments, you could try importing that version of apiclient.discovery with this youtube_search. But all in all, if I had this problem I'd back up to what I really want to accomplish and try another approach to the problem. It may not be worth all the futzing around to make this youtube_search work. Deborah David Amadi wrote, on February 05, 2017 2:45 PM: If I understand your previous email well, I've created the bits I highlighted below and I'm still getting error messages - see below. I'm running the python script on a Macbook Pro. DEVELOPER_KEY = "AIza*****" YOUTUBE_API_SERVICE_NAME = "youtube" YOUTUBE_API_VERSION = "v3" GOOGLE_APPLICATION_CREDENTIALS = "/Users/*path*/Downloads/Youtube API-67ty5.json" def youtube_search(options): youtube = build(YOUTUBE_API_SERVICE_NAME, GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) And here's the error message that was printed on my console: build() takes at most 2 positional arguments (3 given) Traceback (most recent call last): File "/Users/*path*/ML with Python/youtube_search.py", line 71, in youtube_search(args) File "/Users/*path*/ML with Python/youtube_search.py", line 19, in youtube_search youtube = build(YOUTUBE_API_SERVICE_NAME, GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) File "/Users/*path*/anaconda/lib/python3.5/site-packages/oauth2client/_helper s.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/*path*/anaconda/lib/python3.5/site-packages/googleapiclient/disc overy.py", line 223, in build requested_url, discovery_http, cache_discovery, cache) File "/Users/*path*/anaconda/lib/python3.5/site-packages/googleapiclient/disc overy.py", line 270, in _retrieve_discovery_doc resp, content = http.request(actual_url) AttributeError: 'str' object has no attribute 'request' On 5 February 2017 at 19:58, Deborah Swanson wrote: OK David, I'm also a Linux newbie and my Linux PC has a dead power supply that I've been too sick to fix. Since it is a choice (given my limited energy resources), I've spent my time learning python on a very old Windows PC instead of reviving the Linux PC, and I've never used Google Apps. However, looking at the links provided, this is how I'd proceed: 1) At https://developers.google.com/ identity/protocols/application-default-cre dentials (Google Application Default Credentials | Google Identity Platform), are the insructions you need to follow. Now, I think you make up your own path and filename for the environment variable, the more iffy question is what file it shoud be. And the answer appears to be that it's a file you download. It then tells you to follow several instructions, using the https://console.developers. google.com/projectselector/apis/credentials page, to get a file that automatically downloads to your computer. 2) It then says you have to make the environment variable GOOGLE_APPLICATION_CREDENTIALS, specifying that it points to the file you just downloaded. I don't have a working Linux machine to test this on, at least it sounds like you're using Linux (let me know if you're using Windows and I can test it there), but to permanently set an environment variable in Linux: Edit ~/.bashrc and add the line: export GOOGLE_APPLICATION_CREDENTIALS= (instructions are different for Windows, or Linux using something other than bash for shell scripts) That's what you need to do, and then your youtube_search.py should work (haha, well you can try it and see). Deborah David Amadi wrote, on February 05, 2017 9:38 AM > > Hi Deborah, > > > Thank you for stepping in, I appreciate it. > > > Here's the python code I'm working with - > http://codegists.com/snippet/ python/youtube_searchpy_valeraradu_python > > > To be honest, I don't know how to define the environment > variable GOOGLE_APPLICATION_CREDENTIALS to point to a file > defining the credentials. In the python script (link in the > above line), you could only add the API Key, which I obtained > and provided - but the script won't run. > > > I look forward to hearing back from you. > > > Regards, > David > > > On 5 February 2017 at 03:16, Deborah Swanson > wrote: > > david.amadi at digital.beis.gov. uk wrote, on > Saturday, February 04, 2017 3:39 PM > > > > Hello All, > > > > I'm a newbie to python programming - got into it > > > predominately for the purposes of machine learning and data > mining and > > even though I've committed several weeks to learning the scripting > > language, I have struggled to fully grasp how it works. > > > > I'm looking to scrape title, video Id, view Count, like > Count, dislike > > Count, comment Count, favourite Count etc off YouTube using > a python > > script I found via an online tutorial. > > > > I have installed 'unidecode' and 'google-api-python-client' > packages > > via my terminal. I have also enabled YouTube Data Api V3 and I'm > > getting the error below each time I run the script. > > > > Could anyone please point me in the right direction? > > > > Thanks a lot in advance for your help > > > > **** > > Traceback (most recent call last): > > File "/Users/*Path*/ML with Python/youtube_search.py", > line 73, in > > > > youtube_search(args) > > File "/Users/*Path*/ML with Python/youtube_search.py", > line 21, in > > youtube_search > > youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, > > developerKey=DEVELOPER_KEY) > > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/discovery.py", line 226, in build > > credentials=credentials) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/_helpers.py", line 133, in positional_wrapper > > return wrapped(*args, **kwargs) > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > > lient/discovery.py", line 358, in build_from_document > > credentials = _auth.default_credentials() > > File > "/Users/*Path*/anaconda/lib/python3.5/site-packages/googleapic > > lient/_auth.py", line 41, in default_credentials > > return > > oauth2client.client.GoogleCredentials.get_application_default() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1264, in get_application_default > > return GoogleCredentials._get_implicit_credentials() > > File > > "/Users/*Path*/anaconda/lib/python3.5/site-packages/oauth2clie > > nt/client.py", line 1254, in _get_implicit_credentials > > raise ApplicationDefaultCredentialsError(ADC_HELP_MSG) > > oauth2client.client.ApplicationDefaultCredentialsError: The > > Application Default Credentials are not available. They are > > available if running in Google Compute Engine. Otherwise, the > > environment variable GOOGLE_APPLICATION_CREDENTIALS must be > > defined pointing to a file defining the credentials. See > > https://developers.google.com/ accounts/docs/application-defaul > t-credentials for more information. > > -- > Communications with the Department for Business, Innovation > and Skills may be automatically logged, monitored and/or > recorded for legal purposes. > -- > > I'm also a relatively new python coder (little over a year), > but it seems plain to me that "The Application Default > Credentials ... are available if running in Google Compute > Engine. Otherwise, the environment variable > GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a > file defining the credentials. See > https://developers.google.com/ accounts/docs/application-defaul t-credenti als for more information." If you're not running in Google Compute Engine, you need to make a file defining the credentials, and supposedly there are instructions or requirements in the link above. Looks like Anaconda tried to get the credentials by calling its _auth.default_credentials() function: credentials = _auth.default_credentials() from "python3.5/site-packages/googleapiclient/discovery.py", but it failed. I think you need to read that link and make the file it wants to find. Deborah -- David Amadi Digital Performance Analyst | Apprenticeships Service Department for Business, Energy and Industrial Strategy david.amadi at digital.beis.gov. uk M. 07904776300 Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. -- David Amadi Digital Performance Analyst | Apprenticeships Service Department for Business, Energy and Industrial Strategy david.amadi at digital.beis.gov.uk M. 07904776300 Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. Communications with the Department for Business, Innovation and Skills may be automatically logged, monitored and/or recorded for legal purposes. From breamoreboy at gmail.com Sun Feb 5 19:12:47 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 5 Feb 2017 16:12:47 -0800 (PST) Subject: Beginner help In-Reply-To: <6c0e1e1f-185a-48cc-98fd-c9afbc9f708c@googlegroups.com> References: <6c0e1e1f-185a-48cc-98fd-c9afbc9f708c@googlegroups.com> Message-ID: On Sunday, February 5, 2017 at 11:27:19 PM UTC, pha... at u.rochester.edu wrote: > Hi everyone. I'm new to python and have hit a bit of a wall with an assignment I'm working on. I created a number of classes and instantiated them, now I need to create a list out of them. I am looking for something more elegant than appending each object to the list as I instantiate it. I tried setting up an if loop, but didn't get far with it. Any suggestions? If loop, for heavens sake :) Try one of these beasties instead https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions Kindest regards. Mark Lawrence. From python at mrabarnett.plus.com Sun Feb 5 19:34:01 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 6 Feb 2017 00:34:01 +0000 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: <003101d2800d$2f33f4b0$27b23dae@sambora> References: <003101d2800d$2f33f4b0$27b23dae@sambora> Message-ID: On 2017-02-06 00:08, Deborah Swanson wrote: > Hi David, > > Well, I really don't know the first thing about Macs, but it looks like > you got the download and environment variable right, at least it's not > complaining about that. > > Looks like the current problem is with the parameters you're passing to > the build() function. I don't have the apiclient module installed, so I > can't look at it for myself, but here is the critical error: > > build() takes at most 2 positional arguments (3 given) > > This means that you've given too many parameters to build(). I'm a bit > confused because your youtube_search()code is calling build() with: > > youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, > developerKey=DEVELOPER_KEY), > > which is passing 3 arguments, That's 2 positional arguments and 1 keyword argument. > and you're using > > youtube = build(YOUTUBE_API_SERVICE_NAME, > GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, > developerKey=DEVELOPER_KEY), > > which is passing 4 arguments. That's 3 positional arguments and 1 keyword argument. > In either case the error you're getting is > complaining that build() only takes 2 arguments, so it's not matching up > with either youtube_search's code or your call to build(). > It's expecting at most 2 positional arguments. Any additional arguments would have to be keyword arguments. [snip] From python at deborahswanson.net Sun Feb 5 20:13:30 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sun, 5 Feb 2017 17:13:30 -0800 Subject: Python: How do I resolve oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available? In-Reply-To: Message-ID: <003e01d28016$3afe0ed0$27b23dae@sambora> MRAB wrote, on February 05, 2017 4:34 PM > > On 2017-02-06 00:08, Deborah Swanson wrote: > > Hi David, > > > > Well, I really don't know the first thing about Macs, but it looks > > like you got the download and environment variable right, at least > > it's not complaining about that. > > > > Looks like the current problem is with the parameters > you're passing > > to the build() function. I don't have the apiclient module > installed, > > so I can't look at it for myself, but here is the critical error: > > > > build() takes at most 2 positional arguments (3 given) > > > > This means that you've given too many parameters to > build(). I'm a bit > > confused because your youtube_search()code is calling build() with: > > > > youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, > > developerKey=DEVELOPER_KEY), > > > > which is passing 3 arguments, > > That's 2 positional arguments and 1 keyword argument. Well, I said I'm not an experienced python coder, and in this case I've always seen 'self' as the unspecified argument, but it could be a key argument. Would be nice to see all of the code and messages. And getting a look at the build() function would tell it all. > > and you're using > > > > youtube = build(YOUTUBE_API_SERVICE_NAME, > > GOOGLE_APPLICATION_CREDENTIALS, YOUTUBE_API_VERSION, > > developerKey=DEVELOPER_KEY), > > > > which is passing 4 arguments. > > That's 3 positional arguments and 1 keyword argument. > > > In either case the error you're getting is > > complaining that build() only takes 2 arguments, so it's > not matching > > up with either youtube_search's code or your call to build(). > > > > It's expecting at most 2 positional arguments. Any additional > arguments > would have to be keyword arguments. > > [snip] > If you're right that the error message says build() is expecting 2 positional arguments and 1 keyword argument, then simply dropping the GOOGLE_APPLICATION_CREDENTIALS argument might just work. From steve+python at pearwood.info Sun Feb 5 21:19:21 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 06 Feb 2017 13:19:21 +1100 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5897dd2a$0$1591$c3e8da3$5496439d@news.astraweb.com> On Mon, 6 Feb 2017 09:26 am, Chris Angelico wrote: > On Mon, Feb 6, 2017 at 9:07 AM, Steve D'Aprano > wrote: >> As far as "regular backups", well, you're just not thinking deviously >> enough. If I were to write a ransomware application, running as the >> regular user, I would have the application encrypt files and emails just >> a few at a time, over a period of many weeks, gradually increasing the >> rate. By the time the victim has realised that their files have been >> encrypted, their backups have been compromised too: you can restore from >> backup, but you'll be restoring the encrypted version. >> > > If you commit everything to git git is not a synonym for "version control system". You're going to put the emails you send into subversion before hitting send? What about the emails you receive? How about graphic and video designers? Just how well does hg cope with gigabytes of video data? *Maybe* applications like LibreOffice could be integrated with VCS, and that's actually not a bad idea, but for binary files that can get costly. Yes, hard drives are big, but if every time you hit Ctrl-S on a 400K spreadsheet you end up with another copy, plus overhead, that adds up quickly. Not everyone is going to either want, or be able to, keep multiple years worth of versioning data for all their content. (Let's not forget that Android phones and tablets are Linux too. Some people use Android as their main, or only, computing platform, with relatively restricted resources. Not everyone has 10TB of storage on their computer.) > and keep an eye on your diffs before you push, Every time you push a file, you're going to check whether every other file in your home directory has changed? > the encryption would have to be _extremely_ sneaky. For > starters, it'd have to infect the 'git' command, so it has all the > same protections other people have been talking about. Not at all. All it would need to do is mess with the git database behind the scenes: when encrypting file X, delete all the diffs for X. Or simply corrupt the repository. If people did this, then there would be an arms race with others writing software to repair the damage to the repository, or changing their backup regimen to include backing up the repo history (if you're not already doing this) but then the same problem applies: if the repo is corrupted subtly enough, you may not notice until the backups are all corrupted too. And of course, sometimes backups don't work... https://www.theregister.co.uk/2017/02/01/gitlab_data_loss/ > It'd need to > somehow make the SHA1s match, or else simultaneously infect my system > and whereever I'm pushing to (which is sometimes GitHub and sometimes > my own server). So the first thing is to infect everyone's git so it > accepts the corrupted files as well as the correct ones... AND it has > to still be able to show diffs, or I'd notice it very quickly. How will you notice? You edit file "funny_video_about_a_cat.mp4" and push it into VCS. Meanwhile the malware encrypts "birthday_party_invites.doc", pushes it into the repo *as you*, and once the encrypted version is there, deletes all the past history for that file. Because the encrypted version is now in the repo, you won't notice the file has changed until you try to open it in the word processor. Which you might not do until your next birthday, a year from now. Unless you make a habit of studying *in detail* the complete history of everything in the repo, *and* have a good enough memory to say "wait a minute, I didn't edit that file on November 16th, something is suspicious", a sufficiently sneaky and clever ransomware app will be able to subvert your VCS. Of course, so long as there are millions of Windows users with no backups, there are easier fish to fry. But consider that not all malware is targeted at arbitrary people. "Advanced Persistent Threats" may be targeted at you specifically, for any value of "you". > Is that still frightening complacency? Talk is cheap -- do you actually push everything into VCS and regularly check it for corruption and unexpected changes, or are you just suggesting this is what you will do if and when such sneaky malware starts targeting Linux users? Personally I'm not losing any sleep over this -- but that's because I am complacent, secure in the knowledge that I'm not important enough to have the sort of enemies that will build an APT against me, and that ordinary ransomware criminals have got easier targets to go after. :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Feb 5 22:06:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 6 Feb 2017 14:06:39 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: <5897dd2a$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> <5897dd2a$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Feb 6, 2017 at 1:19 PM, Steve D'Aprano wrote: > On Mon, 6 Feb 2017 09:26 am, Chris Angelico wrote: > >> On Mon, Feb 6, 2017 at 9:07 AM, Steve D'Aprano >> wrote: >>> As far as "regular backups", well, you're just not thinking deviously >>> enough. If I were to write a ransomware application, running as the >>> regular user, I would have the application encrypt files and emails just >>> a few at a time, over a period of many weeks, gradually increasing the >>> rate. By the time the victim has realised that their files have been >>> encrypted, their backups have been compromised too: you can restore from >>> backup, but you'll be restoring the encrypted version. >>> >> >> If you commit everything to git > > git is not a synonym for "version control system". I know. And someone could use Mercurial for the same purpose. I happen to use git. > You're going to put the emails you send into subversion before hitting send? > > What about the emails you receive? Those are all handled on my server, not my own system. I'm actually toying with putting those in git too, but at the moment, they're not. And it'd never be subversion. I don't like subversion. It would be git. > How about graphic and video designers? Just how well does hg cope with > gigabytes of video data? I've no idea, but I know that git can handle large amounts of data. (There are a couple of extensions that make it easier.) > *Maybe* applications like LibreOffice could be integrated with VCS, and > that's actually not a bad idea, but for binary files that can get costly. > Yes, hard drives are big, but if every time you hit Ctrl-S on a 400K > spreadsheet you end up with another copy, plus overhead, that adds up > quickly. Not everyone is going to either want, or be able to, keep multiple > years worth of versioning data for all their content. So don't use spreadsheets like that. If you're paranoid, *use plain text*. I invoice my clients in plain text, and keep them in a git repo. > (Let's not forget that Android phones and tablets are Linux too. Some people > use Android as their main, or only, computing platform, with relatively > restricted resources. Not everyone has 10TB of storage on their computer.) I don't have a phone so I can't speak to its practicality. >> and keep an eye on your diffs before you push, > > Every time you push a file, you're going to check whether every other file > in your home directory has changed? In that repository, yes. It's all done by the one command "git status". The only files you could corrupt without me noticing are those mentioned in .gitignore, which are either safe to lose (eg intermediate files, created from tracked files) or very private and small (eg my SSH private key - if someone corrupts that, I'll have to generate a new one, but that happens if I lose my HD too). >> the encryption would have to be _extremely_ sneaky. For >> starters, it'd have to infect the 'git' command, so it has all the >> same protections other people have been talking about. > > Not at all. All it would need to do is mess with the git database behind the > scenes: when encrypting file X, delete all the diffs for X. Or simply > corrupt the repository. Uhh, you can't just delete all the diffs. That would change the commit hash, and git would instantly notice. Corrupting the repo might work, but it's highly likely to break the 'git push', which I would notice. Also, I frequently eyeball 'gitk', so you would also have to hide it from the diffs. > If people did this, then there would be an arms race with others writing > software to repair the damage to the repository, or changing their backup > regimen to include backing up the repo history (if you're not already doing > this) but then the same problem applies: if the repo is corrupted subtly > enough, you may not notice until the backups are all corrupted too. Actually, it's more going to be that corruption will have to be done such that the SHA1 doesn't change *and* I don't spot it in the diff (if you just add a bunch of random junk to the file so its SHA1 is the same, I'm going to notice that). > And of course, sometimes backups don't work... > > https://www.theregister.co.uk/2017/02/01/gitlab_data_loss/ > > >> It'd need to >> somehow make the SHA1s match, or else simultaneously infect my system >> and whereever I'm pushing to (which is sometimes GitHub and sometimes >> my own server). So the first thing is to infect everyone's git so it >> accepts the corrupted files as well as the correct ones... AND it has >> to still be able to show diffs, or I'd notice it very quickly. > > How will you notice? You edit file "funny_video_about_a_cat.mp4" and push it > into VCS. Meanwhile the malware encrypts "birthday_party_invites.doc", > pushes it into the repo *as you*, and once the encrypted version is there, > deletes all the past history for that file. I would see that, because that file shouldn't have changed. And you can't just delete the past history. It doesn't work that way. > Because the encrypted version is now in the repo, you won't notice the file > has changed until you try to open it in the word processor. Which you might > not do until your next birthday, a year from now. Right. But I wouldn't have that as a word processor file - it would be a text file (reStructuredText or Markdown), so I would see the diff. > Unless you make a habit of studying *in detail* the complete history of > everything in the repo, *and* have a good enough memory to say "wait a > minute, I didn't edit that file on November 16th, something is suspicious", > a sufficiently sneaky and clever ransomware app will be able to subvert > your VCS. I don't study the *entire* history - just the bit since I last pushed (usually less than a day), and eyeball the most recent commit or two as a means of regaining context. Sufficiently-sneaky ransomware could rewrite all the history, but it would have to (a) rewrite it such that the SHA1s are the same, and (b) corrupt a minimum of two, possibly three, versions simultaneously, because otherwise there'll be a version that I can easily and quickly pull down. > Of course, so long as there are millions of Windows users with no backups, > there are easier fish to fry. But consider that not all malware is targeted > at arbitrary people. "Advanced Persistent Threats" may be targeted at you > specifically, for any value of "you". Right. I used to be the most obscure target in the world (running OS/2), but now that I'm on Linux, I'm on a major platform, and thus targetable. >> Is that still frightening complacency? > > Talk is cheap -- do you actually push everything into VCS and regularly > check it for corruption and unexpected changes, or are you just suggesting > this is what you will do if and when such sneaky malware starts targeting > Linux users? Yes, I do - not because I'm paranoid about this kind of malicious attack, but because I want to know about other configuration changes. If I do some Twitch streaming and OBS changes its configuration, even if I don't consciously commit that, I'll notice it the next time I do anything in my "general private stuff" repository - such as invoicing a client, which happens about every weekday. So that'll show up in 'git status', and I'll know to go look at it. Naive corruption that just changes the file and deletes the original would be utterly useless for all text files (because it's blatantly obvious and would result in an immediate checkout), and mostly useless for binary files (even if I commit the corrupted file, I can always checkout the old version later). So it would have to be a git-aware corruption. > Personally I'm not losing any sleep over this -- but that's because I am > complacent, secure in the knowledge that I'm not important enough to have > the sort of enemies that will build an APT against me, and that ordinary > ransomware criminals have got easier targets to go after. > > :-) This is true. This is very true. But the cool thing is that malice can be blocked by the same techniques that block accidental mistakes, and it's worth deploying those techniques because mistakes WILL HAPPEN. Frequently. My habit of keeping everything in git has saved me from at least one hard drive melt-down (I lost about half a day of work in rebuilding the computer and reauthenticating myself with my new keys), and it probably saves me from stupid fat-fingering at least a few times a month. If ever I get hit by a malicious attack, it would have to be a very sneaky one to be more dangerous than my own stupidity :-) ChrisA From jcroy95014 at yahoo.com Mon Feb 6 00:03:01 2017 From: jcroy95014 at yahoo.com (Jean-Claude Roy) Date: Mon, 6 Feb 2017 05:03:01 +0000 (UTC) Subject: Python installer hangs in Windows 7 References: <765703485.1601081.1486357381135.ref@mail.yahoo.com> Message-ID: <765703485.1601081.1486357381135@mail.yahoo.com> ? I am trying to install Python 3.6.0 on a Windows 7 computer. The download of 29.1 MB is successful and I get the nextwindow.? I?choose the "install now" selection and thatopens the Setup Program window. Now the trouble starts:I get "Installing:" and the Initialization progress...and nothing else. There is no additional disk activity, no progress on initialization, andeverything appears dead.? Even after 20 minutes there is zero progress. I've repeated this as both a user and the administrator of this Windowscomputer.? I get the same results in either case. If I go to the task manager it shows that Python 3.6.0 (32-bit) setup is running.? If I try to end the task Iget the message that the program is not responding. Do you have any suggestions as to how I can get past this? Thank you. From best_lay at yahoo.com Mon Feb 6 01:11:52 2017 From: best_lay at yahoo.com (Wildman) Date: Mon, 06 Feb 2017 00:11:52 -0600 Subject: best way to ensure './' is at beginning of sys.path? References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, 06 Feb 2017 09:07:34 +1100, Steve D'Aprano wrote: > On Sun, 5 Feb 2017 07:01 pm, Wildman wrote: > >> Sure, you >> could trick someone into running a program that could >> mess with $HOME but that is all. ?For anyone, like me, >> that makes regular backups, that is not a big problem. >> To do any real damage to the system or install a key >> logger or some other malicious software, root access >> would be required. > > The complacency of Linux users (and I include myself here) is frightening. No comment. :-) > Why do you value the OS more than your own personal files? In the worst > case, you could re-install the OS is a couple of hours effort. Losing your > personal files, your home directory and email, could be irreplaceable. I wold not say I value the OS more. It is that anything I have that I consider important does not stay in $HOME very long without being backed up or moved to an external drive. > You're also ignoring the possibility of privilege-escalation attacks. The odds of that happening is very low. You should know that. There are very few actual exploits in the wild. Whenever one is discovered, it is fixed quickly. You would be hard pressed to find more than a few examples of where a vulnerability was actually exploited. > As far as "regular backups", well, you're just not thinking deviously > enough. If I were to write a ransomware application, running as the regular > user, I would have the application encrypt files and emails just a few at a > time, over a period of many weeks, gradually increasing the rate. By the > time the victim has realised that their files have been encrypted, their > backups have been compromised too: you can restore from backup, but you'll > be restoring the encrypted version. > > Obviously this requires tuning. How many files will people be willing to > just write-off as lost rather than pay the ransom? How quickly do you > accelerate the process of encrypting files to maximize the number of people > who will pay? I should explain a few things that will make my position clearer. First of all, I am not advocating for anyone to change their computing practices. If you are comfortable with your methods, who am I to tell you different? I am an amateur programmer and therefore I do not make a living writing code. If I suddenly lost all my code, it would not be the end of the world for me. I would have enjoyment writing it again. Because of this I am not very paranoid when it come to my computer data, although I do practice safe surfing when it comes to the internet. Scripting and Java stays off unless it is needed by a 'known' site. Also, I never click unknown links without doing a little sniffing first. And last I would like to say that I admit some of the scenarios you and others have laid out could happen, but, in my circumstance, it is very unlikely. One would have a hard time placing a program on my computer and running it without me knowing about it. No, that is not a challenge. :-) -- GNU/Linux user #557453 The cow died so I don't need your bull! From tjreedy at udel.edu Mon Feb 6 02:52:58 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 6 Feb 2017 02:52:58 -0500 Subject: Python installer hangs in Windows 7 In-Reply-To: <765703485.1601081.1486357381135@mail.yahoo.com> References: <765703485.1601081.1486357381135.ref@mail.yahoo.com> <765703485.1601081.1486357381135@mail.yahoo.com> Message-ID: On 2/6/2017 12:03 AM, Jean-Claude Roy via Python-list wrote: > I am trying to install Python 3.6.0 on a Windows 7 computer. > The download of 29.1 MB is successful and I get the nextwindow. I choose the "install now" selection and thatopens the Setup Program window. > Now the trouble starts:I get "Installing:" and the Initialization progress...and nothing else. > There is no additional disk activity, no progress on initialization, andeverything appears dead. Even after 20 minutes there is zero progress. > I've repeated this as both a user and the administrator of this Windowscomputer. I get the same results in either case. > If I go to the task manager it shows that Python 3.6.0 (32-bit) setup is running. If I try to end the task Iget the message that the program is not responding. > Do you have any suggestions as to how I can get past this? Unless you verified the checksum of the downloaded binary, I would download again and retry. -- Terry Jan Reedy From python at deborahswanson.net Mon Feb 6 03:23:11 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 6 Feb 2017 00:23:11 -0800 Subject: How to add months to a date (datetime object)? In-Reply-To: <3c0bf486-ab97-45de-8a20-92caf13bf03d@googlegroups.com> Message-ID: <000701d28052$41c97100$27b23dae@sambora> bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > > for start of month to the beginning of next month > > from datetime import timedelta > from dateutil.relativedelta import relativedelta > > end_date = start_date + relativedelta(months=delta_period) + > timedelta(days=-delta_period) Where do you define 'delta_period', and what is your question? From python at deborahswanson.net Mon Feb 6 03:25:40 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 6 Feb 2017 00:25:40 -0800 Subject: Get Result based on the Top value In-Reply-To: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> Message-ID: <000801d28052$9aa03570$27b23dae@sambora> Meeran Rizvi wrote, on February 03, 2017 4:05 AM > > Hello guys, > Here i am creating a GUI which will act as a search engine. > When you search for a data in search box it will fetch the > results from the very first page in browser and it is > displayed in a Text widget When the TOP value is 10 it should > display only the first 10 results based on numbers.if it is > 20 it should display first 20 results based on the number. > How should i do that? Here is my script from Tkinter > import * import mechanize from bs4 import BeautifulSoup root = Tk() > > def clear_search():#Reset the search field > Search.delete(0,END) > Result.delete("1.0",END) > > def fun(self=0): > new = Search.get() > url = "https://duckduckgo.com/" > br = mechanize.Browser() > br.set_handle_robots(False) > br.open(url) > br.select_form(name="x") > br["q"] = str(new) > res = br.submit() > content = res.read() > #print content > soup = BeautifulSoup(content,"html.parser") > mylink = soup.find_all('a') > v = 0 > for i in mylink: > try: > if i.attrs['class'][0] == "result__a": > v = v + 1 > Result.insert(END,v) > Result.insert(END,i.text) > Result.insert(END,'\n') > elif i.attrs['class'][0] == "result__snippet": > Result.insert(END,i.text) > Result.insert(END,'\n') > Result.insert(END,i.attrs['href']) > Result.insert(END,'\n ') > > Result.insert(END,'------------------------------------------- > -------------------------------') > > except KeyError: > pass > > > with open("result1.xls", "w") as f: > f.write(content) > > Value = Label(root,text="Value:", font="-weight bold") > Value.grid(row=0,column=0,padx=15) > > Search = Entry(root,width=50) > Search.grid(row=0,column=1) > > Top = Label(root,text="TOP",font="-weight bold") > Top.grid(row=1,column=0) > > var = StringVar(root) > var.set('10') > > Dropdownlist = OptionMenu(root,var,'10','20','50','100') > Dropdownlist.grid(row=1,column=1,sticky="w") > > > Go = Button(root,text="GO",width=5,command=fun) > Go.bind("",fun) > Go.grid(row=1,column=1) > > > > > Reset = Button(root,text="RESET",width=5,command=clear_search) > Reset.grid(row=1,column=1,sticky="e") > > Result = Text(root,height=20,width=75) > Result.grid(row=2,column=1,padx=50,pady=10) > Scroll = Scrollbar(root,command=Result.yview) > Scroll.grid(row=2,column=2,sticky=N+S) > Result.config(yscrollcommand=Scroll.set) > > root.mainloop() > > What is your question? If you're getting an error, please include all of it. From __peter__ at web.de Mon Feb 6 04:16:27 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 06 Feb 2017 10:16:27 +0100 Subject: Get Result based on the Top value References: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> Message-ID: Meeran Rizvi wrote: > Hello guys, > Here i am creating a GUI which will act as a search engine. > When you search for a data in search box it will fetch the results from > the very first page in browser and it is displayed in a Text widget When > the TOP value is 10 it should display only the first 10 results based on > numbers.if it is 20 it should display first 20 results based on the > number. How should i do that? Do you know how to get the integer value from the OptionMenu (or rather the associated StringVar)? Use that to check if the current number of results (of which you are keeping track already with the v variable) has reached the limit. Then use a break statement to exit the for loop. > Here is my script > from Tkinter import * > import mechanize > from bs4 import BeautifulSoup > root = Tk() > > def clear_search():#Reset the search field > Search.delete(0,END) > Result.delete("1.0",END) > > def fun(self=0): > new = Search.get() > url = "https://duckduckgo.com/" > br = mechanize.Browser() > br.set_handle_robots(False) > br.open(url) > br.select_form(name="x") > br["q"] = str(new) > res = br.submit() > content = res.read() > #print content > soup = BeautifulSoup(content,"html.parser") > mylink = soup.find_all('a') > v = 0 > for i in mylink: > try: > if i.attrs['class'][0] == "result__a": > v = v + 1 > Result.insert(END,v) > Result.insert(END,i.text) > Result.insert(END,'\n') > elif i.attrs['class'][0] == "result__snippet": > Result.insert(END,i.text) > Result.insert(END,'\n') > Result.insert(END,i.attrs['href']) > Result.insert(END,'\n ') > Result.insert(END,'--------------------------------------------------------------------------') > > except KeyError: > pass > > > with open("result1.xls", "w") as f: > f.write(content) > > Value = Label(root,text="Value:", font="-weight bold") > Value.grid(row=0,column=0,padx=15) > > Search = Entry(root,width=50) > Search.grid(row=0,column=1) > > Top = Label(root,text="TOP",font="-weight bold") > Top.grid(row=1,column=0) > > var = StringVar(root) > var.set('10') > > Dropdownlist = OptionMenu(root,var,'10','20','50','100') > Dropdownlist.grid(row=1,column=1,sticky="w") > > > Go = Button(root,text="GO",width=5,command=fun) > Go.bind("",fun) > Go.grid(row=1,column=1) > > > > > Reset = Button(root,text="RESET",width=5,command=clear_search) > Reset.grid(row=1,column=1,sticky="e") > > Result = Text(root,height=20,width=75) > Result.grid(row=2,column=1,padx=50,pady=10) > Scroll = Scrollbar(root,command=Result.yview) > Scroll.grid(row=2,column=2,sticky=N+S) > Result.config(yscrollcommand=Scroll.set) > > root.mainloop() > > From hmmeeranrizvi18 at gmail.com Mon Feb 6 05:53:50 2017 From: hmmeeranrizvi18 at gmail.com (Meeran Rizvi) Date: Mon, 6 Feb 2017 02:53:50 -0800 (PST) Subject: Get Result based on the Top value In-Reply-To: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> References: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> Message-ID: <96485f11-a5c2-4c9e-a82f-38486e71f2a5@googlegroups.com> On Friday, February 3, 2017 at 5:35:13 PM UTC+5:30, Meeran Rizvi wrote: > Hello guys, > Here i am creating a GUI which will act as a search engine. > When you search for a data in search box it will fetch the results from the very first page in browser and it is displayed in a Text widget > When the TOP value is 10 it should display only the first 10 results based on numbers.if it is 20 it should display first 20 results based on the number. > How should i do that? > Here is my script > > from Tkinter import * > import mechanize > from bs4 import BeautifulSoup > root = Tk() > > def clear_search():#Reset the search field > Search.delete(0,END) > Result.delete("1.0",END) > > def fun(self=0): > new = Search.get() > url = "https://duckduckgo.com/" > br = mechanize.Browser() > br.set_handle_robots(False) > br.open(url) > br.select_form(name="x") > br["q"] = str(new) > res = br.submit() > content = res.read() > #print content > soup = BeautifulSoup(content,"html.parser") > mylink = soup.find_all('a') > v = 0 > for i in mylink: > try: > if i.attrs['class'][0] == "result__a": > v = v + 1 > Result.insert(END,v) > Result.insert(END,i.text) > Result.insert(END,'\n') > elif i.attrs['class'][0] == "result__snippet": > Result.insert(END,i.text) > Result.insert(END,'\n') > Result.insert(END,i.attrs['href']) > Result.insert(END,'\n ') > Result.insert(END,'--------------------------------------------------------------------------') > > except KeyError: > pass > > > with open("result1.xls", "w") as f: > f.write(content) > > Value = Label(root,text="Value:", font="-weight bold") > Value.grid(row=0,column=0,padx=15) > > Search = Entry(root,width=50) > Search.grid(row=0,column=1) > > Top = Label(root,text="TOP",font="-weight bold") > Top.grid(row=1,column=0) > > var = StringVar(root) > var.set('10') > > Dropdownlist = OptionMenu(root,var,'10','20','50','100') > Dropdownlist.grid(row=1,column=1,sticky="w") > > > Go = Button(root,text="GO",width=5,command=fun) > Go.bind("",fun) > Go.grid(row=1,column=1) > > > > > Reset = Button(root,text="RESET",width=5,command=clear_search) > Reset.grid(row=1,column=1,sticky="e") > > Result = Text(root,height=20,width=75) > Result.grid(row=2,column=1,padx=50,pady=10) > Scroll = Scrollbar(root,command=Result.yview) > Scroll.grid(row=2,column=2,sticky=N+S) > Result.config(yscrollcommand=Scroll.set) > > root.mainloop() > > Here i got it,using [var.get()] When the Top value is change to 10 it displays only the first 10 results,if it is changed to 20 it displays first 20 results.But i got a problem with my GUI the scroll bar height is not same as my Text widget.even i used sticky its height remains to be short only. from Tkinter import * import mechanize from bs4 import BeautifulSoup root = Tk() options = ["10","20","30"] var = StringVar(root) var.set('10') def clear_search():#Reset the search field Search.delete(0,END) Result.delete("1.0",END) def fun(self=0): new = Search.get() url = "https://duckduckgo.com/" br = mechanize.Browser() br.set_handle_robots(False) br.open(url) br.select_form(name="x") br["q"] = str(new) res = br.submit() content = res.read() #print content soup = BeautifulSoup(content,"html.parser") mylink = soup.find_all('a') v = 1 b = var.get() for i in mylink: try: if v <= int(b): if i.attrs['class'][0] == "result__a": Result.insert(END,v ) Result.insert(END,i.text) Result.insert(END,'\n') elif i.attrs['class'][0] == "result__snippet": Result.insert(END,i.text) Result.insert(END,'\n') Result.insert(END,i.attrs['href']) Result.insert(END,'\n ') Result.insert(END,'--------------------------------------------------------------------') v = v + 1 except KeyError: pass with open("result1.xls", "w") as f: f.write(content) Value = Label(root,text="Value:", font="-weight bold") Value.grid(row=0,column=0,sticky="W") Search = Entry(root,width=50) Search.grid(row=0,column=1) Top = Label(root,text="TOP",font="-weight bold") Top.grid(row=0,column=2,sticky="W") Go = Button(root,text="GO",width=5,command=fun) Go.bind("",fun) Go.grid(row=0,column=4,sticky="W") Dropdownlist = OptionMenu(root,var,*options) Dropdownlist.grid(row=0,column=3,padx=5,sticky="W") Reset = Button(root,text="RESET",width=5,command=clear_search) Reset.grid(row=0,column=5,padx=5,sticky="W") Result = Text(root,height=20,width=69) Result.place(x=10, y=40) Scroll = Scrollbar(root,command=Result.yview) Scroll.grid(row=1,column=6,sticky=N) Result.config(yscrollcommand=Scroll.set) root.mainloop() From hmmeeranrizvi18 at gmail.com Mon Feb 6 07:29:35 2017 From: hmmeeranrizvi18 at gmail.com (Meeran Rizvi) Date: Mon, 6 Feb 2017 04:29:35 -0800 (PST) Subject: Get Result based on the Top value In-Reply-To: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> References: <41f093af-9d3f-4157-88e9-72072b0b992d@googlegroups.com> Message-ID: <711af729-337e-467b-8778-c6fae3cea58f@googlegroups.com> On Friday, February 3, 2017 at 5:35:13 PM UTC+5:30, Meeran Rizvi wrote: > Hello guys, > Here i am creating a GUI which will act as a search engine. > When you search for a data in search box it will fetch the results from the very first page in browser and it is displayed in a Text widget > When the TOP value is 10 it should display only the first 10 results based on numbers.if it is 20 it should display first 20 results based on the number. > How should i do that? > Here is my script > > from Tkinter import * > import mechanize > from bs4 import BeautifulSoup > root = Tk() > > def clear_search():#Reset the search field > Search.delete(0,END) > Result.delete("1.0",END) > > def fun(self=0): > new = Search.get() > url = "https://duckduckgo.com/" > br = mechanize.Browser() > br.set_handle_robots(False) > br.open(url) > br.select_form(name="x") > br["q"] = str(new) > res = br.submit() > content = res.read() > #print content > soup = BeautifulSoup(content,"html.parser") > mylink = soup.find_all('a') > v = 0 > for i in mylink: > try: > if i.attrs['class'][0] == "result__a": > v = v + 1 > Result.insert(END,v) > Result.insert(END,i.text) > Result.insert(END,'\n') > elif i.attrs['class'][0] == "result__snippet": > Result.insert(END,i.text) > Result.insert(END,'\n') > Result.insert(END,i.attrs['href']) > Result.insert(END,'\n ') > Result.insert(END,'--------------------------------------------------------------------------') > > except KeyError: > pass > > > with open("result1.xls", "w") as f: > f.write(content) > > Value = Label(root,text="Value:", font="-weight bold") > Value.grid(row=0,column=0,padx=15) > > Search = Entry(root,width=50) > Search.grid(row=0,column=1) > > Top = Label(root,text="TOP",font="-weight bold") > Top.grid(row=1,column=0) > > var = StringVar(root) > var.set('10') > > Dropdownlist = OptionMenu(root,var,'10','20','50','100') > Dropdownlist.grid(row=1,column=1,sticky="w") > > > Go = Button(root,text="GO",width=5,command=fun) > Go.bind("",fun) > Go.grid(row=1,column=1) > > > > > Reset = Button(root,text="RESET",width=5,command=clear_search) > Reset.grid(row=1,column=1,sticky="e") > > Result = Text(root,height=20,width=75) > Result.grid(row=2,column=1,padx=50,pady=10) > Scroll = Scrollbar(root,command=Result.yview) > Scroll.grid(row=2,column=2,sticky=N+S) > Result.config(yscrollcommand=Scroll.set) > > root.mainloop() > > i just changed these things to fix the scroll bar height. Using ipady to increase the height of a scrollbar. Result = Text(root,height=20,width=69) Result.grid(rowspan=10,columnspan=40,sticky='W',padx=5) Scroll = Scrollbar(root,command=Result.yview) Scroll.grid(row=1,column=6,ipady=135) Result.config(yscrollcommand=Scroll.set) From skip.montanaro at gmail.com Mon Feb 6 07:56:38 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 6 Feb 2017 06:56:38 -0600 Subject: Strategies when hunting for dictionary corruption Message-ID: I'm wrapping some C++ libraries using pybind11. On the pybind11 side of things, I've written a simple type converter between Python's datetime.date objects and our internal C++ date objects. The first thing the type converter needs to do is to insure that the datetime C API is initialized: if (!PyDateTimeAPI) { PyDateTime_IMPORT; } This is failing because a dictionary internal to the import mechanism is being corrupted, the extensions dict in import.c. I highly doubt this is a Python issue. It's much more likely to be a bug in our C++ libraries or in pybind11. The problem is, dictionaries being rather dynamic objects (growing and sometimes shrinking), the actual data location which might be corrupted isn't fixed, but can move around. I've got a Python interpreter configured using --with-pydebug, it's compiled with -ggdb -O0, and I'm ready to watch some memory locations, but it's not clear what exactly I should watch or how/when to move my watchpoint(s) around. Should I not be thinking in terms of watchpoints? Is there a better way to approach this problem? (I also have valgrind at my disposal, but am not very skilled in its use.) Thx, Skip From bcare003 at odu.edu Mon Feb 6 09:16:46 2017 From: bcare003 at odu.edu (Bryan Carey) Date: Mon, 6 Feb 2017 09:16:46 -0500 Subject: Modify Setup In-Reply-To: References: Message-ID: Good evening! I just installed both python 3.4 & the PyCharm IDE this evening. While trying to run a simple "Hello World" program, I keep getting "Modify Setup" pop up windows. In addition to that, the output of my program is not displayed. Could you help me resolve my issue? On Sat, Feb 4, 2017 at 3:25 PM, Bryan Carey wrote: > Good evening! I just installed both python 3.4 & the PyCharm IDE this > evening. While trying to run a simple "Hello World" program, I keep getting > "Modify Setup" pop up windows. In addition to that, the output of my > program is not displayed. Could you help me resolve my issue? > > On Fri, Feb 3, 2017 at 7:00 PM, Bryan Carey wrote: > >> Good evening! I just installed both python 3.4 & the PyCharm IDE this >> evening. While trying to run a simple "Hello World" program, I keep getting >> "Modify Setup" pop up windows. In addition to that, the output of my >> program is not displayed. Could you help me resolve my issue? >> > > From hmmeeranrizvi18 at gmail.com Mon Feb 6 09:43:06 2017 From: hmmeeranrizvi18 at gmail.com (Meeran Rizvi) Date: Mon, 6 Feb 2017 06:43:06 -0800 (PST) Subject: search for a data in my browser via script Message-ID: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> Hello guys, Here i am writing a script which will open my chrome browser and opens the URL www.google.com. But how to search for a data via script. for example i need to search for 'Rose' in google.com via script. how to do that? import webbrowser url="www.google.com" chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' webbrowser.get(chrome_path) webbrowser.open(url) From torriem at gmail.com Mon Feb 6 10:55:02 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 6 Feb 2017 08:55:02 -0700 Subject: search for a data in my browser via script In-Reply-To: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> References: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> Message-ID: On 02/06/2017 07:43 AM, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > For most things, you can use the "urllib" module in the standard library. https://docs.python.org/3/howto/urllib2.html urllib will let you post to a url, which you'll have to do to get a result from a search engine back. From Deborah_Swanson at f38.n261.z1 Mon Feb 6 12:51:50 2017 From: Deborah_Swanson at f38.n261.z1 (Deborah_Swanson at f38.n261.z1) Date: Mon, 06 Feb 2017 17:51:50 +0000 Subject: Coding issue with XML for word document Message-ID: <2424167298@f38.n261.z1.ftn> From: "Deborah Swanson" I don't see any Python in your code. The python.org list isn't generally familiar with Microsoft proprietary code, which is often quite different from other versions. Try asking this question in Microsoft's Office/Word Document forums. Good luck! accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM Subject: Coding issue with XML for word document > > > I am constructing a sentence to insert into a word xml > template. My code is below > > ------------------------ > #Create a list > > if len(myList) > 0: > if len(myList) > 1: > testText = list_format(myList) + " dealers." > else: > myText = myList[0] + " dealer." > > #Contruct sentence > > #myBlurb = "My > Favorite Cars - My favorite cars are > available at " + myText + "" > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > else: > myBlurb = "" > > > ------------------------------------------- > > If a list exists (not = 0) Then > > I am trying to get the following output: > > My Favorite Cars - My favorite cars are available at (list of > dealers) dealers. > > There is a newline above and below this line. The first three > words and the dash are bold. All other words are regular. > > This works > > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > > But when I try to bold, it bombs. It does not seem to like > either of these (+ myText + "") (separately or > together) appended on the end. > > Thia does not work > > myBlurb = "My Favorite > Cars - My favorite cars are available > at " + myText + " > > What am I missing here? > -- > https://mail.python.org/mailman/listinfo/python-list > From Deborah_Swanson at f38.n261.z1 Mon Feb 6 12:51:50 2017 From: Deborah_Swanson at f38.n261.z1 (Deborah_Swanson at f38.n261.z1) Date: Mon, 06 Feb 2017 17:51:50 +0000 Subject: Coding issue with XML for word document Message-ID: <3309402562@f38.n261.z1.ftn> From: Deborah_Swanson at f38.n261.z1 From: "Deborah Swanson" I don't see any Python in your code. The python.org list isn't generally familiar with Microsoft proprietary code, which is often quite different from other versions. Try asking this question in Microsoft's Office/Word Document forums. Good luck! accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM Subject: Coding issue with XML for word document > > > I am constructing a sentence to insert into a word xml > template. My code is below > > ------------------------ > #Create a list > > if len(myList) > 0: > if len(myList) > 1: > testText = list_format(myList) + " dealers." > else: > myText = myList[0] + " dealer." > > #Contruct sentence > > #myBlurb = "My > Favorite Cars - My favorite cars are > available at " + myText + "" > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > else: > myBlurb = "" > > > ------------------------------------------- > > If a list exists (not = 0) Then > > I am trying to get the following output: > > My Favorite Cars - My favorite cars are available at (list of > dealers) dealers. > > There is a newline above and below this line. The first three > words and the dash are bold. All other words are regular. > > This works > > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > > But when I try to bold, it bombs. It does not seem to like > either of these (+ myText + "") (separately or > together) appended on the end. > > Thia does not work > > myBlurb = "My Favorite > Cars - My favorite cars are available > at " + myText + " > > What am I missing here? > -- > https://mail.python.org/mailman/listinfo/python-list > From Deborah_Swanson at f38.n261.z1 Mon Feb 6 12:51:50 2017 From: Deborah_Swanson at f38.n261.z1 (Deborah_Swanson at f38.n261.z1) Date: Mon, 06 Feb 2017 17:51:50 +0000 Subject: Coding issue with XML for word document Message-ID: <147906467@f38.n261.z1.ftn> From: Deborah_Swanson at f38.n261.z1 From: Deborah_Swanson at f38.n261.z1 From: "Deborah Swanson" I don't see any Python in your code. The python.org list isn't generally familiar with Microsoft proprietary code, which is often quite different from other versions. Try asking this question in Microsoft's Office/Word Document forums. Good luck! accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM Subject: Coding issue with XML for word document > > > I am constructing a sentence to insert into a word xml > template. My code is below > > ------------------------ > #Create a list > > if len(myList) > 0: > if len(myList) > 1: > testText = list_format(myList) + " dealers." > else: > myText = myList[0] + " dealer." > > #Contruct sentence > > #myBlurb = "My > Favorite Cars - My favorite cars are > available at " + myText + "" > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > else: > myBlurb = "" > > > ------------------------------------------- > > If a list exists (not = 0) Then > > I am trying to get the following output: > > My Favorite Cars - My favorite cars are available at (list of > dealers) dealers. > > There is a newline above and below this line. The first three > words and the dash are bold. All other words are regular. > > This works > > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > > But when I try to bold, it bombs. It does not seem to like > either of these (+ myText + "") (separately or > together) appended on the end. > > Thia does not work > > myBlurb = "My Favorite > Cars - My favorite cars are available > at " + myText + " > > What am I missing here? > -- > https://mail.python.org/mailman/listinfo/python-list > From kelvidpang at f38.n261.z1 Mon Feb 6 13:28:08 2017 From: kelvidpang at f38.n261.z1 (kelvidpang at f38.n261.z1) Date: Mon, 06 Feb 2017 18:28:08 +0000 Subject: search for a data in my browser via script References: <3358149734@f38.n261.z1.ftn> Message-ID: <604344209@f38.n261.z1.ftn> From: kelvidpang at gmail.com On Monday, 6 February 2017 22:43:17 UTC+8, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > You may try to search for selenium for python, which has chrome driver.. it works like charm. From kelvidpang at f38.n261.z1 Mon Feb 6 13:28:08 2017 From: kelvidpang at f38.n261.z1 (kelvidpang at f38.n261.z1) Date: Mon, 06 Feb 2017 18:28:08 +0000 Subject: search for a data in my browser via script Message-ID: <2349529954@f38.n261.z1.ftn> From: kelvidpang at f38.n261.z1 From: kelvidpang at gmail.com On Monday, 6 February 2017 22:43:17 UTC+8, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > You may try to search for selenium for python, which has chrome driver.. it works like charm. From kelvidpang at f38.n261.z1 Mon Feb 6 13:28:08 2017 From: kelvidpang at f38.n261.z1 (kelvidpang at f38.n261.z1) Date: Mon, 06 Feb 2017 18:28:08 +0000 Subject: search for a data in my browser via script Message-ID: <660023256@f38.n261.z1.ftn> From: kelvidpang at f38.n261.z1 From: kelvidpang at f38.n261.z1 From: kelvidpang at gmail.com On Monday, 6 February 2017 22:43:17 UTC+8, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > You may try to search for selenium for python, which has chrome driver.. it works like charm. From gordon at panix.com Mon Feb 6 16:54:22 2017 From: gordon at panix.com (John Gordon) Date: Mon, 6 Feb 2017 21:54:22 +0000 (UTC) Subject: How to add months to a date (datetime object)? References: <3c0bf486-ab97-45de-8a20-92caf13bf03d@googlegroups.com> <000701d28052$41c97100$27b23dae@sambora> Message-ID: In "Deborah Swanson" writes: > bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > > > > for start of month to the beginning of next month > > > > from datetime import timedelta > > from dateutil.relativedelta import relativedelta > > > > end_date = start_date + relativedelta(months=delta_period) + > > timedelta(days=-delta_period) > Where do you define 'delta_period', and what is your question? There is no question; it is an answer in response to the original post asking how to add months to a datetime object. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From John_Gordon at f38.n261.z1 Mon Feb 6 16:54:22 2017 From: John_Gordon at f38.n261.z1 (John_Gordon at f38.n261.z1) Date: Mon, 06 Feb 2017 21:54:22 +0000 Subject: How to add months to a date (datetime object)? References: <2689836081@f38.n261.z1.ftn> Message-ID: <1907056859@f38.n261.z1.ftn> From: John Gordon In "Deborah Swanson" writes: > bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > > > > for start of month to the beginning of next month > > > > from datetime import timedelta > > from dateutil.relativedelta import relativedelta > > > > end_date = start_date + relativedelta(months=delta_period) + > > timedelta(days=-delta_period) > Where do you define 'delta_period', and what is your question? There is no question; it is an answer in response to the original post asking how to add months to a datetime object. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From John_Gordon at f38.n261.z1 Mon Feb 6 16:54:22 2017 From: John_Gordon at f38.n261.z1 (John_Gordon at f38.n261.z1) Date: Mon, 06 Feb 2017 21:54:22 +0000 Subject: How to add months to a date (datetime object)? Message-ID: <4048462826@f38.n261.z1.ftn> From: John_Gordon at f38.n261.z1 From: John Gordon In "Deborah Swanson" writes: > bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > > > > for start of month to the beginning of next month > > > > from datetime import timedelta > > from dateutil.relativedelta import relativedelta > > > > end_date = start_date + relativedelta(months=delta_period) + > > timedelta(days=-delta_period) > Where do you define 'delta_period', and what is your question? There is no question; it is an answer in response to the original post asking how to add months to a datetime object. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From John_Gordon at f38.n261.z1 Mon Feb 6 16:54:22 2017 From: John_Gordon at f38.n261.z1 (John_Gordon at f38.n261.z1) Date: Mon, 06 Feb 2017 21:54:22 +0000 Subject: How to add months to a date (datetime object)? Message-ID: <1052615262@f38.n261.z1.ftn> From: John_Gordon at f38.n261.z1 From: John_Gordon at f38.n261.z1 From: John Gordon In "Deborah Swanson" writes: > bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > > > > for start of month to the beginning of next month > > > > from datetime import timedelta > > from dateutil.relativedelta import relativedelta > > > > end_date = start_date + relativedelta(months=delta_period) + > > timedelta(days=-delta_period) > Where do you define 'delta_period', and what is your question? There is no question; it is an answer in response to the original post asking how to add months to a datetime object. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From python at mrabarnett.plus.com Mon Feb 6 17:17:13 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 6 Feb 2017 22:17:13 +0000 Subject: How to add months to a date (datetime object)? In-Reply-To: References: <3c0bf486-ab97-45de-8a20-92caf13bf03d@googlegroups.com> <000701d28052$41c97100$27b23dae@sambora> Message-ID: On 2017-02-06 21:54, John Gordon wrote: > In "Deborah Swanson" writes: > >> bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM >> > >> > for start of month to the beginning of next month >> > >> > from datetime import timedelta >> > from dateutil.relativedelta import relativedelta >> > >> > end_date = start_date + relativedelta(months=delta_period) + >> > timedelta(days=-delta_period) > >> Where do you define 'delta_period', and what is your question? > > There is no question; it is an answer in response to the original > post asking how to add months to a datetime object. > As far as I can find, the posts were from March 2009! From accessnewbie at gmail.com Mon Feb 6 20:00:25 2017 From: accessnewbie at gmail.com (accessnewbie at gmail.com) Date: Mon, 6 Feb 2017 17:00:25 -0800 (PST) Subject: Coding issue with XML for word document Message-ID: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> I am constructing a sentence to insert into a word xml template. My code is below ------------------------ #Create a list if len(myList) > 0: if len(myList) > 1: testText = list_format(myList) + " dealers." else: myText = myList[0] + " dealer." #Contruct sentence #myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" else: myBlurb = "" ------------------------------------------- If a list exists (not = 0) Then I am trying to get the following output: My Favorite Cars - My favorite cars are available at (list of dealers) dealers. There is a newline above and below this line. The first three words and the dash are bold. All other words are regular. This works myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" But when I try to bold, it bombs. It does not seem to like either of these (+ myText + "") (separately or together) appended on the end. Thia does not work myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + " What am I missing here? From python at deborahswanson.net Mon Feb 6 20:42:37 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 6 Feb 2017 17:42:37 -0800 Subject: How to add months to a date (datetime object)? In-Reply-To: Message-ID: <00af01d280e3$76abae60$27b23dae@sambora> MRAB wrote,on February 06, 2017 2:17 PM > > On 2017-02-06 21:54, John Gordon wrote: > > In "Deborah > > Swanson" writes: > > > >> bajimicbiga at gmail.com wrote, on February 02, 2017 2:44 AM > >> > > >> > for start of month to the beginning of next month > >> > > >> > from datetime import timedelta > >> > from dateutil.relativedelta import relativedelta > >> > > >> > end_date = start_date + relativedelta(months=delta_period) + > >> > timedelta(days=-delta_period) > > > >> Where do you define 'delta_period', and what is your question? > > > > There is no question; it is an answer in response to the > original post > > asking how to add months to a datetime object. > > > As far as I can find, the posts were from March 2009! So that's why I couldn't find any previous posts with that title. And it's hard to tell, sometimes newcomers fumble around and their first successful post comes in with a Re: in front of the title. From python at deborahswanson.net Mon Feb 6 20:51:51 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 6 Feb 2017 17:51:51 -0800 Subject: Coding issue with XML for word document In-Reply-To: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> Message-ID: <00b001d280e4$c102ca10$27b23dae@sambora> I don't see any Python in your code. The python.org list isn't generally familiar with Microsoft proprietary code, which is often quite different from other versions. Try asking this question in Microsoft's Office/Word Document forums. Good luck! accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM Subject: Coding issue with XML for word document > > > I am constructing a sentence to insert into a word xml > template. My code is below > > ------------------------ > #Create a list > > if len(myList) > 0: > if len(myList) > 1: > testText = list_format(myList) + " dealers." > else: > myText = myList[0] + " dealer." > > #Contruct sentence > > #myBlurb = "My > Favorite Cars - My favorite cars are > available at " + myText + "" > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > else: > myBlurb = "" > > > ------------------------------------------- > > If a list exists (not = 0) Then > > I am trying to get the following output: > > My Favorite Cars - My favorite cars are available at (list of > dealers) dealers. > > There is a newline above and below this line. The first three > words and the dash are bold. All other words are regular. > > This works > > myBlurb = "My Favorite Cars - My favorite cars are > available at " + myText + "" > > But when I try to bold, it bombs. It does not seem to like > either of these (+ myText + "") (separately or > together) appended on the end. > > Thia does not work > > myBlurb = "My Favorite > Cars - My favorite cars are available > at " + myText + " > > What am I missing here? > -- > https://mail.python.org/mailman/listinfo/python-list > From kelvidpang at gmail.com Mon Feb 6 21:19:44 2017 From: kelvidpang at gmail.com (kelvidpang at gmail.com) Date: Mon, 6 Feb 2017 18:19:44 -0800 (PST) Subject: Python installer hangs in Windows 7 In-Reply-To: References: <765703485.1601081.1486357381135.ref@mail.yahoo.com> <765703485.1601081.1486357381135@mail.yahoo.com> Message-ID: On Monday, 6 February 2017 13:16:24 UTC+8, Jean-Claude Roy wrote: > ? I am trying to install Python 3.6.0 on a Windows 7 computer. > The download of 29.1 MB is successful and I get the nextwindow.? I?choose the "install now" selection and thatopens the Setup Program window. > Now the trouble starts:I get "Installing:" and the Initialization progress...and nothing else. > There is no additional disk activity, no progress on initialization, andeverything appears dead.? Even after 20 minutes there is zero progress. > I've repeated this as both a user and the administrator of this Windowscomputer.? I get the same results in either case. > If I go to the task manager it shows that Python 3.6.0 (32-bit) setup is running.? If I try to end the task Iget the message that the program is not responding. > Do you have any suggestions as to how I can get past this? > Thank you. On Monday, 6 February 2017 13:16:24 UTC+8, Jean-Claude Roy wrote: > ? I am trying to install Python 3.6.0 on a Windows 7 computer. > The download of 29.1 MB is successful and I get the nextwindow.? I?choose the "install now" selection and thatopens the Setup Program window. > Now the trouble starts:I get "Installing:" and the Initialization progress...and nothing else. > There is no additional disk activity, no progress on initialization, andeverything appears dead.? Even after 20 minutes there is zero progress. > I've repeated this as both a user and the administrator of this Windowscomputer.? I get the same results in either case. > If I go to the task manager it shows that Python 3.6.0 (32-bit) setup is running.? If I try to end the task Iget the message that the program is not responding. > Do you have any suggestions as to how I can get past this? > Thank you. On Monday, 6 February 2017 13:16:24 UTC+8, Jean-Claude Roy wrote: > ? I am trying to install Python 3.6.0 on a Windows 7 computer. > The download of 29.1 MB is successful and I get the nextwindow.? I?choose the "install now" selection and thatopens the Setup Program window. > Now the trouble starts:I get "Installing:" and the Initialization progress...and nothing else. > There is no additional disk activity, no progress on initialization, andeverything appears dead.? Even after 20 minutes there is zero progress. > I've repeated this as both a user and the administrator of this Windowscomputer.? I get the same results in either case. > If I go to the task manager it shows that Python 3.6.0 (32-bit) setup is running.? If I try to end the task Iget the message that the program is not responding. > Do you have any suggestions as to how I can get past this? > Thank you. alternatively try install python 2.x to try it out. From kelvidpang at gmail.com Mon Feb 6 21:28:08 2017 From: kelvidpang at gmail.com (kelvidpang at gmail.com) Date: Mon, 6 Feb 2017 18:28:08 -0800 (PST) Subject: search for a data in my browser via script In-Reply-To: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> References: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> Message-ID: On Monday, 6 February 2017 22:43:17 UTC+8, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > You may try to search for selenium for python, which has chrome driver.. it works like charm. From kelvidpang at gmail.com Tue Feb 7 01:22:04 2017 From: kelvidpang at gmail.com (Kelvid Pang) Date: Mon, 6 Feb 2017 22:22:04 -0800 (PST) Subject: gmail api Message-ID: <32a3c9f3-55ff-4aa2-a33e-84b989046ff6@googlegroups.com> hi, I am trying to gmail api with reference to this URL: https://developers.google.com/gmail/api/quickstart/python But I couldn't find the 'gmail-python-quickstart.json' file. Any one can help? thanks. From steve at pearwood.info Tue Feb 7 02:53:41 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 07 Feb 2017 07:53:41 GMT Subject: Coding issue with XML for word document References: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> Message-ID: <58997d05$0$1589$c3e8da3$5496439d@news.astraweb.com> On Mon, 06 Feb 2017 17:00:25 -0800, accessnewbie wrote: > I am constructing a sentence to insert into a word xml template. My code > is below > > #Create a list > if len(myList) > 0: > if len(myList) > 1: What is myList? Where does it come from? > testText = list_format(myList) + " dealers." What does list_format() do? > else: > myText = myList[0] + " dealer." Why does one branch set the variable "testText" and the other branch set a different variable "myText"? > #Contruct sentence > > #myBlurb = "My Favorite Cars - > My favorite cars are available at > " + myText + "" > > myBlurb = "My Favorite Cars - My favorite cars are available > at " + myText + "" > else: > myBlurb = "" [...] > This works > > myBlurb = "My Favorite Cars - My favorite cars are available at " > + myText + "" As far as I can see, that is exactly the same as the line of code above. > But when I try to bold, it bombs. It does not seem to like either of > these (+ myText + "") (separately or together) appended on the > end. How are you trying to bold? If you're programming in a word processor, and you apply bold to your code, that won't work. What do you mean, "bombs"? Do you literally mean you are running a classic Macintosh from the 1990s and you are getting a fatal System Error in a "Bomb" dialog box? Otherwise, please be more precise: do you get a Blue Screen of Death? A kernel panic? Does the Python interpreter segfault? Or do you get an ordinary Python traceback? If it is an ordinary Python traceback, don't keep it a secret -- tell us what it says. COPY AND PASTE the traceback, in full, not just the last line. > Thia does not work > > myBlurb = "My Favorite Cars - > My favorite cars are available at " > + myText + " You are missing a close quote at the very end: #WRONG " #RIGHT "" -- Steve From hmmeeranrizvi18 at gmail.com Tue Feb 7 03:10:55 2017 From: hmmeeranrizvi18 at gmail.com (Meeran Rizvi) Date: Tue, 7 Feb 2017 00:10:55 -0800 (PST) Subject: search for a data in my browser via script In-Reply-To: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> References: <1f0667ab-d152-44ff-974b-1fce405ee54c@googlegroups.com> Message-ID: <6f78d6da-20ef-4a77-80ed-f873d85c0119@googlegroups.com> On Monday, February 6, 2017 at 8:13:17 PM UTC+5:30, Meeran Rizvi wrote: > Hello guys, > Here i am writing a script which will open my chrome browser and opens the URL www.google.com. > But how to search for a data via script. > for example i need to search for 'Rose' in google.com via script. > how to do that? > > > import webbrowser > url="www.google.com" > chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' > webbrowser.get(chrome_path) > webbrowser.open(url) > This works as easy,when you search for a data in google.com via chrome import webbrowser search_word = 'Rose' url = 'https://www.google.com/#q={}'.format(search_word) chrome_path = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe %s' webbrowser.get(chrome_path) webbrowser.open(url) From dieter at handshake.de Tue Feb 7 04:14:06 2017 From: dieter at handshake.de (dieter) Date: Tue, 07 Feb 2017 10:14:06 +0100 Subject: Strategies when hunting for dictionary corruption References: Message-ID: <87lgtifjmp.fsf@handshake.de> Skip Montanaro writes: > I'm wrapping some C++ libraries using pybind11. On the pybind11 side of > things, I've written a simple type converter between Python's datetime.date > objects and our internal C++ date objects. The first thing the type > converter needs to do is to insure that the datetime C API is initialized: > > if (!PyDateTimeAPI) { PyDateTime_IMPORT; } > > This is failing because a dictionary internal to the import mechanism is > being corrupted, the extensions dict in import.c. I highly doubt this is a > Python issue. It's much more likely to be a bug in our C++ libraries or in > pybind11. > > The problem is, dictionaries being rather dynamic objects (growing and > sometimes shrinking), the actual data location which might be corrupted > isn't fixed, but can move around. I've got a Python interpreter configured > using --with-pydebug, it's compiled with -ggdb -O0, and I'm ready to watch > some memory locations, but it's not clear what exactly I should watch or > how/when to move my watchpoint(s) around. Should I not be thinking in terms > of watchpoints? Is there a better way to approach this problem? (I also > have valgrind at my disposal, but am not very skilled in its use.) I would use a form of binary search approach -- unless there is an obvious cause, such as GIL not hold. For the "binary search approach", you would need a non destructive way to detect the corruption during a "gdb" debugging session. There is a set of "gdb" macros to manage (e.g. print out) Python objects from the debugger. Based on them, you might develop similar macros to access dicts and check for their validity. Once you have a way to detect the corruption, you can step into the "PyDateTime_IMPORT" and find out where the corruption happens. From Deborah_Swanson at f38.n261.z1 Tue Feb 7 06:47:42 2017 From: Deborah_Swanson at f38.n261.z1 (Deborah_Swanson at f38.n261.z1) Date: Tue, 07 Feb 2017 11:47:42 +0000 Subject: Coding issue with XML for word document Message-ID: <3105052823@f38.n261.z1.ftn> From: "Deborah Swanson" Cute, whoever you are. Very cute. (I know how to sign up for fake email addresses too.) > -----Original Message----- > From: Python-list > [mailto:python-list-bounces+python=deborahswanson.net at python.o > rg] On Behalf Of Deborah_Swanson at f38.n261.z1 > Sent: Monday, February 06, 2017 9:52 AM > To: python-list at python.org > Subject: Re: Coding issue with XML for word document > > > From: "Deborah Swanson" > > I don't see any Python in your code. The python.org list > isn't generally familiar with Microsoft proprietary code, > which is often quite different from other versions. Try > asking this question in Microsoft's Office/Word Document > forums. Good luck! > > > accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM > Subject: Coding issue with XML for word document > > > > > > I am constructing a sentence to insert into a word xml template. My > > code is below > > > > ------------------------ > > #Create a list > > > > if len(myList) > 0: > > if len(myList) > 1: > > testText = list_format(myList) + " dealers." > > else: > > myText = myList[0] + " dealer." > > > > #Contruct sentence > > > > #myBlurb = "My > > Favorite Cars - My favorite cars are > available > > at " + myText + "" > > myBlurb = "My Favorite Cars - My favorite cars are > > available at " + myText + "" > > else: > > myBlurb = "" > > > > > > ------------------------------------------- > > > > If a list exists (not = 0) Then > > > > I am trying to get the following output: > > > > My Favorite Cars - My favorite cars are available at (list of > > dealers) dealers. > > > > There is a newline above and below this line. The first three words > > and the dash are bold. All other words are regular. > > > > This works > > > > myBlurb = "My Favorite Cars - My favorite cars are > available at > > " + myText + "" > > > > But when I try to bold, it bombs. It does not seem to like > either of > > these (+ myText + "") (separately or > > together) appended on the end. > > > > Thia does not work > > > > myBlurb = "My Favorite Cars - > > My favorite cars are available at > " > > + myText + " > > > > What am I missing here? > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From python at lucidity.plus.com Tue Feb 7 08:50:54 2017 From: python at lucidity.plus.com (Erik) Date: Tue, 7 Feb 2017 13:50:54 +0000 Subject: Strategies when hunting for dictionary corruption In-Reply-To: References: Message-ID: <47de83e1-a13a-1599-fb9a-db451e4c1f6e@lucidity.plus.com> On 06/02/17 12:56, Skip Montanaro wrote: > Is there a better way to approach this problem? (I also > have valgrind at my disposal, but am not very skilled in its use.) valgrind is what I was going to suggest. You have to compile Python with extra flags to get it to integrate well with valgrind though, because of its custom allocators (if you're just trying to run it now and are confused by the results, that might be why). valgrind (or rather, valgrind's "memcheck" module) has command line flags for giving deeper information than the default, but at the expense of tracking more and slowing things down. E. From gang.yang.ctr at mail.mil Tue Feb 7 12:41:13 2017 From: gang.yang.ctr at mail.mil (Yang, Gang CTR (US)) Date: Tue, 7 Feb 2017 17:41:13 +0000 Subject: How to configure trusted CA certificates for SSL client? Message-ID: <82AE9B2456FC17429CDD874AE33B89608BF67192@UCOLHPUC.easf.csd.disa.mil> Hi, I'm using Python 3.X (3.5 on Windows 2008 and 3.4 on CentOS 6.7) and encountered an SSL client side CA certificates issue. The issue came up when a third-party package (django-cas-ng) tried to verify the CAS service ticket (ST) by calling CAS server using requests.get(...) and failed with CERTIFICATE_VERIFY_FAILED error. The CAS server is accessed by HTTPS with a self-signed server certificate. Following some suggestions on the internet, I've tried to modify django-cas-ng's code to call requests.get(..) with verify parameter, such as requests.get(..., verify=False) and requests.get(..., verify="CAS server cert"). Both workarounds worked, but I can't change third-party package code. I also tried to add the CAS server cert to the underlying OS (Windows 2008 and CentOS 6.7), but it did not help. My question is where does SSL client code get the trusted CA certificates from, from Python or the underlying OS? What configuration do I need in order for the SSL client to conduct the SSL handshake successfully? Appreciate any help! Gang Gang Yang Shonborn-Becker Systems Inc. (SBSI) Contractor Engineering Supporting SEC Office: 732-982-8561, x427 Cell: 732-788-7501 Email: gang.yang.ctr at mail.mil From gangyang7 at gmail.com Tue Feb 7 12:43:03 2017 From: gangyang7 at gmail.com (gangyang7 at gmail.com) Date: Tue, 7 Feb 2017 09:43:03 -0800 (PST) Subject: How to configure SSL client trusted CA certificates? Message-ID: Hi, I'm using Python 3.X (3.5 on Windows 2008 and 3.4 on CentOS 6.7) and encountered an SSL client side CA certificates issue. The issue came up when a third-party package (django-cas-ng) tried to verify the CAS service ticket (ST) by calling CAS server using requests.get(...) and failed with CERTIFICATE_VERIFY_FAILED error. The CAS server is accessed by HTTPS with a self-signed server certificate. Following some suggestions on the internet, I've tried to modify django-cas-ng's code to call requests.get(..) with verify parameter, such as requests.get(..., verify=False) and requests.get(..., verify="CAS server cert"). Both workarounds worked, but I can't change third-party package code. I also tried to add the CAS server cert to the underlying OS (Windows 2008 and CentOS 6.7), but it did not help. My question is where does SSL client code get the trusted CA certificates from, from Python or the underlying OS? What configuration do I need in order for the SSL client to conduct the SSL handshake successfully? Appreciate any help! Gang From ethan at stoneleaf.us Tue Feb 7 13:35:12 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 07 Feb 2017 10:35:12 -0800 Subject: Strategies when hunting for dictionary corruption In-Reply-To: References: Message-ID: <589A1360.5070305@stoneleaf.us> On 02/06/2017 04:56 AM, Skip Montanaro wrote: > I'm wrapping some C++ libraries using pybind11. I asked one of the core-devs about it, and he suggested: > Python 3.6 got a new C define in Objects/dictobjet.c > which might help to track the bug: > > /* Uncomment to check the dict content in _PyDict_CheckConsistency() */ > /* #define DEBUG_PYDICT */ > > Uncomment and recompile Python (in debug mode). > > Victor -- ~Ethan~ From python at deborahswanson.net Tue Feb 7 14:47:42 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 7 Feb 2017 11:47:42 -0800 Subject: Coding issue with XML for word document In-Reply-To: <2424167298@f38.n261.z1.ftn> Message-ID: <002e01d2817b$0c18fb50$27b23dae@sambora> Cute, whoever you are. Very cute. (I know how to sign up for fake email addresses too.) > -----Original Message----- > From: Python-list > [mailto:python-list-bounces+python=deborahswanson.net at python.o > rg] On Behalf Of Deborah_Swanson at f38.n261.z1 > Sent: Monday, February 06, 2017 9:52 AM > To: python-list at python.org > Subject: Re: Coding issue with XML for word document > > > From: "Deborah Swanson" > > I don't see any Python in your code. The python.org list > isn't generally familiar with Microsoft proprietary code, > which is often quite different from other versions. Try > asking this question in Microsoft's Office/Word Document > forums. Good luck! > > > accessnewbie at gmail.com wrote, on February 06, 2017 5:00 PM > Subject: Coding issue with XML for word document > > > > > > I am constructing a sentence to insert into a word xml template. My > > code is below > > > > ------------------------ > > #Create a list > > > > if len(myList) > 0: > > if len(myList) > 1: > > testText = list_format(myList) + " dealers." > > else: > > myText = myList[0] + " dealer." > > > > #Contruct sentence > > > > #myBlurb = "My > > Favorite Cars - My favorite cars are > available > > at " + myText + "" > > myBlurb = "My Favorite Cars - My favorite cars are > > available at " + myText + "" > > else: > > myBlurb = "" > > > > > > ------------------------------------------- > > > > If a list exists (not = 0) Then > > > > I am trying to get the following output: > > > > My Favorite Cars - My favorite cars are available at (list of > > dealers) dealers. > > > > There is a newline above and below this line. The first three words > > and the dash are bold. All other words are regular. > > > > This works > > > > myBlurb = "My Favorite Cars - My favorite cars are > available at > > " + myText + "" > > > > But when I try to bold, it bombs. It does not seem to like > either of > > these (+ myText + "") (separately or > > together) appended on the end. > > > > Thia does not work > > > > myBlurb = "My Favorite Cars - > > My favorite cars are available at > " > > + myText + " > > > > What am I missing here? > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From torriem at gmail.com Tue Feb 7 15:08:49 2017 From: torriem at gmail.com (Michael Torrie) Date: Tue, 7 Feb 2017 13:08:49 -0700 Subject: What's with all the messages from @f38.n261.z1 Message-ID: Seems like we're getting a bunch of messages on the mailing list that appear to be copies of real member posts that are saying they are from @f38.n261.z1? They don't appear to be deliberate impersonations. Some misconfigured server reflecting messages back to the list perhaps? From skip.montanaro at gmail.com Tue Feb 7 15:24:56 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 7 Feb 2017 14:24:56 -0600 Subject: Strategies when hunting for dictionary corruption In-Reply-To: <589A1360.5070305@stoneleaf.us> References: <589A1360.5070305@stoneleaf.us> Message-ID: Thanks all. I am stuck on 2.7 for the foreseeable future, so the DEBUG_PYDICT won't help. I think we're set though. for the moment, we seem to have moved past that bug (might have just been ordering in the pybind11 wrapper). Skip On Tue, Feb 7, 2017 at 12:35 PM, Ethan Furman wrote: > On 02/06/2017 04:56 AM, Skip Montanaro wrote: > >> I'm wrapping some C++ libraries using pybind11. > > > I asked one of the core-devs about it, and he suggested: > >> Python 3.6 got a new C define in Objects/dictobjet.c >> which might help to track the bug: >> >> /* Uncomment to check the dict content in _PyDict_CheckConsistency() */ >> /* #define DEBUG_PYDICT */ >> >> Uncomment and recompile Python (in debug mode). >> >> Victor > > > -- > ~Ethan~ > -- > https://mail.python.org/mailman/listinfo/python-list From random832 at fastmail.com Tue Feb 7 15:27:57 2017 From: random832 at fastmail.com (Random832) Date: Tue, 07 Feb 2017 15:27:57 -0500 Subject: What's with all the messages from @f38.n261.z1 In-Reply-To: References: Message-ID: <1486499277.231598.873559968.0994964D@webmail.messagingengine.com> On Tue, Feb 7, 2017, at 15:08, Michael Torrie wrote: > Seems like we're getting a bunch of messages on the mailing list that > appear to be copies of real member posts that are saying they are from > @f38.n261.z1? They don't appear to be deliberate impersonations. Some > misconfigured server reflecting messages back to the list perhaps? Looking at it (compared to the original), It has Usenet headers for comp.lang.python (and a path header indicating an origin at eternal-september.org), and headers associated with FidoNet and something called "Prism BBS". From pavol.lisy at gmail.com Tue Feb 7 16:22:39 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Tue, 7 Feb 2017 22:22:39 +0100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> <5897dd2a$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/6/17, Chris Angelico wrote: > On Mon, Feb 6, 2017 at 1:19 PM, Steve D'Aprano > wrote: [...] >> How about graphic and video designers? Just how well does hg cope with >> gigabytes of video data? > > I've no idea, but I know that git can handle large amounts of data. > (There are a couple of extensions that make it easier.) But there could be problem with "keeping eye" part of your premise. ("If you commit everything to git and keep an eye on your diffs before you push, the encryption would have to be _extremely_ sneaky"). From rosuav at gmail.com Tue Feb 7 16:33:46 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 8 Feb 2017 08:33:46 +1100 Subject: best way to ensure './' is at beginning of sys.path? In-Reply-To: References: <20170203222542.GA63533@cskk.homeip.net> <5qKdnYfSdphnvgvFnZ2dnUU7-b-dnZ2d@giganews.com> <8eSdnWYGT5NcRgvFnZ2dnUU7-IOdnZ2d@giganews.com> <5897a228$0$1601$c3e8da3$5496439d@news.astraweb.com> <5897dd2a$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 8, 2017 at 8:22 AM, Pavol Lisy wrote: >>> How about graphic and video designers? Just how well does hg cope with >>> gigabytes of video data? >> >> I've no idea, but I know that git can handle large amounts of data. >> (There are a couple of extensions that make it easier.) > > But there could be problem with "keeping eye" part of your premise. > ("If you commit everything to git and keep an eye on your diffs before > you push, the encryption would have to be _extremely_ sneaky"). True, which is why as much as possible should be done with raw video data that never changes, and then text files that indicate the alterations. The raw video data gets recorded once and then kept as-is, and then your editing work gets saved into some textual form (maybe a JSON file, maybe a series of commands) that you can plausibly diff. The only time you actually add binary files is when you do initial recording work, and a diff that affects those files should always be suspicious. (Obviously it would be horrendously inefficient to work from the raw files all the time. But the edited files could be gitignored, since they can be reconstructed from the originals; share them for convenience, but don't sweat their loss.) ChrisA From pavol.lisy at gmail.com Tue Feb 7 17:04:32 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Tue, 7 Feb 2017 23:04:32 +0100 Subject: Coding issue with XML for word document In-Reply-To: <58997d05$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> <58997d05$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/7/17, Steven D'Aprano wrote: > On Mon, 06 Feb 2017 17:00:25 -0800, accessnewbie wrote: [...] >> But when I try to bold, it bombs. It does not seem to like either of >> these (+ myText + "") (separately or together) appended on the >> end. > > How are you trying to bold? If you're programming in a word processor, > and you apply bold to your code, that won't work. > > What do you mean, "bombs"? Do you literally mean you are running a > classic Macintosh from the 1990s and you are getting a fatal System Error > in a "Bomb" dialog box? > > Otherwise, please be more precise: do you get a Blue Screen of Death? A > kernel panic? Does the Python interpreter segfault? Or do you get an > ordinary Python traceback? > > If it is an ordinary Python traceback, don't keep it a secret -- tell us > what it says. COPY AND PASTE the traceback, in full, not just the last > line. I am afraid he means that it create mishmash in word document... Dear accessnewbie I propose divide problem to smaller subproblems. For example make python code as simple as possible (avoid python part if you could! As Deborah wrote it is very probably not problem in python) And try to explicitly try to write example text where is bold part. #myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" myBlurb = "My Favorite Cars - My favorite cars are available at my garage " if it is still wrong try to make more and more similar text to: myBlurb = "My Favorite Cars - My favorite cars are available at my garage" I mean for example remove "" part etc. Second - This is probably not your problem (unless you have special char in your example myList) but if you could have xml spec chars in your input then python could help. See example -> >>> from xml.sax.saxutils import escape >>> escape("< & >") '< & >' myBlurb = "My Favorite Cars - My favorite cars are available at " + escape(myText) + "" # this is safer than your code! I hope you understand why it is useful. :) From Pavol_Lisy at f38.n261.z1 Tue Feb 7 18:04:32 2017 From: Pavol_Lisy at f38.n261.z1 (Pavol_Lisy at f38.n261.z1) Date: Tue, 07 Feb 2017 23:04:32 +0000 Subject: Coding issue with XML for word document Message-ID: <213741425@f38.n261.z1.ftn> From: Pavol Lisy On 2/7/17, Steven D'Aprano wrote: > On Mon, 06 Feb 2017 17:00:25 -0800, accessnewbie wrote: [...] >> But when I try to bold, it bombs. It does not seem to like either of >> these (+ myText + "") (separately or together) appended on the >> end. > > How are you trying to bold? If you're programming in a word processor, > and you apply bold to your code, that won't work. > > What do you mean, "bombs"? Do you literally mean you are running a > classic Macintosh from the 1990s and you are getting a fatal System Error > in a "Bomb" dialog box? > > Otherwise, please be more precise: do you get a Blue Screen of Death? A > kernel panic? Does the Python interpreter segfault? Or do you get an > ordinary Python traceback? > > If it is an ordinary Python traceback, don't keep it a secret -- tell us > what it says. COPY AND PASTE the traceback, in full, not just the last > line. I am afraid he means that it create mishmash in word document... Dear accessnewbie I propose divide problem to smaller subproblems. For example make python code as simple as possible (avoid python part if you could! As Deborah wrote it is very probably not problem in python) And try to explicitly try to write example text where is bold part. #myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + "" myBlurb = "My Favorite Cars - My favorite cars are available at my garage " if it is still wrong try to make more and more similar text to: myBlurb = "My Favorite Cars - My favorite cars are available at my garage" I mean for example remove "" part etc. Second - This is probably not your problem (unless you have special char in your example myList) but if you could have xml spec chars in your input then python could help. See example -> >>> from xml.sax.saxutils import escape >>> escape("< & >") '< & >' myBlurb = "My Favorite Cars - My favorite cars are available at " + escape(myText) + "" # this is safer than your code! I hope you understand why it is useful. :) From michael.vilain at gmail.com Wed Feb 8 01:30:45 2017 From: michael.vilain at gmail.com (MeV) Date: Tue, 7 Feb 2017 22:30:45 -0800 (PST) Subject: How to configure trusted CA certificates for SSL client? In-Reply-To: References: <82AE9B2456FC17429CDD874AE33B89608BF67192@UCOLHPUC.easf.csd.disa.mil> Message-ID: On Tuesday, February 7, 2017 at 9:42:54 AM UTC-8, Yang, Gang CTR (US) wrote: > My question is where does SSL client code get the trusted CA certificates from, from Python or the underlying OS? What configuration do I need in order for the SSL client to conduct the SSL handshake successfully? > When I setup TLS for a K/V pair database with self-signed certs, I could supply two types of certs -peer certs with the IP and DNS name tied to the node signed by the self-signed CA I generated -client cert that had the IP and DNS for ALL the machines in the cluster signed by the CA To connect to the any of the clients, I had to provide the client cert, key, and the CA. That's just the way the software built. For browsers, each browser has a CA from various authorities and you add an intermediate key signed by one of those authorities (e.g. Verisign which costs $2000) to the browser to allow you to access a specific domain through a wildcard cert. My K/V software didn't support that. Look at how CentOS 6 stores it's certs in /etc/pki. I think where you'd put your CA + client key bundle. Which means you have to do this for all machines that run your code. That's not optimal or at all portable. You'll probably have to deliver the app as a VM or a docker container to put it all together. Good luck. From stephane at wirtel.be Wed Feb 8 03:50:35 2017 From: stephane at wirtel.be (Stephane Wirtel) Date: Wed, 08 Feb 2017 09:50:35 +0100 Subject: Python Events in 2017, Need your help. In-Reply-To: <20170109095449.v4ual5czl5mc7xxs@sg1> References: <20170109095449.v4ual5czl5mc7xxs@sg1> Message-ID: <7B2FBD81-C43C-46C2-B5E5-B2F2BFC400A5@wirtel.be> Thank you, with your help, we have added events on the t-shirt. Now, Could you just add them on python.org/events ? Have a nice day, Stephane On 9 Jan 2017, at 10:54, Stephane Wirtel via Python-list wrote: > Dear Community, > > For the PythonFOSDEM [1] on 4th and 5th February in Belgium, I would > like to present some slides with the Python events around the World. > Based on https://python.org/events, I have noted that there are > missing events, for example: > > * PyCon Otto: Italy > * PyCon UK: United Kingdom > * PyCon CA: Canada > * PyCon Ireland: Ireland > * PyCon France: France > > Some of these events are not yet announced and I understand they are > in the second semester, and thus, they don't know the location and the > dates, excepted for PyCon Otto (April). > > In fact, I have noted that we know some big events in the Python > community (for example: PyCon US and EuroPython) but do you know the > others events, maybe the local event, PyCon IE, PyCon UK or PyCon IT. > > I like to know where there is a PyCon or a Django Conf or a PyData > Event. > > In fact, I think we can help the Python Community if we submit all the > events in https://python.org/events. > > This page has been created by the PSF and is maintained by some > volunteers. > > I know this list of events: > * PyCon Cameroon : 20-23 Jav, Cameroon > * PythonFOSDEM : 4-5 Feb, Belgium > * PyCon Colombia : 10-12 Feb, Colombia > * PyCon Pune : 16-20 Feb, India > * Swiss Python Summit : 17-18 Feb, Switzerland > * IrPyCon : 17-18 Feb, Iran > * PyCon SK : 10-13 Mar, Slovakia > * Django Europe : 3-8 Apr, Italy > * PyCon Otto : 6-9 Apr, Italy > * Python Sudeste : 5-7 Mai, Brazil > * GeoPython : 8-11 May, Switzerland > * PyCon US : 17-26 May, USA > * EuroPython : July, Italy > * PyCon AU : 3-9 Aug, Australia > * PyCon UK : September, United Kingdom > * PyCon CA : November, Canada > * PyCon Ireland : October, Ireland > * PyCon FR : October/November, France > > And you ? > Please, could you check on https://www.python.org/events/ , if you are > an organizer, please add your event. > > If you think there is a missing event, please, send me the info via > [email](mailto:stephane at wirtel.be) or via my [twitter > account](https://twitter.com/matrixise) and I will add it on my > slides. > > I would like to present your event. > > Thank you so much for your help. > > Stephane Wirtel > > [1] https://www.python-fosdem.org > > -- > St?phane Wirtel - http://wirtel.be - @matrixise > -- > https://mail.python.org/mailman/listinfo/python-list From Michael_Torrie at f38.n261.z1 Wed Feb 8 03:55:12 2017 From: Michael_Torrie at f38.n261.z1 (Michael_Torrie at f38.n261.z1) Date: Wed, 08 Feb 2017 08:55:12 +0000 Subject: How to store properties Message-ID: <4035312961@f38.n261.z1.ftn> From: Michael Torrie On 02/08/2017 04:26 AM, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. > Would a JSON file be a good idea? I've used ConfigParser before. I think the relative simpleness of .INI files. If I was expecting end users to manipulate a config file, I would use ConfigParser. But 90% of the time, I just use another python module. For my purposes, there is no security risk to it. Due to its nature, Django also uses python modules to define configuration such as URL maps, etc. When I'm using GTK+, I would use the gsettings facilities of GTK+ to store and retrieve config variables. And with Qt, I'd use QSettings. I think in both cases they use backends appropriate for the operating system. On Windows, that would be registry settings in the user hive. From Cecil at decebal.nl Wed Feb 8 06:26:57 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Wed, 08 Feb 2017 12:26:57 +0100 Subject: How to store properties Message-ID: <87efz83ou6.fsf@Equus.decebal.nl> In Java you (can) use a properties file store configuration. What is the best way to do something like that in Python? I saw ConfigParser, but have the feeling that it is not really used. Would a JSON file be a good idea? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From peter.heitzer at rz.uni-regensburg.de Wed Feb 8 07:11:55 2017 From: peter.heitzer at rz.uni-regensburg.de (Peter Heitzer) Date: 8 Feb 2017 12:11:55 GMT Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: Cecil Westerhof wrote: >In Java you (can) use a properties file store configuration. What is >the best way to do something like that in Python? >I saw ConfigParser, but have the feeling that it is not really used. >Would a JSON file be a good idea? If you only want to read the configuration, just use an ordinary file you import. For example config.py contains the lines: username=myuser server=myserver password=secret In your script: import config Now you can referenc all the variables via config., e.g. config.username Another method would be a dictionary for your config. You could pickle and unpickle it. -- Dipl.-Inform(FH) Peter Heitzer, peter.heitzer at rz.uni-regensburg.de From chololennon at hotmail.com Wed Feb 8 07:34:56 2017 From: chololennon at hotmail.com (Cholo Lennon) Date: Wed, 8 Feb 2017 09:34:56 -0300 Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: On 02/08/2017 08:26 AM, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. I use it a lot ;-) > Would a JSON file be a good idea? > -- Cholo Lennon Bs.As. ARG From dpalao.python at gmail.com Wed Feb 8 07:40:36 2017 From: dpalao.python at gmail.com (David Palao) Date: Wed, 8 Feb 2017 13:40:36 +0100 Subject: How to store properties In-Reply-To: References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: 2017-02-08 13:34 GMT+01:00 Cholo Lennon : > On 02/08/2017 08:26 AM, Cecil Westerhof wrote: >> >> In Java you (can) use a properties file store configuration. What is >> the best way to do something like that in Python? >> I saw ConfigParser, but have the feeling that it is not really used. > > > I use it a lot ;-) > > >> Would a JSON file be a good idea? >> > > > -- > Cholo Lennon > Bs.As. > ARG > -- > https://mail.python.org/mailman/listinfo/python-list Me too. From Cecil at decebal.nl Wed Feb 8 07:46:37 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Wed, 08 Feb 2017 13:46:37 +0100 Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: <87y3xg26ky.fsf@Equus.decebal.nl> On Wednesday 8 Feb 2017 13:11 CET, Peter Heitzer wrote: > Cecil Westerhof wrote: >> In Java you (can) use a properties file store configuration. What >> is the best way to do something like that in Python? I saw >> ConfigParser, but have the feeling that it is not really used. >> Would a JSON file be a good idea? > > If you only want to read the configuration, just use an ordinary > file you import. For example config.py contains the lines: > username=myuser > server=myserver > password=secret > > In your script: > > import config > > Now you can referenc all the variables via config., e.g. > config.username That I know, but it is a security risk. > Another method would be a dictionary for your config. You could > pickle and unpickle it. Is pickle not discouraged? Better to use a JSON file I think. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From neilc at norwich.edu Wed Feb 8 08:41:23 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Wed, 8 Feb 2017 13:41:23 +0000 (UTC) Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: On 2017-02-08, Cholo Lennon wrote: > On 02/08/2017 08:26 AM, Cecil Westerhof wrote: >> In Java you (can) use a properties file store configuration. >> What is the best way to do something like that in Python? I >> saw ConfigParser, but have the feeling that it is not really >> used. > > I use it a lot ;-) Me too. I wrote a script once to convert all my .cfg files to JSON at one point while trying out a switch from Python to Go, but never made the changeover. -- Neil Cerutti From Cecil_Westerhof at f38.n261.z1 Wed Feb 8 08:46:36 2017 From: Cecil_Westerhof at f38.n261.z1 (Cecil_Westerhof at f38.n261.z1) Date: Wed, 08 Feb 2017 13:46:36 +0000 Subject: How to store properties References: <2253557732@f38.n261.z1.ftn> Message-ID: <2549486914@f38.n261.z1.ftn> From: Cecil Westerhof On Wednesday 8 Feb 2017 13:11 CET, Peter Heitzer wrote: > Cecil Westerhof wrote: >> In Java you (can) use a properties file store configuration. What >> is the best way to do something like that in Python? I saw >> ConfigParser, but have the feeling that it is not really used. >> Would a JSON file be a good idea? > > If you only want to read the configuration, just use an ordinary > file you import. For example config.py contains the lines: > username=myuser > server=myserver > password=secret > > In your script: > > import config > > Now you can referenc all the variables via config., e.g. > config.username That I know, but it is a security risk. > Another method would be a dictionary for your config. You could > pickle and unpickle it. Is pickle not discouraged? Better to use a JSON file I think. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From torriem at gmail.com Wed Feb 8 10:55:13 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 8 Feb 2017 08:55:13 -0700 Subject: How to store properties In-Reply-To: <87efz83ou6.fsf@Equus.decebal.nl> References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: <9014381b-386f-1aa9-e106-cca7744fbfa8@gmail.com> On 02/08/2017 04:26 AM, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. > Would a JSON file be a good idea? I've used ConfigParser before. I think the relative simpleness of .INI files. If I was expecting end users to manipulate a config file, I would use ConfigParser. But 90% of the time, I just use another python module. For my purposes, there is no security risk to it. Due to its nature, Django also uses python modules to define configuration such as URL maps, etc. When I'm using GTK+, I would use the gsettings facilities of GTK+ to store and retrieve config variables. And with Qt, I'd use QSettings. I think in both cases they use backends appropriate for the operating system. On Windows, that would be registry settings in the user hive. From rgaddi at highlandtechnology.invalid Wed Feb 8 12:31:09 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 8 Feb 2017 09:31:09 -0800 Subject: How to store properties In-Reply-To: <87efz83ou6.fsf@Equus.decebal.nl> References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: On 02/08/2017 03:26 AM, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. > Would a JSON file be a good idea? > In the interest of trying to be cross-platform and support fallbacks and work natively into the system and play nicely with my Qt ecosystem I decided to use PySide.QtCore.QSettings instead of ConfigParser for a major application framework I put together. Bloody mistake that was, but now I'm stuck with it. JSON's cute, but the format doesn't support inline comments. If you really need complex arrays and nested data structures in your configuration files, you're probably wrong. ConfirgParser if you care about security. Import a raw Python file if you don't. Solved problem and move on; wheel reinvention is for suckers. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From python at deborahswanson.net Wed Feb 8 14:31:47 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Wed, 8 Feb 2017 11:31:47 -0800 Subject: gmail api In-Reply-To: <32a3c9f3-55ff-4aa2-a33e-84b989046ff6@googlegroups.com> Message-ID: <001e01d28241$fd84f540$27b23dae@sambora> Kelvid Pang wrote, on February 06, 2017 10:22 PM > > hi, > > I am trying to gmail api with reference to this URL: > https://developers.google.com/gmail/api/quickstart/python > > > But I couldn't find the > 'gmail-python-quickstart.json' file. Any one can help? thanks. You likely won't have a 'gmail-python-quickstart.json' file if you didn't download one from Google APIs. Download it in substep 'g' (Download JSON) in the "Step 1: Turn on the Gmail API" instructions. (But you have to do substeps 'a' thru 'f' first.) If you then want to find it and make modifications, and follow the applicable instructions. From tjreedy at udel.edu Wed Feb 8 15:34:53 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 8 Feb 2017 15:34:53 -0500 Subject: How to store properties In-Reply-To: <87efz83ou6.fsf@Equus.decebal.nl> References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: On 2/8/2017 6:26 AM, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. IDLE uses it for default and user configuration files. -- Terry Jan Reedy From accessnewbie at gmail.com Wed Feb 8 16:11:51 2017 From: accessnewbie at gmail.com (accessnewbie at gmail.com) Date: Wed, 8 Feb 2017 13:11:51 -0800 (PST) Subject: Coding issue with XML for word document In-Reply-To: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> References: <4aef3690-426c-43a1-b603-2f84fde79f4f@googlegroups.com> Message-ID: <9c8ad28a-1619-483b-bd18-77815d15af27@googlegroups.com> > myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + " > > What am I missing here? Sorry about posting in python group. My XML coding was embedded in python script. I will do better next time. The problem was with placement of tags. The following worked: myBlurb = "My Favorite Cars - My favorite cars are available at " + myText + " I needed to put the carriage return within the wrapper. I also needed to add the text elements and before and after the variable. From oegeeks at gmail.com Wed Feb 8 19:56:19 2017 From: oegeeks at gmail.com (Andreas Paeffgen) Date: Wed, 8 Feb 2017 18:56:19 -0600 Subject: subprocess problem Message-ID: The Problem with the subprocess code is: Using the sourcecode functioning as normal. The frozen app with cx_freeze on every platform just returns an empty result Here is the code in short: def get_path_pandoc(): settings = QSettings('Pandoc', 'PanConvert') path_pandoc = settings.value('path_pandoc','') if not os.path.isfile(path_pandoc): if platform.system() == 'Darwin' or os.name == 'posix': args = ['which', 'pandoc'] p = subprocess.Popen( args, stdin=subprocess.PIPE, stdout=subprocess.PIPE) path_pandoc = str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8')) The whole problematic code can be checked on http://github.com/apaeffgen/panconvert in source/converters/interface_pandoc.py I debugged it with some QMessage-Boxes. I just know, that the returnd result is empty. I tried some stderr on this code also. But also the error message is empty. Any hints what could be changed in the code, so it also works in the frozen app? At the moment i use python3.4 to 3.5 depending on the platform (Mac, Win, Linux) P.S. Tried also some code with invoking the bash and afterwords the which pandoc code. To know avail also From cs at zip.com.au Wed Feb 8 23:54:52 2017 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 9 Feb 2017 15:54:52 +1100 Subject: subprocess problem In-Reply-To: References: Message-ID: <20170209045452.GA41763@cskk.homeip.net> On 08Feb2017 18:56, Andreas Paeffgen wrote: >The Problem with the subprocess code is: Using the sourcecode >functioning as normal. >The frozen app with cx_freeze on every platform just returns an empty result I don't know what the sentence above above cx_freeze means. >Here is the code in short: >def get_path_pandoc(): > settings = QSettings('Pandoc', 'PanConvert') > path_pandoc = settings.value('path_pandoc','') > if not os.path.isfile(path_pandoc): > if platform.system() == 'Darwin' or os.name == 'posix': > args = ['which', 'pandoc'] > p = subprocess.Popen( > args, > stdin=subprocess.PIPE, "which" doesn't read any input. You probably want "subprocess.DEVNULL" instead of "subprocess.PIPE" for stdin. > stdout=subprocess.PIPE) > path_pandoc = > str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8')) > >The whole problematic code can be checked on >http://github.com/apaeffgen/panconvert >in source/converters/interface_pandoc.py > >I debugged it with some QMessage-Boxes. I just know, that the returnd >result is empty. >I tried some stderr on this code also. But also the error message is empty. What does "which pandoc" say to you at a terminal prompt? The code above relies on that returning you the path to the pandoc executable. If it is not in your path then you will get an empty result. Also, the "which" programme should have a zero exit status if it finds and prints the path to pandoc, and a nonzero exit status otherwise? Find out what exit status it returns. This is available from the "p" object after communicate() returns. See the subprocess doco for details. The easy thing to do is probably to call "p.check_returncode()" after communicate() returns. >Any hints what could be changed in the code, so it also works in the >frozen app? What's a frozen app? Maybe that is your problem: when you run the programme by hand it works, but in this "forzen" state it doesn't. The "which" command isn't magic. It searchs for commands by looking for executables in the directories listed in the $PATH environment variable. Get your program to print or display the value of "os.environ['PATH']" before invoking "which". The difference may tell you something useful. >At the moment i use python3.4 to 3.5 depending on the platform (Mac, >Win, Linux) Thank you for this information; many people forget to supply it. >P.S. Tried also some code with invoking the bash and afterwords the which >pandoc code. To know avail also From a shell prompt (such as bash's prompt) issuing the command: which pandoc should print the executable path, _if_ your $PATH includes the right directory. You can look at $PATH from the shell with the command: echo $PATH Do you know where pandoc normally resides on your system? Cheers, Cameron Simpson From nomail at com.invalid Thu Feb 9 02:03:32 2017 From: nomail at com.invalid (ast) Date: Thu, 9 Feb 2017 08:03:32 +0100 Subject: Decorator Message-ID: <589c1447$0$3337$426a74cc@news.free.fr> Hi In python courses I read, it is explained that @decor def f(): pass is equivalent to: def f(): pass f = decor(f) But that's not always true. See this code class Temperature: def __init__(self): self.value = 0 # @property def celsius(self): return self.value celsius = property(celsius) # @celsius.setter def celsius(self, value): <-- overwrites previous celsius self.value = value celsius = celsius.setter(celsius) <-- error here When you define the setter function named celsius, it overwrites the previous property object also named celsius ... and it fails: celsius = celsius.setter(celsius) AttributeError: 'function' object has no attribute 'setter' any comment ? From rosuav at gmail.com Thu Feb 9 02:19:07 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 9 Feb 2017 18:19:07 +1100 Subject: Decorator In-Reply-To: <589c1447$0$3337$426a74cc@news.free.fr> References: <589c1447$0$3337$426a74cc@news.free.fr> Message-ID: On Thu, Feb 9, 2017 at 6:03 PM, ast wrote: > class Temperature: > def __init__(self): > self.value = 0 > > # @property > def celsius(self): return self.value > celsius = property(celsius) > # @celsius.setter def celsius(self, value): <-- > overwrites previous celsius > self.value = value > > celsius = celsius.setter(celsius) <-- error here > The difference is that the decorator line is evaluated before the function is defined. Try this: class Temperature: def __init__(self): self.value = 0 #@property def celsius(self): return self.value celsius = property(celsius) #@celsius.setter _tmp = celsius.setter def celsius(self, value): self.value = value celsius = _tmp(celsius) del _tmp Now it'll work. (Actually, there are some other very minor subtleties; the name isn't temporarily bound to the undecorated function prior to the decorator being called. But in this case, it's simply an order of evaluation.) In CPython (3.7 on Debian Linux, fwiw), the decorated function is processed like this: @celsius.setter #_tmp = celsius.setter def celsius(self, value): self.value = value #celsius = _tmp(celsius) #del _tmp 11 28 LOAD_NAME 5 (celsius) 30 LOAD_ATTR 6 (setter) 32 LOAD_CONST 5 () 34 LOAD_CONST 4 ('Temperature..Temperature.celsius') 36 MAKE_FUNCTION 0 38 CALL_FUNCTION 1 40 STORE_NAME 5 (celsius) In other words: 1) Evaluate "celsius.setter" and save that on the stack 2) Fetch up the compiled code for the undecorated function 3) Grab the name of the function (it's a constant) 4) Create a function. Don't save it anywhere yet, just put it on the stack. 5) Call the function from step 1, passing the function from step 4 as a parameter. 6) Whatever it returns, save that under the name "celsius". That's very detailed and nitty-gritty, but that's what really happens when you use "@celsius.setter" in your code. ChrisA From wolfgang.maier at biologie.uni-freiburg.de Thu Feb 9 03:08:45 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Thu, 9 Feb 2017 09:08:45 +0100 Subject: subprocess problem In-Reply-To: References: Message-ID: <16db325b-a36f-dcb3-28f9-3cac3d0b5c63@biologie.uni-freiburg.de> On 09.02.2017 01:56, Andreas Paeffgen wrote: > The Problem with the subprocess code is: Using the sourcecode > functioning as normal. > The frozen app with cx_freeze on every platform just returns an empty > result > > Here is the code in short: > def get_path_pandoc(): > > > > > settings = QSettings('Pandoc', 'PanConvert') > > path_pandoc = settings.value('path_pandoc','') > > > > > if not os.path.isfile(path_pandoc): > > > > > if platform.system() == 'Darwin' or os.name == 'posix': > > args = ['which', 'pandoc'] > > p = subprocess.Popen( > > args, > > stdin=subprocess.PIPE, > > stdout=subprocess.PIPE) > > > > > path_pandoc = > str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8')) > > > The whole problematic code can be checked on > http://github.com/apaeffgen/panconvert > in source/converters/interface_pandoc.py > Checking your repo I found that get_path_pandoc, the function from which you took the code snippet above, will always return None if os.path.isfile(path_pandoc). This is probably not what you are intending. Do you know if path_pandoc is maybe set to an existing file in your frozen code already so the whole 'which' or 'where' branch is never executed? Best, Wolfgang From wolfgang.maier at biologie.uni-freiburg.de Thu Feb 9 03:09:02 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Thu, 9 Feb 2017 09:09:02 +0100 Subject: subprocess problem In-Reply-To: References: Message-ID: On 09.02.2017 01:56, Andreas Paeffgen wrote: > The Problem with the subprocess code is: Using the sourcecode > functioning as normal. > The frozen app with cx_freeze on every platform just returns an empty > result > > Here is the code in short: > def get_path_pandoc(): > > > > > settings = QSettings('Pandoc', 'PanConvert') > > path_pandoc = settings.value('path_pandoc','') > > > > > if not os.path.isfile(path_pandoc): > > > > > if platform.system() == 'Darwin' or os.name == 'posix': > > args = ['which', 'pandoc'] > > p = subprocess.Popen( > > args, > > stdin=subprocess.PIPE, > > stdout=subprocess.PIPE) > > > > > path_pandoc = > str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8')) > > > The whole problematic code can be checked on > http://github.com/apaeffgen/panconvert > in source/converters/interface_pandoc.py > Checking your repo I found that get_path_pandoc, the function from which you took the code snippet above, will always return None if os.path.isfile(path_pandoc). This is probably not what you are intending. Do you know if path_pandoc is maybe set to an existing file in your frozen code already so the whole 'which' or 'where' branch is never executed? Best, Wolfgang From dieter at handshake.de Thu Feb 9 03:43:07 2017 From: dieter at handshake.de (dieter) Date: Thu, 09 Feb 2017 09:43:07 +0100 Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> <87y3xg26ky.fsf@Equus.decebal.nl> Message-ID: <8760kjrbz8.fsf@handshake.de> Cecil Westerhof writes: > ... >> If you only want to read the configuration, just use an ordinary >> file you import. For example config.py contains the lines: >> username=myuser >> server=myserver >> password=secret >> >> In your script: >> >> import config >> >> Now you can referenc all the variables via config., e.g. >> config.username > > That I know, but it is a security risk. It is a security risk if you allow potential attackers to modify the Python files. Then, however, those attackers could also modify the Python code itself (rather than the config file). Thus, the risk may not much increase (depending on how different the protection for the config file is compared to that for other Python source code). > ... >> Another method would be a dictionary for your config. You could >> pickle and unpickle it. > > Is pickle not discouraged? Better to use a JSON file I think. "pickle", too, has a potential security risk -- if you allow unpickling from untrusted source. Usually, however, configuration comes from trusted sources. However, if JSON has sufficient data type support for you, go for it. From steve at pearwood.info Thu Feb 9 03:48:28 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 09 Feb 2017 08:48:28 GMT Subject: Decorator References: <589c1447$0$3337$426a74cc@news.free.fr> Message-ID: <589c2cdc$0$1584$c3e8da3$5496439d@news.astraweb.com> On Thu, 09 Feb 2017 08:03:32 +0100, ast wrote: > Hi > > In python courses I read, it is explained that > > @decor > def f(): > pass > > is equivalent to: > > def f(): > pass > > f = decor(f) > > But that's not always true. See this code [...] > any comment ? Congratulations, you've found a microscopic corner of the language where the two different ways of applying decorators are different. Are you just sharing with us, or do you think there is a problem that needs to be solved? You're absolutely correct that when using `property`, trying to apply it by hand instead of using the @ syntax is tricky. So don't apply it by hand, use the @ syntax. -- Steve From Cecil at decebal.nl Thu Feb 9 04:13:53 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Thu, 09 Feb 2017 10:13:53 +0100 Subject: How to store properties References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: <87tw8320by.fsf@Equus.decebal.nl> On Wednesday 8 Feb 2017 12:26 CET, Cecil Westerhof wrote: > In Java you (can) use a properties file store configuration. What is > the best way to do something like that in Python? > I saw ConfigParser, but have the feeling that it is not really used. > Would a JSON file be a good idea? Thanks for all the input. I think I go for configparser (as it is called in python3). It looks very promising. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From rosuav at gmail.com Thu Feb 9 04:47:44 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 9 Feb 2017 20:47:44 +1100 Subject: How to store properties In-Reply-To: <8760kjrbz8.fsf@handshake.de> References: <87efz83ou6.fsf@Equus.decebal.nl> <87y3xg26ky.fsf@Equus.decebal.nl> <8760kjrbz8.fsf@handshake.de> Message-ID: On Thu, Feb 9, 2017 at 7:43 PM, dieter wrote: > "pickle", too, has a potential security risk -- if you allow > unpickling from untrusted source. Usually, however, configuration > comes from trusted sources. Pickle's other downside is that it's an opaque binary file, unlike ConfigParser, JSON, and Python code, which are human-readable text. Letting the end user edit your configs is often a feature, not a bug. ChrisA From nomail at com.invalid Thu Feb 9 06:29:29 2017 From: nomail at com.invalid (ast) Date: Thu, 9 Feb 2017 12:29:29 +0100 Subject: Decorator In-Reply-To: <589c2cdc$0$1584$c3e8da3$5496439d@news.astraweb.com> References: <589c1447$0$3337$426a74cc@news.free.fr> <589c2cdc$0$1584$c3e8da3$5496439d@news.astraweb.com> Message-ID: <589c52a0$0$19764$426a74cc@news.free.fr> "Steven D'Aprano" a ?crit dans le message de news:589c2cdc$0$1584$c3e8da3$5496439d at news.astraweb.com... > On Thu, 09 Feb 2017 08:03:32 +0100, ast wrote: > Congratulations, you've found a microscopic corner of the language where > the two different ways of applying decorators are different. > > Are you just sharing with us, or do you think there is a problem that > needs to be solved? I just wanted to have a confirmation. It's OK with Chris Angelico's detailed answer. Ty From steve+python at pearwood.info Thu Feb 9 06:46:07 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 09 Feb 2017 22:46:07 +1100 Subject: Rename file without overwriting existing files References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> On Mon, 30 Jan 2017 09:39 pm, Peter Otten wrote: >>>> def rename(source, dest): > ... os.link(source, dest) > ... os.unlink(source) > ... >>>> rename("foo", "baz") >>>> os.listdir() > ['bar', 'baz'] >>>> rename("bar", "baz") > Traceback (most recent call last): > File "", line 1, in > File "", line 2, in rename > FileExistsError: [Errno 17] File exists: 'bar' -> 'baz' Thanks Peter! That's not quite ideal, as it isn't atomic: it is possible that the link will succeed, but the unlink won't. But I prefer that over the alternative, which is over-writing a file and causing data loss. So to summarise, os.rename(source, destination): - is atomic on POSIX systems, if source and destination are both on the same file system; - may not be atomic on Windows? - may over-write an existing destination on POSIX systems, but not on Windows; - and it doesn't work across file systems. os.replace(source, destination) is similar, except that it may over-write an existing destination on Windows as well as on POSIX systems. The link/unlink trick: - avoids over-writing existing files on POSIX systems at least; - but maybe not Windows? - isn't atomic, so in the worst case you end up with two links to the one file; - but os.link may not be available on all platforms; - and it won't work across file systems. Putting that all together, here's my attempt at a version of file rename which doesn't over-write existing files: import os import shutil def rename(src, dest): """Rename src to dest only if dest doesn't already exist (almost).""" if hasattr(os, 'link'): try: os.link(src, dest) except OSError: pass else: os.unlink(src) return # Fallback to an implementation which is vulnerable to a # Time Of Check to Time Of Use bug. # Try to reduce the window for this race condition by minimizing # the number of lookups needed between one call and the next. move = shutil.move if not os.file.exists(dest): move(src, dest) else: raise shutil.Error("Destination path '%s' already exists" % dest) Any comments? Any bugs? Any cross-platform way to slay this TOCTOU bug once and for all? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Feb 9 06:47:54 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 09 Feb 2017 22:47:54 +1100 Subject: Rename file without overwriting existing files References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> <8537g0t74b.fsf@benfinney.id.au> Message-ID: <589c56eb$0$1606$c3e8da3$5496439d@news.astraweb.com> On Tue, 31 Jan 2017 11:17 am, Ben Finney wrote: > Peter Otten <__peter__ at web.de> writes: > >> http://stackoverflow.com/questions/3222341/how-to-rename-without-race-conditions >> >> and from a quick test it appears to work on Linux: > > By ?works on Linux?, I assume you mean ?works on filesystems that use > inodes and hard links?. That is not true for all filesystems, even on > Linux. Indeed it is not, and we're often very sloppy about describing file system differences as if they were OS differences. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jussi.piitulainen at helsinki.fi Thu Feb 9 07:54:41 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 09 Feb 2017 14:54:41 +0200 Subject: Rename file without overwriting existing files References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano writes: > On Mon, 30 Jan 2017 09:39 pm, Peter Otten wrote: > >>>>> def rename(source, dest): >> ... os.link(source, dest) >> ... os.unlink(source) >> ... >>>>> rename("foo", "baz") >>>>> os.listdir() >> ['bar', 'baz'] >>>>> rename("bar", "baz") >> Traceback (most recent call last): >> File "", line 1, in >> File "", line 2, in rename >> FileExistsError: [Errno 17] File exists: 'bar' -> 'baz' > > > Thanks Peter! > > That's not quite ideal, as it isn't atomic: it is possible that the link > will succeed, but the unlink won't. But I prefer that over the alternative, > which is over-writing a file and causing data loss. > > So to summarise, os.rename(source, destination): > > - is atomic on POSIX systems, if source and destination are both on the > same file system; > > - may not be atomic on Windows? > > - may over-write an existing destination on POSIX systems, but not on > Windows; > > - and it doesn't work across file systems. > > os.replace(source, destination) is similar, except that it may over-write an > existing destination on Windows as well as on POSIX systems. > > > The link/unlink trick: > > - avoids over-writing existing files on POSIX systems at least; > > - but maybe not Windows? > > - isn't atomic, so in the worst case you end up with two links to > the one file; > > - but os.link may not be available on all platforms; > > - and it won't work across file systems. > > > Putting that all together, here's my attempt at a version of file rename > which doesn't over-write existing files: > > > import os > import shutil > > def rename(src, dest): > """Rename src to dest only if dest doesn't already exist (almost).""" > if hasattr(os, 'link'): > try: > os.link(src, dest) > except OSError: > pass > else: > os.unlink(src) > return > # Fallback to an implementation which is vulnerable to a > # Time Of Check to Time Of Use bug. > # Try to reduce the window for this race condition by minimizing > # the number of lookups needed between one call and the next. > move = shutil.move > if not os.file.exists(dest): > move(src, dest) > else: > raise shutil.Error("Destination path '%s' already exists" % dest) > > > > Any comments? Any bugs? Any cross-platform way to slay this TOCTOU bug once > and for all? To claim the filename before crossing a filesystem boundary, how about: 1) create a temporary file in the target directory (tempfile.mkstemp) 2) link the temporary file to the target name (in the same directory) 3) unlink the temporary name 4) now it should be safe to move the source file to the target name 5) set permissions and whatever other attributes there are? Or maybe copy the source file to the temporary name, link the copy to the target name, unlink the temporary name, unlink the source file; failing the link step: unlink the temporary name but do not unlink the source file. From eryksun at gmail.com Thu Feb 9 08:07:33 2017 From: eryksun at gmail.com (eryk sun) Date: Thu, 9 Feb 2017 13:07:33 +0000 Subject: Rename file without overwriting existing files In-Reply-To: <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Feb 9, 2017 at 11:46 AM, Steve D'Aprano wrote: > > So to summarise, os.rename(source, destination): > > - is atomic on POSIX systems, if source and destination are both on the > same file system; > - may not be atomic on Windows? > - may over-write an existing destination on POSIX systems, but not on > Windows; > - and it doesn't work across file systems. On Windows in 2.7 and prior to 3.3, os.rename will silently copy and delete when the destination isn't on the same volume. It may even silently leave the original file in place in some cases -- e.g. when the file is read-only and the user isn't allowed to modify the file attributes. If the destination is on the same volume, renaming should be atomic via the system calls NtOpenFile and NtSetInformationFile. Ultimately it depends on the file system implementation of IRP_MJ_SET_INFORMATION, FileRenameInformation [1]. > The link/unlink trick: > - avoids over-writing existing files on POSIX systems at least; > - but maybe not Windows? This works for renaming files on Windows as long as the file system supports hard links (e.g. NTFS). It's not documented on MSDN, but WinAPI CreateHardLink is implemented by calling NtSetInformationFile to set the FileLinkInformation, with ReplaceIfExists set to FALSE, so it fails if the destination exists. Note that this does not allow renaming directories. See the note for FileLinkInformation [1]; NTFS doesn't allow directory hard links. But why bother with this 'trick' on Windows? [1]: https://msdn.microsoft.com/en-us/library/ff549366 From oegeeks at gmail.com Thu Feb 9 09:34:20 2017 From: oegeeks at gmail.com (Andreas Paeffgen) Date: Thu, 9 Feb 2017 08:34:20 -0600 Subject: subprocess problem References: Message-ID: Maybe i could use another trick to circumvent the problems in the frozen app? The frozen apps can be downloaded here: https://sourceforge.net/projects/panconvert/files/Newest/ @Cameron: 1. I use PyQT5 for a creating a gui app. To run the app on other systems, where no QT5 and PyQT5 is installed, a bundle is created with all relevant libraries, so it will start independently. This app bundle, is called a frozen app. Not only the source code, but all it dependencies are distributed including python3 itself. 2. In the test-environment pandoc is installed in /usr/local/bin/pandoc (Mac) or /bin/pandoc (Linux). Which returns the path correctly, at the terminal (shell), in Python and in the Gui-App Panconvert. 3. I am aware of the limitations of which. There is a fallback to manually insert the path. So the app will function anyway. And i want to avoid to search the whole filesystem. This takes minutes on my test machines, so this is no option. 4. I suppose that something is different from executing the code in the frozen state. The app e.g. on Mac is not started from a python shell or a comand shell. But even on linux, when started from a shell it does not work. 5. I will try the hint with dev.null and the p.check_returncode() @Wolfgang 1. The function should be executed only, if the path is unknown. If the user enters the path manually in the settings and pandoc exists, the function should not be executed to save computation time. 2. Only if the manually entered path is not correct or empty the function should be executed, hence 'if not os.path.isfile(path_pandoc)' 3. The function fills in the path automatically if which returns a value. This works in the source code From oegeeks at gmail.com Thu Feb 9 12:16:18 2017 From: oegeeks at gmail.com (Andreas Paeffgen) Date: Thu, 9 Feb 2017 11:16:18 -0600 Subject: subprocess problem References: Message-ID: I guess which does not return an error code. If it does not find anything, the return is just blank. If it finds something, the path is returned. So the change of code did not help, because there is just no error message. Could there be a $path problem in the subprocess started inside the binary? From best_lay at yahoo.com Thu Feb 9 12:59:34 2017 From: best_lay at yahoo.com (Wildman) Date: Thu, 09 Feb 2017 11:59:34 -0600 Subject: subprocess problem References: Message-ID: On Thu, 09 Feb 2017 11:16:18 -0600, Andreas Paeffgen wrote: > I guess which does not return an error code. If it does not find > anything, the return is just blank. If it finds something, the path is > returned. > > So the change of code did not help, because there is just no error message. > Could there be a $path problem in the subprocess started inside the binary? Here is a method I frequently use to replace the which command. (air code) import os pathlist = os.environ["PATH"].split(":") def which(target) for p in pathlist: fullpath = p + "/" + target if os.path.isfile(fullpath): return fullpath, True return "", False target, found = which("pandoc") if found: target will contain the full path to pandoc else: pandoc was not found -- GNU/Linux user #557453 The cow died so I don't need your bull! From gd.usenet at spamfence.net Thu Feb 9 13:26:10 2017 From: gd.usenet at spamfence.net (=?UTF-8?Q?G=c3=bcnther_Dietrich?=) Date: Thu, 9 Feb 2017 19:26:10 +0100 Subject: subprocess problem In-Reply-To: References: Message-ID: <1bf3835e-4d11-3b2b-5d79-5ea78a7845dc@spamfence.net> Am 09.02.17 um 18:16 schrieb Andreas Paeffgen: > I guess which does not return an error code. In fact, id does return a return code. Here an example: | honk:~ gdie$ which bash; echo $? | /bin/bash | 0 | honk:~ gdie$ which wzlbrmpf; echo $? | 1 It is 0 on success, 1 for a failure. Exactly the result I would expect from an unixoid program. On Ubuntu 16.04, 'man which' tells of three possible exit states: 0, 1 and 2. On Mac OS, the return codes are not documented. It seems, there are only 0 (success) and 1 (failure). Normally, on unixoid operating systems, return code 0 indicates success, anything else non-success, failure or error. > If it does not find > anything, the return is just blank. If it finds something, the path is > returned. That is the result on STDOUT. It is _not_ the return code! > So the change of code did not help, because there is just no error message. Do you expect an error message, or a return code? An error message would -- normally -- appear on STDERR. > Could there be a $path problem in the subprocess started inside the binary? That's for sure. I already had this problem with py2exe. I solved it by passing a directory containing a suitable PATH entry to the env argument of subprocess.Popen(). Problem gone. If you call 'python myscript.py' on the shell command line, the python script will inherit a copy the shell's environment. If you start a frozen python script, there is no shell's environment, it could inherit. It is up to the tool you used to freeze the script (cx_freeze, py2exe, etc.), which environment contents it will see. Best regards, G?nther From cs at zip.com.au Thu Feb 9 17:50:32 2017 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 10 Feb 2017 09:50:32 +1100 Subject: subprocess problem In-Reply-To: References: Message-ID: <20170209225032.GA67436@cskk.homeip.net> On 09Feb2017 11:16, Andreas Paeffgen wrote: >I guess which does not return an error code. If it does not find >anything, the return is just blank. If it finds something, the path is >returned. > >So the change of code did not help, because there is just no error message. >Could there be a $path problem in the subprocess started inside the binary? You're confusing the error code (an integer returned from _every_ exiting process) and the process' stdout and stderr, which are data streams. This is why I suggested the check_returncode() method, which examines the error code. Cheers, Cameron Simpson From cs at zip.com.au Thu Feb 9 17:53:32 2017 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 10 Feb 2017 09:53:32 +1100 Subject: subprocess problem In-Reply-To: References: Message-ID: <20170209225332.GA78558@cskk.homeip.net> On 09Feb2017 11:59, Wildman wrote: >Here is a method I frequently use to replace the which >command. (air code) > >import os >pathlist = os.environ["PATH"].split(":") > >def which(target) > for p in pathlist: > fullpath = p + "/" + target > if os.path.isfile(fullpath): > return fullpath, True > return "", False Cleaner is to return the path if found, or None if not. >target, found = which("pandoc") >if found: > target will contain the full path to pandoc >else: > pandoc was not found You want to check for execute permssion too. Cheers, Cameron Simpson From tjreedy at udel.edu Thu Feb 9 18:18:57 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 9 Feb 2017 18:18:57 -0500 Subject: Decorator In-Reply-To: <589c1447$0$3337$426a74cc@news.free.fr> References: <589c1447$0$3337$426a74cc@news.free.fr> Message-ID: On 2/9/2017 2:03 AM, ast wrote: > Hi > > In python courses I read, it is explained that > > @decor > def f(): > pass > > is equivalent to: > > def f(): > pass > > f = decor(f) > > But that's not always true. See this code Which is why the official docs now say 'roughly equivalent' ''' For example, the following code @f1(arg) @f2 def func(): pass is roughly equivalent to def func(): pass func = f1(arg)(f2(func)) except that the original function is not temporarily bound to the name func. ''' Perhaps "There are also cases in which evaluating 'fi(arg)' before rather than after the def statement makes a difference." should be added. -- Terry Jan Reedy From rosuav at gmail.com Thu Feb 9 18:35:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 10 Feb 2017 10:35:28 +1100 Subject: Decorator In-Reply-To: References: <589c1447$0$3337$426a74cc@news.free.fr> Message-ID: On Fri, Feb 10, 2017 at 10:18 AM, Terry Reedy wrote: > Perhaps "There are also cases in which evaluating 'fi(arg)' before rather > than after the def statement makes a difference." should be added. Perhaps not. The word "roughly" covers the odd edge cases, and Python has a general principle of evaluating top-to-bottom, left-to-right [1], which covers this situation. ChrisA [1] if not a conditional expression, else inside-to-outside From tjreedy at udel.edu Thu Feb 9 18:37:56 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 9 Feb 2017 18:37:56 -0500 Subject: The Dominion Message-ID: A video ad for Anaconda from its maker (satirizing a well know data-related film). I found it amusing, in a good way. https://www.youtube.com/watch?v=0XDqc5wTve0 PS. I don't use Anaconda and I am not posting this to promote it, nor do I necessarily agree with everything said, but I do appreciate it as a creative Python-related product, of a sort I would like to see more of. -- Terry Jan Reedy From eryksun at gmail.com Thu Feb 9 19:03:53 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 10 Feb 2017 00:03:53 +0000 Subject: subprocess problem In-Reply-To: <20170209225032.GA67436@cskk.homeip.net> References: <20170209225032.GA67436@cskk.homeip.net> Message-ID: On Thu, Feb 9, 2017 at 10:50 PM, Cameron Simpson wrote: > This is why I suggested the check_returncode() method, which examines the > error code. You must be thinking of the returncode attribute, which isn't a method. check_returncode() is a method of the CompletedProcess object that's returned by subprocess.run(), which was added in 3.5. The OP is using both 3.4 and 3.5, so run() isn't a practical option. The older check_output() function also checks the return code and raises a subprocess.CalledProcessError if the command fails. For example: >>> subprocess.check_output(['which', 'ls']) b'/bin/ls\n' >>> subprocess.check_output(['which', 'pandoc']) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.5/subprocess.py", line 626, in check_output **kwargs).stdout File "/usr/lib/python3.5/subprocess.py", line 708, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['which', 'pandoc']' returned non-zero exit status 1 From best_lay at yahoo.com Thu Feb 9 19:05:57 2017 From: best_lay at yahoo.com (Wildman) Date: Thu, 09 Feb 2017 18:05:57 -0600 Subject: subprocess problem References: <20170209225332.GA78558@cskk.homeip.net> Message-ID: On Fri, 10 Feb 2017 09:53:32 +1100, Cameron Simpson wrote: > On 09Feb2017 11:59, Wildman wrote: >>Here is a method I frequently use to replace the which >>command. (air code) >> >>import os >>pathlist = os.environ["PATH"].split(":") >> >>def which(target) >> for p in pathlist: >> fullpath = p + "/" + target >> if os.path.isfile(fullpath): >> return fullpath, True >> return "", False > > Cleaner is to return the path if found, or None if not. I don't see this as an important issue but you are right. >>target, found = which("pandoc") >>if found: >> target will contain the full path to pandoc >>else: >> pandoc was not found > > You want to check for execute permssion too. For my uses of the above code, it is unlikely that the 'target' would not have the exec bit set. But, your suggestion is valid due to the fact that I should not take anything for granted. > Cheers, > Cameron Simpson Corrected code: def which(target) for p in pathlist: fullpath = p + "/" + target if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK): return fullpath, True return None, False Thanks. -- GNU/Linux user #557453 Keyboard not detected! Press any key to continue... From python at mrabarnett.plus.com Thu Feb 9 19:36:37 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 10 Feb 2017 00:36:37 +0000 Subject: subprocess problem In-Reply-To: References: <20170209225332.GA78558@cskk.homeip.net> Message-ID: On 2017-02-10 00:05, Wildman via Python-list wrote: > On Fri, 10 Feb 2017 09:53:32 +1100, Cameron Simpson wrote: > >> On 09Feb2017 11:59, Wildman wrote: >>>Here is a method I frequently use to replace the which >>>command. (air code) >>> >>>import os >>>pathlist = os.environ["PATH"].split(":") >>> >>>def which(target) >>> for p in pathlist: >>> fullpath = p + "/" + target >>> if os.path.isfile(fullpath): >>> return fullpath, True >>> return "", False >> >> Cleaner is to return the path if found, or None if not. > > I don't see this as an important issue but you are > right. > >>>target, found = which("pandoc") >>>if found: >>> target will contain the full path to pandoc >>>else: >>> pandoc was not found >> >> You want to check for execute permssion too. > > For my uses of the above code, it is unlikely that > the 'target' would not have the exec bit set. But, > your suggestion is valid due to the fact that I > should not take anything for granted. > >> Cheers, >> Cameron Simpson > > Corrected code: > > def which(target) > for p in pathlist: > fullpath = p + "/" + target > if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK): > return fullpath, True > return None, False > > Thanks. > The implication was that you don't need the True/False, just the string/None. From eryksun at gmail.com Thu Feb 9 19:38:18 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 10 Feb 2017 00:38:18 +0000 Subject: subprocess problem In-Reply-To: References: <20170209225332.GA78558@cskk.homeip.net> Message-ID: On Fri, Feb 10, 2017 at 12:05 AM, Wildman via Python-list wrote: > > Corrected code: > > def which(target) > for p in pathlist: > fullpath = p + "/" + target > if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK): > return fullpath, True > return None, False Use shutil.which: https://docs.python.org/3/library/shutil.html#shutil.which From cs at zip.com.au Thu Feb 9 20:56:13 2017 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 10 Feb 2017 12:56:13 +1100 Subject: subprocess problem In-Reply-To: References: Message-ID: <20170210015613.GA19690@cskk.homeip.net> On 10Feb2017 00:03, eryk sun wrote: >On Thu, Feb 9, 2017 at 10:50 PM, Cameron Simpson wrote: >> This is why I suggested the check_returncode() method, which examines the >> error code. > >You must be thinking of the returncode attribute, which isn't a >method. check_returncode() is a method of the CompletedProcess object >that's returned by subprocess.run(), which was added in 3.5. The OP is >using both 3.4 and 3.5, so run() isn't a practical option. The communicate method mentioned it; I hadn't realised it was new with 3.5, good point. >The older check_output() function also checks the return code and >raises a subprocess.CalledProcessError if the command fails. For >example: [...] Cool, Cameron Simpson From adam14711993 at gmail.com Thu Feb 9 21:33:54 2017 From: adam14711993 at gmail.com (adam14711993 at gmail.com) Date: Thu, 9 Feb 2017 18:33:54 -0800 (PST) Subject: int vs. float Message-ID: Hello, My computer programming professor challenged me to figure out a way to manipulate my program to display one error message if the user input is a zero or a negative number, and a separate error message if the user input is a decimal number. My program starts out: number_purchases_str = int(input("Enter the number of packages you're buying:")) number_purchases = int(number_purchases_str) format(number_purchases, "12,") So if the user input is, for example, -9 or 0, the program will result with: Sorry, you must order at least one package." What I cannot figure out is how to write it so that if my user input is, for example, 1.5, the program will result with: Sorry, you can only order whole packages. I understand that because I am starting out by assigning my number_purchases_str to be an int, when the user enters a float that is a conflict and will crash. My professor apparently believes there is a way to accomplish this. Any help or advice would be greatly appreciated. Thank you. From rosuav at gmail.com Thu Feb 9 21:42:20 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 10 Feb 2017 13:42:20 +1100 Subject: int vs. float In-Reply-To: References: Message-ID: On Fri, Feb 10, 2017 at 1:33 PM, wrote: > I understand that because I am starting out by assigning my number_purchases_str to be an int, when the user enters a float that is a conflict and will crash. > > My professor apparently believes there is a way to accomplish this. Any help or advice would be greatly appreciated. What's actually happening is that you're converting a string to an integer. You can play with this at the interactive prompt: >>> int("1") 1 >>> int("9") 9 >>> int("-3") -3 >>> int("1.5") Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '1.5' >>> int("hello") Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: 'hello' The user always enters a string. Fundamentally, people don't type numbers - they type strings. What you're looking for is a way to let the user type digits that you interpret as an integer. Fortunately, "crash" in Python really means "raise an exception". Explore exception handling and you'll find out how you can *catch* those exceptions you understand. I won't give you the whole solution, but you should be able to research it from there. To distinguish between "1.5" and "hello", you could attempt to convert the digits to a float - if that succeeds, the user entered a fractional number of packages. Or maybe you don't care about that difference. Enjoy Pythonning! ChrisA From kelvidpang at gmail.com Fri Feb 10 00:53:36 2017 From: kelvidpang at gmail.com (Kelvid Pang) Date: Thu, 9 Feb 2017 21:53:36 -0800 (PST) Subject: gmail api In-Reply-To: <32a3c9f3-55ff-4aa2-a33e-84b989046ff6@googlegroups.com> References: <32a3c9f3-55ff-4aa2-a33e-84b989046ff6@googlegroups.com> Message-ID: On Tuesday, 7 February 2017 14:22:32 UTC+8, Kelvid Pang wrote: > hi, > > I am trying to gmail api with reference to this URL: https://developers.google.com/gmail/api/quickstart/python > > But I couldn't find the 'gmail-python-quickstart.json' file. Any one can help? thanks. First of all, I have figured out the gmail api. thanks for those who give me the guide, apparently I didn't follow the guide 100%. Side note, why reply in the email doesn't appear at the groups.google here, so other viewers can also know the status of the thread? From jldunn2000 at gmail.com Fri Feb 10 03:35:41 2017 From: jldunn2000 at gmail.com (loial) Date: Fri, 10 Feb 2017 00:35:41 -0800 (PST) Subject: Extract sigle file from zip file based on file extension Message-ID: <01c525f1-87bf-4eb5-a0d3-2d60e6876535@googlegroups.com> I need to be able to extract a single file from a .zip file in python. The zip file will contain many files. The file to extract will be the only .csv file in the zip, but the full name of the csv file will not be known. Can this be done in python? From jussi.piitulainen at helsinki.fi Fri Feb 10 03:59:19 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 10 Feb 2017 10:59:19 +0200 Subject: Extract sigle file from zip file based on file extension References: <01c525f1-87bf-4eb5-a0d3-2d60e6876535@googlegroups.com> Message-ID: loial writes: > I need to be able to extract a single file from a .zip file in python. > > The zip file will contain many files. The file to extract will be the > only .csv file in the zip, but the full name of the csv file will not > be known. > > Can this be done in python? Find the one member name that ends with ".csv". If the following assignment crashes, it wasn't true that there is exactly one such. with zipfile.ZipFile(path, "r") as f: [member] = [name for name in f.namelist() if name.endswith(".csv")] # extract member here now that you know its full name From __peter__ at web.de Fri Feb 10 04:03:57 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 10 Feb 2017 10:03:57 +0100 Subject: Extract sigle file from zip file based on file extension References: <01c525f1-87bf-4eb5-a0d3-2d60e6876535@googlegroups.com> Message-ID: loial wrote: > I need to be able to extract a single file from a .zip file in python. > The zip file will contain many files. The file to extract will be the only > .csv file in the zip, but the full name of the csv file will not be known. > > Can this be done in python? See It should not take you more than five minutes to work this out yourself. Hint: have a look at the extract() and namelist() methods. From amit at phpandmore.net Fri Feb 10 04:15:26 2017 From: amit at phpandmore.net (Amit Yaron) Date: Fri, 10 Feb 2017 11:15:26 +0200 Subject: Extract sigle file from zip file based on file extension In-Reply-To: <01c525f1-87bf-4eb5-a0d3-2d60e6876535@googlegroups.com> References: <01c525f1-87bf-4eb5-a0d3-2d60e6876535@googlegroups.com> Message-ID: <589D84AE.6000501@phpandmore.net> On 10/02/17 10:35, loial wrote: > I need to be able to extract a single file from a .zip file in python. > The zip file will contain many files. The file to extract will be the only .csv file in the zip, but the full name of the csv file will not be known. > > Can this be done in python? > Yes, you can use the 'zipfile' module to access the members of a zip file in python. To learn more, click https://docs.python.org/2/library/zipfile.html From uday3prakash at gmail.com Fri Feb 10 04:44:59 2017 From: uday3prakash at gmail.com (udhay prakash pethakamsetty) Date: Fri, 10 Feb 2017 01:44:59 -0800 (PST) Subject: check MS SQL database status, and execute preconfigured jobs Message-ID: <424d6762-c57c-4155-96b8-8ddc57043fd5@googlegroups.com> Hi I am using pyodbc to connect to MS SQL server. I got a result for cursor.execute('SELECT * FROM sys.databases') statement. But, I dont know what fields among them must be considered to ensure that database is up and running and is fine. IN this https://www.mssqltips.com/sqlservertip/1477/methods-to-determine-the-status-of-a-sql-server-database/, example 1 lists all the fields. Post that, I want execute some procedures, I can do them with cursor.execute() in the same way. BUT, as I am new to MS SQL db, can someone let me know which fields must be taken into consideration to ensure that db is up and running fine. From amit at phpandmore.net Fri Feb 10 06:59:45 2017 From: amit at phpandmore.net (Amit Yaron) Date: Fri, 10 Feb 2017 13:59:45 +0200 Subject: int vs. float In-Reply-To: References: Message-ID: <589DAB31.5040400@phpandmore.net> On 10/02/17 04:33, adam14711993 at gmail.com wrote: > Hello, > > My computer programming professor challenged me to figure out a way to manipulate my program to display one error message if the user input is a zero or a negative number, and a separate error message if the user input is a decimal number. My program starts out: > > number_purchases_str = int(input("Enter the number of packages you're buying:")) > number_purchases = int(number_purchases_str) > format(number_purchases, "12,") > > So if the user input is, for example, -9 or 0, the program will result with: Sorry, you must order at least one package." > > What I cannot figure out is how to write it so that if my user input is, for example, 1.5, the program will result with: Sorry, you can only order whole packages. > > I understand that because I am starting out by assigning my number_purchases_str to be an int, when the user enters a float that is a conflict and will crash. > > My professor apparently believes there is a way to accomplish this. Any help or advice would be greatly appreciated. > > Thank you. > The function int(str) raises the exception ValueError if the input string does not represent an integer. Use: try: num=int(str) except ValueError e: print "Error: not an integer" From amit at phpandmore.net Fri Feb 10 06:59:45 2017 From: amit at phpandmore.net (Amit Yaron) Date: Fri, 10 Feb 2017 13:59:45 +0200 Subject: int vs. float In-Reply-To: References: Message-ID: <589DAB31.5040400@phpandmore.net> On 10/02/17 04:33, adam14711993 at gmail.com wrote: > Hello, > > My computer programming professor challenged me to figure out a way to manipulate my program to display one error message if the user input is a zero or a negative number, and a separate error message if the user input is a decimal number. My program starts out: > > number_purchases_str = int(input("Enter the number of packages you're buying:")) > number_purchases = int(number_purchases_str) > format(number_purchases, "12,") > > So if the user input is, for example, -9 or 0, the program will result with: Sorry, you must order at least one package." > > What I cannot figure out is how to write it so that if my user input is, for example, 1.5, the program will result with: Sorry, you can only order whole packages. > > I understand that because I am starting out by assigning my number_purchases_str to be an int, when the user enters a float that is a conflict and will crash. > > My professor apparently believes there is a way to accomplish this. Any help or advice would be greatly appreciated. > > Thank you. > The function int(str) raises the exception ValueError if the input string does not represent an integer. Use: try: num=int(str) except ValueError e: print "Error: not an integer" From Cecil at decebal.nl Fri Feb 10 07:47:40 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Fri, 10 Feb 2017 13:47:40 +0100 Subject: Repeating of posts Message-ID: <87lgte1ac3.fsf@Equus.decebal.nl> I see a lot of post delivered multiple times. Is that a problem at my side, or are others experiencing this also? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From melezhik at gmail.com Fri Feb 10 07:52:02 2017 From: melezhik at gmail.com (melezhik at gmail.com) Date: Fri, 10 Feb 2017 04:52:02 -0800 (PST) Subject: Outthentic - full support for Python Message-ID: <779134e2-520b-45ad-9f3a-40437c1d70bc@googlegroups.com> Hi! Whether you are operations guy, devops or developer from time to time you deal with scripts development for various tasks - handy utilities, automation scripts, monitoring tools, so on. Outthentic - is universal, language independent framework to encourage scripts development in easy and fun manner. At version of 0.2.25 I have added full support for Python. https://github.com/melezhik/outthentic In case you need more information here is short introduction into it - "Outthentic ? quick way to develop users scenarios" - https://sparrowdo.wordpress.com/2017/02/09/outthentic-quick-way-to-develop-user-scenarios/ From od at numericable.fr Fri Feb 10 09:05:01 2017 From: od at numericable.fr (stalker5) Date: Fri, 10 Feb 2017 15:05:01 +0100 Subject: problem with Python 3.6 + PyX Message-ID: Hello everybody ! I'm have a problem with this little program #---d?but--- from pyx import * g = graph.graphxy(width=8, x=graph.axis.linear(min=-15, max=15)) g.plot(graph.data.function("y(x)=sin(x)/x")) g.writeEPSfile("function") g.writePDFfile("function") g.writeSVGfile("function") #--- fin--- Python protest : --begin error message Running script: "/Users/draper/Documents/2016-2017/PYTHON/pyx test.py" Traceback (most recent call last): File "/Users/draper/Documents/2016-2017/PYTHON/pyx test.py", line 5, in g.writeEPSfile("function") File "/usr/local/lib/python3.6/site-packages/pyx/canvas.py", line 50, in wrappedindocument return method(d, file, **write_kwargs) File "/usr/local/lib/python3.6/site-packages/pyx/document.py", line 185, in writeEPSfile pswriter.EPSwriter(self, f, **kwargs) File "/usr/local/lib/python3.6/site-packages/pyx/pswriter.py", line 142, in __init__ page.processPS(pagefile, self, acontext, registry, pagebbox) File "/usr/local/lib/python3.6/site-packages/pyx/document.py", line 132, in processPS self._process("processPS", *args) File "/usr/local/lib/python3.6/site-packages/pyx/document.py", line 78, in _process bbox.set(self.canvas.bbox()) # this bbox is not accurate File "/usr/local/lib/python3.6/site-packages/pyx/graph/graph.py", line 181, in bbox self.finish() File "/usr/local/lib/python3.6/site-packages/pyx/graph/graph.py", line 303, in finish self.doaxes() File "/usr/local/lib/python3.6/site-packages/pyx/graph/graph.py", line 580, in doaxes self.dolayout() File "/usr/local/lib/python3.6/site-packages/pyx/graph/graph.py", line 564, in dolayout self.doaxiscreate(axisname) File "/usr/local/lib/python3.6/site-packages/pyx/graph/graph.py", line 240, in doaxiscreate self.axes[axisname].create() File "/usr/local/lib/python3.6/site-packages/pyx/graph/axis/axis.py", line 591, in create self.canvas = self.axis.create(self.data, self.positioner, self.graphtexrunner, self.errorname) File "/usr/local/lib/python3.6/site-packages/pyx/graph/axis/axis.py", line 250, in create return _regularaxis._create(self, data, positioner, graphtexrunner, self.parter, self.rater, errorname) File "/usr/local/lib/python3.6/site-packages/pyx/graph/axis/axis.py", line 220, in _create variants[0].storedcanvas = layout(variants[0]) File "/usr/local/lib/python3.6/site-packages/pyx/graph/axis/axis.py", line 141, in layout self.painter.paint(canvas, data, self, positioner) File "/usr/local/lib/python3.6/site-packages/pyx/graph/axis/painter.py", line 192, in paint t.temp_labelbox = canvas.texrunner.text_pt(t.temp_x_pt, t.temp_y_pt, t.label, labelattrs) File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1428, in wrapped return f(self, *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1459, in text_pt return self.instance.text_pt(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1278, in text_pt left_pt, right_pt, height_pt, depth_pt = self.do_typeset(expr, self.texmessages_run_default + self.texmessages_run + texmessages) File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1201, in do_typeset self.do_start() File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1344, in do_start super().do_start() File "/usr/local/lib/python3.6/site-packages/pyx/text.py", line 1156, in do_start self.popen = config.Popen(cmd, stdin=config.PIPE, stdout=config.PIPE, stderr=config.STDOUT, bufsize=0) File "/usr/local/lib/python3.6/site-packages/pyx/config.py", line 190, in Popen return subprocess.Popen(cmd, *args, **kwargs) File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 707, in __init__ restore_signals, start_new_session) File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 1326, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'tex' -- end eroor message -- I search some solution, but without success. Anybody have a idea ?? Thx a lot. Have a nice day. -- Olivier From stefan_ml at behnel.de Fri Feb 10 09:06:58 2017 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 10 Feb 2017 15:06:58 +0100 Subject: Python Events in 2017, Need your help. In-Reply-To: <20170109095449.v4ual5czl5mc7xxs@sg1> References: <20170109095449.v4ual5czl5mc7xxs@sg1> Message-ID: Hi! Germany has three major Python events planned this year: - PyCon-Web in M?nchen (May 27-28th) - Python-Camp in K?ln (April 8-9th) - PyCon-DE in Karlsruhe (October, dates TBA). http://pyconweb.org/ https://python-verband.org/informieren/events/pythoncamp-2017 Stefan Stephane Wirtel via PSF-Community schrieb am 09.01.2017 um 10:54: > Dear Community, > > For the PythonFOSDEM [1] on 4th and 5th February in Belgium, I would like > to present some slides with the Python events around the World. Based on > https://python.org/events, I have noted that there are missing events, for > example: > > * PyCon Otto: Italy > * PyCon UK: United Kingdom > * PyCon CA: Canada > * PyCon Ireland: Ireland > * PyCon France: France > > Some of these events are not yet announced and I understand they are in the > second semester, and thus, they don't know the location and the dates, > excepted for PyCon Otto (April). > > In fact, I have noted that we know some big events in the Python community > (for example: PyCon US and EuroPython) but do you know the others events, > maybe the local event, PyCon IE, PyCon UK or PyCon IT. > > I like to know where there is a PyCon or a Django Conf or a PyData Event. > > In fact, I think we can help the Python Community if we submit all the > events in https://python.org/events. > > This page has been created by the PSF and is maintained by some volunteers. > > I know this list of events: > * PyCon Cameroon : 20-23 Jav, Cameroon > * PythonFOSDEM : 4-5 Feb, Belgium > * PyCon Colombia : 10-12 Feb, Colombia > * PyCon Pune : 16-20 Feb, India > * Swiss Python Summit : 17-18 Feb, Switzerland > * IrPyCon : 17-18 Feb, Iran > * PyCon SK : 10-13 Mar, Slovakia > * Django Europe : 3-8 Apr, Italy > * PyCon Otto : 6-9 Apr, Italy > * Python Sudeste : 5-7 Mai, Brazil > * GeoPython : 8-11 May, Switzerland > * PyCon US : 17-26 May, USA > * EuroPython : July, Italy > * PyCon AU : 3-9 Aug, Australia > * PyCon UK : September, United Kingdom > * PyCon CA : November, Canada > * PyCon Ireland : October, Ireland > * PyCon FR : October/November, France > > And you ? > Please, could you check on https://www.python.org/events/ , if you are an > organizer, please add your event. > > If you think there is a missing event, please, send me the info via > [email](mailto:stephane at wirtel.be) or via my [twitter > account](https://twitter.com/matrixise) and I will add it on my slides. > > I would like to present your event. > > Thank you so much for your help. > > Stephane Wirtel > > [1] https://www.python-fosdem.org > From lele at metapensiero.it Fri Feb 10 10:02:38 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Fri, 10 Feb 2017 16:02:38 +0100 Subject: problem with Python 3.6 + PyX References: Message-ID: <87inoirsvl.fsf@metapensiero.it> stalker5 writes: > I'm have a problem with this little program > ... > raise child_exception_type(errno_num, err_msg) > FileNotFoundError: [Errno 2] No such file or directory: 'tex' PyX requires TeX to render textual glyphs into the image. Do you have it installed? If so, is it "reachable" by your $PATH variable? A quick test to re-render my PyX-based images with Python 3.6 didn't reveal any problem... ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From grant.b.edwards at gmail.com Fri Feb 10 10:25:14 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 10 Feb 2017 15:25:14 +0000 (UTC) Subject: Repeating of posts References: <87lgte1ac3.fsf@Equus.decebal.nl> Message-ID: On 2017-02-10, Cecil Westerhof wrote: > I see a lot of post delivered multiple times. Is that a problem at my > side, or are others experiencing this also? I see it too. Same as a a month or two back, but not as much. -- Grant Edwards grant.b.edwards Yow! Do you like "TENDER at VITTLES"? gmail.com From rgaddi at highlandtechnology.invalid Fri Feb 10 11:20:05 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Fri, 10 Feb 2017 08:20:05 -0800 Subject: Repeating of posts In-Reply-To: References: <87lgte1ac3.fsf@Equus.decebal.nl> Message-ID: On 02/10/2017 07:25 AM, Grant Edwards wrote: > On 2017-02-10, Cecil Westerhof wrote: > >> I see a lot of post delivered multiple times. Is that a problem at my >> side, or are others experiencing this also? > > I see it too. Same as a a month or two back, but not as much. > I pinged Wolfgang, who runs eternal-september, and he figured out why comp.lang.python has been getting all those weird echos from f38.n261.nz, hunted down the malefactor and put a stop to it. So that part of the problem should at least be fixed. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From mail at timgolden.me.uk Fri Feb 10 12:10:39 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 10 Feb 2017 17:10:39 +0000 Subject: Repeating of posts In-Reply-To: References: <87lgte1ac3.fsf@Equus.decebal.nl> Message-ID: <491167e8-50e1-341e-eaef-c268a704ff83@timgolden.me.uk> On 10/02/2017 16:20, Rob Gaddi wrote: > On 02/10/2017 07:25 AM, Grant Edwards wrote: >> On 2017-02-10, Cecil Westerhof wrote: >> >>> I see a lot of post delivered multiple times. Is that a problem at my >>> side, or are others experiencing this also? >> >> I see it too. Same as a a month or two back, but not as much. >> > > > I pinged Wolfgang, who runs eternal-september, and he figured out why > comp.lang.python has been getting all those weird echos from > f38.n261.nz, hunted down the malefactor and put a stop to it. So that > part of the problem should at least be fixed. > Thanks very much. We've been filtering them out at the gateway to the mailing list as a practical expedient, but we hadn't got as far as trying to find the source. TJG From oegeeks at gmail.com Fri Feb 10 12:54:23 2017 From: oegeeks at gmail.com (Andreas Paeffgen) Date: Fri, 10 Feb 2017 11:54:23 -0600 Subject: subprocess problem References: Message-ID: Thanks for all the great advice. I tested all the possibilities. So far no luck. If i start the app from a terminal, the solutions work. But not, if i start the app-bundle (There is no connection to a terminal) 1. Copied the path to p.subprocess 2. Used the def wich: python function 3. Used shutil When there is no Terminal, there is no error message / status code message. From eproser at gmail.com Fri Feb 10 13:11:39 2017 From: eproser at gmail.com (eproser at gmail.com) Date: Fri, 10 Feb 2017 10:11:39 -0800 (PST) Subject: os.path.isfile Message-ID: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Hello NG Python 3.5.2 Windows 10 os.path.isfile() no recognise file with double dot? eg. match.cpython-35.pyc Please somebody know something about that? Thank You in advance From pkpearson at nowhere.invalid Fri Feb 10 14:15:36 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 10 Feb 2017 19:15:36 GMT Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> Message-ID: On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron wrote: > On 10/02/17 04:33, adam14711993 at gmail.com wrote: >> Hello, >> >> My computer programming professor challenged me to figure out a way >> to manipulate my program to display one error message if the user >> input is a zero or a negative number, and a separate error message if >> the user input is a decimal number. My program starts out: [snip] > The function int(str) raises the exception ValueError if the input > string does not represent an integer. > > Use: > > try: > num=int(str) > except ValueError e: > print "Error: not an integer" What should happen if the user types "1.0"? To be flexible about this possibility, you could accept the number as a float, and then complain if int(num) != num. -- To email me, substitute nowhere->runbox, invalid->com. From vincent.vande.vyvre at telenet.be Fri Feb 10 14:50:58 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 10 Feb 2017 20:50:58 +0100 Subject: os.path.isfile In-Reply-To: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : > Hello NG > > Python 3.5.2 > > Windows 10 > > os.path.isfile() no recognise file with double dot? > > eg. match.cpython-35.pyc > > Please somebody know something about that? > > Thank You in advance > Interesting, you're right. Python 3.4.3 (default, Nov 17 2016, 01:08:31) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc ') False Why ? From rosuav at gmail.com Fri Feb 10 15:28:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 11 Feb 2017 07:28:05 +1100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: On Sat, Feb 11, 2017 at 6:50 AM, Vincent Vande Vyvre wrote: > Interesting, you're right. > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import os >>>> >>>> os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc >>>> ') > False Works on my system, in 3.4, 3.5, 3.6, and 3.7: Python 3.4.4 (default, Apr 17 2016, 16:02:33) [GCC 5.3.1 20160409] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.path.isfile("__pycache__/foo.cpython-36.pyc") True There's gotta be a difference here somewhere. ChrisA From python at mrabarnett.plus.com Fri Feb 10 15:30:34 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 10 Feb 2017 20:30:34 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: <9129dcc2-ba4a-a5e9-d13a-45ea33358125@mrabarnett.plus.com> On 2017-02-10 19:50, Vincent Vande Vyvre wrote: > Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >> Hello NG >> >> Python 3.5.2 >> >> Windows 10 >> >> os.path.isfile() no recognise file with double dot? >> >> eg. match.cpython-35.pyc >> >> Please somebody know something about that? >> >> Thank You in advance >> > Interesting, you're right. > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import os > >>> > os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc > ') > False > > > Why ? > It looks like the path has a space on the end. Is that true? (I haven't been able to reproduce the problem, incidentally.) From __peter__ at web.de Fri Feb 10 15:36:38 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 10 Feb 2017 21:36:38 +0100 Subject: os.path.isfile References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: Vincent Vande Vyvre wrote: > Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >> Hello NG >> >> Python 3.5.2 >> >> Windows 10 >> >> os.path.isfile() no recognise file with double dot? >> >> eg. match.cpython-35.pyc >> >> Please somebody know something about that? >> >> Thank You in advance >> > Interesting, you're right. > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import os > >>> > os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc > ') > False > > > Why ? No idea in the case of the OP -- on Windows a string literal containing a sequence like "...\t..." is always a good candidate. But you probably sneaked in a trailing newline or space -- i. e. 'twas a pasting mishap ;) From python at lucidity.plus.com Fri Feb 10 15:46:16 2017 From: python at lucidity.plus.com (Erik) Date: Fri, 10 Feb 2017 20:46:16 +0000 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> Message-ID: <5d46a235-c5c9-ce1a-b65c-51d0e7a897e8@lucidity.plus.com> On 10/02/17 19:15, Peter Pearson wrote: > On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron wrote: >> Use: >> >> try: >> num=int(str) >> except ValueError e: >> print "Error: not an integer" > > What should happen if the user types "1.0"? > > To be flexible about this possibility, you could accept the number > as a float, and then complain if int(num) != num. Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 0.99999999999999995 1.0 >>> f = float("0.99999999999999995") >>> i = int(f) >>> i 1 >>> f 1.0 >>> i == f True >>> E. From vincent.vande.vyvre at telenet.be Fri Feb 10 16:03:04 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 10 Feb 2017 22:03:04 +0100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: <9c5e7307-c2a3-5842-49fc-453de1ce7582@telenet.be> Le 10/02/17 ? 21:36, Peter Otten a ?crit : > Vincent Vande Vyvre wrote: > >> Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >>> Hello NG >>> >>> Python 3.5.2 >>> >>> Windows 10 >>> >>> os.path.isfile() no recognise file with double dot? >>> >>> eg. match.cpython-35.pyc >>> >>> Please somebody know something about that? >>> >>> Thank You in advance >>> >> Interesting, you're right. >> >> Python 3.4.3 (default, Nov 17 2016, 01:08:31) >> [GCC 4.8.4] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> import os >> >>> >> > os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc >> ') >> False >> >> >> Why ? > No idea in the case of the OP -- on Windows a string literal containing a > sequence like "...\t..." is always a good candidate. > > But you probably sneaked in a trailing newline or space -- i. e. 'twas a > pasting mishap ;) > > No, it's a copy-paste. No trailing newline. From vincent.vande.vyvre at telenet.be Fri Feb 10 16:09:43 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 10 Feb 2017 22:09:43 +0100 Subject: os.path.isfile In-Reply-To: <9c5e7307-c2a3-5842-49fc-453de1ce7582@telenet.be> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <9c5e7307-c2a3-5842-49fc-453de1ce7582@telenet.be> Message-ID: Le 10/02/17 ? 22:03, Vincent Vande Vyvre a ?crit : > Le 10/02/17 ? 21:36, Peter Otten a ?crit : >> Vincent Vande Vyvre wrote: >> >>> Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >>>> Hello NG >>>> >>>> Python 3.5.2 >>>> >>>> Windows 10 >>>> >>>> os.path.isfile() no recognise file with double dot? >>>> >>>> eg. match.cpython-35.pyc >>>> >>>> Please somebody know something about that? >>>> >>>> Thank You in advance >>>> >>> Interesting, you're right. >>> >>> Python 3.4.3 (default, Nov 17 2016, 01:08:31) >>> [GCC 4.8.4] on linux >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> import os >>> >>> >>> >> os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc >> >>> ') >>> False >>> >>> >>> Why ? >> No idea in the case of the OP -- on Windows a string literal >> containing a >> sequence like "...\t..." is always a good candidate. >> >> But you probably sneaked in a trailing newline or space -- i. e. 'twas a >> pasting mishap ;) >> >> > No, it's a copy-paste. > > No trailing newline. > OooPs, no newline but just a space. I'm using frequently the copy-paste of file name into a terminal and I don't never see that. From od at numericable.fr Fri Feb 10 18:49:25 2017 From: od at numericable.fr (stalker5) Date: Sat, 11 Feb 2017 00:49:25 +0100 Subject: problem with Python 3.6 + PyX In-Reply-To: References: <87inoirsvl.fsf@metapensiero.it> Message-ID: Yes, I have a Tex engine on my system. How can i fix the problem with the PATH ? Even if i execute the .py script directly by python (I mean :not inside a LateX file) pyx need to reach the TeX system ? Is there a config file to fix the PATH ? Where ? Thanks a lot !! Ciao -- Olivier > PyX requires TeX to render textual glyphs into the image. Do you have it > installed? If so, is it "reachable" by your $PATH variable? > > A quick test to re-render my PyX-based images with Python 3.6 didn't reveal > any problem... > > ciao, lele. > From dan at tombstonezero.net Fri Feb 10 21:17:26 2017 From: dan at tombstonezero.net (Dan Sommers) Date: Sat, 11 Feb 2017 02:17:26 -0000 (UTC) Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> <5d46a235-c5c9-ce1a-b65c-51d0e7a897e8@lucidity.plus.com> Message-ID: On Fri, 10 Feb 2017 20:46:16 +0000, Erik wrote: > Python 3.5.2 (default, Nov 17 2016, 17:05:23) > [GCC 5.4.0 20160609] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> 0.99999999999999995 > 1.0 > >>> f = float("0.99999999999999995") > >>> i = int(f) > >>> i > 1 > >>> f > 1.0 > >>> i == f > True > >>> > > E. At least it works both ways: Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linux Type "help", "copyright", "credits" or "license" for more information. >>> s = "33333333333333333333333333333333333333333333333333333333333333333" >>> f = float(s) >>> i = int(s) >>> i 33333333333333333333333333333333333333333333333333333333333333333 >>> f 3.3333333333333333e+64 >>> i == f False From cs at zip.com.au Fri Feb 10 21:23:12 2017 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 11 Feb 2017 13:23:12 +1100 Subject: subprocess problem In-Reply-To: References: Message-ID: <20170211022312.GA24986@cskk.homeip.net> On 10Feb2017 11:54, Andreas Paeffgen wrote: >Thanks for all the great advice. >I tested all the possibilities. So far no luck. If i start the app >from a terminal, the solutions work. But not, if i start the >app-bundle (There is no connection to a terminal) > >1. Copied the path to p.subprocess >2. Used the def wich: python function >3. Used shutil > >When there is no Terminal, there is no error message / status code message. Please show us you code containing the status code check. I would do some debugging inside the function. Consider writing to a special purpose log file, for example like this: def your_function(.......): with open('/path/to/your/logfile.txt', 'a') as logfp: print("PATH=".os.environ['PATH'], file=logfp) p=Popen(.......) p.communicate(...) print("p.returncode=%r" % (p.returncode)) and any other interesting values that occur to you. This is really simple, and doesn't require a terminal or using the logging module. You can 'tail -f /path/to/your/logfile.txt' in another terminal to watch stuff happen as you exercise the program. Cheers, Cameron Simpson From eryksun at gmail.com Fri Feb 10 22:01:23 2017 From: eryksun at gmail.com (eryk sun) Date: Sat, 11 Feb 2017 03:01:23 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <9c5e7307-c2a3-5842-49fc-453de1ce7582@telenet.be> Message-ID: On Fri, Feb 10, 2017 at 9:09 PM, Vincent Vande Vyvre wrote: > Le 10/02/17 ? 22:03, Vincent Vande Vyvre a ?crit : >> Le 10/02/17 ? 21:36, Peter Otten a ?crit : >>> Vincent Vande Vyvre wrote: >>>> Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >>>>> >>>>> Python 3.5.2 >>>>> >>>>> Windows 10 >>>>> >>>>> os.path.isfile() no recognise file with double dot? >>>>> >>>>> eg. match.cpython-35.pyc >>>>> >>>> Interesting, you're right. >>>> >>>> Python 3.4.3 (default, Nov 17 2016, 01:08:31) >>>> [GCC 4.8.4] on linux >>>> Type "help", "copyright", "credits" or "license" for more information. >>>> >>> import os >>>> >>> os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid. >>>> cpython-34.pyc ') >>>> False >>>> >>>> Why ? >>> >>> No idea in the case of the OP -- on Windows a string literal containing a >>> sequence like "...\t..." is always a good candidate. >>> >>> But you probably sneaked in a trailing newline or space -- i. e. 'twas a >>> pasting mishap ;) >>> >> No, it's a copy-paste. >> >> No trailing newline. >> > OooPs, no newline but just a space. It works for me in Windows 10: >>> sys.version_info[:3], sys.getwindowsversion()[:2] ((3, 5, 2), (10, 0)) >>> os.path.isfile(r'__pycache__\match.cpython-35.pyc') True We know the problem isn't a typo with trailing spaces or dots, since Windows strips them from the last path component to be compatible with MS-DOS (which had a weak excuse to cut corners because it was written in 8086 assembly): >>> os.path.isfile(r'__pycache__\match.cpython-35.pyc ... ') True Using a file with a name like that in Windows requires a fully-qualified path prefixed by \\?\ to bypass DOS compatibility processing. From amit at phpandmore.net Sat Feb 11 02:00:22 2017 From: amit at phpandmore.net (Amit Yaron) Date: Sat, 11 Feb 2017 09:00:22 +0200 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> Message-ID: <589EB686.6070308@phpandmore.net> On 10/02/17 21:15, Peter Pearson wrote: > On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron wrote: >> On 10/02/17 04:33, adam14711993 at gmail.com wrote: >>> Hello, >>> >>> My computer programming professor challenged me to figure out a way >>> to manipulate my program to display one error message if the user >>> input is a zero or a negative number, and a separate error message if >>> the user input is a decimal number. My program starts out: > [snip] >> The function int(str) raises the exception ValueError if the input >> string does not represent an integer. >> >> Use: >> >> try: >> num=int(str) >> except ValueError e: >> print "Error: not an integer" > > What should happen if the user types "1.0"? > > To be flexible about this possibility, you could accept the number > as a float, and then complain if int(num) != num. > Another option: Use 'float' instead of 'int'. and check using the method 'is_integer' of floating point numbers: >>> 3.5.is_integer() False >>> 4.0.is_integer() True From amit at phpandmore.net Sat Feb 11 02:00:22 2017 From: amit at phpandmore.net (Amit Yaron) Date: Sat, 11 Feb 2017 09:00:22 +0200 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> Message-ID: <589EB686.6070308@phpandmore.net> On 10/02/17 21:15, Peter Pearson wrote: > On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron wrote: >> On 10/02/17 04:33, adam14711993 at gmail.com wrote: >>> Hello, >>> >>> My computer programming professor challenged me to figure out a way >>> to manipulate my program to display one error message if the user >>> input is a zero or a negative number, and a separate error message if >>> the user input is a decimal number. My program starts out: > [snip] >> The function int(str) raises the exception ValueError if the input >> string does not represent an integer. >> >> Use: >> >> try: >> num=int(str) >> except ValueError e: >> print "Error: not an integer" > > What should happen if the user types "1.0"? > > To be flexible about this possibility, you could accept the number > as a float, and then complain if int(num) != num. > Another option: Use 'float' instead of 'int'. and check using the method 'is_integer' of floating point numbers: >>> 3.5.is_integer() False >>> 4.0.is_integer() True From robertvstepp at gmail.com Sat Feb 11 02:47:32 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 11 Feb 2017 01:47:32 -0600 Subject: int vs. float In-Reply-To: <589EB686.6070308@phpandmore.net> References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> Message-ID: On Sat, Feb 11, 2017 at 1:00 AM, Amit Yaron wrote: > On 10/02/17 21:15, Peter Pearson wrote: >> >> On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron >> wrote: >>> >>> On 10/02/17 04:33, adam14711993 at gmail.com wrote: >>>> My computer programming professor challenged me to figure out a way >>>> to manipulate my program to display one error message if the user >>>> input is a zero or a negative number, and a separate error message if >>>> the user input is a decimal number. My program starts out: [snip] >> What should happen if the user types "1.0"? >> >> To be flexible about this possibility, you could accept the number >> as a float, and then complain if int(num) != num. >> > Another option: > Use 'float' instead of 'int'. and check using the method 'is_integer' of > floating point numbers: > >>>> 3.5.is_integer() > False >>>> 4.0.is_integer() > True According to the OP's professor's challenge, the OP needs to recognize an input of "4.0" as a float and "4" as an integer, and to respond with an error message in the float case, or "decimal number" case as the OP phrased it. Apparently only positive integers are acceptable input; all other inputs should generate an appropriate error message by input type. -- boB From marko at pacujo.net Sat Feb 11 03:24:32 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 11 Feb 2017 10:24:32 +0200 Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> Message-ID: <87a89t879b.fsf@elektro.pacujo.net> boB Stepp : > According to the OP's professor's challenge, the OP needs to recognize > an input of "4.0" as a float and "4" as an integer, and to respond > with an error message in the float case, or "decimal number" case as > the OP phrased it. Apparently only positive integers are acceptable > input; all other inputs should generate an appropriate error message > by input type. Haven't been following the discussion, but this should be simply: ast.literal_eval("...") For example: >>> ast.literal_eval("( 1.0, 3 )").__class__.__name__ 'tuple' Marko From amit at phpandmore.net Sat Feb 11 06:07:21 2017 From: amit at phpandmore.net (Amit Yaron) Date: Sat, 11 Feb 2017 13:07:21 +0200 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> Message-ID: <589EF069.1030503@phpandmore.net> On 11/02/17 09:47, boB Stepp wrote: > On Sat, Feb 11, 2017 at 1:00 AM, Amit Yaron wrote: >> On 10/02/17 21:15, Peter Pearson wrote: >>> >>> On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron >>> wrote: >>>> >>>> On 10/02/17 04:33, adam14711993 at gmail.com wrote: > >>>>> My computer programming professor challenged me to figure out a way >>>>> to manipulate my program to display one error message if the user >>>>> input is a zero or a negative number, and a separate error message if >>>>> the user input is a decimal number. My program starts out: > > [snip] > >>> What should happen if the user types "1.0"? >>> >>> To be flexible about this possibility, you could accept the number >>> as a float, and then complain if int(num) != num. >>> >> Another option: >> Use 'float' instead of 'int'. and check using the method 'is_integer' of >> floating point numbers: >> >>>>> 3.5.is_integer() >> False >>>>> 4.0.is_integer() >> True > > According to the OP's professor's challenge, the OP needs to recognize > an input of "4.0" as a float and "4" as an integer, and to respond > with an error message in the float case, or "decimal number" case as > the OP phrased it. Apparently only positive integers are acceptable > input; all other inputs should generate an appropriate error message > by input type. > > Here's the output of the int function with a string argument: >>> int("4.0") Traceback (most recent call last): File "", line 1, in You can use exception handling to detect numbers that do not match the regular expression for integers, and comparisons to check if a number is negative. From amit at phpandmore.net Sat Feb 11 06:07:21 2017 From: amit at phpandmore.net (Amit Yaron) Date: Sat, 11 Feb 2017 13:07:21 +0200 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> Message-ID: <589EF069.1030503@phpandmore.net> On 11/02/17 09:47, boB Stepp wrote: > On Sat, Feb 11, 2017 at 1:00 AM, Amit Yaron wrote: >> On 10/02/17 21:15, Peter Pearson wrote: >>> >>> On Fri, 10 Feb 2017 13:59:45 +0200, Amit Yaron >>> wrote: >>>> >>>> On 10/02/17 04:33, adam14711993 at gmail.com wrote: > >>>>> My computer programming professor challenged me to figure out a way >>>>> to manipulate my program to display one error message if the user >>>>> input is a zero or a negative number, and a separate error message if >>>>> the user input is a decimal number. My program starts out: > > [snip] > >>> What should happen if the user types "1.0"? >>> >>> To be flexible about this possibility, you could accept the number >>> as a float, and then complain if int(num) != num. >>> >> Another option: >> Use 'float' instead of 'int'. and check using the method 'is_integer' of >> floating point numbers: >> >>>>> 3.5.is_integer() >> False >>>>> 4.0.is_integer() >> True > > According to the OP's professor's challenge, the OP needs to recognize > an input of "4.0" as a float and "4" as an integer, and to respond > with an error message in the float case, or "decimal number" case as > the OP phrased it. Apparently only positive integers are acceptable > input; all other inputs should generate an appropriate error message > by input type. > > Here's the output of the int function with a string argument: >>> int("4.0") Traceback (most recent call last): File "", line 1, in You can use exception handling to detect numbers that do not match the regular expression for integers, and comparisons to check if a number is negative. From saxri89 at gmail.com Sat Feb 11 13:34:52 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 11 Feb 2017 10:34:52 -0800 (PST) Subject: DJANGO image processing Message-ID: <67826482-b7f7-4151-998c-41f9fc3f9c4b@googlegroups.com> I want to create a django app with the base64 help where the users can upload images(specific ".tiff" ext) using DJANGO forms without model and without that images store in my server. and i will the users can be get back new processing image. i have success with encode/decode image with base64 and i can display that image in template using safari because that can display TIFF images. here the code : views.py def convert_to_base64(image_file): base64_string = base64.encodestring(image_file) return "data:image/png;base64,"+base64_string @csrf_exempt def index(request): form = ImageUploadForm(request.POST or None, request.FILES or None) if request.method == "POST" and form.is_valid(): image_file = request.FILES['image'].read() base64_string = convert_to_base64(image_file) file = base64_string.decode('utf8') return render_to_response("blog/success.html", {"image_file":file}) return render_to_response('blog/calc.html', {'form': form}, RequestContext(request)) my problem is now after the base64 i cant use that image for image processing(work fine in standalone python script) and i take two error : RuntimeError at / not a string in the browser and that message in command line : "GET / HTTP/1.1" 200 346 ERROR 4: `II*' does not exist in the file system, and is not recognised as a supported dataset name. here the code : @csrf_exempt def index(request): form = ImageUploadForm(request.POST or None, request.FILES or None) if request.method == "POST" and form.is_valid(): image_file = request.FILES['image'].read() base64_string = convert_to_base64(image_file) file = base64_string.decode('utf8') file_like = cStringIO.StringIO(image_file) flow_direction_uri = "output.tif" routing.flow_direction_d_inf(file_like, flow_direction_uri) return render_to_response("blog/success.html", {"image_file":file}) return render_to_response('blog/calc.html', {'form': form}, RequestContext(request)) and if i use file = base64_string.decode('utf8') as input in my function then i take that error : AttributeError at / 'NoneType' object has no attribute 'GetRasterBand' and in command line i take to like that : AIAAgACAAIAAgACAAIAAgACAAIAAgACA AIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAA gACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACA AIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAA with to many lines. From python at lucidity.plus.com Sat Feb 11 17:33:53 2017 From: python at lucidity.plus.com (Erik) Date: Sat, 11 Feb 2017 22:33:53 +0000 Subject: int vs. float In-Reply-To: References: <589DAB31.5040400@phpandmore.net> <5d46a235-c5c9-ce1a-b65c-51d0e7a897e8@lucidity.plus.com> Message-ID: <44733eea-9575-8a2e-6526-6c5e971926a7@lucidity.plus.com> [Dan, this isn't "aimed" at you personally, it's just a follow-up on the general point I am (and I think you are also) making] On 11/02/17 02:17, Dan Sommers wrote: > At least it works both ways: > > Python 3.5.3 (default, Jan 19 2017, 14:11:04) > [GCC 6.3.0 20170118] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> s = "33333333333333333333333333333333333333333333333333333333333333333" >>>> f = float(s) >>>> i = int(s) Yes, but Peter said "accept the number as a float, and then complain if int(num) != num.", so he meant use "i = int(f)" (i.e., convert the float to an int, not the original string). Anyway, that's just as bad, but in a different way ;) Using your example: Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> s = "33333333333333333333333333333333333333333333333333333333333333333" >>> f = float(s) >>> f 3.3333333333333333e+64 >>> i = int(f) >>> i 33333333333333333069679542094544616940918707231410151788522766336 >>> i == f True >>> That's poor if "i" ends up being the value actually used later when the input is processed - it's NOT the value the user entered! I imagine that this particular use-case is not trying to teach the OP about float/int issues, but about how to parse numerical strings from user-input (so these extreme cases are not expected to be part of the input range). I might be wrong, though. However, these sorts of issues are a very important consideration for people writing real-world code where the input might be coming from an external source such as a database or output from a language which uses fixed-point or some other precise representation of decimals (e.g., COBOL). Whenever you convert something that may not be an integer in the appropriate range to a float (using IEEE 754, at least) you must be aware that you may lose precision. The answer in this case is to do something similar to what Peter suggested, but to use Python's 'decimal' type as the intermediate representation rather than the float() type: Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import Decimal as dec >>> s = "0.99999999999999995" >>> d = dec(s) >>> i = int(d) >>> d Decimal('0.99999999999999995') >>> i 0 >>> i == d False >>> s = "33333333333333333333333333333333333333333333333333333333333333333" >>> d = dec(s) >>> i = int(d) >>> d Decimal('33333333333333333333333333333333333333333333333333333333333333333') >>> i 33333333333333333333333333333333333333333333333333333333333333333 >>> i == d True No surprises there ;) E. From steve+python at pearwood.info Sat Feb 11 22:27:31 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 14:27:31 +1100 Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> <87a89t879b.fsf@elektro.pacujo.net> Message-ID: <589fd623$0$1590$c3e8da3$5496439d@news.astraweb.com> On Sat, 11 Feb 2017 07:24 pm, Marko Rauhamaa wrote: > boB Stepp : > >> According to the OP's professor's challenge, the OP needs to recognize >> an input of "4.0" as a float and "4" as an integer, and to respond >> with an error message in the float case, or "decimal number" case as >> the OP phrased it. Apparently only positive integers are acceptable >> input; all other inputs should generate an appropriate error message >> by input type. > > Haven't been following the discussion, but this should be simply: > > ast.literal_eval("...") > > For example: > > >>> ast.literal_eval("( 1.0, 3 )").__class__.__name__ > 'tuple' In what way does returning a tuple match the requirement "return an int or a float or generate an error message"? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Feb 11 22:34:11 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 Feb 2017 14:34:11 +1100 Subject: int vs. float In-Reply-To: <589fd623$0$1590$c3e8da3$5496439d@news.astraweb.com> References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> <87a89t879b.fsf@elektro.pacujo.net> <589fd623$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 12, 2017 at 2:27 PM, Steve D'Aprano wrote: >> >> For example: >> >> >>> ast.literal_eval("( 1.0, 3 )").__class__.__name__ >> 'tuple' > > > In what way does returning a tuple match the requirement "return an int or a > float or generate an error message"? Easy. You just use a version of Python in which tuple is a subclass of int. ChrisA From steve+python at pearwood.info Sat Feb 11 22:38:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 14:38:27 +1100 Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> Message-ID: <589fd8b4$0$1603$c3e8da3$5496439d@news.astraweb.com> On Sat, 11 Feb 2017 06:00 pm, Amit Yaron wrote: > Another option: > Use 'float' instead of 'int'. and check using the method 'is_integer' > of floating point numbers: > > >>> 3.5.is_integer() > False > >>> 4.0.is_integer() > True A bad option... py> float('12345678901234567') 1.2345678901234568e+16 Notice the last digit of the mantissa? Beyond a certain value, *all* floats are integer, because floats don't have enough precision to include a fractional portion. py> float('12345678901234567.12345').is_integer() True The safest way to treat this is to attempt to convert to an int, if that succeeds return the int; if it fails, try to convert to a float, and if that succeeds, return the float. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Feb 11 22:48:17 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 14:48:17 +1100 Subject: os.path.isfile References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: <589fdb02$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sat, 11 Feb 2017 06:50 am, Vincent Vande Vyvre wrote: > Le 10/02/17 ? 19:11, eproser at gmail.com a ?crit : >> Hello NG >> >> Python 3.5.2 >> >> Windows 10 >> >> os.path.isfile() no recognise file with double dot? >> >> eg. match.cpython-35.pyc >> >> Please somebody know something about that? >> >> Thank You in advance >> > Interesting, you're right. > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import os > >>> > os.path.isfile('/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc > ') > False > > > Why ? How do you know that file exists? Why choose such a long file name where you risk introducing spaces or newlines due to word-wrapping? I cannot replicate your result using Python 3.5 on Linux: py> path = '/tmp/spam.eggs.cheese.txt' py> with open(path, 'w') as f: ... f.write('something') ... 9 py> import os py> print(os.path.isfile(path)) True I expect that you have a space at the end of the file name: '/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc ' instead of '/home/vincent/oqapy-3/trunk/__pycache__/grid.cpython-34.pyc' -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Feb 11 22:52:09 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 14:52:09 +1100 Subject: os.path.isfile References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> Message-ID: <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sat, 11 Feb 2017 05:11 am, eproser at gmail.com wrote: > Hello NG > > Python 3.5.2 > > Windows 10 > > os.path.isfile() no recognise file with double dot? > > eg. match.cpython-35.pyc I doubt that very much. I expect you are probably writing something like this: path = 'My Documents\testing\path\match.cpython-35.pyc' os.path.isfile(path) and not realising that you have a TAB character in your string from the \t, or something similar. In Python, you should always use forward slashes for paths, even on Windows. path = 'My Documents/testing/path/match.cpython-35.pyc' Try printing the repr() of your path. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Feb 11 23:09:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 15:09:08 +1100 Subject: Rename file without overwriting existing files References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <589fdfe5$0$1604$c3e8da3$5496439d@news.astraweb.com> On Fri, 10 Feb 2017 12:07 am, eryk sun wrote: > On Thu, Feb 9, 2017 at 11:46 AM, Steve D'Aprano > wrote: >> >> So to summarise, os.rename(source, destination): >> >> - is atomic on POSIX systems, if source and destination are both on the >> same file system; >> - may not be atomic on Windows? >> - may over-write an existing destination on POSIX systems, but not on >> Windows; >> - and it doesn't work across file systems. > > On Windows in 2.7 and prior to 3.3, os.rename will silently copy and > delete when the destination isn't on the same volume. Will the copy overwrite an existing file? > It may even > silently leave the original file in place in some cases -- e.g. when > the file is read-only and the user isn't allowed to modify the file > attributes. > > If the destination is on the same volume, renaming should be atomic > via the system calls NtOpenFile and NtSetInformationFile. Ultimately > it depends on the file system implementation of > IRP_MJ_SET_INFORMATION, FileRenameInformation [1]. > >> The link/unlink trick: >> - avoids over-writing existing files on POSIX systems at least; >> - but maybe not Windows? > > This works for renaming files on Windows as long as the file system > supports hard links (e.g. NTFS). It's not documented on MSDN, but > WinAPI CreateHardLink is implemented by calling NtSetInformationFile > to set the FileLinkInformation, with ReplaceIfExists set to FALSE, so > it fails if the destination exists. Note that this does not allow > renaming directories. See the note for FileLinkInformation [1]; NTFS > doesn't allow directory hard links. But why bother with this 'trick' > on Windows? I don't know, that's why I'm asking for guidance. I don't have a Windows system to test on. On Windows, how would you implement a file rename (potentially across file system boundaries) which will not overwrite existing files? Just by calling os.rename()? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Sat Feb 11 23:20:05 2017 From: eryksun at gmail.com (eryk sun) Date: Sun, 12 Feb 2017 04:20:05 +0000 Subject: os.path.isfile In-Reply-To: <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 12, 2017 at 3:52 AM, Steve D'Aprano wrote: > In Python, you should always use forward slashes for paths, even on Windows. There are cases where slash doesn't work (e.g. some command lines; \\?\ prefixed paths; registry subkey paths), so it's simpler to follow a rule to always convert paths to backslash on Windows, using os.path.normpath or pathlib.Path. From rosuav at gmail.com Sat Feb 11 23:29:20 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 Feb 2017 15:29:20 +1100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 12, 2017 at 3:20 PM, eryk sun wrote: > On Sun, Feb 12, 2017 at 3:52 AM, Steve D'Aprano > wrote: >> In Python, you should always use forward slashes for paths, even on Windows. > > There are cases where slash doesn't work (e.g. some command lines; > \\?\ prefixed paths; registry subkey paths), so it's simpler to follow > a rule to always convert paths to backslash on Windows, using > os.path.normpath or pathlib.Path. Registry subkeys aren't paths, and the other two cases are extremely narrow. Convert slashes to backslashes ONLY in the cases where you actually need to. ChrisA From steve+python at pearwood.info Sat Feb 11 23:50:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 15:50:18 +1100 Subject: int vs. float References: <589DAB31.5040400@phpandmore.net> <589EB686.6070308@phpandmore.net> <87a89t879b.fsf@elektro.pacujo.net> <589fd623$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <589fe98b$0$1586$c3e8da3$5496439d@news.astraweb.com> On Sun, 12 Feb 2017 02:34 pm, Chris Angelico wrote: > On Sun, Feb 12, 2017 at 2:27 PM, Steve D'Aprano > wrote: >>> >>> For example: >>> >>> >>> ast.literal_eval("( 1.0, 3 )").__class__.__name__ >>> 'tuple' >> >> >> In what way does returning a tuple match the requirement "return an int >> or a float or generate an error message"? > > Easy. You just use a version of Python in which tuple is a subclass of > int. Wouldn't a subclass of float be more appropriate? After all, floats have a mantissa and an exponent, which kinda makes them a tuple... *tongue firmly in cheek* -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Feb 11 23:53:48 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 12 Feb 2017 15:53:48 +1100 Subject: os.path.isfile References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> On Sun, 12 Feb 2017 03:20 pm, eryk sun wrote: > On Sun, Feb 12, 2017 at 3:52 AM, Steve D'Aprano > wrote: >> In Python, you should always use forward slashes for paths, even on >> Windows. > > There are cases where slash doesn't work (e.g. some command lines; > \\?\ prefixed paths; registry subkey paths), so it's simpler to follow > a rule to always convert paths to backslash on Windows, using > os.path.normpath or pathlib.Path. Okay, that makes sense. But we agree that when writing paths as string literals, you should (nearly) always use forward slashes in Python. Why not raw strings? Because raw strings aren't actually fully raw, and consequently they don't allow the string to end with a backslash: py> s = r'documents\' File "", line 1 s = r'documents\' ^ SyntaxError: EOL while scanning string literal (I still don't understand why this isn't just treated as a bug in raw string parsing and fixed...) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From pbonfanti2 at inwindNO.it Sun Feb 12 15:19:36 2017 From: pbonfanti2 at inwindNO.it (Paolo) Date: Sun, 12 Feb 2017 21:19:36 +0100 Subject: scrivere file Message-ID: Buonasera, ? da un p? che sto cercando come esercizio di scrivere un file al contrario. Mi spiego meglio ho un file con N righe e vorrei scriverne un altro con gli stessi dati ma la 1? riga deve diventare l' ultima. Es. file di partenza riga1 riga2 riga3 riga4 file riscritto riga4 riga3 riga2 riga1 Grazie anticipamente per i consigli -- Paolo From auriocus at gmx.de Sun Feb 12 15:38:44 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 12 Feb 2017 21:38:44 +0100 Subject: scrivere file In-Reply-To: References: Message-ID: Am 12.02.17 um 21:19 schrieb Paolo: > Buonasera, ? da un p? che sto cercando come esercizio di scrivere un > file al contrario. > Mi spiego meglio ho un file con N righe e vorrei scriverne un altro con > gli stessi dati ma la 1? riga deve diventare l' ultima. > Es. file di partenza > riga1 > riga2 > riga3 > riga4 > > file riscritto > > riga4 > riga3 > riga2 > riga1 > > Grazie anticipamente per i consigli tac file2 does that. You need to read the whole file into memory and print the lines in reversed order. BTW, this is an English group. Your question is clearly stated and therefore can be understood with minimal knowledge of Italian, but be prepared to discuss it in English in the following. Christian From bc at freeuk.com Sun Feb 12 16:07:36 2017 From: bc at freeuk.com (BartC) Date: Sun, 12 Feb 2017 21:07:36 +0000 Subject: scrivere file In-Reply-To: References: Message-ID: On 12/02/2017 20:38, Christian Gollwitzer wrote: > Am 12.02.17 um 21:19 schrieb Paolo: >> Buonasera, ? da un p? che sto cercando come esercizio di scrivere un >> file al contrario. >> Mi spiego meglio ho un file con N righe e vorrei scriverne un altro con >> gli stessi dati ma la 1? riga deve diventare l' ultima. >> Es. file di partenza >> riga1 >> riga2 >> riga3 >> riga4 >> >> file riscritto >> >> riga4 >> riga3 >> riga2 >> riga1 >> >> Grazie anticipamente per i consigli > > > tac file2 > > does that. In what language? In Python, I had to do something like this: f=open("file1","r") lines=f.readlines() lines.reverse() f.close() g=open("file2","w") g.writelines(lines) g.close() -- Bartc From auriocus at gmx.de Sun Feb 12 16:46:56 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 12 Feb 2017 22:46:56 +0100 Subject: scrivere file In-Reply-To: References: Message-ID: Am 12.02.17 um 22:07 schrieb BartC: > On 12/02/2017 20:38, Christian Gollwitzer wrote: >> Am 12.02.17 um 21:19 schrieb Paolo: >>> Buonasera, ? da un p? che sto cercando come esercizio di scrivere un >>> file al contrario. >>> Mi spiego meglio ho un file con N righe e vorrei scriverne un altro con >>> gli stessi dati ma la 1? riga deve diventare l' ultima. >> >> tac file2 >> >> does that. > > In what language? Shell script. > In Python, I had to do something like this: > Thank you for doing his homework. Christian From bc at freeuk.com Sun Feb 12 17:44:07 2017 From: bc at freeuk.com (BartC) Date: Sun, 12 Feb 2017 22:44:07 +0000 Subject: scrivere file In-Reply-To: References: Message-ID: On 12/02/2017 21:46, Christian Gollwitzer wrote: > Am 12.02.17 um 22:07 schrieb BartC: >> On 12/02/2017 20:38, Christian Gollwitzer wrote: >>> Am 12.02.17 um 21:19 schrieb Paolo: >>>> Buonasera, ? da un p? che sto cercando come esercizio di scrivere un >>>> file al contrario. >>>> Mi spiego meglio ho un file con N righe e vorrei scriverne un altro con >>>> gli stessi dati ma la 1? riga deve diventare l' ultima. >>> >>> tac file2 >>> >>> does that. >> >> In what language? > > Shell script. > >> In Python, I had to do something like this: >> > > Thank you for doing his homework. 'google python reverse a file' Sorry, have I done it again...? From python at lucidity.plus.com Sun Feb 12 18:56:45 2017 From: python at lucidity.plus.com (Erik) Date: Sun, 12 Feb 2017 23:56:45 +0000 Subject: os.path.isfile In-Reply-To: <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> On 12/02/17 04:53, Steve D'Aprano wrote: > py> s = r'documents\' > File "", line 1 > s = r'documents\' > ^ > SyntaxError: EOL while scanning string literal > > > (I still don't understand why this isn't just treated as a bug in raw string > parsing and fixed...) I would imagine that it's something to do with the way the parser works in edge cases such as those below where it struggles to determine how many string tokens there are (is there a PEP that describes how this could be easily fixed?) Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> r'sp\'am' "ha\m" "sp\\'amha\\m" >>> r'sp\'am\' "ha\m" File "", line 1 r'sp\'am\' "ha\m" ^ SyntaxError: EOL while scanning string literal >>> r'sp\'am\' "ha\m' 'sp\\\'am\\\' "ha\\m' >>> Actually, while contriving those examples, I noticed that sometimes when using string literal concatenation, the 'rawness' of the initial string is sometimes applied to the following string and sometimes not: >>> "hello \the" r"worl\d" 'hello \theworl\\d' Makes sense - the initial string is not raw, the concatenated string is. >>> r"hello \the" "worl\d" 'hello \\theworl\\d' Slightly surprising. The concatenated string adopts the initial string's 'rawness'. >>> "hello \the" r"worl\d" "\t" 'hello \theworl\\d\t' The initial string is not raw, the following string is. The string following _that_ becomes raw too. >>> r"hello \the" "worl\d" "\t" 'hello \\theworl\\d\t' The initial string is raw. The following string adopts that (same as the second example), but the _next_ string does not! >>> r"hello \the" "worl\d" r"\t" 'hello \\theworl\\d\\t' ... and this example is the same as before, but makes the third string "raw" again by explicitly declaring it as such. Presumably (I haven't checked), this also applies to u-strings and f-strings - is this a documented and known "wart"/edge-case or is it something that should be defined and fixed? E. From python at lucidity.plus.com Sun Feb 12 19:08:29 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 13 Feb 2017 00:08:29 +0000 Subject: os.path.isfile In-Reply-To: <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: <6ad4040b-e5ad-b5b2-92db-19fad066f5d0@lucidity.plus.com> On 12/02/17 23:56, Erik wrote: >>>> r"hello \the" "worl\d" > 'hello \\theworl\\d' > > Slightly surprising. The concatenated string adopts the initial string's > 'rawness'. > >>>> "hello \the" r"worl\d" "\t" > 'hello \theworl\\d\t' > > The initial string is not raw, the following string is. The string > following _that_ becomes raw too. That's clearly wrong - I mean the final string does _not_ become raw even though it follows a raw string (my point was that the previous example shows a non-raw string becoming raw because it follows a raw string). E. From rosuav at gmail.com Sun Feb 12 19:11:19 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 13 Feb 2017 11:11:19 +1100 Subject: os.path.isfile In-Reply-To: <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On Mon, Feb 13, 2017 at 10:56 AM, Erik wrote: > Actually, while contriving those examples, I noticed that sometimes when > using string literal concatenation, the 'rawness' of the initial string is > sometimes applied to the following string and sometimes not: > >>>> "hello \the" r"worl\d" > 'hello \theworl\\d' > > Makes sense - the initial string is not raw, the concatenated string is. > >>>> r"hello \the" "worl\d" > 'hello \\theworl\\d' > > Slightly surprising. The concatenated string adopts the initial string's > 'rawness'. > >>>> "hello \the" r"worl\d" "\t" > 'hello \theworl\\d\t' > > The initial string is not raw, the following string is. The string following > _that_ becomes raw too. > >>>> r"hello \the" "worl\d" "\t" > 'hello \\theworl\\d\t' > > The initial string is raw. The following string adopts that (same as the > second example), but the _next_ string does not! > >>>> r"hello \the" "worl\d" r"\t" > 'hello \\theworl\\d\\t' > > ... and this example is the same as before, but makes the third string "raw" > again by explicitly declaring it as such. > > Presumably (I haven't checked), this also applies to u-strings and f-strings > - is this a documented and known "wart"/edge-case or is it something that > should be defined and fixed? Firstly, be aware that there's no such thing as a "raw string" - what you have is a "raw string literal". It's a purely syntactic feature. The repr of a string indicates the contents of the string object, and raw literals and non-raw literals can produce the same resulting string objects. The string "\t" gets shown in the repr as "\t". It is a string consisting of one character, U+0009, a tab. The string r"\t" is shown as "\\t" and consists of two characters, REVERSE SOLIDUS and LATIN SMALL LETTER T. That might be why you think there's confusing stuff happening :) ChrisA From rosuav at gmail.com Sun Feb 12 19:13:35 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 13 Feb 2017 11:13:35 +1100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On Mon, Feb 13, 2017 at 11:11 AM, Chris Angelico wrote: > The string "\t" gets shown in the repr as "\t". It is a string > consisting of one character, U+0009, a tab. The string r"\t" is shown > as "\\t" and consists of two characters, REVERSE SOLIDUS and LATIN > SMALL LETTER T. That might be why you think there's confusing stuff > happening :) Oh, and the other trap you can fall into is the reverse of that: >>> "worl\d" 'worl\\d' This one actually triggers a warning in sufficiently-recent Pythons: $ python3 -Wall Python 3.7.0a0 (default:cebc9c7ad195, Jan 24 2017, 06:55:19) [GCC 6.2.0 20161027] on linux Type "help", "copyright", "credits" or "license" for more information. >>> "worl\d" :1: DeprecationWarning: invalid escape sequence \d 'worl\\d' A future Python may define \d to mean something else, or may trigger an error on this. ChrisA From python at lucidity.plus.com Sun Feb 12 19:23:42 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 13 Feb 2017 00:23:42 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: <61fb4c64-fc81-38d4-de34-6d53f71cd942@lucidity.plus.com> On 13/02/17 00:11, Chris Angelico wrote: > Firstly, be aware that there's no such thing as a "raw string" - what > you have is a "raw string literal". It's a purely syntactic feature. I do understand that. When I said "is raw"/"rawness", I am talking about what the _parser_ is doing. I don't think there's a "raw string object" ;) > The string "\t" gets shown in the repr as "\t". It is a string > consisting of one character, U+0009, a tab. The string r"\t" is shown > as "\\t" and consists of two characters, REVERSE SOLIDUS and LATIN > SMALL LETTER T. That might be why you think there's confusing stuff > happening :) OK, so please explain one of my examples: >>>>> r"hello \the" "worl\d" "\t" >> 'hello \\theworl\\d\t' >> >> The initial string is raw. The following string adopts that (same as the >> second example), but the _next_ string does not! Why is the first string token parsed as a "raw" string, the second string token also parsed as a "raw" string (without the 'r' prefix), but the third string token NOT? That's what I'm trying to point out. E. From python at lucidity.plus.com Sun Feb 12 19:29:46 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 13 Feb 2017 00:29:46 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On 13/02/17 00:13, Chris Angelico wrote: > On Mon, Feb 13, 2017 at 11:11 AM, Chris Angelico wrote: >> The string "\t" gets shown in the repr as "\t". It is a string >> consisting of one character, U+0009, a tab. The string r"\t" is shown >> as "\\t" and consists of two characters, REVERSE SOLIDUS and LATIN >> SMALL LETTER T. That might be why you think there's confusing stuff >> happening :) > > Oh, and the other trap you can fall into is the reverse of that: > >>>> "worl\d" > 'worl\\d' > > This one actually triggers a warning in sufficiently-recent Pythons: Fair point, but you're going off at a tangent. I just stuck a backslash on a random letter to see which string tokens were/were not being treated as "raw" by the parser. Next time I'll use \v or something. You're focusing on something that is beside the point I'm trying to make ;) E. From python at lucidity.plus.com Sun Feb 12 19:34:27 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 13 Feb 2017 00:34:27 +0000 Subject: os.path.isfile In-Reply-To: <61fb4c64-fc81-38d4-de34-6d53f71cd942@lucidity.plus.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> <61fb4c64-fc81-38d4-de34-6d53f71cd942@lucidity.plus.com> Message-ID: <20e14a52-5358-3bcd-0730-d5df3327a3af@lucidity.plus.com> On 13/02/17 00:23, Erik wrote: >>>>>> r"hello \the" "worl\d" "\t" >>> 'hello \\theworl\\d\t' >>> >>> The initial string is raw. The following string adopts that (same as the >>> second example), but the _next_ string does not! > > Why is the first string token parsed as a "raw" string, the second > string token also parsed as a "raw" string (without the 'r' prefix), but > the third string token NOT? > > That's what I'm trying to point out. OK, I get it now - because '\d' is not a valid escape sequence, then even in a non-raw string literal, the '\' is treated as a literal backslash character (not an escape). So, the second string token is NOT being treated as "raw", it just looks that way from the repr (and as you point out, a newer version of the parser which explicitly complains about invalid escape sequences removes that ambiguity). Cool ;) E. From rosuav at gmail.com Sun Feb 12 19:34:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 13 Feb 2017 11:34:48 +1100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On Mon, Feb 13, 2017 at 11:29 AM, Erik wrote: > On 13/02/17 00:13, Chris Angelico wrote: >> >> On Mon, Feb 13, 2017 at 11:11 AM, Chris Angelico wrote: >>> >>> The string "\t" gets shown in the repr as "\t". It is a string >>> consisting of one character, U+0009, a tab. The string r"\t" is shown >>> as "\\t" and consists of two characters, REVERSE SOLIDUS and LATIN >>> SMALL LETTER T. That might be why you think there's confusing stuff >>> happening :) >> >> >> Oh, and the other trap you can fall into is the reverse of that: >> >>>>> "worl\d" >> >> 'worl\\d' >> >> This one actually triggers a warning in sufficiently-recent Pythons: > > > Fair point, but you're going off at a tangent. I just stuck a backslash on a > random letter to see which string tokens were/were not being treated as > "raw" by the parser. Next time I'll use \v or something. You're focusing on > something that is beside the point I'm trying to make ;) Except that I'm not. Here, look: OK, so please explain one of my examples: >>>>> r"hello \the" "worl\d" "\t" >> 'hello \\theworl\\d\t' >> >> The initial string is raw. The following string adopts that (same as the >> second example), but the _next_ string does not! Why is the first string token parsed as a "raw" string, the second string token also parsed as a "raw" string (without the 'r' prefix), but the third string token NOT? >>> r"hello \the" "worl\d" "\t" 'hello \\theworl\\d\t' >>> r"hello \the" "worl\t" "\d" 'hello \\theworl\t\\d' >>> "hello \the" "worl\d" "\t" 'hello \theworl\\d\t' >>> "hello \the" "worl\t" "\d" 'hello \theworl\t\\d' The unit "\t" always means U+0009, even if it's following a raw string literal; and the unit "\d" always means "\\d", regardless of the rawness of any of the literals involved. The thing that's biting you here is that unrecognized escapes get rendered as backslash followed by letter, which is why that now produces a warning. ChrisA From rosuav at gmail.com Sun Feb 12 19:35:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 13 Feb 2017 11:35:13 +1100 Subject: os.path.isfile In-Reply-To: <20e14a52-5358-3bcd-0730-d5df3327a3af@lucidity.plus.com> References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> <61fb4c64-fc81-38d4-de34-6d53f71cd942@lucidity.plus.com> <20e14a52-5358-3bcd-0730-d5df3327a3af@lucidity.plus.com> Message-ID: On Mon, Feb 13, 2017 at 11:34 AM, Erik wrote: > OK, I get it now - because '\d' is not a valid escape sequence, then even in > a non-raw string literal, the '\' is treated as a literal backslash > character (not an escape). > > So, the second string token is NOT being treated as "raw", it just looks > that way from the repr (and as you point out, a newer version of the parser > which explicitly complains about invalid escape sequences removes that > ambiguity). > Correct. Our emails crossed in delivery. You've nailed it. ChrisA From python at lucidity.plus.com Sun Feb 12 19:40:44 2017 From: python at lucidity.plus.com (Erik) Date: Mon, 13 Feb 2017 00:40:44 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On 13/02/17 00:34, Chris Angelico wrote: > The unit "\t" always means U+0009, even if it's following a raw string > literal; and the unit "\d" always means "\\d", regardless of the > rawness of any of the literals involved. The thing that's biting you > here is that unrecognized escapes get rendered as backslash followed > by letter, which is why that now produces a warning. Yes, and I'm with you (we just cross-posted). FWIW, if you'd have written the above as your first response I wouldn't have argued ;) You alluded to it, for sure ... :D E. From rosuav at gmail.com Sun Feb 12 19:43:27 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 13 Feb 2017 11:43:27 +1100 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: On Mon, Feb 13, 2017 at 11:40 AM, Erik wrote: > FWIW, if you'd have written the above as your first response I wouldn't have > argued ;) You alluded to it, for sure ... :D Nothing wrong with respectfully arguing. It's one of the best ways to zero in on the truth :) ChrisA From steve+python at pearwood.info Sun Feb 12 20:58:46 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 13 Feb 2017 12:58:46 +1100 Subject: os.path.isfile References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> <589fea5d$0$1586$c3e8da3$5496439d@news.astraweb.com> <2e027760-ac15-943f-3705-a7a6a7a80ae9@lucidity.plus.com> Message-ID: <58a112d8$0$1617$c3e8da3$5496439d@news.astraweb.com> On Mon, 13 Feb 2017 11:43 am, Chris Angelico wrote: > On Mon, Feb 13, 2017 at 11:40 AM, Erik wrote: >> FWIW, if you'd have written the above as your first response I wouldn't >> have argued ;) You alluded to it, for sure ... :D > > Nothing wrong with respectfully arguing. It's one of the best ways to > zero in on the truth :) Is this the right room for an argument? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Sun Feb 12 22:31:10 2017 From: eryksun at gmail.com (eryk sun) Date: Mon, 13 Feb 2017 03:31:10 +0000 Subject: Rename file without overwriting existing files In-Reply-To: <589fdfe5$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <588ea9c8$0$1605$c3e8da3$5496439d@news.astraweb.com> <20170130043303.GA78047@cskk.homeip.net> <588f127e$0$1586$c3e8da3$5496439d@news.astraweb.com> <589c5681$0$1606$c3e8da3$5496439d@news.astraweb.com> <589fdfe5$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 12, 2017 at 4:09 AM, Steve D'Aprano wrote: > On Fri, 10 Feb 2017 12:07 am, eryk sun wrote: > >> On Thu, Feb 9, 2017 at 11:46 AM, Steve D'Aprano >> wrote: >>> >>> So to summarise, os.rename(source, destination): >>> >>> - is atomic on POSIX systems, if source and destination are both on the >>> same file system; >>> - may not be atomic on Windows? >>> - may over-write an existing destination on POSIX systems, but not on >>> Windows; >>> - and it doesn't work across file systems. >> >> On Windows in 2.7 and prior to 3.3, os.rename will silently copy and >> delete when the destination isn't on the same volume. > > Will the copy overwrite an existing file? 2.7/3.2 calls MoveFile. This is effectively the same as (but not necessarily implemented by) MoveFileEx with the flag MOVEFILE_COPY_ALLOWED. It will not replace an existing file whether or not the target is on the same volume. For a cross-volume move, it's effectively a CopyFile followed by DeleteFile. If deleting the source file fails, it tries to reset the file attributes and retries the delete. > On Windows, how would you implement a file rename (potentially across file > system boundaries) which will not overwrite existing files? Just by calling > os.rename()? I'd prefer to always call shutil.move on Windows. In 2.7 the os.rename call in shutil.move will copy and delete for a cross-volume move of a file, but fail for a directory. In the latter case shutil.move falls back on shutil.copytree and shutil.rmtree. The OS won't allow replacing an existing file with a directory, so that's not a problem. In 3.3+ os.rename doesn't move files across volumes on any platform -- I think. In this case the default copy2 function used by shutil.move is a problem for everyone. Ideally we want "x" mode instead of "w" mode for creating the destination file. It raises a FileExistsError. 3.5+ allows using a custom copy function to implement this. From eryksun at gmail.com Sun Feb 12 23:48:23 2017 From: eryksun at gmail.com (eryk sun) Date: Mon, 13 Feb 2017 04:48:23 +0000 Subject: os.path.isfile In-Reply-To: References: <9c43f6d9-1cae-4097-ab15-a8f161ba10bb@googlegroups.com> <589fdbea$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Feb 12, 2017 at 4:29 AM, Chris Angelico wrote: > Registry subkeys aren't paths, and the other two cases are extremely > narrow. Convert slashes to backslashes ONLY in the cases where you > actually need to. \\?\ paths are required to exceed MAX_PATH (a paltry 260 characters) or to avoid quirks of DOS paths (e.g. DOS device names, stripping trailing dots or spaces). Some programs require backslashes in paths passed on the command line -- e.g. more.com, but it works if the path is quoted; obviously reg.exe for registry paths (they are paths); findstr.exe (a grep-like program); mountvol.exe needs a \\?\ Volume GUID path; and running "C:/Windows/system32/cmd.exe" parses "/cmd.exe" in its own name as "/c md.exe", which is more fun if you have an "md.exe" in PATH. In cases where you can use slash in the filesystem API, Windows is doing the conversion for you by rewriting a copy of the path with slash replaced by backslash -- among other normalizations. I'm not saying that relying on the Windows base API to do this work for you is bad, just as a rule it's simpler to call normpath on path literals because you don't have to worry about edge cases like remembering to normalize the path before passing it as a command-line argument. If you're using pathlib, it already does this for you automatically: >>> p = pathlib.Path('spam/eggs') >>> os.fspath(p) 'spam\\eggs' There's been a significant effort to make pathlib interoperate with the rest of the standard library in 3.6. The point above registry subkeys inspires me to stray into Windows internals stuff, so everyone can stop reading at this point... Of course subkeys are relative paths. They're just not file-system paths. Paths of named object types (e.g. Device, File, Key, Section, Event, WindowStation, etc) are rooted in a single object namespace. The only path separator in this namespace is backslash. Forward slash is handled as a regular name character, except file systems in particular reserve it for the sake of POSIX and DOS compatibility. Here's a broad overview of what the object manager's ObOpenObjectByName function does (the "Ob" prefix is for the object manager), which gets called by system services such as NtOpenFile and NtOpenKeyEx to open a named object. The object manager implements Directory and SymbolicLink objects, so it's the first system to parse a path, starting with the root directory. It continues parsing path elements until it reaches an object type that's managed by another system. Then it passes control to that object's ParseProcedure. For a Key this is CmpParseKey (the "Cm" prefix is for the configuration manager). For a Device such as a disk volume, it's IopParseDevice (the "Io" prefix is for the I/O manager). Assuming there isn't an object-type mismatch (e.g. calling NtOpenFile on a registry key) and the object is successfully created or referenced, then a handle created in the calling process handle table is returned to the system service, which returns it to the user-mode caller. For example, parsing "C:/Program Files/Python36" first gets rewritten by the runtime library as "\??\C:\Program Files\Python36" (consider this a raw string, please). The object manager first parses "\??C:" by looking for it as "\Sessions\0\DosDevices\[LogonSessionId]\C:" That's unlikely for the C: drive (though possible). Next it checks for "\Global??\C:". That should be a symbolic link to something like "\Device\HarddiskVolume2". Next it calls the parse procedure for this device object, IopParseDevice, which sees that this is a volume device that's handled by a file-system driver. Say the context of this parse is in the middle of an NtOpenFile call. In this case, the I/O manager creates a File object (which references the remaining path "\Program Files\Python36") and an I/O request packet (IRP) to be serviced by the file-system device stack. If the file system supports reparse points, such as NTFS junctions and symbolic links, the IRP might be completed with a STATUS_REPARSE code and a new path to parse. Finally, if the open succeeds, the object manager creates a handle for the File object in the process handle table, and the handle value is returned to the caller. Now consider calling NtOpenKeyEx to open a registry key. The master registry hive has a root key named "\Registry", and two commonly used subkeys "\Registry\Machine" and "\Registry\Users". We typically reference the latter two keys via the pseudo-handles HKEY_LOCAL_MACHINE (HKLM) and HKEY_USERS (HKU) -- because these also work when accessing a remote registry over RPC. Say we're trying to open "HKLM\Software\Python\PythonCore". The real local path is "\Registry\Machine\Software\Python\PythonCore". The first thing to do is open and cache the real handle for the HKLM pseudo-handle, by opening "\Registry\Machine". NtOpenKeyEx calls ObOpenObjectByName, and the object manager begins parsing the path. It hands off parsing to the ParseProcedure of the "\Registry" object, CmpParseKey, which returns a pointer reference to the "\Registry\Machine" key object. The object manager creates a handle for the object in the process handle table, and NtOpenKeyEx returns this handle to the caller. It's little known, but the registry also supports symbolic links, so CmpParseKey may return STATUS_REPARSE with a new path for the object manager to parse. Next it does a relative open on the path "Software\Python\PythonCore" using the "\Registry\Machine" handle as the RootDirectory for the ObjectAttributes of the open. A relative open for a disk volume works the same way (e.g. opening a file relative to a handle for the working directory). The interesting thing about the documented registry API is that it exposes this native ability to open relative to a handle (like Unix *at system calls). Similar functionality could be supported in CreateFile by extending the sized SECURITY_ATTRIBUTES structure to add a RootDirectory field. As is you have to call NtCreateFile or NtOpenFile to get this functionality, which isn't supported. Let's check this out in the debugger. First Windows opens "\Registry\Machine" to cache the real handle for its HKLM pseudo-handle. Breakpoint 0 hit ntdll!NtOpenKeyEx: 00007ffd`b29b82d0 4c8bd1 mov r10,rcx 0:000> !obja @r8 Obja +000000913bfef878 at 000000913bfef878: Name is \REGISTRY\MACHINE OBJ_CASE_INSENSITIVE 0:000> r rcx rcx=000000913bfef838 0:000> pt ntdll!NtOpenKeyEx+0x14: 00007ffd`b29b82e4 c3 ret The handle returned is 0x70: 0:000> dq 913bfef838 l1 00000091`3bfef838 00000000`00000070 0:000> g Next it opens the relative path "Software\Python\PythonCore". Breakpoint 0 hit ntdll!NtOpenKeyEx: 00007ffd`b29b82d0 4c8bd1 mov r10,rcx 0:000> !obja @r8 Obja +000000913bfef6d0 at 000000913bfef6d0: Name is Software\Python\PythonCore OBJ_CASE_INSENSITIVE The RootDirectory field is 0x70, the handle for "\Registry\Machine", as we can easily see in the kernel debugger when looking at the above address (0x913bfef6d0): lkd> ?? (nt!_OBJECT_ATTRIBUTES *)0x913bfef6d0 struct _OBJECT_ATTRIBUTES * 0x00000091`3bfef6d0 +0x000 Length : 0x30 +0x008 RootDirectory : 0x00000000`00000070 Void +0x010 ObjectName : 0x00000091`3bfef978 _UNICODE_STRING "Software\Python\PythonCore" +0x018 Attributes : 0x40 +0x020 SecurityDescriptor : (null) +0x028 SecurityQualityOfService : (null) lkd> !handle 0x70 3 0070: Object: ffffe00a8ddfbf70 GrantedAccess: 000f003f (Audit) ... Name: \REGISTRY\MACHINE To close this discussion out, here's another problem involving slash in named objects. The Windows base API creates many per-session objects in a "BaseNamedObjects" directory located at "\Sessions\[SessionId]\BaseNamedObjects" or globally in "\BaseNamedObjects". It's a dumping ground for objects that don't have a better place to call home. Within the session directory there's a "Global" symbolic link to the system-wide "\BaseNamedObjects" directory. When creating a named object, you can use that link to name the object globally for all sessions. For example, creating a shared-memory section named "Global\MySharedMemory" actually creates "\BaseNamedObjects\MySharedMemory". But if you accidentally write "Global/MySharedMemory", you'll instead create an object named with a literal slash in the local session's BaseNamedObjects. I've seen this problem before in a Stack Overflow question. People get lulled into a false belief that the Windows API will handle forward slashes as path separators in anything that's pathlike (and indeed is actually implemented as a relative path under the hood), but that's only the file-system API. From jhlobin at gmail.com Mon Feb 13 07:17:18 2017 From: jhlobin at gmail.com (jhlobin at gmail.com) Date: Mon, 13 Feb 2017 04:17:18 -0800 (PST) Subject: Problems with scp script in Python Message-ID: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> I have an SCP script that auto completes just fine from the terminal window in Pi. ?When I use either subprocess or os to try and have it run under Python it does not do the file transfer from Pi to my Ubuntu machine. ?What am I doing wrong? ?Here is the script:? from subprocess import call? cmd='scp -i id_rsa sample.txt?jack at 192.168.1.103?/home/jack/pi_data'? call (cmd.split ())? Thank you? From Joaquin.Alzola at lebara.com Mon Feb 13 07:26:41 2017 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Mon, 13 Feb 2017 12:26:41 +0000 Subject: Problems with scp script in Python In-Reply-To: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> References: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> Message-ID: >I have an SCP script that auto completes just fine from the terminal window in Pi. When I use either subprocess or os to try and have it run under Python it does not do the file >transfer from Pi to my Ubuntu machine. What am I doing wrong? Here is the script: >from subprocess import call >cmd='scp -i id_rsa sample.txt jack at 192.168.1.103 /home/jack/pi_data' >call (cmd.split ()) Do it manually first and check that it is working. Then if you will not expect any output: os.system(''scp -i id_rsa sample.txt jack at 192.168.1.103:/home/jack/pi_data') I put a colon as well when I copy to a specific directory. This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From jhlobin at gmail.com Mon Feb 13 07:34:30 2017 From: jhlobin at gmail.com (jhlobin at gmail.com) Date: Mon, 13 Feb 2017 04:34:30 -0800 (PST) Subject: Problems with scp script in Python In-Reply-To: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> References: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> Message-ID: Sorry, I did use a colon after the remote machine address, mistyped above. It does work ok under the terminal window. From torriem at gmail.com Mon Feb 13 08:24:05 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 13 Feb 2017 06:24:05 -0700 Subject: Problems with scp script in Python In-Reply-To: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> References: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> Message-ID: <1742325d-695a-2494-fdc2-70c09375a045@gmail.com> On 2017-02-13 05:17 AM, jhlobin at gmail.com wrote: > I have an SCP script that auto completes just fine from the terminal window in Pi. When I use either subprocess or os to try and have it run under Python it does not do the file transfer from Pi to my Ubuntu machine. What am I doing wrong? Here is the script: > > from subprocess import call > cmd='scp -i id_rsa sample.txt jack at 192.168.1.103 /home/jack/pi_data' > call (cmd.split ()) > > Thank you Did you try specifying the full path for scp? From steve+python at pearwood.info Mon Feb 13 09:49:19 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 14 Feb 2017 01:49:19 +1100 Subject: Problems with scp script in Python References: <89399433-0ebd-47d3-a161-3a0eee47750e@googlegroups.com> Message-ID: <58a1c770$0$1588$c3e8da3$5496439d@news.astraweb.com> On Mon, 13 Feb 2017 11:17 pm, jhlobin at gmail.com wrote: > I have an SCP script that auto completes just fine from the terminal > window in Pi. ?When I use either subprocess or os to try and have it run > under Python it does not do the file transfer from Pi to my Ubuntu > machine. ?What am I doing wrong? ?Here is the script: First thing you are doing wrong: you are re-typing the script instead of copying and pasting it. There's at least one error in your script below: a missing colon. How many other errors did you make when retyping? There's no way for us to tell. Second thing you are doing wrong: keeping the exception traceback a secret from us. How are we supposed to diagnose the error when we can't see what the error is? Even if we wanted to run your code, we can't: we'll get a completely different error, since we don't have access to your network and user account. > from subprocess import call > cmd='scp -i id_rsa sample.txt?jack at 192.168.1.103?/home/jack/pi_data' > call (cmd.split ()) > > Thank you If there is no exception traceback, is there any output at all? Again, don't type it from memory, COPY and PASTE the output as you see it, exactly. If you remove any private information (like a plain text password) make it obvious that has been removed by replacing it with a string of asterisks **** . -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lauren.sophia1998 at gmail.com Mon Feb 13 11:30:32 2017 From: lauren.sophia1998 at gmail.com (lauren.sophia1998 at gmail.com) Date: Mon, 13 Feb 2017 08:30:32 -0800 (PST) Subject: Problems with scripts Message-ID: Hello! I have 2 python assignments that I just can't figure out. The first one returns the same thing no matter what I input and the second won't accept "done" to stop the program and return answers. Please help! 1) print("How old are you: 17, 18, 19, or 20?") answer = input("> ") if answer == 17 or 18 or 19 or 20: print("Wow, you are old!") elif answer != 17 or 18 or 19 or 20: if type(answer) is int or float: print("You just can't follow drections, can you? Choose either 17, 18, 19, or 20.") input("> ") elif type(answer) is str: print("That isn't even a number. Choose either 17, 18, 19, or 20.") input("> ") 2) print("This program keeps track of the prices of items and how many items bought at that price. Enter 'done' to return the answers.") item_num = 1 total_price = 0 price = input("What is the price of item number " + str(item_num) + " ? ") total_items = 0 how_many = input("How many items at that price? ") while price or how_many != "done": total_price = int(total_price) + int(price) total_items = int(total_items) + int(how_many) item_num = item_num + 1 price = input("What is the price of item number " + str(item_num) + " ? ") how_many = input("How many items at that price? ") if input == done: print("The total price is $" + str(total_price) + ". And the number of items for that price is " + str(total_items) + ".") From joel.goldstick at gmail.com Mon Feb 13 11:55:50 2017 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Mon, 13 Feb 2017 11:55:50 -0500 Subject: Problems with scripts In-Reply-To: References: Message-ID: On Mon, Feb 13, 2017 at 11:30 AM, wrote: > Hello! I have 2 python assignments that I just can't figure out. The first one returns the same thing no matter what I input and the second won't accept "done" to stop the program and return answers. Please help! > > 1) > print("How old are you: 17, 18, 19, or 20?") > answer = input("> ") > if answer == 17 or 18 or 19 or 20: You can't do if like you are trying to do. First, you are comparing to numbers, but you have a string for answer. So, convert answer to an int, and then look up 'if answer in (17,18, etc.): > print("Wow, you are old!") > elif answer != 17 or 18 or 19 or 20: > if type(answer) is int or float: > print("You just can't follow drections, can you? Choose either 17, 18, 19, or 20.") > input("> ") > elif type(answer) is str: > print("That isn't even a number. Choose either 17, 18, 19, or 20.") > input("> ") > > > > 2) > print("This program keeps track of the prices of items and how many items bought at that price. Enter 'done' to return the answers.") > item_num = 1 > total_price = 0 > price = input("What is the price of item number " + str(item_num) + " ? ") > total_items = 0 > how_many = input("How many items at that price? ") > while price or how_many != "done": > total_price = int(total_price) + int(price) > total_items = int(total_items) + int(how_many) > item_num = item_num + 1 > price = input("What is the price of item number " + str(item_num) + " ? ") > how_many = input("How many items at that price? ") > > if input == done: > print("The total price is $" + str(total_price) + ". And the number of items for that price is " + str(total_items) + ".") > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From best_lay at yahoo.com Mon Feb 13 12:54:02 2017 From: best_lay at yahoo.com (Wildman) Date: Mon, 13 Feb 2017 11:54:02 -0600 Subject: Problems with scripts References: Message-ID: On Mon, 13 Feb 2017 08:30:32 -0800, lauren.sophia1998 wrote: > Hello! I have 2 python assignments that I just can't figure out. The first one returns the same thing no matter what I input and the second won't accept "done" to stop the program and return answers. Please help! > > 1) > print("How old are you: 17, 18, 19, or 20?") > answer = input("> ") > if answer == 17 or 18 or 19 or 20: > print("Wow, you are old!") > elif answer != 17 or 18 or 19 or 20: > if type(answer) is int or float: > print("You just can't follow drections, can you? Choose either 17, 18, 19, or 20.") > input("> ") > elif type(answer) is str: > print("That isn't even a number. Choose either 17, 18, 19, or 20.") > input("> ") Incorrect use of 'or'. if answer == 17 or answer == 18 or answer == 19 or answer == 20: elif answer != 17 and answer != 18 and answer != 19 and answer != 20: if type(answer) is int or type(answer) is float: > 2) > print("This program keeps track of the prices of items and how many items bought at that price. Enter 'done' to return the answers.") > item_num = 1 > total_price = 0 > price = input("What is the price of item number " + str(item_num) + " ? ") > total_items = 0 > how_many = input("How many items at that price? ") > while price or how_many != "done": > total_price = int(total_price) + int(price) > total_items = int(total_items) + int(how_many) > item_num = item_num + 1 > price = input("What is the price of item number " + str(item_num) + " ? ") > how_many = input("How many items at that price? ") > > if input == done: > print("The total price is $" + str(total_price) + ". And the number of items for that price is " + str(total_items) + ".") Same here: while price != "done" and how_many != "done": -- GNU/Linux user #557453 May the Source be with you. From lauren.sophia1998 at gmail.com Mon Feb 13 13:03:05 2017 From: lauren.sophia1998 at gmail.com (Lauren Fugate) Date: Mon, 13 Feb 2017 10:03:05 -0800 (PST) Subject: Problems with scripts In-Reply-To: References: Message-ID: <8f77d5f6-d49f-42e1-ac88-5c18a8c1e86c@googlegroups.com> So I changed my code to this: print("How old are you: 17, 18, 19, or 20?") answer = input("> ") if int(answer) == 17 or 18 or 19 or 20: print("Wow, you are old!") else: print("You just can't follow drections, can you? Choose either 17, 18, 19, or 20.") input("> ") and now if I input any of the preferred answers it will print "Wow, you are old!" like it should. But if I enter a word, like "hi", it will return an error. From lauren.sophia1998 at gmail.com Mon Feb 13 13:08:11 2017 From: lauren.sophia1998 at gmail.com (Lauren Fugate) Date: Mon, 13 Feb 2017 10:08:11 -0800 (PST) Subject: Problems with scripts In-Reply-To: References: Message-ID: <255ae39a-538e-4b19-9edb-64a19e7c63e8@googlegroups.com> So I tried both of these and they didn't change anything, the python shell printed the same things... From alister.ware at ntlworld.com Mon Feb 13 13:23:07 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 13 Feb 2017 18:23:07 GMT Subject: Problems with scripts References: <255ae39a-538e-4b19-9edb-64a19e7c63e8@googlegroups.com> Message-ID: On Mon, 13 Feb 2017 10:08:11 -0800, Lauren Fugate wrote: > So I tried both of these and they didn't change anything, the python > shell printed the same things... No answers (answering you homework for you will not teach you anything useful) but some hints to help you think about the problem 1) why check for each option explicitly, why not simply check if the input is between acceptable limits. 2) instead of checking for valid inputs, why not react to the invalid ones 3) what is the error you are getting when you try to change your input to an int, can you TRY* to do anything with this error? * this is a very big hint -- YOW!! Now I understand advanced MICROBIOLOGY and th' new TAX REFORM laws!! From best_lay at yahoo.com Mon Feb 13 13:53:04 2017 From: best_lay at yahoo.com (Wildman) Date: Mon, 13 Feb 2017 12:53:04 -0600 Subject: Problems with scripts References: <255ae39a-538e-4b19-9edb-64a19e7c63e8@googlegroups.com> Message-ID: On Mon, 13 Feb 2017 10:08:11 -0800, Lauren Fugate wrote: > So I tried both of these and they didn't change anything, the python shell printed the same things... The first assignment is overly complicated. The extra input functions are useless. There is no loopback to check the input. Also, input returns a string not an int or float. Try this: x = ["17","18","19","20"] answer = None print("How old are you: 17, 18, 19, or 20?") while answer not in x: answer = input("> ") if answer in x: print("Wow, you are old!") else: print("You just can't follow drections, can you? Choose either 17, 18, 19, or 20.") Keep in mind that the above code does not give you a way out. One of the expected numbers must be entered. I'll leave the second one for you to figure out. -- GNU/Linux user #557453 The cow died so I don't need your bull! From steve+python at pearwood.info Mon Feb 13 18:41:58 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 14 Feb 2017 10:41:58 +1100 Subject: Problems with scripts References: Message-ID: <58a24448$0$1591$c3e8da3$5496439d@news.astraweb.com> On Tue, 14 Feb 2017 03:30 am, lauren.sophia1998 at gmail.com wrote: > Hello! I have 2 python assignments that I just can't figure out. The first > one returns the same thing no matter what I input and the second won't > accept "done" to stop the program and return answers. Please help! Hi Lauren, Best to ask one question per post, unless they are very closely related. You might like to resend your second question in a new message, with a different SUBJECT line. > print("How old are you: 17, 18, 19, or 20?") > answer = input("> ") After this line, "answer" is a string. Even if it looks like a number, it is still a string. Have you learned about the differences between strings and integers yet? How do you think you might change answer to an int, or a float, if and only if it can, but leave it as a string if not? > if answer == 17 or 18 or 19 or 20: > print("Wow, you are old!") Nice try, but not quite! Python code is very close to English, but not *that* close. In English, you can say: if the answer equals 17 or 18 or 19 or 20 but Python doesn't understand that. You can write this instead: if answer == 17 or answer == 18 or answer == 19 or answer == 20: but perhaps an easier way is: if answer in (17, 18, 19, 20): (But remember, at the moment answer is still a string. Strings are never equal to ints, even if they look the same.) > elif answer != 17 or 18 or 19 or 20: There's no need to repeat the test, just say else: instead. > if type(answer) is int or float: > print("You just can't follow drections, can you? Choose either 17, > 18, 19, or 20.") input("> ") > elif type(answer) is str: > print("That isn't even a number. Choose either 17, 18, 19, or > 20.") input("> ") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From Cecil at decebal.nl Mon Feb 13 18:51:36 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Tue, 14 Feb 2017 00:51:36 +0100 Subject: Multimedia and Python Message-ID: <87tw7xwsxj.fsf@Equus.decebal.nl> At the moment I use Bash scripts with Image Magick, ffmpeg and ghostscript to do some multimedia stuff. But I want to make the switch to Python. An initial search does point me to PyMedia. Is this the correct tool, or is there a better one? Initially I just want to cut a part from a MP3 file. Nothing fancy: from a point until a point. Later on I want to do some more advanced stuff: - Adding text and/or graphics to the video. - Converting of video. - Putting graphics and audio together to create a video. - Creating graphics like: https://www.facebook.com/TimeManagementTipsAndSkills/photos/a.301775443228177.69771.104189696320087/1347649318640779/ just from the text. - ? -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From c.curtit at gmail.com Tue Feb 14 11:41:12 2017 From: c.curtit at gmail.com (c.curtit at gmail.com) Date: Tue, 14 Feb 2017 08:41:12 -0800 (PST) Subject: memory access ordering between processes in python 3.6 Message-ID: <89d8d5ed-7dc7-45f8-ba17-7a29e9c7f85e@googlegroups.com> Hi everyone, I'm looking at building an application that allows lockless interprocess data passing using a ring buffer style (sorta) shared memory zone. My initial plan is to create a shared memory zone (probably using C functions thru ctypes), then write data to it with a writer process, then the reader process comes to read that data in its own time. Basically the zone is read/written using ctypes or memoryview objects. The zone (say 1GB) is split in many small buckets approx 10k in size. Each bucket has a flag either showing 0 (free) or 1 (needs reading) in a header. The flag is reset by the reader when done. and is set by the writer once the full bucket has been written to memory and is ready to be read. I know I have to be carefull to make sure writes appear in the proper order to the reader process (that is : data writen by the writer needs to appear before the flag is set by the writter process, from the point of view of the reader process, which is absolutely not a given on modern cpu). So questions : *does python takes care of that automagically, and that's why I havn't found anything on the subject while googling ? (I imagine that within one single multithreaded process the GIL does that neatly, but what of multiprocess applications ?) *or is there a native mechanism or module that allows to play with memory barriers like C does allow one to (https://www.kernel.org/doc/Documentation/memory-barriers.txt). * or do I have to write my own shims to C functions wrapping C/asm memory barrier instructions ? (sounds lighter weight than pulling in the various python asm modules I have around) Any answers much appreciated. Please don't : * tell me to use message queues (that works with some trickery, it's not the purpose of this question) * tell me to use array from the the multiprocessing module, or the data proxy classes (or explain me if the locking scheme in that context can be lockless :) ) Cheers ! Charlie From josemsuarezsierra at gmail.com Tue Feb 14 12:13:48 2017 From: josemsuarezsierra at gmail.com (=?UTF-8?Q?Jos=C3=A9_Manuel_Su=C3=A1rez_Sierra?=) Date: Tue, 14 Feb 2017 09:13:48 -0800 (PST) Subject: Doubt with files Message-ID: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> hello, im trying to read a rtf or txt file with this python script: with open(dirFichero,'r') as reader: for line in reader: print line the problem is that shown is : {\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf810 {\fonttbl\f0\fswiss\fcharset0 Helvetica;} {\colortbl;\red255\green255\blue255;} {\*\expandedcolortbl;;} \paperw11900\paperh16840\margl1440\margr1440\vieww10800\viewh8400\viewkind0 \pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\pardirnatural\partightenfactor0 \f0\fs24 \cf0 1l2m,1svo,1lme} INSTEAD of: 1l2m,1svo,1lme How could I fix it? THANK YOU From Joaquin.Alzola at lebara.com Tue Feb 14 12:31:16 2017 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Tue, 14 Feb 2017 17:31:16 +0000 Subject: Doubt with files In-Reply-To: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> References: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> Message-ID: >with open(dirFichero,'r') as reader: > for line in reader: > print line > >the problem is that shown is : >{\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf810 >{\fonttbl\f0\fswiss\fcharset0 Helvetica;} >{\colortbl;\red255\green255\blue255;} >{\*\expandedcolortbl;;} >\paperw11900\paperh16840\margl1440\margr1440\vieww10800\viewh8400\viewkind0 >\pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\pardirnatural\partightenfactor0 > >\f0\fs24 \cf0 1l2m,1svo,1lme} > >INSTEAD of: >1l2m,1svo,1lme > >How could I fix it? I do not know exactly what encoding rtf or the txt file has but you can use: with open(dirFichero,'r',encoding='utf-8') as reader: This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From robertvstepp at gmail.com Tue Feb 14 12:38:28 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Tue, 14 Feb 2017 11:38:28 -0600 Subject: Doubt with files In-Reply-To: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> References: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> Message-ID: On Tue, Feb 14, 2017 at 11:13 AM, Jos? Manuel Su?rez Sierra wrote: > hello, > im trying to read a rtf or txt file with this python script: > > with open(dirFichero,'r') as reader: > for line in reader: > print line > > the problem is that shown is : > > {\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf810 > > {\fonttbl\f0\fswiss\fcharset0 Helvetica;} > > {\colortbl;\red255\green255\blue255;} > > {\*\expandedcolortbl;;} > > \paperw11900\paperh16840\margl1440\margr1440\vieww10800\viewh8400\viewkind0 > > \pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\pardirnatural\partightenfactor0 > > > > \f0\fs24 \cf0 1l2m,1svo,1lme} > > > INSTEAD of: > 1l2m,1svo,1lme > > How could I fix it? Your program seems to be faithfully printing out the contents of your rtf file as a txt file, showing all of the rtf formatting code. If you want a different result, I suggest you search online for something like: python read write rtf -- boB From robertvstepp at gmail.com Tue Feb 14 12:40:14 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Tue, 14 Feb 2017 11:40:14 -0600 Subject: Doubt with files In-Reply-To: References: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> Message-ID: On Tue, Feb 14, 2017 at 11:31 AM, Joaquin Alzola wrote: > >>with open(dirFichero,'r') as reader: > > for line in reader: > > print line >> >>the problem is that shown is : >>{\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf810 >>{\fonttbl\f0\fswiss\fcharset0 Helvetica;} >>{\colortbl;\red255\green255\blue255;} >>{\*\expandedcolortbl;;} >>\paperw11900\paperh16840\margl1440\margr1440\vieww10800\viewh8400\viewkind0 >>\pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx5102\tx5669\tx6236\tx6803\pardirnatural\partightenfactor0 >> >>\f0\fs24 \cf0 1l2m,1svo,1lme} >> >>INSTEAD of: >>1l2m,1svo,1lme >> >>How could I fix it? > > I do not know exactly what encoding rtf or the txt file has but you can use: > > with open(dirFichero,'r',encoding='utf-8') as reader: This won't work. It will still display the rtf format coding, which the OP apparently does not want. -- boB From pkpearson at nowhere.invalid Tue Feb 14 12:44:18 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 14 Feb 2017 17:44:18 GMT Subject: Doubt with files References: <21721e4d-c33f-454a-8490-941ae849529b@googlegroups.com> Message-ID: On Tue, 14 Feb 2017 09:13:48 -0800 (PST), Jos? Manuel Su?rez Sierra wrote: > hello, > im trying to read a rtf or txt file with this python script: > > with open(dirFichero,'r') as reader: > for line in reader: > print line > > the problem is that shown is : [suppressing blank lines] > > {\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf810 > {\fonttbl\f0\fswiss\fcharset0 Helvetica;} > {\colortbl;\red255\green255\blue255;} > {\*\expandedcolortbl;;} > \paperw11900\paperh16840\margl1440\margr1440\vieww10800\viewh8400\viewkind0 > \pard\tx566\tx1133\tx1700\tx2267\tx2834\tx3401\tx3968\tx4535\tx [snip] > \f0\fs24 \cf0 1l2m,1svo,1lme} > > INSTEAD of: > 1l2m,1svo,1lme > > How could I fix it? An RTF file *is* a text file, with formatting information inserted as text strings. Your program is reading and printing an RTF file. If all you want is the part of the file's content that will be visible when it has been rendered into graphics, you'll need to convert the file from RTF to "plain" text at some point. I'd do that with LibreOffice. Alternativey, I see that there's a PyRTF package that might do what you want from within your Python program. -- To email me, substitute nowhere->runbox, invalid->com. From kwa at kuwata-lab.com Tue Feb 14 17:44:03 2017 From: kwa at kuwata-lab.com (Makoto Kuwata) Date: Wed, 15 Feb 2017 07:44:03 +0900 Subject: WANT: bad code in python (for refactoring example) Message-ID: Hi, Is there any *just right* python code to refactor? In other words, I'm finding bad code example in python. (background) I'm teaching Python to some novice programmer, and want to show refactoring example to them. (required) * not good code * not too large (for novice programmer) * not too complex (for novice programmer) * released under open source license If you know good material in github or bitbucket to refactor, let me know it. -- regards, kwatch From irmen.NOSPAM at xs4all.nl Tue Feb 14 18:12:54 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Wed, 15 Feb 2017 00:12:54 +0100 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: Message-ID: <58a38ef5$0$673$e4fe514c@news.xs4all.nl> On 14-2-2017 23:44, Makoto Kuwata wrote: > Hi, > > Is there any *just right* python code to refactor? > In other words, I'm finding bad code example in python. > > (background) > I'm teaching Python to some novice programmer, and > want to show refactoring example to them. > > (required) > * not good code > * not too large (for novice programmer) > * not too complex (for novice programmer) > * released under open source license > > If you know good material in github or bitbucket to refactor, > let me know it. > > -- > regards, > kwatch > No code in text form, but I find the following video by Raymond Hettinger very clear about the values of refactoring otherwise seemingly fine code https://youtu.be/wf-BqAjZb8M?t=12m39s code starts at 12 min. 40 sec. It does require a bit of background knowledge about Python and the PEP8 code style and API design. Irmen From steve at pearwood.info Wed Feb 15 01:28:30 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 15 Feb 2017 06:28:30 GMT Subject: WANT: bad code in python (for refactoring example) References: Message-ID: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> On Wed, 15 Feb 2017 07:44:03 +0900, Makoto Kuwata wrote: > Hi, > > Is there any *just right* python code to refactor? > In other words, I'm finding bad code example in python. Try looking at the ActiveState website for recipes in Python. Especially look at the ones with negative ratings, but even positively rated recipes are often nonsense. E.g. http://code.activestate.com/recipes/580750 does nothing more that define echo = sys.stdout.write Why not use sys.stdout.write directly? Or print? If I saw somebody using this recipe in production code, in the way shown, I'd refactor it to just use print. There's no advantage to re-inventing the wheel this way. -- Steve From p.oseidon at datec.at Wed Feb 15 04:33:01 2017 From: p.oseidon at datec.at (poseidon) Date: Wed, 15 Feb 2017 10:33:01 +0100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? Message-ID: In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It contains the line /home/poseidon/tau4/swr/py3/src In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it should be possible to write import tau4 in my programs. But it isn't. Despite the fact that /home/poseidon/tau4/swr/py3/src is in sys.path, I get a ModuleNotFoundError. It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the site-packages dir: ln -s /home/poseidon/tau4/swr/py3/src /usr/lib/python3.6/site-packages/tau4 https://docs.python.org/3.6/library/site.html suggests that PTH files only work relative to the site-packages dir. But digging around in e.g. StackOverflow I got the impression that absolute paths should work as well. If so, what am I doing wrong? I'm on Arch Linux, Python 3.6. Kind regards Paul From kwa at kuwata-lab.com Wed Feb 15 04:36:08 2017 From: kwa at kuwata-lab.com (Makoto Kuwata) Date: Wed, 15 Feb 2017 18:36:08 +0900 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: Thanks Irmen and Steven, I'm sorry that my explanation is not enough. I'm looking for bad python code, not refactoring examples. I can do (and want to do) refactor bad code by myself. Is there any bad python code (or project) with proper size? # I found a lot of bad PHP code in github, but didn't find bad Python code. -- regards, makoto kuwata On Wed, Feb 15, 2017 at 3:28 PM, Steven D'Aprano wrote: > On Wed, 15 Feb 2017 07:44:03 +0900, Makoto Kuwata wrote: > > > Hi, > > > > Is there any *just right* python code to refactor? > > In other words, I'm finding bad code example in python. > > > Try looking at the ActiveState website for recipes in Python. Especially > look at the ones with negative ratings, but even positively rated recipes > are often nonsense. > > E.g. http://code.activestate.com/recipes/580750 > > does nothing more that define > > echo = sys.stdout.write > > Why not use sys.stdout.write directly? Or print? If I saw somebody using > this recipe in production code, in the way shown, I'd refactor it to just > use print. There's no advantage to re-inventing the wheel this way. > > > > -- > Steve > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at rece.vub.ac.be Wed Feb 15 05:49:50 2017 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 15 Feb 2017 11:49:50 +0100 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: Op 15-02-17 om 07:28 schreef Steven D'Aprano: > On Wed, 15 Feb 2017 07:44:03 +0900, Makoto Kuwata wrote: > >> Hi, >> >> Is there any *just right* python code to refactor? >> In other words, I'm finding bad code example in python. > > Try looking at the ActiveState website for recipes in Python. Especially > look at the ones with negative ratings, but even positively rated recipes > are often nonsense. > > E.g. http://code.activestate.com/recipes/580750 > > does nothing more that define > > echo = sys.stdout.write > > Why not use sys.stdout.write directly? Or print? If I saw somebody using > this recipe in production code, in the way shown, I'd refactor it to just > use print. There's no advantage to re-inventing the wheel this way. On reason to use this is for some easy "logging", you use echo to help in debugging and afterwards you can either define echo as an empty function or something easy to find to comment out. -- Antoon Pardon. From dpalao.python at gmail.com Wed Feb 15 05:53:29 2017 From: dpalao.python at gmail.com (David Palao) Date: Wed, 15 Feb 2017 11:53:29 +0100 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: 2017-02-15 10:36 GMT+01:00 Makoto Kuwata : > Thanks Irmen and Steven, > > I'm sorry that my explanation is not enough. > I'm looking for bad python code, not refactoring examples. > I can do (and want to do) refactor bad code by myself. > > Is there any bad python code (or project) with proper size? > > # I found a lot of bad PHP code in github, but didn't find bad Python code. > > -- > regards, > makoto kuwata > > > On Wed, Feb 15, 2017 at 3:28 PM, Steven D'Aprano > wrote: > >> On Wed, 15 Feb 2017 07:44:03 +0900, Makoto Kuwata wrote: >> >> > Hi, >> > >> > Is there any *just right* python code to refactor? >> > In other words, I'm finding bad code example in python. >> >> >> Try looking at the ActiveState website for recipes in Python. Especially >> look at the ones with negative ratings, but even positively rated recipes >> are often nonsense. >> >> E.g. http://code.activestate.com/recipes/580750 >> >> does nothing more that define >> >> echo = sys.stdout.write >> >> Why not use sys.stdout.write directly? Or print? If I saw somebody using >> this recipe in production code, in the way shown, I'd refactor it to just >> use print. There's no advantage to re-inventing the wheel this way. >> >> >> >> -- >> Steve >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > -- > https://mail.python.org/mailman/listinfo/python-list Hello, IMHO most code can be refactored. Forme the point is how you want to teach it and what you want to teach exactly. For instance, if you are following a tdd approach, then the project you choose, should better come with that in mind. If you are not considering tests, for whatever reason, I guess any simple enough project would be good. Of course, it depends on the details of what you want to teach. BTW, at work I have been told to organize a course about testing and good programming practices with Python. I was thinking in developing an example by myself... Best David From wolfgang.maier at biologie.uni-freiburg.de Wed Feb 15 06:16:39 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 15 Feb 2017 12:16:39 +0100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On 15.02.2017 10:33, poseidon wrote: > In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It contains > the line > > /home/poseidon/tau4/swr/py3/src > > In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it > should be possible to write > > import tau4 > > in my programs. No, that's not what you should expect! A path file contains paths to be added at interpreter startup to the package/module search path stored in sys.path. That is, in your example, if you put a file tau4.py or a tau4 directory with the __init__.py file inside into /home/poseidon/tau4/swr/py3/src, *then* you could import tau4. > It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the > site-packages dir: > > ln -s /home/poseidon/tau4/swr/py3/src /usr/lib/python3.6/site-packages/tau4 > Well this works because now Python finds (following the symlink) a tau4 package (i.e., a directory with that name and an __init__.py file inside) in /usr/lib/python3.6/site-packages. The .pth file is not involved in this at all. > https://docs.python.org/3.6/library/site.html suggests that PTH files > only work relative to the site-packages dir. But digging around in e.g. > StackOverflow I got the impression that absolute paths should work as > well. If so, what am I doing wrong? > > I'm on Arch Linux, Python 3.6. > > Kind regards > Paul > > From steve+python at pearwood.info Wed Feb 15 07:24:26 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 15 Feb 2017 23:24:26 +1100 Subject: WANT: bad code in python (for refactoring example) References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: <58a4487c$0$1589$c3e8da3$5496439d@news.astraweb.com> On Wed, 15 Feb 2017 09:49 pm, Antoon Pardon wrote: > Op 15-02-17 om 07:28 schreef Steven D'Aprano: [...] >> Why not use sys.stdout.write directly? Or print? If I saw somebody using >> this recipe in production code, in the way shown, I'd refactor it to just >> use print. There's no advantage to re-inventing the wheel this way. > > On reason to use this is for some easy "logging", you use echo to help > in debugging and afterwards you can either define echo as an empty > function or something easy to find to comment out. In Python 3 you can always add def print(*args, **kw): pass in your module to disable printing. But a better way, in my opinion, is to use your editor to search for: print( and replace with: #print( But even better, once you get to the point of putting print calls in more than two or three places in your code, you should probably invest the time to learn how to use the logging module. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.oseidon at datec.at Wed Feb 15 07:42:06 2017 From: p.oseidon at datec.at (poseidon) Date: Wed, 15 Feb 2017 13:42:06 +0100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On 15/02/17 12:16, Wolfgang Maier wrote: > On 15.02.2017 10:33, poseidon wrote: >> In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It contains >> the line >> >> /home/poseidon/tau4/swr/py3/src >> >> In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it >> should be possible to write >> >> import tau4 >> >> in my programs. > > > No, that's not what you should expect! > A path file contains paths to be added at interpreter startup to the > package/module search path stored in sys.path. That is, in your example, > if you put a file tau4.py or a tau4 directory with the __init__.py file > inside into /home/poseidon/tau4/swr/py3/src, *then* you could import tau4. > >> It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the >> site-packages dir: >> >> ln -s /home/poseidon/tau4/swr/py3/src >> /usr/lib/python3.6/site-packages/tau4 >> > > Well this works because now Python finds (following the symlink) a tau4 > package (i.e., a directory with that name and an __init__.py file > inside) in /usr/lib/python3.6/site-packages. The .pth file is not > involved in this at all. > Yes, removed it (symlink still there) and it still works. But then, what are pth files for? I'd just place a symlink to the package and am done with. The path doesn't seem to be needed in sys.path (where it would go if placed in a pth file). If I write from tau4 import datalogging that works, too. So no need for the path being in sys.path (i.e. in a pth file)? >> https://docs.python.org/3.6/library/site.html suggests that PTH files >> only work relative to the site-packages dir. But digging around in e.g. >> StackOverflow I got the impression that absolute paths should work as >> well. If so, what am I doing wrong? >> >> I'm on Arch Linux, Python 3.6. >> >> Kind regards >> Paul >> >> > From spiess.benjamin at googlemail.com Wed Feb 15 08:27:03 2017 From: spiess.benjamin at googlemail.com (spiess.benjamin at googlemail.com) Date: Wed, 15 Feb 2017 05:27:03 -0800 (PST) Subject: Create ordering of points Message-ID: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> Hello !:) I've got a problem which I would really like to solve. I got a cloud of points (in the simplest example its a 2-dimensional cloud of points). First, I want to set one of the points as the initial (or middle) point. Starting from there, the next points with the closest distance to the initial points are to be found. Afterwards, the closest points to these points are to be written out. These points shall be connected like it is depicted in this figure http://fs5.directupload.net/images/170215/b835zixm.jpg With this approach, the "fastest" path from every point to the defined initial point are specified by this pattern. Does somebody know a good technique for this problem? or can even give a hint to a existing python procedure? I would be really grateful! Regards Benny From wolfgang.maier at biologie.uni-freiburg.de Wed Feb 15 08:34:13 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 15 Feb 2017 14:34:13 +0100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On 15.02.2017 13:42, poseidon wrote: > On 15/02/17 12:16, Wolfgang Maier wrote: >> On 15.02.2017 10:33, poseidon wrote: >>> In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It contains >>> the line >>> >>> /home/poseidon/tau4/swr/py3/src >>> >>> In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it >>> should be possible to write >>> >>> import tau4 >>> >>> in my programs. >> >> >> No, that's not what you should expect! >> A path file contains paths to be added at interpreter startup to the >> package/module search path stored in sys.path. That is, in your example, >> if you put a file tau4.py or a tau4 directory with the __init__.py file >> inside into /home/poseidon/tau4/swr/py3/src, *then* you could import >> tau4. >> >>> It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the >>> site-packages dir: >>> >>> ln -s /home/poseidon/tau4/swr/py3/src >>> /usr/lib/python3.6/site-packages/tau4 >>> >> >> Well this works because now Python finds (following the symlink) a tau4 >> package (i.e., a directory with that name and an __init__.py file >> inside) in /usr/lib/python3.6/site-packages. The .pth file is not >> involved in this at all. >> > > Yes, removed it (symlink still there) and it still works. But then, what > are pth files for? I'd just place a symlink to the package and am done > with. The path doesn't seem to be needed in sys.path (where it would go > if placed in a pth file). If I write > > from tau4 import datalogging > > that works, too. So no need for the path being in sys.path (i.e. in a > pth file)? > I guess a major point of .pth files is that you only have one or a small number of files with a clear purpose polluting the containing directory. Of course, you could put symlinks to all your packages and modules into site-packages, but what's the point of putting them somewhere else in the first place? Also, you cannot create symlinks across devices, but .pth files will work. Best, Wolfgang From antoon.pardon at rece.vub.ac.be Wed Feb 15 08:37:23 2017 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 15 Feb 2017 14:37:23 +0100 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <58a4487c$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> <58a4487c$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1064aa40-072f-f8c4-2488-fd80b46f0c08@rece.vub.ac.be> Op 15-02-17 om 13:24 schreef Steve D'Aprano: > On Wed, 15 Feb 2017 09:49 pm, Antoon Pardon wrote: > >> Op 15-02-17 om 07:28 schreef Steven D'Aprano: > [...] >>> Why not use sys.stdout.write directly? Or print? If I saw somebody using >>> this recipe in production code, in the way shown, I'd refactor it to just >>> use print. There's no advantage to re-inventing the wheel this way. >> On reason to use this is for some easy "logging", you use echo to help >> in debugging and afterwards you can either define echo as an empty >> function or something easy to find to comment out. > In Python 3 you can always add > > def print(*args, **kw): > pass > > in your module to disable printing. But a better way, in my opinion, is to > use your editor to search for: > > print( > > and replace with: > > #print( You don't seem to understand, I don't want to disable all printing, only the diagnostics. That is easier to do if I use a different name for printing diagnostics than for regular I/O. Whether I disable it by defining an empty function or by commenting them out, doesn't really matter with this regard. > But even better, once you get to the point of putting print calls in more > than two or three places in your code, you should probably invest the time > to learn how to use the logging module. I know how to use the logging module. It is my experience that for a lot of rather small projects, the hassle of setting it up, is not worth it. YMMV. -- Antoon Pardon From duncan at invalid.invalid Wed Feb 15 10:03:53 2017 From: duncan at invalid.invalid (duncan smith) Date: Wed, 15 Feb 2017 15:03:53 +0000 Subject: Create ordering of points In-Reply-To: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> References: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> Message-ID: On 15/02/17 13:27, spiess.benjamin at googlemail.com wrote: > Hello !:) > I've got a problem which I would really like to solve. > I got a cloud of points (in the simplest example its a 2-dimensional cloud of points). > First, I want to set one of the points as the initial (or middle) point. Starting from there, the next points with the closest distance to the initial points are to be found. Afterwards, the closest points to these points are to be written out. These points shall be connected like it is depicted in this figure http://fs5.directupload.net/images/170215/b835zixm.jpg > > With this approach, the "fastest" path from every point to the defined initial point are specified by this pattern. > > Does somebody know a good technique for this problem? or can even give a hint to a existing python procedure? > I would be really grateful! > > > Regards > Benny > Maybe https://en.wikipedia.org/wiki/Shortest-path_tree? Duncan From p.oseidon at datec.at Wed Feb 15 10:34:07 2017 From: p.oseidon at datec.at (poseidon) Date: Wed, 15 Feb 2017 16:34:07 +0100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On 15/02/17 14:34, Wolfgang Maier wrote: > On 15.02.2017 13:42, poseidon wrote: >> On 15/02/17 12:16, Wolfgang Maier wrote: >>> On 15.02.2017 10:33, poseidon wrote: >>>> In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It >>>> contains >>>> the line >>>> >>>> /home/poseidon/tau4/swr/py3/src >>>> >>>> In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it >>>> should be possible to write >>>> >>>> import tau4 >>>> >>>> in my programs. >>> >>> >>> No, that's not what you should expect! >>> A path file contains paths to be added at interpreter startup to the >>> package/module search path stored in sys.path. That is, in your example, >>> if you put a file tau4.py or a tau4 directory with the __init__.py file >>> inside into /home/poseidon/tau4/swr/py3/src, *then* you could import >>> tau4. >>> >>>> It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the >>>> site-packages dir: >>>> >>>> ln -s /home/poseidon/tau4/swr/py3/src >>>> /usr/lib/python3.6/site-packages/tau4 >>>> >>> >>> Well this works because now Python finds (following the symlink) a tau4 >>> package (i.e., a directory with that name and an __init__.py file >>> inside) in /usr/lib/python3.6/site-packages. The .pth file is not >>> involved in this at all. >>> >> >> Yes, removed it (symlink still there) and it still works. But then, what >> are pth files for? I'd just place a symlink to the package and am done >> with. The path doesn't seem to be needed in sys.path (where it would go >> if placed in a pth file). If I write >> >> from tau4 import datalogging >> >> that works, too. So no need for the path being in sys.path (i.e. in a >> pth file)? >> > > I guess a major point of .pth files is that you only have one or a small > number of files with a clear purpose polluting the containing directory. > Of course, you could put symlinks to all your packages and modules into > site-packages, but what's the point of putting them somewhere else in > the first place? Also, you cannot create symlinks across devices, but > .pth files will work. Thank you, Wolfgang. Did the following: Put a symlink tau4 in /home/poseidon/tau4/swr/py3/src pointing to this directory. Removed the symlink in the site-packages. Restore the pth file tau4.pth containing the line /home/poseidon/tau4/swr/py3/src. Worx! Thank you for sheding light on this! Paul > > Best, > Wolfgang > From steve+python at pearwood.info Wed Feb 15 10:34:19 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 16 Feb 2017 02:34:19 +1100 Subject: PTH files: Abs paths not working as expected. Symlinks needed? References: Message-ID: <58a474fc$0$1586$c3e8da3$5496439d@news.astraweb.com> On Wed, 15 Feb 2017 11:42 pm, poseidon wrote: > Yes, removed it (symlink still there) and it still works. But then, what > are pth files for? Good question. I don't actually know anyone that uses pth files, so perhaps they're unnecessary. But the principle behind them is that they can be used to customize the search path Python uses for locating modules. There are (at least) six ways to customize this path: (1) Have your application import sys and modify sys.path on startup. (2) Set the environment variable PYTHONPATH. (3) For those building and distributing their own Python environment, e.g. Linux distros, you can customise the site.py file. (4) For site-wide customization, create a sitecustomize.py file in (for example) /usr/local/lib/python3.5/site-packages/ or the equivalent for your system, and modify sys.path. (5) For a per-user customization, create usercustomize.py in (for example) ~/.local/lib/python3.5/site-packages/ or the equivalent for your system. Again, import sys and modify sys.path. (6) Or place a pth file in one of the site-packages locations. See the documentation for the site module for more detail: https://docs.python.org/3/library/site.html > I'd just place a symlink to the package and am done > with. The path doesn't seem to be needed in sys.path (where it would go > if placed in a pth file). If I write > > from tau4 import datalogging > > that works, too. So no need for the path being in sys.path (i.e. in a > pth file)? Where are you placing the symlink? Python will follow symlinks, but they have to be in the PYTHONPATH to be found in the first place. And not all platforms support symlinks. (Windows, I think.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Wed Feb 15 10:39:06 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 15 Feb 2017 15:39:06 +0000 Subject: Create ordering of points References: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> Message-ID: <87fujfqx9h.fsf@bsb.me.uk> spiess.benjamin at googlemail.com writes: > I've got a problem which I would really like to solve. > I got a cloud of points (in the simplest example its a 2-dimensional > cloud of points). > First, I want to set one of the points as the initial (or middle) > point. Starting from there, the next points with the closest distance > to the initial points are to be found. Afterwards, the closest points > to these points are to be written out. These points shall be connected > like it is depicted in this figure > http://fs5.directupload.net/images/170215/b835zixm.jpg > > With this approach, the "fastest" path from every point to the defined > initial point are specified by this pattern. The fastest path will be a direct line -- every point connected to the root (you are building a tree). Are you perhaps looking for what is call the minimum spanning tree? This is not really a Python question. Some language Usenet groups and mailing lists (this is both) don't like general programming questions and some don't mind. I'm not sure what comp.lang.python's view is on these things, but groups like comp.programming and even comp.theory are, on the surface, better places to ask. Unfortunately they are full of cranks. though they can be ignored. -- Ben. From pavol.lisy at gmail.com Wed Feb 15 11:36:25 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 15 Feb 2017 17:36:25 +0100 Subject: Create ordering of points In-Reply-To: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> References: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> Message-ID: On 2/15/17, spiess.benjamin--- via Python-list wrote: > Hello !:) > I've got a problem which I would really like to solve. > I got a cloud of points (in the simplest example its a 2-dimensional cloud > of points). > First, I want to set one of the points as the initial (or middle) point. > Starting from there, the next points with the closest distance to the > initial points are to be found. Afterwards, the closest points to these > points are to be written out. These points shall be connected like it is > depicted in this figure > http://fs5.directupload.net/images/170215/b835zixm.jpg > > With this approach, the "fastest" path from every point to the defined > initial point are specified by this pattern. > > Does somebody know a good technique for this problem? or can even give a > hint to a existing python procedure? > I would be really grateful! Something like https://networkx.github.io/documentation/networkx-1.10/reference/algorithms.shortest_paths.html could help? Or maybe this https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.algorithms.mst.minimum_spanning_tree.html?highlight=minimum_spanning_tree ? PL. From g.starck at gmail.com Wed Feb 15 13:05:50 2017 From: g.starck at gmail.com (gst) Date: Wed, 15 Feb 2017 10:05:50 -0800 (PST) Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: <58a474fc$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <58a474fc$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: Le mercredi 15 f?vrier 2017 10:34:42 UTC-5, Steve D'Aprano a ?crit?: > On Wed, 15 Feb 2017 11:42 pm, poseidon wrote: > > > Yes, removed it (symlink still there) and it still works. But then, what > > are pth files for? > > > Good question. I don't actually know anyone that uses pth files, so perhaps > they're unnecessary. > when used with the "import " mechanism they can be used as a startup hook for instance, which can be pretty useful. From rosuav at gmail.com Wed Feb 15 13:51:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 16 Feb 2017 05:51:13 +1100 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <1064aa40-072f-f8c4-2488-fd80b46f0c08@rece.vub.ac.be> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> <58a4487c$0$1589$c3e8da3$5496439d@news.astraweb.com> <1064aa40-072f-f8c4-2488-fd80b46f0c08@rece.vub.ac.be> Message-ID: On Thu, Feb 16, 2017 at 12:37 AM, Antoon Pardon wrote: >> But a better way, in my opinion, is to >> use your editor to search for: >> >> print( >> >> and replace with: >> >> #print( > > You don't seem to understand, I don't want to disable all printing, only the > diagnostics. That is easier to do if I use a different name for printing > diagnostics than for regular I/O. Whether I disable it by defining an empty > function or by commenting them out, doesn't really matter with this regard. > >> But even better, once you get to the point of putting print calls in more >> than two or three places in your code, you should probably invest the time >> to learn how to use the logging module. > > I know how to use the logging module. It is my experience that for a lot > of rather small projects, the hassle of setting it up, is not worth it. > YMMV. Once you get to the point of saying either "but #print( might comment out only the first line, so I can't wrap it" or "but #print( would comment out too much, I only want to toggle some of the statements", it's time to set up logging. Also, if you're building a daemon that should play nicely with a larger system (eg a system process on Unix/Linux), the logging module makes that work a lot easier, since you can easily redirect it to a file or whatever you want. ChrisA From jladasky at itu.edu Wed Feb 15 14:17:41 2017 From: jladasky at itu.edu (jladasky at itu.edu) Date: Wed, 15 Feb 2017 11:17:41 -0800 (PST) Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: On Wednesday, February 15, 2017 at 2:52:55 AM UTC-8, Antoon Pardon wrote: > Op 15-02-17 om 07:28 schreef Steven D'Aprano: > > E.g. http://code.activestate.com/recipes/580750 > > > > does nothing more that define > > > > echo = sys.stdout.write > > > > Why not use sys.stdout.write directly? Or print? If I saw somebody using > > this recipe in production code, in the way shown, I'd refactor it to just > > use print. There's no advantage to re-inventing the wheel this way. > > On reason to use this is for some easy "logging", you use echo to help > in debugging and afterwards you can either define echo as an empty > function or something easy to find to comment out. > > -- > Antoon Pardon. Agreed, I have done things like this myself, but within functions, not at the global level. The "echo" name could be assigned to some other function elsewhere in the code. I use the name reassignment to direct the function's logging information -- to a console, to a GUI, or to "/dev/null" by creating an empty function. From oegeeks at gmail.com Wed Feb 15 15:15:36 2017 From: oegeeks at gmail.com (Andreas Paeffgen) Date: Wed, 15 Feb 2017 14:15:36 -0600 Subject: subprocess problem References: <20170211022312.GA24986@cskk.homeip.net> Message-ID: On 2017-02-11 02:23:12 +0000, Cameron Simpson said: > def your_function(.......): > with open('/path/to/your/logfile.txt', 'a') as logfp: > print("PATH=".os.environ['PATH'], file=logfp) > p=Popen(.......) > p.communicate(...) > print("p.returncode=%r" % (p.returncode)) > > and any other interesting values that occur to you. This is really simple, and > doesn't require a terminal or using the logging module. > > You can 'tail -f /path/to/your/logfile.txt' in another terminal to watch stuff > happen as you exercise the program. Thanks for the great advice. I found out, that the frozen app does not pass the normal path variable. All the tricks with passing the correct environment did not work. I just hardcoded the existing standard path. After that it worked. I think it is a problem with frozen apps in general. Cx_Freeze has other problems than Pyinstaller. But all need some workarounds for the problems. From dotancohen at gmail.com Wed Feb 15 16:34:55 2017 From: dotancohen at gmail.com (Dotan Cohen) Date: Wed, 15 Feb 2017 23:34:55 +0200 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: Message-ID: I think that we can help each other! This is my own code, back when I was writing Python like PHP: https://github.com/dotancohen/burton It is badly in need of a Pythonic refactor! Have at it, and I'll be happy to accept any pull requests. On Wed, Feb 15, 2017 at 12:44 AM, Makoto Kuwata wrote: > Hi, > > Is there any *just right* python code to refactor? > In other words, I'm finding bad code example in python. > > (background) > I'm teaching Python to some novice programmer, and > want to show refactoring example to them. > > (required) > * not good code > * not too large (for novice programmer) > * not too complex (for novice programmer) > * released under open source license > > If you know good material in github or bitbucket to refactor, > let me know it. > > -- > regards, > kwatch > -- > https://mail.python.org/mailman/listinfo/python-list -- Dotan Cohen http://gibberish.co.il http://what-is-what.com From python at lucidity.plus.com Wed Feb 15 16:53:33 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 15 Feb 2017 21:53:33 +0000 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> Hi, On 15/02/17 09:36, Makoto Kuwata wrote: > I'm sorry that my explanation is not enough. > I'm looking for bad python code, not refactoring examples. I think you need to explain what you mean by "bad". Do you mean something like: i = 0 data = ["bad", "example", "of", "python", "code"] while i < len(data): process(data[i]) i = i + 1 ... which could be better expressed as: data = ["good", "example", "of", "python", "code"] for d in data: process(data) ? If so, then you're asking for "unpythonic" code examples (code that is written in the style of a different language that doesn't have some of Python's syntactic sugar and underlying mechanisms). I suspect that you can find many short examples like the above on sites like StackOverflow, but finding a substantial code-base example (which is what I think you mean by "proper size") is unlikely, as anyone writing a large project in Python is almost certainly going to have learned and adopted the "pythonic" idioms long before their project became as large as you appear to be asking for. If you _don't_ mean what I suggested above, please explain further (Python code examples of what you think is "bad" vs "good" would be useful). E. From python at lucidity.plus.com Wed Feb 15 17:05:13 2017 From: python at lucidity.plus.com (Erik) Date: Wed, 15 Feb 2017 22:05:13 +0000 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> Message-ID: <60e5338d-9950-b43e-872d-8f97b8d56e3b@lucidity.plus.com> On 15/02/17 21:53, Erik wrote: > process(data) Before I get jumped on by a pack of rabid wolves, I of course meant: process(d) E. From ned at nedbatchelder.com Wed Feb 15 20:19:17 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 15 Feb 2017 17:19:17 -0800 (PST) Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: <4a7b0574-e3c9-45f5-8c82-123ccd897a52@googlegroups.com> On Wednesday, February 15, 2017 at 8:34:45 AM UTC-5, Wolfgang Maier wrote: > On 15.02.2017 13:42, poseidon wrote: > > On 15/02/17 12:16, Wolfgang Maier wrote: > >> On 15.02.2017 10:33, poseidon wrote: > >>> In /usr/lib/python3.6/site-packages I wrote a file tau4.pth. It contains > >>> the line > >>> > >>> /home/poseidon/tau4/swr/py3/src > >>> > >>> In /home/poseidon/tau4/swr/py3/src there's an __init__.py file, so it > >>> should be possible to write > >>> > >>> import tau4 > >>> > >>> in my programs. > >> > >> > >> No, that's not what you should expect! > >> A path file contains paths to be added at interpreter startup to the > >> package/module search path stored in sys.path. That is, in your example, > >> if you put a file tau4.py or a tau4 directory with the __init__.py file > >> inside into /home/poseidon/tau4/swr/py3/src, *then* you could import > >> tau4. > >> > >>> It works, if I set a symlink to /home/poseidon/tau4/swr/py3/src in the > >>> site-packages dir: > >>> > >>> ln -s /home/poseidon/tau4/swr/py3/src > >>> /usr/lib/python3.6/site-packages/tau4 > >>> > >> > >> Well this works because now Python finds (following the symlink) a tau4 > >> package (i.e., a directory with that name and an __init__.py file > >> inside) in /usr/lib/python3.6/site-packages. The .pth file is not > >> involved in this at all. > >> > > > > Yes, removed it (symlink still there) and it still works. But then, what > > are pth files for? I'd just place a symlink to the package and am done > > with. The path doesn't seem to be needed in sys.path (where it would go > > if placed in a pth file). If I write > > > > from tau4 import datalogging > > > > that works, too. So no need for the path being in sys.path (i.e. in a > > pth file)? > > > > I guess a major point of .pth files is that you only have one or a small > number of files with a clear purpose polluting the containing directory. > Of course, you could put symlinks to all your packages and modules into > site-packages, but what's the point of putting them somewhere else in > the first place? Also, you cannot create symlinks across devices, but > .pth files will work. Also, not all operating systems support symlinks. --Ned. From steve at pearwood.info Wed Feb 15 21:25:26 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 16 Feb 2017 02:25:26 GMT Subject: Crappy Python code of the day Message-ID: <58a50d96$0$2816$c3e8da3$76491128@news.astraweb.com> This has been in production for months, the writer of the code has left and the new maintainer has been asked to find out why it has been crashing with UnboundLocalError: try: result = future.result() except requests.exceptions.ConnectionError as e: pass resp = self.client.service.GetModifiedTerminals( __inject={'reply': result.content}) Traceback (most recent call last): [incriminating details removed for the protection of the guilty] UnboundLocalError: local variable 'result' referenced before assignment https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ -- Steve From no.email at nospam.invalid Wed Feb 15 22:08:59 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 15 Feb 2017 19:08:59 -0800 Subject: WANT: bad code in python (for refactoring example) References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> Message-ID: <87mvdmomr8.fsf@nightsong.com> Antoon Pardon writes: > On reason to use this is for some easy "logging" I think it's better to use the actual logging module. I generally start a new program with print statements but convert them to logging after there's enough code to want to be more organized about it. From no.email at nospam.invalid Wed Feb 15 22:15:20 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 15 Feb 2017 19:15:20 -0800 Subject: Create ordering of points References: <89058ce7-7645-439d-8a5a-57200d34cc75@googlegroups.com> Message-ID: <87inoaomgn.fsf@nightsong.com> spiess.benjamin at googlemail.com writes: > Does somebody know a good technique for this problem? or can even give > a hint to a existing python procedure? You're asking about nearest neighbor search, which is the topic of a huge literature: https://en.wikipedia.org/wiki/Nearest_neighbor_search From rosuav at gmail.com Wed Feb 15 22:28:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 16 Feb 2017 14:28:57 +1100 Subject: Crappy Python code of the day In-Reply-To: <58a50d96$0$2816$c3e8da3$76491128@news.astraweb.com> References: <58a50d96$0$2816$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Feb 16, 2017 at 1:25 PM, Steven D'Aprano wrote: > This has been in production for months, the writer of the code has left > and the new maintainer has been asked to find out why it has been > crashing with UnboundLocalError: > > > try: > result = future.result() > except requests.exceptions.ConnectionError as e: > pass > resp = self.client.service.GetModifiedTerminals( > __inject={'reply': result.content}) > > > Traceback (most recent call last): > [incriminating details removed for the protection of the guilty] > UnboundLocalError: local variable 'result' referenced before assignment > Without even looking at the link.... 'except pass' around an assignment. Unless there's a preceding "result = some-other-object", that's going to annoyingly fail. Maybe "except return None"? ChrisA From steve at pearwood.info Wed Feb 15 23:15:59 2017 From: steve at pearwood.info (Steven D'Aprano) Date: 16 Feb 2017 04:15:59 GMT Subject: Crappy Python code of the day References: <58a50d96$0$2816$c3e8da3$76491128@news.astraweb.com> Message-ID: <58a5277f$0$2816$c3e8da3$76491128@news.astraweb.com> On Thu, 16 Feb 2017 14:28:57 +1100, Chris Angelico wrote: > On Thu, Feb 16, 2017 at 1:25 PM, Steven D'Aprano > wrote: >> This has been in production for months, the writer of the code has left >> and the new maintainer has been asked to find out why it has been >> crashing with UnboundLocalError: [...] > Without even looking at the link.... 'except pass' around an assignment. > Unless there's a preceding "result = some-other-object", > that's going to annoyingly fail. Maybe "except return None"? Oh, we know why the code is failing. We don't need help diagnosing the UnboundLocalError exception. You're right: there's an except pass around an assignment, so if the assignment fails, `result` never gets set. But the real WTF is that the ConnectionError is just thrown away. There's no attempt to recover from it, or log it, or try connecting again... the end result is that the application dies with an unhelpful UnboundLocalError, and (until today) we had no idea what the actual cause of the failure was. [Name changed to protect the guilty] Thanks Aloysius!!! -- Steve From alister.ware at ntlworld.com Thu Feb 16 04:43:29 2017 From: alister.ware at ntlworld.com (alister) Date: Thu, 16 Feb 2017 09:43:29 GMT Subject: WANT: bad code in python (for refactoring example) Message-ID: <5vepA.41985$bt2.1855@fx40.am4> On Wed, 15 Feb 2017 19:08:59 -0800, Paul Rubin wrote: > Antoon Pardon writes: >> On reason to use this is for some easy "logging" > > I think it's better to use the actual logging module. I generally start > a new program with print statements but convert them to logging after > there's enough code to want to be more organized about it. my default python template file includes basic logging setup from the start (all 3 lines of it ) -- "...if the church put in half the time on covetousness that it does on lust, this would be a better world." - Garrison Keillor, "Lake Wobegon Days" From Cecil at decebal.nl Thu Feb 16 06:00:36 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Thu, 16 Feb 2017 12:00:36 +0100 Subject: WANT: bad code in python (for refactoring example) References: <5vepA.41985$bt2.1855@fx40.am4> Message-ID: <87lgt6wgbv.fsf@Equus.decebal.nl> On Thursday 16 Feb 2017 10:43 CET, alister wrote: > On Wed, 15 Feb 2017 19:08:59 -0800, Paul Rubin wrote: > >> Antoon Pardon writes: >>> On reason to use this is for some easy "logging" >> >> I think it's better to use the actual logging module. I generally >> start a new program with print statements but convert them to >> logging after there's enough code to want to be more organized >> about it. > > my default python template file includes basic logging setup from > the start (all 3 lines of it ) You should have shared your template, or at least those three lines. :-P -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From alister.ware at ntlworld.com Thu Feb 16 09:18:17 2017 From: alister.ware at ntlworld.com (alister) Date: Thu, 16 Feb 2017 14:18:17 GMT Subject: WANT: bad code in python (for refactoring example) References: <5vepA.41985$bt2.1855@fx40.am4> <87lgt6wgbv.fsf@Equus.decebal.nl> Message-ID: On Thu, 16 Feb 2017 12:00:36 +0100, Cecil Westerhof wrote: > On Thursday 16 Feb 2017 10:43 CET, alister wrote: > >> On Wed, 15 Feb 2017 19:08:59 -0800, Paul Rubin wrote: >> >>> Antoon Pardon writes: >>>> On reason to use this is for some easy "logging" >>> >>> I think it's better to use the actual logging module. I generally >>> start a new program with print statements but convert them to logging >>> after there's enough code to want to be more organized about it. >> >> my default python template file includes basic logging setup from the >> start (all 3 lines of it ) > > You should have shared your template, or at least those three lines. > :-P import logging logger=logging.getLogger() logging.basicConfig(level=logging.DEBUG) now simply add logger.debug('debug message') wherever needed instead of print statements alternatively import logging logger=logging.getLogger() logging.basicConfig(filename='logfile',level=logging.DEBUG) if you would prefer to log to a file instead. simples :-) -- A New Way of Taking Pills A physician one night in Wisconsin being disturbed by a burglar, and having no ball or shot for his pistol, noiselessly loaded the weapon with small, hard pills, and gave the intruder a "prescription" which he thinks will go far towards curing the rascal of a very bad ailment. -- Nevada Morning Transcript, January 30, 1861 From tjreedy at udel.edu Thu Feb 16 16:53:55 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 16 Feb 2017 16:53:55 -0500 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On 2/15/2017 7:42 AM, poseidon wrote: > what are pth files for? They are for extending (mainly) lib/site-packages. To repeat what I have posted before: every time I install a new version of Python, I add (copy) python.pth containing 'F:/python' (without the quotes). This makes my directory of modules and packages, F:/python, an extension of the new site-packages. In effect, the .pth file 'installs', all at once, all my personal projects in the new version site-packages. This is done without symlinking each individual project directory. It is also done *without* uninstalling projects from previous versions. As a result, files can be run with -m from a command line just as they could be when in site-packages. For instance, 'py -3.6 -m xploro' runs F:/python/xploro/__main__ with python 3.6. And I can replace '3.6' with 3.5 or 2.7' I can also use absolute imports within a project/package the same as if the xploro directory were in each versions site-packages. For instance, 'from xplore import test' imports F:/python/xplore/test. There is no hassle with relative imports, and if I ever put the package on PyPI, no change to imports will be needed. I just verified that dropping python.pth into the site-packages directory of my locally built python 3.7.0a0 lets me easily test that xploro tests run on the next version of python. -- Terry Jan Reedy From kwa at kuwata-lab.com Thu Feb 16 21:03:29 2017 From: kwa at kuwata-lab.com (Makoto Kuwata) Date: Fri, 17 Feb 2017 11:03:29 +0900 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> Message-ID: Thanks Erik, On Thu, Feb 16, 2017 at 6:53 AM, Erik wrote: > (which is what I think you mean by "proper size") > As I explained at my first post, I'm teaching Python to novice programmers. Therefore refactoring example code should be in the size what they can understand. It is hard for them to read and understand over than thousand lines of code. Almost of all projects have code over than thousand lines of code, but it is possible to quote a part of them for refactoring example, I think. > > (Python code examples of what you think is "bad" vs "good" would be > useful). > You are right. Bad code Example: # https://codewords.recurse.com/issues/one/an-introduction-to-functional-programming from random import random def move_cars(car_positions): return map(lambda x: x + 1 if random() > 0.3 else x, car_positions) def output_car(car_position): return '-' * car_position def run_step_of_race(state): return {'time': state['time'] - 1, 'car_positions': move_cars(state['car_positions'])} def draw(state): print '' print '\n'.join(map(output_car, state['car_positions'])) def race(state): draw(state) if state['time']: race(run_step_of_race(state)) race({'time': 5, 'car_positions': [1, 1, 1]}) Refactoring example: from random import random class Car(object): def __init__(self): self.position = 1 def move(self): if random() > 0.3: self.position += 1 return self.position class Race(object): def __init__(self, n_cars=3): self._cars = [ Car() for _ in range(n_cars) ] def round(self): for car in self._cars: car.move() def report(self): print("") for car in self._cars: print('-' * car.position) def run(self, n_rounds=5): self.report() for _ in range(n_rounds): self.round() self.report() if __name__ == '__main__': Race(3).run(5) -- regards, makoto kuwata From rustompmody at gmail.com Thu Feb 16 21:55:23 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 16 Feb 2017 18:55:23 -0800 (PST) Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: On Friday, February 17, 2017 at 3:24:32 AM UTC+5:30, Terry Reedy wrote: > On 2/15/2017 7:42 AM, poseidon wrote: > > > what are pth files for? > > They are for extending (mainly) lib/site-packages. Hey Terry! This needs to get into more public docs than a one-off post on a newsgroup/ML From jussi.piitulainen at helsinki.fi Fri Feb 17 04:07:18 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 17 Feb 2017 11:07:18 +0200 Subject: WANT: bad code in python (for refactoring example) References: <58a3f50d$0$2776$c3e8da3$76491128@news.astraweb.com> <704fd0c8-0138-110e-5693-6f678625c952@lucidity.plus.com> Message-ID: Makoto Kuwata writes: > On Thu, Feb 16, 2017 at 6:53 AM, Erik wrote: >> >> (Python code examples of what you think is "bad" vs "good" would be >> useful). > > You are right. > > Bad code Example: > > # > https://codewords.recurse.com/issues/one/an-introduction-to-functional-programming > > from random import random > > def move_cars(car_positions): > return map(lambda x: x + 1 if random() > 0.3 else x, > car_positions) > > def output_car(car_position): > return '-' * car_position > > def run_step_of_race(state): > return {'time': state['time'] - 1, > 'car_positions': move_cars(state['car_positions'])} > > def draw(state): > print '' > print '\n'.join(map(output_car, state['car_positions'])) > > def race(state): > draw(state) > if state['time']: > race(run_step_of_race(state)) > > race({'time': 5, > 'car_positions': [1, 1, 1]}) Here's a rewrite in functional style, which I consider somewhat better than the original: from random import random def move(positions): return [ pos + bool(random() > 0.3) for pos in positions ] def race(positions, steps): for step in range(steps): positions = move(positions) yield positions def draw(positions): print(*('-' * pos for pos in positions), sep = '\n') if __name__ == '__main__': for step, positions in enumerate(race([1,1,1], 5)): step and print() draw(positions) I did a number of things, but mainly I reconceptualized the race itself explicitly as a sequence of positions. While a Python generator is an extremely stateful object, with some care it works well in functional style. > Refactoring example: > > from random import random > > class Car(object): > > def __init__(self): > self.position = 1 > > def move(self): > if random() > 0.3: > self.position += 1 > return self.position > > class Race(object): > > def __init__(self, n_cars=3): > self._cars = [ Car() for _ in range(n_cars) ] > > def round(self): > for car in self._cars: > car.move() > > def report(self): > print("") > for car in self._cars: > print('-' * car.position) > > def run(self, n_rounds=5): > self.report() > for _ in range(n_rounds): > self.round() > self.report() > > if __name__ == '__main__': > Race(3).run(5) If you want to refactor bad code into better code, it would be more honest to start with code that is already in your preferred style. That would be a good lesson. Now you've taken questionable code in a somewhat functional style and refactored it into object-oriented style. But you don't call them that. You call them bad code and good code. That's a bad lesson. It conflates issues. Take good functional code, refactor it into your preferred style. Also do the reverse. That would be a good lesson, assuming your students are ready for such discussion. From kwa at kuwata-lab.com Fri Feb 17 08:47:57 2017 From: kwa at kuwata-lab.com (Makoto Kuwata) Date: Fri, 17 Feb 2017 22:47:57 +0900 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: Message-ID: On Thu, Feb 16, 2017 at 6:34 AM, Dotan Cohen wrote: > I think that we can help each other! This is my own code, back when I > was writing Python like PHP: > https://github.com/dotancohen/burton Year, this is a good example code for me to do refactoring. I created a pull request. This will be shown to my students. https://github.com/dotancohen/burton/pull/20/files I hope you favor this PR. -- regards, makoto kuwata From cfkaran2 at gmail.com Sat Feb 18 07:50:19 2017 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 18 Feb 2017 07:50:19 -0500 Subject: PTH files: Abs paths not working as expected. Symlinks needed? In-Reply-To: References: Message-ID: <5E1F28B6-7276-4354-A062-24F56A3C4771@gmail.com> On Feb 16, 2017, at 9:55 PM, Rustom Mody wrote: > On Friday, February 17, 2017 at 3:24:32 AM UTC+5:30, Terry Reedy wrote: >> On 2/15/2017 7:42 AM, poseidon wrote: >> >>> what are pth files for? >> >> They are for extending (mainly) lib/site-packages. > > > Hey Terry! > This needs to get into more public docs than a one-off post on a newsgroup/ML +1! This is the first I've heard of this, and it sounds INCREDIBLY useful! Thanks, Cem Karan From tfente at gmail.com Sat Feb 18 12:38:32 2017 From: tfente at gmail.com (TTaglo) Date: Sat, 18 Feb 2017 09:38:32 -0800 (PST) Subject: print odd numbers of lines from tekst WITHOUT space between lines Message-ID: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> i = 1 f = open ('rosalind_ini5(1).txt') for line in f.readlines(): if i % 2 == 0: print line i += 1 How do i get output without breaks between the lines? Result: Other things just make you swear and curse When you're chewing on life's gristle, don't grumble give a whistle This will help things turn out for the best Always look on the bright side of life From robertvstepp at gmail.com Sat Feb 18 12:55:34 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 18 Feb 2017 11:55:34 -0600 Subject: print odd numbers of lines from tekst WITHOUT space between lines In-Reply-To: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: On Sat, Feb 18, 2017 at 11:38 AM, TTaglo wrote: > i = 1 > f = open ('rosalind_ini5(1).txt') > for line in f.readlines(): > if i % 2 == 0: > print line > i += 1 > > > How do i get output without breaks between the lines? If you use "print line," (Note the trailing comma.), this should suppress the line break that the print statement normally inserts. Another suggestion might be to use enumerate() instead of using a manual counter in your for loop. HTH! -- boB From alister.ware at ntlworld.com Sat Feb 18 12:59:40 2017 From: alister.ware at ntlworld.com (alister) Date: Sat, 18 Feb 2017 17:59:40 GMT Subject: print odd numbers of lines from tekst WITHOUT space between lines References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: On Sat, 18 Feb 2017 11:55:34 -0600, boB Stepp wrote: > On Sat, Feb 18, 2017 at 11:38 AM, TTaglo wrote: >> i = 1 f = open ('rosalind_ini5(1).txt') >> for line in f.readlines(): >> if i % 2 == 0: >> print line >> i += 1 >> >> >> How do i get output without breaks between the lines? > > If you use "print line," (Note the trailing comma.), this should > suppress the line break that the print statement normally inserts. > > Another suggestion might be to use enumerate() instead of using a manual > counter in your for loop. > > HTH! or simply strip the cr/lf of the end of each line -- Smell from unhygenic janitorial staff wrecked the tape heads From best_lay at yahoo.com Sat Feb 18 13:03:24 2017 From: best_lay at yahoo.com (Wildman) Date: Sat, 18 Feb 2017 12:03:24 -0600 Subject: print odd numbers of lines from tekst WITHOUT space between lines References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: On Sat, 18 Feb 2017 09:38:32 -0800, TTaglo wrote: > i = 1 > f = open ('rosalind_ini5(1).txt') > for line in f.readlines(): > if i % 2 == 0: > print line > i += 1 > > > How do i get output without breaks between the lines? > > Result: > > Other things just make you swear and curse > > When you're chewing on life's gristle, don't grumble give a whistle > > This will help things turn out for the best > > Always look on the bright side of life In Python 3 you can do this: print(line, end="") For Python 2 use this: import sys . . . sys.stdout.write(line) Don' forget... f.close() -- GNU/Linux user #557453 The cow died so I don't need your bull! From daniel.4ndersson at gmail.com Sat Feb 18 13:29:23 2017 From: daniel.4ndersson at gmail.com (Daniel Andersson) Date: Sat, 18 Feb 2017 10:29:23 -0800 (PST) Subject: How to inherit classmethod constructors? Message-ID: <692e13cf-a1be-479f-9c7d-6eb38916567d@googlegroups.com> I have a class "A" below, that have a very simple __init__ method, which only wants a "needle". class A: def __init__(self, needle): self.needle = needle @classmethod def from_haystack(cls, haystack): # ... # Many lines of code (omitted here) # that process haystack to obtain a needle # ... # ... # The obtained needle is then used to # return an instance return cls(needle) The class also has a classmethod: "from_haystack" that is used as an alternative to __init__ for initalizing an instance. My problem is that: 1. I'm not allowed to refactor A. 2. I need a class B which must inherit A. 3. B must use a haystack to initialize itself. 4. Therefore, I want to reuse "from_haystack" from A in some way so I don't have to repeat the code. My naive solution was this: class B(A): def __init__(self, haystack, spam, eggs): super().from_haystack(haystack) # Won't work self.spam = spam self.eggs = eggs The use of super() here will call "from_haystack" with cls = B, which won't work, since the return line will be "return B(needle)" and B can't handle that. What is idiomatic solution to this problem? Thanks, From breamoreboy at gmail.com Sat Feb 18 14:50:52 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 18 Feb 2017 11:50:52 -0800 (PST) Subject: print odd numbers of lines from tekst WITHOUT space between lines In-Reply-To: References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: <62d28ab3-7f3e-4f72-a2a1-bd03a746da1b@googlegroups.com> On Saturday, February 18, 2017 at 6:03:37 PM UTC, Wildman wrote: > On Sat, 18 Feb 2017 09:38:32 -0800, TTaglo wrote: > > > i = 1 > > f = open ('rosalind_ini5(1).txt') > > for line in f.readlines(): > > if i % 2 == 0: > > print line > > i += 1 > > > > > > How do i get output without breaks between the lines? > > > > Result: > > > > Other things just make you swear and curse > > > > When you're chewing on life's gristle, don't grumble give a whistle > > > > This will help things turn out for the best > > > > Always look on the bright side of life > > In Python 3 you can do this: > > print(line, end="") > > For Python 2 use this: > > import sys > . > . > . > sys.stdout.write(line) > > Don' forget... > f.close() > > -- > GNU/Linux user #557453 > The cow died so I don't need your bull! For Python 2, strictly from memory:- from __future__ import print_function print(line, end="") Kindest regards. Mark Lawrence. From tfente at gmail.com Sat Feb 18 15:07:48 2017 From: tfente at gmail.com (TTaglo) Date: Sat, 18 Feb 2017 12:07:48 -0800 (PST) Subject: print odd numbers of lines from tekst WITHOUT space between lines In-Reply-To: References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: Op zaterdag 18 februari 2017 18:55:46 UTC+1 schreef boB Stepp: > On Sat, Feb 18, 2017 at 11:38 AM, TTaglo wrote: > > i = 1 > > f = open ('rosalind_ini5(1).txt') > > for line in f.readlines(): > > if i % 2 == 0: > > print line > > i += 1 > > > > > > How do i get output without breaks between the lines? > > If you use "print line," (Note the trailing comma.), this should > suppress the line break that the print statement normally inserts. > > Another suggestion might be to use enumerate() instead of using a > manual counter in your for loop. > > HTH! > > > > -- > boB thanks for the help ! From tfente at gmail.com Sat Feb 18 15:08:16 2017 From: tfente at gmail.com (TTaglo) Date: Sat, 18 Feb 2017 12:08:16 -0800 (PST) Subject: print odd numbers of lines from tekst WITHOUT space between lines In-Reply-To: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: Op zaterdag 18 februari 2017 18:38:43 UTC+1 schreef TTaglo: > i = 1 > f = open ('rosalind_ini5(1).txt') > for line in f.readlines(): > if i % 2 == 0: > print line > i += 1 > > > How do i get output without breaks between the lines? > > Result: > > Other things just make you swear and curse > > When you're chewing on life's gristle, don't grumble give a whistle > > This will help things turn out for the best > > Always look on the bright side of life thank you all! From __peter__ at web.de Sat Feb 18 15:20:41 2017 From: __peter__ at web.de (Peter Otten) Date: Sat, 18 Feb 2017 21:20:41 +0100 Subject: How to inherit classmethod constructors? References: <692e13cf-a1be-479f-9c7d-6eb38916567d@googlegroups.com> Message-ID: Daniel Andersson wrote: > > I have a class "A" below, that have a very simple > __init__ method, which only wants a "needle". > > > class A: > def __init__(self, needle): > self.needle = needle > > @classmethod > def from_haystack(cls, haystack): > # ... > # Many lines of code (omitted here) > # that process haystack to obtain a needle > # ... > # ... > # The obtained needle is then used to > # return an instance > return cls(needle) > > > The class also has a classmethod: "from_haystack" > that is used as an alternative to __init__ for > initalizing an instance. > > My problem is that: > 1. I'm not allowed to refactor A. > 2. I need a class B which must inherit A. > 3. B must use a haystack to initialize > itself. > 4. Therefore, I want to reuse "from_haystack" from A > in some way so I don't have to repeat the code. > > My naive solution was this: > > > class B(A): > def __init__(self, haystack, spam, eggs): > super().from_haystack(haystack) # Won't work > self.spam = spam > self.eggs = eggs > > > The use of super() here will call "from_haystack" with > cls = B, which won't work, since the return line will be > "return B(needle)" and B can't handle that. > > What is idiomatic solution to this problem? If A is a heap class you can change the class after the from_haystack() invocation: class B(A): def __new__(cls, needle, spam, eggs): inst = A.from_haystack(needle) inst.__class__ = cls return inst def __init__(self, needle, spam, eggs): super().__init__(needle) self.spam = spam self.eggs = eggs From pavel.aronsky at gmail.com Sat Feb 18 15:38:12 2017 From: pavel.aronsky at gmail.com (ddbug) Date: Sat, 18 Feb 2017 12:38:12 -0800 (PST) Subject: How to access installed scripts on Windows? Message-ID: I am very perplexed by inability to tell the Windows installer (bdist_wininst or pip) where to install scripts (or "entry points"). By default (and I don't see other options) scripts go to %USERPROFILE%/Appdata/Roaming/Python/Scripts. Ok. Now what? How the user is going to call these scripts: add this location to PATH? Then, should I add a custom script for this into the bdist_wininst installer? It looks like the "py" launcher does not help to find scripts that are installed into that per-user location. What is the "best known good" method to easily run scripts installed in per-user location, from command line (aka "dos window") ? Thanks, Pavel A From __peter__ at web.de Sat Feb 18 15:41:54 2017 From: __peter__ at web.de (Peter Otten) Date: Sat, 18 Feb 2017 21:41:54 +0100 Subject: print odd numbers of lines from tekst WITHOUT space between lines References: <8d354598-3cce-43d8-8fb2-e707d7ae9573@googlegroups.com> Message-ID: TTaglo wrote: > i = 1 > f = open ('rosalind_ini5(1).txt') > for line in f.readlines(): > if i % 2 == 0: > print line > i += 1 > > > How do i get output without breaks between the lines? I'm late to the show, but here are a few unsolicited remarks to your code. (1) The best way to open a file is with open(filename) as f: # use the file # at this point the file is closed without the need to invoke # f.close() explicitly (2) The readlines() method reads the complete file into a list. This can become a problem if the file is large. It is better to iterate over the file directly: for line in f: # use the line (3) If you read the file into a list you can get all odd lines with for odd_line in f.readlines()[1::2]: # use the odd line (4) If you follow my advice from (2) and want to avoid the in-memory list there's an equivalent itertools.islice() function that you can use like this: import itertools for odd_line in itertools.islice(f, 1, None, 2): # use the odd line (5) sys.stdout.write() was already mentioned, but there is also a sys.stdout.writelines() meathod which takes a sequence of strings. With that your code may become import itertools import sys with open('rosalind_ini5(1).txt') as f: sys.stdout.writelines(itertools.islice(f, 1, None, 2)) From arequipeno at gmail.com Sat Feb 18 17:47:29 2017 From: arequipeno at gmail.com (Ian Pilcher) Date: Sat, 18 Feb 2017 16:47:29 -0600 Subject: Noob confused by ConfigParser defaults Message-ID: I am trying to use ConfigParser for the first time (while also writing my first quasi-serious Python program). Assume that I want to parse a a configuration file of the following form: [section-1] option-1 = value1 option-2 = value2 [section-2] option-1 = value3 option-2 = value4 How do a set a default for option-1 *only* in section-1? -- ======================================================================== Ian Pilcher arequipeno at gmail.com -------- "I grew up before Mark Zuckerberg invented friendship" -------- ======================================================================== From eryksun at gmail.com Sat Feb 18 19:40:16 2017 From: eryksun at gmail.com (eryk sun) Date: Sun, 19 Feb 2017 00:40:16 +0000 Subject: How to access installed scripts on Windows? In-Reply-To: References: Message-ID: On Sat, Feb 18, 2017 at 8:38 PM, ddbug wrote: > I am very perplexed by inability to tell the Windows installer (bdist_wininst or pip) where to > install scripts (or "entry points"). > > By default (and I don't see other options) scripts go to > %USERPROFILE%/Appdata/Roaming/Python/Scripts. That's the scripts directory that's used by the --user setup in 2.7 and formerly in 3.x. Python 3.5+ uses the "PythonXY" subdirectory instead, e.g. "%AppData%\Python\Python35\Scripts". Note that you should use "%AppData%" for the user's roaming application data because the directory is relocatable. The --user setup is rarely used when installing packages. Typically packages are installed in the main site-packages and Scripts directories. The installer has an option to update PATH to include this Scripts directory. You can also develop using venv virtual environments. You can symlink or shell-shortcut to the activation script of a virtual environment. > the "py" launcher does not help to find scripts that are installed into that per-user location. I dislike the idea of automatically searching script directories, but there could be a command-line option for this. > command line (aka "dos window") ? The cmd shell is called the Command Prompt. It's a console application used on NT versions of Windows (e.g. Windows XP to 10). Older systems that extended DOS (e.g. Windows 98) used a DOS window and the classic COMMAND.COM shell. From dotancohen at gmail.com Sun Feb 19 14:43:52 2017 From: dotancohen at gmail.com (Dotan Cohen) Date: Sun, 19 Feb 2017 21:43:52 +0200 Subject: WANT: bad code in python (for refactoring example) In-Reply-To: References: Message-ID: There are some nice changes in here. You can mention to the students that I really appreciated the work done on the menu loop. You took an example of code that had grown piecemeal and become reasonably unmaintainable, and refactored it to be very maintainable in the future. Good luck to all class participants! On Fri, Feb 17, 2017 at 3:47 PM, Makoto Kuwata wrote: > On Thu, Feb 16, 2017 at 6:34 AM, Dotan Cohen wrote: > >> I think that we can help each other! This is my own code, back when I >> was writing Python like PHP: >> https://github.com/dotancohen/burton > > > Year, this is a good example code for me to do refactoring. > > I created a pull request. This will be shown to my students. > > https://github.com/dotancohen/burton/pull/20/files > > I hope you favor this PR. > > -- > regards, > makoto kuwata > -- > https://mail.python.org/mailman/listinfo/python-list -- Dotan Cohen http://gibberish.co.il http://what-is-what.com From python at deborahswanson.net Sun Feb 19 22:01:07 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sun, 19 Feb 2017 19:01:07 -0800 Subject: Python application launcher (for Python code) Message-ID: <003701d28b25$9579bd40$27b23dae@sambora> I could probably write this myself, but I'm wondering if this hasn't already been done many times. Anyone have some git links or other places to download from? From ben+python at benfinney.id.au Mon Feb 20 02:27:27 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 20 Feb 2017 18:27:27 +1100 Subject: Python application launcher (for Python code) References: <003701d28b25$9579bd40$27b23dae@sambora> Message-ID: <8560k5mie8.fsf@benfinney.id.au> "Deborah Swanson" writes: > I could probably write this myself, but I'm wondering if this hasn't > already been done many times. Can you describe what you are looking for, in enough detail that we can know whether it's already been done as you want it? -- \ ?God forbid that any book should be banned. The practice is as | `\ indefensible as infanticide.? ?Dame Rebecca West | _o__) | Ben Finney From ben+python at benfinney.id.au Mon Feb 20 02:39:40 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 20 Feb 2017 18:39:40 +1100 Subject: Noob confused by ConfigParser defaults References: Message-ID: <851sutmhtv.fsf@benfinney.id.au> Ian Pilcher writes: > How do a set a default for option-1 *only* in section-1? I think you misinderstand the semantics of what ?configparser? expects : Default values [?] are used in interpolation if an option used is not defined elsewhere. When default_section is given, it specifies the name for the special section holding default values for other sections and interpolation purposes (normally named "DEFAULT"). The default values are a special pseudo-section that holds defaults for the named options *anywhere else* in the configuration. It's not done per-section, it's done for the entire configuration input. This is IIUC because of the long-standing semantics of the format that inspired the ConfigParser behaviour. -- \ ?Software patents provide one more means of controlling access | `\ to information. They are the tool of choice for the internet | _o__) highwayman.? ?Anthony Taylor | Ben Finney From __peter__ at web.de Mon Feb 20 04:45:12 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 20 Feb 2017 10:45:12 +0100 Subject: Noob confused by ConfigParser defaults References: Message-ID: Ian Pilcher wrote: > I am trying to use ConfigParser for the first time (while also writing > my first quasi-serious Python program). Assume that I want to parse a > a configuration file of the following form: > > [section-1] > option-1 = value1 > option-2 = value2 > > [section-2] > option-1 = value3 > option-2 = value4 > > How do a set a default for option-1 *only* in section-1? You can provide a default value in your code with parser = configparser.ConfigParser() parser.read(configfile) value = parser.get("section-1", "option-1", fallback="default value") Or you can invent your own convention that the default values for "section-1" are found in "section-1-defaults", and then look it up with a small helper function: $ cat cpdemo.ini [section-1-defaults] option-2=default2 [section-1] option-1 = value1 [section-2] option-1 = value2 $ cat cpdemo.py import configparser parser = configparser.ConfigParser() parser.read("cpdemo.ini") def get(section, option, **kw): try: return parser.get(section, option) except configparser.NoOptionError: try: return parser.get(section + "-defaults", option, **kw) except (configparser.NoSectionError, configparser.NoOptionError): pass raise for section in ["section-1", "section-2"]: for option in ["option-1", "option-2"]: value = get(section, option, fallback="#missing") print("{}/{} = {}".format(section, option, value)) $ python3 cpdemo.py section-1/option-1 = value1 section-1/option-2 = default2 section-2/option-1 = value2 section-2/option-2 = #missing From grant.b.edwards at gmail.com Mon Feb 20 10:36:43 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 20 Feb 2017 15:36:43 +0000 (UTC) Subject: Python application launcher (for Python code) References: <003701d28b25$9579bd40$27b23dae@sambora> Message-ID: On 2017-02-20, Deborah Swanson wrote: > I could probably write this myself, but I'm wondering if this hasn't > already been done many times. Yes, it has. > Anyone have some git links or other places to download from? See the "website" and "repository" links on the pages below: https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29 https://en.wikipedia.org/wiki/Bourne_shell https://en.wikipedia.org/wiki/KornShell https://en.wikipedia.org/wiki/C_shell https://en.wikipedia.org/wiki/Almquist_shell -- Grant Edwards grant.b.edwards Yow! I was making donuts at and now I'm on a bus! gmail.com From python at deborahswanson.net Mon Feb 20 10:44:27 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 07:44:27 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <8560k5mie8.fsf@benfinney.id.au> Message-ID: <003f01d28b90$385d8450$27b23dae@sambora> Ben Finney wrote, on February 19, 2017 11:27 PM > > "Deborah Swanson" writes: > > > I could probably write this myself, but I'm wondering if this hasn't > > already been done many times. > > Can you describe what you are looking for, in enough detail > that we can know whether it's already been done as you want it? > > -- > \ "God forbid that any book should be banned. The practice is as | > `\ indefensible as infanticide." >-Dame Rebecca West | > _o__) | > Ben Finney I deliberately left the question open-ended because I'm curious what's out there. I've studied and practiced Python for a little over a year, but I've spent that time mostly writing my own code and I don't really know much about what and where to look for in modules and packages. Basically, I now have quite a few Python programs I use frequently, and as time goes on my collection and uses of it will grow. Right now I just want a way to select which one I'd like to run and run it. I'd like it to be a standalone application and some sort of system of categories would be nice. I'm migrating tasks I've always done in Excel to Python, and I have a sketchy idea of features I'd like to open Excel with, but I hate Excel VBA so much that I haven't written an on_Open macro for Excel yet. What I'd like to open with is mostly a menu of macros I'd like to have available for any code I'm running, possibly opening different environments for different kinds of tasks, that sort of thing. I also plan to use sqlite3 for permanent data storage, matplotlib for charts, and tkinter for interfaces. That's all in the planning stages, but one thing that seems like an obvious need is a way to keep related code and its associated data, charts, etc, easily accessible to each other, like they are when they're all bundled together in an Excel workbook. I have a few ideas about how to do that, but I'm also interested in what other people have done. I probably won't know exactly what I want until I have one and use it for awhile. I've been keeping my code for daily computing open in my IDE and using the IDE for a launcher, but it's getting a little crowded, and I'd like to access those bits separately from code I'm currently working on. Deborah From arequipeno at gmail.com Mon Feb 20 10:59:48 2017 From: arequipeno at gmail.com (Ian Pilcher) Date: Mon, 20 Feb 2017 09:59:48 -0600 Subject: Noob confused by ConfigParser defaults In-Reply-To: <851sutmhtv.fsf@benfinney.id.au> References: <851sutmhtv.fsf@benfinney.id.au> Message-ID: On 02/20/2017 01:39 AM, Ben Finney wrote: > I think you misinderstand the semantics of what ?configparser? expects > : You are absolutely correct. Thank you! -- ======================================================================== Ian Pilcher arequipeno at gmail.com -------- "I grew up before Mark Zuckerberg invented friendship" -------- ======================================================================== From arequipeno at gmail.com Mon Feb 20 11:00:25 2017 From: arequipeno at gmail.com (Ian Pilcher) Date: Mon, 20 Feb 2017 10:00:25 -0600 Subject: Noob confused by ConfigParser defaults In-Reply-To: References: Message-ID: On 02/20/2017 03:45 AM, Peter Otten wrote: > You can provide a default value in your code with > > parser = configparser.ConfigParser() > parser.read(configfile) > value = parser.get("section-1", "option-1", fallback="default value") Perfect. Thank you! -- ======================================================================== Ian Pilcher arequipeno at gmail.com -------- "I grew up before Mark Zuckerberg invented friendship" -------- ======================================================================== From python at deborahswanson.net Mon Feb 20 11:05:04 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 08:05:04 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <003f01d28b90$385d8450$27b23dae@sambora> Message-ID: <004401d28b93$19d79d60$27b23dae@sambora> I should also say that right now I'm using Windows XP, but hope very soon to have Linux again. Ideally, this launcher would work in both. I wrote, on February 20, 2017 7:44 AM > > Ben Finney wrote, on February 19, 2017 11:27 PM > > > > "Deborah Swanson" writes: > > > > > I could probably write this myself, but I'm wondering if > this hasn't > > > > already been done many times. > > > > Can you describe what you are looking for, in enough detail > > that we can know whether it's already been done as you want it? > > > > -- > > \ "God forbid that any book should be banned. The > practice is as > | > > `\ indefensible as infanticide." > >-Dame Rebecca West | > > _o__) > | > > Ben Finney > > I deliberately left the question open-ended because I'm > curious what's out there. I've studied and practiced Python > for a little over a year, but I've spent that time mostly > writing my own code and I don't really know much about what > and where to look for in modules and packages. > > Basically, I now have quite a few Python programs I use > frequently, and as time goes on my collection and uses of it > will grow. Right now I just want a way to select which one > I'd like to run and run it. I'd like it to be a standalone > application and some sort of system of categories would be nice. > > I'm migrating tasks I've always done in Excel to Python, and > I have a sketchy idea of features I'd like to open Excel > with, but I hate Excel VBA so much that I haven't written an > on_Open macro for Excel yet. What I'd like to open with is > mostly a menu of macros I'd like to have available for any > code I'm running, possibly opening different environments for > different kinds of tasks, that sort of thing. I also plan to > use sqlite3 for permanent data storage, matplotlib for > charts, and tkinter for interfaces. That's all in the > planning stages, but one thing that seems like an obvious > need is a way to keep related code and its associated data, > charts, etc, easily accessible to each other, like they are > when they're all bundled together in an Excel workbook. I > have a few ideas about how to do that, but I'm also > interested in what other people have done. > > I probably won't know exactly what I want until I have one > and use it for awhile. I've been keeping my code for daily > computing open in my IDE and using the IDE for a launcher, > but it's getting a little crowded, and I'd like to access > those bits separately from code I'm currently working on. > > Deborah > > -- > https://mail.python.org/mailman/listinfo/python-list > From python at deborahswanson.net Mon Feb 20 12:04:42 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 09:04:42 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <000f01d28b9b$6e7ccc70$27b23dae@sambora> Grant Edwards wrote, on February 20, 2017 7:37 AM > > On 2017-02-20, Deborah Swanson wrote: > > > I could probably write this myself, but I'm wondering if > this hasn't > > already been done many times. > > Yes, it has. > > > Anyone have some git links or other places to download from? > > See the "website" and "repository" links on the pages below: > https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29 https://en.wikipedia.org/wiki/Bourne_shell https://en.wikipedia.org/wiki/KornShell https://en.wikipedia.org/wiki/C_shell https://en.wikipedia.org/wiki/Almquist_shell -- Grant Edwards grant.b.edwards Yow! I was making donuts at and now I'm on a bus! gmail.com That's a good idea, you can do just about anything from a shell, and I read that Linus Torvalds never uses anything except the shell. I should have also said that right now I'm only working in Windows XP on a really old crappy PC, although I had Linux on my good PC and hope to have it back again soon. (Long story, but I'm a very sick person with limited time and physical energy to get real work done each day.) So when my good PC died I had to choose whether to learn Python or to invest a lot of time getting getting machines flying right. Guess which I choose? ;) Since I will be in Windows for yet awhile, it would be ideal to find an application that will work in both Windows and Linux, and maybe a shell installed in Windows is the way to go. But even within the shell I'd want something to act as a Python application launcher (see my previous post for details), although using just the shell itself to locate and launch Python code is an option. Thanks for the links! I'll likely be installing a shell one way or another to do this, so I'm sure these will come in handy. Deborah From ganesh1pal at gmail.com Mon Feb 20 12:55:21 2017 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 20 Feb 2017 23:25:21 +0530 Subject: Python - decode('hex') Message-ID: Hello Python world, I am on Linux and Python 2.7 , I need some help to figure out what is going wrong in the below piece of code I am trying to replace seven bytes in the hexdumfile at a given offset Program 1 works fine #!/usr/bin/python from qa.utils.easy_popen import run import pdb def replace_hex(fname, offset, newdata): #pdb.set_trace() with open(fname, 'r+b') as f: #Show current contents f.seek(offset) stringdata = f.read(len(newdata)) print 'Current data:' print '%08X: %s\n' % (offset, stringdata.encode('hex')) #Replace data at offset with newdata f.seek(offset) f.write(newdata) f.close() #print the data cmd = 'hexdump -C %s' % (fname) print cmd out, err, ret = run(cmd) print "Hexdump file data for corrupting" print out, err, ret return fname fname = '/tmp/2017-02-20_08-42-22/file1.raw' offset = 0x1 newdata = '64000101057804'.decode('hex') replace_hex(fname, offset, newdata) -200-1# python file_2.py Current data: 00000001: 6400010105789a hexdump -C /tmp/2017-02-20_08-42-22/file1.raw Hexdump file data for corrupting 00000000 06 64 00 01 01 05 78 04 73 1c ab 58 08 f9 8b 3b |.d....x.s..X...;| ==> Replaced 00000010 1e 00 a1 00 01 00 00 00 64 00 01 01 05 78 02 00 |........d....x..| Program 2 ( Doesn't work , some junk character got replaced ) g-200-1# vi file_2.py #!/usr/bin/python from qa.utils.easy_popen import run import pdb def replace_hex(fname, offset, newdata): #pdb.set_trace() with open(fname, 'r+b') as f: #Show current contents f.seek(offset) stringdata = f.read(len(newdata)) print 'Current data:' print '%08X: %s\n' % (offset, stringdata.encode('hex')) #Replace data at offset with newdata f.seek(offset) f.write(newdata) f.close() #print the data cmd = 'hexdump -C %s' % (fname) print cmd out, err, ret = run(cmd) print "Hexdump file data for corrupting" print out, err, ret return fname fname = '/tmp/2017-02-20_08-42-22/file1.raw' offset = 0x10 newdata = "" newdata = '64000101057804' newdata.decode('hex') replace_hex(fname, offset, newdata) g-200-1# python file_2.py Current data: 00000010: 1e00a10001000000640001010578 hexdump -C /tmp/2017-02-20_08-42-22/MOB_file1.raw Hexdump file data for corrupting 00000000 06 64 00 01 01 05 78 04 73 1c ab 58 08 f9 8b 3b |.d....x.s..X...;| 00000010 36 34 30 30 30 31 30 31 30 35 37 38 30 34 02 00 |64000101057804..|===> Some junk values got writen ??? 00000020 e0 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 00000030 00 03 10 21 bf cd 05 80 00 00 00 00 00 00 00 00 |...!............| 00000040 58 02 00 00 00 00 00 00 00 20 00 00 00 00 00 00 |X........ ......| 00000050 01 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 |................| 00000060 00 00 00 00 00 00 00 00 74 1c ab 58 a0 84 2b 00 |........t..X..+.| 00000070 74 1c ab 58 a0 84 2b 00 fd ff fd ff 00 00 00 00 |t..X..+.........| 00000080 02 04 25 40 04 00 01 64 00 01 01 05 78 04 31 30 |..%@...d....x.10| 00000090 35 37 38 00 02 84 02 64 00 01 01 05 78 30 30 30 |578....d....x000| 000000a0 31 30 30 30 32 00 0f fe 94 ba 15 00 00 00 00 00 |10002...........| 000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 000001d0 00 00 00 00 00 22 ff ff ff ff 20 03 00 00 00 00 |.....".... .....| 000001e0 00 00 00 0b 04 00 0b 01 00 02 01 0a 00 01 00 64 |...............d| 000001f0 06 00 00 01 00 02 97 0a 33 9b 3f 32 f3 62 b1 15 |........3.?2.b..| Questions: 1. The only difference between both the programs the difference are just the below lines. newdata = '64000101057804'.decode('hex') and newdata = "" newdata = '64000101057804' newdata.decode('hex') What is happening here and how do I fix this in program 2 ? for my eyes there doesn't look any difference . question 2: I am using the variable newdata because I can't hardcore the value , I have to keep changing this every time the function is called, will return it as a string help me slove this problem def get_data() : return str(data) new_data =get_data(input) Regards, Ganesh From python.list at tim.thechases.com Mon Feb 20 13:10:37 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 20 Feb 2017 12:10:37 -0600 Subject: Noob confused by ConfigParser defaults In-Reply-To: References: Message-ID: <20170220121037.05862bcd@bigbox.christie.dr> On 2017-02-20 10:45, Peter Otten wrote: > value = parser.get("section-1", "option-1", fallback="default > value") Huh. Do you remember when this was added? I see it in the 3.x docs, but not the 2.x docs. I remember writing my own wrappers multiple times for exactly these purposes, even to the point of opening an issue in the hopes of getting something of the like included https://lists.gt.net/python/bugs/827378?do=post_view_threaded So I'm glad to see it arrived; just curious when that happened officially. -tkc From rhodri at kynesim.co.uk Mon Feb 20 13:30:48 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 20 Feb 2017 18:30:48 +0000 Subject: Python - decode('hex') In-Reply-To: References: Message-ID: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> On 20/02/17 17:55, Ganesh Pal wrote: > 1. The only difference between both the programs the difference are just > the below lines. > > newdata = '64000101057804'.decode('hex') > > and > > newdata = "" > newdata = '64000101057804' > newdata.decode('hex') > > > What is happening here and how do I fix this in program 2 ? for my eyes > there doesn't look any difference . Python strings are immutable; methods like decode() create a brand new string for you. What your program 2 version does is to name the string of hex digits "newdata", decode it as hex into a new string and then throw that new string away. Your program 1 version by contrast decodes the string of digits as hex and then names is "newdata", throwing the original string of digits away. > question 2: > > I am using the variable newdata because I can't hardcore the value , I > have to keep changing this every time the function is called, will return > it as a string help me slove this problem > > def get_data() : > return str(data) > > new_data =get_data(input) That seems like a lot of work and syntax errors to do new_data = raw_input() (or new_data = input() if you're writing Python 3, which you weren't) Since you probably want to do this for your file name too, you might want to pass them as command line parameters to the script. Look up sys.argv[] in the standard library, or the argparse module if you're feeling keen. -- Rhodri James *-* Kynesim Ltd From python at mrabarnett.plus.com Mon Feb 20 14:00:21 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 20 Feb 2017 19:00:21 +0000 Subject: Python - decode('hex') In-Reply-To: References: Message-ID: <82603fa6-ec77-7ffb-e0c6-740422a42eca@mrabarnett.plus.com> On 2017-02-20 17:55, Ganesh Pal wrote: > Hello Python world, > > I am on Linux and Python 2.7 , I need some help to figure out what is > going wrong in the below piece of code > > I am trying to replace seven bytes in the hexdumfile at a given offset > > > Program 1 works fine > > #!/usr/bin/python > from qa.utils.easy_popen import run > import pdb > > def replace_hex(fname, offset, newdata): > #pdb.set_trace() > with open(fname, 'r+b') as f: > #Show current contents > f.seek(offset) > stringdata = f.read(len(newdata)) > print 'Current data:' > print '%08X: %s\n' % (offset, stringdata.encode('hex')) > > #Replace data at offset with newdata > f.seek(offset) > f.write(newdata) There's no need to explicitly close the file because when you leave the 'with' statement, it'll close the file for you; that's why it's used! > f.close() It's best to unindent the following lines so that they're outside the 'with' statement (and the file will have been closed). > #print the data > cmd = 'hexdump -C %s' % (fname) > print cmd > out, err, ret = run(cmd) > print "Hexdump file data for corrupting" > print out, err, ret > return fname > > fname = '/tmp/2017-02-20_08-42-22/file1.raw' > offset = 0x1 > newdata = '64000101057804'.decode('hex') > > replace_hex(fname, offset, newdata) > > > -200-1# python file_2.py > Current data: > 00000001: 6400010105789a > > hexdump -C /tmp/2017-02-20_08-42-22/file1.raw > Hexdump file data for corrupting > 00000000 06 64 00 01 01 05 78 04 73 1c ab 58 08 f9 8b 3b > |.d....x.s..X...;| ==> Replaced > 00000010 1e 00 a1 00 01 00 00 00 64 00 01 01 05 78 02 00 > |........d....x..| > [snip] From ganesh1pal at gmail.com Mon Feb 20 14:43:54 2017 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Tue, 21 Feb 2017 01:13:54 +0530 Subject: Python - decode('hex') In-Reply-To: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> References: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> Message-ID: On Feb 21, 2017 12:17 AM, "Rhodri James" wrote: On 20/02/17 17:55, Ganesh Pal wrote: > 1. The only difference between both the programs the difference are just > the below lines. > > newdata = '64000101057804'.decode('hex') > > and > > newdata = "" > newdata = '64000101057804' > newdata.decode('hex') > > > What is happening here and how do I fix this in program 2 ? for my eyes > there doesn't look any difference . > Python strings are immutable; methods like decode() create a brand new string for you. What your program 2 version does is to name the string of hex digits "newdata", decode it as hex into a new string and then throw that new string away. Your program 1 version by contrast decodes the string of digits as hex and then names is "newdata", throwing the original string of digits away Thanks for the reply James. How can I make my program 2 look like program 1 , any hacks ? because I get newdata value( as a hx return value Of type string )from a function. Regards, Ganesh From python at mrabarnett.plus.com Mon Feb 20 15:04:20 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 20 Feb 2017 20:04:20 +0000 Subject: Python - decode('hex') In-Reply-To: References: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> Message-ID: On 2017-02-20 19:43, Ganesh Pal wrote: > On Feb 21, 2017 12:17 AM, "Rhodri James" wrote: > > On 20/02/17 17:55, Ganesh Pal wrote: > >> 1. The only difference between both the programs the difference are just >> the below lines. >> >> newdata = '64000101057804'.decode('hex') >> >> and >> >> newdata = "" >> newdata = '64000101057804' >> newdata.decode('hex') >> >> >> What is happening here and how do I fix this in program 2 ? for my eyes >> there doesn't look any difference . >> > > Python strings are immutable; methods like decode() create a brand new > string for you. What your program 2 version does is to name the string of > hex digits "newdata", decode it as hex into a new string and then throw > that new string away. Your program 1 version by contrast decodes the > string of digits as hex and then names is "newdata", throwing the original > string of digits away > > > Thanks for the reply James. > > How can I make my program 2 look like program 1 , any hacks ? because I > get newdata value( as a hx return value Of type string )from a function. > In this: newdata.decode('hex') The .decode method doesn't change newdata in-place, it _returns_ the result. You're doing anything with that result. You're not binding (assigning) it to a name. You're not passing it into a function. You're not doing _anything_ with it. You're just letting it be discarded, thrown away. You could ask the user for the hex string: hex_string = raw_input('Enter the hex string: ') and then decode it: newdata = hex_string.decode('hex') From kw at codebykevin.com Mon Feb 20 16:15:07 2017 From: kw at codebykevin.com (Kevin Walzer) Date: Mon, 20 Feb 2017 16:15:07 -0500 Subject: Python application launcher (for Python code) In-Reply-To: References: <003701d28b25$9579bd40$27b23dae@sambora> Message-ID: On 2/19/17 10:01 PM, Deborah Swanson wrote: > I could probably write this myself, but I'm wondering if this hasn't > already been done many times. Anyone have some git links or other > places to download from? > What do you mean by "application launcher"? It's not more complicated than "python my script.py." Run your console of choice on any platform that supports Python. -- Kevin Walzer Code by Kevin/Mobile Code by Kevin http://www.codebykevin.com http://www.wtmobilesoftware.com From python at deborahswanson.net Mon Feb 20 17:56:31 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 14:56:31 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <005b01d28bcc$942975a0$27b23dae@sambora> Kevin Walzer wrote, on February 20, 2017 1:15 PM > > On 2/19/17 10:01 PM, Deborah Swanson wrote: > > I could probably write this myself, but I'm wondering if > this hasn't > > already been done many times. Anyone have some git links or other > > places to download from? > > > What do you mean by "application launcher"? It's not more complicated > than "python my script.py." Run your console of choice on any > platform > that supports Python. > > -- > Kevin Walzer > Code by Kevin/Mobile Code by Kevin > http://www.codebykevin.com > http://www.wtmobilesoftware.com I'm sorry, I thought people could figure out that I wasn't talking about running just one script. In case you didn't see my 2nd post responding to this question, I said: I deliberately left the question open-ended because I'm curious what's out there. I've studied and practiced Python for a little over a year, but I've spent that time mostly writing my own code and I don't really know much about what and where to look for in modules and packages. Basically, I now have quite a few Python programs I use frequently, and as time goes on my collection and uses of it will grow. Right now I just want a way to select which one I'd like to run and run it. I'd like it to be a standalone application and some sort of system of categories would be nice. I'm migrating tasks I've always done in Excel to Python, and I have a sketchy idea of features I'd like to open Excel with, but I hate Excel VBA so much that I haven't written an on_Open macro for Excel yet. What I'd like to open with is mostly a menu of macros I'd like to have available for any code I'm running, possibly opening different environments for different kinds of tasks, that sort of thing. I also plan to use sqlite3 for permanent data storage, matplotlib for charts, and tkinter for interfaces. That's all in the planning stages, but one thing that seems like an obvious need is a way to keep related code and its associated data, charts, etc, easily accessible to each other, like they are when they're all bundled together in an Excel workbook. I have a few ideas about how to do that, but I'm also interested in what other people have done. I probably won't know exactly what I want until I have one and use it for awhile. I've been keeping my code for daily computing open in my IDE and using the IDE for a launcher, but it's getting a little crowded, and I'd like to access those bits separately from code I'm currently working on. I should also say that right now I'm using Windows XP, but hope very soon to have Linux again. Ideally, this launcher would work in both. From __peter__ at web.de Mon Feb 20 18:04:09 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 21 Feb 2017 00:04:09 +0100 Subject: Noob confused by ConfigParser defaults References: <20170220121037.05862bcd@bigbox.christie.dr> Message-ID: Tim Chase wrote: > On 2017-02-20 10:45, Peter Otten wrote: >> value = parser.get("section-1", "option-1", fallback="default >> value") > > Huh. Do you remember when this was added? I see it in the 3.x docs, > but not the 2.x docs. I remember writing my own wrappers multiple > times for exactly these purposes, even to the point of opening an > issue in the hopes of getting something of the like included > > https://lists.gt.net/python/bugs/827378?do=post_view_threaded > > So I'm glad to see it arrived; just curious when that happened > officially. I don't use configparser regularly, so I had to look around myself and found https://docs.python.org/3.2/whatsnew/3.2.html#configparser """ A number of smaller features were also introduced, like support for specifying encoding in read operations, specifying fallback values for get- functions, or reading directly from dictionaries and strings. (All changes contributed by ?ukasz Langa.) """ ?ukasz also closed your bug report http://bugs.python.org/issue8666#msg113414 but the superseding issue is itself superseded... From steve+python at pearwood.info Mon Feb 20 19:22:23 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 21 Feb 2017 11:22:23 +1100 Subject: Python - decode('hex') References: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> Message-ID: <58ab8841$0$1618$c3e8da3$5496439d@news.astraweb.com> On Tue, 21 Feb 2017 06:43 am, Ganesh Pal wrote: > How can I make my program 2 look like program 1 I don't understand the question. If you want program 2 to look like program 1, you can edit program 2 and change it to look like program 1. But why not just use program 1? It already looks like program 1. In an earlier post, you even pointed out that there is just a single, three line difference between the two programs: [quote] 1. The only difference between both the programs the difference are just the below lines. newdata = '64000101057804'.decode('hex') and newdata = "" newdata = '64000101057804' newdata.decode('hex') [end quote] So you select the three lines which don't work in your editor, hit DELETE, then type the one line which does work. If you want to understand WHY the three lines don't work, add "print newdata" to the end: newdata = "" newdata = '64000101057804' newdata.decode('hex') print newdata If you still don't understand what is going on, try this example instead: x = 0 x = 999 x + 1 print x versus: x = 999 x = x + 1 print x or even better: x = 999 + 1 print x -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From simon+python at bleah.co.uk Mon Feb 20 19:36:06 2017 From: simon+python at bleah.co.uk (Simon Ward) Date: Tue, 21 Feb 2017 00:36:06 +0000 Subject: Python application launcher (for Python code) In-Reply-To: <005b01d28bcc$942975a0$27b23dae@sambora> References: <005b01d28bcc$942975a0$27b23dae@sambora> Message-ID: On 20 February 2017 22:56:31 GMT+00:00, Deborah Swanson wrote: > Basically, I now have quite a few Python programs I use frequently, > and > as time goes on my collection and uses of it will grow. Right now I > just > want a way to select which one I'd like to run and run it A Python application should be treated as any other application and be runnable by whatever mechanisms the operating system provides, there's your application launcher. The main niggle is that a Python file is not a "native" executable, so the OS or shell running under the OS needs to be told to use the Python interpreter to run it. For many shells on Linux (or the kernel itself) this can be done with the "shebang" on the first line. For Windows you generally associate the Python file type with the Python interpreter (done when Python is installed), or create shortcuts specifying the Python interpreter as the executable (or make executable files using some other tool). >?I'd like it to be a standalone application and some sort of system of categories > would be nice. The freedesktop.org menu specification[1] provides a common format for defining application menus including organisation such as categories and hierarchies. The Windows start menu can be arranged how you wish, but I'm not aware of an implementation of the menu specification on Windows. [1]: https://www.freedesktop.org/wiki/Specifications/menu-spec/ > I'm migrating tasks I've always done in Excel to Python, and I have a > sketchy idea of features I'd like to open Excel with, but I hate Excel > VBA so much that I haven't written an on_Open macro for Excel yet. > What > I'd like to open with is mostly a menu of macros I'd like to have > available for any code I'm running, possibly opening different > environments for different kinds of tasks, that sort of thing. I also > plan to use sqlite3 for permanent data storage, matplotlib for charts, > and tkinter for interfaces. That's all in the planning stages, but one > thing that seems like an obvious need is a way to keep related code > and > its associated data, charts, etc, easily accessible to each other, > like > they are when they're all bundled together in an Excel workbook. I > have > a few ideas about how to do that, but I'm also interested in what > other > people have done. Possibly what you actually want is a development environment. Mine is made up of a shell with convenience functions, a text editor and a few other independent things. Some prefer to use an IDE, such as IDLE[2], where this is all integrated into one convenient application. >From some of what you said above I suggest taking a look at Jupyter Notebook[3] and/or the underlying iPython[4] shell. [2]: https://docs.python.org/3/library/idle.html [3]: http://jupyter.org/ [4]: https://ipython.org/ Simon -- Sent from Kaiten Mail. Please excuse my brevity. From steve+python at pearwood.info Mon Feb 20 19:53:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 21 Feb 2017 11:53:27 +1100 Subject: Python application launcher (for Python code) References: <000f01d28b9b$6e7ccc70$27b23dae@sambora> Message-ID: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> On Tue, 21 Feb 2017 04:04 am, Deborah Swanson wrote: > That's a good idea, you can do just about anything from a shell, and I > read that Linus Torvalds never uses anything except the shell. [...] > Since I will be in Windows for yet awhile, it would be ideal to find an > application that will work in both Windows and Linux, and maybe a shell > installed in Windows is the way to go. Windows has a console application with a shell. It is pretty bare bones, but it works. If I remember the steps correctly (and hopefully some Windows user will step in and correct me if I get it wrong): Start menu > Run > type "cmd" ENTER I'm told that serious Windows power users don't use cmd.exe or command.com (or is it the other way around?) any more, they use Powershell, but I know nothing about that. If you're using Windows 10, you might look at Windows Subsystem for Linux: https://blogs.windows.com/buildingapps/2016/03/30/run-bash-on-ubuntu-on-windows/ > But even within the shell I'd > want something to act as a Python application launcher (see my previous > post for details), although using just the shell itself to locate and > launch Python code is an option. Normally the way I would launch applications is: - from the Start menu (yes, many Linux desktops have a Start menu); - from the shell. The Start menu is configurable under Linux, although I've never done so, I've always let the OS manage it. At the shell, there are plenty of ways to organise things. It depends on how you want to launch your apps: python path/to/my/application.py python -m application.py path/to/my/application.py application.py except on Windows you might have to use python.exe or py.exe instead of python. The first is the most flexible: you can organise your scripts and applications however you like, wherever you like, and then just explicitly run them. The last is the trickiest: on Linux, the file application.py needs three things: - it must have the appropriate magic "hash bang" line as the first line of the file; - it must be flagged as executable; - it must be located somewhere on the executable PATH. I'm not sure what the Windows equivalents to that are. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Feb 20 20:25:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 21 Feb 2017 12:25:14 +1100 Subject: Python application launcher (for Python code) References: <8560k5mie8.fsf@benfinney.id.au> <003f01d28b90$385d8450$27b23dae@sambora> Message-ID: <58ab96fd$0$1595$c3e8da3$5496439d@news.astraweb.com> On Tue, 21 Feb 2017 02:44 am, Deborah Swanson wrote: [...] > Basically, I now have quite a few Python programs I use frequently, and > as time goes on my collection and uses of it will grow. Right now I just > want a way to select which one I'd like to run and run it. I'd like it > to be a standalone application and some sort of system of categories > would be nice. Here's a possible work-flow that might be suitable for you. I'm going to write it in Linux-speak, you can hopefully translate it to Windows yourself. (1) Organise your scripts into separate directories according to your preferred way of categorising them: /home/steve/python/fun/ /home/steve/python/serious/ /home/steve/python/work/ Move each script into the appropriate directory. (2) Add each category to the PYTHONPATH. One easy way to do so is by adding the directories to a .pth file. Create the file: /home/steve/.local/lib/python3.5/site-packages/default.pth (the name is not important so long as it ends with .pth; the location is) containing the three lines: /home/steve/python/fun/ /home/steve/python/serious/ /home/steve/python/work/ See the documentation for the site module for more detail: https://docs.python.org/3/library/site.html (3) Now you can run any of your scripts by name, without caring precisely where it is: python3 -m myscript or you can explicitly choose one: cd ~ # change to my home directory python3 python/fun/myscript.py > I'm migrating tasks I've always done in Excel to Python, and I have a > sketchy idea of features I'd like to open Excel with, but I hate Excel > VBA so much that I haven't written an on_Open macro for Excel yet. You might consider using LibreOffice. It is intended to be backwards compatible with Excel, but is scriptable using four different languages: Basic Python Javascript BeanShell Start here: https://duckduckgo.com/html/?q=scripting+libreoffice -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Mon Feb 20 20:50:00 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 21 Feb 2017 12:50:00 +1100 Subject: Python application launcher (for Python code) References: <8560k5mie8.fsf@benfinney.id.au> <003f01d28b90$385d8450$27b23dae@sambora> Message-ID: <85wpckl3cn.fsf@benfinney.id.au> "Deborah Swanson" writes: > Basically, I now have quite a few Python programs I use frequently, and > as time goes on my collection and uses of it will grow. Right now I just > want a way to select which one I'd like to run and run it. I'd like it > to be a standalone application and some sort of system of categories > would be nice. I agree with others that you appear to be describing a ?shell?: In computing, a shell is a user interface for access to an operating system's services. In general, operating system shells use either a command-line interface (CLI) or graphical user interface (GUI), depending on a computer's role and particular operation. When writing your own programs, organise them on the filesystem in a way that makes sense for you (Python's modules, packages, and imports will respond to your filesystem layout once you learn how to arrange them). Then, use the operating system shell to invoke the program you want on each occasion. -- \ ?We can't depend for the long run on distinguishing one | `\ bitstream from another in order to figure out which rules | _o__) apply.? ?Eben Moglen, _Anarchism Triumphant_, 1999 | Ben Finney From python at deborahswanson.net Mon Feb 20 21:16:14 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 18:16:14 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: <008401d28be8$7a5ceeb0$27b23dae@sambora> Steve D'Aprano wrote, on February 20, 2017 4:53 PM > > On Tue, 21 Feb 2017 04:04 am, Deborah Swanson wrote: > > > That's a good idea, you can do just about anything from a shell, and I > > read that Linus Torvalds never uses anything except the shell. > [...] > > Since I will be in Windows for yet awhile, it would be ideal to find > > an application that will work in both Windows and Linux, and maybe a > > shell installed in Windows is the way to go. > > Windows has a console application with a shell. It is pretty > bare bones, but it works. If I remember the steps correctly > (and hopefully some Windows user will step in and correct me > if I get it wrong): > > Start menu > Run > type "cmd" ENTER Yes, I'm familiar with Windows' shell. It's basically (pun intended) unchanged since Windows 2.0, maybe all the way back to MSDOS, except for the additions of new built-in commands. (Some of which are useful, but most are pretty useless.) > I'm told that serious Windows power users don't use cmd.exe > or command.com (or is it the other way around?) any more, > they use Powershell, but I know nothing about that. Yes, I've used Powershell, a couple decades ago, but it would be a learning curve I'd rather stay off of to go back to it now. I'd rather spend the time and energy to get up and running in Linux. It was my plan all along to migrate to Linux and mostly leave Windows behind, but that plan ran into some delays. Mostly by my good PC breaking down, and my health went downhill again, about the same time. Funny how you can't really schedule those things, or make them be over with because you want them to be. > If you're using Windows 10, you might look at Windows > Subsystem for Linux: > > https://blogs.windows.com/buildingapps/2016/03/30/run-bash-on- > ubuntu-on-windows/ I can't stand any opsys Mickeysoft made after Windows XP, and I'd rather have Linux back again than to pay big bucks to buy both the software and a new PC, just to fight with Windows 10 for a mere subset of Linux. I read about "Cooperative Linux" the other day, which runs concurrently with Windows and doesn't require booting back and forth between them, or anything like the resource-hogging virtualization system I had on my good PC. Cooperative could be ideal for this ancient, low resource PC I have for now, but I haven't thoroughly checked it out yet. Or simply installing a Linux shell in Windows as Grant Edwards suggested, but that seems fraught with problems. Maybe not, but if Cooperative works as advertised, that's a better bet. > > But even within the shell I'd > > want something to act as a Python application launcher (see my previous > > post for details), although using just the shell itself to locate and > > launch Python code is an option. > > Normally the way I would launch applications is: > > - from the Start menu (yes, many Linux desktops have a Start menu); > > - from the shell. > > > The Start menu is configurable under Linux, although I've > never done so, I've always let the OS manage it. > > At the shell, there are plenty of ways to organise things. It > depends on how you want to launch your apps: > > > python path/to/my/application.py > > python -m application.py > > path/to/my/application.py > > application.py That may be so, and maybe what I want to do could be much more easily done in Linux than it is in Windows. I really didn't get that far with Python in Linux before my PC with Linux on it died, so I honestly just don't know and hadn't thought about it. Quite possibly Cooperative Linux is my ticket out of this mess. > except on Windows you might have to use python.exe or py.exe > instead of python. I think my time is better spent getting Linux back one way or another, I've tried several approaches in Windows XP and the tinkering needed to make it work well just doesn't seem worth it to me. > The first is the most flexible: you can organise your scripts > and applications however you like, wherever you like, and > then just explicitly run them. The other thing that occurs to me is that it's relatively easy to run processes concurrently in Linux, so I could have my "on_Open" macro running while any other Python code is running. The "on_Open" macro should be accessible by the other running Python processes, but I've read enough by now to see that it can be done, but there's a bit to learn about how to do it. > The last is the trickiest: on Linux, the file application.py > needs three things: > > - it must have the appropriate magic "hash bang" line as the > first line of the file; > > - it must be flagged as executable; > > - it must be located somewhere on the executable PATH. Thanks, I've copied these items into my project file, hopefully to be used soon. > I'm not sure what the Windows equivalents to that are. Me neither, and I'm less than enthused about figuring it out. > > > -- > Steve > "Cheer up," they said, "things could be worse." So I cheered > up, and sure enough, things got worse. > Thanks Steve. It's possible I've been going at this all wrong. Like it or not, I have a Microsoft mindset, all the more deeply burned in from working there as a software tester as long as I did, and much as I've tried to shed that mindset I still don't always see the blinders til someone else points out a better way. In Windows you likely would need an application launcher to efficiently do what I described, and it might be possible to easily do it all simply from a Linux shell. To be determined. That's why I inititially asked an open-ended question, to see what other people are doing, and I may have got too caught up with people asking exactly what I wanted to do. From python at deborahswanson.net Mon Feb 20 21:40:29 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 18:40:29 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <85wpckl3cn.fsf@benfinney.id.au> Message-ID: <008d01d28beb$dddefe80$27b23dae@sambora> Ben Finney wrote, on February 20, 2017 5:50 PM > > "Deborah Swanson" writes: > > > Basically, I now have quite a few Python programs I use frequently, > > and as time goes on my collection and uses of it will grow. Right now > > I just want a way to select which one I'd like to run and run it. I'd > > like it to be a standalone application and some sort of system of > > categories would be nice. > > I agree with others that you appear to be describing a "shell": > > In computing, a shell is a user interface for access to > an operating > system's services. In general, operating system shells > use either a > command-line interface (CLI) or graphical user interface (GUI), > depending on a computer's role and particular operation. > > Yes, though Windows is a bit snarlier than Linux, especially since the only way I could get a Python3 install out of Anaconda was by sacrificing their installed console, paths, etc. I got 3.4.3 before they slammed the door on Windows XP, and it works fine if you access it from an IDE like PyCharm, but not so fine if you try to get to it in a DOS box (cmd.com). Another bunch of good reasons to get Linux back asap. > When writing your own programs, organise them on the > filesystem in a way that makes sense for you (Python's > modules, packages, and imports will respond to your > filesystem layout once you learn how to arrange them). > > Then, use the operating system shell to invoke the program > you want on each occasion. > > -- > \ "We can't depend for the long run on > distinguishing one | > `\ bitstream from another in order to figure out > which rules | > _o__) apply." -Eben Moglen, _Anarchism > Triumphant_, 1999 | > Ben Finney Sounds like a plan! From python.list at tim.thechases.com Mon Feb 20 21:56:11 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 20 Feb 2017 20:56:11 -0600 Subject: Noob confused by ConfigParser defaults In-Reply-To: References: <20170220121037.05862bcd@bigbox.christie.dr> Message-ID: <20170220205611.42f40808@bigbox.christie.dr> On 2017-02-21 00:04, Peter Otten wrote: > Tim Chase wrote: >> On 2017-02-20 10:45, Peter Otten wrote: >>> value = parser.get("section-1", "option-1", fallback="default >>> value") >> >> Huh. Do you remember when this was added? > > I don't use configparser regularly, so I had to look around myself > and found > > https://docs.python.org/3.2/whatsnew/3.2.html#configparser > > > ?ukasz also closed your bug report Thanks for hunting that down, and thanks also to ?ukasz for shepherding in that long-desired change. -tkc From python at deborahswanson.net Mon Feb 20 23:16:50 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 20:16:50 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <009a01d28bf9$5361ed90$27b23dae@sambora> Simon Ward wroe, on February 20, 2017 4:36 PM > > On 20 February 2017 22:56:31 GMT+00:00, Deborah Swanson > wrote: > > Basically, I now have quite a few Python programs I use frequently, > > and as time goes on my collection and uses of it will grow. Right now > > I just want a way to select which one I'd like to run and run it > > A Python application should be treated as any other > application and be runnable by whatever mechanisms the > operating system provides, there's your application launcher. > The main niggle is that a Python file is not a "native" > executable, so the OS or shell running under the OS needs to > be told to use the Python interpreter to run it. For many > shells on Linux (or the kernel itself) this can be done with > the "shebang" on the first line. For Windows you generally > associate the Python file type with the Python interpreter > (done when Python is installed), or create shortcuts > specifying the Python interpreter as the executable (or make > executable files using some other tool). That "main niggle" is a doozie on Windows XP with Anaconda Python 3.4.3 (see my previous post for a more detailed explanation. Getting Linux up and running seems my best hope at this point. > >?I'd like it to be a standalone application and some sort of system of > >categories would be nice. > > The freedesktop.org menu specification[1] provides a common > format for defining application menus including organisation > such as categories and hierarchies. The Windows start menu > can be arranged how you wish, but I'm not aware of an > implementation of the menu specification on Windows. > > [1]: https://www.freedesktop.org/wiki/Specifications/menu-spec/ Thanks Simon, this may be what I was looking for, though I wasn't enirely sure what form it would take or my commitment to a solution workable in Windows. I am quite familiar with Windows' Start Menu, but beyond changing the order of program shortcuts and maybe putting groups of them in folders to further collapse the size of the menu, there isn't much else you can do with it. > > > I'm migrating tasks I've always done in Excel to Python, and I have a > > sketchy idea of features I'd like to open Excel with, but I hate Excel > > VBA so much that I haven't written an on_Open macro for Excel yet. > > What I'd like to open with is mostly a menu of macros I'd like to have > > available for any code I'm running, possibly opening different > > environments for different kinds of tasks, that sort of thing. I also > > plan to use sqlite3 for permanent data storage, matplotlib for charts, > > and tkinter for interfaces. That's all in the planning stages, but one > > thing that seems like an obvious need is a way to keep related code and > > its associated data, charts, etc, easily accessible to each other, like > > they are when they're all bundled together in an Excel workbook. I have > > a few ideas about how to do that, but I'm also interested in what other > > people have done. > > Possibly what you actually want is a development environment. > Mine is made up of a shell with convenience functions, a text > editor and a few other independent things. Some prefer to use > an IDE, such as IDLE[2], where this is all integrated into > one convenient application. That's more along the lines of what I was looking for. > From some of what you said above I suggest taking a look at > Jupyter Notebook[3] and/or the underlying iPython[4] shell. > > [2]: https://docs.python.org/3/library/idle.html > [3]: http://jupyter.org/ > [4]: https://ipython.org/ > > Simon > -- > Sent from Kaiten Mail. Please excuse my brevity. I've looked at Jupyter, and it might be very useful, but it will also have to wait til I'm on Linux again, where I can get complete installs of Anaconda. Thanks for your comments, the freedesktop.org menu definitely sounds worth checking out. Deborah From python at deborahswanson.net Mon Feb 20 23:42:49 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Mon, 20 Feb 2017 20:42:49 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <58ab96fd$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: <009b01d28bfc$f4a59f00$27b23dae@sambora> Steve D'Aprano wrote, on February 20, 2017 5:25 PM > > On Tue, 21 Feb 2017 02:44 am, Deborah Swanson wrote: > > [...] > > Basically, I now have quite a few Python programs I use frequently, > > and as time goes on my collection and uses of it will grow. > Right now > > I just want a way to select which one I'd like to run and > run it. I'd > > like it to be a standalone application and some sort of system of > > categories would be nice. > > Here's a possible work-flow that might be suitable for you. > I'm going to write it in Linux-speak, you can hopefully > translate it to Windows yourself. > > (1) Organise your scripts into separate directories according > to your preferred way of categorising them: > > /home/steve/python/fun/ > /home/steve/python/serious/ > /home/steve/python/work/ > > Move each script into the appropriate directory. > > (2) Add each category to the PYTHONPATH. One easy way to do > so is by adding the directories to a .pth file. Create the file: > > /home/steve/.local/lib/python3.5/site-packages/default.pth > > (the name is not important so long as it ends with .pth; the > location is) > > containing the three lines: > > /home/steve/python/fun/ > /home/steve/python/serious/ > /home/steve/python/work/ > See the documentation for the site module for more detail: > >https://docs.python.org/3/library/site.html >(3) Now you can run any of your scripts by name, without caring precisely where it is: python3 -m myscript or you can explicitly choose one: cd ~ # change to my home directory python3 python/fun/myscript.py Thanks, Steve. This is similar to what I thought I could do on Windows, but with no nicety like the .pth file and its associated usefulness. > I'm migrating tasks I've always done in Excel to Python, and I have a > sketchy idea of features I'd like to open Excel with, but I hate Excel > VBA so much that I haven't written an on_Open macro for Excel yet. You might consider using LibreOffice. It is intended to be backwards compatible with Excel, but is scriptable using four different languages: Basic Python Javascript BeanShell Start here: https://duckduckgo.com/html/?q=scripting+libreoffice -- Steve "Cheer up," they said, "things could be worse." So I cheered up, and sure enough, things got worse. I have LibreOffice 5 (no Python scripting), and the scripting with Python in some later version may be nice, but the interface itself is clunkier than Excel, and maybe as flakey and unreliable as Excel. I don't know that, but I've only sporadically used LibreOffice for less than 5 years, where I've been using other spreadsheets since 1988, mostly Excel and in the beginning Lotus 123 (that was on a real DOS box, the kind you only had floppy drives on). They're all clunky in one aspect or another, and I just want the spreadsheet functionality in straight Python, even if it has to be DIY, but to get all of Excel's (or LibreOffice's) functionality, I'll need the tool I've been asking about in this thread, and several others if this group knows anything about them. I'll maybe write more about that in another topic, but probably not right away. Not sure if there's all that much interest in the topic here. Thanks for your comments, Deborah From jzakiya at gmail.com Mon Feb 20 23:48:15 2017 From: jzakiya at gmail.com (jzakiya at gmail.com) Date: Mon, 20 Feb 2017 20:48:15 -0800 (PST) Subject: integer nth roots for arbitrary sized integer Message-ID: <712a251a-e94f-478e-ad88-c3ee7bc567b5@googlegroups.com> FYI for people who need this. I was looking here http://www.codecodex.com/wiki/Calculate_an_integer_square_root for an efficient/fast algorithm for computing integer squareroots. Based on one of the C version examples, I think this is the most efficient/fastest that always produces correct results for arbitrary sized integers. >>> def isqrt(n): ... root, bitn_mask = 0, (1 << (n.bit_length() // 2 + 1)) ... while bitn_mask != 0: ... root |= bitn_mask ... if (root * root) > n: root ^= bitn_mask ... bitn_mask >>= 1 ... return root ... >>> n = 10**35; isqrt(n) 316227766016837933 >>> n = 10**111 >>> isqrt(n) 31622776601683793319988935444327185337195551393252168268L >>> n = 10**110; isqrt(n) 10000000000000000000000000000000000000000000000000000000L >>> n = 10**210; isqrt(n) 1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000L >>> n = 10**211; isqrt(n) 3162277660168379331998893544432718533719555139325216826857504852792594438639238221344248108379300295187347L With a small modification, the algorithm will find the exact integer nth root for arbitrary sized integers. >>> def irootn(num, n): ... root, bitn_mask = 0, (1 << (num.bit_length() // n + 1)) ... while bitn_mask != 0: ... root |= bitn_mask ... if root**n > num: root ^= bitn_mask ... bitn_mask >>= 1 ... return root ... >>> n = 10**35; isqrt(n) 316227766016837933 >>> n = 10**35; irootn(n, 2) 316227766016837933 >>> n = 10**39; irootn(n, 2) 31622776601683793319L >>> n = 10**39; irootn(n, 3) 10000000000000 >>> n = 10**39; irootn(n, 4) 5623413251 >>> n = 10**144; irootn(n, 2) 1000000000000000000000000000000000000000000000000000000000000000000000000L >>> n = 10**144; irootn(n, 3) 1000000000000000000000000000000000000000000000000L >>> n = 10**144; irootn(n, 4) 1000000000000000000000000000000000000L >>> n = 10**144; irootn(n, 6) 1000000000000000000000000L >>> n = 10**144; irootn(n, 8) 1000000000000000000 >>> n = 10**144; irootn(n, 11) 12328467394420 >>> n = 10**144; irootn(n, 12) 1000000000000 >>> From ganesh1pal at gmail.com Tue Feb 21 00:49:40 2017 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Tue, 21 Feb 2017 11:19:40 +0530 Subject: Python - decode('hex') In-Reply-To: References: <085a1719-e143-a36b-1796-e8ac3fbb3192@kynesim.co.uk> Message-ID: Got it , MRAB, Thanks for the explanation it was such a simple thing I was breaking my head over it On Tue, Feb 21, 2017 at 1:34 AM, MRAB wrote: > On 2017-02-20 19:43, Ganesh Pal wrote: > >> On Feb 21, 2017 12:17 AM, "Rhodri James" wrote: >> >> On 20/02/17 17:55, Ganesh Pal wrote: >> >> 1. The only difference between both the programs the difference are just >>> the below lines. >>> >>> newdata = '64000101057804'.decode('hex') >>> >>> and >>> >>> newdata = "" >>> newdata = '64000101057804' >>> newdata.decode('hex') >>> >>> >>> What is happening here and how do I fix this in program 2 ? for my >>> eyes >>> there doesn't look any difference . >>> >>> >> Python strings are immutable; methods like decode() create a brand new >> string for you. What your program 2 version does is to name the string of >> hex digits "newdata", decode it as hex into a new string and then throw >> that new string away. Your program 1 version by contrast decodes the >> string of digits as hex and then names is "newdata", throwing the original >> string of digits away >> >> >> Thanks for the reply James. >> >> How can I make my program 2 look like program 1 , any hacks ? because I >> get newdata value( as a hx return value Of type string )from a function. >> >> In this: > > newdata.decode('hex') > > The .decode method doesn't change newdata in-place, it _returns_ the > result. > > You're doing anything with that result. You're not binding (assigning) it > to a name. You're not passing it into a function. You're not doing > _anything_ with it. You're just letting it be discarded, thrown away. > > You could ask the user for the hex string: > > hex_string = raw_input('Enter the hex string: ') > > and then decode it: > > newdata = hex_string.decode('hex') > > -- > https://mail.python.org/mailman/listinfo/python-list > From pertoger at gmail.com Tue Feb 21 04:03:09 2017 From: pertoger at gmail.com (pertoger at gmail.com) Date: Tue, 21 Feb 2017 10:03:09 +0100 Subject: Distributing "Python 3.4 for iOS" Message-ID: <58ac024e.12142e0a.cb56f.7f87@mx.google.com> I distribute learning apps via an MDM server. There are two possibilities for deploying apps; either to users via their Apple-ID or directly to devices (iPads). Alas my users don?t have Apple-IDs, so I have to distribute apps directly to their devices. Sadly ?Python 3.4 for iOS? is in Appstore not set to distribute from Mobile Device Management servers to devices (iPads), only users. Does anybody know some means of contacting the right instance/person to alter this? ? Gratefully, Per-Johnny T?gersen IKT-manager for Vik and Berg schools in S?mna, Norway pertoger at gmail.com +47 951 38 649 Sent by E-mail for Windows 10 From bc at freeuk.com Tue Feb 21 08:52:24 2017 From: bc at freeuk.com (BartC) Date: Tue, 21 Feb 2017 13:52:24 +0000 Subject: Python application launcher (for Python code) In-Reply-To: References: <8560k5mie8.fsf@benfinney.id.au> <003f01d28b90$385d8450$27b23dae@sambora> Message-ID: On 20/02/2017 15:44, Deborah Swanson wrote: > Ben Finney wrote, on February 19, 2017 11:27 PM >> >> "Deborah Swanson" writes: >> >>> I could probably write this myself, but I'm wondering if this hasn't > >>> already been done many times. >> >> Can you describe what you are looking for, in enough detail >> that we can know whether it's already been done as you want it? > I deliberately left the question open-ended because I'm curious what's > out there. I've studied and practiced Python for a little over a year, > but I've spent that time mostly writing my own code and I don't really > know much about what and where to look for in modules and packages. > > Basically, I now have quite a few Python programs I use frequently, and > as time goes on my collection and uses of it will grow. Right now I just > want a way to select which one I'd like to run and run it. I'd like it > to be a standalone application and some sort of system of categories > would be nice. If you use Windows, then the OS can take care of this. You don't need a separate application, just open up a directory showing a list of .py files, and click on one. Then if '.py' is correctly associated with the right Python version, it will run Python on it. To group into categories, just put the .py files into separate, aptly named directories. The only problem I found, when launching in GUI mode, was that a console-mode Python program would briefly open a console window that would then promptly disappear if it finished immediately and there was no interaction. -- bartc From skip.montanaro at gmail.com Tue Feb 21 09:06:01 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 21 Feb 2017 08:06:01 -0600 Subject: Derivative timezone? Message-ID: Is it possible to create a timezone which is a derivative of an existing timezone? I'd like to create one which is several hours different ahead of America/Chicago, but follows Chicago's DST changes. My initial attempt was to create a timezone object using pytz: tz = pytz.timezone("America/Chicago") then add eight hours to its _utcoffset attribute, but that doesn't work. Is "creative" timezone construction possible, or am I limited to what pytz finds in the Olson database? Thx, Skip From carlopires at gmail.com Tue Feb 21 09:23:29 2017 From: carlopires at gmail.com (carlopires at gmail.com) Date: Tue, 21 Feb 2017 06:23:29 -0800 (PST) Subject: str.format fails with JSON? Message-ID: Hi, When I run this piece of code: 'From {"value": 1}, value={value}'.format(value=1) Python complains about the missing "value" parameter (2.7.12 and 3.6.x): Traceback (most recent call last): File "test_format.py", line 1, in 'From {"value": 1}, value={value}'.format(value=1) KeyError: '"value" But according to the format string syntax (https://docs.python.org/2/library/string.html): replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] "}" field_name ::= arg_name ("." attribute_name | "[" element_index "]")* arg_name ::= [identifier | integer] attribute_name ::= identifier element_index ::= integer | index_string index_string ::= + conversion ::= "r" | "s" format_spec ::= The replacement_field, which in this case, is composed by an identifier, shouldn't have quotation marks. Here is the lexical definition for an identifier (according to the documentation): identifier ::= (letter|"_") (letter | digit | "_")* letter ::= lowercase | uppercase lowercase ::= "a"..."z" uppercase ::= "A"..."Z" digit ::= "0"..."9" So according to the specification, {value} should be recognized as a valid format string identifier and {"value"} should be ignored. Python seems to not follow the specification in the documentation. Anything inside the keys is accepted as identifier. From skip.montanaro at gmail.com Tue Feb 21 09:34:04 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 21 Feb 2017 08:34:04 -0600 Subject: Derivative timezone? In-Reply-To: References: Message-ID: I figured out that I can drop down to a lower level and use a manually constructed tz file to create the desired timezone object. The pytz package has a build_tzinfo() function which takes a zone name and an open file object). Skip On Tue, Feb 21, 2017 at 8:06 AM, Skip Montanaro wrote: > Is it possible to create a timezone which is a derivative of an > existing timezone? I'd like to create one which is several hours > different ahead of America/Chicago, but follows Chicago's DST changes. > My initial attempt was to create a timezone object using pytz: > > tz = pytz.timezone("America/Chicago") > > then add eight hours to its _utcoffset attribute, but that doesn't > work. Is "creative" timezone construction possible, or am I limited to > what pytz finds in the Olson database? > > Thx, > > Skip From rosuav at gmail.com Tue Feb 21 09:38:54 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 22 Feb 2017 01:38:54 +1100 Subject: str.format fails with JSON? In-Reply-To: References: Message-ID: On Wed, Feb 22, 2017 at 1:23 AM, wrote: > Hi, > > When I run this piece of code: > > 'From {"value": 1}, value={value}'.format(value=1) Firstly, this code is in error; I suspect you wanted a couple of literal braces, so what you want is: >>> 'From {{"value": 1}}, value={value}'.format(value=1) 'From {"value": 1}, value=1' So everything from here on is about exactly *what* error gets raised. > Python complains about the missing "value" parameter (2.7.12 and 3.6.x): > > Traceback (most recent call last): > File "test_format.py", line 1, in > 'From {"value": 1}, value={value}'.format(value=1) > KeyError: '"value" > > But according to the format string syntax (https://docs.python.org/2/library/string.html): > > The replacement_field, which in this case, is composed by an identifier, shouldn't have quotation marks. > > So according to the specification, {value} should be recognized as a valid format string identifier and {"value"} should be ignored. > > Python seems to not follow the specification in the documentation. Anything inside the keys is accepted as identifier. I agree. It'd be beneficial to add that kind of a check, as a means of detecting cases where someone forgot to double a brace. Something like: InvalidKeyError: '"value"' - did you intend to use {{ }} ? Not sure how hard the check would be to write, but it'd help to highlight a particular class of bug. ChrisA . From __peter__ at web.de Tue Feb 21 09:48:11 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 21 Feb 2017 15:48:11 +0100 Subject: str.format fails with JSON? References: Message-ID: carlopires at gmail.com wrote: > Hi, > > When I run this piece of code: > > 'From {"value": 1}, value={value}'.format(value=1) > > Python complains about the missing "value" parameter (2.7.12 and 3.6.x): > > Traceback (most recent call last): > File "test_format.py", line 1, in > 'From {"value": 1}, value={value}'.format(value=1) > KeyError: '"value" > > But according to the format string syntax > (https://docs.python.org/2/library/string.html): > > replacement_field ::= "{" [field_name] ["!" conversion] [":" format_spec] > "}" > field_name ::= arg_name ("." attribute_name | "[" element_index > "]")* > arg_name ::= [identifier | integer] > attribute_name ::= identifier > element_index ::= integer | index_string > index_string ::= + > conversion ::= "r" | "s" > format_spec ::= > > The replacement_field, which in this case, is composed by an identifier, > shouldn't have quotation marks. Here is the lexical definition for an > identifier (according to the documentation): > > identifier ::= (letter|"_") (letter | digit | "_")* > letter ::= lowercase | uppercase > lowercase ::= "a"..."z" > uppercase ::= "A"..."Z" > digit ::= "0"..."9" > > So according to the specification, {value} should be recognized as a valid > format string identifier and {"value"} should be ignored. I'd rather say '{"value"}' unspecified as "value" isn't a proper field_name and literal {} have to be escaped. > > Python seems to not follow the specification in the documentation. > Anything inside the keys is accepted as identifier. It does not enforce that you follow the specification, but if you do and properly escape the braces by doubling them you get the correct result >>> 'From {{"value": 1}}, value={value}'.format(value=1) 'From {"value": 1}, value=1' From jussi.piitulainen at helsinki.fi Tue Feb 21 09:50:19 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 21 Feb 2017 16:50:19 +0200 Subject: str.format fails with JSON? References: Message-ID: carlopires at gmail.com writes: > Hi, > > When I run this piece of code: > > 'From {"value": 1}, value={value}'.format(value=1) > > Python complains about the missing "value" parameter (2.7.12 and > 3.6.x): Perhaps you know this, but just to be sure, and for the benefit of any reader who doesn't: double the braces in the format string when you don't mean them to be interpreted as a parameter. > But according to the format string syntax > (https://docs.python.org/2/library/string.html): [- -] > So according to the specification, {value} should be recognized as a > valid format string identifier and {"value"} should be ignored. > > Python seems to not follow the specification in the documentation. > Anything inside the keys is accepted as identifier. I think raising an error is more helpful than ignoring it. I think. From python at deborahswanson.net Tue Feb 21 10:16:48 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 07:16:48 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <0jboac9lljerue8iafeag8fdf3am64k29q@4ax.com> Message-ID: <004001d28c55$85d5b610$27b23dae@sambora> Dennis Lee Bieber wrote, on February 21, 2017 4:19 AM > > On Mon, 20 Feb 2017 18:16:14 -0800, "Deborah Swanson" > declaimed the following: > > > >Yes, I've used Powershell, a couple decades ago, but it would be a > > Uhm... PowerShell 1.0 only came out ONE decade ago... > > Be afraid -- PowerShell has supposedly gone open-source > with versions for OS-X and Ubuntu! > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ Really? We used software called Powershell a couple decades ago in the 90s, as an improvement on the DOS box. I didn't like it much and I was going through software like candy those days. Maybe that version disappeared. It may have re-emerged in an improved form a decade later. My health collapsed in 2003, and I was not using any unfamiliar software until a couple years ago, so I guess I've never used something called Powershell that appeared a decade ago. From carlopires at gmail.com Tue Feb 21 10:19:24 2017 From: carlopires at gmail.com (Carlo Pires) Date: Tue, 21 Feb 2017 07:19:24 -0800 (PST) Subject: str.format fails with JSON? In-Reply-To: References: Message-ID: <79d89827-289b-47ac-9da8-f22fdad31a6a@googlegroups.com> Em ter?a-feira, 21 de fevereiro de 2017 11:39:13 UTC-3, Chris Angelico escreveu: > On Wed, Feb 22, 2017 at 1:23 AM, Carlo Pires wrote: > > Hi, > > > > When I run this piece of code: > > > > 'From {"value": 1}, value={value}'.format(value=1) > > Firstly, this code is in error; I suspect you wanted a couple of > literal braces, so what you want is: What I wanted was an output without errors, like: >From {"value": 1}, value=1 I think Python should ignore value not matching integers or identifiers. This will make str.format more robust and usable. It cannot be used if the text has JSON as string, for instance. From rosuav at gmail.com Tue Feb 21 10:28:10 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 22 Feb 2017 02:28:10 +1100 Subject: str.format fails with JSON? In-Reply-To: <79d89827-289b-47ac-9da8-f22fdad31a6a@googlegroups.com> References: <79d89827-289b-47ac-9da8-f22fdad31a6a@googlegroups.com> Message-ID: On Wed, Feb 22, 2017 at 2:19 AM, Carlo Pires wrote: > What I wanted was an output without errors, like: > > From {"value": 1}, value=1 > > I think Python should ignore value not matching integers or identifiers. This will make str.format more robust and usable. It cannot be used if the text has JSON as string, for instance. No, Python should most definitely not just ignore them. If the behaviour is changed, it should be to catch the error in a more friendly way, stating that a format directive was not a valid identifier. To get the behaviour you want, double the braces that you want to be literal, as I did in my example in my first reply. Format strings are a mini-language. Like with any other, you need to speak the grammar of that language. To include quotes and apostrophes in a string literal, you need to either escape them with backslashes, or use something else to surround the string; and since a backslash can escape a quote, a backslash must itself be escaped. That's the nature of the mini-language of string literals. When you layer in the format codes, other characters gain meaning. If, instead, you layer in printf formatting, braces won't be significant, but percent signs will be. Etcetera. You can't escape this. (Pun intended.) ChrisA From rosuav at gmail.com Tue Feb 21 10:30:27 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 22 Feb 2017 02:30:27 +1100 Subject: Python application launcher (for Python code) In-Reply-To: <004001d28c55$85d5b610$27b23dae@sambora> References: <0jboac9lljerue8iafeag8fdf3am64k29q@4ax.com> <004001d28c55$85d5b610$27b23dae@sambora> Message-ID: On Wed, Feb 22, 2017 at 2:16 AM, Deborah Swanson wrote: > Really? We used software called Powershell a couple decades ago in the > 90s, as an improvement on the DOS box. I didn't like it much and I was > going through software like candy those days. Maybe that version > disappeared. It may have re-emerged in an improved form a decade later. With a name like that, there've almost certainly been multiple things using it. The PowerShell that comes with modern Windowses came out in 2006, and IMO it's less of a shell language and more of a scripting language. And if I want to use a scripting language for program flow control, I'll use Python or Pike, fankyouwerrymuch. ChrisA From python at deborahswanson.net Tue Feb 21 10:43:51 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 07:43:51 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <004d01d28c59$4d1b52e0$27b23dae@sambora> BartC wrote, on February 21, 2017 5:52 AM > > On 20/02/2017 15:44, Deborah Swanson wrote: > > Ben Finney wrote, on February 19, 2017 11:27 PM > >> > >> "Deborah Swanson" writes: > >> > >>> I could probably write this myself, but I'm wondering if > this hasn't > > > >>> already been done many times. > >> > >> Can you describe what you are looking for, in enough > detail that we > >> can know whether it's already been done as you want it? > > > I deliberately left the question open-ended because I'm > curious what's > > out there. I've studied and practiced Python for a little > over a year, > > but I've spent that time mostly writing my own code and I > don't really > > know much about what and where to look for in modules and packages. > > > > Basically, I now have quite a few Python programs I use frequently, > > and as time goes on my collection and uses of it will grow. > Right now > > I just want a way to select which one I'd like to run and > run it. I'd > > like it to be a standalone application and some sort of system of > > categories would be nice. > > If you use Windows, then the OS can take care of this. You > don't need a > separate application, just open up a directory showing a list of .py > files, and click on one. > > Then if '.py' is correctly associated with the right Python > version, it > will run Python on it. > > To group into categories, just put the .py files into separate, aptly > named directories. > > The only problem I found, when launching in GUI mode, was that a > console-mode Python program would briefly open a console window that > would then promptly disappear if it finished immediately and > there was > no interaction. > > -- > bartc Yes, I can see that only one person who has responded so far is remotely familiar with what an application launcher is, and that person only alluded to one, and hinted at some features I saw that could possibly be used as an application launcher. I like Linux for this job, as it has a number of capabilities that Windows doesn't have, and I was looking for an improvement on what I can do in Windows. If you do a lot of computing it's nice to have tools and code you commonly use, and only the ones you've chosen, conveniently available from one interface. This interface could have other functionalities itself. I was asking if anyone knew of Python code that acts in this way, and it appears so far that the answer is no. Deborah From bob.lunnon at cantab.net Tue Feb 21 10:47:16 2017 From: bob.lunnon at cantab.net (Robert William Lunnon) Date: Tue, 21 Feb 2017 15:47:16 -0000 (UTC) Subject: problems installing pylab Message-ID: <49261.10.2.0.9.1487692036.squirrel@secure2.aluminati.net> Dear Python I am trying to install pylab alongside python 3.6. However when I type python -m pip install pylab I get the message No module named site In the documentation [documentation for installing python modules in python 3.6.0 documentation] it says: The above example assumes that the option to adjust the system PATH environment variable was selected when installing python. How do I do this? I am running Windows 10 Looking forward to hearing from you Bob From python at deborahswanson.net Tue Feb 21 10:53:11 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 07:53:11 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <005701d28c5a$9af52cb0$27b23dae@sambora> Chris Angelico wrote, on February 21, 2017 7:30 AM > > On Wed, Feb 22, 2017 at 2:16 AM, Deborah Swanson > wrote: > > Really? We used software called Powershell a couple decades ago in > > the 90s, as an improvement on the DOS box. I didn't like it much and I > > was going through software like candy those days. Maybe that version > > disappeared. It may have re-emerged in an improved form a decade > > later. > > With a name like that, there've almost certainly been > multiple things using it. The PowerShell that comes with > modern Windowses came out in 2006, and IMO it's less of a > shell language and more of a scripting language. And if I > want to use a scripting language for program flow control, > I'll use Python or Pike, fankyouwerrymuch. > > ChrisA Yes, the name "Powershell" would have an appeal to a certain kind of would-be "power user", if anyone uses that term anymore. It makes sense that the idea would keep cropping up. And my sentiments exactly on scripting languages other than straight Python, which is why I was originally asking if anyone knew of an application launcher, written in Python for launching Python code. From grant.b.edwards at gmail.com Tue Feb 21 11:38:51 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 21 Feb 2017 16:38:51 +0000 (UTC) Subject: Python application launcher (for Python code) References: <004d01d28c59$4d1b52e0$27b23dae@sambora> Message-ID: On 2017-02-21, Deborah Swanson wrote: > Yes, I can see that only one person who has responded so far is remotely > familiar with what an application launcher is, and that person only > alluded to one, and hinted at some features I saw that could possibly be > used as an application launcher. I think I've read every posting in this thread, and I _still_ don't know what you mean by "application launcher". > I like Linux for this job, as it has a number of capabilities that > Windows doesn't have, and I was looking for an improvement on what I can > do in Windows. If you do a lot of computing it's nice to have tools and > code you commonly use, and only the ones you've chosen, conveniently > available from one interface. Isn't that what shortcuts and start menu and all the other shiny-bits on GUI desktop environments do? [I don't use a "desktop" so maybe I'm wrong.] The "root menu" provided by many window managers is also a common way to do this (that's something I do use). A command-line shell like bash is also a common way to do this. And yes, there are plenty of them available for Windows. > This interface could have other functionalities itself. > > I was asking if anyone knew of Python code that acts in this way, What way? I can't figure out how what you want is not what's provided by any of the standard GUI desktop/shells or command-line shells. Are you looking for a 1980s-style menu driven UI that were common before GUIs came into widespread use? > and it appears so far that the answer is no. I think the problem is that you're unable to describe what you want in a way that's understandable to the people that might be able to answer your question. -- Grant Edwards grant.b.edwards Yow! VICARIOUSLY experience at some reason to LIVE!! gmail.com From eryksun at gmail.com Tue Feb 21 12:09:48 2017 From: eryksun at gmail.com (eryk sun) Date: Tue, 21 Feb 2017 17:09:48 +0000 Subject: Python application launcher (for Python code) In-Reply-To: <58ab96fd$0$1595$c3e8da3$5496439d@news.astraweb.com> References: <8560k5mie8.fsf@benfinney.id.au> <003f01d28b90$385d8450$27b23dae@sambora> <58ab96fd$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Feb 21, 2017 at 1:25 AM, Steve D'Aprano wrote: > (2) Add each category to the PYTHONPATH. One easy way to do so is by adding > the directories to a .pth file. PYTHONPATH isn't a synonym for sys.path. The PYTHONPATH environment variable gets used by every installed interpreter, which can be a problem if it's set permanently. Using .pth files, as suggested here, is one way around this. In general, virtual environments [1] are convenient for development. Install what you need for a project from PyPI, a local wheel cache, or directly from VCS. IDEs such as PyCharm have integrated support for virtual environments. Entry-point scripts in a virtual environment should have a shebang that executes the environment's interpreter, so they can be run even when the environment isn't active. [1]: https://docs.python.org/3/library/venv.html The rest of this message is just for Windows. To support shebangs, make sure that .py[w] files are associated with the py launcher, which is installed with Python 3. The installer defaults to associating .py[w] files with the Python.File and Python.NoConFile program identifiers (ProgIds). Change it back if you've modified it. Don't use the command prompt's assoc command for this. Use the control panel "Default Programs -> Set Associations" dialog (i.e. associate a file type or protocol with a program). The descriptions for .py and .pyw in this dialog should be "Python File" and "Python File (no console)". When you double click on the entries for .py and .pyw, the associated program should be "Python" with a rocket on the icon. If not, and you see it in the list, select it. Otherwise ensure that the Python.File and Python.NoConFile ProgIds are configured to use the py.exe and pyw.exe launchers. If the launcher was installed for all users (i.e. per machine), you can use ftype in an *elevated* command prompt to query and set Python.[NoCon]File. For example: C:\>ftype Python.File Python.File="C:\Windows\py.exe" "%1" %* C:\>ftype Python.NoConFile Python.NoConFile="C:\Windows\pyw.exe" "%1" %* If the launcher was installed for the current user only (i.e. per user), then you cannot use the ftype command since it only accesses the local-machine settings. Also, even if you've installed for all users, maybe you have an old value in HKCU, which takes precedence over HKLM when the system reads the merged HKCR view. Use regedit to manually edit the per-user keys if they exist. If "HKCU\Software\Classes\Python.File\shell\open\command" exists, its default value should be a template command to run the fully-qualified path to py.exe plus "%1" for the target script and %* for the command-line arguments, as shown above. Similarly, if "HKCU\Software\Classes\Python.NoConFile\shell\open\command" exists, it should run the same command but with pyw.exe substituted for py.exe. From jf_byrnes at comcast.net Tue Feb 21 12:50:50 2017 From: jf_byrnes at comcast.net (Jim) Date: Tue, 21 Feb 2017 11:50:50 -0600 Subject: Python application launcher (for Python code) In-Reply-To: <004d01d28c59$4d1b52e0$27b23dae@sambora> References: <004d01d28c59$4d1b52e0$27b23dae@sambora> Message-ID: On 02/21/2017 09:43 AM, Deborah Swanson wrote: > BartC wrote, on February 21, 2017 5:52 AM >> >> On 20/02/2017 15:44, Deborah Swanson wrote: >>> Ben Finney wrote, on February 19, 2017 11:27 PM >>>> >>>> "Deborah Swanson" writes: >>>> >>>>> I could probably write this myself, but I'm wondering if >> this hasn't >>> >>>>> already been done many times. >>>> >>>> Can you describe what you are looking for, in enough >> detail that we >>>> can know whether it's already been done as you want it? >> >>> I deliberately left the question open-ended because I'm >> curious what's >>> out there. I've studied and practiced Python for a little >> over a year, >>> but I've spent that time mostly writing my own code and I >> don't really >>> know much about what and where to look for in modules and packages. >>> >>> Basically, I now have quite a few Python programs I use frequently, >>> and as time goes on my collection and uses of it will grow. >> Right now >>> I just want a way to select which one I'd like to run and >> run it. I'd >>> like it to be a standalone application and some sort of system of >>> categories would be nice. >> >> If you use Windows, then the OS can take care of this. You >> don't need a >> separate application, just open up a directory showing a list of .py >> files, and click on one. >> >> Then if '.py' is correctly associated with the right Python >> version, it >> will run Python on it. >> >> To group into categories, just put the .py files into separate, aptly >> named directories. >> >> The only problem I found, when launching in GUI mode, was that a >> console-mode Python program would briefly open a console window that >> would then promptly disappear if it finished immediately and >> there was >> no interaction. >> >> -- >> bartc > > Yes, I can see that only one person who has responded so far is remotely > familiar with what an application launcher is, and that person only > alluded to one, and hinted at some features I saw that could possibly be > used as an application launcher. > > I like Linux for this job, as it has a number of capabilities that > Windows doesn't have, and I was looking for an improvement on what I can > do in Windows. If you do a lot of computing it's nice to have tools and > code you commonly use, and only the ones you've chosen, conveniently > available from one interface. This interface could have other > functionalities itself. > > I was asking if anyone knew of Python code that acts in this way, and it > appears so far that the answer is no. > > Deborah > If you switch to Linux you might look at a program called Drawers. I use it on Ubuntu 14.04 and 16.04. It sits on the launcher panel and if you click it it opens up and you can click on programs to start them. Most things you can drag and drop on it, some things you must do a little editing to get them to start. It is written in Python I think. On Mint 18 I use a program called MyLauncher. It sits on the top panel and will drop down a menu of programs to start. It's a little more hands on as you must add your programs to a config file to get them in the launcher. Don't think this one is written in Python. Both are in the OS's repositories. Regards, Jim From pavol.lisy at gmail.com Tue Feb 21 13:00:19 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Tue, 21 Feb 2017 19:00:19 +0100 Subject: Python application launcher (for Python code) In-Reply-To: <005701d28c5a$9af52cb0$27b23dae@sambora> References: <005701d28c5a$9af52cb0$27b23dae@sambora> Message-ID: On 2/21/17, Deborah Swanson wrote: > Chris Angelico wrote, on February 21, 2017 7:30 AM >> >> On Wed, Feb 22, 2017 at 2:16 AM, Deborah Swanson >> wrote: >> > Really? We used software called Powershell a couple decades ago in >> > the 90s, as an improvement on the DOS box. I didn't like it much and > I >> > was going through software like candy those days. Maybe that version > >> > disappeared. It may have re-emerged in an improved form a decade >> > later. >> >> With a name like that, there've almost certainly been >> multiple things using it. The PowerShell that comes with >> modern Windowses came out in 2006, and IMO it's less of a >> shell language and more of a scripting language. And if I >> want to use a scripting language for program flow control, >> I'll use Python or Pike, fankyouwerrymuch. >> >> ChrisA > > Yes, the name "Powershell" would have an appeal to a certain kind of > would-be "power user", if anyone uses that term anymore. It makes sense > that the idea would keep cropping up. > > And my sentiments exactly on scripting languages other than straight > Python, which is why I was originally asking if anyone knew of an > application launcher, written in Python for launching Python code. > > -- > https://mail.python.org/mailman/listinfo/python-list Maybe try to look at https://github.com/spyder-ide/spyder . It could satisfy your expectations. You can edit, run, debug, statically analyze python codes and more. And it is open source written in python... From best_lay at yahoo.com Tue Feb 21 13:02:50 2017 From: best_lay at yahoo.com (Wildman) Date: Tue, 21 Feb 2017 12:02:50 -0600 Subject: Request Help With ttk.Notebook Tabs Message-ID: Python 3.4.2 Linux platform I am working on a program that has tabs created with ttk.Notebook. The code for creating the tabs is working but there is one thing I have not been able to figure out. As is, the tabs are located up against the lower edge of the caption bar. I would like to have them a little lower to make room above the tabs for other widgets such as labels and/or command buttons. Here is the code I use to create the window and tabs... class Window(tk.Frame): def __init__(self, master=None): tk.Frame.__init__(self, master) master.title("My Program") nb = ttk.Notebook(root, width=600, height=340) tab1 = tk.Frame(nb) tab2 = tk.Frame(nb) nb.add(tab1, text="Tab1") nb.add(tab2, text="Tab2") nb.pack(expand=1, fill="both") I have tried the grid and place methods to move the tabs down but neither works. I have tried the methods both before and after the pack method. Here is an example of what I have tried... nb.grid(row=2, column=2) or nb.place(x=10, y=10) Would be appreciated if anyone could provide any guidance. -- GNU/Linux user #557453 Keyboard not detected! Press any key to continue... From rgaddi at highlandtechnology.invalid Tue Feb 21 13:14:00 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 21 Feb 2017 10:14:00 -0800 Subject: Python application launcher (for Python code) In-Reply-To: References: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> <008401d28be8$7a5ceeb0$27b23dae@sambora> Message-ID: On 02/20/2017 06:16 PM, Deborah Swanson wrote: > [snip lots about using Windows but rather be > using Linux but not wanting to have to spend lots of > energy switching right now] You know, I'm always reluctant to recommend it, because it can definitely get you tied in knots. But you're about the ideal candidate for looking into https://www.cygwin.com/ When my primary PC had to be Windows I used it fairly successfully for years. It works quite well for many things, and can be bullied into working well enough for others. It will probably do what you seem like you might want, which is a functioning shell under Windows, and yes it works all the way back to XP. At the low low price of free, you'll definitely get your money's worth. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From best_lay at yahoo.com Tue Feb 21 13:22:11 2017 From: best_lay at yahoo.com (Wildman) Date: Tue, 21 Feb 2017 12:22:11 -0600 Subject: Request Help With ttk.Notebook Tabs References: Message-ID: <6bKdnYiihpHOGDHFnZ2dnUU7-K-dnZ2d@giganews.com> On Tue, 21 Feb 2017 12:02:50 -0600, Wildman wrote: > Python 3.4.2 > Linux platform > > > I am working on a program that has tabs created with ttk.Notebook. > The code for creating the tabs is working but there is one thing I > have not been able to figure out. As is, the tabs are located up > against the lower edge of the caption bar. I would like to have > them a little lower to make room above the tabs for other widgets > such as labels and/or command buttons. Here is the code I use to > create the window and tabs... > > class Window(tk.Frame): > def __init__(self, master=None): > tk.Frame.__init__(self, master) > master.title("My Program") > nb = ttk.Notebook(root, width=600, height=340) > tab1 = tk.Frame(nb) > tab2 = tk.Frame(nb) > nb.add(tab1, text="Tab1") > nb.add(tab2, text="Tab2") > nb.pack(expand=1, fill="both") > > I have tried the grid and place methods to move the tabs down but > neither works. I have tried the methods both before and after the > pack method. Here is an example of what I have tried... > > nb.grid(row=2, column=2) > or > nb.place(x=10, y=10) > > Would be appreciated if anyone could provide any guidance. I posted too quickly. I found my answer... nb.pack(expand=1, fill="both", padx=20, pady=20) Sorry. -- GNU/Linux user #557453 The cow died so I don't need your bull! From python at mrabarnett.plus.com Tue Feb 21 13:22:31 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 21 Feb 2017 18:22:31 +0000 Subject: Request Help With ttk.Notebook Tabs In-Reply-To: References: Message-ID: <0583be69-3393-aaa7-5a06-619a3dea3fa4@mrabarnett.plus.com> On 2017-02-21 18:02, Wildman via Python-list wrote: > Python 3.4.2 > Linux platform > > > I am working on a program that has tabs created with ttk.Notebook. > The code for creating the tabs is working but there is one thing I > have not been able to figure out. As is, the tabs are located up > against the lower edge of the caption bar. I would like to have > them a little lower to make room above the tabs for other widgets > such as labels and/or command buttons. Here is the code I use to > create the window and tabs... > > class Window(tk.Frame): > def __init__(self, master=None): > tk.Frame.__init__(self, master) > master.title("My Program") > nb = ttk.Notebook(root, width=600, height=340) > tab1 = tk.Frame(nb) > tab2 = tk.Frame(nb) > nb.add(tab1, text="Tab1") > nb.add(tab2, text="Tab2") > nb.pack(expand=1, fill="both") > > I have tried the grid and place methods to move the tabs down but > neither works. I have tried the methods both before and after the > pack method. Here is an example of what I have tried... > > nb.grid(row=2, column=2) > or > nb.place(x=10, y=10) > > Would be appreciated if anyone could provide any guidance. > If you want other widgets above the Notebook widget, why don't you just put them there in a frame? From best_lay at yahoo.com Tue Feb 21 13:36:42 2017 From: best_lay at yahoo.com (Wildman) Date: Tue, 21 Feb 2017 12:36:42 -0600 Subject: Request Help With ttk.Notebook Tabs References: <0583be69-3393-aaa7-5a06-619a3dea3fa4@mrabarnett.plus.com> Message-ID: On Tue, 21 Feb 2017 18:22:31 +0000, MRAB wrote: > On 2017-02-21 18:02, Wildman via Python-list wrote: >> Python 3.4.2 >> Linux platform >> >> >> I am working on a program that has tabs created with ttk.Notebook. >> The code for creating the tabs is working but there is one thing I >> have not been able to figure out. As is, the tabs are located up >> against the lower edge of the caption bar. I would like to have >> them a little lower to make room above the tabs for other widgets >> such as labels and/or command buttons. Here is the code I use to >> create the window and tabs... >> >> class Window(tk.Frame): >> def __init__(self, master=None): >> tk.Frame.__init__(self, master) >> master.title("My Program") >> nb = ttk.Notebook(root, width=600, height=340) >> tab1 = tk.Frame(nb) >> tab2 = tk.Frame(nb) >> nb.add(tab1, text="Tab1") >> nb.add(tab2, text="Tab2") >> nb.pack(expand=1, fill="both") >> >> I have tried the grid and place methods to move the tabs down but >> neither works. I have tried the methods both before and after the >> pack method. Here is an example of what I have tried... >> >> nb.grid(row=2, column=2) >> or >> nb.place(x=10, y=10) >> >> Would be appreciated if anyone could provide any guidance. >> > If you want other widgets above the Notebook widget, why don't you just > put them there in a frame? Thanks for the reply. That approach would work but I found a simple method that serves the purpose using padx/pady. I was doing my research in the areas of grid and place. Didn't think about pack. -- GNU/Linux user #557453 The cow died so I don't need your bull! From tjreedy at udel.edu Tue Feb 21 13:40:39 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 21 Feb 2017 13:40:39 -0500 Subject: problems installing pylab In-Reply-To: <49261.10.2.0.9.1487692036.squirrel@secure2.aluminati.net> References: <49261.10.2.0.9.1487692036.squirrel@secure2.aluminati.net> Message-ID: On 2/21/2017 10:47 AM, Robert William Lunnon wrote: > I am trying to install pylab alongside python 3.6. On Windows 10 as you said at the bottom. > However when I type > > python -m pip install pylab Try py -3.6 -m pip install pylab The '.6' should not be needed, but will make the command fail if you another 3.x but not 3.6. > I get the message > > No module named site If any python.exe ran, /Lib/site.py should be present and you should not see that. What happens if you enter just 'python'? > In the documentation [documentation for installing python modules in > python 3.6.0 documentation] it says: The above example assumes that the > option to adjust the system PATH environment variable was selected when > installing python. In most cases, one is better to just use the py command on Windows. If there is only one python.exe installed >py will start it. If multiple pythons are installed, the latest is the default but you can use the flags to select -2 or -3 to select the latest of each or -x.y to select a specific version that is not the latest of the x series. (I still type 'python' (now for 3.6) because a) it works since I made the install selection, and b) 20 years of habit ;-).) > How do I do this? I am running Windows 10 You can repair or replace the existing install, but I don't suggest it. -- Terry Jan Reedy From grant.b.edwards at gmail.com Tue Feb 21 14:21:14 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 21 Feb 2017 19:21:14 +0000 (UTC) Subject: Python application launcher (for Python code) References: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> <008401d28be8$7a5ceeb0$27b23dae@sambora> Message-ID: On 2017-02-21, Rob Gaddi wrote: > On 02/20/2017 06:16 PM, Deborah Swanson wrote: > > > [snip lots about using Windows but rather be > > using Linux but not wanting to have to spend lots of > > energy switching right now] > > You know, I'm always reluctant to recommend it, because it can > definitely get you tied in knots. But you're about the ideal candidate > for looking into https://www.cygwin.com/ There are other ways to get "shell and unix-utilities" for Windows that are less "drastic" than Cygwin: they don't try to provide a complete a Unix development environment or the illusion of Unix filesystem semantics. [Remember: a Unix shell without a set of accompanying utilitys is pretty useless, since Unix shells don't have all of the "built-in" commands that Windows shells do.] NB: I haven't used any of these lately... http://unxutils.sourceforge.net/ http://www.mingw.org/ (look for msys) https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058 (git's bash-terminal) https://sourceforge.net/projects/ezwinports/ There were a couple another very highly recommended commercial Unix shell+utilities products (MKS Toolkit, Interix) that I used to use. But AFAIK, they all got bought and killed by Microsoft. -- Grant Edwards grant.b.edwards Yow! Please come home with at me ... I have Tylenol!! gmail.com From kwpolska at gmail.com Tue Feb 21 14:26:22 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Tue, 21 Feb 2017 20:26:22 +0100 Subject: Python application launcher (for Python code) In-Reply-To: References: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> <008401d28be8$7a5ceeb0$27b23dae@sambora> Message-ID: On 21 February 2017 at 20:21, Grant Edwards wrote: > On 2017-02-21, Rob Gaddi wrote: >> On 02/20/2017 06:16 PM, Deborah Swanson wrote: >> >> > [snip lots about using Windows but rather be >> > using Linux but not wanting to have to spend lots of >> > energy switching right now] >> >> You know, I'm always reluctant to recommend it, because it can >> definitely get you tied in knots. But you're about the ideal candidate >> for looking into https://www.cygwin.com/ > > There are other ways to get "shell and unix-utilities" for Windows > that are less "drastic" than Cygwin: they don't try to provide a > complete a Unix development environment or the illusion of Unix > filesystem semantics. [Remember: a Unix shell without a set of > accompanying utilitys is pretty useless, since Unix shells don't have > all of the "built-in" commands that Windows shells do.] > > NB: I haven't used any of these lately... > > http://unxutils.sourceforge.net/ > http://www.mingw.org/ (look for msys) > https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058 (git's bash-terminal) > https://sourceforge.net/projects/ezwinports/ > > There were a couple another very highly recommended commercial Unix > shell+utilities products (MKS Toolkit, Interix) that I used to > use. But AFAIK, they all got bought and killed by Microsoft. Git Bash, or basically msys, is pretty reasonable. But if you are on Windows 10, you might like the built-in Windows Subsystem for Linux (aka Bash on Ubuntu on Windows) more ? it?s real Linux that runs alongside Windows, but less crazy than Cygwin. -- Chris Warrick PGP: 5EAAEA16 From tjreedy at udel.edu Tue Feb 21 14:38:35 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 21 Feb 2017 14:38:35 -0500 Subject: Request Help With ttk.Notebook Tabs In-Reply-To: References: Message-ID: On 2/21/2017 1:02 PM, Wildman via Python-list wrote: > Python 3.4.2 > Linux platform > > > I am working on a program that has tabs created with ttk.Notebook. > The code for creating the tabs is working but there is one thing I > have not been able to figure out. As is, the tabs are located up > against the lower edge of the caption bar. I would like to have > them a little lower to make room above the tabs for other widgets > such as labels and/or command buttons. Here is the code I use to > create the window and tabs... > > class Window(tk.Frame): > def __init__(self, master=None): > tk.Frame.__init__(self, master) > master.title("My Program") > nb = ttk.Notebook(root, width=600, height=340) > tab1 = tk.Frame(nb) > tab2 = tk.Frame(nb) > nb.add(tab1, text="Tab1") > nb.add(tab2, text="Tab2") > nb.pack(expand=1, fill="both") This is too minimal. If I copy, paste, and run the above, it will fail. I will not guess what other code (which you should minimize) you actually had to make this run. See https://stackoverflow.com/help/mcve, which is *great* advice that most SO unfortunately ignore. > I have tried the grid and place methods to move the tabs down but > neither works. I have tried the methods both before and after the > pack method. Here is an example of what I have tried... > > nb.grid(row=2, column=2) > or > nb.place(x=10, y=10) > > Would be appreciated if anyone could provide any guidance. I have not specifically used Notebook, but 0. You MUST NOT mix geometry methods within a particular toplevel or frame. From what you wrote (without complete code), you might have. 1. All widgets that go within a particular container widget must have that container as parent. Although you put the notebook code in the window(frame) init, its parent is root, so it will appear outside the window, if indeed the window is made visible (which it is not in the code you posted. So either get rid of the window(frame) code or put the notebook in the windows instance nb = ttk.Notebook(self, width=600, height=340) The former is easier for minimal proof-of-concept testing, the latter may be better for final development and maintenance. 2. Packing is order dependent. 3. Gridding is not order dependent, but empty rows and column take no space. In any case, the following works for me on 3.6 and Win10. import tkinter as tk from tkinter import ttk root = tk.Tk() e = ttk.Label(root, text='Widget outside of notebook') nb = ttk.Notebook(root, width=600, height=340) tab1 = ttk.Frame(nb) tab2 = ttk.Frame(nb) ttk.Label(tab1, text='Text on notebook tab 1').pack() ttk.Label(tab2, text='Tab 2 text').pack() nb.add(tab1, text="Tab1") nb.add(tab2, text="Tab2") e.grid(row=0) nb.grid(row=1) #root.mainloop() # not needed when run from IDLE -- Terry Jan Reedy From python at deborahswanson.net Tue Feb 21 14:59:49 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 11:59:49 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <004801d28c7d$106f83b0$27b23dae@sambora> Jim wrote, on February 21, 2017 9:51 AM > > On 02/21/2017 09:43 AM, Deborah Swanson wrote: > > I like Linux for this job, as it has a number of capabilities that > > Windows doesn't have, and I was looking for an improvement on what I > > can do in Windows. If you do a lot of computing it's nice to have > > tools and code you commonly use, and only the ones you've chosen, > > conveniently available from one interface. This interface could have > > other functionalities itself. > > > > I was asking if anyone knew of Python code that acts in this way, and > > it appears so far that the answer is no. > > > > Deborah > > > > If you switch to Linux you might look at a program called Drawers. I use > it on Ubuntu 14.04 and 16.04. It sits on the launcher panel and if you > click it it opens up and you can click on programs to start them. Most > things you can drag and drop on it, some things you must do a little > editing to get them to start. It is written in Python I think. > > On Mint 18 I use a program called MyLauncher. It sits on the top panel > and will drop down a menu of programs to start. It's a little more hands > on as you must add your programs to a config file to get them in the > launcher. Don't think this one is written in Python. > > Both are in the OS's repositories. > > Regards, Jim Thanks, Jim. This is exactly the type of program I was looking for. A number of people have made some very helpful suggestions on ways to get a Linux shell installed in Windows, but since I really don't want to burn up a lot of time (any time at all actually) for a Windows solution, I think I'll have a go with Cooperative Linux first. It's a way of installing Linux on a separate partition that runs concurrently with Windows and doesn't require booting back and forth between them - why go cheap when you can have the real thing, which is what you originally wanted anyway? Might not be that much trouble either, though I think the installation is tricky. I'll try both Drawers and taking a look at MyLauncher's code. Chances are MyLauncher, or the parts of it I want, can be rewritten in Python. But if Drawers is sufficient and no better recommendations surface, I'll probably just go with that. Originally I wanted Python code that would run in Windows (for minimal fuss to get an application launcher now), and I might look at Drawers for that. But chances are that an application launcher would make heavy use of the os, and in Drawers that would be Linux and not Windows. I think Cooperative Linux should be my next stop for this project. Deborah From python at deborahswanson.net Tue Feb 21 15:11:49 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 12:11:49 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <004f01d28c7e$bc90a650$27b23dae@sambora> Grant Edwards wrote, on February 21, 2017 11:21 AM > > On 2017-02-21, Rob Gaddi wrote: > > On 02/20/2017 06:16 PM, Deborah Swanson wrote: > > > > > [snip lots about using Windows but rather be > > > using Linux but not wanting to have to spend lots of > > > energy switching right now] > > > > You know, I'm always reluctant to recommend it, because it can > > definitely get you tied in knots. But you're about the > ideal candidate > > for looking into https://www.cygwin.com/ > > There are other ways to get "shell and unix-utilities" for > Windows that are less "drastic" than Cygwin: they don't try > to provide a complete a Unix development environment or the > illusion of Unix filesystem semantics. [Remember: a Unix > shell without a set of accompanying utilitys is pretty > useless, since Unix shells don't have all of the "built-in" > commands that Windows shells do.] > > NB: I haven't used any of these lately... > http://unxutils.sourceforge.net/ http://www.mingw.org/ (look for msys) https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058 (git's bash-terminal) https://sourceforge.net/projects/ezwinports/ There were a couple another very highly recommended commercial Unix shell+utilities products (MKS Toolkit, Interix) that I used to use. But AFAIK, they all got bought and killed by Microsoft. Thanks Grant for these suggestions. Microsoft may have killed them at the time, but useful software can resurface a number of ways. I'll keep your list and go looking for them someday. I think something like Drawers or MyLauncher that Jim mentioned would fit the bill for my immediate use, but old Unix utilities sound kinda neat in themselves, and well worth looking into in my free time. (haha, free time you say? But seriously I'd like to check them out.) Deborah From python at deborahswanson.net Tue Feb 21 15:20:06 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 12:20:06 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <005001d28c7f$e4d8bbb0$27b23dae@sambora> Chris Warrick wrote, on February 21, 2017 11:26 AM > > On 21 February 2017 at 20:21, Grant Edwards > wrote: > > On 2017-02-21, Rob Gaddi wrote: > >> On 02/20/2017 06:16 PM, Deborah Swanson wrote: > >> > >> > [snip lots about using Windows but rather be > >> > using Linux but not wanting to have to spend lots of energy > >> > switching right now] > >> > >> You know, I'm always reluctant to recommend it, because it can > >> definitely get you tied in knots. But you're about the ideal > >> candidate for looking into https://www.cygwin.com/ > > > > There are other ways to get "shell and unix-utilities" for Windows > > that are less "drastic" than Cygwin: they don't try to provide a > > complete a Unix development environment or the illusion of Unix > > filesystem semantics. [Remember: a Unix shell without a set of > > accompanying utilitys is pretty useless, since Unix shells > don't have > > all of the "built-in" commands that Windows shells do.] > > > > NB: I haven't used any of these lately... > > > > http://unxutils.sourceforge.net/ > > http://www.mingw.org/ (look for msys) > > > https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f 058 (git's bash-terminal) > https://sourceforge.net/projects/ezwinports/ > > There were a couple another very highly recommended commercial Unix > shell+utilities products (MKS Toolkit, Interix) that I used to > use. But AFAIK, they all got bought and killed by Microsoft. Git Bash, or basically msys, is pretty reasonable. But if you are on Windows 10, you might like the built-in Windows Subsystem for Linux (aka Bash on Ubuntu on Windows) more - it's real Linux that runs alongside Windows, but less crazy than Cygwin. -- Chris Warrick PGP: 5EAAEA16 Thanks Chris, I kind of thought useful software like this would resurface in some form, and you given me msys' new name (Git Bash), or one of them anyway. From python at deborahswanson.net Tue Feb 21 15:33:01 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 12:33:01 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <004f01d28c7e$bc90a650$27b23dae@sambora> Message-ID: <005401d28c81$b2964cb0$27b23dae@sambora> I wrote, on February 21, 2017 12:12 PM > > Grant Edwards wrote, on February 21, 2017 11:21 AM > > NB: I haven't used any of these lately... > > > http://unxutils.sourceforge.net/ > http://www.mingw.org/ (look for msys) > https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058 > (git's bash-terminal) > https://sourceforge.net/projects/ezwinports/ > > There were a couple another very highly recommended commercial Unix > shell+utilities products (MKS Toolkit, Interix) that I used to > use. But AFAIK, they all got bought and killed by Microsoft. > > > Thanks Grant for these suggestions. Microsoft may have killed > them at the time, but useful software can resurface a number > of ways. I'll keep your list and go looking for them > someday. I think something like Drawers or MyLauncher that > Jim mentioned would fit the bill for my immediate use, but > old Unix utilities sound kinda neat in themselves, and well > worth looking into in my free time. (haha, free time you say? > But seriously I'd like to check them out.) > > Deborah Whoops, I just noticed on rereading that only MKS Toolkit and Interix were bought and killed by Microsoft, not your entire list. Guess they can't buy open source code anyway. While this is sad for MKS Toolkit and Interix, it does mean the others are probably still available as open source. I used to work in a partially Unix house in mechanical engineering at Ingersoll-Rand. I did DOS and the brand new ever Windows (1.0?) at the time, but I remember looking at MKS Toolkit on the Unix machines to see if it could be adapted to Windows. Very sad if it is lost for good. From ethan at stoneleaf.us Tue Feb 21 16:13:10 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 21 Feb 2017 13:13:10 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <004801d28c7d$106f83b0$27b23dae@sambora> References: <004801d28c7d$106f83b0$27b23dae@sambora> Message-ID: <58ACAD66.6040503@stoneleaf.us> On 02/21/2017 11:59 AM, Deborah Swanson wrote: > I think I'll have a go with Cooperative Linux first It certainly looks interesting, but doesn't seem to have any activity since late 2014. -- ~Ethan~ From python at deborahswanson.net Tue Feb 21 17:58:50 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 21 Feb 2017 14:58:50 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <58ACAD66.6040503@stoneleaf.us> Message-ID: <008101d28c96$118e30c0$27b23dae@sambora> Ethan Furman wrote, on February 21, 2017 1:13 PM > > On 02/21/2017 11:59 AM, Deborah Swanson wrote: > > > I think I'll have a go with Cooperative Linux first > > It certainly looks interesting, but doesn't seem to have any > activity since late 2014. > > -- > ~Ethan~ I can't remember where I picked up the link for Cooperative Linux (aka coLinux), but the newsletter Import Python is a good bet, and it would have been in an issue no more than 2 weeks ago. Maybe somebody's trying to revive it, or just spread the word. The article certainly glowed about how useful it is. Guess I'll download and try it, it's a sourceforge page. So if it downloads and installs (that may be the tricky part), I'm probably set to go. I don't have a problem with old software and it seems some open software isn't updated for long periods of time. If it works and does something useful for me, I'm fine with it. Some of my oldest spreadsheets I use daily are about 15 years old, and so is the Excel I run them in. ;) Deborah From alfredocabrera4 at gmail.com Wed Feb 22 01:44:42 2017 From: alfredocabrera4 at gmail.com (alfredocabrera4 at gmail.com) Date: Tue, 21 Feb 2017 22:44:42 -0800 (PST) Subject: python decorator Message-ID: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> I have a python function and a decorator that works just fine. def fun_cache(function): memo = {} def wrapper(*args): if args in memo: return memo[args] else: rv = function(*args) memo[args] = rv return rv return wrapper @fun_cache def fib(n): if (n < 2): return 1 else: return fib(n-1) + fib(n-2) assert(fib(0) == 1) assert(fib(3) == 3) assert(fib(6) == 13) assert(fib(10) == 89) assert(fib(30) == 1346269) assert(fib(100) == 573147844013817084101) assert(fib(400) == 284812298108489611757988937681460995615380088782304890986477195645969271404032323901) Now I will like to call the @fun_cache decorator like this : @fun_cache(cache={}) Any help? From anonimista at outlook.com Wed Feb 22 02:06:34 2017 From: anonimista at outlook.com (Joe Anonimist) Date: Tue, 21 Feb 2017 23:06:34 -0800 (PST) Subject: [ANN] New Slack Group Dedicated to learning Python Message-ID: <46826180-2012-45f4-8a6f-2dc02d358a40@googlegroups.com> Hello all! Me and a few other Python enthusiasts started a Slack group dedicated to learning Python. All of us know the basics of Python and our goal is to acquire new skills that would help us get jobs as Python developers. Our plan is to collaborate on Python projects to get experience and become Python professionals. If you are interested send me your email address and I'll send you an invitation to join the group. If you have a Reddit account you can visit our subreddit https://www.reddit.com/r/pystudygroup/ and PM me your mail for an invitation. From cs at zip.com.au Wed Feb 22 02:19:36 2017 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 22 Feb 2017 18:19:36 +1100 Subject: python decorator In-Reply-To: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> Message-ID: <20170222071936.GA2403@cskk.homeip.net> On 21Feb2017 22:44, alfredocabrera4 at gmail.com wrote: >I have a python function and a decorator that works just fine. > >def fun_cache(function): > memo = {} > def wrapper(*args): > if args in memo: > return memo[args] > else: > rv = function(*args) > memo[args] = rv > return rv > return wrapper > > >@fun_cache >def fib(n): > if (n < 2): return 1 > else: return fib(n-1) + fib(n-2) > >assert(fib(0) == 1) >assert(fib(3) == 3) >assert(fib(6) == 13) >assert(fib(10) == 89) >assert(fib(30) == 1346269) >assert(fib(100) == 573147844013817084101) >assert(fib(400) == 284812298108489611757988937681460995615380088782304890986477195645969271404032323901) > > >Now I will like to call the @fun_cache decorator like this : > >@fun_cache(cache={}) > >Any help? You need to write "fun_cache" as a function accepting a cache argument, and returning a decorator that uses that cache. Like: def fun_cache(cache): def fun_cache_decorator(function): def wrapper(*args): ... save things in cache instead of memo ... return wrapper return fun_cache_decorator Cheers, Cameron Simpson From anonimista at outlook.com Wed Feb 22 02:25:17 2017 From: anonimista at outlook.com (Joe Anonimist) Date: Tue, 21 Feb 2017 23:25:17 -0800 (PST) Subject: [ANN] New Slack Group Dedicated to learning Python In-Reply-To: <46826180-2012-45f4-8a6f-2dc02d358a40@googlegroups.com> References: <46826180-2012-45f4-8a6f-2dc02d358a40@googlegroups.com> Message-ID: On Wednesday, 22 February 2017 08:07:01 UTC+1, Joe Anonimist wrote: > Hello all! > > Me and a few other Python enthusiasts started a Slack group dedicated to learning Python. All of us know the basics of Python and our goal is to acquire new skills that would help us get jobs as Python developers. Our plan is to collaborate on Python projects to get experience and become Python professionals. > > If you are interested send me your email address and I'll send you an invitation to join the group. If you have a Reddit account you can visit our subreddit https://www.reddit.com/r/pystudygroup/ and PM me your mail for an invitation. Invitation sent :) From alfredocabrera4 at gmail.com Wed Feb 22 02:39:38 2017 From: alfredocabrera4 at gmail.com (Argentinian Black ops lll) Date: Tue, 21 Feb 2017 23:39:38 -0800 (PST) Subject: python decorator In-Reply-To: References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <20170222071936.GA2403@cskk.homeip.net> Message-ID: Thanks for the quick response...! you saved me a lot of time, thank you! From alfredocabrera4 at gmail.com Wed Feb 22 02:49:47 2017 From: alfredocabrera4 at gmail.com (Argentinian Black ops lll) Date: Tue, 21 Feb 2017 23:49:47 -0800 (PST) Subject: python decorator In-Reply-To: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> Message-ID: <3a9a86b7-386f-4fe6-9a19-1a20812041fd@googlegroups.com> *** SOLVED *** From Cecil at decebal.nl Wed Feb 22 04:47:41 2017 From: Cecil at decebal.nl (Cecil Westerhof) Date: Wed, 22 Feb 2017 10:47:41 +0100 Subject: python decorator References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <3a9a86b7-386f-4fe6-9a19-1a20812041fd@googlegroups.com> Message-ID: <874lzmwo8y.fsf@Equus.decebal.nl> On Wednesday 22 Feb 2017 08:49 CET, Argentinian Black ops lll wrote: > *** SOLVED *** It would be nice if you shared the solution. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From breamoreboy at gmail.com Wed Feb 22 04:57:26 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 22 Feb 2017 01:57:26 -0800 (PST) Subject: problems installing pylab In-Reply-To: References: <49261.10.2.0.9.1487692036.squirrel@secure2.aluminati.net> Message-ID: <5040d3a4-629b-4562-92c9-45f27b11fed7@googlegroups.com> On Tuesday, February 21, 2017 at 3:55:40 PM UTC, Robert William Lunnon wrote: > Dear Python > > I am trying to install pylab alongside python 3.6. However when I type > > python -m pip install pylab > > I get the message > > No module named site > > In the documentation [documentation for installing python modules in > python 3.6.0 documentation] it says: The above example assumes that the > option to adjust the system PATH environment variable was selected when > installing python. > > How do I do this? > > I am running Windows 10 > > Looking forward to hearing from you > > Bob If you actually need matplotlib I suggest that you read this http://stackoverflow.com/questions/11469336/what-is-the-difference-between-pylab-and-pyplot Kindest regards. Mark Lawrence. From kasperseidem.gamer at gmail.com Wed Feb 22 05:27:12 2017 From: kasperseidem.gamer at gmail.com (Kasper) Date: Wed, 22 Feb 2017 02:27:12 -0800 (PST) Subject: I need help with a game in (turtle graphics - python) Message-ID: <7f438d21-c371-430a-9c0a-67259de7c988@googlegroups.com> Hi! How can i make the score stop blinking and how can i make a high score table, in this game made with python? (this game is made on a Macbook) (there are some files in the game that i don't haven't copied into this file! like pyth.GIF) ________________________________________________________________________________________________________________ #turtle game (take down the yellow astroides 1v1) import turtle import math import random import os #asks for players name print("player 1 uses W,A,S,D and player 2 are using arrow Up, Down, Left, Right.") print("write player 1's name") name1 = input() print("write player 2's name") name2 = input() #set up screen wn = turtle.Screen() wn.bgcolor("black") wn.bgpic("pyth.GIF") #borders mypen = turtle.Turtle() mypen.penup() mypen.setposition(-300,-300) mypen.pendown() mypen.pensize(3) mypen.color("white") mypen.left(90) mypen.forward(600) mypen.right(90) mypen.forward(600) mypen.right(90) mypen.forward(600) mypen.right(90) mypen.forward(600) mypen.hideturtle() #creates the scores score1 = 0 score2 = 0 #create player 1 player = turtle.Turtle() player.color("blue") player.shape("triangle") player.penup() player.speed(0) player.setposition(285, 285) #create player 2 player2 = turtle.Turtle() player2.color("white") player2.shape("triangle") player2.penup() player2.speed(0) player2.setposition(-285, -285) #creats goals maxGoals = 1 goals = [] for count in range(maxGoals): goals.append(turtle.Turtle()) goals[count].color("yellow") goals[count].shape("circle") goals[count].penup() goals[count].speed(0) goals[count].setposition(random.randint(-290, 290), random.randint(-290, 290)) #set speed variable speed = 3 #define functions def turnleft(): player.left(30) def turnright(): player.right(30) def turnright2(): player2.right(30) def turnleft2(): player2.left(30) def up(): player.forward(50) def up2(): player2.forward(50) def down(): player.right(180) def down2(): player2.right(180) def speedup(): global speed speed += 1 def stop(): global speed speed = 0 def reset(): goals[count].setposition(random.randint(-290, 290), random.randint(-290, 290)) player.setposition(random.randint(-290, 290), random.randint(-290, 290)) player2.setposition(random.randint(-290, 290), random.randint(-290, 290)) def isCollision(t1, t2): d = math.sqrt(math.pow(t1.xcor()-t2.xcor(),2) + math.pow(t1.ycor()-t2.ycor(),2)) if d < 20: return True else: return False #set keyboard bindings turtle.listen() turtle.onkey(turnleft, "a") turtle.onkey(turnleft2, "Left") turtle.onkey(turnright, "d") turtle.onkey(turnright2, "Right") turtle.onkey(speedup, "o") turtle.onkey(stop, "b") turtle.onkey(down, "s") turtle.onkey(down2, "Down") turtle.onkey(up, "w") turtle.onkey(up2, "Up") turtle.onkey(reset, "r") while True: player.forward(speed) player2.forward(speed) #boundary checking player if player.xcor() > 290 or player.xcor() < -290: player.right(180) os.system("afplay bing.mp3&") #boundary Checking player if player.ycor() > 290 or player.ycor() < -290: player.right(180) os.system("afplay bing.mp3&") #boundary checking player2 if player2.xcor() > 290 or player2.xcor() < -290: player2.right(180) os.system("afplay bing.mp3&") #boundary Checking player2 if player2.ycor() > 290 or player2.ycor() < -290: player2.right(180) os.system("afplay bing.mp3&") #boundary checking if goals[count].xcor() > 290 or goals[count].xcor() < -290: goals[count].right(180) os.system("afplay bing.mp3&") #boundary Checking if goals[count].ycor() > 290 or goals[count].ycor() < -290: goals[count].right(180) os.system("afplay bing.mp3&") #move ball for count in range(maxGoals): goals[count].forward(1) #collision checking with goals if isCollision(player, goals[count]): goals[count].setposition(random.randint(-300, 300), random.randint(-300, 300)) goals[count].right(random.randint(0,360)) os.system("afplay yes.mp3&") score1 += 2 speed += 0.5 #collision checking with goals if isCollision(player2, goals[count]): goals[count].setposition(random.randint(-300, 300), random.randint(-300, 300)) goals[count].right(random.randint(0,360)) os.system("afplay yes.mp3&") score2 += 2 speed += 0.5 #collision checking with player 2 if isCollision(player, player2): player.setposition(random.randint(-300, 300), random.randint(-290, 290)) player.right(random.randint(0,360)) os.system("afplay yes.mp3&") score1 -= 1 speed += 0.5 if isCollision(player2, player): player2.setposition(random.randint(-290, 290), random.randint(-290, 290)) player2.right(random.randint(0,360)) os.system("afplay yes.mp3&") score2 -= 1 speed += 0.5 #draws the score on the screen mypen.undo() mypen.penup() mypen.hideturtle() mypen.setposition(-290, 310) scorestring1 = (name1 + ": %s" + " points ") %score1 scorestring2 = (name2 + ": %s" + " points") %score2 mypen.write(scorestring1 + scorestring2, False, align="left", font=("Arial",14, "normal")) From steve+python at pearwood.info Wed Feb 22 05:35:48 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 22 Feb 2017 21:35:48 +1100 Subject: python decorator References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <3a9a86b7-386f-4fe6-9a19-1a20812041fd@googlegroups.com> <874lzmwo8y.fsf@Equus.decebal.nl> Message-ID: <58ad6985$0$1597$c3e8da3$5496439d@news.astraweb.com> On Wed, 22 Feb 2017 08:47 pm, Cecil Westerhof wrote: > On Wednesday 22 Feb 2017 08:49 CET, Argentinian Black ops lll wrote: > >> *** SOLVED *** > > It would be nice if you shared the solution. I believe Cameron's post contains the bones of a solution. Here's my untested solution. def func_cache(cache): # Create a decorator that uses cache. def decorate(function): @functools.wraps(function) def wrapper(*args): try: result = cache[args] except KeyError: result = function(*args) cache[args] = result except TypeError: result = function(*args) return result return wrapper return decorate @func_cache({}): def something(x): ... -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From __peter__ at web.de Wed Feb 22 09:27:21 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 22 Feb 2017 15:27:21 +0100 Subject: I need help with a game in (turtle graphics - python) References: <7f438d21-c371-430a-9c0a-67259de7c988@googlegroups.com> Message-ID: Kasper wrote: > How can i make the score stop blinking The following part > #draws the score on the screen > mypen.undo() > mypen.penup() > mypen.hideturtle() > mypen.setposition(-290, 310) > scorestring1 = (name1 + ": %s" + " points ") %score1 > scorestring2 = (name2 + ": %s" + " points") %score2 > mypen.write(scorestring1 + scorestring2, False, align="left", > font=("Arial",14, "normal")) of your code removes what was written before and then draws something new, hence the blinking. One easy improvement is to call the above only when the score has changed: # untested ... # outside the loop: old_score1 = None old_score2 = None ... # in the loop: if old_score1 != score1 or old_score2 != score2: mypen.undo() mypen.penup() mypen.hideturtle() mypen.setposition(-290, 310) scorestring1 = (name1 + ": %s" + " points ") %score1 scorestring2 = (name2 + ": %s" + " points") %score2 mypen.write(scorestring1 + scorestring2, False, align="left", font=("Arial",14, "normal")) # make sure the if-suite is not executed again # unless there was a change old_score1 = score1 old_score2 = score2 > and how can i make a high score table, in this game made with python? You can store the (score, name) pairs in a file that you update when the program terminates. Do you know how to read/write a file? Given a file containing 10 Jim 13 Sue 5 Dave you can use the split() method break a line into the score and the name, and int() to convert the score from a string to an integer. When you put the entries into a list it is easy to sort them in reverse order: >>> highscores = [(10, "Jim"), (13, "Sue"), (5, "Dave")] >>> for score, name in sorted(highscores, reverse=True): ... print("%3d %s" % (score, name)) ... 13 Sue 10 Jim 5 Dave From kasperseidem.gamer at gmail.com Wed Feb 22 09:54:28 2017 From: kasperseidem.gamer at gmail.com (Kasper) Date: Wed, 22 Feb 2017 06:54:28 -0800 (PST) Subject: I need help with a game in (turtle graphics - python) In-Reply-To: References: <7f438d21-c371-430a-9c0a-67259de7c988@googlegroups.com> Message-ID: Thanks! This helped me! From pavol.lisy at gmail.com Wed Feb 22 10:16:06 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Wed, 22 Feb 2017 16:16:06 +0100 Subject: python decorator In-Reply-To: <58ad6985$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <3a9a86b7-386f-4fe6-9a19-1a20812041fd@googlegroups.com> <874lzmwo8y.fsf@Equus.decebal.nl> <58ad6985$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2/22/17, Steve D'Aprano wrote: > On Wed, 22 Feb 2017 08:47 pm, Cecil Westerhof wrote: > >> On Wednesday 22 Feb 2017 08:49 CET, Argentinian Black ops lll wrote: >> >>> *** SOLVED *** >> >> It would be nice if you shared the solution. > > I believe Cameron's post contains the bones of a solution. > > Here's my untested solution. > > > def func_cache(cache): > # Create a decorator that uses cache. > def decorate(function): > @functools.wraps(function) > def wrapper(*args): > try: > result = cache[args] > except KeyError: > result = function(*args) > cache[args] = result > except TypeError: > result = function(*args) > return result > return wrapper > return decorate > > > @func_cache({}): > def something(x): > ... Maybe this technique could be reusable (and maybe part of functools?) With this decorator: def wrap_args(decorator): def decor_out(*args, **kwargs): def decor_in(func): return decorator(func, *args, **kwargs) return decor_in return decor_out Alfredo needs to change only (*) next 2 lines: def fun_cache(function): memo = {} to: @wrap_args def fun_cache(function, cache): memo = cache (*) - Steve's improvements (for example using functools.wraps) are good to consider as well! :) (but maybe catching TypeError could more problems hide than solve) From grant.b.edwards at gmail.com Wed Feb 22 10:51:34 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 22 Feb 2017 15:51:34 +0000 (UTC) Subject: Python application launcher (for Python code) References: <58ab8f89$0$1589$c3e8da3$5496439d@news.astraweb.com> <008401d28be8$7a5ceeb0$27b23dae@sambora> Message-ID: On 2017-02-21, Chris Warrick wrote: > Git Bash, or basically msys, is pretty reasonable. But if you are on > Windows 10, you might like the built-in Windows Subsystem for Linux > (aka Bash on Ubuntu on Windows) more ? it?s real Linux that runs > alongside Windows, but less crazy than Cygwin. Cygwin is indeed crazy. I've been in awe of it for 15 years. Even sshd works (if you're carefuly and a little lucky). It's a medium-sized miracle that Cygwin works as well as it does. For a few years, I distributed and supported a custom snapshot of base Cygwin plus a software development kit for a specific hardware product. It was not fun. Cygwin is a bit brittle and presents many mysterious failure modes. But, had I not seen it myself, I never would have believed Cygwin would work in a useful way. In then end, we gave up on supporting Cygwin for our Windows customers. It's actually easier to install Ubuntu in a VM and have them use that. The last customer I had who was trying to install Cyginw and use it for development spent weeks and never got everything to work with Cygwin. About two hours after I finally convinced him to try Linux on a VM, I got an e-mail saying his Linux VM as installed, the devleopment tools were installed on that, and he was happily building his applications. -- Grant Edwards grant.b.edwards Yow! I know how to do at SPECIAL EFFECTS!! gmail.com From ganesh1pal at gmail.com Wed Feb 22 12:03:31 2017 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Wed, 22 Feb 2017 22:33:31 +0530 Subject: If statement with or operator Message-ID: Hello Friends, I need suggestion on the if statement in the below code , all that I was trying to do was to add a check i.e if any one of the functions return True then break the loop. end_time = time.time() + 300 umount_completed = False while time.time() < end_time: if attempt_umount() is True or df_output_lines() is True: logging.info("umount completed sucessfully") umount_completed = True break time.sleep(15) assert umount_completed, "ERROR: The umount failed Exiting" Can this be optimized better , please suggest. I am on Linux and Python 2.7 Regards. Ganesh From alister.ware at ntlworld.com Wed Feb 22 12:25:20 2017 From: alister.ware at ntlworld.com (alister) Date: Wed, 22 Feb 2017 17:25:20 GMT Subject: If statement with or operator Message-ID: <4QjrA.292380$FC4.42097@fx19.am4> On Wed, 22 Feb 2017 22:33:31 +0530, Ganesh Pal wrote: > Hello Friends, > > I need suggestion on the if statement in the below code , all that I > was trying to do was to add a check i.e if any one of the functions > return True then break the loop. > > > end_time = time.time() + 300 > umount_completed = False while time.time() < end_time: > if attempt_umount() is True or df_output_lines() is True: > logging.info("umount completed sucessfully") > umount_completed = True break > time.sleep(15) > assert umount_completed, "ERROR: The umount failed Exiting" > > Can this be optimized better , please suggest. I am on Linux and > Python 2.7 > > > Regards. > Ganesh a simple if attempt_umount() or df_output_lines(): would suffice (Q argument on whether the unnecessary is True makes the code more readable or not) there may also be better ways to perform your unmount but without knowing the details of the code it is difficult to say -- "Nuclear war would really set back cable." - Ted Turner From braxtonalfred at shaw.ca Wed Feb 22 12:26:54 2017 From: braxtonalfred at shaw.ca (Braxton Alfred) Date: Wed, 22 Feb 2017 09:26:54 -0800 Subject: CSV Message-ID: <000301d28d30$dd2a28c0$977e7a40$@ca> Why does this not run? It is right out of the CSV file in the Standard Lib. Python ver 3.4.4, 64 bit. import csv """ READ EXCEL FILE """ filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' with open (filename, newline = ' ') as bp: dialect = csv.excel reader = csv.reader(bp) for row in reader: print (row) Marc From pkpearson at nowhere.invalid Wed Feb 22 12:38:37 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 22 Feb 2017 17:38:37 GMT Subject: If statement with or operator References: Message-ID: On Wed, 22 Feb 2017 22:33:31 +0530, Ganesh Pal wrote: [snip] > I need suggestion on the if statement in the below code , all that I was > trying to do was to add a check i.e if any one of the functions return > True then break the loop. > > end_time = time.time() + 300 > umount_completed = False > while time.time() < end_time: > if attempt_umount() is True or df_output_lines() is True: > logging.info("umount completed sucessfully") > umount_completed = True > break > time.sleep(15) > assert umount_completed, "ERROR: The umount failed Exiting" The if statement is healthy and reasonable. If you have a guarantee that attempt_umount and df_output_lines return only True or False, then this would be equivalent: if attempt_umount() or df_output_lines(): However, at the bottom, checking for completion with "assert" is (as I understand it) weak because assertions may be ignored in optimized situations. I'd raise RuntimeError. Also, an consider this alternative loop structure: while not (attempt_umount() or df_output_lines()): if end_time < time.time(): raise RuntimeError("The umount failed. Exiting.") time.sleep(15) logging.info("umount completed successfully") -- To email me, substitute nowhere->runbox, invalid->com. From ian.g.kelly at gmail.com Wed Feb 22 12:45:31 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 22 Feb 2017 10:45:31 -0700 Subject: python decorator In-Reply-To: References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <3a9a86b7-386f-4fe6-9a19-1a20812041fd@googlegroups.com> <874lzmwo8y.fsf@Equus.decebal.nl> <58ad6985$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Feb 22, 2017 at 8:16 AM, Pavol Lisy wrote: > Maybe this technique could be reusable (and maybe part of functools?) > > With this decorator: > > def wrap_args(decorator): > def decor_out(*args, **kwargs): > def decor_in(func): > return decorator(func, *args, **kwargs) > return decor_in > return decor_out > > Alfredo needs to change only (*) next 2 lines: > > def fun_cache(function): > memo = {} > > to: > > @wrap_args > def fun_cache(function, cache): > memo = cache > > (*) - Steve's improvements (for example using functools.wraps) are > good to consider as well! :) > (but maybe catching TypeError could more problems hide than solve) When all the arguments are optional it's generally desirable that @foo and @foo() be equivalent. See: https://blogs.it.ox.ac.uk/inapickle/2012/01/05/python-decorators-with-optional-arguments/ I would augment that example by making the optional arguments keyword-only to prevent accidentally passing one as the function to be decorated. Hopefully this feature would also be part of any general reusable solution. From irmen.NOSPAM at xs4all.nl Wed Feb 22 13:05:31 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Wed, 22 Feb 2017 19:05:31 +0100 Subject: python decorator In-Reply-To: References: <409ff091-7590-42ca-9986-e6c5f16534b8@googlegroups.com> <20170222071936.GA2403@cskk.homeip.net> Message-ID: <58add2eb$0$663$e4fe514c@news.xs4all.nl> On 22-2-2017 8:39, Argentinian Black ops lll wrote: > Thanks for the quick response...! you saved me a lot of time, thank you! > I don't know if you want/have to use your own custom caching decorator, but are you aware of the lru_cache decorator that's part of the standard library? https://docs.python.org/3/library/functools.html#functools.lru_cache Irmen From rosuav at gmail.com Wed Feb 22 13:06:17 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 23 Feb 2017 05:06:17 +1100 Subject: CSV In-Reply-To: <000301d28d30$dd2a28c0$977e7a40$@ca> References: <000301d28d30$dd2a28c0$977e7a40$@ca> Message-ID: On Thu, Feb 23, 2017 at 4:26 AM, Braxton Alfred wrote: > filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' Use forward slashes instead. Also, if something isn't working, post the actual output - probably an exception. Don't assume that we can read your mind, even when we can. ChrisA From irmen.NOSPAM at xs4all.nl Wed Feb 22 13:09:56 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Wed, 22 Feb 2017 19:09:56 +0100 Subject: CSV In-Reply-To: References: <000301d28d30$dd2a28c0$977e7a40$@ca> Message-ID: <58add3f4$0$749$e4fe514c@news.xs4all.nl> On 22-2-2017 18:26, Braxton Alfred wrote: > Why does this not run? It is right out of the CSV file in the Standard Lib. What does "not run" mean. We can't help if you are not telling us the exact error message you're getting (if any) > filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' That being said, there's at least a problem with this line because you're not escaping the backslashes. So the resulting filename string is not what you intended it to be (try printing it to see what I mean). Several solutions possible: - switch to using forward slashes / this works on windows too. - use a raw string literal r'.....' - or escape the backslashes \\ Irmen From python at deborahswanson.net Wed Feb 22 13:50:18 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Wed, 22 Feb 2017 10:50:18 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <001801d28d3c$83743170$27b23dae@sambora> Grant Edwards wrote, on February 22, 2017 7:52 AM > > On 2017-02-21, Chris Warrick wrote: > > > Git Bash, or basically msys, is pretty reasonable. But if you are on > > Windows 10, you might like the built-in Windows Subsystem for Linux > > (aka Bash on Ubuntu on Windows) more - it's real Linux that runs > > alongside Windows, but less crazy than Cygwin. > > Cygwin is indeed crazy. > > I've been in awe of it for 15 years. > > Even sshd works (if you're carefuly and a little lucky). > > It's a medium-sized miracle that Cygwin works as well as it > does. For a few years, I distributed and supported a custom > snapshot of base Cygwin plus a software development kit for a > specific hardware product. It was not fun. Cygwin is a bit > brittle and presents many mysterious failure modes. But, had > I not seen it myself, I never would have believed Cygwin > would work in a useful way. > > In then end, we gave up on supporting Cygwin for our Windows > customers. It's actually easier to install Ubuntu in a VM > and have them use that. The last customer I had who was > trying to install Cyginw and use it for development spent > weeks and never got everything to work with Cygwin. About > two hours after I finally convinced him to try Linux on a VM, > I got an e-mail saying his Linux VM as installed, the > devleopment tools were installed on that, and he was happily > building his applications. > > -- > Grant Edwards grant.b.edwards Yow! I > know how to do > at SPECIAL EFFECTS!! > gmail.com I tried Cygwin once. I had a fairly complex library of C functions I wanted to rewrite portions of in Python, and I wanted to run the C code first, and be able to use a running C version to check my Python version against. I couldn't afford Visual Studio, especially since there was only this one project to use it for, and Cygwin seemed like a reasonable alternative. Cygwin has an avid fan club of followers for using Cygwin to write and execute C code, and as an IDE, I think. But I couldn't get it to run right, and I usually get along pretty well with software. Not an experience I'm eager to repeat. Deborah From rhodri at kynesim.co.uk Wed Feb 22 13:56:38 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 22 Feb 2017 18:56:38 +0000 Subject: If statement with or operator In-Reply-To: References: Message-ID: <5328ab07-9c16-27ed-174f-3cfb8521a0e2@kynesim.co.uk> On 22/02/17 17:38, Peter Pearson wrote: > On Wed, 22 Feb 2017 22:33:31 +0530, Ganesh Pal wrote: > [snip] >> I need suggestion on the if statement in the below code , all that I was >> trying to do was to add a check i.e if any one of the functions return >> True then break the loop. >> >> end_time = time.time() + 300 >> umount_completed = False >> while time.time() < end_time: >> if attempt_umount() is True or df_output_lines() is True: >> logging.info("umount completed sucessfully") >> umount_completed = True >> break >> time.sleep(15) >> assert umount_completed, "ERROR: The umount failed Exiting" > > The if statement is healthy and reasonable. Quite to the contrary, the if statement should be discouraged with great vigour unless the OP is certain that the functions return only the object True for success, and not just values that are true in a boolean context. if attempt_umount() or df_output_lines(): is far more likely to do what you expect. -- Rhodri James *-* Kynesim Ltd From gandalf at shopzeus.com Wed Feb 22 14:50:46 2017 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Wed, 22 Feb 2017 20:50:46 +0100 Subject: concurrent futures, async futures and await Message-ID: I'm in a situation where I would like to refactor some code to use native coroutine functions (aync def) instead of "normal" coroutines that use yield. Most of the code is asnyc, but some of the operations are performed in different threads (using concurrent.futures.Executor). We have concurrent.futures.Future and asyncio.Future. The former one an be used to execute code in a different thread, the later one can be awaited. Since concurrent.futures.Future does not implement the __await__ method, it cannot be awaited in an ioloop. For example, if I want to read from a file in a different thread, then I can submit that as a task to a ThreadPoolExecutor, but it is not possible to await for it. It is still possible to read from the file in the same thread, but obviously I do not want to do that (because file read can block the loop, and I do not want that). In other words: this makes it impossible to refactor coroutine functions to native coroutine functions. In my concrete use case, coroutines are used in tornado's ioloop, and tornado handles both types of futures *if they are yielded*. But once I switch to async def, yield becomes await and the native await is incompatible with concurrent.futures.Future. Both the await keyword and concurrent.futures.Future are in the standard library. They should be compatible. Is there a reason why concurrent.futures does not implement __await__ ? Thanks, Laszlo From jussi.piitulainen at helsinki.fi Wed Feb 22 15:15:58 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Wed, 22 Feb 2017 22:15:58 +0200 Subject: CSV References: <000301d28d30$dd2a28c0$977e7a40$@ca> Message-ID: Braxton Alfred writes: > Why does this not run? It is right out of the CSV file in the Standard Lib. > > > > > Python ver 3.4.4, 64 bit. > > > > > > > > import csv > """ READ EXCEL FILE """ > filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' '\b' is backspace. A couple of months ago I actually met a filename with a backspace in it. I renamed the file. Or maybe I removed it, I forget. But don't you get a SyntaxError from '\user'? [snip] From python at mrabarnett.plus.com Wed Feb 22 16:37:24 2017 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 22 Feb 2017 21:37:24 +0000 Subject: How to store properties In-Reply-To: References: <87efz83ou6.fsf@Equus.decebal.nl> Message-ID: <6cafc7b6-ce6d-c43a-adcc-bd6fff228817@mrabarnett.plus.com> On 2017-02-22 21:08, Gilmeh Serda wrote: > On Wed, 08 Feb 2017 09:31:09 -0800, Rob Gaddi wrote: > >> JSON's cute, but the format doesn't support inline comments. > > { > "your_key": "whatever", > "COMMENT": "blah", > "meh": [ > "yup", > "yep", > "yip" > ], > "COMMENT": "mooh" > } > > Works fine for me. As long as you don't have any use for your "COMMENT" > key elsewhere, it will continue to work. > That's a dictionary, so if you read it in and then write it out again, the order might change, and one of the comments will be lost. From breamoreboy at gmail.com Wed Feb 22 16:52:29 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 22 Feb 2017 13:52:29 -0800 (PST) Subject: CSV In-Reply-To: References: <000301d28d30$dd2a28c0$977e7a40$@ca> Message-ID: <2d722532-b97c-42f8-8ba6-abae24d08cbb@googlegroups.com> On Wednesday, February 22, 2017 at 5:55:47 PM UTC, Braxton Alfred wrote: > Why does this not run? It is right out of the CSV file in the Standard Lib. > > Python ver 3.4.4, 64 bit. > > import csv > """ READ EXCEL FILE """ > filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' > with open (filename, newline = ' ') as bp: > dialect = csv.excel > reader = csv.reader(bp) > for row in reader: > print (row) > > Marc Of course it will run, assuming that there are no syntax errors. Besides that using raw strings or forward slashes will probably help. Of course we don't actually know as you haven't shown us the actual error that you get, and our crystal balls broke down long ago owing to over use. You might like to read http://www.catb.org/~esr/faqs/smart-questions.html and then http://sscce.org/ or even http://stackoverflow.com/help/mcve and then ask again. Kindest regards. Mark Lawrence. From nathan.ernst at gmail.com Wed Feb 22 19:39:11 2017 From: nathan.ernst at gmail.com (Nathan Ernst) Date: Wed, 22 Feb 2017 18:39:11 -0600 Subject: CSV In-Reply-To: <2d722532-b97c-42f8-8ba6-abae24d08cbb@googlegroups.com> References: <000301d28d30$dd2a28c0$977e7a40$@ca> <2d722532-b97c-42f8-8ba6-abae24d08cbb@googlegroups.com> Message-ID: One other thing besides the issues noted with filename - newline is set to a space. It should be set to an empty string. See: https://docs.python.org/3/library/csv.html#id3 Regards, Nate On Wed, Feb 22, 2017 at 3:52 PM, wrote: > On Wednesday, February 22, 2017 at 5:55:47 PM UTC, Braxton Alfred wrote: > > Why does this not run? It is right out of the CSV file in the Standard > Lib. > > > > Python ver 3.4.4, 64 bit. > > > > import csv > > """ READ EXCEL FILE """ > > filename = 'c:\users\user\my documents\Braxton\Excel\personal\bp.csv' > > with open (filename, newline = ' ') as bp: > > dialect = csv.excel > > reader = csv.reader(bp) > > for row in reader: > > print (row) > > > > Marc > > Of course it will run, assuming that there are no syntax errors. > > Besides that using raw strings or forward slashes will probably help. > > Of course we don't actually know as you haven't shown us the actual error > that you get, and our crystal balls broke down long ago owing to over use. > > You might like to read http://www.catb.org/~esr/faqs/smart-questions.html > and then http://sscce.org/ or even http://stackoverflow.com/help/mcve and > then ask again. > > Kindest regards. > > Mark Lawrence. > -- > https://mail.python.org/mailman/listinfo/python-list > From python at deborahswanson.net Wed Feb 22 23:49:24 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Wed, 22 Feb 2017 20:49:24 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <00cc01d28d90$350fa330$27b23dae@sambora> Dennis Lee Bieber wrote, on February 22, 2017 6:48 PM > > On Wed, 22 Feb 2017 10:50:18 -0800, "Deborah Swanson" > declaimed the following: > > > >first, and be able to use a running C version to check my Python > >version against. I couldn't afford Visual Studio, especially since > >there was > > Must have been quite some time ago, since VS Express > versions have been available since 2005. > > Granted, Express has limitations... But if porting C > code, you would likely not have been affected -- it's > unlikely your code used MFC... > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ Didn't even look. Visual Studio has always been pricey, and it never occurred to me that they might have a free or cheap version now. This was last summer, and I ended up just writing the Python by looking at the C, and attempting to verify against a data set I knew the correct answers for. But it wasn't looking exactly right and I dropped the project. I'll remember VS Express if I revisit this, so thanks! Deborah From 17chiue at gmail.com Thu Feb 23 00:08:35 2017 From: 17chiue at gmail.com (17chiue at gmail.com) Date: Wed, 22 Feb 2017 21:08:35 -0800 (PST) Subject: First Live Online Python Syntax Checker Message-ID: Hi everyone! :) I just wanted to share with everyone a lightweight online tool I created called PythonBuddy (https://github.com/ethanchewy/PythonBuddy) which actively lints Python online. I made this so that MOOCs like edX or codecademy could easily embed and use this on their courses. Also, PythonBuddy could help alleviate the frustrations with setting up a programming environment. Just thought it might be helpful for those who want to program in Python online with a helpful live syntax checker. Please leave a ? if you found this project to be helpful! I'm currently trying to convert this project to an XBlock so that online courses (like 6.00.1x) can embed a live python syntax checker for quizzes and assignments online. From sntc99 at gmail.com Thu Feb 23 02:05:01 2017 From: sntc99 at gmail.com (sntc99 at gmail.com) Date: Wed, 22 Feb 2017 23:05:01 -0800 (PST) Subject: any one used moviepy please come in!!! I need help, thanks! In-Reply-To: <12117181-ec3b-4dd6-9f58-7b0c81f0ae20@googlegroups.com> References: <5ec6d36f-5200-4882-98ce-c317deab35b3@googlegroups.com> <12117181-ec3b-4dd6-9f58-7b0c81f0ae20@googlegroups.com> Message-ID: On Thursday, January 26, 2017 at 8:12:24 AM UTC+5:30, Tony Chen wrote: > On Wednesday, January 25, 2017 at 8:34:01 PM UTC+13, Chris Angelico wrote: > > On Wed, Jan 25, 2017 at 6:22 PM, Tony Chen wrote: > > > This error can be due to the fact that ImageMagick is not installed on your computer, or (for Windows users) that you didn't specify the path to the ImageMagick binary in file conf.py, or.that the path you specified is incorrect > > > > So... is ImageMagick installed? > > > > ChrisA > > Okay... Yes it's installed. This problem has been solved haha. Thank you for replying. Hi Tony, same issue facing by me. kindly help me please list the solution From ian.g.kelly at gmail.com Thu Feb 23 02:38:30 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 23 Feb 2017 00:38:30 -0700 Subject: concurrent futures, async futures and await In-Reply-To: References: Message-ID: On Wed, Feb 22, 2017 at 12:50 PM, Nagy L?szl? Zsolt wrote: > > I'm in a situation where I would like to refactor some code to use > native coroutine functions (aync def) instead of "normal" coroutines > that use yield. Most of the code is asnyc, but some of the operations > are performed in different threads (using concurrent.futures.Executor). > > We have concurrent.futures.Future and asyncio.Future. The former one an > be used to execute code in a different thread, the later one can be > awaited. Since concurrent.futures.Future does not implement the > __await__ method, it cannot be awaited in an ioloop. For example, if I > want to read from a file in a different thread, then I can submit that > as a task to a ThreadPoolExecutor, but it is not possible to await for > it. It is still possible to read from the file in the same thread, but > obviously I do not want to do that (because file read can block the > loop, and I do not want that). > > In other words: this makes it impossible to refactor coroutine functions > to native coroutine functions. > > In my concrete use case, coroutines are used in tornado's ioloop, and > tornado handles both types of futures *if they are yielded*. But once I > switch to async def, yield becomes await and the native await is > incompatible with concurrent.futures.Future. > > Both the await keyword and concurrent.futures.Future are in the standard > library. They should be compatible. Is there a reason why > concurrent.futures does not implement __await__ ? My guess: because asyncio wouldn't know what to do with a concurrent.futures.Future. The tornado docs say that "You can also use tornado.gen.convert_yielded to convert anything that would work with yield into a form that will work with await": http://www.tornadoweb.org/en/stable/guide/coroutines.html#python-3-5-async-and-await From tomasz.wisniewski.gm at gmail.com Thu Feb 23 03:14:23 2017 From: tomasz.wisniewski.gm at gmail.com (tomasz.wisniewski.gm at gmail.com) Date: Thu, 23 Feb 2017 00:14:23 -0800 (PST) Subject: [Glitch?] Python has just stopped working In-Reply-To: References: Message-ID: <66175bbb-9ecb-4fcc-82ee-8e32bca9c643@googlegroups.com> W dniu wtorek, 16 lutego 2016 21:09:50 UTC+1 u?ytkownik Theo Hamilton napisa?: > I woke up two days ago to find out that python literally won't work any > more. I have looked everywhere, asked multiple Stack Overflow questions, > and am ready to give up. Whenever I run python (3.5), I get the following > message: > > Fatal Python error: Py_initialize: unable to load the file system codec > ImportError: No module named 'encodings' > > Current thread 0x00002168 (most recent call first): > > If there's anything you know that I could do to fix this, then please tell > me. I've tried uninstalling and reparing, so it's not those. Thanks! You have to set your PYTHONHOME variable in Windows. for example PYTHONHOME='E:\Python36' I had the same problem and this worked. From python at deborahswanson.net Thu Feb 23 04:38:48 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 01:38:48 -0800 Subject: Namedtuples problem Message-ID: <001701d28db8$a2ea03f0$27b23dae@sambora> This is how the list of namedtuples is originally created from a csv: infile = open("E:\\Coding projects\\Pycharm\\Moving\\Moving 2017 in - test.csv") rows = csv.reader(infile)fieldnames = next(rows) Record = namedtuple("Record", fieldnames) records = [Record._make(fieldnames)] records.extend(Record._make(row) for row in rows) Thanks to Peter Otten for this succinct code, and to Greg Ewing for suggesting namedtuples for this type of problem to begin with. Namedtuples worked beautifully for the first two thirds of this code, but I've run into a snag attempting to proceed. Here's my code up to the snag, and I'll explain afterwards what I'm trying to do: import operator records[1:] = sorted(records[1:], key=operator.attrgetter("title", "Date")) groups = defaultdict() for r in records[1:]: # if the key doesn't exist, make a new group if r.title not in groups.keys(): groups[r.title] = [r] # if key (group) exists, append this record else: groups[r.title].append(r) # make lookup table: indices for field names records_idx = {} for idx, label in enumerate(records[0]): records_idx[label] = idx LABELS = ['Location', 'ST', 'co', 'miles', 'first', 'Kind', 'Notes'] # look at field values for each label on group for group in groups.values(): values = [] for idx, row in enumerate(group): for label in LABELS: values.append(group[[idx][records_idx[label]]]) <-snag I want to get lists of field values from the list of namedtuples, one list of field values for each row in each group (groups are defined in the section beginning with "groups = defaultdict()". LABELS defines the field names for the columns of field values of interest. So all the locations in this group would be in one list, all the states in another list, etc. (Jussi, I'm looking at your suggestion for the next part.) (I'm quite sure this bit of code could be written with list and dict comprehensions, but here I was just trying to get it to work, and comprehensions still confuse me a little.) Using the debugger's watch window, from group[[idx][records_idx[label]]], I get: idx = {int}: 0 records_idx[label] = {int}: 4 which is the correct indices for the first row of the current group (idx = 0) and the first field label in LABELS, 'Location' (records_idx[label] = 4). And if I look at group[0][4] = 'Longview' this is also correct. Longview is the Location field value for the first row of this group. However, group[[idx][records_idx[label]]] gets an Index Error: list index out of range I've run into this kind of problem with namedtuples before, trying to access field values with variable names, like: label = 'Location' records.label and I get something like "'records' has no attribute 'label'. This can be fixed by using the subscript form and an index, like: for idx, r in enumerate(records): ... records[idx] = r But here, I get the Index Error and I'm a bit baffled why. Both subscripts evaluate to valid indices and give the correct value when explicitly used. Can anyone see why I'm getting this Index error? and how to fix it? From python at lucidity.plus.com Thu Feb 23 05:08:38 2017 From: python at lucidity.plus.com (Erik) Date: Thu, 23 Feb 2017 10:08:38 +0000 Subject: Namedtuples problem In-Reply-To: <001701d28db8$a2ea03f0$27b23dae@sambora> References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: Hi, On 23/02/17 09:38, Deborah Swanson wrote: > group[[idx][records_idx[label]]] > gets an Index Error: list index out of range [snip] > Can anyone see why I'm getting this Index error? and how to fix it? It looks to me like you are indexing into a single-element list that you are creating using the literal list syntax in the middle of the expression. If we were to break the expression into parts to make it a bit simpler to refer to discuss: ridx = records_idx[label] group[[idx][ridx]] You can now more easily see that 'group' is being indexed by the expression "[idx][ridx]". What does that mean? [idx] is creating a single-element list using literal list syntax. This is then indexed using 'ridx' (using, perhaps confusingly, the exact same syntax to do a different thing). The result of *that* expression is then being used to index 'group', but it won't get that far because you'll get the exception if 'ridx' is anything but zero. So the initial problem at least is the extra [] around 'idx' which is creating a list on the fly for you. E. From __peter__ at web.de Thu Feb 23 05:34:11 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 23 Feb 2017 11:34:11 +0100 Subject: Namedtuples problem References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: Deborah Swanson wrote: > This is how the list of namedtuples is originally created from a csv: > > infile = open("E:\\Coding projects\\Pycharm\\Moving\\Moving 2017 in - > test.csv") > rows = csv.reader(infile)fieldnames = next(rows) > Record = namedtuple("Record", fieldnames) > records = [Record._make(fieldnames)] > records.extend(Record._make(row) for row in rows) > > Thanks to Peter Otten for this succinct code, and to Greg Ewing for > suggesting namedtuples for this type of problem to begin with. > > Namedtuples worked beautifully for the first two thirds of this code, > but I've run into a snag attempting to proceed. > > Here's my code up to the snag, and I'll explain afterwards what I'm > trying to do: > > import operator > records[1:] = sorted(records[1:], key=operator.attrgetter("title", > "Date")) > > groups = defaultdict() > for r in records[1:]: > # if the key doesn't exist, make a new group > if r.title not in groups.keys(): > groups[r.title] = [r] > # if key (group) exists, append this record > else: > groups[r.title].append(r) > > # make lookup table: indices for field names > records_idx = {} > for idx, label in enumerate(records[0]): > records_idx[label] = idx > > LABELS = ['Location', 'ST', 'co', 'miles', 'first', 'Kind', 'Notes'] # > look at field values for each label on group for group in > groups.values(): > values = [] > for idx, row in enumerate(group): > for label in LABELS: > values.append(group[[idx][records_idx[label]]]) > <-snag > > I want to get lists of field values from the list of namedtuples, one > list of field values for each row in each group (groups are defined in > the section beginning with "groups = defaultdict()". > > LABELS defines the field names for the columns of field values of > interest. So all the locations in this group would be in one list, all > the states in another list, etc. (Jussi, I'm looking at your suggestion > for the next part.) > > (I'm quite sure this bit of code could be written with list and dict > comprehensions, but here I was just trying to get it to work, and > comprehensions still confuse me a little.) > > Using the debugger's watch window, from > group[[idx][records_idx[label]]], I get: > > idx = {int}: 0 > records_idx[label] = {int}: 4 > > which is the correct indices for the first row of the current group (idx > = 0) and the first field label in LABELS, 'Location' (records_idx[label] > = 4). > > And if I look at > > group[0][4] = 'Longview' > > this is also correct. Longview is the Location field value for the first > row of this group. > > However, > > group[[idx][records_idx[label]]] > gets an Index Error: list index out of range > > I've run into this kind of problem with namedtuples before, trying to > access field values with variable names, like: > > label = 'Location' > records.label > > and I get something like "'records' has no attribute 'label'. This can > be fixed by using the subscript form and an index, like: > > for idx, r in enumerate(records): > ... > records[idx] = r > > But here, I get the Index Error and I'm a bit baffled why. Both > subscripts evaluate to valid indices and give the correct value when > explicitly used. > > Can anyone see why I'm getting this Index error? and how to fix it? I'm not completely sure I can follow you, but you seem to be mixing two problems (1) split a list into groups (2) convert a list of rows into a list of columns and making a kind of mess in the process. Functions to the rescue: #untested def split_into_groups(records, key): groups = defaultdict(list) for record in records: # no need to check if a group already exists # an empty list will automatically added for every # missing key groups[key(record)].append(record) return groups def extract_column(records, name): # you will agree that extracting one column is easy :) return [getattr(record, name) for record in records] def extract_columns(records, names): # we can build on that to make a list of columns return [extract_column(records, name) for name in names] wanted_columns = ['Location', ...] records = ... groups = split_into_groups(records, operator.attrgetter("title")) Columns = namedtuple("Columns", wanted_columns) for title, group in groups.items(): # for easier access we turn the list of columns # into a namedtuple of columns groups[title] = Columns._make(extract_columns(wanted_columns)) If all worked well you should now be able to get a group with group["whatever"] and all locations for that group with group["whatever"].Locations If there is a bug you can pinpoint the function that doesn't work and ask for specific help on that one. From __peter__ at web.de Thu Feb 23 05:57:05 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 23 Feb 2017 11:57:05 +0100 Subject: Namedtuples problem References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: Peter Otten wrote: > Functions to the rescue: On second thought this was still more code than necessary. Why not calculate a column on demand? Here's a self-containe example: $ cat swanson_grouped_columns_virtual.py import operator from collections import defaultdict, namedtuple def split_into_groups(records, key, GroupClass=list): groups = defaultdict(GroupClass) for record in records: groups[key(record)].append(record) return groups class Group(list): def __getattr__(self, name): return [getattr(item, name) for item in self] Record = namedtuple("Record", "title location") records = [ Record("foo", "here"), Record("foo", "there"), Record("bar", "everywhere"), ] groups = split_into_groups( records, operator.attrgetter("title"), Group ) print(groups["foo"].location) $ python3 swanson_grouped_columns_virtual.py ['here', 'there'] From eryksun at gmail.com Thu Feb 23 06:17:02 2017 From: eryksun at gmail.com (eryk sun) Date: Thu, 23 Feb 2017 11:17:02 +0000 Subject: [Glitch?] Python has just stopped working In-Reply-To: <66175bbb-9ecb-4fcc-82ee-8e32bca9c643@googlegroups.com> References: <66175bbb-9ecb-4fcc-82ee-8e32bca9c643@googlegroups.com> Message-ID: On Thu, Feb 23, 2017 at 8:14 AM, wrote: > W dniu wtorek, 16 lutego 2016 21:09:50 UTC+1 u?ytkownik Theo Hamilton napisa?: >> I woke up two days ago to find out that python literally won't work any >> more. I have looked everywhere, asked multiple Stack Overflow questions, >> and am ready to give up. Whenever I run python (3.5), I get the following >> message: >> >> Fatal Python error: Py_initialize: unable to load the file system codec >> ImportError: No module named 'encodings' >> >> Current thread 0x00002168 (most recent call first): >> >> If there's anything you know that I could do to fix this, then please tell >> me. I've tried uninstalling and reparing, so it's not those. Thanks! > > You have to set your PYTHONHOME variable in Windows. for example > > PYTHONHOME='E:\Python36' > > I had the same problem and this worked. This variable should not be needed and never set permanently. If standard CPython can't start without PYTHONHOME, then there's something wrong with your installation. Run set PYTHONHOME= to clear it in the current command prompt, and then try to run Python. From billy.earney at gmail.com Thu Feb 23 07:45:27 2017 From: billy.earney at gmail.com (Billy Earney) Date: Thu, 23 Feb 2017 06:45:27 -0600 Subject: any one used moviepy please come in!!! I need help, thanks! In-Reply-To: References: <5ec6d36f-5200-4882-98ce-c317deab35b3@googlegroups.com> <12117181-ec3b-4dd6-9f58-7b0c81f0ae20@googlegroups.com> Message-ID: imagemagick (ie, convert) has a policy.xml file that stops you from accessing any resource that starts with '@'. type 'identify -list policy' the first line should tell you where your policy.xml file is located. My policy.xml file is lcoated at /etc/ImageMagick-6/policy.xml open that file, and goto to the end and comment out (or remove the line that reads) since this is xml, you can comment out this line by appending the line with Hope this helps! On Thu, Feb 23, 2017 at 1:05 AM, wrote: > On Thursday, January 26, 2017 at 8:12:24 AM UTC+5:30, Tony Chen wrote: > > On Wednesday, January 25, 2017 at 8:34:01 PM UTC+13, Chris Angelico > wrote: > > > On Wed, Jan 25, 2017 at 6:22 PM, Tony Chen > wrote: > > > > This error can be due to the fact that ImageMagick is not installed > on your computer, or (for Windows users) that you didn't specify the path > to the ImageMagick binary in file conf.py, or.that the path you specified > is incorrect > > > > > > So... is ImageMagick installed? > > > > > > ChrisA > > > > Okay... Yes it's installed. This problem has been solved haha. Thank you > for replying. > > Hi Tony, > same issue facing by me. kindly help me > please list the solution > -- > https://mail.python.org/mailman/listinfo/python-list > From peterreavey32 at gmail.com Thu Feb 23 08:23:27 2017 From: peterreavey32 at gmail.com (peterreavey32 at gmail.com) Date: Thu, 23 Feb 2017 05:23:27 -0800 (PST) Subject: end= Message-ID: Using python at home as oppose to using it at school and getting a syntax error for the line end= but this has worked previously at school. I think the version I've got at home is newer than the one at school - should this affect it? From rosuav at gmail.com Thu Feb 23 08:28:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 24 Feb 2017 00:28:16 +1100 Subject: end= In-Reply-To: References: Message-ID: On Fri, Feb 24, 2017 at 12:23 AM, wrote: > Using python at home as oppose to using it at school and getting a syntax error for the line end= but this has worked previously at school. I think the version I've got at home is newer than the one at school - should this affect it? > Yes, it definitely can make a difference, but more likely, you're running an _older_ version at home. The "end=" syntax on the print function is available in Python 3.x and not normally in Python 2.x. Upgrade your home system to the latest (Python 3.6 currently), and the syntax error should disappear. ChrisA From steve+python at pearwood.info Thu Feb 23 09:11:07 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 24 Feb 2017 01:11:07 +1100 Subject: Namedtuples problem References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: <58aeed7c$0$1610$c3e8da3$5496439d@news.astraweb.com> On Thu, 23 Feb 2017 08:38 pm, Deborah Swanson wrote: > However, > > group[[idx][records_idx[label]]] > gets an Index Error: list index out of range That's not very helpful: judging from that line alone, there could be as many as THREE places in that line of code that might generate IndexError. When you have code complex enough that you can't understand the error it gives, break it up into separate steps: # group[[idx][records_idx[label]]] i = records_idx[label] j = [idx][i] group[j] Now at least you will find out which step is failing! Since records_idx is a dict, if that fails it should give KeyError, not IndexError. So it won't be that line, at least not if my reading of your code is correct. The most likely suspect is the second indexing operation, which looks mighty suspicious: [idx][i] What a strange thing to do: you create a list with a single item, idx, and then index into that list with some value i. Either i is 0, in which case you get idx, or else you get an error. So effectively, you are looking up: group[idx] or failing. > I've run into this kind of problem with namedtuples before, This is nothing to do with namedtuples. You happen to be using namedtuples elsewhere in the code, but that's not what is failing. namedtuple is just an innocent bystander. > trying to > access field values with variable names, like: > > label = 'Location' > records.label > > and I get something like "'records' has no attribute 'label'. Which has nothing to do with this error. When the attribute name is known only in a variable, the simplest way to look it up is: getattr(records, label) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From pavol.lisy at gmail.com Thu Feb 23 09:19:03 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Thu, 23 Feb 2017 15:19:03 +0100 Subject: Namedtuples problem In-Reply-To: References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: On 2/23/17, Peter Otten <__peter__ at web.de> wrote: > Peter Otten wrote: > >> Functions to the rescue: > > On second thought this was still more code than necessary. If we are talking about less code than necessary then probably we could use something from python's ecosystem... >>> import pandas as pd # really useful package >>> import io # I simlulate file in csv with header (like Deborah has) >>> data = """\ what,location foo,here foo,there bar,eberywhere""" >>> df = pd.read_csv(io.StringIO(data)) # reading csv into DataFrame is very simple >>> print(df[df.what == 'foo']) # data where what == foo what location 0 foo here 1 foo there >>> list(df[df.what=='foo'].location.values) # list of values ['here', 'there'] >>> list(df.itertuples()) # if we want to use namedtuples [Pandas(Index=0, what='foo', location='here'), Pandas(Index=1, what='foo', location='there'), Pandas(Index=2, what='bar', location='eberywhere')] etc. From __peter__ at web.de Thu Feb 23 10:19:00 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 23 Feb 2017 16:19 +0100 Subject: Namedtuples problem References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: Pavol Lisy wrote: > If we are talking about less code than necessary then probably we > could use something from python's ecosystem... > >>>> import pandas as pd # really useful package Yes, pandas looks like the perfect fit here. There is a learning curve, though, so that it will probably pay off only after the second project ;) For example here's what my session might have looked like: >>> df = pd.read_csv("what_location.csv") >>> df what location 0 foo here 1 foo there 2 bar eberywhere [3 rows x 2 columns] >>> df.groupby("what") >>> print(df.groupby("what")) >>> df.groupby("what")["foo"] Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/pandas/core/groupby.py", line 2477, in __getitem__ raise KeyError(str(key)) KeyError: 'foo' >>> df.groupby("what").keys() Traceback (most recent call last): File "", line 1, in TypeError: 'str' object is not callable >>> df.groupby("what").keys 'what' Frustrating. PS: >>> g.groups {'foo': [0, 1], 'bar': [2]} From chololennon at hotmail.com Thu Feb 23 10:19:51 2017 From: chololennon at hotmail.com (Cholo Lennon) Date: Thu, 23 Feb 2017 12:19:51 -0300 Subject: Python application launcher (for Python code) References: <8560k5mie8.fsf@benfinney.id.au> <003f01d28b90$385d8450$27b23dae@sambora> Message-ID: On 02/21/2017 10:52 AM, BartC wrote: > > The only problem I found, when launching in GUI mode, was that a > console-mode Python program would briefly open a console window that > would then promptly disappear if it finished immediately and there was > no interaction. If you want to avoid a console just use pythonw.exe with your applications instead of python.exe (modify your launcher scripts to do that or change the registration for .py/pyc extensions in the OS configuration -by default python.exe is the associated application) Regards -- Cholo Lennon Bs.As. ARG From torriem at gmail.com Thu Feb 23 10:43:11 2017 From: torriem at gmail.com (Michael Torrie) Date: Thu, 23 Feb 2017 08:43:11 -0700 Subject: Python application launcher (for Python code) In-Reply-To: <00cc01d28d90$350fa330$27b23dae@sambora> References: <00cc01d28d90$350fa330$27b23dae@sambora> Message-ID: <25be7339-c409-4692-ea0e-848860691b3c@gmail.com> On 2017-02-22 09:49 PM, Deborah Swanson wrote: > Didn't even look. Visual Studio has always been pricey, and it never > occurred to me that they might have a free or cheap version now. You can get the full edition of Visual Studio, called Visual Studio Community Edition for free. They still offer Visual Studio Express, but I think they recommend the full community edition to most people now. The biggest downside to the VS Community Edition is that it has to phone home and log in to MS's developer web site from time to time to stay active. Sigh. MS almost gets it, but not quite. From chololennon at hotmail.com Thu Feb 23 10:58:24 2017 From: chololennon at hotmail.com (Cholo Lennon) Date: Thu, 23 Feb 2017 12:58:24 -0300 Subject: Python application launcher (for Python code) References: <005b01d28bcc$942975a0$27b23dae@sambora> Message-ID: On 02/20/2017 07:56 PM, Deborah Swanson wrote: > Basically, I now have quite a few Python programs I use frequently, and > as time goes on my collection and uses of it will grow. Right now I just > want a way to select which one I'd like to run and run it. I'd like it > to be a standalone application and some sort of system of categories > would be nice. Well, there are a lot of possibilities (the amount of responses shows that), but in this situation I'd simply create a Windows shortcut for every python script. Later, shortcuts can be put in a folder (or in start menu folder) If the script has no screen input/output I'd edit the shortcut and prefix the script pathname with pythonw.exe (to avoid the empty console). Also I'd add the python path if it is not in the user/system PATH or if you have multiple python installations) Regards -- Cholo Lennon Bs.As. ARG From gandalf at shopzeus.com Thu Feb 23 11:44:47 2017 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 23 Feb 2017 17:44:47 +0100 Subject: concurrent futures, async futures and await In-Reply-To: References: Message-ID: > My guess: because asyncio wouldn't know what to do with a > concurrent.futures.Future. I see that as a problem. > > The tornado docs say that "You can also use > tornado.gen.convert_yielded to convert anything that would work with > yield into a form that will work with await": > http://www.tornadoweb.org/en/stable/guide/coroutines.html#python-3-5-async-and-await It is true if you yield it from a normal tornado.gen.coroutine, and then let tornado's own ioloop handle it. But if you try to refactor to a native coroutine and await for it (instead of yield), then it will stop working. It is because the await statement is not implemented by tornado. It is implemented in core Python, and it does not support a concurrent futures. But hey, concurrent futures are part of the standard library, so they *should* work together. Here is an example that makes the problem clear. This one works: executor = ThreadPoolExecutor(4) @tornado.gen.coroutine def produce_chunks(filename, chunk_size=8192): with open(filename,"rb") as fin: chunk = yield executor.submit(f.read, chunk_size) process_chunk(chunk) It is because "yield executor.submit" will yield the concurrent future to the ioloop's handler, and tornado's ioloop is clever enough to detect it, and wrap it in a thread-safe way. But this won't work: async def produce_chunks(filename, chunk_size=8192): with open(filename,"rb") as fin: chunk = await executor.submit(f.read, chunk_size) process_chunk(chunk) Simply because the concurrent future returned by executor.submit does not implement __await__ and so it cannot be awaited for. Right now asyncio does not know how to handle this. But I think it should, because it is part of the standard library, and there are very important use cases (like the one above) that cannot be implemented without concurrent futures. In the above example, file.read will almost certainly block the execution, and there is no platform independent way of doing an async file read operation other than doing it in a different thread. AFAIK async file reads are not supported in the Linux kernel, and the only way to do this is to use a thread. Of course, asyncio should not care if the executor is doing the task in a different thread or a different process. All I'm saying is that concurrent.futures.Future should implement the __await__ method, and asyncio should be able to use it. From python at deborahswanson.net Thu Feb 23 12:24:47 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 09:24:47 -0800 Subject: Namedtuples problem In-Reply-To: Message-ID: <006501d28df9$bbd2c050$27b23dae@sambora> Thanks all of you for replying. I'm going to have to study your responses a bit before I can respond. I wrote this code the way I did because I was getting errors that I stopped getting with these fixes and I got the correct results from this code, up until the last line. All of you introduced concepts that I haven't seen or thought of before and I'll need to look at them carefully. Peter's right about pandas and we talked about this the first time we visited this problem. pandas is a steep learning curve, and it makes more sense to stick with straight Python this first time. This spreadsheet problem is fairly typical of several I need to do, and I will be using pandas for future problems. But this time I'll stick with learning and understanding what I've got. Thanks again all, and I'll write back when I have a handle on what you've said, and probably I'll have more questions to ask. From ian.g.kelly at gmail.com Thu Feb 23 13:33:15 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 23 Feb 2017 11:33:15 -0700 Subject: concurrent futures, async futures and await In-Reply-To: References: Message-ID: On Thu, Feb 23, 2017 at 9:44 AM, Nagy L?szl? Zsolt wrote: > >> My guess: because asyncio wouldn't know what to do with a >> concurrent.futures.Future. > I see that as a problem. >> >> The tornado docs say that "You can also use >> tornado.gen.convert_yielded to convert anything that would work with >> yield into a form that will work with await": >> http://www.tornadoweb.org/en/stable/guide/coroutines.html#python-3-5-async-and-await > It is true if you yield it from a normal tornado.gen.coroutine, and then > let tornado's own ioloop handle it. But if you try to refactor to a > native coroutine and await for it (instead of yield), then it will stop > working. It is because the await statement is not implemented by > tornado. It is implemented in core Python, and it does not support a > concurrent futures. But hey, concurrent futures are part of the standard > library, so they *should* work together. > > Here is an example that makes the problem clear. > > This one works: > > executor = ThreadPoolExecutor(4) > > @tornado.gen.coroutine > def produce_chunks(filename, chunk_size=8192): > with open(filename,"rb") as fin: > chunk = yield executor.submit(f.read, chunk_size) > process_chunk(chunk) > > > It is because "yield executor.submit" will yield the concurrent future > to the ioloop's handler, and tornado's ioloop is clever enough to detect > it, and wrap it in a thread-safe way. > > But this won't work: > > async def produce_chunks(filename, chunk_size=8192): > with open(filename,"rb") as fin: > chunk = await executor.submit(f.read, chunk_size) > process_chunk(chunk) > > > Simply because the concurrent future returned by executor.submit does > not implement __await__ and so it cannot be awaited for. I get that, but what happens if you try wrapping the executor.submit call with tornado.gen.convert_yielded as the tornado docs suggest and as I suggested above? > Right now asyncio does not know how to handle this. But I think it > should, because it is part of the standard library, and there are very > important use cases (like the one above) that cannot be implemented > without concurrent futures. In the above example, file.read will almost > certainly block the execution, and there is no platform independent way > of doing an async file read operation other than doing it in a different > thread. AFAIK async file reads are not supported in the Linux kernel, > and the only way to do this is to use a thread. > > Of course, asyncio should not care if the executor is doing the task in > a different thread or a different process. All I'm saying is that > concurrent.futures.Future should implement the __await__ method, and > asyncio should be able to use it. I found this in the original PEP at http://legacy.python.org/dev/peps/pep-3156/#futures: """ In the future (pun intended) we may unify asyncio.Future and concurrent.futures.Future, e.g. by adding an __iter__() method to the latter that works with yield from. To prevent accidentally blocking the event loop by calling e.g. result() on a Future that's not done yet, the blocking operation may detect that an event loop is active in the current thread and raise an exception instead. However the current PEP strives to have no dependencies beyond Python 3.3, so changes to concurrent.futures.Future are off the table for now. """ Maybe we're now far enough into the future that this could be reconsidered. In the meantime, the user does have other options: 1) For tornado, use tornado.gen.convert_yielded. 2) For asyncio, create an asyncio future and link them by setting a callback on the concurrent future that propagates the result (using call_soon_threadsafe since the callback may run in another thread). 3) Better, don't use concurrent.futures directly in the first place; instead use https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.AbstractEventLoop.run_in_executor which runs a function in an Executor and wraps it up via #2 transparently. From skip.montanaro at gmail.com Thu Feb 23 14:23:09 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 23 Feb 2017 13:23:09 -0600 Subject: Metaclass to rewrite class __doc__? Message-ID: I stumbled upon this use of a metaclass to modify function docstrings: http://www.jesshamrick.com/2013/04/17/rewriting-python-docstrings-with-a-metaclass/ In certain circumstances (mostly when extending some Pybind11 wrappers), it might be nice to extend/modify a class's docstring in a similar manner. Alas, my first attempt at that was unsuccessful. All I was trying to do was the following (skipping any error checking): class _RewriteDocMeta(type): def __new__(cls, name, parents, attrs): cls.__doc__ = parents[0].__doc__ + "\n\n" + cls.__doc__ return super(_RewriteDocMeta, cls).__new__(cls, name, parents, attrs) Using that as the __metaclass__ for a class resulted in: ... '__doc__' of 'type' objects is not writable. Does this imply a requirement for a metaclass metaclass? In Jess Hamrick's blog, she built entirely new function objects out of their constituent bits, then set their __doc__ attributes. She didn't attempt to replace/modify an existing function (or unbound method) docstring. I fear that might be required in this case as well. This works: class F(object): "F doc" pass G = type(F)("G", (F,), {"__doc__": F.__doc__ + "\n" + "G doc"}) but I'm not sure how to translate that into a working __metaclass__ example (my brain is slowly exploding [*] as I write this)... Thx, Skip [*] Whatever happened to Don Beaudry? From __peter__ at web.de Thu Feb 23 15:02:13 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 23 Feb 2017 21:02:13 +0100 Subject: Metaclass to rewrite class __doc__? References: Message-ID: Skip Montanaro wrote: > I stumbled upon this use of a metaclass to modify function docstrings: > > http://www.jesshamrick.com/2013/04/17/rewriting-python-docstrings-with-a-metaclass/ > > In certain circumstances (mostly when extending some Pybind11 wrappers), > it might be nice to extend/modify a class's docstring in a similar manner. > Alas, my first attempt at that was unsuccessful. All I was trying to do > was the following (skipping any error checking): > > class _RewriteDocMeta(type): > def __new__(cls, name, parents, attrs): > cls.__doc__ = parents[0].__doc__ + "\n\n" + cls.__doc__ Here cls is _RewriteDocMeta, but I think you want to change the not yet existing class, i. e. an instance of _RewriteDocMeta. As its attributes are in the attrs dict you have to update that dictionary: $ cat demo.py class Meta(type): def __new__(cls, classname, bases, attrs): attrs["__doc__"] = "{}: {}".format( classname, attrs.get("__doc__") or "" ) return type.__new__(cls, classname, bases, attrs) class A(metaclass=Meta): "hello" print(A.__doc__) class B(A): "goodbye" print(B.__doc__) class C(metaclass=Meta): pass print(C.__doc__) $ python3 demo.py A: hello B: goodbye C: $ > return super(_RewriteDocMeta, cls).__new__(cls, name, parents, > attrs) > > Using that as the __metaclass__ for a class resulted in: > > ... '__doc__' of 'type' objects is not writable. > > Does this imply a requirement for a metaclass metaclass? In Jess Hamrick's > blog, she built entirely new function objects out of their constituent > bits, then set their __doc__ attributes. She didn't attempt to > replace/modify an existing function (or unbound method) docstring. I fear > that might be required in this case as well. > > This works: > > class F(object): > "F doc" > pass > > G = type(F)("G", (F,), {"__doc__": F.__doc__ + "\n" + "G doc"}) > > but I'm not sure how to translate that into a working __metaclass__ > example (my brain is slowly exploding > [*] as I write this)... > > Thx, > > Skip > > [*] Whatever happened to Don Beaudry? From juan0christian at gmail.com Thu Feb 23 16:33:06 2017 From: juan0christian at gmail.com (Juan C.) Date: Thu, 23 Feb 2017 21:33:06 +0000 (UTC) Subject: How to create a Python 3.6 traceroute without SOCK RAW? Message-ID: <641BCF333CE60F80.5F265467-84C6-4079-9DE9-C2F32CD36B8D@mail.outlook.com> I need to implement a traceroute inside my script but I can't escalate privileges. ?Unix uses UDP for traceroute, but I didn't find any material regarding UDP traceroute in Python. From irmen.NOSPAM at xs4all.nl Thu Feb 23 17:42:22 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Thu, 23 Feb 2017 23:42:22 +0100 Subject: How to create a Python 3.6 traceroute without SOCK RAW? In-Reply-To: References: <641BCF333CE60F80.5F265467-84C6-4079-9DE9-C2F32CD36B8D@mail.outlook.com> Message-ID: <58af654d$0$674$e4fe514c@news.xs4all.nl> On 23-2-2017 22:33, Juan C. wrote: > I need to implement a traceroute inside my script but I can't escalate privileges. Unix uses UDP for traceroute, but I didn't find any material regarding UDP traceroute in Python. import os os.system("traceroute www.google.com") From python at deborahswanson.net Thu Feb 23 17:51:49 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 14:51:49 -0800 Subject: Namedtuples problem In-Reply-To: Message-ID: <006801d28e27$6ba1f2d0$27b23dae@sambora> Peter Otten wrote, on February 23, 2017 2:34 AM > > Deborah Swanson wrote: > > > > Can anyone see why I'm getting this Index error? and how to fix it? > > I'm not completely sure I can follow you, but you seem to be > mixing two problems > > (1) split a list into groups > (2) convert a list of rows into a list of columns Actually, I was trying to duplicate your original intentions, which I thought were quite excellent, but turned out to cause problems when I tried to use the code you gave. Your original intention was to make a dictionary with the keys being each of the unique titles in all the records, and the values are the complete records that contain the unique title. (Each rental listing can have many records, each with the same title.) > and making a kind of mess in the process. Functions to the rescue: I'm sorry you think I made a mess of it and I agree that the code I wrote is clumsy, although it does work and gives the correct results up to the last line I gave. I was hoping, among other things, that you would help me clean it up, so let's look at what you said. > #untested > > def split_into_groups(records, key): > groups = defaultdict(list) > for record in records: > # no need to check if a group already exists > # an empty list will automatically added for every > # missing key > groups[key(record)].append(record) > return groups I used this approach the first time I tried this for both defaultdict and OrderedDict, and for both of them I immediately got a KeyError for the first record. groups is empty, so the title for the first record wouldn't already be in groups. Just to check, I commented out the extra lines that I added to handle new keys in my code and immediately got the same KeyError. My guess is that while standard dictionaries will automatically make a new key if it isn't found in the dict, defaultdict and OrderedDict will not. So it seems you need to handle new keys yourself. Unless you think I'm doing something wrong and dicts from collections should also automatically make new keys. Rightly or wrongly, I chose not to use functions in my attempt, just to keep the steps sequential and understandable. Probably I should have factored out the functions before I posted it. > def extract_column(records, name): > # you will agree that extracting one column is easy :) > return [getattr(record, name) for record in records] > > def extract_columns(records, names): > # we can build on that to make a list of columns > return [extract_column(records, name) for name in names] > > wanted_columns = ['Location', ...] > records = ... > groups = split_into_groups(records, operator.attrgetter("title")) > > Columns = namedtuple("Columns", wanted_columns) > for title, group in groups.items(): > # for easier access we turn the list of columns > # into a namedtuple of columns > groups[title] = Columns._make(extract_columns(wanted_columns)) This approach essentially reverses the order of the steps from what I did, making the columns first and then grouping the records by title. Either order should work in principle. > If all worked well you should now be able to get a group with > > group["whatever"] > > and all locations for that group with > > group["whatever"].Locations > > If there is a bug you can pinpoint the function that doesn't work and ask > for specific help on that one. I'll play with your code and see if it works better than what I had. I can see right off that the line group["whatever"].Locations will fail because group only has a 'Location' field and doesn't have a 'Locations' field. Running it in the watch window confirms, and it gets: AttributeError: 'Record' object has no attribute 'Locations' Earlier on I tried several methods to get an anology to your line group["whatever"].Locations, and failing to do it straightforwardly is part of why my code is so convoluted. Many thanks for your reply. Quite possibly getting the columns first and grouping the records second will be an improvement. From python at deborahswanson.net Thu Feb 23 18:00:53 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 15:00:53 -0800 Subject: Namedtuples problem In-Reply-To: Message-ID: <000001d28e28$afbde720$27b23dae@sambora> > -----Original Message----- > From: Erik [mailto:python at lucidity.plus.com] > Sent: Thursday, February 23, 2017 2:09 AM > To: python at deborahswanson.net; python-list at python.org > Subject: Re: Namedtuples problem > > > Hi, > > On 23/02/17 09:38, Deborah Swanson wrote: > > group[[idx][records_idx[label]]] > > gets an IndexError: list index out of range > > [snip] > > > Can anyone see why I'm getting this Index error? and how to fix it? > > It looks to me like you are indexing into a single-element > list that you > are creating using the literal list syntax in the middle of > the expression. Actually, group is essentially a 2-element list. Each group has a list of rows, and each row has a set of fields. group has to be indexed by row index and field index. (This is a namedtuple configuration.) The weirdness is that group[0][4] gets the right answer, but group[[idx][records_idx[label]]], where idx = 0 and records_idx[label]] = 4 gets the IndexError. > If we were to break the expression into parts to make it a > bit simpler > to refer to discuss: > > ridx = records_idx[label] > group[[idx][ridx]] > > You can now more easily see that 'group' is being indexed by the > expression "[idx][ridx]". What does that mean? > > [idx] is creating a single-element list using literal list > syntax. This > is then indexed using 'ridx' (using, perhaps confusingly, the > exact same > syntax to do a different thing). > > The result of *that* expression is then being used to index > 'group', but > it won't get that far because you'll get the exception if 'ridx' is > anything but zero. > > So the initial problem at least is the extra [] around 'idx' which is > creating a list on the fly for you. > > E. > From juan0christian at gmail.com Thu Feb 23 18:20:22 2017 From: juan0christian at gmail.com (Juan C.) Date: Thu, 23 Feb 2017 20:20:22 -0300 Subject: How to create a Python 3.6 traceroute without SOCK RAW? In-Reply-To: <58af654d$0$674$e4fe514c@news.xs4all.nl> References: <641BCF333CE60F80.5F265467-84C6-4079-9DE9-C2F32CD36B8D@mail.outlook.com> <58af654d$0$674$e4fe514c@news.xs4all.nl> Message-ID: On Thu, Feb 23, 2017 at 7:42 PM, Irmen de Jong wrote: > > import os > os.system("traceroute www.google.com") Indeed, that would work, but it isn't a great approach in my opinion because I would rely on a system command, in this case Windows uses tracert while UNIX uses traceroute, one could fix that with a if-else to check the os and then use the correct command, but it just make it feel even more like a workaround and not a definitive solution. From loza_pl at hotmail.com Thu Feb 23 18:21:28 2017 From: loza_pl at hotmail.com (Pablo Lozano) Date: Thu, 23 Feb 2017 23:21:28 +0000 Subject: Spyder 3.1.3 update Message-ID: Good day, I installed the Spyder 3.6 IDE from the Anaconda package and at the start k to the IDE I get the Spyder Update saying Spyder 3.1.3 is available. Did it incorrectly install Anaconda? As far as I know Spyder 3.1.3 is rather old and the latest is 3.6 Regards Pablo From irmen.NOSPAM at xs4all.nl Thu Feb 23 18:35:28 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Fri, 24 Feb 2017 00:35:28 +0100 Subject: How to create a Python 3.6 traceroute without SOCK RAW? In-Reply-To: References: <641BCF333CE60F80.5F265467-84C6-4079-9DE9-C2F32CD36B8D@mail.outlook.com> <58af654d$0$674$e4fe514c@news.xs4all.nl> Message-ID: <58af71bf$0$698$e4fe514c@news.xs4all.nl> On 24-2-2017 0:20, Juan C. wrote: > On Thu, Feb 23, 2017 at 7:42 PM, Irmen de Jong wrote: >> >> import os >> os.system("traceroute www.google.com") > > Indeed, that would work, but it isn't a great approach in my opinion > because I would rely on a system command, in this case Windows uses > tracert while UNIX uses traceroute, one could fix that with a if-else > to check the os and then use the correct command, but it just make it > feel even more like a workaround and not a definitive solution. > I don't consider this a workaround at all. Why reinvent the wheel? Also: once you're going to deal with low level socket API you're going to run into platform differences anyway. Irmen From Irv at furrypants.com Thu Feb 23 18:50:59 2017 From: Irv at furrypants.com (Irv Kalb) Date: Thu, 23 Feb 2017 15:50:59 -0800 Subject: Namedtuples problem In-Reply-To: <000001d28e28$afbde720$27b23dae@sambora> References: <000001d28e28$afbde720$27b23dae@sambora> Message-ID: <3E25CE60-2949-4E04-8B31-53250CA2C420@furrypants.com> > On Feb 23, 2017, at 3:00 PM, Deborah Swanson > wrote: > > The weirdness is that > > group[0][4] > > gets the right answer, but > > group[[idx][records_idx[label]]], > where idx = 0 and records_idx[label]] = 4 If that's the case, then I think you need this instead: group[idx][records_idx[label]] Irv From python at mrabarnett.plus.com Thu Feb 23 18:58:35 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 23 Feb 2017 23:58:35 +0000 Subject: Namedtuples problem In-Reply-To: <006801d28e27$6ba1f2d0$27b23dae@sambora> References: <006801d28e27$6ba1f2d0$27b23dae@sambora> Message-ID: <155d8915-90b7-0e95-94bf-0b9f86569d8d@mrabarnett.plus.com> On 2017-02-23 22:51, Deborah Swanson wrote: > Peter Otten wrote, on February 23, 2017 2:34 AM [snip] >> #untested >> >> def split_into_groups(records, key): >> groups = defaultdict(list) >> for record in records: >> # no need to check if a group already exists >> # an empty list will automatically added for every >> # missing key >> groups[key(record)].append(record) >> return groups > > I used this approach the first time I tried this for both defaultdict > and OrderedDict, and for both of them I immediately got a KeyError for > the first record. groups is empty, so the title for the first record > wouldn't already be in groups. > If the key isn't already in the dictionary, a defaultdict will create the entry whereas a normal dict will raise KeyError. > Just to check, I commented out the extra lines that I added to handle > new keys in my code and immediately got the same KeyError. > > My guess is that while standard dictionaries will automatically make a > new key if it isn't found in the dict, defaultdict and OrderedDict will > not. So it seems you need to handle new keys yourself. Unless you think > I'm doing something wrong and dicts from collections should also > automatically make new keys. > defaultdict will, dict and OrderedDict won't. [snip] From python at lucidity.plus.com Thu Feb 23 19:02:53 2017 From: python at lucidity.plus.com (Erik) Date: Fri, 24 Feb 2017 00:02:53 +0000 Subject: Namedtuples problem In-Reply-To: <000001d28e28$afbde720$27b23dae@sambora> References: <000001d28e28$afbde720$27b23dae@sambora> Message-ID: <3c2b553c-baad-3b55-1ff0-92b5a89f5ce3@lucidity.plus.com> On 23/02/17 23:00, Deborah Swanson wrote: >> It looks to me like you are indexing into a single-element >> list that you >> are creating using the literal list syntax in the middle of >> the expression. > > Actually, group is essentially a 2-element list. Each group has a list > of rows, and each row has a set of fields. group has to be indexed by > row index and field index. (This is a namedtuple configuration.) > > The weirdness is that > > group[0][4] > > gets the right answer, but > > group[[idx][records_idx[label]]], > where idx = 0 and records_idx[label]] = 4 > > gets the IndexError. So remove the outermost square brackets then so the two expressions are the same (what I - and also Steven - mentioned is correct: you are creating a single-element list and indexing it (and then using the result of that, should it work, to index 'group')). The same thing as "group[0][4]" in your example is: group[idx][records_idx[label]] (assuming you have all those things correct - I haven't studied your code in a lot of detail). Your new example expressed using the original construct you posted is: group[[0][4]] ... see the extra enclosing square brackets? E. From python at mrabarnett.plus.com Thu Feb 23 19:06:20 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 24 Feb 2017 00:06:20 +0000 Subject: Namedtuples problem In-Reply-To: <000001d28e28$afbde720$27b23dae@sambora> References: <000001d28e28$afbde720$27b23dae@sambora> Message-ID: On 2017-02-23 23:00, Deborah Swanson wrote: > > >> -----Original Message----- >> From: Erik [mailto:python at lucidity.plus.com] >> Sent: Thursday, February 23, 2017 2:09 AM >> To: python at deborahswanson.net; python-list at python.org >> Subject: Re: Namedtuples problem >> >> >> Hi, >> >> On 23/02/17 09:38, Deborah Swanson wrote: >> > group[[idx][records_idx[label]]] >> > gets an IndexError: list index out of range >> >> [snip] >> >> > Can anyone see why I'm getting this Index error? and how to fix it? >> >> It looks to me like you are indexing into a single-element >> list that you >> are creating using the literal list syntax in the middle of >> the expression. > > Actually, group is essentially a 2-element list. Each group has a list > of rows, and each row has a set of fields. group has to be indexed by > row index and field index. (This is a namedtuple configuration.) > > The weirdness is that > > group[0][4] > > gets the right answer, but > > group[[idx][records_idx[label]]], > where idx = 0 and records_idx[label]] = 4 > > gets the IndexError. > It's not weird. As Erik says below, and I'll repeat here, you have: group[[idx][records_idx[label]]] That consists of the expression: [idx][records_idx[label]] within: group[...] The [idx] creates a one-element list. You then try to subscript it with records_idx[label]. If records_idx[label] is anything other than 0 (or -1), it'll raise IndexError because there's only one element in that list. If you substitute idx == 0 and records_idx[label]] == 4 into: group[[idx][records_idx[label]]] you'll get: group[[0][4]] which is not the same thing as group[0][4]! >> If we were to break the expression into parts to make it a >> bit simpler >> to refer to discuss: >> >> ridx = records_idx[label] >> group[[idx][ridx]] >> >> You can now more easily see that 'group' is being indexed by the >> expression "[idx][ridx]". What does that mean? >> >> [idx] is creating a single-element list using literal list >> syntax. This >> is then indexed using 'ridx' (using, perhaps confusingly, the >> exact same >> syntax to do a different thing). >> >> The result of *that* expression is then being used to index >> 'group', but >> it won't get that far because you'll get the exception if 'ridx' is >> anything but zero. >> >> So the initial problem at least is the extra [] around 'idx' which is >> creating a list on the fly for you. >> From Irv at furrypants.com Thu Feb 23 19:19:04 2017 From: Irv at furrypants.com (Irv Kalb) Date: Thu, 23 Feb 2017 16:19:04 -0800 Subject: Disallowing instantiation of super class Message-ID: Hi, I have built a set of three classes: - A super class, let's call it: Base - A class that inherits from Base, let's call that: ClassA - Another class that inherits from Base, let's call that: ClassB ClassA and ClassB have some code in their __init__ methods that set some instance variables to different values. After doing so, they call the the __init__ method of their common super class (Base) to set some other instance variables to some common values. This all works great. Instances of ClassA and ClassB do just what I want them to. I would like to add is some "insurance" that I (or someone else who uses my code) never instantiates my Base class, It is not intended to be instantiated because some of the needed instance variables are only created in the __init__ method of ClassA and ClassB. I am looking for some way in the Base's __init__ method to determine if the method was called directly: instanceOfBase = Base(... some data ...) # I want this case to generate an error I tried using "isinstance(self, Base)", but it returns True when I instantiate an object from ClassA, from ClassB, or from Base. If I can find a way to determine that the caller is attempting to instantiate Base directly, I will raise an exception. Thanks, Irv (If it makes a difference, I am doing this currently in Python 2.7 - please don't beat me up about that.) From ben.usenet at bsb.me.uk Thu Feb 23 19:23:48 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 24 Feb 2017 00:23:48 +0000 Subject: Namedtuples problem References: <000001d28e28$afbde720$27b23dae@sambora> Message-ID: <874lzk5td7.fsf@bsb.me.uk> "Deborah Swanson" writes: >> -----Original Message----- >> From: Erik [mailto:python at lucidity.plus.com] >> Sent: Thursday, February 23, 2017 2:09 AM >> To: python at deborahswanson.net; python-list at python.org >> Subject: Re: Namedtuples problem >> >> >> Hi, >> >> On 23/02/17 09:38, Deborah Swanson wrote: >> > group[[idx][records_idx[label]]] >> > gets an IndexError: list index out of range >> >> [snip] >> >> > Can anyone see why I'm getting this Index error? and how to fix it? >> >> It looks to me like you are indexing into a single-element >> list that you >> are creating using the literal list syntax in the middle of >> the expression. > > Actually, group is essentially a 2-element list. Each group has a list > of rows, and each row has a set of fields. group has to be indexed by > row index and field index. (This is a namedtuple configuration.) > > The weirdness is that > > group[0][4] Sure. No trouble there. > gets the right answer, but > > group[[idx][records_idx[label]]], > where idx = 0 and records_idx[label]] = 4 > > gets the IndexError. group is not involved in the index error. Just write [idx][records_idx[label]] and you'll get the error. Wrapping that up in group[...] won't make a difference. In fact, since idx is 0 and records_idx[label]] is 4, you get the same error (for the same reason) by typing [0][4] into a Python REPL. Does it still seem weird? This is just another shot at explaining the issue because, sometimes, different words can help. I thought the message you are replying to (as well as others) explained it well, but it obviously it did not hit the spot. -- Ben. From python at mrabarnett.plus.com Thu Feb 23 19:40:58 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 24 Feb 2017 00:40:58 +0000 Subject: Disallowing instantiation of super class In-Reply-To: References: Message-ID: <861d5d77-c806-13de-4c84-25feac90626c@mrabarnett.plus.com> On 2017-02-24 00:19, Irv Kalb wrote: > Hi, > > I have built a set of three classes: > > - A super class, let's call it: Base > > - A class that inherits from Base, let's call that: ClassA > > - Another class that inherits from Base, let's call that: ClassB > > ClassA and ClassB have some code in their __init__ methods that set some instance variables to different values. After doing so, they call the the __init__ method of their common super class (Base) to set some other instance variables to some common values. This all works great. Instances of ClassA and ClassB do just what I want them to. > > I would like to add is some "insurance" that I (or someone else who uses my code) never instantiates my Base class, It is not intended to be instantiated because some of the needed instance variables are only created in the __init__ method of ClassA and ClassB. I am looking for some way in the Base's __init__ method to determine if the method was called directly: > > instanceOfBase = Base(... some data ...) # I want this case to generate an error > > I tried using "isinstance(self, Base)", but it returns True when I instantiate an object from ClassA, from ClassB, or from Base. > > If I can find a way to determine that the caller is attempting to instantiate Base directly, I will raise an exception. > > Thanks, > > Irv > > (If it makes a difference, I am doing this currently in Python 2.7 - please don't beat me up about that.) > Apart from renaming Base to _Base as a hint, you could put Base's initialisation code in, say, '_init' and have Base's __init__ just raise an exception. ClassA and ClassB would then call Base's _init instead of its __init__. From __peter__ at web.de Thu Feb 23 19:58:45 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 24 Feb 2017 01:58:45 +0100 Subject: Disallowing instantiation of super class References: Message-ID: Irv Kalb wrote: > Hi, > > I have built a set of three classes: > > - A super class, let's call it: Base > > - A class that inherits from Base, let's call that: ClassA > > - Another class that inherits from Base, let's call that: ClassB > > ClassA and ClassB have some code in their __init__ methods that set some > instance variables to different values. After doing so, they call the the > __init__ method of their common super class (Base) to set some other > instance variables to some common values. This all works great. > Instances of ClassA and ClassB do just what I want them to. > > I would like to add is some "insurance" that I (or someone else who uses > my code) never instantiates my Base class, It is not intended to be > instantiated because some of the needed instance variables are only > created in the __init__ method of ClassA and ClassB. I am looking for > some way in the Base's __init__ method to determine if the method was > called directly: > > instanceOfBase = Base(... some data ...) # I want this case to > generate an error > > I tried using "isinstance(self, Base)", but it returns True when I > instantiate an object from ClassA, from ClassB, or from Base. > > If I can find a way to determine that the caller is attempting to > instantiate Base directly, I will raise an exception. > > Thanks, > > Irv > > (If it makes a difference, I am doing this currently in Python 2.7 - > please don't beat me up about that.) >>> import abc >>> class Base: ... __metaclass__ = abc.ABCMeta ... @abc.abstractmethod ... def __init__(self): ... self.z = self.x + self.y ... >>> class A(Base): ... def __init__(self): ... self.x = 2 ... self.y = 3 ... super(A, self).__init__() ... >>> A().z 5 >>> Base() Traceback (most recent call last): File "", line 1, in TypeError: Can't instantiate abstract class Base with abstract methods __init__ From python at deborahswanson.net Thu Feb 23 20:34:43 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 17:34:43 -0800 Subject: Namedtuples problem In-Reply-To: Message-ID: <001b01d28e3e$2d06e9b0$27b23dae@sambora> MRAB wrote, on Thursday, February 23, 2017 4:06 PM > > On 2017-02-23 23:00, Deborah Swanson wrote: > > > > > >> -----Original Message----- > >> From: Erik [mailto:python at lucidity.plus.com] > >> Sent: Thursday, February 23, 2017 2:09 AM > >> To: python at deborahswanson.net; python-list at python.org > >> Subject: Re: Namedtuples problem > >> > >> > >> Hi, > >> > >> On 23/02/17 09:38, Deborah Swanson wrote: > >> > group[[idx][records_idx[label]]] > >> > gets an IndexError: list index out of range > >> > >> [snip] > >> > >> > Can anyone see why I'm getting this Index error? and how > to fix it? > >> > >> It looks to me like you are indexing into a single-element > list that > >> you are creating using the literal list syntax in the middle of > >> the expression. > > > > Actually, group is essentially a 2-element list. Each group > has a list > > of rows, and each row has a set of fields. group has to be > indexed by > > row index and field index. (This is a namedtuple configuration.) > > > > The weirdness is that > > > > group[0][4] > > > > gets the right answer, but > > > > group[[idx][records_idx[label]]], > > where idx = 0 and records_idx[label]] = 4 > > > > gets the IndexError. > > > It's not weird. As Erik says below, and I'll repeat here, you have: > > group[[idx][records_idx[label]]] > > That consists of the expression: > > [idx][records_idx[label]] > > within: > > group[...] > > The [idx] creates a one-element list. Yes, group[idx] gives you a row from group > You then try to subscript it with records_idx[label]. If > records_idx[label] is anything other than 0 (or -1), it'll raise > IndexError because there's only one element in that list. No, there are currently 17 elements in a record that records_idx[label] could be referring to in my running code, depending on which field 'label' refers to. Please see the breakdown of what a typical group looks like that I gave Steve. If you can't find it, let me know and I'll repeat it. > If you substitute idx == 0 and records_idx[label]] == 4 into: > > group[[idx][records_idx[label]]] > > you'll get: > > group[[0][4]] > > which is not the same thing as group[0][4]! No it isn't! And that's probably where the mysterious IndexError is coming from! Sickening how crosseyed I can get working with these things for too long at a stretch. And in fact, deleting that outer set of square brackets not only get's rid of the IndexError, but group[idx][records_idx[label]] now fers to the correct set of field values I was trying to access. Thank you, thank you! From python at deborahswanson.net Thu Feb 23 20:38:32 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 17:38:32 -0800 Subject: Namedtuples problem In-Reply-To: <874lzk5td7.fsf@bsb.me.uk> Message-ID: <001c01d28e3e$b5bec2a0$27b23dae@sambora> Ben Bacarisse wrote, on February 23, 2017 4:24 PM > > "Deborah Swanson" writes: > > >> -----Original Message----- > >> From: Erik [mailto:python at lucidity.plus.com] > >> Sent: Thursday, February 23, 2017 2:09 AM > >> To: python at deborahswanson.net; python-list at python.org > >> Subject: Re: Namedtuples problem > >> > >> > >> Hi, > >> > >> On 23/02/17 09:38, Deborah Swanson wrote: > >> > group[[idx][records_idx[label]]] > >> > gets an IndexError: list index out of range > >> > >> [snip] > >> > >> > Can anyone see why I'm getting this Index error? and how > to fix it? > >> > >> It looks to me like you are indexing into a single-element > >> list that you > >> are creating using the literal list syntax in the middle of > >> the expression. > > > > Actually, group is essentially a 2-element list. Each group > has a list > > of rows, and each row has a set of fields. group has to be > indexed by > > row index and field index. (This is a namedtuple configuration.) > > > > The weirdness is that > > > > group[0][4] > > Sure. No trouble there. > > > gets the right answer, but > > > > group[[idx][records_idx[label]]], > > where idx = 0 and records_idx[label]] = 4 > > > > gets the IndexError. > > group is not involved in the index error. Just write > > [idx][records_idx[label]] > > and you'll get the error. Wrapping that up in group[...] > won't make a difference. In fact, since idx is 0 and > records_idx[label]] is 4, you get the same error (for the > same reason) by typing > > [0][4] > > into a Python REPL. Does it still seem weird? No, and you also put your finger on (psrt of) the real problem. See the reply I just sent to MRAB. > This is just another shot at explaining the issue because, > sometimes, different words can help. I thought the message > you are replying to (as well as others) explained it well, > but it obviously it did not hit the spot. > > > -- > Ben No, the spot wasn't getting hit, but it has been now. Thanks for replying. From python at deborahswanson.net Thu Feb 23 20:41:03 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 17:41:03 -0800 Subject: Namedtuples problem In-Reply-To: <3E25CE60-2949-4E04-8B31-53250CA2C420@furrypants.com> Message-ID: <001d01d28e3f$0fe81ec0$27b23dae@sambora> Irv Kalb wrote, on February 23, 2017 3:51 PM > > > On Feb 23, 2017, at 3:00 PM, Deborah Swanson > > > wrote: > > > > The weirdness is that > > > > group[0][4] > > > > gets the right answer, but > > > > group[[idx][records_idx[label]]], > > where idx = 0 and records_idx[label]] = 4 > > > If that's the case, then I think you need this instead: > > group[idx][records_idx[label]] > > Irv Yes, that's exactly right and it works in my running code. Three of you have now spotted the problem (and explained it in terms I can recognize). Thank you all! From python at deborahswanson.net Thu Feb 23 20:49:24 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 17:49:24 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <001f01d28e40$3a81ec50$27b23dae@sambora> Cholo Lennon wrote, on February 23, 2017 7:58 AM > > On 02/20/2017 07:56 PM, Deborah Swanson wrote: > > Basically, I now have quite a few Python programs I use frequently, > > and as time goes on my collection and uses of it will grow. > Right now > > I just want a way to select which one I'd like to run and > run it. I'd > > like it to be a standalone application and some sort of system of > > categories would be nice. > > Well, there are a lot of possibilities (the amount of responses shows > that), but in this situation I'd simply create a Windows shortcut for > every python script. Later, shortcuts can be put in a folder (or in > start menu folder) > > If the script has no screen input/output I'd edit the shortcut and > prefix the script pathname with pythonw.exe (to avoid the empty > console). Also I'd add the python path if it is not in the > user/system > PATH or if you have multiple python installations) > > Regards > > > -- > Cholo Lennon > Bs.As. > ARG Those are all good ideas, as was your first post, and I'm saving them to my project folder. I'd prefer to get a launcher that's already written, but I might end up rolling my own. From python at deborahswanson.net Thu Feb 23 20:54:46 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 17:54:46 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <25be7339-c409-4692-ea0e-848860691b3c@gmail.com> Message-ID: <002001d28e40$fa384ad0$27b23dae@sambora> Michael Torrie wrote, on February 23, 2017 7:43 AM > > On 2017-02-22 09:49 PM, Deborah Swanson wrote: > > Didn't even look. Visual Studio has always been pricey, and it never > > occurred to me that they might have a free or cheap version now. > > You can get the full edition of Visual Studio, called Visual Studio > Community Edition for free. They still offer Visual Studio Express, but > I think they recommend the full community edition to most people now. > The biggest downside to the VS Community Edition is that it has to phone > home and log in to MS's developer web site from time to time to stay > active. Sigh. MS almost gets it, but not quite. Another free version of Visual Studio, wonders never cease! As for it phoning home, I won't use it for long, and then I might not ever use it again. Wonder what value they think this has, other than giving them a nosecount of how many active copies there are at any given time. From python at deborahswanson.net Thu Feb 23 21:24:05 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 18:24:05 -0800 Subject: Namedtuples problem In-Reply-To: <3c2b553c-baad-3b55-1ff0-92b5a89f5ce3@lucidity.plus.com> Message-ID: <002101d28e45$12c25920$27b23dae@sambora> Erik wrote, on February 23, 2017 4:03 PM > > On 23/02/17 23:00, Deborah Swanson wrote: > >> It looks to me like you are indexing into a single-element > list that > >> you are creating using the literal list syntax in the middle of > >> the expression. > > > > Actually, group is essentially a 2-element list. Each group > has a list > > of rows, and each row has a set of fields. group has to be > indexed by > > row index and field index. (This is a namedtuple configuration.) > > > > The weirdness is that > > > > group[0][4] > > > > gets the right answer, but > > > > group[[idx][records_idx[label]]], > > where idx = 0 and records_idx[label]] = 4 > > > > gets the IndexError. > > So remove the outermost square brackets then so the two expressions are > the same (what I - and also Steven - mentioned is correct: you are > creating a single-element list and indexing it (and then using the > result of that, should it work, to index 'group')). I see that clearly now, I just couldn't understand it from what you and Steven were saying. > The same thing as "group[0][4]" in your example is: > > group[idx][records_idx[label]] > > (assuming you have all those things correct - I haven't studied your > code in a lot of detail). You have it exactly correct, so no code study required. ;) > Your new example expressed using the original construct you posted is: > > group[[0][4]] > > ... see the extra enclosing square brackets? > > E. Yes, I see them now, and deleting them makes everything work like it should. From Irv at furrypants.com Thu Feb 23 23:53:15 2017 From: Irv at furrypants.com (Irv Kalb) Date: Thu, 23 Feb 2017 20:53:15 -0800 Subject: Disallowing instantiation of super class In-Reply-To: <861d5d77-c806-13de-4c84-25feac90626c@mrabarnett.plus.com> References: <861d5d77-c806-13de-4c84-25feac90626c@mrabarnett.plus.com> Message-ID: <486E1A8B-20AC-41EB-BEC7-CC9C534F4552@furrypants.com> > On Feb 23, 2017, at 4:40 PM, MRAB wrote: > > On 2017-02-24 00:19, Irv Kalb wrote: >> Hi, >> >> I have built a set of three classes: >> >> - A super class, let's call it: Base >> >> - A class that inherits from Base, let's call that: ClassA >> >> - Another class that inherits from Base, let's call that: ClassB >> >> ClassA and ClassB have some code in their __init__ methods that set some instance variables to different values. After doing so, they call the the __init__ method of their common super class (Base) to set some other instance variables to some common values. This all works great. Instances of ClassA and ClassB do just what I want them to. >> >> I would like to add is some "insurance" that I (or someone else who uses my code) never instantiates my Base class, It is not intended to be instantiated because some of the needed instance variables are only created in the __init__ method of ClassA and ClassB. I am looking for some way in the Base's __init__ method to determine if the method was called directly: >> >> instanceOfBase = Base(... some data ...) # I want this case to generate an error >> >> I tried using "isinstance(self, Base)", but it returns True when I instantiate an object from ClassA, from ClassB, or from Base. >> >> If I can find a way to determine that the caller is attempting to instantiate Base directly, I will raise an exception. >> >> Thanks, >> >> Irv >> >> (If it makes a difference, I am doing this currently in Python 2.7 - please don't beat me up about that.) >> > Apart from renaming Base to _Base as a hint, you could put Base's initialisation code in, say, '_init' and have Base's __init__ just raise an exception. > > ClassA and ClassB would then call Base's _init instead of its __init__. > > -- > https://mail.python.org/mailman/listinfo/python-list > MRAB and Peter: Thank you very much for your solutions. I had considered both of these, but was wondering if there was another way. Peter's solution does exactly what I want, but I'm not ready to get into metaclasses. So I decided to go with MRAB's approach. I've modified my code and it seems to work great. Thank you both. Irv From greg.ewing at canterbury.ac.nz Fri Feb 24 00:07:07 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 24 Feb 2017 18:07:07 +1300 Subject: Namedtuples problem In-Reply-To: References: <001701d28db8$a2ea03f0$27b23dae@sambora> Message-ID: Deborah Swanson wrote: > I've run into this kind of problem with namedtuples before, trying to > access field values with variable names, like: > > label = 'Location' > records.label If you need to access an attribute whose name is computed at run time, you want getattr() and setattr(): value = getattr(record, label) setattr(record, label, value) Using these, I suspect you'll be able to eliminate all the stuff you've got that messes around with indexes to access field values, making the code a lot simpler and easier to follow. -- Greg From greg.ewing at canterbury.ac.nz Fri Feb 24 00:12:52 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 24 Feb 2017 18:12:52 +1300 Subject: Python application launcher (for Python code) In-Reply-To: References: <25be7339-c409-4692-ea0e-848860691b3c@gmail.com> <002001d28e40$fa384ad0$27b23dae@sambora> Message-ID: Deborah Swanson wrote: > As for it phoning home... Wonder what value they think this has, other than > giving them a nosecount of how many active copies there are at any given > time. It gives them a kill switch. If they decide they don't want it to be free any more, they can stop us from using it. "Free" to Microsoft is all about beer, not freedom. -- Greg From python at deborahswanson.net Fri Feb 24 00:27:40 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 21:27:40 -0800 Subject: Namedtuples problem In-Reply-To: Message-ID: <002601d28e5e$b8503560$27b23dae@sambora> Gregory Ewing wrote, on February 23, 2017 9:07 PM > > Deborah Swanson wrote: > > I've run into this kind of problem with namedtuples before, > trying to > > access field values with variable names, like: > > > > label = 'Location' > > records.label > > If you need to access an attribute whose name is computed > at run time, you want getattr() and setattr(): > > value = getattr(record, label) > > setattr(record, label, value) > > Using these, I suspect you'll be able to eliminate all > the stuff you've got that messes around with indexes to > access field values, making the code a lot simpler and > easier to follow. > > -- > Greg Steve D'Aprano mentioned using getattr(record, label) and I thought that might be the way to untangle a lot of the mess I made. Been futzing around with different ways to use it, but I'll look at setattr(record, label, value) first thing in the morning. Thanks bunches! Deborah From python at deborahswanson.net Fri Feb 24 00:59:22 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Thu, 23 Feb 2017 21:59:22 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <002701d28e63$25b45f10$27b23dae@sambora> Gregory Ewing wrote, on February 23, 2017 9:13 PM > > Deborah Swanson wrote: > > As for it phoning home... Wonder what value they think this has, other > > than giving them a nosecount of how many active copies there are at > > any given time. > > It gives them a kill switch. If they decide they don't want > it to be free any more, they can stop us from using it. > > "Free" to Microsoft is all about beer, not freedom. > > -- > Greg Ah, now that you mention it, a kill switch does sound right up Microsoft's alley. Well, they won't kill mine, not if I have my VPN on while I'm using it. Buggers, are we getting to the point that we need to be using VPNs all the time now? It's sorta like sex. Remember what it was like before STDs got so deadly? Or street racing and playing chicken - that'll end you up in jail now, nevermind that the drivers are consenting adults. Not much is done bythely with thoughtless freedom anymore, and we have to protect ourselves from prying gubmint and corporate eyes, ears and wires. Isn't freedom what the USA always used to be about? Makes me sick when they say soldiers are fighting for our freedom in the Middle East. We don't have any freedom or privacy or rights of ownership nowadays for anyone to be fighting to defend. But of course it's all lies now anyway. Deborah From pavol.lisy at gmail.com Fri Feb 24 03:04:22 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Fri, 24 Feb 2017 09:04:22 +0100 Subject: Spyder 3.1.3 update In-Reply-To: References: Message-ID: On 2/24/17, Pablo Lozano wrote: > Good day, > > I installed the Spyder 3.6 IDE from the Anaconda package and at the start k > to the IDE I get the Spyder Update saying Spyder 3.1.3 is available. Did it > incorrectly install Anaconda? As far as I know Spyder 3.1.3 is rather old > and the latest is 3.6 Spyder is spyder and python is python. If you look at ipython console in your spyder window than you probably see something like: Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00) ... 3.1.3 is latest version of spyder (and not very old (2017-02-20)): https://github.com/spyder-ide/spyder/blob/master/CHANGELOG.md From steve+python at pearwood.info Fri Feb 24 04:24:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 24 Feb 2017 20:24:27 +1100 Subject: Disallowing instantiation of super class References: Message-ID: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> On Fri, 24 Feb 2017 11:19 am, Irv Kalb wrote: > Hi, > > I have built a set of three classes: > > - A super class, let's call it: Base [...] > I would like to add is some "insurance" that I (or someone else who uses > my code) never instantiates my Base class, It is not intended to be > instantiated because some of the needed instance variables are only > created in the __init__ method of ClassA and ClassB. class Base: def __init__(self): if type(self) is Base: raise TypeError("cannot instantiate Base class") Does that help? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From wolfgang.maier at biologie.uni-freiburg.de Fri Feb 24 05:21:00 2017 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Fri, 24 Feb 2017 11:21:00 +0100 Subject: Disallowing instantiation of super class In-Reply-To: References: Message-ID: On 24.02.2017 01:19, Irv Kalb wrote: > Hi, > I have built a set of three classes: > - A super class, let's call it: Base > > - A class that inherits from Base, let's call that: ClassA > > - Another class that inherits from Base, let's call that: ClassB > > ClassA and ClassB have some code in their __init__ methods that set some instance variables to different values. After doing so, they call the the __init__ method of their common super class (Base) to set some other instance variables to some common values. This all works great. Instances of ClassA and ClassB do just what I want them to. > [...] > > If I can find a way to determine that the caller is attempting to instantiate Base directly, I will raise an exception. > A pattern I'm using sometimes to achieve this is: class Base: def __init__(self): self.set_common() self.set_specific() def set_common(self): self.a = 10 def set_specific(self): raise NotImplementedError() class A(Base): def set_specific(self): self.b = 20 class B(Base): def set_specific(self): self.b = 30 Of course, MRAB's and Peter's suggestion are very similar ideas. From michelletan123 at gmail.com Fri Feb 24 06:18:19 2017 From: michelletan123 at gmail.com (Michelle Tan) Date: Fri, 24 Feb 2017 19:18:19 +0800 Subject: Error installing python on Windows Message-ID: Hello all I am new to python. Trying to install Python and encountered this error message : "The program can't start because api-ms-win-crt-runtime-I1-1-0.dll is missing from your computer." Tried to repair and reinstall Python however i may have missed out some installation. What should I do to resolve this issue? Thanks for your help! From chololennon at hotmail.com Fri Feb 24 07:32:17 2017 From: chololennon at hotmail.com (Cholo Lennon) Date: Fri, 24 Feb 2017 09:32:17 -0300 Subject: Python application launcher (for Python code) References: <005b01d28bcc$942975a0$27b23dae@sambora> Message-ID: On 02/23/2017 11:15 PM, Dennis Lee Bieber wrote: > On Thu, 23 Feb 2017 12:58:24 -0300, Cholo Lennon > declaimed the following: > > >> If the script has no screen input/output I'd edit the shortcut and >> prefix the script pathname with pythonw.exe (to avoid the empty >> console). Also I'd add the python path if it is not in the user/system >> PATH or if you have multiple python installations) >> > If a normal Windows install, just change the extension to .pyw -- the > normal association is for .pyw to use pythonw.exe > Ahhh... it's good to know :-) thanks > > C:\Users\Wulfraed>assoc .pyw > .pyw=Python.NoConFile > > C:\Users\Wulfraed>ftype python.noconfile > python.noconfile="C:\Python27\pythonw.exe" "%1" %* > > C:\Users\Wulfraed> > -- Cholo Lennon Bs.As. ARG From c.c.wood at gmail.com Fri Feb 24 07:38:22 2017 From: c.c.wood at gmail.com (ChrisW) Date: Fri, 24 Feb 2017 04:38:22 -0800 (PST) Subject: Python 3.6 installation doesn't add launcher to PATH Message-ID: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> The installation guidelines for Python 3.6 say: "Per-user installations of Python do not add the launcher to PATH unless the option was selected on installation." (https://docs.python.org/3/using/windows.html#from-the-command-line). However, I've installed Python 3.6 with the 'include PATH' checkbox ticked for my user only, and although C:\Windows\py.exe exists, it has not been added to my PATH. I also tried installing for all users, and this also doesn't add it to the PATH. Is this a bug, or have I done something wrong?! From kelvan at ist-total.org Fri Feb 24 10:08:27 2017 From: kelvan at ist-total.org (Florian Schweikert) Date: Fri, 24 Feb 2017 16:08:27 +0100 Subject: Error installing python on Windows In-Reply-To: References: Message-ID: <7d705eb1-849b-04a6-cae7-d417b580824d@ist-total.org> Hi Michelle, On 24/02/17 12:18, Michelle Tan wrote: > I am new to python. Welcome to Python. > Trying to install Python What Python version are you trying to install and what's your Windows version? -- Florian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 862 bytes Desc: OpenPGP digital signature URL: From rgacote at appropriatesolutions.com Fri Feb 24 11:42:33 2017 From: rgacote at appropriatesolutions.com (Ray Cote) Date: Fri, 24 Feb 2017 11:42:33 -0500 Subject: python3.5m vs libpython3.5m on CentOS6 Message-ID: Hello: Attempting to install latest psycopg2 (2.6.2) on current CentOS6 with a custom install Python3.5. Getting the error: /usr/bin/ld: cannot find -lpython3.5m Don?t have a python3.5m library on the system. Do have libpython3.5m.so.1.0 with a soft link to libpython3.5m.so. Thought adding a new python3.5m.so softlink to libpython3.5m.so.1.0 and re-running ldconfig would do the trick, but still get the same error. Any hints appreciated. ?Ray From irmen.NOSPAM at xs4all.nl Fri Feb 24 13:08:50 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Fri, 24 Feb 2017 19:08:50 +0100 Subject: Python 3.6 installation doesn't add launcher to PATH In-Reply-To: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> References: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> Message-ID: <58b076b1$0$740$e4fe514c@news.xs4all.nl> On 24-2-2017 13:38, ChrisW wrote: > The installation guidelines for Python 3.6 say: > > "Per-user installations of Python do not add the launcher to PATH unless the option was selected on installation." (https://docs.python.org/3/using/windows.html#from-the-command-line). > > However, I've installed Python 3.6 with the 'include PATH' checkbox ticked for my user only, and although C:\Windows\py.exe exists, it has not been added to my PATH. > > I also tried installing for all users, and this also doesn't add it to the PATH. > > Is this a bug, or have I done something wrong?! > Hm, I don't remember having this issue when installing this myself. Are you sure you're checking it from a newly opened cmd.exe *after* the install finished? Already opened windows won't see the change until restarted. Irmen From irmen.NOSPAM at xs4all.nl Fri Feb 24 13:12:44 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Fri, 24 Feb 2017 19:12:44 +0100 Subject: Error installing python on Windows In-Reply-To: References: Message-ID: <58b0779b$0$740$e4fe514c@news.xs4all.nl> On 24-2-2017 12:18, Michelle Tan wrote: > Hello all > > I am new to python. > > Trying to install Python and encountered this error message : "The program > can't start because api-ms-win-crt-runtime-I1-1-0.dll is missing from your > computer." > > Tried to repair and reinstall Python however i may have missed out some > installation. What should I do to resolve this issue? Thanks for your > help! > Make sure your windows is up to date! That dll should be installed as part of one of the regular windows updates. Google may be able to tell you the specific KB number of the particular update. Irmen From none at invalid.com Fri Feb 24 13:31:03 2017 From: none at invalid.com (mm0fmf) Date: Fri, 24 Feb 2017 18:31:03 +0000 Subject: Error installing python on Windows In-Reply-To: <58b0779b$0$740$e4fe514c@news.xs4all.nl> References: <58b0779b$0$740$e4fe514c@news.xs4all.nl> Message-ID: On 24/02/2017 18:12, Irmen de Jong wrote: > On 24-2-2017 12:18, Michelle Tan wrote: >> Hello all >> >> I am new to python. >> >> Trying to install Python and encountered this error message : "The program >> can't start because api-ms-win-crt-runtime-I1-1-0.dll is missing from your >> computer." >> >> Tried to repair and reinstall Python however i may have missed out some >> installation. What should I do to resolve this issue? Thanks for your >> help! >> > > Make sure your windows is up to date! That dll should be installed as part of one of the > regular windows updates. Google may be able to tell you the specific KB number of the > particular update. > > > Irmen > It always surprises me that people don't seem to cut & paste the error message into a search engine (such as Google) to see if it's a common issue. https://www.google.ch/search?q=api-ms-win-crt-runtime-I1-1-0.dll+is+missing That search returns a variety of pages explaining what the error is and how to fix it and took me under 15secs to get plenty of help. That is always going to be quicker than a newsgroup. I'd always search for initial help on error messages etc. on a search engine and come to places like this to discuss the problem in depth and detail if there was no help or I didn't understand what the answers meant. KB2999226 is the update that is missing. From python at mrabarnett.plus.com Fri Feb 24 13:40:38 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 24 Feb 2017 18:40:38 +0000 Subject: Disallowing instantiation of super class In-Reply-To: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2017-02-24 09:24, Steve D'Aprano wrote: > On Fri, 24 Feb 2017 11:19 am, Irv Kalb wrote: > >> Hi, >> >> I have built a set of three classes: >> >> - A super class, let's call it: Base > [...] >> I would like to add is some "insurance" that I (or someone else who uses >> my code) never instantiates my Base class, It is not intended to be >> instantiated because some of the needed instance variables are only >> created in the __init__ method of ClassA and ClassB. > > class Base: > def __init__(self): > if type(self) is Base: > raise TypeError("cannot instantiate Base class") > > > Does that help? > D'oh! Never thought of that! :-) The OP is using Python 2.7, so you'll need to use new-style classes: class Base(object): def __init__(self): if type(self) is Base: raise TypeError("cannot instantiate Base class") From kar6308 at gmail.com Fri Feb 24 13:54:07 2017 From: kar6308 at gmail.com (kar6308 at gmail.com) Date: Fri, 24 Feb 2017 10:54:07 -0800 (PST) Subject: Python replace multiple strings (m*n) combination Message-ID: I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. for example is the search string is not a variable we can say re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... Pseudo code for string in genobj_of_million_strings: pattern = re.compile('|'.join(regex_map.keys())) return pattern.sub(lambda x: regex_map[x], string) From python at mrabarnett.plus.com Fri Feb 24 14:47:51 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 24 Feb 2017 19:47:51 +0000 Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: Message-ID: <3f33cc14-93a8-0164-b218-4b6b16a4571b@mrabarnett.plus.com> On 2017-02-24 18:54, kar6308 at gmail.com wrote: > I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. > > What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. > > for example is the search string is not a variable we can say > > re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... > > Pseudo code > > for string in genobj_of_million_strings: > pattern = re.compile('|'.join(regex_map.keys())) > return pattern.sub(lambda x: regex_map[x], string) > Here's an example: import re # A dict of the replacements. mapping = {'one': 'unu', 'two': 'du', 'three': 'tri', 'four': 'kvar', 'five': 'kvin'} # The text that we're searching. text = 'one two three four five six seven eight nine ten' # It's best to put the strings we're looking for into reverse order in # case one of the keys is the prefix of another. ordered_keys = sorted(mapping.keys(), reverse=True) ordered_values = [mapping[key] for key in ordered_keys] # Build the pattern, putting each key in its own group. # I'm assuming that the keys are all pure literals, that they don't # contain anything that's treated specially by regex. You could escape # the key (using re.escape(...)) if that's not the case. pattern = re.compile('|'.join('(%s)' % key for key in ordered_keys)) # When we find a match, the match object's .lastindex attribute will # tell us which group (i.e. key) matched. We can then look up the # replacement. We also need to take into account that the groups are # numbered from 1, whereas list items are numbered from 0. new_text = pattern.sub(lambda m: ordered_values[m.lastindex - 1], text) It might be faster (timing it would be a good idea) if you could put all of the rows into a single string (or a number of rows into a single srting), process that string, and then split up the result. If none of the rows contain '\n', then you could join them together with that, otherwise just pick some other character. From python at lucidity.plus.com Fri Feb 24 16:44:16 2017 From: python at lucidity.plus.com (Erik) Date: Fri, 24 Feb 2017 21:44:16 +0000 Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: Message-ID: <081ae044-8b1c-3cf1-a9a5-57fe6f1545d5@lucidity.plus.com> On 24/02/17 18:54, kar6308 at gmail.com wrote: > for example is the search string is not a variable we can say > > re.search(r"\$%^search_text", "replace_text", "some_text") but when I > read from the dict where shd I place the "r" keyword, unfortunately > putting inside key doesnt work "r key" like this.... Do you mean the 'r' that is in front of the first argument to re.search()? If not, ignore this response ;) If so, then you need to understand what "raw string" literals are. They are a construct that the *parser* understands in order to build a string based on what is between the quotation marks. The prefix 'r' on a string literal just means that certain characters (such as '\') will not be treated specially in the way they are for regular string literals. In your case, r"\$%search_text" is the same as "\\$%search_text". In the second case, because it's a regular string literal where the backslash is important, the backslash needs to be escaped (with a leading backslash) - otherwise the parser thinks that backslash is itself an escape character. You can try all of this at the Python REPL shell: Python 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> r"\$%search_text" '\\$%search_text' >>> "\\$%search_text" '\\$%search_text' The two string objects that exist at runtime (which are the things printed out) are the same regardless of how you express them in the source code. Therefore, you don't need to do anything with 'r' inside your key - if it's not expressed in Python source code, then the "raw" prefix does not come into it. Regards, E. From kar6308 at gmail.com Fri Feb 24 17:18:13 2017 From: kar6308 at gmail.com (kar) Date: Fri, 24 Feb 2017 14:18:13 -0800 (PST) Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: <3f33cc14-93a8-0164-b218-4b6b16a4571b@mrabarnett.plus.com> Message-ID: <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> On Friday, February 24, 2017 at 11:48:22 AM UTC-8, MRAB wrote: > On 2017-02-24 18:54, kar6308 at gmail.com wrote: > > I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. > > > > What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. > > > > for example is the search string is not a variable we can say > > > > re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... > > > > Pseudo code > > > > for string in genobj_of_million_strings: > > pattern = re.compile('|'.join(regex_map.keys())) > > return pattern.sub(lambda x: regex_map[x], string) > > > Here's an example: > > import re > > # A dict of the replacements. > mapping = {'one': 'unu', 'two': 'du', 'three': 'tri', 'four': 'kvar', > 'five': 'kvin'} > > # The text that we're searching. > text = 'one two three four five six seven eight nine ten' > > # It's best to put the strings we're looking for into reverse order in > # case one of the keys is the prefix of another. > ordered_keys = sorted(mapping.keys(), reverse=True) > ordered_values = [mapping[key] for key in ordered_keys] > > # Build the pattern, putting each key in its own group. > # I'm assuming that the keys are all pure literals, that they don't > # contain anything that's treated specially by regex. You could escape > # the key (using re.escape(...)) if that's not the case. > pattern = re.compile('|'.join('(%s)' % key for key in ordered_keys)) > > # When we find a match, the match object's .lastindex attribute will > # tell us which group (i.e. key) matched. We can then look up the > # replacement. We also need to take into account that the groups are > # numbered from 1, whereas list items are numbered from 0. > new_text = pattern.sub(lambda m: ordered_values[m.lastindex - 1], text) > > > It might be faster (timing it would be a good idea) if you could put all > of the rows into a single string (or a number of rows into a single > srting), process that string, and then split up the result. If none of > the rows contain '\n', then you could join them together with that, > otherwise just pick some other character. Thanks, what is the idea behind storing the keys and values in a list, I assume looking up for a value in a map is faster getting the value from the list. Also I like the idea of combining multiple rows into one string and passing it. I would batch up the million rows in to strings and give it a shot. From steve+python at pearwood.info Fri Feb 24 17:35:10 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 25 Feb 2017 09:35:10 +1100 Subject: Disallowing instantiation of super class References: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <58b0b51f$0$22141$c3e8da3$5496439d@news.astraweb.com> On Sat, 25 Feb 2017 05:40 am, MRAB wrote: >> class Base: >> def __init__(self): >> if type(self) is Base: >> raise TypeError("cannot instantiate Base class") >> >> >> Does that help? >> > D'oh! Never thought of that! :-) > > The OP is using Python 2.7, so you'll need to use new-style classes: > > class Base(object): > def __init__(self): > if type(self) is Base: > raise TypeError("cannot instantiate Base class") For "classic classes", you can use this: if self.__class__ is Base: raise TypeError("cannot instantiate Base class") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From python at mrabarnett.plus.com Fri Feb 24 17:36:18 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 24 Feb 2017 22:36:18 +0000 Subject: Python replace multiple strings (m*n) combination In-Reply-To: <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> References: <3f33cc14-93a8-0164-b218-4b6b16a4571b@mrabarnett.plus.com> <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> Message-ID: On 2017-02-24 22:18, kar wrote: > On Friday, February 24, 2017 at 11:48:22 AM UTC-8, MRAB wrote: >> On 2017-02-24 18:54, kar6308 at gmail.com wrote: >> > I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. >> > >> > What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. >> > >> > for example is the search string is not a variable we can say >> > >> > re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... >> > >> > Pseudo code >> > >> > for string in genobj_of_million_strings: >> > pattern = re.compile('|'.join(regex_map.keys())) >> > return pattern.sub(lambda x: regex_map[x], string) >> > >> Here's an example: >> >> import re >> >> # A dict of the replacements. >> mapping = {'one': 'unu', 'two': 'du', 'three': 'tri', 'four': 'kvar', >> 'five': 'kvin'} >> >> # The text that we're searching. >> text = 'one two three four five six seven eight nine ten' >> >> # It's best to put the strings we're looking for into reverse order in >> # case one of the keys is the prefix of another. >> ordered_keys = sorted(mapping.keys(), reverse=True) >> ordered_values = [mapping[key] for key in ordered_keys] >> >> # Build the pattern, putting each key in its own group. >> # I'm assuming that the keys are all pure literals, that they don't >> # contain anything that's treated specially by regex. You could escape >> # the key (using re.escape(...)) if that's not the case. >> pattern = re.compile('|'.join('(%s)' % key for key in ordered_keys)) >> >> # When we find a match, the match object's .lastindex attribute will >> # tell us which group (i.e. key) matched. We can then look up the >> # replacement. We also need to take into account that the groups are >> # numbered from 1, whereas list items are numbered from 0. >> new_text = pattern.sub(lambda m: ordered_values[m.lastindex - 1], text) >> >> >> It might be faster (timing it would be a good idea) if you could put all >> of the rows into a single string (or a number of rows into a single >> srting), process that string, and then split up the result. If none of >> the rows contain '\n', then you could join them together with that, >> otherwise just pick some other character. > > Thanks, what is the idea behind storing the keys and values in a list, I assume looking up for a value in a map is faster getting the value from the list. > If you're only looking for _exact_ literal matches, then you can use a dict because you know exactly what it'll find, but if you're ignoring the case, or are going to, say, look for any digits using \d, and, therefore, don't know exactly what it'll find (might be uppercase, might be lowercase, might be any digit), then this technique is better. I'm just hedging my bets! > Also I like the idea of combining multiple rows into one string and passing it. I would batch up the million rows in to strings and give it a shot. > If your computer has enough memory for it, then yes. From python at lucidity.plus.com Fri Feb 24 17:41:39 2017 From: python at lucidity.plus.com (Erik) Date: Fri, 24 Feb 2017 22:41:39 +0000 Subject: Python replace multiple strings (m*n) combination In-Reply-To: <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> References: <3f33cc14-93a8-0164-b218-4b6b16a4571b@mrabarnett.plus.com> <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> Message-ID: <23b1fc07-18b7-5754-0161-3f5b65634225@lucidity.plus.com> On 24/02/17 22:18, kar wrote: > Thanks, what is the idea behind storing the keys and values in a list, I assume looking up for a value in a map is faster getting the value from the list. What do you not understand? MRAB's extensive comments explain what is being done and why. In summary, the keys and values are put into lists so that the keys can be given an explicit order for the re.compile() and then the subsequent lookup when a match is found is via a numeric index (which is even faster for a list than a key lookup is for a dict (map)). E. From kar6308 at gmail.com Fri Feb 24 17:57:32 2017 From: kar6308 at gmail.com (kar6308 at gmail.com) Date: Fri, 24 Feb 2017 14:57:32 -0800 (PST) Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: <3f33cc14-93a8-0164-b218-4b6b16a4571b@mrabarnett.plus.com> <94166cde-062b-4dba-bee2-a83dae41eaf4@googlegroups.com> <23b1fc07-18b7-5754-0161-3f5b65634225@lucidity.plus.com> Message-ID: <53584650-7285-4ac7-9b74-f5bf858e7cf6@googlegroups.com> On Friday, February 24, 2017 at 2:41:58 PM UTC-8, Erik wrote: > On 24/02/17 22:18, kar wrote: > > Thanks, what is the idea behind storing the keys and values in a list, I assume looking up for a value in a map is faster getting the value from the list. > > What do you not understand? MRAB's extensive comments explain what is > being done and why. > > In summary, the keys and values are put into lists so that the keys can > be given an explicit order for the re.compile() and then the subsequent > lookup when a match is found is via a numeric index (which is even > faster for a list than a key lookup is for a dict (map)). > > E. I was trying to understand the performance aspect, so idx lookup is faster than map, got it. From Irv at furrypants.com Fri Feb 24 18:31:56 2017 From: Irv at furrypants.com (Irv Kalb) Date: Fri, 24 Feb 2017 15:31:56 -0800 Subject: Disallowing instantiation of super class In-Reply-To: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <58affbcb$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <9C88EAFF-52C9-4779-B511-74DD0BA0EC57@furrypants.com> > On Feb 24, 2017, at 1:24 AM, Steve D'Aprano wrote: > > On Fri, 24 Feb 2017 11:19 am, Irv Kalb wrote: > >> Hi, >> >> I have built a set of three classes: >> >> - A super class, let's call it: Base > [...] >> I would like to add is some "insurance" that I (or someone else who uses >> my code) never instantiates my Base class, It is not intended to be >> instantiated because some of the needed instance variables are only >> created in the __init__ method of ClassA and ClassB. > > class Base: > def __init__(self): > if type(self) is Base: > raise TypeError("cannot instantiate Base class") > > > Does that help? > > Double D'oh!! That's exactly what I was looking for. (Why didn't I think of that?) Thanks very much! Irv From greg.ewing at canterbury.ac.nz Sat Feb 25 00:08:32 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 25 Feb 2017 18:08:32 +1300 Subject: Python application launcher (for Python code) In-Reply-To: References: <002701d28e63$25b45f10$27b23dae@sambora> Message-ID: Deborah Swanson wrote: > Well, they won't kill mine, not if I have my VPN on while I'm using it. How will a VPN help? If it needs to phone home and perform a secret handshake before it will run, and they turn off the secret handshake server, you're hosed. -- Greg From rosuav at gmail.com Sat Feb 25 01:04:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 25 Feb 2017 17:04:24 +1100 Subject: Python application launcher (for Python code) In-Reply-To: References: <002701d28e63$25b45f10$27b23dae@sambora> Message-ID: On Sat, Feb 25, 2017 at 4:08 PM, Gregory Ewing wrote: > Deborah Swanson wrote: >> >> Well, they won't kill mine, not if I have my VPN on while I'm using it. > > > How will a VPN help? If it needs to phone home and perform a > secret handshake before it will run, and they turn off the secret > handshake server, you're hosed. It'll help because you'll know that that's what's going on (because blocking it from phoning home will insta-kill the program). If blocking it DOESN'T stop it from running, you know you're safe. A useful trick for verifying what you already strongly suspect. ChrisA From python at deborahswanson.net Sat Feb 25 01:26:15 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Fri, 24 Feb 2017 22:26:15 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <00bd01d28f30$119d7660$27b23dae@sambora> Gregory Ewing wrote, on February 24, 2017 9:09 PM > > Deborah Swanson wrote: > > Well, they won't kill mine, not if I have my VPN on while I'm using > > it. > > How will a VPN help? If it needs to phone home and perform a > secret handshake before it will run, and they turn off the > secret handshake server, you're hosed. > > -- > Greg Well rats. Skull duggery on the net is a lot more sophisticated than when I saw it last (15 years ago). Getting up to speed with it is on my list, but quite aways down. Best strategy is to only plan on using it a few times and then uninstall. Maybe restore that install from a drive backup if I need it again, then overwrite with an up to date backup. There's only one C library I want to rewrite parts of in Python, and if I get it right the first time I probably won't have need of Visual Studio again. I really just want to see the execution path through a very weirdly constructed sub path in the C Library. Once I have that I can get the rest from the code. From python at deborahswanson.net Sat Feb 25 01:29:28 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Fri, 24 Feb 2017 22:29:28 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <00be01d28f30$84e4e040$27b23dae@sambora> Chris Angelico wrote, February 24, 2017 10:04 PM > > On Sat, Feb 25, 2017 at 4:08 PM, Gregory Ewing > wrote: > > Deborah Swanson wrote: > >> > >> Well, they won't kill mine, not if I have my VPN on while > I'm using > >> it. > > > > > > How will a VPN help? If it needs to phone home and perform a secret > > handshake before it will run, and they turn off the secret > handshake > > server, you're hosed. > > It'll help because you'll know that that's what's going on > (because blocking it from phoning home will insta-kill the > program). If blocking it DOESN'T stop it from running, you > know you're safe. > > A useful trick for verifying what you already strongly suspect. > > ChrisA There's still something creepy about the phoning home. I'd rather catch it between phone calls if possible. Maybe turning off all internet access (pulling the wire) while using it? From eryksun at gmail.com Sat Feb 25 02:20:30 2017 From: eryksun at gmail.com (eryk sun) Date: Sat, 25 Feb 2017 07:20:30 +0000 Subject: Python 3.6 installation doesn't add launcher to PATH In-Reply-To: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> References: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> Message-ID: On Fri, Feb 24, 2017 at 12:38 PM, ChrisW wrote: > However, I've installed Python 3.6 with the 'include PATH' checkbox ticked > for my user only, and although C:\Windows\py.exe exists, it has not been > added to my PATH. > > I also tried installing for all users, and this also doesn't add it to the PATH. > > Is this a bug, or have I done something wrong?! What the installer adds to PATH is the Python36[-32] directory that has python.exe and the Scripts directory that has pip.exe. The recommended way to run Python is via the "python" command not "py". The py launcher is mainly for shell executing script files with shebang processing. That said people do use py -X[-Y[-32]] as a convenient way to run multiple versions of Python without modifying PATH. As to adding SystemRoot to PATH, it should be there already. This doesn't matter if the program lets CreateProcess search because it always searches system directories before PATH. But it matters in a common case; cmd.exe does its own search and only searches PATH (and also the current directory if the system is configured insecurely). From breamoreboy at gmail.com Sat Feb 25 07:49:13 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 25 Feb 2017 04:49:13 -0800 (PST) Subject: Python application launcher (for Python code) In-Reply-To: References: <25be7339-c409-4692-ea0e-848860691b3c@gmail.com> <002001d28e40$fa384ad0$27b23dae@sambora> Message-ID: <4e18242f-66a5-4d02-ae7c-b7cdc3e897d3@googlegroups.com> On Friday, February 24, 2017 at 1:54:39 AM UTC, Deborah Swanson wrote: > Michael Torrie wrote, on February 23, 2017 7:43 AM > > > > On 2017-02-22 09:49 PM, Deborah Swanson wrote: > > > Didn't even look. Visual Studio has always been pricey, and it never > > > > occurred to me that they might have a free or cheap version now. > > > > You can get the full edition of Visual Studio, called Visual Studio > > Community Edition for free. They still offer Visual Studio Express, > but > > I think they recommend the full community edition to most people now. > > The biggest downside to the VS Community Edition is that it has to > phone > > home and log in to MS's developer web site from time to time to stay > > active. Sigh. MS almost gets it, but not quite. > > Another free version of Visual Studio, wonders never cease! > > As for it phoning home, I won't use it for long, and then I might not > ever use it again. Wonder what value they think this has, other than > giving them a nosecount of how many active copies there are at any given > time. As an alternative to Visual Studio Community Edition, which takes forever and a day to download and install, you might like to give Visual Studio Code a try https://code.visualstudio.com/ Kindest regards. Mark Lawrence. From sntc99 at gmail.com Sat Feb 25 12:19:48 2017 From: sntc99 at gmail.com (Santhosh Kumar) Date: Sat, 25 Feb 2017 22:49:48 +0530 Subject: any one used moviepy please come in!!! I need help, thanks! In-Reply-To: References: <5ec6d36f-5200-4882-98ce-c317deab35b3@googlegroups.com> <12117181-ec3b-4dd6-9f58-7b0c81f0ae20@googlegroups.com> Message-ID: Thanks a lot , I got this solution after installing ghostscript Regards sandy On Thu, Feb 23, 2017 at 6:15 PM, Billy Earney wrote: > imagemagick (ie, convert) has a policy.xml file that stops you from > accessing any resource that starts with '@'. > > type 'identify -list policy' > the first line should tell you where your policy.xml file is located. My > policy.xml file is lcoated at /etc/ImageMagick-6/policy.xml > > open that file, and goto to the end and comment out (or remove the line > that reads) > > > > since this is xml, you can comment out this line by appending the line > with > > Hope this helps! > > On Thu, Feb 23, 2017 at 1:05 AM, wrote: > >> On Thursday, January 26, 2017 at 8:12:24 AM UTC+5:30, Tony Chen wrote: >> > On Wednesday, January 25, 2017 at 8:34:01 PM UTC+13, Chris Angelico >> wrote: >> > > On Wed, Jan 25, 2017 at 6:22 PM, Tony Chen < >> tonytonytony0724 at gmail.com> wrote: >> > > > This error can be due to the fact that ImageMagick is not installed >> on your computer, or (for Windows users) that you didn't specify the path >> to the ImageMagick binary in file conf.py, or.that the path you specified >> is incorrect >> > > >> > > So... is ImageMagick installed? >> > > >> > > ChrisA >> > >> > Okay... Yes it's installed. This problem has been solved haha. Thank >> you for replying. >> >> Hi Tony, >> same issue facing by me. kindly help me >> please list the solution >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > From python at deborahswanson.net Sat Feb 25 13:03:45 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sat, 25 Feb 2017 10:03:45 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <005801d28f91$8247a910$27b23dae@sambora> Dennis Lee Bieber wrote, on February 25, 2017 4:50 AM > > On Fri, 24 Feb 2017 22:26:15 -0800, "Deborah Swanson" > declaimed the following: > > > > >Well rats. Skull duggery on the net is a lot more sophisticated than > >when I saw it last (15 years ago). Getting up to speed with it is on my > >list, but quite aways down. Best strategy is to only plan on using it > >a few times and then uninstall. Maybe restore that install from a drive > >backup if I need it again, then overwrite with an up to date backup. > > > Probably won't help... There's likely some cryptic > registry entry encoding the date of last "phone home" (and > maybe number of times started since that date too). If it's > by date, you'd have to fake the date on the computer (I > wouldn't be surprised if the first thing it does is ensure > NTP sync is on, and run a time-server sync). If, in > association, there is a use counter, you'd have to restore an > earlier value of the registry. > > And I'm quite certain M$ uninstall won't remove the > registry entries of usage, just so a reinstall (or restore > from backup) can't bypass the system. > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ Didn't think of that one either, but it's easy to export the involved keys right after installing and restore them before each use. I like the idea of pulling the wire (cutting internet access) when I use it. Then nobody (including the app) is getting in or out to look at or do anything while the app is running. I won't use it much or often. From songofacandy at gmail.com Sat Feb 25 14:08:51 2017 From: songofacandy at gmail.com (INADA Naoki) Date: Sun, 26 Feb 2017 04:08:51 +0900 Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: Message-ID: If you can use third party library, I think you can use Aho-Corasick algorithm. https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm https://pypi.python.org/pypi/pyahocorasick/ On Sat, Feb 25, 2017 at 3:54 AM, wrote: > I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. > > What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. > > for example is the search string is not a variable we can say > > re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... > > Pseudo code > > for string in genobj_of_million_strings: > pattern = re.compile('|'.join(regex_map.keys())) > return pattern.sub(lambda x: regex_map[x], string) > -- > https://mail.python.org/mailman/listinfo/python-list From cfkaran2 at gmail.com Sat Feb 25 15:13:14 2017 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 25 Feb 2017 15:13:14 -0500 Subject: Python replace multiple strings (m*n) combination In-Reply-To: References: Message-ID: <4A537885-0A18-464A-A7FB-BC43578A4D6B@gmail.com> Another possibility is to form a suffix array (https://en.wikipedia.org/wiki/Suffix_array#Applications) as an index for the string, and then search for patterns within the suffix array. The basic idea is that you index the string you're searching over once, and then look for patterns within it. The main problem with this method is how you're doing the replacements. If your replacement text can create a new string that matches a different regex that occurs later on, then you really should use what INADA Naoki suggested. Thanks, Cem Karan On Feb 25, 2017, at 2:08 PM, INADA Naoki wrote: > If you can use third party library, I think you can use Aho-Corasick algorithm. > > https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm > > https://pypi.python.org/pypi/pyahocorasick/ > > On Sat, Feb 25, 2017 at 3:54 AM, wrote: >> I have a task to search for multiple patterns in incoming string and replace with matched patterns, I'm storing all pattern as keys in dict and replacements as values, I'm using regex for compiling all the pattern and using the sub method on pattern object for replacement. But the problem I have a tens of millions of rows, that I need to check for pattern which is about 1000 and this is turns out to be a very expensive operation. >> >> What can be done to optimize it. Also I have special characters for matching, where can I specify raw string combinations. >> >> for example is the search string is not a variable we can say >> >> re.search(r"\$%^search_text", "replace_text", "some_text") but when I read from the dict where shd I place the "r" keyword, unfortunately putting inside key doesnt work "r key" like this.... >> >> Pseudo code >> >> for string in genobj_of_million_strings: >> pattern = re.compile('|'.join(regex_map.keys())) >> return pattern.sub(lambda x: regex_map[x], string) >> -- >> https://mail.python.org/mailman/listinfo/python-list > -- > https://mail.python.org/mailman/listinfo/python-list From python at deborahswanson.net Sat Feb 25 16:19:37 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sat, 25 Feb 2017 13:19:37 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <8sr3bc1k4okti5456q0f74q7mlhmt5ib6h@4ax.com> Message-ID: <001d01d28fac$dede6770$27b23dae@sambora> > -----Original Message----- > From: Python-list > [mailto:python-list-bounces+python=deborahswanson.net at python.o > rg] On Behalf Of Dennis Lee Bieber > Sent: Saturday, February 25, 2017 12:58 PM > To: python-list at python.org > Subject: Re: Python application launcher (for Python code) > > > On Sat, 25 Feb 2017 10:03:45 -0800, "Deborah Swanson" > declaimed the following: > > > >Didn't think of that one either, but it's easy to export the > involved > >keys right after installing and restore them before each use. > > > If you can find them -- might be something hidden under > one of those cryptic-looking GUID format. Oh, I'm an old hand at finding Microsoft GUIDs. As a tester at Microsoft, when a nightly build's uninstall failed (which was often because the setup team was so crappy), we had to find and delete every blasted registry key and installed file for the application. I had 50+ servers in my lab, so I identified all the keys and files on one server and sent the delete operations out over the network to the others. This method was laborious, but a fraction of the work needed to wipe and fresh install 50+ servers. > Might have to export the entire registry first, > install, export entire registry, and run a differences of the two > > >I like the idea of pulling the wire (cutting internet access) when I > >use it. Then nobody (including the app) is getting in or out to look at > >or do anything while the app is running. I won't use it much > or often. > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ From wanderer at dialup4less.com Sat Feb 25 17:06:37 2017 From: wanderer at dialup4less.com (Wanderer) Date: Sat, 25 Feb 2017 14:06:37 -0800 (PST) Subject: Python application launcher (for Python code) In-Reply-To: References: <005801d28f91$8247a910$27b23dae@sambora> Message-ID: On Saturday, February 25, 2017 at 1:03:40 PM UTC-5, Deborah Swanson wrote: > Dennis Lee Bieber wrote, on February 25, 2017 4:50 AM > > > > On Fri, 24 Feb 2017 22:26:15 -0800, "Deborah Swanson" > > declaimed the following: > > > > > > > >Well rats. Skull duggery on the net is a lot more sophisticated than > > >when I saw it last (15 years ago). Getting up to speed with it is on > my > > >list, but quite aways down. Best strategy is to only plan on using > it > > >a few times and then uninstall. Maybe restore that install from a > drive > > >backup if I need it again, then overwrite with an up to date backup. > > > > > Probably won't help... There's likely some cryptic > > registry entry encoding the date of last "phone home" (and > > maybe number of times started since that date too). If it's > > by date, you'd have to fake the date on the computer (I > > wouldn't be surprised if the first thing it does is ensure > > NTP sync is on, and run a time-server sync). If, in > > association, there is a use counter, you'd have to restore an > > earlier value of the registry. > > > > And I'm quite certain M$ uninstall won't remove the > > registry entries of usage, just so a reinstall (or restore > > from backup) can't bypass the system. > > > > -- > > Wulfraed Dennis Lee Bieber AF6VN > > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > Didn't think of that one either, but it's easy to export the involved > keys right after installing and restore them before each use. > > I like the idea of pulling the wire (cutting internet access) when I use > it. Then nobody (including the app) is getting in or out to look at or > do anything while the app is running. I won't use it much or often. If you want to turn internet access on and off more easily in Windows 7, go to Control Panel\ Network and Sharing Center and in the left hand menu click on Change Adapter Settings. Create a shortcut to Local Area Connect and put it on your desktop. Local Area Connect has the internet enable and disable button. I also use Networx to keep track of who is phoning home. From python at deborahswanson.net Sat Feb 25 17:44:37 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sat, 25 Feb 2017 14:44:37 -0800 Subject: Python application launcher (for Python code) In-Reply-To: Message-ID: <002a01d28fb8$bec08c50$27b23dae@sambora> Wanderer wrote, on February 25, 2017 2:07 PM > > If you want to turn internet access on and off more easily in > Windows 7, go to Control Panel\ Network and Sharing Center > and in the left hand menu click on Change Adapter Settings. > Create a shortcut to Local Area Connect and put it on your > desktop. Local Area Connect has the internet enable and > disable button. I also use Networx to keep track of who is > phoning home. Oh, "pulling the wire" and seeing who's trying to call home is quick and easy enough, in a number of different ways. I don't have Win7 nor want it, but as this thread evolves, perhaps someone else reading this will find your information useful. From python at deborahswanson.net Sun Feb 26 02:02:50 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Sat, 25 Feb 2017 23:02:50 -0800 Subject: Python application launcher (for Python code) In-Reply-To: <4e18242f-66a5-4d02-ae7c-b7cdc3e897d3@googlegroups.com> Message-ID: <003101d28ffe$580d56a0$27b23dae@sambora> breamoreboy at gmail.com wrote, on February 25, 2017 4:49 AM > > On Friday, February 24, 2017 at 1:54:39 AM UTC, Deborah Swanson wrote: > > Michael Torrie wrote, on February 23, 2017 7:43 AM > > > > > > On 2017-02-22 09:49 PM, Deborah Swanson wrote: > > > > Didn't even look. Visual Studio has always been pricey, and it > > > > never > > > > > > occurred to me that they might have a free or cheap version now. > > > > > > You can get the full edition of Visual Studio, called > Visual Studio > > > Community Edition for free. They still offer Visual > Studio Express, > > but > > > I think they recommend the full community edition to most people > > > now. > > > The biggest downside to the VS Community Edition is that it has to > > phone > > > home and log in to MS's developer web site from time to > time to stay > > > active. Sigh. MS almost gets it, but not quite. > > > > Another free version of Visual Studio, wonders never cease! > > > > As for it phoning home, I won't use it for long, and then I > might not > > ever use it again. Wonder what value they think this has, > other than > > giving them a nosecount of how many active copies there are at any > > given time. > > As an alternative to Visual Studio Community Edition, which > takes forever and a day to download and install, you might > like to give Visual Studio Code a try https://code.visualstudio.com/ > > Kindest regards. > > Mark Lawrence. Thanks Mark, I will take a look at it. It sounds like a real pain to keep on top of all the invasions from Microsoft and who knows else, but likely that's all set up in Visual Studio Community's .NET installation code. I don't/won't have any .NET after about version 3 on my computers, so it just occurs that I may not even be able/willing to install it. Later versions of dotNet not only phone home, but can also serve as a backdoor to the NSA and other government nasties. That's one reason why they're all so keen on killing XP, because anything before SP3 can't be instrumented in that way, and a major reason why I'm sticking with XP SP2 until I can get back on Linux. And on top of that, now you say Visual Studio Community Edition takes a long time to download and install, even if you're willing to live with all that comes with it (I'm not). I almost have a verifiably working version in Python of the funcionality I want from that C library, so I may just bag Visual Studio. It would be nice to see what calculations they're exactly doing in that C library because the authors are authorities in this area, but the body of calculations is all written up in other sources. Once I have a Python version I can verify against another dataset I have and it checks with the written sources, I may not care whether I have that C library to compare to or not. But if I really want to see all the execution steps in the C code, going with Visual Studio Code is probably the way to go. Heck, I thought it would be a shortcut to download the open source C code and model my code after it, but it's turned into a rats nest of trouble, which likely won't tell me anything I don't already know at this point. Deborah From best_lay at yahoo.com Sun Feb 26 02:16:51 2017 From: best_lay at yahoo.com (Wildman) Date: Sun, 26 Feb 2017 01:16:51 -0600 Subject: Problem With Tkinter Font Message-ID: Python 3.4.2 Tkinter 8.6 Linux I want to set the font in a GUI program I am working on. Here is the pertinent code I am using... from tkinter import font myfont = font.Font(family='Helvetica', size=10, weight='bold') Here is the error I get... Traceback (most recent call last): File "./test.py", line 41, in myfont = font.Font(family='Helvetica', size=10, weight="bold") File "/usr/lib/python3.4/tkinter/font.py", line 93, in __init__ tk.call("font", "create", self.name, *font) AttributeError: 'NoneType' object has no attribute 'call' >From my research, the syntax is correct but I am having doubts. Can anyone clarify? -- GNU/Linux user #557453 The cow died so I don't need your bull! From __peter__ at web.de Sun Feb 26 03:17:00 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 26 Feb 2017 09:17 +0100 Subject: Problem With Tkinter Font References: Message-ID: Wildman via Python-list wrote: > Python 3.4.2 > Tkinter 8.6 > Linux > > I want to set the font in a GUI program I am working on. > Here is the pertinent code I am using... > > from tkinter import font > > myfont = font.Font(family='Helvetica', size=10, weight='bold') > > Here is the error I get... > > Traceback (most recent call last): > File "./test.py", line 41, in > myfont = font.Font(family='Helvetica', size=10, weight="bold") > File "/usr/lib/python3.4/tkinter/font.py", line 93, in __init__ > tk.call("font", "create", self.name, *font) > AttributeError: 'NoneType' object has no attribute 'call' > > From my research, the syntax is correct but I am having > doubts. Can anyone clarify? If you do not provide the root argument there is still an implicit dependency: """ class Font: ... def __init__(self, root=None, font=None, name=None, exists=False, **options): if not root: root = tkinter._default_root tk = getattr(root, 'tk', root) """ >>> import tkinter >>> from tkinter import font >>> font.Font(family="Helpvetica", size=10, weight="bold") Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.4/tkinter/font.py", line 93, in __init__ tk.call("font", "create", self.name, *font) AttributeError: 'NoneType' object has no attribute 'call' >>> root = tkinter.Tk() >>> font.Font(family="Helpvetica", size=10, weight="bold") So you have to create the main window before you can create a Font. From michael.gauthier.uni at gmail.com Sun Feb 26 09:13:30 2017 From: michael.gauthier.uni at gmail.com (michael.gauthier.uni at gmail.com) Date: Sun, 26 Feb 2017 06:13:30 -0800 (PST) Subject: Help debugging code - Negative lookahead problem Message-ID: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> Hi everyone, So here is my problem. I have a bunch of tweets and various metadata that I want to analyze for sociolinguistic purposes. In order to do this, I'm trying to infer users' ages thanks to the information they provide in their bio, among others. For that I'm using regular expressions to match a couple of recurring patterns in users' bio, like a user mentioning a number followed by various spellings of "years old" as in: "John, 30 years old, engineer." The reason why I'm using regexes for this is that there are actually very few ways people use to mention there age on Twitter, so just three or four regexes would allow me to infer the age of most users in my dataset. However, in this case I also want to check for what comes after "years old", as many people mention their children's age, and I don't want this to be incorrectly associated to the user's age, as in: "John, father of a 12 year old kid, engineer" So cases as the one above should be ignored, so that I can only keep users for whom a valid age can be inferred. My program looks like this: import csv import re with open("test_corpus.csv") as corpus: corpus_read = csv.reader(corpus, delimiter=",") for row in corpus_read: if re.findall(r"\d{2}\s?(?=years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year old\s?(?!son|daughter|kid|child))",row[5].lower()): age = re.findall(r"\d{2}\s?",row[5].lower()) for i in age: print(i) The program seems to work in some cases, but in the small test file I created to try it out, it incorrectly matches the age mentioned in the string "I have a 12 yo son", and returns 12 as a matched age, which I don't want it to. I'm guessing this has something to do with brackets or delimiters at some point in the program, but I spent a few days on it, and I could not find anything helpful around here or on other forums, so any help would be appreciated. Thus, the actual question is: how to make the program not recognize 12 in "John, father of a 12 year old kid, engineer" as the age of the user, based on the program I already have? I am somewhat new at programming, so apologies if I forgot to mention something important, do not hesitate to tell me if you need more details. Thanks in advance for any help you could provide! From c.c.wood at gmail.com Sun Feb 26 10:00:36 2017 From: c.c.wood at gmail.com (ChrisW) Date: Sun, 26 Feb 2017 07:00:36 -0800 (PST) Subject: Python 3.6 installation doesn't add launcher to PATH In-Reply-To: <58b076b1$0$740$e4fe514c@news.xs4all.nl> References: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> <58b076b1$0$740$e4fe514c@news.xs4all.nl> Message-ID: On Friday, 24 February 2017 18:09:01 UTC, Irmen de Jong wrote: > On 24-2-2017 13:38, ChrisW wrote: > > The installation guidelines for Python 3.6 say: > > > > "Per-user installations of Python do not add the launcher to PATH unless the option was selected on installation." (https://docs.python.org/3/using/windows.html#from-the-command-line). > > > > However, I've installed Python 3.6 with the 'include PATH' checkbox ticked for my user only, and although C:\Windows\py.exe exists, it has not been added to my PATH. > > > > I also tried installing for all users, and this also doesn't add it to the PATH. > > > > Is this a bug, or have I done something wrong?! > > > > Hm, I don't remember having this issue when installing this myself. > > Are you sure you're checking it from a newly opened cmd.exe *after* the install > finished? Already opened windows won't see the change until restarted. > > Irmen Yeh - it definitely still happens with a new cmd window... From c.c.wood at gmail.com Sun Feb 26 10:09:33 2017 From: c.c.wood at gmail.com (ChrisW) Date: Sun, 26 Feb 2017 07:09:33 -0800 (PST) Subject: Python 3.6 installation doesn't add launcher to PATH In-Reply-To: References: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> Message-ID: On Saturday, 25 February 2017 07:21:30 UTC, eryk sun wrote: > On Fri, Feb 24, 2017 at 12:38 PM, ChrisW wrote: > > However, I've installed Python 3.6 with the 'include PATH' checkbox ticked > > for my user only, and although C:\Windows\py.exe exists, it has not been > > added to my PATH. > > > > I also tried installing for all users, and this also doesn't add it to the PATH. > > > > Is this a bug, or have I done something wrong?! > > What the installer adds to PATH is the Python36[-32] directory that > has python.exe and the Scripts directory that has pip.exe. The > recommended way to run Python is via the "python" command not "py". > The py launcher is mainly for shell executing script files with > shebang processing. That said people do use py -X[-Y[-32]] as a > convenient way to run multiple versions of Python without modifying > PATH. > > As to adding SystemRoot to PATH, it should be there already. This > doesn't matter if the program lets CreateProcess search because it > always searches system directories before PATH. But it matters in a > common case; cmd.exe does its own search and only searches PATH (and > also the current directory if the system is configured insecurely). Hmm, that seems to contradict what the docs say: `py` is recommended for Windows users who want to run various versions (not just for executing scripts). SystemRoot is not in my PATH variable. From jonas at wielicki.name Sun Feb 26 10:30:35 2017 From: jonas at wielicki.name (Jonas Wielicki) Date: Sun, 26 Feb 2017 16:30:35 +0100 Subject: [ANN] aioxmpp 0.8 released Message-ID: <6383807.RxgDzUcQAG@sinistra> Dear subscribers, I am pleased to announce the release of aioxmpp 0.8. The current release can be obtained from GitHub [1] (check out the v0.8.0 tag or the master branch) or PyPI [2]. The HTML documentation can be found at [3]. Examples can be found in the GitHub repository, in the examples sub directory. aioxmpp is a Python library based on asyncio. It implements the client side of the XMPP protocol (RFC 6120 and others). For a more detailed description of the package, please review the README of the package on GitHub [1] or on PyPI [2]. For more information on XMPP, please see [8]. aioxmpp is licensed under the terms of the GNU Lesser General Public License Version 3.0 or later. Version 0.8 is a feature release. Major features include: * Support for executing Ad-Hoc Commands [XEP-0050] with peers. * Easier creation of new protocol services (aioxmpp.service.Service subclasses) using decorators stanza handlers and other niceties. * End-to-end/Integration tests against XMPP servers are now supported. Many of the other changes were under the hood. Please refer to the changelog for this release [9] for details and breaking changes compared to the previous release. Bugs, feature requests, patches and questions can be directed to either the aioxmpp mailing list [4], the GitHub issue tracker [5] or the XMPP Multi-User chat [6], whatever floats your boat. Please direct security-relevant issue reports directly to me (jonas at wielicki.name), preferably encrypted using my GPG public key [7]. best regards and happy-asyncio-ing, Jonas Wielicki [1]: https://github.com/horazont/aioxmpp [2]: https://pypi.python.org/pypi/aioxmpp [3]: https://docs.zombofant.net/aioxmpp/0.8/ [4]: https://lists.zombofant.net/mailman/listinfo/aioxmpp-devel [5]: https://github.com/horazont/aioxmpp/issues [6]: aioxmpp at conference.zombofant.net [7]: https://sks-keyservers.net/pks/lookup?op=get&search=0xE5EDE5AC679E300F AA5A 78FF 508D 8CF4 F355 F682 E5ED E5AC 679E 300F [8]: https://xmpp.org/ [9]: https://docs.zombofant.net/aioxmpp/0.8/api/changelog.html#version-0-8 [XEP-0050]: https://xmpp.org/extensions/xep-0050.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part. URL: From best_lay at yahoo.com Sun Feb 26 10:35:54 2017 From: best_lay at yahoo.com (Wildman) Date: Sun, 26 Feb 2017 09:35:54 -0600 Subject: Problem With Tkinter Font References: Message-ID: On Sun, 26 Feb 2017 09:17:00 +0100, Peter Otten wrote: > Wildman via Python-list wrote: > >> Python 3.4.2 >> Tkinter 8.6 >> Linux >> >> I want to set the font in a GUI program I am working on. >> Here is the pertinent code I am using... >> >> from tkinter import font >> >> myfont = font.Font(family='Helvetica', size=10, weight='bold') >> >> Here is the error I get... >> >> Traceback (most recent call last): >> File "./test.py", line 41, in >> myfont = font.Font(family='Helvetica', size=10, weight="bold") >> File "/usr/lib/python3.4/tkinter/font.py", line 93, in __init__ >> tk.call("font", "create", self.name, *font) >> AttributeError: 'NoneType' object has no attribute 'call' >> >> From my research, the syntax is correct but I am having >> doubts. Can anyone clarify? > > If you do not provide the root argument there is still an implicit > dependency: > > """ > class Font: > ... > def __init__(self, root=None, font=None, name=None, exists=False, > **options): > if not root: > root = tkinter._default_root > tk = getattr(root, 'tk', root) > > """ > >>>> import tkinter >>>> from tkinter import font >>>> font.Font(family="Helpvetica", size=10, weight="bold") > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3.4/tkinter/font.py", line 93, in __init__ > tk.call("font", "create", self.name, *font) > AttributeError: 'NoneType' object has no attribute 'call' >>>> root = tkinter.Tk() >>>> font.Font(family="Helpvetica", size=10, weight="bold") > > > So you have to create the main window before you can create a Font. OK, that makes sense. I knew I was doing something dumb or in this case, not doing it. Thank you. -- GNU/Linux user #557453 The cow died so I don't need your bull! From python at mrabarnett.plus.com Sun Feb 26 11:15:59 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 26 Feb 2017 16:15:59 +0000 Subject: Help debugging code - Negative lookahead problem In-Reply-To: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> References: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> Message-ID: <4b6a8296-062b-c71f-0526-b71d0497bcab@mrabarnett.plus.com> On 2017-02-26 14:13, michael.gauthier.uni at gmail.com wrote: > Hi everyone, > > So here is my problem. I have a bunch of tweets and various metadata that I want to analyze for sociolinguistic purposes. In order to do this, I'm trying to infer users' ages thanks to the information they provide in their bio, among others. For that I'm using regular expressions to match a couple of recurring patterns in users' bio, like a user mentioning a number followed by various spellings of "years old" as in: > > "John, 30 years old, engineer." > > The reason why I'm using regexes for this is that there are actually very few ways people use to mention there age on Twitter, so just three or four regexes would allow me to infer the age of most users in my dataset. However, in this case I also want to check for what comes after "years old", as many people mention their children's age, and I don't want this to be incorrectly associated to the user's age, as in: > > "John, father of a 12 year old kid, engineer" > > > So cases as the one above should be ignored, so that I can only keep users for whom a valid age can be inferred. > My program looks like this: > > > import csv > import re > > with open("test_corpus.csv") as corpus: > corpus_read = csv.reader(corpus, delimiter=",") > for row in corpus_read: > if re.findall(r"\d{2}\s?(?=years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year old\s?(?!son|daughter|kid|child))",row[5].lower()): > age = re.findall(r"\d{2}\s?",row[5].lower()) > for i in age: > print(i) > > > The program seems to work in some cases, but in the small test file I created to try it out, it incorrectly matches the age mentioned in the string "I have a 12 yo son", and returns 12 as a matched age, which I don't want it to. I'm guessing this has something to do with brackets or delimiters at some point in the program, but I spent a few days on it, and I could not find anything helpful around here or on other forums, so any help would be appreciated. > > Thus, the actual question is: how to make the program not recognize 12 in "John, father of a 12 year old kid, engineer" as the age of the user, based on the program I already have? > > I am somewhat new at programming, so apologies if I forgot to mention something important, do not hesitate to tell me if you need more details. > > Thanks in advance for any help you could provide! > Remove some of the alternatives and it should become clearer: r"\d{2}\s?(?=years old\s?|year old\s?(?!son|daughter|kid|child))" The negative lookahead is in the last alternative and applies only to that one. Adding a non-capture group (?:...) around the alternatives: r"\d{2}\s?(?=(?:years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year old\s?)(?!son|daughter|kid|child))" Another point: the \d{2} will match only 2 digits and there's nothing to prevent it from matching the end of a series of digits, so but if the person was, say, 101 years old, it would match "01". Try matching 2 or more digits with \d{2,}. From michael.gauthier.uni at gmail.com Sun Feb 26 12:15:58 2017 From: michael.gauthier.uni at gmail.com (michael.gauthier.uni at gmail.com) Date: Sun, 26 Feb 2017 09:15:58 -0800 (PST) Subject: Help debugging code - Negative lookahead problem In-Reply-To: References: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> <4b6a8296-062b-c71f-0526-b71d0497bcab@mrabarnett.plus.com> Message-ID: Hi MRAB, Thanks for taking time to look at my problem! I tried your solution: r"\d{2}\s?(?=(?:years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year old\s?)(?!son|daughter|kid|child))" but unfortunately it does seem not work. Also, I tried adding the negative lookaheads after every one of the alternatives, but it does not work either, so the problem does not seem to be that the negative lookahead applies only to the last proposition... : ( Also, \d{2} will only match two single digits, and won't match the last two digits of 101, so at least this is fine! : ) Any other idea to improve that code? I'm starting to get desperate... Thanks again for your help anyways, I really appreciate it! ; ) From eryksun at gmail.com Sun Feb 26 12:27:11 2017 From: eryksun at gmail.com (eryk sun) Date: Sun, 26 Feb 2017 17:27:11 +0000 Subject: Python 3.6 installation doesn't add launcher to PATH In-Reply-To: References: <7e7e73a3-7a7a-42c6-a7c5-0473a79ef16d@googlegroups.com> Message-ID: On Sun, Feb 26, 2017 at 3:09 PM, ChrisW wrote: > On Saturday, 25 February 2017 07:21:30 UTC, eryk sun wrote: >> On Fri, Feb 24, 2017 at 12:38 PM, ChrisW wrote: >> > However, I've installed Python 3.6 with the 'include PATH' checkbox ticked >> > for my user only, and although C:\Windows\py.exe exists, it has not been >> > added to my PATH. >> > >> > I also tried installing for all users, and this also doesn't add it to the PATH. >> > >> > Is this a bug, or have I done something wrong?! >> >> What the installer adds to PATH is the Python36[-32] directory that >> has python.exe and the Scripts directory that has pip.exe. The >> recommended way to run Python is via the "python" command not "py". >> The py launcher is mainly for shell executing script files with >> shebang processing. That said people do use py -X[-Y[-32]] as a >> convenient way to run multiple versions of Python without modifying >> PATH. >> >> As to adding SystemRoot to PATH, it should be there already. This >> doesn't matter if the program lets CreateProcess search because it >> always searches system directories before PATH. But it matters in a >> common case; cmd.exe does its own search and only searches PATH (and >> also the current directory if the system is configured insecurely). > > Hmm, that seems to contradict what the docs say: `py` is recommended for > Windows users who want to run various versions (not just for executing scripts). "python" is the recommended way to run Python that's cross-platform. The last thing I'd want is to see tutorials and FAQs filled with caveats that Windows users should use py -X[.Y[-32]] instead of python. It can be assumed that the environment is configured correctly for the python command to run the desired version of the interpreter. The best way to do that is to develop using virtual environments. Steve Dower (Microsoft) and Nick Coghlan (Red Hat) have made suggestions for a new "python" launcher that can run multiple versions and would be available on all platforms. I've yet to see any specific discussion about this idea on python-ideas or python-dev -- at least not to the point that someone is willing to write a PEP and start coding. It's just a pipe dream for now. > SystemRoot is not in my PATH variable. Open the environment variables editor in the system properties window; double-click on "Path" in the system variables and insert the following at the beginning: %SystemRoot%\System32;%SystemRoot%;%SystemRoot%\System32\Wbem; These are present by default when Windows is installed -- except %SystemRoot% may already be expanded to C:\Windows, or wherever Windows is installed on the system. Someone or some program messed up if they're missing. From __peter__ at web.de Sun Feb 26 13:57:22 2017 From: __peter__ at web.de (Peter Otten) Date: Sun, 26 Feb 2017 19:57:22 +0100 Subject: Help debugging code - Negative lookahead problem References: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> <4b6a8296-062b-c71f-0526-b71d0497bcab@mrabarnett.plus.com> Message-ID: michael.gauthier.uni at gmail.com wrote: > Hi MRAB, > > Thanks for taking time to look at my problem! > > I tried your solution: > > r"\d{2}\s?(?=(?:years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year > old\s?)(?!son|daughter|kid|child))" > > but unfortunately it does seem not work. Also, I tried adding the negative > lookaheads after every one of the alternatives, but it does not work > either, so the problem does not seem to be that the negative lookahead > applies only to the last proposition... : ( > > Also, \d{2} will only match two single digits, and won't match the last > two digits of 101, so at least this is fine! : ) > > Any other idea to improve that code? I'm starting to get desperate... If your code becomes too complex to manage it break it into simpler parts. In this case you can use two simple regular expressions: >>> age = re.compile(r"\d+") >>> child = re.compile(r"\s+kid") >>> text = "42 bar baz foo 12 kid" >>> for candidate in age.finditer(text): ... if child.match(text, candidate.end()): ... print("Kid's age:", candidate.group()) ... else: ... print("Author's age:", candidate.group()) ... Author's age: 42 Kid's age: 12 Applying that idea (and the principle to break everything into dead easy parts) to your problem: $ cat demo.py import re def longest_first(text): return sorted(text.splitlines(), key=len, reverse=True) YEARS = longest_first("""\ year years year old years old yo ys o """) CHILDREN = longest_first("""\ son daughter kid child """) YEARS_RE = r"\b(?P\d+) ({})".format("|".join(YEARS)) re_years = re.compile(YEARS_RE) CHILD_RE = r" ({})\b".format("|".join(CHILDREN)) re_child = re.compile(CHILD_RE) def followed_by_child(candidate): return re_child.match(candidate.string, candidate.end()) CORPUS = """\ jester, 42 years old, 20 years kidding 12 years kid engineer, 30 years engineer, 30 years old daughter """.splitlines() for text in CORPUS: print(text) for m in re_years.finditer(text): age = m.group("age") if followed_by_child(m): print(" rejected:", age) else: print(" accepted:", age) $ python3 demo.py jester, 42 years old, 20 years kidding accepted: 42 accepted: 20 12 years kid rejected: 12 engineer, 30 years accepted: 30 engineer, 30 years old daughter rejected: 30 From python at mrabarnett.plus.com Sun Feb 26 14:07:10 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 26 Feb 2017 19:07:10 +0000 Subject: Help debugging code - Negative lookahead problem In-Reply-To: References: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> <4b6a8296-062b-c71f-0526-b71d0497bcab@mrabarnett.plus.com> Message-ID: On 2017-02-26 17:15, michael.gauthier.uni at gmail.com wrote: > Hi MRAB, > > Thanks for taking time to look at my problem! > > I tried your solution: > > r"\d{2}\s?(?=(?:years old\s?|yo\s?|yr old\s?|y o\s?|yrs old\s?|year > old\s?)(?!son|daughter|kid|child))" > > but unfortunately it does seem not work. Also, I tried adding the negative lookaheads after every one of the alternatives, but it does not work either, so the problem does not seem to be that the negative lookahead applies only to > the last proposition... : ( > > Also, \d{2} will only match two single digits, and won't match the last two digits of 101, so at least this is fine! : ) > > Any other idea to improve that code? I'm starting to get desperate... > > Thanks again for your help anyways, I really appreciate it! ; ) > Ah, OK. I see what the problem is. (I should've marked it as "untested". :-() It matches r"yo\s?" against "yo " (the r"\s?" consumes the space) and then the "son" alternative against "son", but that's a _negative_ lookahead, so it _fails_, so it backtracks. It retries the r"\s?", which now matches an empty string (doesn't consume the space), and then the "son" alternative against " son", which fails, but that's a _negative_ lookahead, so it _succeeds_. And the regex as a whole matches. Ideally I'd want to use a possessive quantifier or atomic group, but they aren't supported by the re module, so the workaround is to move the check for whitespace: r"\d{2}\s?(?=(?:years old|yo|yr old|y o|yrs old|year old)(?!\s?son|\s?daughter|\s?kid|\s?child))" From michael.gauthier.uni at gmail.com Sun Feb 26 14:39:39 2017 From: michael.gauthier.uni at gmail.com (michael.gauthier.uni at gmail.com) Date: Sun, 26 Feb 2017 11:39:39 -0800 (PST) Subject: Help debugging code - Negative lookahead problem In-Reply-To: References: <890481df-c56f-4900-ac3a-79fb45fd94e7@googlegroups.com> <4b6a8296-062b-c71f-0526-b71d0497bcab@mrabarnett.plus.com> Message-ID: <7c5cb941-e1c8-4ef9-a790-fe538260d1cf@googlegroups.com> WOW, many thanks guys, Peter, and MRAB, for your time, help, and explanations! Peter, yes, you're right, when things get too complicated I should definitely try to split things up, and thus split the difficulties (ah, Descartes... ^^), thanks for the advice! MRAB, your code is now working, thank you SO MUCH! You just put an end to 4 days of headaches because of that code! And most importantly, thanks for the explanation, I can now understand what was wrong in the code and not reproduce the same mistake, your help is invaluable and I can now go on with the rest of my analysis. Thanks again guys, I posted this question on several forums, and you're the only ones who have been able to actually help me solve the issue, you're the best! ; ) From contact at stridebird.com Sun Feb 26 15:15:43 2017 From: contact at stridebird.com (Pete Dowdell) Date: Sun, 26 Feb 2017 20:15:43 +0000 Subject: Exposing all methods of a class Message-ID: <3afdc9ac-0679-f970-b604-e3367ca9788f@stridebird.com> I use Python, mainly with Django, for work. I was wondering if anyone has encountered an editor that could display a class with all inherited methods included in the editor's view of the class code. I am kind of envisaging the inherited code would be displayed differently (say, grey vs black), and labelled with the name of the parent class so that it would provide a one-stop view of the entire class structure. I edit using vim, and use ctags to explore up the inheritance chain which is useful enough, but I feel it should be quite possible for the editor to search through the imports and grab the parent classes to insert into the editor view. Any suggestions? Maybe there are better ways of navigating inheritance but it does seem logical to expose the whole class code in one place, suitably annotated. I feel a plugin coming on. Ta, Pete From tjreedy at udel.edu Sun Feb 26 17:48:31 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 26 Feb 2017 17:48:31 -0500 Subject: Talk Python to Me with GvR Message-ID: https://player.backtracks.fm/talkpython/m/100-guido-van-rossum Both audio and transcript. There is more discussion of Guido's pre-Python work than I have read before. Discussion of Python 3 migration starts at 37:05. Guido says harder than expected because Python was more popular than he was aware of at the time. -- Terry Jan Reedy From jobmattcon at gmail.com Sun Feb 26 23:08:51 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 26 Feb 2017 20:08:51 -0800 (PST) Subject: dictionary of pickle error when get it in multiprocessing and has name error Message-ID: <160b2708-8594-43e5-9545-fc3b187785f7@googlegroups.com> before cloususerlogin Unexpected error: after map pool ... passwordlist = pickle.load( open( str(currentworkingdirectory) + "\\decryptedsecret.p", "rb" ) ) def processInput(host): try: decrypt_file(sys.argv[1], str(currentworkingdirectory) + "\\encryptedsecret.p", str(currentworkingdirectory) + "\\decryptedsecret.p", 64*1024) if os.path.isfile(str(currentworkingdirectory) + "\\decryptedsecret.p") == True: print("exist file") else: print("not exist file") print(str(currentworkingdirectory) + "\\decryptedsecret.p") HOST = host print(host) print("before cloususerlogin") password = passwordlist["clouduserlogin"] <--------------- Name error print(host) text_file.write(host+"\n") try: print("before recursiveconnect") recursiveconnect(host,"") print("after recursiveconnect") except: print "inner Unexpected error:", sys.exc_info()[0] except: print "Unexpected error:", sys.exc_info()[0] resultstringedge = "" def easy_parallize(): pool = Pool(4) print("after pool") hostlist = [ "192.168.1.1" ] #foo = Foo() #results = pool.apply_async(processInput,args=(hostlist,)) results = pool.map(processInput, hostlist) print("after map pool") cleaned = [x for x in results if not x is None] cleaned = np.asarray(cleaned) pool.close() pool.join() print("cleaned") return cleaned if __name__ == '__main__': currentworkingdirectory = str(os.getcwd()).replace("\\","\\\\") text_file = open(newpath+"\\"+"errorstest.txt", "w") easy_parallize() print("111") #for host in hostlist: #if os.path.isfile(str(currentworkingdirectory) + "\\decryptedsecret.p") == True: #try: #os.remove(str(currentworkingdirectory) + "\\decryptedsecret.p") #except: #print("ignored error") print("here close 1") text_file.close() print("here close 2") text_file10.close() From gandalf at shopzeus.com Mon Feb 27 02:26:23 2017 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Mon, 27 Feb 2017 08:26:23 +0100 Subject: concurrent futures, async futures and await In-Reply-To: References: Message-ID: >> >> Simply because the concurrent future returned by executor.submit does >> not implement __await__ and so it cannot be awaited for. > I get that, but what happens if you try wrapping the executor.submit > call with tornado.gen.convert_yielded as the tornado docs suggest and > as I suggested above? Thanks, it did the right thing. :-) (The tornado docs needs to be updated where it talks about changing normal coroutines to native coroutines.) >> Of course, asyncio should not care if the executor is doing the task in >> a different thread or a different process. All I'm saying is that >> concurrent.futures.Future should implement the __await__ method, and >> asyncio should be able to use it. > I found this in the original PEP at > http://legacy.python.org/dev/peps/pep-3156/#futures: > > """ > In the future (pun intended) we may unify asyncio.Future and > concurrent.futures.Future, e.g. by adding an __iter__() method to the > latter that works with yield from. To prevent accidentally blocking > the event loop by calling e.g. result() on a Future that's not done > yet, the blocking operation may detect that an event loop is active in > the current thread and raise an exception instead. However the current > PEP strives to have no dependencies beyond Python 3.3, so changes to > concurrent.futures.Future are off the table for now. > """ I was not aware of this. Thank you for pointing out. > > Maybe we're now far enough into the future that this could be > reconsidered. In the meantime, the user does have other options: > > 1) For tornado, use tornado.gen.convert_yielded. > > 2) For asyncio, create an asyncio future and link them by setting a > callback on the concurrent future that propagates the result (using > call_soon_threadsafe since the callback may run in another thread). > > 3) Better, don't use concurrent.futures directly in the first place; > instead use https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.AbstractEventLoop.run_in_executor > which runs a function in an Executor and wraps it up via #2 > transparently. That is the best. I have discovered another anomaly: asyncio names this "run_in_executor" but tornado names it "run_on_executor" ( http://www.tornadoweb.org/en/stable/concurrent.html#tornado.concurrent.run_on_executor ) I hope eventually they will become fully compatible. :-) Thanks for your help! Laszlo From __peter__ at web.de Mon Feb 27 03:28:45 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 27 Feb 2017 09:28:45 +0100 Subject: dictionary of pickle error when get it in multiprocessing and has name error References: <160b2708-8594-43e5-9545-fc3b187785f7@googlegroups.com> Message-ID: Ho Yeung Lee wrote: > before cloususerlogin > Unexpected error: > after map pool > > > ... > passwordlist = pickle.load( open( str(currentworkingdirectory) + > "\\decryptedsecret.p", "rb" ) ) According to https://docs.python.org/dev/library/multiprocessing.html#programming-guidelines you cannot use global resources on Windows: """ Explicitly pass resources to child processes On Unix using the fork start method, a child process can make use of a shared resource created in a parent process using a global resource. However, it is better to pass the object as an argument to the constructor for the child process. Apart from making the code (potentially) compatible with Windows and the other start methods this also ensures that as long as the child process is still alive the object will not be garbage collected in the parent process. This might be important if some resource is freed when the object is garbage collected in the parent process. So for instance from multiprocessing import Process, Lock def f(): ... do something using "lock" ... if __name__ == '__main__': lock = Lock() for i in range(10): Process(target=f).start() should be rewritten as from multiprocessing import Process, Lock def f(l): ... do something using "l" ... if __name__ == '__main__': lock = Lock() for i in range(10): Process(target=f, args=(lock,)).start() """ From vincent.vande.vyvre at telenet.be Mon Feb 27 07:57:16 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Mon, 27 Feb 2017 13:57:16 +0100 Subject: Usage of ast. Message-ID: I've this strange error: Python 3.4.3 (default, Nov 17 2016, 01:08:31) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ast >>> l = "print('hello world')" >>> ast.literal_eval(l) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.4/ast.py", line 84, in literal_eval return _convert(node_or_string) File "/usr/lib/python3.4/ast.py", line 83, in _convert raise ValueError('malformed node or string: ' + repr(node)) ValueError: malformed node or string: <_ast.Call object at 0x7fcf955871d0> Is it an import question ? Vincent From rosuav at gmail.com Mon Feb 27 08:09:54 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 00:09:54 +1100 Subject: Usage of ast. In-Reply-To: References: Message-ID: On Mon, Feb 27, 2017 at 11:57 PM, Vincent Vande Vyvre wrote: > I've this strange error: > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import ast >>>> l = "print('hello world')" >>>> ast.literal_eval(l) > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3.4/ast.py", line 84, in literal_eval > return _convert(node_or_string) > File "/usr/lib/python3.4/ast.py", line 83, in _convert > raise ValueError('malformed node or string: ' + repr(node)) > ValueError: malformed node or string: <_ast.Call object at 0x7fcf955871d0> > > > Is it an import question ? The message is a little confusing, but the error comes from the fact that literal_eval permits a very few legal operations, and calling a function isn't one of them. So when you try to evaluate the "literal" that you've given it, you get back an error saying that a Call is 'malformed'. More accurate would be to say that it's not permitted. ChrisA From jussi.piitulainen at helsinki.fi Mon Feb 27 08:19:49 2017 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 27 Feb 2017 15:19:49 +0200 Subject: Usage of ast. References: Message-ID: Vincent Vande Vyvre writes: > I've this strange error: > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> import ast >>>> l = "print('hello world')" >>>> ast.literal_eval(l) > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3.4/ast.py", line 84, in literal_eval > return _convert(node_or_string) > File "/usr/lib/python3.4/ast.py", line 83, in _convert > raise ValueError('malformed node or string: ' + repr(node)) > ValueError: malformed node or string: <_ast.Call object at 0x7fcf955871d0> > > > Is it an import question ? print('hello world') is not a literal. Literals are expressions that somehow stand for themselves. Try help(ast.literal_eval) for a more detailed definition. literal_eval('x') # not ok: x is not a literal literal_eval('"x"') # ok: "x" is a literal From vincent.vande.vyvre at telenet.be Mon Feb 27 08:44:51 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Mon, 27 Feb 2017 14:44:51 +0100 Subject: Usage of ast. In-Reply-To: References: Message-ID: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Le 27/02/17 ? 14:09, Chris Angelico a ?crit : > On Mon, Feb 27, 2017 at 11:57 PM, Vincent Vande Vyvre > wrote: >> I've this strange error: >> >> Python 3.4.3 (default, Nov 17 2016, 01:08:31) >> [GCC 4.8.4] on linux >> Type "help", "copyright", "credits" or "license" for more information. >>>>> import ast >>>>> l = "print('hello world')" >>>>> ast.literal_eval(l) >> Traceback (most recent call last): >> File "", line 1, in >> File "/usr/lib/python3.4/ast.py", line 84, in literal_eval >> return _convert(node_or_string) >> File "/usr/lib/python3.4/ast.py", line 83, in _convert >> raise ValueError('malformed node or string: ' + repr(node)) >> ValueError: malformed node or string: <_ast.Call object at 0x7fcf955871d0> >> >> >> Is it an import question ? > The message is a little confusing, but the error comes from the fact > that literal_eval permits a very few legal operations, and calling a > function isn't one of them. So when you try to evaluate the "literal" > that you've given it, you get back an error saying that a Call is > 'malformed'. More accurate would be to say that it's not permitted. > > ChrisA OK, it's coherent with the secure execution. Thanks Vincent From jon+usenet at unequivocal.eu Mon Feb 27 09:18:29 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 27 Feb 2017 14:18:29 -0000 (UTC) Subject: Usage of ast. References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On 2017-02-27, Vincent Vande Vyvre wrote: > Le 27/02/17 ? 14:09, Chris Angelico a ?crit : >> The message is a little confusing, but the error comes from the fact >> that literal_eval permits a very few legal operations, and calling a >> function isn't one of them. So when you try to evaluate the "literal" >> that you've given it, you get back an error saying that a Call is >> 'malformed'. More accurate would be to say that it's not permitted. > > OK, it's coherent with the secure execution. "execution" isn't really the right way to describe literal_eval(). It isn't an code executor or even an expression evaluator, all it does is turns a Python literal in source form into the value that source represents. You can't do literal_eval("2*3") for example, because that's an expression not a literal. The only reason you can do literal_eval("2+3") (in Python>=3.2) is because of a side effect of the fact that complex number literals are, well, complex ("2+3j"), and the support for +/- wasn't restricted to complex literals only. From rosuav at gmail.com Mon Feb 27 09:18:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 01:18:33 +1100 Subject: Usage of ast. In-Reply-To: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On Tue, Feb 28, 2017 at 12:44 AM, Vincent Vande Vyvre wrote: > OK, it's coherent with the secure execution. > Yep. Here's a PR to make the message a bit clearer, though: https://github.com/python/cpython/pull/340 >>> ast.literal_eval("print('hello world')") Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.7/ast.py", line 87, in literal_eval return _convert(node_or_string) File "/usr/local/lib/python3.7/ast.py", line 85, in _convert raise ValueError('%s not allowed in literal' % type(node).__name__) ValueError: Call not allowed in literal ChrisA From rosuav at gmail.com Mon Feb 27 09:37:22 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 01:37:22 +1100 Subject: Usage of ast. In-Reply-To: References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On Tue, Feb 28, 2017 at 1:18 AM, Jon Ribbens wrote: > On 2017-02-27, Vincent Vande Vyvre wrote: >> Le 27/02/17 ? 14:09, Chris Angelico a ?crit : >>> The message is a little confusing, but the error comes from the fact >>> that literal_eval permits a very few legal operations, and calling a >>> function isn't one of them. So when you try to evaluate the "literal" >>> that you've given it, you get back an error saying that a Call is >>> 'malformed'. More accurate would be to say that it's not permitted. >> >> OK, it's coherent with the secure execution. > > "execution" isn't really the right way to describe literal_eval(). > It isn't an code executor or even an expression evaluator, all it > does is turns a Python literal in source form into the value that > source represents. You can't do literal_eval("2*3") for example, > because that's an expression not a literal. > > The only reason you can do literal_eval("2+3") (in Python>=3.2) > is because of a side effect of the fact that complex number literals > are, well, complex ("2+3j"), and the support for +/- wasn't restricted > to complex literals only. Actually it does execute, as you can see from the source code. The name "literal" is slightly incorrect technically, as it will run some code that isn't actually a literal (nor even covered by constant folding, which "2+3j" is - it's two literals and a binary operator, but constant folded at compile time). >>> ast.literal_eval("[1,2,3]") [1, 2, 3] This isn't a literal, and it's evaluated by literal_eval. It's basically a "safe eval of simple expressions". ChrisA From jon+usenet at unequivocal.eu Mon Feb 27 11:17:58 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 27 Feb 2017 16:17:58 -0000 (UTC) Subject: Usage of ast. References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On 2017-02-27, Chris Angelico wrote: > On Tue, Feb 28, 2017 at 1:18 AM, Jon Ribbens wrote: >> "execution" isn't really the right way to describe literal_eval(). >> It isn't an code executor or even an expression evaluator, all it >> does is turns a Python literal in source form into the value that >> source represents. You can't do literal_eval("2*3") for example, >> because that's an expression not a literal. >> >> The only reason you can do literal_eval("2+3") (in Python>=3.2) >> is because of a side effect of the fact that complex number literals >> are, well, complex ("2+3j"), and the support for +/- wasn't restricted >> to complex literals only. > > Actually it does execute, as you can see from the source code. I'm not sure what you mean by that. I was looking at the source code (and its history) when I wrote my post, and I would be hard pushed to describe it as "executing" anything. > The name "literal" is slightly incorrect technically, as it will run > some code that isn't actually a literal (nor even covered by constant > folding, which "2+3j" is - it's two literals and a binary operator, > but constant folded at compile time). Actually the name is more correct than the code is! It is clear that what it is trying to do is exactly as I described above and as its name indicates. The fact that in Python >= 3.2 it allows binary addition and subtraction of numeric values is an undocumented extension to the specification resulting from a side effect of the implementation. >>>> ast.literal_eval("[1,2,3]") > [1, 2, 3] > > This isn't a literal, and it's evaluated by literal_eval. Actually, that is a literal, in the computer science sense: "a literal is a notation for representing a fixed value in source code". I realise it isn't a literal in the sense of "as defined by section 2.4 of the Python Language Reference", but then perhaps most people would expect the word to have its more generic meaning. Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as it is an example of a "list display", but that form is only a subset of what "list display" means (displays include comprehensions), so it is not true to say that literal_eval() can evaluate displays. If we had to choose a way to describe it, it seems unlikely that there's a better description than "list literal". (See also the "What's new" announcement for Python 3.2 where the addition of set support to literal_eval() was described as adding "set literals". https://docs.python.org/3/whatsnew/3.2.html#ast ) > It's basically a "safe eval of simple expressions". What I was trying to say was that I think that is a very misleading way of describing it. It is not entirely untrue, but it is much closer to being false than it is to being true. It isn't "safe eval", and it is not intended to deal with any expressions. (See also http://bugs.python.org/issue22525 ) From rosuav at gmail.com Mon Feb 27 11:36:07 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 03:36:07 +1100 Subject: Usage of ast. In-Reply-To: References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On Tue, Feb 28, 2017 at 3:17 AM, Jon Ribbens wrote: > On 2017-02-27, Chris Angelico wrote: >> Actually it does execute, as you can see from the source code. > > I'm not sure what you mean by that. I was looking at the source code > (and its history) when I wrote my post, and I would be hard pushed to > describe it as "executing" anything. You could say that it "interprets" the AST. It has exactly the same kind of if/elif tree of "this kind of thing? do this" that a naive and unoptimized language interpreter would use. Does that not count as execution? >>>>> ast.literal_eval("[1,2,3]") >> [1, 2, 3] >> >> This isn't a literal, and it's evaluated by literal_eval. > > Actually, that is a literal, in the computer science sense: "a literal > is a notation for representing a fixed value in source code". > I realise it isn't a literal in the sense of "as defined by section > 2.4 of the Python Language Reference", but then perhaps most people > would expect the word to have its more generic meaning. So what _is_ a literal? Can you have a literal that evaluates to a non-constant, or a non-literal that gets compiled to a constant? Here are examples of both: >>> dis.dis(lambda: [1,2,3]) 1 0 LOAD_CONST 1 (1) 2 LOAD_CONST 2 (2) 4 LOAD_CONST 3 (3) 6 BUILD_LIST 3 8 RETURN_VALUE By your definition, this is a literal, but it most surely does not represent a fixed value. Every time you call this function, it will construct and return a brand new list. >>> dis.dis(lambda x: x in {1,2,3}) 1 0 LOAD_FAST 0 (x) 2 LOAD_CONST 4 (frozenset({1, 2, 3})) 4 COMPARE_OP 6 (in) 6 RETURN_VALUE There's no literal syntax "frozenset({1, 2, 3})", but that got loaded as a constant. > Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as > it is an example of a "list display", but that form is only a subset > of what "list display" means (displays include comprehensions), so > it is not true to say that literal_eval() can evaluate displays. > If we had to choose a way to describe it, it seems unlikely that > there's a better description than "list literal". It's a list display. > (See also the "What's new" announcement for Python 3.2 where the > addition of set support to literal_eval() was described as adding > "set literals". https://docs.python.org/3/whatsnew/3.2.html#ast ) Technically they're called "set display", too. Does it count as a literal if and only if all members are literals? That's a pretty tricky concept to try to bake into a definition, but that's exactly the recursive definition that literal_eval actually uses. >> It's basically a "safe eval of simple expressions". > > What I was trying to say was that I think that is a very misleading > way of describing it. It is not entirely untrue, but it is much closer > to being false than it is to being true. It isn't "safe eval", and it > is not intended to deal with any expressions. > > (See also http://bugs.python.org/issue22525 ) Given the many subtle differences in usage of the word "literal", I'm happy with it using that name. It's not 100% accurate, but it covers the non-pedantic uses of the term. Its docstring most definitely says "expression", though, because there are some non-literal expressions that it DOES evaluate, and for good reason. ChrisA From jon+usenet at unequivocal.eu Mon Feb 27 11:58:29 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 27 Feb 2017 16:58:29 -0000 (UTC) Subject: Usage of ast. References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On 2017-02-27, Chris Angelico wrote: > On Tue, Feb 28, 2017 at 3:17 AM, Jon Ribbens wrote: >> On 2017-02-27, Chris Angelico wrote: >>> Actually it does execute, as you can see from the source code. >> >> I'm not sure what you mean by that. I was looking at the source code >> (and its history) when I wrote my post, and I would be hard pushed to >> describe it as "executing" anything. > > You could say that it "interprets" the AST. It has exactly the same > kind of if/elif tree of "this kind of thing? do this" that a naive and > unoptimized language interpreter would use. Does that not count as > execution? Seeing as most of it is code along the lines of "is this an integer constant? if so the value is that constant": no, I think "execution" is a misleading word to use. >> Actually, that is a literal, in the computer science sense: "a literal >> is a notation for representing a fixed value in source code". >> I realise it isn't a literal in the sense of "as defined by section >> 2.4 of the Python Language Reference", but then perhaps most people >> would expect the word to have its more generic meaning. > > So what _is_ a literal? It's "a notation for representing a fixed value in source code", I just said so right there above. > Can you have a literal that evaluates to a non-constant, or a > non-literal that gets compiled to a constant? Here are examples of both: > >>>> dis.dis(lambda: [1,2,3]) > By your definition, this is a literal, No, it isn't - it's pushing it somewhat to call a function a "value" (especially in the context of when words like "literal" were coined), and it's certainly not a "fixed value" in this sense. (Yes, I know the function itself does not change. That doesn't make it a "fixed value".) > but it most surely does not represent a fixed value. So it isn't a literal "by my definition" after all (which isn't "my" definition by the way). > There's no literal syntax "frozenset({1, 2, 3})", but that got loaded > as a constant. So that isn't a literal either - and it isn't accepted by literal_eval. Jolly good. What's the problem? >> Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as >> it is an example of a "list display", but that form is only a subset >> of what "list display" means (displays include comprehensions), so >> it is not true to say that literal_eval() can evaluate displays. >> If we had to choose a way to describe it, it seems unlikely that >> there's a better description than "list literal". > > It's a list display. I just explained right there in the text you quoted why that is not sufficient as a description. Just repeating part of what I already said back to me and ignoring the rest isn't going to progress things very much. >> (See also the "What's new" announcement for Python 3.2 where the >> addition of set support to literal_eval() was described as adding >> "set literals". https://docs.python.org/3/whatsnew/3.2.html#ast ) > > Technically they're called "set display", too. Does it count as a > literal if and only if all members are literals? That seems to be the unavoidable consequence of the definition, yes. > That's a pretty tricky concept to try to bake into a definition, but > that's exactly the recursive definition that literal_eval actually uses. So the definition is useful and accurate then! > Given the many subtle differences in usage of the word "literal", I'm > happy with it using that name. It's not 100% accurate, but it covers > the non-pedantic uses of the term. You seem to have got turned around and are now arguing in favour of my position and against your own previous position. > Its docstring most definitely says "expression", though, because > there are some non-literal expressions that it DOES evaluate, and > for good reason. ... the good reason being that they are literals. From rosuav at gmail.com Mon Feb 27 12:18:40 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 04:18:40 +1100 Subject: Usage of ast. In-Reply-To: References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On Tue, Feb 28, 2017 at 3:58 AM, Jon Ribbens wrote: > On 2017-02-27, Chris Angelico wrote: >> On Tue, Feb 28, 2017 at 3:17 AM, Jon Ribbens wrote: >>> On 2017-02-27, Chris Angelico wrote: >>>> Actually it does execute, as you can see from the source code. >>> >>> I'm not sure what you mean by that. I was looking at the source code >>> (and its history) when I wrote my post, and I would be hard pushed to >>> describe it as "executing" anything. >> >> You could say that it "interprets" the AST. It has exactly the same >> kind of if/elif tree of "this kind of thing? do this" that a naive and >> unoptimized language interpreter would use. Does that not count as >> execution? > > Seeing as most of it is code along the lines of "is this an integer > constant? if so the value is that constant": no, I think "execution" > is a misleading word to use. That's true of a lot of executors. Doesn't change the fact that it will look at a UnaryOp or BinOp and (if it's [U]Add or [U]Sub) perform the corresponding operation. That's execution. >>> Actually, that is a literal, in the computer science sense: "a literal >>> is a notation for representing a fixed value in source code". >>> I realise it isn't a literal in the sense of "as defined by section >>> 2.4 of the Python Language Reference", but then perhaps most people >>> would expect the word to have its more generic meaning. >> >> So what _is_ a literal? > > It's "a notation for representing a fixed value in source code", > I just said so right there above. > >> Can you have a literal that evaluates to a non-constant, or a >> non-literal that gets compiled to a constant? Here are examples of both: >> >>>>> dis.dis(lambda: [1,2,3]) > >> By your definition, this is a literal, > > No, it isn't - it's pushing it somewhat to call a function a "value" > (especially in the context of when words like "literal" were coined), > and it's certainly not a "fixed value" in this sense. (Yes, I know the > function itself does not change. That doesn't make it a "fixed value".) > >> but it most surely does not represent a fixed value. > > So it isn't a literal "by my definition" after all (which isn't "my" > definition by the way). It's completely acceptable to ast.literal_eval. You said earlier that literal_eval really does evaluate literals, not expressions. So how can this simultaneously be acceptable to literal_eval but not a literal, of literal_eval evaluates literals? (Woodchucks chucking wood come to mind.) >> There's no literal syntax "frozenset({1, 2, 3})", but that got loaded >> as a constant. > > So that isn't a literal either - and it isn't accepted by literal_eval. > Jolly good. What's the problem? According to the disassembly (see my original post), it can be a constant. Not even a literal, yet it's a constant that gets baked into the function's attributes. >>> Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as >>> it is an example of a "list display", but that form is only a subset >>> of what "list display" means (displays include comprehensions), so >>> it is not true to say that literal_eval() can evaluate displays. >>> If we had to choose a way to describe it, it seems unlikely that >>> there's a better description than "list literal". >> >> It's a list display. > > I just explained right there in the text you quoted why that is not > sufficient as a description. Just repeating part of what I already > said back to me and ignoring the rest isn't going to progress things > very much. My point is that it truly IS a list display, despite your concerns. "42" is a subset of the definition of integer literals, but that doesn't stop it from being one. >> That's a pretty tricky concept to try to bake into a definition, but >> that's exactly the recursive definition that literal_eval actually uses. > > So the definition is useful and accurate then! Yet it disagrees with the comp sci definition that you espoused earlier. >> Given the many subtle differences in usage of the word "literal", I'm >> happy with it using that name. It's not 100% accurate, but it covers >> the non-pedantic uses of the term. > > You seem to have got turned around and are now arguing in favour of my > position and against your own previous position. Nope. I still believe that literal_eval isn't precisely evaluating literals in the Python sense, but that it evaluates a subset of expressions that can be safely parsed. >> Its docstring most definitely says "expression", though, because >> there are some non-literal expressions that it DOES evaluate, and >> for good reason. > > ... the good reason being that they are literals. Yeah, but we haven't settled a definition that actually means that they are. ChrisA From formisc at gmail.com Mon Feb 27 15:09:13 2017 From: formisc at gmail.com (Andrew Zyman) Date: Mon, 27 Feb 2017 12:09:13 -0800 (PST) Subject: panda, column access Message-ID: <55a53c18-c7b4-4b88-b3e1-da93d6c50287@googlegroups.com> Hello, i'm trying to understand what is the difference between the below code. And how do i access the column's data without hardcoding the column name in such a way that i'll be able to convert it's values into the list of strings? I appreciate your help. Short version: colnames='hostname' data = pandas.read_csv(input_file, names=colnames) list_data = data.hostname.tolist() or list_data = data['hostname'].tolist() returns the list like - ['hostname','hostname1'] - exactly what i want. but neither of the below versions work as expected (returning the list of strings): list_data = data[0].tolist() -> errors with "return self._getitem_column(key)" list_data = data[colnames].values.tolist() -> the list_data is : [ ['hostname'],['hostname1'...] list_data = data.tolist() -> errors with "AttributeError: 'DataFrame' object has no attribute 'tolist'" More details: [code] file ./Data/domain.csv ./domain.csv: ASCII text, with CRLF line terminators head -3 ./Data/domain.csv sunqq.ux.yyy.zzzz sunww.ux.yyy.zzzz sunee.ux.yyy.zzzz [/code] all work is done in Python 2.7. Working version [code] input_file = r'Data/2domain.csv' colnames = ['hostname'] data = pandas.read_csv(input_file, names=colnames) list_data = data.hostname.tolist() list_data = list(set(list_data)) print list_data [/code] Non working version: [code] input_file = r'Data/2domain.csv' colnames = ['hostname'] data = pandas.read_csv(input_file, names=colnames) #list_data = data.hostname.tolist() # neither list_data = data[0].tolist() list_data = data[colnames].tolist() list_data = data['QUERY'].tolist() list_data = data.tolist() # return the list I expected list_data = list(set(list_data)) print list_data [/code] From formisc at gmail.com Mon Feb 27 15:50:46 2017 From: formisc at gmail.com (Andrew Zyman) Date: Mon, 27 Feb 2017 12:50:46 -0800 (PST) Subject: panda, column access Message-ID: Hello, i'd appreciate an explanation about the differences in the code versions below. I'm trying to get the list of the strings out of the column and avoid hardcoding the column name. cat Data/2domain.csv hostname hostname1 hostname2 ... ... Working version(s): Python2.7: input_file = r'Data/2domain.csv' colnames = ['hostname'] data = pandas.read_csv(input_file, names=colnames,header=0) list_data = data.hostname.tolist() # or #list_data = data['hostname'].tolist() print list_data ['hostname','hostname1','hostname3'....] And confusion is with these versions: colnames = ['hostname'] data = pandas.read_csv(input_file, names=colnames,header=0) list_data = data[colnames].tolist() -AttributeError: 'DataFrame' object has no attribute 'tolist' list_data = data[colnames].get_values().tolist() - [ ['hostname'],['hostname1']..] list_data = data[colnames].get_values() - [ ['hostname'], ['hostname1'] .. ] Thank you AZ [working code] [/working code] [no working code ] [/not working code ] From pavol.lisy at gmail.com Mon Feb 27 16:27:06 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Mon, 27 Feb 2017 22:27:06 +0100 Subject: panda, column access In-Reply-To: <55a53c18-c7b4-4b88-b3e1-da93d6c50287@googlegroups.com> References: <55a53c18-c7b4-4b88-b3e1-da93d6c50287@googlegroups.com> Message-ID: On 2/27/17, Andrew Zyman wrote: > Hello, > i'm trying to understand what is the difference between the below code. > And how do i access the column's data without hardcoding the column name in > such a way that i'll be able to convert it's values into the list of > strings? > > I appreciate your help. > > Short version: > colnames='hostname' I think you meant: colnames = ['hostname'] > data = pandas.read_csv(input_file, names=colnames) > list_data = data.hostname.tolist() or list_data = data['hostname'].tolist() > returns the list like - ['hostname','hostname1'] - exactly what i want. > but neither of the below versions work as expected (returning the list of > strings): > list_data = data[0].tolist() -> errors with "return > self._getitem_column(key)" > list_data = data[colnames].values.tolist() -> the list_data is : [ > ['hostname'],['hostname1'...] > list_data = data.tolist() -> errors with "AttributeError: 'DataFrame' object > has no attribute 'tolist'" What do you want is probably: >>> data[colnames[0]].tolist() or maybe if you don't want to remember colnames: >>> data[data.columns[0]].tolist() Be careful! Type of data[x] depend on what type is x. >>> type(data['hostname']) # -> pandas.core.series.Series >>> type(data[['hostname']]) # -> pandas.core.frame.DataFrame From formisc at gmail.com Mon Feb 27 16:43:05 2017 From: formisc at gmail.com (Andrew Zyman) Date: Mon, 27 Feb 2017 13:43:05 -0800 (PST) Subject: panda, column access In-Reply-To: References: <55a53c18-c7b4-4b88-b3e1-da93d6c50287@googlegroups.com> Message-ID: On Monday, February 27, 2017 at 4:27:23 PM UTC-5, Pavol Lisy wrote: > On 2/27/17, Andrew Zyman wrote: > I think you meant: > colnames = ['hostname'] arh... Exactly. Now the rest is "unfolding". > What do you want is probably: > >>> data[colnames[0]].tolist() this is great and the rest of the code works. but the output of "list_data = data.hostname.tolist()" is the comma separated strings, and "list_data = data[colnames[0]].get_values()" is space separated list. I suspect the answer to my "why" is in your statements below . > or maybe if you don't want to remember colnames: > >>> data[data.columns[0]].tolist() > Be careful! Type of data[x] depend on what type is x. > >>> type(data['hostname']) # -> pandas.core.series.Series > >>> type(data[['hostname']]) # -> pandas.core.frame.DataFrame Thank you Pavol, very helpful. From jon+usenet at unequivocal.eu Mon Feb 27 19:35:52 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 28 Feb 2017 00:35:52 -0000 (UTC) Subject: Usage of ast. References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On 2017-02-27, Chris Angelico wrote: > On Tue, Feb 28, 2017 at 3:58 AM, Jon Ribbens wrote: >> Seeing as most of it is code along the lines of "is this an integer >> constant? if so the value is that constant": no, I think "execution" >> is a misleading word to use. > > That's true of a lot of executors. Doesn't change the fact that it > will look at a UnaryOp or BinOp and (if it's [U]Add or [U]Sub) perform > the corresponding operation. That's execution. Yes. I didn't say the word "executing" was strictly false, just that it is misleading, which it is. >> So it isn't a literal "by my definition" after all (which isn't "my" >> definition by the way). > > It's completely acceptable to ast.literal_eval. Sorry, I must have missed something here. What are you talking about? "lambda: [1,2,3]" is not acceptable input to ast.literal_eval(), it will throw an exception. >>> There's no literal syntax "frozenset({1, 2, 3})", but that got loaded >>> as a constant. >> >> So that isn't a literal either - and it isn't accepted by literal_eval. >> Jolly good. What's the problem? > > According to the disassembly (see my original post), it can be a > constant. Not even a literal, yet it's a constant that gets baked into > the function's attributes. It's not a literal, and literal_eval() doesn't accept it. A conundrum for the ages indeed. >>>> Python doesn't seem to have a word for "[1, 2, 3]", inasmuch as >>>> it is an example of a "list display", but that form is only a subset >>>> of what "list display" means (displays include comprehensions), so >>>> it is not true to say that literal_eval() can evaluate displays. >>>> If we had to choose a way to describe it, it seems unlikely that >>>> there's a better description than "list literal". >>> >>> It's a list display. >> >> I just explained right there in the text you quoted why that is not >> sufficient as a description. Just repeating part of what I already >> said back to me and ignoring the rest isn't going to progress things >> very much. > > My point is that it truly IS a list display, despite your concerns. Yes. As I already said that I already said, I already said that. As I also already said, please stop repeating me back to me. >>> That's a pretty tricky concept to try to bake into a definition, but >>> that's exactly the recursive definition that literal_eval actually uses. >> >> So the definition is useful and accurate then! > > Yet it disagrees with the comp sci definition that you espoused earlier. In what way? > Nope. I still believe that literal_eval isn't precisely evaluating > literals in the Python sense, Yes, that is also what I already said. It's evaluating literals in the comp sci sense (with an undocumented extension in some versions that should not be relied on as it's a side effect of the implementation). > but that it evaluates a subset of expressions that can be safely > parsed. Yes, but as I already said it is such a tiny subset of expressions that even comparing it to that set is utterly misleading. It's like calling the number 7 "a subset of the real numbers". It's technically true ("the best kind of true!") but leads you in completely the wrong direction. >>> Its docstring most definitely says "expression", though, because >>> there are some non-literal expressions that it DOES evaluate, and >>> for good reason. >> >> ... the good reason being that they are literals. > > Yeah, but we haven't settled a definition that actually means that they are. I'm getting a bizarre sense of deja vu in pointing out that I'm repeating myself, but: "a literal is a notation for representing a fixed value in source code". From rosuav at gmail.com Mon Feb 27 20:16:52 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 28 Feb 2017 12:16:52 +1100 Subject: Usage of ast. In-Reply-To: References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On Tue, Feb 28, 2017 at 11:35 AM, Jon Ribbens wrote: > Sorry, I must have missed something here. What are you talking about? > "lambda: [1,2,3]" is not acceptable input to ast.literal_eval(), it > will throw an exception. [1,2,3] is, though. Go read my previous posts. ChrisA From jf_byrnes at comcast.net Mon Feb 27 23:07:45 2017 From: jf_byrnes at comcast.net (Jim) Date: Mon, 27 Feb 2017 22:07:45 -0600 Subject: python-libxdo? Message-ID: I found this module by accident the other day and decided to try to use it to position some windows on my desktop. # Script to open and position program windows import subprocess from xdo import Xdo xdo = Xdo() subprocess.call('firefox') win = xdo.search_windows(winname = 'Mozilla Firefox') xdo.move_window(win, 14, 83) Traceback (most recent call last): File "/home/jfb/MyProgs/Scripts/place_windows.py", line 9, in win = xdo.search_windows(winname = 'Mozilla Firefox') File "/home/jfb/EVs/env/lib/python3.5/site-packages/xdo/__init__.py", line 708, in search_windows search.winname = winname TypeError: bytes or integer address expected instead of str instance I couldn't find any examples on line, so what do I need to give to search_windows so I can pass a window id to move_window? Thanks, Jim From jnash673 at gmail.com Mon Feb 27 23:16:06 2017 From: jnash673 at gmail.com (Jarod Nash) Date: Mon, 27 Feb 2017 23:16:06 -0500 Subject: Python failed... Repeatedly In-Reply-To: References: Message-ID: I have just started into python and this was my first time setting up my own software. I ended up with pycharm and chose the python 3.6 or whatever translator. Every time I run my hello world program I get the message "invalid translator" and I get several pop ups giving me the options of custom set up, repair, or uninstall I have tried custom set up a few times and repair multiple times and even have gotten successful repairs even and next time I hit test it goes right back to where it was. Finally I was given the information for this from selecting the correct options in the pop ups. Does anyone have suggestions and can someone confirm that pycharm uses cpython instead of anaconda or jython ect. Please? Because I am taking a cpython course. P.s. I have tried visual studio, and like 3 other softwares and none of them had a successful download, and I have windows 10 but am looking into ubuntu. From python at mrabarnett.plus.com Mon Feb 27 23:18:02 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 28 Feb 2017 04:18:02 +0000 Subject: python-libxdo? In-Reply-To: References: Message-ID: On 2017-02-28 04:07, Jim wrote: > I found this module by accident the other day and decided to try to use > it to position some windows on my desktop. > > # Script to open and position program windows > > import subprocess > from xdo import Xdo > > xdo = Xdo() > > subprocess.call('firefox') > win = xdo.search_windows(winname = 'Mozilla Firefox') > xdo.move_window(win, 14, 83) > > Traceback (most recent call last): > File "/home/jfb/MyProgs/Scripts/place_windows.py", line 9, in > win = xdo.search_windows(winname = 'Mozilla Firefox') > File "/home/jfb/EVs/env/lib/python3.5/site-packages/xdo/__init__.py", > line 708, in search_windows > search.winname = winname > TypeError: bytes or integer address expected instead of str instance > > I couldn't find any examples on line, so what do I need to give to > search_windows so I can pass a window id to move_window? > > Thanks, Jim > It looks like there are some docs here: https://rshk.github.io/python-libxdo/index.html From tjreedy at udel.edu Tue Feb 28 01:31:05 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 28 Feb 2017 01:31:05 -0500 Subject: Python failed... Repeatedly In-Reply-To: References: Message-ID: On 2/27/2017 11:16 PM, Jarod Nash wrote: > I have just started into python and this was my first time setting up my > own software. I ended up with pycharm and chose the python 3.6 or whatever > translator. Every time I run my hello world program I get the message > "invalid translator" and I get several pop ups giving me the options of This are not a CPython messages or behavior, so must be coming from PyCharm. > custom set up, repair, or uninstall I have tried custom set up a few times > and repair multiple times and even have gotten successful repairs even and > next time I hit test it goes right back to where it was. Finally I was > given the information for this from selecting the correct options in the > pop ups. Does anyone have suggestions and can someone confirm that pycharm > uses cpython instead of anaconda or jython ect. Please? Because I am taking > a cpython course. > > P.s. I have tried visual studio, and like 3 other softwares and none of > them had a successful download, and I have windows 10 but am looking into > ubuntu. I run standard CPython 3.6 installed on Win 10 with the standard Windows installer from python.org and have no such problems. The same has been mostly true for previous python and windows versions for 20 years. I recommend that you start with that and forget PyCharm, etc., until you have Python running on your machine. -- Terry Jan Reedy From pavol.lisy at gmail.com Tue Feb 28 05:05:28 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Tue, 28 Feb 2017 11:05:28 +0100 Subject: python-libxdo? In-Reply-To: References: Message-ID: On 2/28/17, Jim wrote: simplified: > from xdo import Xdo > xdo = Xdo() > win = xdo.search_windows(winname = 'Mozilla Firefox') > File "/home/jfb/EVs/env/lib/python3.5/site-packages/xdo/__init__.py" > TypeError: bytes or integer address expected instead of str instance Example on github seems to use python2 where bytes and str are same type. Because you have python3, you need to write -> win = xdo.search_windows(winname = b'Mozilla Firefox') PS. this reminds me "Talk Python to Me with GvR" thread on this list. :) How to minimize frustration? Could not have ctypes.Structure something like encoding attribute? From jon+usenet at unequivocal.eu Tue Feb 28 06:17:32 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 28 Feb 2017 11:17:32 -0000 (UTC) Subject: Usage of ast. References: <8a83de85-ca2f-5697-f4c0-8ba150bb2b4f@telenet.be> Message-ID: On 2017-02-28, Chris Angelico wrote: > On Tue, Feb 28, 2017 at 11:35 AM, Jon Ribbens wrote: >> Sorry, I must have missed something here. What are you talking about? >> "lambda: [1,2,3]" is not acceptable input to ast.literal_eval(), it >> will throw an exception. > > [1,2,3] is, though. Go read my previous posts. I already did. Perhaps you should too - as well as forgetting what I have said you seem to be forgetting what you have said! "[1,2,3]" is both a literal by "my" definition, and is accepted by literal_eval(). There is no mystery there. You brought up the idea of functions instead, which are neither literals nor accepted by literal_eval() - no mystery there either. From murthy919 at gmail.com Tue Feb 28 08:00:42 2017 From: murthy919 at gmail.com (Murthy Jn) Date: Tue, 28 Feb 2017 05:00:42 -0800 (PST) Subject: Insert blob data from one db to other db in oracle Message-ID: I have the table structure in one db is number,photo,date,name. the same structure is there in other database in other machine. I want to insert table1 data from db1 into table2 in db2 using python script. Some one could please help me in this.. I have tried like below. for fetching rows db connection cur=db.cursor() SqlFth="select no,photo,date,name from table1; cur.execute(SqlFth) res=cur.fetchall() rows=[] for row in res: no,photo,date,name=row rows.append(row) n1= row[0] f1=row[1] d1=row[1] na1=row[2] for inserting to other db: SqlStr="insert into test (no,photo,date,name) values(:n1,:f1,:d1,:na1)" print SqlStr cur.executemany(SqlStr,rows) res1=cur.fetchone() cur.execute("commit") Please help me in this... From murthy919 at gmail.com Tue Feb 28 08:02:43 2017 From: murthy919 at gmail.com (Murthy Jn) Date: Tue, 28 Feb 2017 05:02:43 -0800 (PST) Subject: Insert blob data from one db to other db in oracle In-Reply-To: References: Message-ID: <6590d3ae-ce09-49dd-a458-fcce1a449e64@googlegroups.com> On Tuesday, February 28, 2017 at 6:30:54 PM UTC+5:30, Murthy Jn wrote: > I have the table structure in one db is number,photo,date,name. > the same structure is there in other database in other machine. > > I want to insert table1 data from db1 into table2 in db2 using python script. > > Some one could please help me in this.. I have tried like below. > > for fetching rows > db connection > cur=db.cursor() > SqlFth="select no,photo,date,name from table1; > cur.execute(SqlFth) > res=cur.fetchall() > rows=[] > for row in res: > no,photo,date,name=row > rows.append(row) > n1= row[0] > f1=row[1] > d1=row[1] > na1=row[2] > > for inserting to other db: > > SqlStr="insert into test (no,photo,date,name) values(:n1,:f1,:d1,:na1)" > print SqlStr > cur.executemany(SqlStr,rows) > res1=cur.fetchone() > cur.execute("commit") > > Please help me in this... while executing I am getting following error :: insert into test (no,photo,date,name) values(:n1,:f1,:d1,:na1) Variable_TypeByValue(): unhandled data type cx_Oracle.LOB From bogdan.dima06 at gmail.com Tue Feb 28 08:11:03 2017 From: bogdan.dima06 at gmail.com (bogdan.dima06 at gmail.com) Date: Tue, 28 Feb 2017 05:11:03 -0800 (PST) Subject: pyaudio Message-ID: <128298dd-cd7f-4f3e-975d-67fb00263525@googlegroups.com> I'm using pyaudio library and recieve after I run the code this message Process finished with exit code -1 Here is the whole code: import pyaudio import numpy as np p = pyaudio.PyAudio() volume = 0.5 # range [0.0, 1.0] fs = 44100 # sampling rate, Hz, must be integer duration = 10.0 # in seconds, may be float f = 440.0 # sine frequency, Hz, may be float # generate samples, note conversion to float32 array samples = (np.sin(2*np.pi*np.arange(fs*duration)*f/fs)).astype(np.float32) # for paFloat32 sample values must be in range [-1.0, 1.0] stream = p.open(format=pyaudio.paFloat32, channels=1, rate=fs, output=True) # play. May repeat with different volume values (if done interactively) stream.write(volume*samples) stream.stop_stream() stream.close() From __peter__ at web.de Tue Feb 28 09:01:35 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 28 Feb 2017 15:01:35 +0100 Subject: pyaudio References: <128298dd-cd7f-4f3e-975d-67fb00263525@googlegroups.com> Message-ID: bogdan.dima06 at gmail.com wrote: > I'm using pyaudio library and recieve after I run the code this message > > > Process finished with exit code -1 How did you invoke the script? When you run it from the command line, do you get a traceback? Once you have such a traceback please show it to us (use cut and paste text, not a screenshot), together with information about the operating system etc. > Here is the whole code: > > import pyaudio > import numpy as np > > p = pyaudio.PyAudio() > > volume = 0.5 # range [0.0, 1.0] > fs = 44100 # sampling rate, Hz, must be integer > duration = 10.0 # in seconds, may be float > f = 440.0 # sine frequency, Hz, may be float > > # generate samples, note conversion to float32 array > samples = (np.sin(2*np.pi*np.arange(fs*duration)*f/fs)).astype(np.float32) > > # for paFloat32 sample values must be in range [-1.0, 1.0] > stream = p.open(format=pyaudio.paFloat32, > channels=1, > rate=fs, > output=True) > > # play. May repeat with different volume values (if done interactively) > stream.write(volume*samples) > > stream.stop_stream() > stream.close() Works for me. From frank at chagford.com Tue Feb 28 09:01:40 2017 From: frank at chagford.com (Frank Millman) Date: Tue, 28 Feb 2017 16:01:40 +0200 Subject: asyncio does not always show the full traceback Message-ID: Hi all I use asyncio in my project, so most of my functions start with 'async' and most of my calls are preceded by 'await'. If an exception is raised, I usually get the full traceback, but sometimes I just get something like the following - Traceback (most recent call last): File "C:\Users\User\aib\aib\test_data.py", line 645, in loop.run_until_complete(main(company)) File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\asyncio\base_events.py", line 466, in run_until_complete return future.result() ValueError: not enough values to unpack (expected 2, got 1) It does not show the line where the ValueError occurs, and I have not managed to narrow down exactly when this happens. Does anyone know what I must change to get the full traceback? Frank Millman From ethan at stoneleaf.us Tue Feb 28 11:23:03 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 28 Feb 2017 08:23:03 -0800 Subject: Exposing all methods of a class In-Reply-To: <3afdc9ac-0679-f970-b604-e3367ca9788f@stridebird.com> References: <3afdc9ac-0679-f970-b604-e3367ca9788f@stridebird.com> Message-ID: <58B5A3E7.1050107@stoneleaf.us> On 02/26/2017 12:15 PM, Pete Dowdell wrote: > I was wondering if anyone has encountered an editor that could display a > class with all inherited methods included in the editor's view of the class code. I do not know of one, but agree it would be quite useful! -- ~Ethan~ From skip.montanaro at gmail.com Tue Feb 28 12:28:14 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 28 Feb 2017 11:28:14 -0600 Subject: Manual parameter substitution in sqlite3 Message-ID: Most of the time (well, all the time if you're smart), you let the database adapter do parameter substitution for you to avoid SQL injection attacks (or stupid users). So: curs.execute("select * from mumble where key = ?", (key,)) If you want to select from several possible keys, it would be nice to be able to do this: curs.execute("select * from mumble where key in (?)", (keys,)) but that doesn't work. Instead, you need to do your own parameter substitution. The quick-and-insecure way to do this is: curs.execute("select * from mumble where key in (%s)" % ",".join([repr(k) for k in keys])) I'm pretty sure that's breakable. Some database adapters provide a function to do explicit substitution (e.g., mySQLdb.escape, psycopg2._param_escape), but the sqlite3 adapter doesn't. Is there a function floating around out there which does the right thing, allowing you to safely construct these sorts of set inclusion clauses? Thx, Skip From rosuav at gmail.com Tue Feb 28 12:40:43 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Mar 2017 04:40:43 +1100 Subject: Manual parameter substitution in sqlite3 In-Reply-To: References: Message-ID: On Wed, Mar 1, 2017 at 4:28 AM, Skip Montanaro wrote: > Some database adapters provide a function to do explicit substitution > (e.g., mySQLdb.escape, psycopg2._param_escape), but the sqlite3 > adapter doesn't. Is there a function floating around out there which > does the right thing, allowing you to safely construct these sorts of > set inclusion clauses? Testing with PostgreSQL (which *does* transform lists) suggests that "in" doesn't work; I used "key = any(%s)". I'd try that with sqlite3 first, just in case it makes a difference. Probably it won't, but worth a try. Second recommendation: Switch to PostgreSQL, because then this happens automatically :) Third recommendation: Instead of making yourself completely vulnerable, just go one level in: curs.execute("select * from mumble where key in (" + ",".join(["?"]*len(keys)) + ")", keys) If this is combined with another parameter, it'd be messier, but you could do something like: curs.execute("select * from mumble where key in (" + ",".join(["?"]*len(keys)) + ") and category = ?", tuple(keys) + (cat,)) Either way, you're still letting the sqlite connector do the processing of the elements, but handling the collection yourself. I may or may not have needed to do this once before.... with MySQL.... but I'm not going to admit to it. Because I also had to use PHP to talk to the same database. And I don't admit to knowing PHP. ChrisA From __peter__ at web.de Tue Feb 28 13:05:32 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 28 Feb 2017 19:05:32 +0100 Subject: Manual parameter substitution in sqlite3 References: Message-ID: Skip Montanaro wrote: > Most of the time (well, all the time if you're smart), you let the > database adapter do parameter substitution for you to avoid SQL > injection attacks (or stupid users). So: > > curs.execute("select * from mumble where key = ?", (key,)) > > If you want to select from several possible keys, it would be nice to > be able to do this: > > curs.execute("select * from mumble where key in (?)", (keys,)) > > but that doesn't work. Instead, you need to do your own parameter > substitution. The quick-and-insecure way to do this is: > > curs.execute("select * from mumble where key in (%s)" % > ",".join([repr(k) for k in keys])) > > I'm pretty sure that's breakable. > > Some database adapters provide a function to do explicit substitution > (e.g., mySQLdb.escape, psycopg2._param_escape), but the sqlite3 > adapter doesn't. Is there a function floating around out there which > does the right thing, allowing you to safely construct these sorts of > set inclusion clauses? > > Thx, > > Skip If all else fails use quote() http://www.sqlite.org/lang_corefunc.html#quote >>> db = sqlite3.connect(":memory:") >>> cs = db.cursor() >>> next(cs.execute("select quote(?)", ("foo 'bar' \"baz\"",)))[0] '\'foo \'\'bar\'\' "baz"\'' >>> print(_) 'foo ''bar'' "baz"' With the documented limitation: >>> next(cs.execute("select quote(?)", ("foo 'bar'\0 \"baz\"",)))[0] "'foo ''bar'''" From skip.montanaro at gmail.com Tue Feb 28 13:40:38 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 28 Feb 2017 12:40:38 -0600 Subject: Manual parameter substitution in sqlite3 In-Reply-To: References: Message-ID: On Tue, Feb 28, 2017 at 11:40 AM, Chris Angelico wrote: > Testing with PostgreSQL (which *does* transform lists) suggests that > "in" doesn't work; I used "key = any(%s)". I'd try that with sqlite3 > first, just in case it makes a difference. Probably it won't, but > worth a try. Yeah, doesn't work in Sqlite. It doesn't appear to have an any() function. > Second recommendation: Switch to PostgreSQL, because then this happens > automatically :) It's on my likely list of upgrades. Sqlite3 was just easier to start with. > Third recommendation: Instead of making yourself completely > vulnerable, just go one level in: That's what I was doing. On Tue, Feb 28, 2017 at 12:05 PM, Peter Otten <__peter__ at web.de> wrote: > If all else fails use quote() > > http://www.sqlite.org/lang_corefunc.html#quote This looks like it will do the trick nicely. I was only looking at the API for the Python package, not considering the Sqlite3 core. To allow simpler migration to another database in the future, I'll likely hide your example in a function. Thanks for the responses. Skip From skip.montanaro at gmail.com Tue Feb 28 13:42:29 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 28 Feb 2017 12:42:29 -0600 Subject: Exposing all methods of a class In-Reply-To: <58B5A3E7.1050107@stoneleaf.us> References: <3afdc9ac-0679-f970-b604-e3367ca9788f@stridebird.com> <58B5A3E7.1050107@stoneleaf.us> Message-ID: Might be worth checking out the speedbar package in GNU Emacs. I'm not sure if (or how) it might handle inheritance. Skip On Tue, Feb 28, 2017 at 10:23 AM, Ethan Furman wrote: > On 02/26/2017 12:15 PM, Pete Dowdell wrote: > >> I was wondering if anyone has encountered an editor that could display a >> class with all inherited methods included in the editor's view of the >> class code. > > > I do not know of one, but agree it would be quite useful! > > -- > ~Ethan~ > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Tue Feb 28 13:42:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 1 Mar 2017 05:42:39 +1100 Subject: Manual parameter substitution in sqlite3 In-Reply-To: References: Message-ID: On Wed, Mar 1, 2017 at 5:40 AM, Skip Montanaro wrote: > On Tue, Feb 28, 2017 at 11:40 AM, Chris Angelico wrote: >> Testing with PostgreSQL (which *does* transform lists) suggests that >> "in" doesn't work; I used "key = any(%s)". I'd try that with sqlite3 >> first, just in case it makes a difference. Probably it won't, but >> worth a try. > > Yeah, doesn't work in Sqlite. It doesn't appear to have an any() function. Not surprised, but worth a try. >> Third recommendation: Instead of making yourself completely >> vulnerable, just go one level in: > > That's what I was doing. > That isn't what you were doing in your post, so it seemed worth mentioning. Sounds like you have the best available options already at your fingertips. Have at it! ChrisA From 17chiue at gmail.com Tue Feb 28 13:51:11 2017 From: 17chiue at gmail.com (17chiue at gmail.com) Date: Tue, 28 Feb 2017 10:51:11 -0800 (PST) Subject: Online Python Editor with Live Syntax Checking Message-ID: <460d95fb-827c-4e1a-91bf-d4cc633a213b@googlegroups.com> Hi everyone, I made a tool called PythonBuddy (http://pythonbuddy.com/). I made this so that MOOCs like edX or codecademy could easily embed and use this on their courses so students wouldn't have to go through the frustrations of setting up a Python environment and jump right into Python programming. Also, professors and teachers could easily set up a server and allow students to quickly test out their code with PythonBuddy online. Github repo: https://github.com/ethanche?/OnlinePythonLinterSyntaxChecker Feel free to ask any questions you have below or any feedback you have! Would love to improve this tool in any way :) From jladasky at itu.edu Tue Feb 28 13:54:17 2017 From: jladasky at itu.edu (jladasky at itu.edu) Date: Tue, 28 Feb 2017 10:54:17 -0800 (PST) Subject: asyncio does not always show the full traceback In-Reply-To: References: Message-ID: <72209011-db09-4ba2-9c5b-f576a30e2d09@googlegroups.com> > Does anyone know what I must change to get the full traceback? Three years ago, I had a similar issue with incomplete tracebacks while using multiprocessing.Pool. The discussion is here: https://groups.google.com/forum/#!msg/comp.lang.python/qKTNNt8uKKU/biNyslh19ncJ;context-place=msg/comp.lang.python/qKTNNt8uKKU/c6K8kVdfTw4J Other, better Python programmers than myself were treating this as a bug that needed to be fixed, and were doing so for Python 3.4. The linked discussion contains a link to the developers' bug reports. It appears that the bug was addressed, but I don't know if it was backported to every version of Python. What version of Python are you using? From skip.montanaro at gmail.com Tue Feb 28 15:02:05 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 28 Feb 2017 14:02:05 -0600 Subject: Manual parameter substitution in sqlite3 In-Reply-To: References: Message-ID: On Tue, Feb 28, 2017 at 12:42 PM, Chris Angelico wrote: > That isn't what you were doing in your post, so it seemed worth > mentioning. Sorry, my original post was a bit abbreviated. I can't copy text from inside to outside, so have to retype everything. I guess I missed that. S From neilc at norwich.edu Tue Feb 28 17:05:59 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Tue, 28 Feb 2017 22:05:59 +0000 (UTC) Subject: Manual parameter substitution in sqlite3 References: Message-ID: On 2017-02-28, Skip Montanaro wrote: > Some database adapters provide a function to do explicit > substitution (e.g., mySQLdb.escape, psycopg2._param_escape), > but the sqlite3 adapter doesn't. It's clunky but you can use sqlite's core "quote" function. quote(X) The quote(X) function returns the text of an SQL literal which is the value of its argument suitable for inclusion into an SQL statement. Strings are surrounded by single-quotes with escapes on interior quotes as needed. BLOBs are encoded as hexadecimal literals. Strings with embedded NUL characters cannot be represented as string literals in SQL and hence the returned string literal is truncated prior to the first NUL. The time I found it useful was when I thought I needed to build a query with parameters where the DB-API didn't allow them, e.g., in the column-names portion of an INSERT statement. quoted_val, = c.execute("SELECT quote(?);", (val,)).fetchone() -- Neil Cerutti From python.list at tim.thechases.com Tue Feb 28 17:22:22 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 28 Feb 2017 16:22:22 -0600 Subject: Manual parameter substitution in sqlite3 In-Reply-To: References: Message-ID: <20170228162222.34a90fe1@bigbox.christie.dr> On 2017-03-01 04:40, Chris Angelico wrote: > curs.execute("select * from mumble where key in (" + > ",".join(["?"]*len(keys)) + ")", keys) > > If this is combined with another parameter, it'd be messier, but you > could do something like: > > curs.execute("select * from mumble where key in (" + > ",".join(["?"]*len(keys)) + ") and category = ?", > tuple(keys) + (cat,)) > > Either way, you're still letting the sqlite connector do the > processing of the elements, but handling the collection yourself. These are the methods I use both with sqlite and with the SQL Server adapter (at $DAYJOB). Works great and also pretty easy to debug if you preformulate the strings into a variable: PLACEHOLDER = "?" # SQL Server's place-holder params = [3, 1, 4, 5, 9] field2 = "Some value" sql = """ SELECT * FROM tbl WHERE field1 IN (%(all_placeholders)s) AND field2 = %(placeholder)s """ % dict( placeholders=",".join( PLACEHOLDER for _ in params ), placeholder=PLACEHOLDER, ) cur.execute(sql, params + [field2,]) -tkc From rantingrickjohnson at gmail.com Tue Feb 28 18:16:07 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 28 Feb 2017 15:16:07 -0800 (PST) Subject: Crappy Python code of the day In-Reply-To: <58a5277f$0$2816$c3e8da3$76491128@news.astraweb.com> References: <58a50d96$0$2816$c3e8da3$76491128@news.astraweb.com> <58a5277f$0$2816$c3e8da3$76491128@news.astraweb.com> Message-ID: <03017984-f1bb-43c6-93d4-6c29c9dfca66@googlegroups.com> On Wednesday, February 15, 2017 at 10:16:14 PM UTC-6, Steven D'Aprano wrote: > Oh, we know why the code is failing. We don't need help > diagnosing the UnboundLocalError exception. You're right: > there's an except pass around an assignment, so if the > assignment fails, `result` never gets set. > > But the real WTF is that the ConnectionError is just thrown > away. There's no attempt to recover from it, or log it, or > try connecting again... the end result is that the > application dies with an unhelpful UnboundLocalError, and > (until today) we had no idea what the actual cause of the > failure was. Perhaps the "writer of the code" (*AHEM* as you so eloquently put it), was disgruntled, and decided to leave a little "present" for the next code monkey to find. And now, thanks to you, he/she is tasting the sweet nectar of revenge. From rantingrickjohnson at gmail.com Tue Feb 28 19:16:49 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 28 Feb 2017 16:16:49 -0800 (PST) Subject: Exposing all methods of a class In-Reply-To: References: <3afdc9ac-0679-f970-b604-e3367ca9788f@stridebird.com> Message-ID: On Sunday, February 26, 2017 at 2:15:58 PM UTC-6, Pete Dowdell wrote: > I use Python, mainly with Django, for work. I was wondering > if anyone has encountered an editor that could display a > class with all inherited methods included in the editor's > view of the class code. Personally, i'm a minimalist when it comes to editing tools so i don't know jack-squat about anything like that (although the likelihood of it existing seems reasonable). But if perusing the inter-webs for days only to become frustrated is not your cup of java, and you dont mind rolling your own "green snake doobie", and you're not offended by the abysmal state of Tkinter (aka: stems-and-seeds), you might take a look at IDLE. Sarcastic Sam Said: "Got caveats?" o_O IDLE has a "class browser" feature (@GUI_XY = File-> Class_Browser) that displays a GUI tree of the currently opened module, although, and unfortunately for you, the scope of detail ends at the physical borders of the module. However, by utilizing the introspection power of python (`import inspect`, for instance), you could probably implement something akin to "comprehensive introspection reform". Of course, that last statement assumes you're _not_ an "all- talk-and-no-action" code monkey. > I am kind of envisaging the inherited code would be > displayed differently (say, grey vs black), and labelled > with the name of the parent class so that it would provide > a one-stop view of the entire class structure. Yeah, which can be quite helpful when you're trying to understand a deeply nested object structure. And let's be honest here... contrary to popular belief, it's _not_ always the third party code that causes us to bang our heads on our desks, no, sometimes, it's just some really wacky code that we wrote a long, long time ago ;-) > I edit using vim, and use ctags to explore up the > inheritance chain which is useful enough, but I feel it > should be quite possible for the editor to search through > the imports and grab the parent classes to insert into the > editor view. Any suggestions? Maybe there are better ways > of navigating inheritance but it does seem logical to > expose the whole class code in one place, suitably > annotated. I feel a plugin coming on. Hey, just be sure to post a link to the repo! ;-) From python at deborahswanson.net Tue Feb 28 19:19:11 2017 From: python at deborahswanson.net (Deborah Swanson) Date: Tue, 28 Feb 2017 16:19:11 -0800 Subject: Exposing all methods of a class In-Reply-To: <58B5A3E7.1050107@stoneleaf.us> Message-ID: <006901d29221$74122d10$27b23dae@sambora> Ethan Furman wrote, on February 28, 2017 8:23 AM > > On 02/26/2017 12:15 PM, Pete Dowdell wrote: > > > I was wondering if anyone has encountered an editor that > could display > > a class with all inherited methods included in the editor's view of > > the class code. > > I do not know of one, but agree it would be quite useful! > > -- > ~Ethan~ I haven't installed StarUML yet, but a TA in one of the MIT courses I took used StarUML to make graphs for the classes and methods we were learning. Sounds exactly like what you guys are talking about. http://staruml.io/ Deborah From jf_byrnes at comcast.net Tue Feb 28 22:12:38 2017 From: jf_byrnes at comcast.net (Jim) Date: Tue, 28 Feb 2017 21:12:38 -0600 Subject: python-libxdo? In-Reply-To: References: Message-ID: On 02/28/2017 04:05 AM, Pavol Lisy wrote: > On 2/28/17, Jim wrote: > > simplified: > >> from xdo import Xdo >> xdo = Xdo() >> win = xdo.search_windows(winname = 'Mozilla Firefox') >> File "/home/jfb/EVs/env/lib/python3.5/site-packages/xdo/__init__.py" >> TypeError: bytes or integer address expected instead of str instance > > Example on github seems to use python2 where bytes and str are same type. > > Because you have python3, you need to write -> > > win = xdo.search_windows(winname = b'Mozilla Firefox') > > PS. this reminds me "Talk Python to Me with GvR" thread on this list. :) > > How to minimize frustration? Could not have ctypes.Structure something > like encoding attribute? > Thanks Pavol, I made the change you suggested and the error is gone. Regards, Jim