From rustompmody at gmail.com Mon Aug 1 00:05:08 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 31 Jul 2016 21:05:08 -0700 (PDT) Subject: Can math.atan2 return INF? In-Reply-To: <579eb9b2$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57697e61$0$1590$c3e8da3$5496439d@news.astraweb.com> <87d1n9kzk9.fsf@bsb.me.uk> <1mp9e9m.s25gk61k28dhhN%pdorange@pas-de-pub-merci.mac.com> <87inx1janx.fsf@bsb.me.uk> <576b5eb4$0$1595$c3e8da3$5496439d@news.astraweb.com> <576b8576$0$1510$c3e8da3$5496439d@news.astraweb.com> <877fdgj7f4.fsf@bsb.me.uk> <576c1207$0$1597$c3e8da3$5496439d@news.astraweb.com> <1mpb97o.f86qm9eqx9uyN%pdorange@pas-de-pub-merci.mac.com> <87shw3es70.fsf@elektro.pacujo.net> <57739650$0$1601$c3e8da3$5496439d@news.astraweb.com> <87mvm4uux1.fsf@elektro.pacujo.net> <5774d75d$0$1617$c3e8da3$5496439d@news.astraweb.com> <0765346e-42a1-48b7-8b7d-e134f4165ce4@googlegroups.com> <57755efb$0$1604$c3e8da3$5496439d@news.astraweb.com> <579eb9b2$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <2909506b-ea11-481a-adf3-4ee6ed654c62@googlegroups.com> On Monday, August 1, 2016 at 8:23:49 AM UTC+5:30, Steven D'Aprano wrote: > On Sat, 30 Jul 2016 03:46 pm, Rustom Mody wrote: > > > Lots of questions... I would guess rhetorical. > > They weren't rhetorical. > > You've made a lot of claims about the origins of computer science, and I've > questioned some of your statements. Answers would be appreciated. > > > > However under the assumption that they are genuine (or could be genuine > > for others than you), I went back and checked. > > I recollected that I started thinking along these lines ? viz. that > > philosophical disputes led to the genesis of computers ? after reading an > > essay by a certain Adam Siepel... which subsequently seems to have fallen > > off the net > > > > I tracked him down and re-posted his essay here: > > http://blog.languager.org/2016/07/mechanism-romanticism-computers.html > > > > Just to be clear ? this is Dr. Adam Siepel's writing reposted with his > > permission > > > The essay is not awful, but I wouldn't shout its praises either. It looks to > me like an undergraduate essay, taking a very narrow and rather naive view > of the field. There's not a lot of references (only nine), which means the > author is (in my opinion) excessively influenced by a small number of > views, and I don't see any sign that he has even made a half-hearted > attempt to seek out alternate views. > > The author makes a claim: > > "... Principia Mathematica, between 1910 and 1913, which in its attempt to > place mathematics squarely in the domain of logic, represented the first > new system of logic since Aristotle" > > but doesn't give any justification for the claim. Why single out the > Principia and ignore the works of the Stoics, Peter Abelard, William of > Ockham, Augustus DeMorgan, Gottlob Frege and most especially George Boole > dismissed? > > I would think that if anyone truly deserved credit for creating a new system > of logic, it should be Boole. But perhaps that's just a matter of opinion > on where you draw the lines. > > http://www.iep.utm.edu/prop-log/#H2 > > That's not really central to his argument, but it does suggest that his > views are quite idiosyncratic. To my mind, that feels like someone claiming > that Stephen Hawking is the first genuinely original physicist since > Newton. Einstein? Schr?dinger? Dirac? Never heard of 'em. > > A rather large section of the essay is an irrelevant (and, I think, > incorrect) digression about "Mechanists" and "Romantics", neither of which > is really relevant to the philosophy of mathematics. He eventually mentions > the Intuitionists, but I don't think he understands them. By linking them > to the Romantics, he seems to think that the Intuitionist school of thought > doesn't require mathematical proofs, or that they are satisfied with > the "intuitively obvious truth" of axioms. But that's not what the > Intuitionists were about: > > https://en.wikipedia.org/wiki/Intuitionism > > He mischaracterises and over-simplifies the argument over the foundations of > mathematics: > > https://en.wikipedia.org/wiki/Brouwer?Hilbert_controversy > > with at least three separate groups involved (Logicists such as Russell, > Formalists such as Hilbert, and Constructivists such as Poincar? -- four > groups if you count Intuitionism separate from Constructionism). He > exaggerates the death of the Logicist school of thought. It continues today > with Second Order Logic. > > And I wonder why you are taking this essay as supporting your position. > According to this essay, the Intuitionists won in mathematics. And yet > Turing and Von Neumann (two major pioneers of computing) were "Mechanists". > If Intuitionism influenced computer science, where is the evidence of this? > Where are the Intuitionist computer scientists? On the contrary, academic > CS seems to have come from the Logicist school of thought, and practical > computer engineering from "whatever works" school of thought. > > None of this even *remotely* supports your assertions such as "[Turing] > wishes to put the soul into the machine". Maybe he did. But this essay > gives no reason to think so, or any reason to think that Turing's personal > beliefs about souls is the slightest bit relevant to computer science. My agreement vis-a-vis this essay is basically this: Abstruse philosophical how-many-angels-on-a-pin type arguments amongst philosophers of math/logic across 19-20 century gave rise to Computers and Computer Science. Some of the other detailed points you make, I agree with, eg. 1. There were more than 2 parties in the dispute 2. There is some arbitrary line-drawing there (Un)fortunately arbitrary line-drawing is the name of the game everywhere: Religion?Philosophy?History?Politics Right now as we speak there looks like a war ? maybe world-war ? in offing: https://www.thenation.com/article/the-united-states-and-nato-are-preparing-for-a-major-war-with-russia/ Possibly democrat-engineered to discredit Trump: https://www.thenation.com/article/neo-mccarthyism-and-olympic-politics-as-more-evidence-of-a-new-cold-war/ All starts with the disorder in the middle-east and a whole lot of arbitrary lines drawn there [Going backward in time] - A line drawn in space called ?Israel? - Based on a line drawn in time called the ?Exodus of Moses? - Based on the supremely authoritative history-record called ?The Bible? (or Torah depending on the speaker) Leaving aside the first and the third, why specifically the Exodus as a definer? Before Canaan they were in Egypt, so Israel=Egypt is equally legitimate And before?before? they were in Eden. Why not set Israel up in Eden? May it just be the (in)convenient fact that knocking the Palestinians out of their homes is easier than removing Mr. God from Eden? Collapse of borders: http://www.independent.co.uk/news/world/middle-east/isis-in-a-borderless-world-the-days-when-we-could-fight-foreign-wars-and-be-safe-at-home-may-be-long-a6741146.html From ian.g.kelly at gmail.com Mon Aug 1 03:05:36 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 1 Aug 2016 01:05:36 -0600 Subject: Can math.atan2 return INF? In-Reply-To: <2909506b-ea11-481a-adf3-4ee6ed654c62@googlegroups.com> References: <57697e61$0$1590$c3e8da3$5496439d@news.astraweb.com> <87d1n9kzk9.fsf@bsb.me.uk> <1mp9e9m.s25gk61k28dhhN%pdorange@pas-de-pub-merci.mac.com> <87inx1janx.fsf@bsb.me.uk> <576b5eb4$0$1595$c3e8da3$5496439d@news.astraweb.com> <576b8576$0$1510$c3e8da3$5496439d@news.astraweb.com> <877fdgj7f4.fsf@bsb.me.uk> <576c1207$0$1597$c3e8da3$5496439d@news.astraweb.com> <1mpb97o.f86qm9eqx9uyN%pdorange@pas-de-pub-merci.mac.com> <87shw3es70.fsf@elektro.pacujo.net> <57739650$0$1601$c3e8da3$5496439d@news.astraweb.com> <87mvm4uux1.fsf@elektro.pacujo.net> <5774d75d$0$1617$c3e8da3$5496439d@news.astraweb.com> <0765346e-42a1-48b7-8b7d-e134f4165ce4@googlegroups.com> <57755efb$0$1604$c3e8da3$5496439d@news.astraweb.com> <579eb9b2$0$1611$c3e8da3$5496439d@news.astraweb.com> <2909506b-ea11-481a-adf3-4ee6ed654c62@googlegroups.com> Message-ID: On Sun, Jul 31, 2016 at 10:05 PM, Rustom Mody wrote: > All starts with the disorder in the middle-east and a whole lot of arbitrary lines > drawn there > [Going backward in time] > - A line drawn in space called ?Israel? > - Based on a line drawn in time called the ?Exodus of Moses? I'm no expert but I believe that Zionism was based upon the idea of Negation of the Diaspora, not Exodus. > - Based on the supremely authoritative history-record called ?The Bible? > (or Torah depending on the speaker) > > Leaving aside the first and the third, why specifically the Exodus as a definer? > Before Canaan they were in Egypt, so Israel=Egypt is equally legitimate > And before?before? they were in Eden. > Why not set Israel up in Eden? Since the Exodus story has it that the Israelites were slaves in Egypt (despite the utter lack of archaeological evidence that they were ever there en masse), placing Israel in Egypt would seem rather insulting. > May it just be the (in)convenient fact that knocking the Palestinians out of > their homes is easier than removing Mr. God from Eden? Fun historical fact: the land of Palestine was so named by the Roman Emperor Hadrian after he kicked the Jews out of their homes and forced them to leave. The name was a Romanization of the ancient land of Philistia, home of the Philistines, despite that they were long gone by this point. So the entire existence of Palestine is basically one big fat Roman middle finger to the Jews. Not that any of this is meant to condone any of the violence in the Middle East. From michael.selik at gmail.com Mon Aug 1 03:13:47 2016 From: michael.selik at gmail.com (Michael Selik) Date: Mon, 01 Aug 2016 07:13:47 +0000 Subject: usage of functools.partial in in parallelism In-Reply-To: References: Message-ID: On Sun, Jul 31, 2016 at 5:47 AM Sivan Greenberg wrote: > That's exactly the answer I was looking for. Thanks. > > I got used too much I guess to solving problems the OOP way, e.g. my code > wraps the session.get invocation with a class to pack together the > arguments and data and also took care of parallelism using gevent, from > within the class. > > While a bit more verbose (or god forbid..unpythonic?) I find it much nore > readable in my taste and easier to pass on to the next code maintainer in > the heritage. > You might benefit from watching the talk "Stop Writing Classes" https://www.youtube.com/watch?v=o9pEzgHorH0 From bart4858 at gmail.com Mon Aug 1 03:55:02 2016 From: bart4858 at gmail.com (bart4858 at gmail.com) Date: Mon, 1 Aug 2016 00:55:02 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <5792cd34$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <579e19e0$0$1595$c3e8da3$5496439d@news.astraweb.com> <7f3efb38-e146-436d-b38e-a0e56427f7dc@googlegroups.com> Message-ID: On Monday, 1 August 2016 01:22:02 UTC+1, Chris Angelico wrote: > On Mon, Aug 1, 2016 at 10:11 AM, wrote: > > (128MB or 128KB? In the 1980s we were all running in 64KB to 640KB of memory. 128MB might be what a well-endowed mainframe might have had!) > > Yes, and we didn't have Python then. When I had a computer with 640KB > of memory, my options were (1) BASIC or (2) 8086 assembly language, > using DEBUG.EXE and its mini-assembler. Later on (much much later), I > added C to the available languages, but it was tedious and annoying, > because one tiny change meant minutes of compilation. This wasn't my experience. I used my own tools and designed them to always be quick enough in use that compilation speed was never really an issue. Not even on 8-bit machines. I was also happily running my interpreters within 640KB (less than that too depending on customers' machines). So that 128MB limit, or *two hundred* times as much memory, was a hardly a limitation! -- Bartc From bart4858 at gmail.com Mon Aug 1 04:05:53 2016 From: bart4858 at gmail.com (bart4858 at gmail.com) Date: Mon, 1 Aug 2016 01:05:53 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> Message-ID: <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> On Monday, 1 August 2016 01:33:37 UTC+1, Chris Angelico wrote: > On Mon, Aug 1, 2016 at 10:21 AM, wrote: > > However I do 'drive' as I've been programming for decades. And I can have an opinion about a model of car that I don't normally drive. An opinion which you might well not get from someone who drives that model for a living. > > > > So've I, but I don't try to tell the Scheme folks that they need to > change the language. You can have an opinion - but it doesn't make it > worth anything. OK. But I both drive, and build my own cars! You think that my nearly 30 years' experience of designing interpreted languages and writing fast bytecode interpreters doesn't make my opinions have any more merit, that's fine. -- Bartc From alister.ware at ntlworld.com Mon Aug 1 05:40:30 2016 From: alister.ware at ntlworld.com (alister) Date: Mon, 01 Aug 2016 09:40:30 GMT Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> Message-ID: On Mon, 01 Aug 2016 09:49:46 +1000, Chris Angelico wrote: > On Mon, Aug 1, 2016 at 9:43 AM, wrote: >> On Sunday, 31 July 2016 21:01:52 UTC+1, Michael Torrie wrote: >> >>> That said, I wish he'd stop posting his arguments here on this list as >>> he clearly doesn't use Python for anything, and hasn't used Python for >>> any real amount of coding. He has no vested interest in Python so why >>> should his opinions matter to us? >> >> What's using Python got to do with it? If I needed to write Python as a >> job, then I'd just get on with it. I probably wouldn't have time to >> post on here! >> >> I don't so can discuss it from a different perspective. > > I should get into Parliament and start passing laws about cars. I'd have > a valuable perspective on it, since I never drive. > > ChrisA Based on my experience of driving laws & road traffic planning that is a mandatory requirement. -- My BIOLOGICAL ALARM CLOCK just went off ... It has noiseless DOZE FUNCTION and full kitchen!! From alister.ware at ntlworld.com Mon Aug 1 05:50:08 2016 From: alister.ware at ntlworld.com (alister) Date: Mon, 01 Aug 2016 09:50:08 GMT Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> Message-ID: On Mon, 01 Aug 2016 01:05:53 -0700, bart4858 wrote: > On Monday, 1 August 2016 01:33:37 UTC+1, Chris Angelico wrote: >> On Mon, Aug 1, 2016 at 10:21 AM, wrote: > >> > However I do 'drive' as I've been programming for decades. And I can >> > have an opinion about a model of car that I don't normally drive. An >> > opinion which you might well not get from someone who drives that >> > model for a living. >> > >> > >> So've I, but I don't try to tell the Scheme folks that they need to >> change the language. You can have an opinion - but it doesn't make it >> worth anything. > > OK. But I both drive, and build my own cars! > > You think that my nearly 30 years' experience of designing interpreted > languages and writing fast bytecode interpreters doesn't make my > opinions have any more merit, that's fine. Actually the more you make these claims the more I think you are suffering from NIH (Not Invented Here) syndrome. there is another poster on the F1 group who initially made comments along the lines "Maybe Ferrari should employ me" at first they were taken as tongue in cheek until it became apparent he actually believed them. without any significant user base for any of your invented Languages I would suggest they are nothing more than toys* & instead of thinking "my designs are much better" you would be better to use the approach "that's not how I would have done it what, do they know that I am missing" * to be fare even a toy language is beyond my own meagre abilities. -- Worst Month of 1981 for Downhill Skiing: August. The lift lines are the shortest, though. -- Steve Rubenstein From steve+comp.lang.python at pearwood.info Mon Aug 1 06:12:15 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 01 Aug 2016 20:12:15 +1000 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> Message-ID: <579f207f$0$11110$c3e8da3@news.astraweb.com> On Monday 01 August 2016 18:05, bart4858 at gmail.com wrote: > You think that my nearly 30 years' experience of designing interpreted > languages and writing fast bytecode interpreters doesn't make my opinions > have any more merit, that's fine. If you are the only one who has ever seen or used these fast bytecode interpreters, then they might as well not exist, in which case, no, your opinions have no merit. No offense intended Bart, but for all we know every single one of your benchmarks are faked, your interpreters are buggy pieces of garbage, and for all your self-promotion, your code wouldn't survive more than six seconds of peer review. -- Steve From bart4858 at gmail.com Mon Aug 1 09:19:39 2016 From: bart4858 at gmail.com (bart4858 at gmail.com) Date: Mon, 1 Aug 2016 06:19:39 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <579f207f$0$11110$c3e8da3@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> Message-ID: <2214300f-92ca-48e2-ae2c-2fd7a27ffc06@googlegroups.com> On Monday, 1 August 2016 11:12:30 UTC+1, Steven D'Aprano wrote: > On Monday 01 August 2016 18:05, bart4858 at gmail.com wrote: > > > You think that my nearly 30 years' experience of designing interpreted > > languages and writing fast bytecode interpreters doesn't make my opinions > > have any more merit, that's fine. > > If you are the only one who has ever seen or used these fast bytecode > interpreters, then they might as well not exist, in which case, no, your > opinions have no merit. > > No offense intended Bart, but for all we know every single one of your > benchmarks are faked, your interpreters are buggy pieces of garbage, and for > all your self-promotion, your code wouldn't survive more than six seconds of > peer review. Sure. I can't pretend the implementations are wonderful. But I think the ideas are good and they would make fine little languages when seriously implementated. Not that it bothers me because I've sold $millions of software implemented 100% in these 'toy' languages (and versions from 20+ years ago too). (If you want to try something out, go to: https://github.com/bartg/langs Download the interpreter pcc32.exe (a Windows binary, but I remember it seemed to work under 'wine' in Linux). And test program jpeg.bc (a binary bytecode file). Run as pcc32 jpeg file.jpg (or as wine pcc32 jpeg file.jpg). If it works, the image will be displayed. The sourcecode for jpeg.bc (its main module) is in jpeg.q. (jpeg.m is a separate static language version.) I don't have the Python version to hand, but that has been posted in the past (and which you have already commented on!). There is also qq.bc, a bytecode compiler. Run as pcc32 qq filename.q, but I haven't provided libraries so only simple programs can be compiled. (Note that pcc32.exe is not accelerated; that version is 2-3 times faster on this test.) I'm working at the moment on a version that works like Python, directly on source code rather than precompiled bytecode. But it needs to be fast enough so I'm aiming for a 1M lines per second compilation speed; I've managed 0.8-0.9Mpls so far on real applications.) -- Bartc From bart4858 at gmail.com Mon Aug 1 09:26:26 2016 From: bart4858 at gmail.com (bart4858 at gmail.com) Date: Mon, 1 Aug 2016 06:26:26 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> Message-ID: <65021545-28c4-4e7d-9d92-a6739fc13d93@googlegroups.com> On Monday, 1 August 2016 10:50:20 UTC+1, alister wrote: > Actually the more you make these claims the more I think you are > suffering from NIH (Not Invented Here) syndrome. That's not surprising. I started out developing hardware such as microprocessor boards and video displays. My languages started as streamlined tools that I needed for my job. Commercial products at the time were poor Chris pointed out (and also expensive). -- Bartc From nicoe at openhex.org Mon Aug 1 10:55:23 2016 From: nicoe at openhex.org (Nicolas =?utf-8?Q?=C3=89vrard?=) Date: Mon, 1 Aug 2016 16:55:23 +0200 Subject: SOAP and Zeep In-Reply-To: <579B9869.1040707@stoneleaf.us> References: <579B9869.1040707@stoneleaf.us> Message-ID: <20160801145523.GB23354@localhost.localdomain> * Ethan Furman [2016-07-29 19:54 +0200]: >Greetings! > >I may have a need in the immediate future to work with SOAP and WSDL >services, and a quick search turned up Zeep >(http://docs.python-zeep.org/en/latest/) -- does anyone have any >experience with it? >Or any other libraries that can be recommended? We used zeep to communicate with SOAP webservices. We choose it over suds because it supports python3. I also happen to have a bug, and the maintainer quickly accepted my patch, so according to me it's a good choice. -- (?> Nicolas ?vrard ( ) Li?ge `? From gordon at address.invalid Mon Aug 1 11:30:47 2016 From: gordon at address.invalid (Gordon Levi) Date: Tue, 02 Aug 2016 01:30:47 +1000 Subject: Why not allow empty code blocks? References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> Message-ID: "D'Arcy J.M. Cain" wrote: >On Sun, 31 Jul 2016 11:53:47 -0400 >"D'Arcy J.M. Cain" wrote: >> On Mon, 01 Aug 2016 00:25:58 +1000 >> On the other hand I have no throwaway accounts. Every address I use >> is a primary one. I have all sorts of methods to block spam. None of >> those methods involves moving my spam problem to someone else. > >For example... > >: Host or domain name not found. Name service >error for name=address.invalid type=AAAA: Host not found > >So I have to examine every address I reply to or deal with the bounce >message later. Way to move your spam problem to someone else. .invalid is specified in RFC 2606 for exactly that purpose. If you don't notice that the address is invalid a server very close to you, probably your SMTP server, will tell you. In contrast, your valid email address will become the target of many spam emails even if you manage to block them. It will also be used as the source of spam so somebody at vex.net may be fooled into believing that they are receiving a proper email. From ganesh1pal at gmail.com Mon Aug 1 11:31:47 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 1 Aug 2016 21:01:47 +0530 Subject: use import * Message-ID: Hi Team , I am a Linux user on python 2,6 . I have a very simple question I was going the zen of python by Tim peters and found an example that demonstrates Explicit is better than implicit """Load the cat, dog, and mouse models so we can edit instances of them.""" def load(): from menagerie.cat.models import * from menagerie.dog.models import * from menagerie.mouse.models import * #----------------------------------------------------------------------- def load(): from menagerie.models import cat as cat_models from menagerie.models import dog as dog_models from menagerie.models import mouse as mouse_models #----------------------------------------------------------------------- print 'Explicit is better than implicit.' I had a question on the above example 1. I haven't used " from menagerie.cat.models import * is it a good programming practice to use import * ? if answer is "NO " then are there situation where you are forced to use import * Regards, Ganesh From PointedEars at web.de Mon Aug 1 11:55:19 2016 From: PointedEars at web.de (Thomas 'PointedEars' Lahn) Date: Mon, 01 Aug 2016 17:55:19 +0200 Subject: use import * References: Message-ID: <5115899.MhkbZ0Pkbq@PointedEars.de> Ganesh Pal wrote: > is it a good programming practice to use [from ? ] import * ? No; but it is not as harmful as one would think either. > if answer is "NO " then are there situation where you are forced to use > import * No. -- PointedEars Twitter: @PointedEars2 Please do not cc me. / Bitte keine Kopien per E-Mail. From ian.g.kelly at gmail.com Mon Aug 1 11:59:47 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 1 Aug 2016 09:59:47 -0600 Subject: use import * In-Reply-To: References: Message-ID: On Mon, Aug 1, 2016 at 9:31 AM, Ganesh Pal wrote: > Hi Team , > > I am a Linux user on python 2,6 . I have a very simple question > > I was going the zen of python by Tim peters and found an example that > demonstrates Explicit is better than implicit > > """Load the cat, dog, and mouse models so we can edit instances of them.""" > def load(): > from menagerie.cat.models import * > from menagerie.dog.models import * > from menagerie.mouse.models import * > #----------------------------------------------------------------------- > def load(): > from menagerie.models import cat as cat_models > from menagerie.models import dog as dog_models > from menagerie.models import mouse as mouse_models > #----------------------------------------------------------------------- > print 'Explicit is better than implicit.' These aren't equivalent. The first implies the existence of the package "menagerie.cat.models" while the second implies the existence of "menagerie.models.cat". > I had a question on the above example > > 1. I haven't used " from menagerie.cat.models import * is it a good > programming practice to use import * ? if answer is "NO " then are > there situation where you are forced to use import * Generally, no. It can be convenient for use in the interactive interpreter or in toy programs where code health is unimportant. If you use it in real work, then you end up in situations where you can't tell just from looking at the source what names are actually being imported. From darcy at Vex.Net Mon Aug 1 12:05:15 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Mon, 1 Aug 2016 12:05:15 -0400 Subject: Using valid emails In-Reply-To: References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> Message-ID: <20160801120515.654f2737@imp> Time to change the subject. On Tue, 02 Aug 2016 01:30:47 +1000 Gordon Levi wrote: > "D'Arcy J.M. Cain" wrote: > >: Host or domain name not found. Name service > >error for name=address.invalid type=AAAA: Host not found > > > >So I have to examine every address I reply to or deal with the bounce > >message later. Way to move your spam problem to someone else. > > .invalid is specified in RFC 2606 for exactly that purpose. If you > don't notice that the address is invalid a server very close to you, Sure so either I have to stop and examine the address or else deal with the bounce from my SMTP server. Isn't that what I said? > probably your SMTP server, will tell you. In contrast, your valid > email address will become the target of many spam emails even if you > manage to block them. It will also be used as the source of spam so > somebody at vex.net may be fooled into believing that they are > receiving a proper email. I'm a little insulted that you think that I would ever allow someone to send mail to Vex.Net pretending to be from Vex.Net. That's just too easy to detect and block. Try it. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From rosuav at gmail.com Mon Aug 1 12:19:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Aug 2016 02:19:58 +1000 Subject: use import * In-Reply-To: References: Message-ID: On Tue, Aug 2, 2016 at 1:31 AM, Ganesh Pal wrote: > I am a Linux user on python 2,6 . I have a very simple question > > I was going the zen of python by Tim peters and found an example that > demonstrates Explicit is better than implicit > > """Load the cat, dog, and mouse models so we can edit instances of them.""" > def load(): > from menagerie.cat.models import * > from menagerie.dog.models import * > from menagerie.mouse.models import * > #----------------------------------------------------------------------- > def load(): > from menagerie.models import cat as cat_models > from menagerie.models import dog as dog_models > from menagerie.models import mouse as mouse_models > #----------------------------------------------------------------------- > print 'Explicit is better than implicit.' > > > I had a question on the above example > > 1. I haven't used " from menagerie.cat.models import * is it a good > programming practice to use import * ? if answer is "NO " then are > there situation where you are forced to use import * > Specifically to that example? No it's never a good idea to do a star-import inside a function. It's completely illegal in Python 3. Normally, you'll want to do your imports at module level, and not with the star. There are a very few places where a star import is the best thing to do, and they're usually because of a tight binding between the two modules. For example, it's common for __init__.py to say "from .something import *", to make a package's module's contents available in the package itself. Also, a Python module with a C accelerator will often start or end by star-importing from the accelerator (eg ast.py "from _ast import *"). If you're not doing something like that, there are only a handful of modules that are really designed with star-imports in mind (eg tkinter, stat), and even then, I personally don't recommend it. The downside of a star-import is that it breaks the common pattern that the first use of a name is its definition. If you see, somewhere in a program, "conn = psycopg2.connect(...)", you can go up to the top of the program to find "import psycopg2" and know that it's a module - even if you don't recognize the name (since it's a third-party one), you can know that it's a module. Similarly, if you see "conn = connect(...)", you can go up to the top - but then it depends on whether the program author wrote "from psycopg2 import connect" or "from psycopg2 import *". In the latter case, you have to *guess* that the name "connect" might have come from there. So, my recommendation is: Use "import *" only when you control both ends of something. Otherwise, be explicit. ChrisA From rosuav at gmail.com Mon Aug 1 12:22:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Aug 2016 02:22:50 +1000 Subject: Using valid emails In-Reply-To: <20160801120515.654f2737@imp> References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <20160801120515.654f2737@imp> Message-ID: On Tue, Aug 2, 2016 at 2:05 AM, D'Arcy J.M. Cain wrote: >> probably your SMTP server, will tell you. In contrast, your valid >> email address will become the target of many spam emails even if you >> manage to block them. It will also be used as the source of spam so >> somebody at vex.net may be fooled into believing that they are >> receiving a proper email. > > I'm a little insulted that you think that I would ever allow someone to > send mail to Vex.Net pretending to be from Vex.Net. That's just too > easy to detect and block. Try it. vex.net. 10800 IN TXT "v=spf1 mx -all" vex.net. 10800 IN MX 10 mail.vex.net. mail.vex.net. 10800 IN A 98.158.139.68 So, if you're not sending mail from 98.158.139.68 and you claim to be vex.net, you should be blocked. Unfortunately, a lot of domains don't have SPF records, or have very convoluted ones (check out gmail.com's SPF for instance). ChrisA From darcy at Vex.Net Mon Aug 1 12:32:19 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Mon, 1 Aug 2016 12:32:19 -0400 Subject: Using valid emails In-Reply-To: <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> Message-ID: <20160801123219.77633305@imp> On Sun, 31 Jul 2016 14:01:26 -0600 Michael Torrie wrote: > > So I have to examine every address I reply to or deal with the > > bounce message later. Way to move your spam problem to someone > > else. > > I've never had this problem. I use the mailing list side of things > and I reply to the list and I don't have bounce issues. Besides Fine for you. I actually prefer to have people explicitly reply to me (and the list) because then it winds up in my INBOX instead of the Python list folder. If I am busy I might skim over the Python folder but I always read my INBOX. The mail will stand out better. If I preferred everything in my Python folder I would just change the filter order. The idea is that as much as possible everyone should be able to get everything that they want. > that, he's not using email; he's using usenet which has never > required a valid email address, real or fake. To demand he use a > real email address is as silly as that guy that lambasts people on > this list for not using real names. That's really not the same thing at all. In fact, I don't even care if the address even works, as long as it doesn't bounce. If I didn't want to get replies I would use NULL at Vex.Net which gets silently dropped and the address suggests that it will not be delivered. But I absolutely don't care if someone uses a fake name. I agree that such a restriction would be silly not to mention totally unenforceable. > That said, I wish he'd stop posting his arguments here on this list as > he clearly doesn't use Python for anything, and hasn't used Python for > any real amount of coding. He has no vested interest in Python so why > should his opinions matter to us? Agreed. That's why I have stopped talking to and listening to him. > As for spam, I hardly think spambots trolling Usenet are as effective > these days as they once were. And most people's anti-spam filters are > pretty good. Besides that, those of us posting to the mailing list > side of things get our real email addresses copied out into Usenet > and I haven't noticed any change in spam levels. Ditto. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From darcy at Vex.Net Mon Aug 1 12:40:14 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Mon, 1 Aug 2016 12:40:14 -0400 Subject: Using valid emails In-Reply-To: References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <20160801120515.654f2737@imp> Message-ID: <20160801124014.214444c2@imp> On Tue, 2 Aug 2016 02:22:50 +1000 Chris Angelico wrote: > > I'm a little insulted that you think that I would ever allow > > someone to send mail to Vex.Net pretending to be from Vex.Net. > > That's just too easy to detect and block. Try it. > > vex.net. 10800 IN TXT "v=spf1 mx -all" > vex.net. 10800 IN MX 10 mail.vex.net. > mail.vex.net. 10800 IN A 98.158.139.68 > > So, if you're not sending mail from 98.158.139.68 and you claim to be > vex.net, you should be blocked. Exactly. I also have a further check. If you try to send me email pretending to be from Vex.Net you get a "5xx Yah Right" error. > Unfortunately, a lot of domains don't have SPF records, or have very > convoluted ones (check out gmail.com's SPF for instance). I can only fix my little corner of the Internet. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From torriem at gmail.com Mon Aug 1 14:38:27 2016 From: torriem at gmail.com (Michael Torrie) Date: Mon, 1 Aug 2016 12:38:27 -0600 Subject: Using valid emails In-Reply-To: <20160801123219.77633305@imp> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <20160801123219.77633305@imp> Message-ID: On 08/01/2016 10:32 AM, D'Arcy J.M. Cain wrote: > On Sun, 31 Jul 2016 14:01:26 -0600 > Michael Torrie wrote: >>> So I have to examine every address I reply to or deal with the >>> bounce message later. Way to move your spam problem to someone >>> else. >> >> I've never had this problem. I use the mailing list side of things >> and I reply to the list and I don't have bounce issues. Besides > > Fine for you. I actually prefer to have people explicitly reply to me > (and the list) because then it winds up in my INBOX instead of the > Python list folder. If I am busy I might skim over the Python folder > but I always read my INBOX. The mail will stand out better. That's not something I would ever want, nor I suspect people who are participating in a lot of e-mail lists. To each his own. But more importantly, this just isn't how Usenet works. You may be using the e-mail gateway, but not everyone is. People coming from Usenet are always going to be replying to the list in general, because as I said before Usenet doesn't care about valid email addresses. You can complain all you want, but you're unlikely to have replies going to your email inbox *and* the list folks reply from Usenet. I myself will almost never reply intentionally to the person and also the list. Duplicated emails just annoy people. And they screw up Gmail's filters on my own account. Of course that's mainly because Gmail is pretty stupid when it comes to dealing with mailing lists and they refuse to fix it. From marko at pacujo.net Mon Aug 1 15:14:21 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 01 Aug 2016 22:14:21 +0300 Subject: Why not allow empty code blocks? References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> Message-ID: <87lh0gcnea.fsf@elektro.pacujo.net> Gordon Levi : > In contrast, your valid email address will become the target of many > spam emails even if you manage to block them. It will also be used as > the source of spam so somebody at vex.net may be fooled into believing > that they are receiving a proper email. Spammers aren't going to cow me into submission. This is my internet, not theirs. Anyway, SPF can prevent sender address spoofing effectively. Marko From marko at pacujo.net Mon Aug 1 15:16:25 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 01 Aug 2016 22:16:25 +0300 Subject: Using valid emails References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <20160801120515.654f2737@imp> Message-ID: <87h9b4cnau.fsf@elektro.pacujo.net> Chris Angelico : > Unfortunately, a lot of domains don't have SPF records, or have very > convoluted ones (check out gmail.com's SPF for instance). That's their problem. Marko From darcy at Vex.Net Mon Aug 1 15:27:29 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Mon, 1 Aug 2016 15:27:29 -0400 Subject: Using valid emails In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <20160801123219.77633305@imp> Message-ID: <20160801152729.5014751c@imp> On Mon, 1 Aug 2016 12:38:27 -0600 Michael Torrie wrote: > That's not something I would ever want, nor I suspect people who are > participating in a lot of e-mail lists. To each his own. Exactly my point. Make the general case something that allows each person to control their own experience. > But more importantly, this just isn't how Usenet works. You may be I have ranted against gatewaying Usenet and email lists in the past. No one listened then. I don't suppose anyone would listen now. At least I was able to filter out Google groups where most of the garbage was coming from. :0 Hir * ^List-Id:.*python-list.python.org * ^From:.*@gmail.com * ^Newsgroups:.* /dev/null -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From bart4858 at gmail.com Mon Aug 1 16:22:58 2016 From: bart4858 at gmail.com (bartc) Date: Mon, 1 Aug 2016 13:22:58 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <579f207f$0$11110$c3e8da3@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> Message-ID: <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> On Monday, 1 August 2016 11:12:30 UTC+1, Steven D'Aprano wrote: > On Monday 01 August 2016 18:05, bart4858 at gmail.com wrote: > No offense intended Bart, but for all we know every single one of your > benchmarks are faked, your interpreters are buggy pieces of garbage, and for > all your self-promotion I missed this comment earlier on, but I can't really let it go. I never promote any actual product of mine, but I might bring in ideas from them to illustrate a point or a different way of doing things (although that usually falls on deaf as the language is already perfect. As is C, funnily enough.) Only a couple of times have links been given (after creating them to make it possible) when people have accused me of making things up! But I'm not selling anything. -- Bartc From rantingrickjohnson at gmail.com Mon Aug 1 19:28:39 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 1 Aug 2016 16:28:39 -0700 (PDT) Subject: Is it possible to draw a BUTTON? In-Reply-To: References: <8e075875-3a7b-467a-9f81-88eb8ac6e445@googlegroups.com> <28fab6aa-176b-49ad-9e5e-cdce2e081940@googlegroups.com> Message-ID: <9c864d9c-d94c-4399-9eee-73e4d6eee133@googlegroups.com> On Thursday, July 28, 2016 at 11:32:19 PM UTC-5, huey.y... at gmail.com wrote: > Yes, Rick's code works great. Well, could you go a little > further? No, i think we've gone far enough. Just FYI, Rick doesn't do homework for free *OR* for a fee. Now, it's time for *YOU* to step u.! If you want more help, you'll first have to prove that our offerings so far have not fallen on deaf ears. Here are two indispensable sites for learning Tkinter: http://effbot.org/tkinterbook/ http://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html I will not write the code for you, but i will give you a basic outline to follow. (Step 1): Learn how to display an image on the Tkinter canvas. (Step 2): In the handler that is bound to the mouse press event, display the image where ever you want. Note: If you want the image to be displayed whilst the mouse button is held down, and then hidden when the mouse button is released, you'll have to bind the events "" for the press, and for the release -- where "N" is the number 1,2 or 3. Look at Christian's code for more details. PS: If you're having trouble, please don't expect us to solve multiple problems at once, instead, try to focus on one step of the problem at a time. If you're going to be a programmer, learning to break down large complex problems into smaller, more manageable chunks, will be paramount to your success. After you get all the "chunks" working, then you can assemble them into a working program. Since the interpreted nature of Python allows you to easily test small "chunks of code" on the command line and see the results, be sure to take advantage of this ability! From lawrencedo99 at gmail.com Mon Aug 1 21:32:25 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 1 Aug 2016 18:32:25 -0700 (PDT) Subject: use import * In-Reply-To: References: Message-ID: On Tuesday, August 2, 2016 at 3:32:11 AM UTC+12, Ganesh Pal wrote: > is it a good programming practice to use import * ? No. Think of the module you?re importing as a can, and each of the names it defines as a worm. So you when you do ?from ?module? import *?, it?s like emptying all the worms in the can onto the floor, and letting them crawl all over your program, polluting your global namespace. Now imagine you change your mind, and want to get all the worms back into the can, because people reading your program (yourself included) cannot be sure where each name is coming from. Not a pleasant process. So the wise programmer avoids getting into this situation. Yes, it can be annoying to have to write module names over and over. Python lets you use alternative names, e.g. import long_module_name as lmn then you say ?lmn.?name?? instead of ?long_module_name.?name??. From lawrencedo99 at gmail.com Mon Aug 1 21:33:44 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 1 Aug 2016 18:33:44 -0700 (PDT) Subject: python and open office In-Reply-To: References: Message-ID: <7c364eae-63a1-410e-997c-becab8071d9b@googlegroups.com> On Thursday, July 28, 2016 at 4:29:34 AM UTC+12, Crane Ugly wrote: > I try to create some scripts that will help me to open and manipulate > OpenOffice documents. Have you looked at odfpy ? From lawrencedo99 at gmail.com Mon Aug 1 21:41:53 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 1 Aug 2016 18:41:53 -0700 (PDT) Subject: ctypes Usage Note Message-ID: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> Sometimes people load a library with ctypes like this: libc = ctypes.cdll.LoadLibrary("libc.so") Don?t do that. Do this instead: libc = ctypes.cdll.LoadLibrary("libc.so.6") What?s the difference? The difference is that the unversioned library comes from the ?development? package, while the versioned one comes from the ?runtime? package. Users of your Python binding should only need to have the runtime package installed to run scripts that use it, not the development package. So: always load the explicitly-versioned library name. I thnk this recommendation should be part of the official ctypes docs , don?t you? From lawrencedo99 at gmail.com Mon Aug 1 21:45:20 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 1 Aug 2016 18:45:20 -0700 (PDT) Subject: `exec`-based routine crashes app upon migration from 3.4.3 to python 3.5.2. In-Reply-To: References: Message-ID: <9c6b75e0-67ba-4495-9ec4-fde591249c36@googlegroups.com> On Friday, July 29, 2016 at 6:25:51 AM UTC+12, Enjoys Math wrote: > exec('obj = ' + objType + '(self)', None, _locals) > obj = _locals['obj'] Why? Why not just obj = objType(self) ? From torriem at gmail.com Mon Aug 1 23:14:42 2016 From: torriem at gmail.com (Michael Torrie) Date: Mon, 1 Aug 2016 21:14:42 -0600 Subject: usage of functools.partial in in parallelism In-Reply-To: References: Message-ID: <5c6c7f8c-8f1a-8fd0-9972-54312f400263@gmail.com> On 08/01/2016 01:13 AM, Michael Selik wrote: > You might benefit from watching the talk "Stop Writing Classes" > https://www.youtube.com/watch?v=o9pEzgHorH0 Great talk! Thanks for posting that. From steve+comp.lang.python at pearwood.info Tue Aug 2 00:14:24 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 02 Aug 2016 14:14:24 +1000 Subject: usage of functools.partial in in parallelism References: <5c6c7f8c-8f1a-8fd0-9972-54312f400263@gmail.com> Message-ID: <57a01e20$0$2809$c3e8da3$76491128@news.astraweb.com> On Tuesday 02 August 2016 13:14, Michael Torrie wrote: > On 08/01/2016 01:13 AM, Michael Selik wrote: >> You might benefit from watching the talk "Stop Writing Classes" >> https://www.youtube.com/watch?v=o9pEzgHorH0 > > Great talk! Thanks for posting that. It is a great talk, but for a counter-view: http://lucumr.pocoo.org/2013/2/13/moar-classes/ Remember: moderation in all things, except for moderation. -- Steve From eryksun at gmail.com Tue Aug 2 00:48:11 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 2 Aug 2016 04:48:11 +0000 Subject: `exec`-based routine crashes app upon migration from 3.4.3 to python 3.5.2. In-Reply-To: <9c6b75e0-67ba-4495-9ec4-fde591249c36@googlegroups.com> References: <9c6b75e0-67ba-4495-9ec4-fde591249c36@googlegroups.com> Message-ID: On Tue, Aug 2, 2016 at 1:45 AM, Lawrence D?Oliveiro wrote: > On Friday, July 29, 2016 at 6:25:51 AM UTC+12, Enjoys Math wrote: > >> exec('obj = ' + objType + '(self)', None, _locals) >> obj = _locals['obj'] > > Why? Why not just > > obj = objType(self) I think you meant to use `globals()[objType](self)`, as was already suggested by Chris Angelico. The value of objType appears to be the type's name, maybe from marshaled data. From eryksun at gmail.com Tue Aug 2 01:17:19 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 2 Aug 2016 05:17:19 +0000 Subject: ctypes Usage Note In-Reply-To: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> References: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> Message-ID: On Tue, Aug 2, 2016 at 1:41 AM, Lawrence D?Oliveiro wrote: > > Don?t do that. Do this instead: > > libc = ctypes.cdll.LoadLibrary("libc.so.6") I recommend using ctypes.CDLL instead. ctypes.cdll is pretty much pointless on Unix systems (and a potential problem on Windows due to cached function pointers). It also doesn't allow passing the mode, handle, and use_errno parameters. As to the library name, you may instead want to use find_library for Unix shared libraries: >>> ctypes.util.find_library('c') 'libc.so.6' >>> ctypes.util.find_library('python3.5m') 'libpython3.5m.so.1.0' It's pretty much pointless on Windows since it just searches PATH for the given name. It doesn't match how Windows LoadLibrary(Ex) really works, which allows precise control over exactly what gets searched via AddDllDirectory, SetDefaultDllDirectories, and activation contexts (i.e. ActivateActCtx). Plus find_library('c') no longer even works in 3.5+ on Windows (it returns None) because the CRT has been split into API sets (currently all mapped to ucrtbase.dll, but presumably that can change with servicing). From lawrencedo99 at gmail.com Tue Aug 2 02:46:03 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 1 Aug 2016 23:46:03 -0700 (PDT) Subject: ctypes Usage Note In-Reply-To: References: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> Message-ID: On Tuesday, August 2, 2016 at 5:18:14 PM UTC+12, eryk sun wrote: > I recommend using ctypes.CDLL instead. ctypes.cdll is pretty much > pointless on Unix systems (and a potential problem on Windows due to > cached function pointers). It also doesn't allow passing the mode, > handle, and use_errno parameters. None of which I currently care about. From honeygne at gmail.com Tue Aug 2 02:55:18 2016 From: honeygne at gmail.com (honeygne at gmail.com) Date: Mon, 1 Aug 2016 23:55:18 -0700 (PDT) Subject: Python text file fetch specific part of line In-Reply-To: References: <20160728070402.GA4848@cskk.homeip.net> Message-ID: <26f5f7b4-611c-45d6-a914-2e57672ee44b@googlegroups.com> On Thursday, July 28, 2016 at 1:00:17 PM UTC+5:30, c... at zip.com.au wrote: > On 27Jul2016 22:12, Arshpreet Singh wrote: > >I am writing Imdb scrapper, and getting available list of titles from IMDB > >website which provide txt file in very raw format, Here is the one part of > >file(http://pastebin.com/fpMgBAjc) as the file provides tags like Distribution > >Votes,Rank,Title I want to parse title names, I tried with readlines() method > >but it returns only list which is quite heterogeneous, is it possible that I > >can parse each value comes under title section? > > Just for etiquette: please just post text snippets like that inline in your > text. Some people don't like fetching random URLs, and some of us are not > always online when reading and replying to email. Either way, having the text > in the message, especially when it is small, is preferable. > > To your question: > > Your sample text looks like this: > > New Distribution Votes Rank Title > 0000000125 1680661 9.2 The Shawshank Redemption (1994) > 0000000125 1149871 9.2 The Godfather (1972) > 0000000124 786433 9.0 The Godfather: Part II (1974) > 0000000124 1665643 8.9 The Dark Knight (2008) > 0000000133 860145 8.9 Schindler's List (1993) > 0000000133 444718 8.9 12 Angry Men (1957) > 0000000123 1317267 8.9 Pulp Fiction (1994) > 0000000124 1209275 8.9 The Lord of the Rings: The Return of the King > (2003) > 0000000123 500803 8.9 Il buono, il brutto, il cattivo (1966) > 0000000133 1339500 8.8 Fight Club (1999) > 0000000123 1232468 8.8 The Lord of the Rings: The Fellowship of the > Ring (2001) > 0000000223 832726 8.7 Star Wars: Episode V - The Empire Strikes Back > (1980) > 0000000233 1243066 8.7 Forrest Gump (1994) > 0000000123 1459168 8.7 Inception (2010) > 0000000223 1094504 8.7 The Lord of the Rings: The Two Towers (2002) > 0000000232 676479 8.7 One Flew Over the Cuckoo's Nest (1975) > 0000000232 724590 8.7 Goodfellas (1990) > 0000000233 1211152 8.7 The Matrix (1999) > > Firstly, I would suggest you not use readlines(), it pulls all the text into > memory. For small text like this is it ok, but some things can be arbitrarily > large, so it is something to avoid if convenient. Normally you can just iterate > over a file and get lines. > > You want "text under the Title." Looking at it, I would be inclined to say that > the first line is a header and the rest consist of 4 columns: a number > (distribution?), a vote count, a rank and the rest (title plus year). > > You can parse data like that like this (untested): > > # presumes `fp` is reading from the text > for n, line in enumerate(fp): > if n == 0: > # heading, skip it > continue > distnum, nvotes, rank, etc = split(line, 3) > ... do stuff with the various fields ... > > I hope that gets you going. If not, return with what code you have, what > happened, and what you actually wanted to happen and we may help further. Thanks I am able to do it with following: https://github.com/alberanid/imdbpy/blob/master/bin/imdbpy2sql.py (it was very helpful) python imdbpy2sql.py -d <.txt files downloaded from IMDB> -u sqlite:/where/to/save/db --sqlite-transactions From antoon.pardon at rece.vub.ac.be Tue Aug 2 03:31:22 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 2 Aug 2016 09:31:22 +0200 Subject: Why not allow empty code blocks? In-Reply-To: <579c23de$0$1596$c3e8da3$5496439d@news.astraweb.com> References: <5792cd34$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <20160729071458.1dfa170f@imp> <579b5d8c$0$1615$c3e8da3$5496439d@news.astraweb.com> <579B603B.60102@rece.vub.ac.be> <579b6a68$0$1612$c3e8da3$5496439d@news.astraweb.com> <579BA15A.4090406@rece.vub.ac.be> <579c23de$0$1596$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57A04C4A.6070201@rece.vub.ac.be> Op 30-07-16 om 05:49 schreef Steven D'Aprano: > On Sat, 30 Jul 2016 04:32 am, Antoon Pardon wrote: > > > Perhaps its *you* who doesn't understand. The subject line talks > about "empty code blocks", and Bart has suggested that instead of having to > write > > ... > > So now you're changing your story, and claiming that you're not actually > replacing an existing except block with pass (because I think any > experienced Python developer, which Bart is not, realises that's not likely > to work). Now you've got a use-case where we start with: I am not changing my story, I am describing it from a different view point. > # before > some code > more processing > > and add an entire try...except around it: > > # step 1 > try: > some code > except SomeException: > debugging code > more processing > > *and then* comment out the debugging code and replace it with pass: > > # after, as stated by you > try: > some code > except SomeException: > # debugging code > pass > more processing No I start with writing the above code and the debugging code uncommented. > Of course it's legitimate to wrap some failing code in a temporary > try...except block for debugging purposes. But that's not an empty code > block, and its not "pass", and it's not relevant to Chris' question. You still don't seem to understand. This code doesn't fail. SomeException can normally safely be ignored. However some of the called functions might throw it wrongly. > And what happens once you have finished debugging the code? According to > your post, you comment out the debugging code, replacing it by "pass". And > you do this *frequently*. That depends what you mean with *frequently*. It depends on the nastyness of the bug I am chasing. > Okay, if you say you do that, I have to believe you. If we take you at your > word, then we can only conclude that your code is littered with the > detritus of old debugging code and unnecessary try...except blocks which do > nothing. That's practically the definition of *poor programming*, leaving > old code commented out to rot, but if you say that's what you do, who am I > to argue that you don't? You are not taking my word for it, you are jumping to conclusions. The fact that I do this at some point, doesn't mean I leave it in indefenatly. > Since I don't actually believe you are that poor a programmer, I suspect > that what you actually do is not replace the debugging code with "pass", > like you claim, but that once you have finished debugging that section of > the code you remove the unneeded try...except block and dead, unused > debugging code, returning it to the "before" state. But it isn't an unneeded try ... except block. I need the try ... except block to ignore an exception that should be harmless at that point. > D'Arcy's suggestion of leaving the debugging code in with a runtime switch > to turn it on and off also makes sense. Yes that makes sense. However if I think that is worth the while I usally just start using the logging module. -- Antoon From munozvvaleria at gmail.com Tue Aug 2 04:21:44 2016 From: munozvvaleria at gmail.com (munozvvaleria at gmail.com) Date: Tue, 2 Aug 2016 01:21:44 -0700 (PDT) Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 Message-ID: Hello, I am new to the programming world but I need to learn this program for a research class that I am taking. I have downloaded Python 3.6.0a3 on a Mac 10.9.5 and realized that I also need to download an Active Tcl for it. I can't find one for this version. Can someone please direct me to it? The most recent one I've been able to find is for Python 3.5.2 Thanks! Valeria From auriocus at gmx.de Tue Aug 2 04:29:29 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 2 Aug 2016 10:29:29 +0200 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: Message-ID: Am 02.08.16 um 10:21 schrieb munozvvaleria at gmail.com: > I am new to the programming world but I need to learn this program for a research class that I am taking. I have downloaded Python 3.6.0a3 on a Mac 10.9.5 and realized that I also need to download an Active Tcl for it. I can't find one for this version. Can someone please direct me to it? The most recent one I've been able to find is for Python 3.5.2 Tcl/Tk is independent from Python; any version should work. For best results on the Mac, try the most recent ActiveTcl from here http://www.activestate.com/activetcl/downloads/thank-you?dl=http://downloads.activestate.com/ActiveTcl/releases/8.6.4.1/ActiveTcl8.6.4.1.299124-macosx10.5-i386-x86_64-threaded.dmg Christian From arsh840 at gmail.com Tue Aug 2 05:14:23 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Tue, 2 Aug 2016 02:14:23 -0700 (PDT) Subject: holding if/else condition in live bitcoin trading Python Message-ID: <84adc952-262e-4a53-b749-0c3ecb09bf01@googlegroups.com> I am making a function that is running in while loop and if/else statements make decisions on based on the condition: here is code: def main_call(): while True: l0_0 = getDiff('btcusd',8) l1_0 = np.tanh(l0_0*0.8446488687) l1_1 = np.tanh(l0_0*-0.5674069006) l1_2 = np.tanh(l0_0*0.8676766445) if(l3_0>0): print l3_0 print auto_order_buy('0.01',str(ticker_last('btcusd'))) elif(l3_0<0): print l3_0 print auto_order_sell('0.01',str(ticker_last('btcusd'))) else: pass now in my function l3_0>0 auto_order_buy() is executed but as program is running in while loop so it gets executed again and again. I have implemented while loop because I am doing live trading. is there any approach I can go with so auto_order_buy() auto_order_sell() function only executed once? From andrea.botti at gmail.com Tue Aug 2 05:50:12 2016 From: andrea.botti at gmail.com (andrea.botti at gmail.com) Date: Tue, 2 Aug 2016 02:50:12 -0700 (PDT) Subject: Recursive csv import functions for Pandas Message-ID: <3ef3445d-ce42-4e22-a080-75dc24ff0933@googlegroups.com> I have put together the following code: ## DEFINES INPUT FILES inputcsvT = ['./input_csv/A08_KI_T*.csv', './input_csv/A08_LR_T*.csv', './input_csv/A08_B1_T*.csv',] #'./input_csv/A10_KI_T*.csv', './input_csv/A10_LR_T*.csv', './input_csv/A10_B1_T*.csv', #'./input_csv/A11_KI_T*.csv', './input_csv/A11_LR_T*.csv', './input_csv/A11_B1_T*.csv', #'./input_csv/A16_KI_T*.csv', './input_csv/A16_LR_T*.csv', './input_csv/A16_B1_T*.csv'] inputcsvR = ['./input_csv/A08_KI_R*.csv', './input_csv/A08_LR_R*.csv', './input_csv/A08_B1_R*.csv',] ## DEFINES FUNCTIONS FOR CSV INPUT, MERGE AND HOURLY RESAMPLING def csv_import_merge_T(f): dfsT = [pd.read_csv(fp, index_col=[0], parse_dates=[0], dayfirst=True, names=['datetime','temp','rh'], header=0) for fp in files] dfT = pd.concat(dfsT) dfT = dfT.drop('rh', 1) #dfT[~dfT.index.duplicated()] # replaced with function below dfT_clean = dfT.reset_index().drop_duplicates('datetime').set_index('datetime') dfTH = dfT_clean.resample('H').bfill() return dfTH def csv_import_merge_R(f): dfsR = [pd.read_csv(fp, index_col=[0], parse_dates=[0], dayfirst=True, names=['datetime','rad'], header=0) for fp in files] dfR = pd.concat(dfsR) #dfR[~dfR.index.duplicated()] # replaced with function below dfR_clean = dfR.reset_index().drop_duplicates('datetime').set_index('datetime') dfRH = dfR_clean.resample('H').mean() return dfRH ## PERFORMS FUNCTIONS FOR ALL Ts AND Rs AND CALCULATES HEATING DEGREE HOURS (HDH) AS R-T WHEN XX_XX_R = XX_XX_T for csvnameT in inputcsvT: files = glob.glob(csvnameT) print ('___'); print (files) csvT = csvnameT[12:20] print csvT t = csv_import_merge_T(files) t.to_csv('./output_csv/'+ csvT + '.csv') for csvnameR in inputcsvR: files = glob.glob(csvnameR) print ('___'); print (files) csvR = csvnameR[12:20] r = csv_import_merge_R(files) print csvR while csvnameR[12:18] == csvnameT[12:18]: print csvR + "=" + csvT r.to_csv('./output_csv/'+ csvR +'.csv') hdh = r.sub(t,axis=0).dropna() hdh[hdh['temp']<=1] = 0 csvHDH = csvnameT[12:18] hdh_week = hdh.temp.resample('W-MON').sum().round(decimals=0) hdh_week.to_csv('./output_csv/HDH/' + csvHDH + '_HDH.csv') The sequence of action should be: for A08_KI - import A08_KI_T1,T2,...Tn, merge them into A08_KI_T and resample them by the hour (room temp.) - import A08_KI_R1,R2,...Rn, merge them into A08_KI_R and resample them by the hour (radiator temp.) - every time XX_XX_R = XX_XX_T (i.e. A08_KI_T and A08_KI_R, or A10_LR_T and A10_LR_R) calculate HDH = R - T. The code I developed has something wrong and the command: while csvnameR[12:18] == csvnameT[12:18]: seems to cause the loop to go on for ever. From rosuav at gmail.com Tue Aug 2 06:21:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Aug 2016 20:21:40 +1000 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: Message-ID: On Tue, Aug 2, 2016 at 6:21 PM, wrote: > I am new to the programming world but I need to learn this program for a research class that I am taking. I have downloaded Python 3.6.0a3 on a Mac 10.9.5 and realized that I also need to download an Active Tcl for it. I can't find one for this version. Can someone please direct me to it? The most recent one I've been able to find is for Python 3.5.2 > To add to what Christian suggested: If you can't get something working using an alpha build of Python (that's an unreleased development version), download Python 3.5.2 instead. That's the current stable release. Using alphas opens you up to new bugs and incompatibilities, and generally isn't something I'd recommend for a new programmer. It's only fools like me who use unreleased software :) ChrisA From antoon.pardon at rece.vub.ac.be Tue Aug 2 06:30:23 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Tue, 2 Aug 2016 12:30:23 +0200 Subject: Why not allow empty code blocks? In-Reply-To: <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> References: <5792cd34$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> Message-ID: <57A0763F.7050809@rece.vub.ac.be> Op 30-07-16 om 18:15 schreef Rustom Mody: > > The more general baby that is significant is that beginners should have > it easy to distinguish procedure and function and python does not naturally aid that. print was something procedure-ish in python2 but the general notion being > absent is a much more significant problem (for beginners) than print. > > ... > > Ok Python is better than Java is better than C++ > But it cannot stand up to scheme as a teaching language > [The MIT Profs who replaced scheme by python admit to as much viz. But AFAIK scheme doesn't aid in distinguishing procedure from function either. -- Antoon. From bc at freeuk.com Tue Aug 2 07:05:51 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 12:05:51 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 31/07/2016 19:58, Terry Reedy wrote: > On 7/31/2016 6:18 AM, BartC wrote: >> repeat N: >> The benefit is not so much performance, but being able to express >> something very easily and quickly. > > The cost of the 'repeat' contraction is that one cannot use the loop > variable, either as part of a modified computation or for monitoring or > debugging purposes. > print(i, for_body_result) > Beginners are often atrocious at debugging, and it seems not to be > taught hardly at all. 'repeat n' erects a barrier to debugging. > Debugging: probing a computation to see what actually happens, as > opposed to what one wanted and expected to happen. (Me, just now ;-) > > One major way of debugging is printing values as they are computed. > Naming values (objects) allows them to be printed without recomputing > the value. In the 'repeat n' context, recomputing would mean adding 3 > lines of debugging code instead of 1. > > i = 0 > repeat n: > a = f(a) > print(i, a) > n += 1 Your objection to a feature such as 'repeat N' doesn't really stack up. Debugging code is stuff that you add temporarily to find out what's going on, then you get rid of it to leave the clean, unfettered lines of the program. You don't want people wondering where that loop index may or may not be used. Anyway, if that was a valid objection, it would apply throughout the language. In list-comps for example (there is an index, but where do you stick the print?). Or in a for-loop iterating over values: a=[10,20,30,40,50] for x in a: print (x) This will print the element, but what's the index? According to you, every such for-loop needs to be written in a form that provides the index of a loop, on the off-chance that someone might want to print its value somewhere in the body! ('for (i,x) in enumerate(a):') I get that people here don't want such a feature, but I don't think this is the reason. I think the real reason is not willing to admit that the language lacks something that could actually be useful, and especially not to an upstart on usenet who is not even an expert in that language. -- Bartc From uri at speedy.net Tue Aug 2 08:11:17 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 2 Aug 2016 15:11:17 +0300 Subject: Installing Python 3 on Windows Message-ID: Hi, I want to install Python 3 on Windows, but I also need Python 2 for Google App Engine SDK. When I type a name of a Python file in command line, I want it to run with Python 3. However, I checked with "print 3/5" and it printed 0 - Python 2. I have the latest versions of Python installed - python-2.7.12.msi and python-3.5.2.exe >python --version Python 2.7.12 >assoc .py .py=Python.File >ftype Python.File Python.File="C:\Python27\python.exe" "%1" %* By the way, it there a way to print the Python version from Python? Thanks, Uri. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ From eldiener at tropicsoft.invalid Tue Aug 2 08:21:46 2016 From: eldiener at tropicsoft.invalid (Edward Diener) Date: Tue, 2 Aug 2016 08:21:46 -0400 Subject: Installing Python 3 on Windows In-Reply-To: References: Message-ID: On 8/2/2016 8:11 AM, Uri Even-Chen wrote: > Hi, > > I want to install Python 3 on Windows, but I also need Python 2 for Google > App Engine SDK. When I type a name of a Python file in command line, I want > it to run with Python 3. However, I checked with "print 3/5" and it printed > 0 - Python 2. I have the latest versions of Python installed > - python-2.7.12.msi and python-3.5.2.exe > >> python --version > Python 2.7.12 > >> assoc .py > .py=Python.File > >> ftype Python.File > Python.File="C:\Python27\python.exe" "%1" %* > > By the way, it there a way to print the Python version from Python? Use the Python Launcher for Windows. From rustompmody at gmail.com Tue Aug 2 08:29:06 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 2 Aug 2016 05:29:06 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <5792cd34$0$1587$c3e8da3$5496439d@news.astraweb.com> <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> Message-ID: On Tuesday, August 2, 2016 at 4:01:25 PM UTC+5:30, Antoon Pardon wrote: > Op 30-07-16 om 18:15 schreef Rustom Mody: > > > > The more general baby that is significant is that beginners should have > > it easy to distinguish procedure and function and python does not naturally aid that. print was something procedure-ish in python2 but the general notion being > > absent is a much more significant problem (for beginners) than print. > > > > ... > > > > Ok Python is better than Java is better than C++ > > But it cannot stand up to scheme as a teaching language > > [The MIT Profs who replaced scheme by python admit to as much viz. > > But AFAIK scheme doesn't aid in distinguishing procedure from function either. True. And my words above seem to say that. However let me restore some more context: On Saturday, July 30, 2016 at 9:45:34 PM UTC+5:30, Rustom Mody wrote: > On Saturday, July 30, 2016 at 8:17:19 PM UTC+5:30, Steven D'Aprano wrote: > > On Sat, 30 Jul 2016 09:39 pm, Rustom Mody wrote: > > > Its a function? ok. > > > Its ?just? a function? Arguable > > > > "Granny Weatherwax, you are a natural-born disputant." > > "I ain't!" > > Heh I really aint :D > At least not for this dispute ? its not my baby > Or rather its a stepbaby of stepbaby > > Diff between > print "x" > and > print("x") > is one char ? the closing ?)? > > To make a dispute about that ? I?ll leave to BartC! > > The more general baby that is significant is that beginners should have > it easy to distinguish procedure and function and python does not naturally aid that. print was something procedure-ish in python2 but the general notion being > absent is a much more significant problem (for beginners) than print. > > Brings me to the even more general baby > But I studied in the 80s and there was greater clarity (about some matters > of course) than now. > eg It was completely natural that in ?school? one studied > ?nice? things like Pascal, Lisp, Prolog, Snobol, APL etc > And in a professional context used ?real? things like > Fortran, Cobol, PL-1 and a little later C. > > Once omnibus languages like C++, Java, C# and Python became popular > the academic vs real-world division has disappeared. > So beginners start with these ?real-world? > And get their brains scrambled > And think it wonderful > > Ok Python is better than Java is better than C++ > But it cannot stand up to scheme as a teaching language > [The MIT Profs who replaced scheme by python admit to as much viz. > MIT on practical reasons for python over scheme: > https://www.wisdomandwonder.com/link/2110/why-mit-switched-from-scheme-to-python > Berkeley on fundamental reasons for the opposite choice: > https://people.eecs.berkeley.edu/~bh/proglang.html So I was talking of 3 very different levels: 1. print x vs print(x) ? a difference too petty for me to waste my time with 2. Procedure vs Function as something very necessary for beginner thinking-ontology which Pascal gets right 3. The fact that the gap between a mainly-for-teaching language and a serious software-engineering-real-world language is not closable And that saying that the same language could be used for both purposes is like arguing that both these delightful ladies are pianists: Martha: https://www.youtube.com/watch?v=YLZLp6AcAi4 Rose : https://www.youtube.com/watch?v=_bjKDJD-CLc Scheme and Pascal happen to be two well-known well-crafted but quite different for-teaching languages From uri at speedy.net Tue Aug 2 08:44:04 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 2 Aug 2016 15:44:04 +0300 Subject: Installing Python 3 on Windows In-Reply-To: References: Message-ID: I don't understand. How do I use the Python Launcher? I didn't find such an option when installing Python 3. http://screencast.com/t/mBXfyMw4jpa http://screencast.com/t/FidjuB68aJ5G *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Tue, Aug 2, 2016 at 3:21 PM, Edward Diener wrote: > On 8/2/2016 8:11 AM, Uri Even-Chen wrote: > >> Hi, >> >> I want to install Python 3 on Windows, but I also need Python 2 for Google >> App Engine SDK. When I type a name of a Python file in command line, I >> want >> it to run with Python 3. However, I checked with "print 3/5" and it >> printed >> 0 - Python 2. I have the latest versions of Python installed >> - python-2.7.12.msi and python-3.5.2.exe >> >> python --version >>> >> Python 2.7.12 >> >> assoc .py >>> >> .py=Python.File >> >> ftype Python.File >>> >> Python.File="C:\Python27\python.exe" "%1" %* >> >> By the way, it there a way to print the Python version from Python? >> > > Use the Python Launcher for Windows. > > -- > https://mail.python.org/mailman/listinfo/python-list > From rustompmody at gmail.com Tue Aug 2 09:28:48 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 2 Aug 2016 06:28:48 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> Message-ID: > I think the real reason is not willing to admit that the language lacks > something that could actually be useful, and especially not to an > upstart on usenet who is not even an expert in that language. And earlier you said: > But dedicated forms (even if they just map to 'while' or 'for') wouldn't > hurt. Syntax is free after all, and it's about expressing exactly what > you mean. You are in effect repeating yourself What you are saying would be true if syntax were free ? or even cheap However a case may be made that syntax is one of the most wastefully expensive hobby with which computer scientists waste their time First some? Turing-completeness or Church-Turing thesis says that all languages are the same ? identical power. And yet in the last ? century of computers there is as chaotic a babel of computer languages as there is of natural languages in the last 5000 years of human history. IOW that L? and L? are same in power means that the difference is entirely in syntax Of course we need to understand syntax as - super-superficial ? lexical - superficial ? CFG grammar ? what is normally called ?syntax? - deeper but still static aspects eg type structure More at http://blog.languager.org/2015/04/poverty-universality-structure-0.html [Heh! I see it is related to another thread of yours!] Ok lets agree we need both C (like) and Python (like) languages because they serve very different purposes. But do we really need Python and Perl and Ruby and Tcl and Pike and ?? Or bash and zsh and ksh and powershell and ?? Now lets try to quantify the cost of this bedlam gcc is 15 million lines of code. How much is that in man-years? I guess somewhere between 100 and 1000 But that?s just the compiler One needs a development environment. Java folks tend towards eclipse C# towards Visual Studio Evidently eclipse is nearly 50 million lines of code(!!) https://dzone.com/articles/eclipse-indigo-released One may expect VS to be of comparable order. And as the ide-divide points out, choosing a fancy language implies choosing an underpowered IDE And vice-versa: http://blog.osteele.com/posts/2004/11/ides/ This is exactly the cost I am talking about: If people did not spend their time inventing newer and newer languages ? especially when that is just fiddling with syntax ? the IDEs and other necessary tooling that is needed for practical usage would have the chance to develop So to amplify Steven?s > No offense intended Bart, but for all we know every single one of your > benchmarks are faked, your interpreters are buggy pieces of garbage, and for > all your self-promotion If your language(s) gain enough traction as to contribute something beyond the language ? Great! All power to you! But if its just one more (misshapen) brick in the babel of computer languages then you are an anti-social element, of course not on a big scale like a terrorist? but let us say of the scale of someone who litters public spaces ? Stupid word! Those who use it should also be considered anti-social From uri at speedy.net Tue Aug 2 09:46:49 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 2 Aug 2016 16:46:49 +0300 Subject: Installing Python 3 on Windows In-Reply-To: References: Message-ID: I have the launcher installed and I configured it with a C:\Windows\py.ini file to run Python 3 when typing "py". But when I type the file name it runs Python 2. >py Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> exit() >python --version Python 2.7.12 >assoc .py .py=Python.File >ftype Python.File Python.File="C:\Python27\python.exe" "%1" %* How do I associate .py files with the launcher? I read this http://stackoverflow.com/questions/5087831/how-should-i-set-default-python-version *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Tue, Aug 2, 2016 at 3:44 PM, Uri Even-Chen wrote: > I don't understand. How do I use the Python Launcher? I didn't find such > an option when installing Python 3. http://screencast.com/t/mBXfyMw4jpa > http://screencast.com/t/FidjuB68aJ5G > > > *Uri Even-Chen* > [image: photo] Phone: +972-54-3995700 > Email: uri at speedy.net > Website: http://www.speedysoftware.com/uri/en/ > > > > > On Tue, Aug 2, 2016 at 3:21 PM, Edward Diener > wrote: > >> On 8/2/2016 8:11 AM, Uri Even-Chen wrote: >> >>> Hi, >>> >>> I want to install Python 3 on Windows, but I also need Python 2 for >>> Google >>> App Engine SDK. When I type a name of a Python file in command line, I >>> want >>> it to run with Python 3. However, I checked with "print 3/5" and it >>> printed >>> 0 - Python 2. I have the latest versions of Python installed >>> - python-2.7.12.msi and python-3.5.2.exe >>> >>> python --version >>>> >>> Python 2.7.12 >>> >>> assoc .py >>>> >>> .py=Python.File >>> >>> ftype Python.File >>>> >>> Python.File="C:\Python27\python.exe" "%1" %* >>> >>> By the way, it there a way to print the Python version from Python? >>> >> >> Use the Python Launcher for Windows. >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > From eryksun at gmail.com Tue Aug 2 09:50:36 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 2 Aug 2016 13:50:36 +0000 Subject: Installing Python 3 on Windows In-Reply-To: References: Message-ID: On Tue, Aug 2, 2016 at 12:11 PM, Uri Even-Chen wrote: > > I want to install Python 3 on Windows, but I also need Python 2 for Google > App Engine SDK. When I type a name of a Python file in command line, I want > it to run with Python 3. However, I checked with "print 3/5" and it printed > 0 - Python 2. I have the latest versions of Python installed > - python-2.7.12.msi and python-3.5.2.exe > >>python --version > Python 2.7.12 > >>assoc .py > .py=Python.File > >>ftype Python.File > Python.File="C:\Python27\python.exe" "%1" %* Open the control panel's "Programs and Features". Look for the most recently installed "Python Launcher"; right-click it and select "Repair". That should fix the Python.File and Python.NoConFile commands. If these commands aren't your current choice in Explorer, open Default Programs->Set Associations. Change the association for .py and .pyw to "Python". If there are multiple "Python" entries in the list, the icon of the correct one should have a rocket (a launcher) in the upper left-hand corner. The launcher handles shebangs and even accepts common Unix paths such as /usr/bin, e.g.: Latest 2.x: #! /usr/bin/python Latest 3.x: #! /usr/bin/python3 Without a shebang the launcher currently defaults to Python 2 if it's installed. To always use Python 3 as the default, set the environment variable `PY_PYTHON=3`. > By the way, i[s] there a way to print the Python version from Python? sys.version is the version string. sys.version_info is a named sequence of version information: >>> sys.version_info sys.version_info(major=2, minor=7, micro=12, releaselevel='final', serial=0) >>> sys.version_info[:] (2, 7, 12, 'final', 0) >>> sys.version_info sys.version_info(major=3, minor=5, micro=2, releaselevel='final', serial=0) >>> sys.version_info[:] (3, 5, 2, 'final', 0) From uri at speedy.net Tue Aug 2 10:12:33 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 2 Aug 2016 17:12:33 +0300 Subject: Installing Python 3 on Windows In-Reply-To: References: Message-ID: Thank you, repair worked! I already have py.ini and don't want to add shebangs, I have at least 50 Python files and I don't want to edit all of them. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Tue, Aug 2, 2016 at 4:50 PM, eryk sun wrote: > On Tue, Aug 2, 2016 at 12:11 PM, Uri Even-Chen wrote: > > > > I want to install Python 3 on Windows, but I also need Python 2 for > Google > > App Engine SDK. When I type a name of a Python file in command line, I > want > > it to run with Python 3. However, I checked with "print 3/5" and it > printed > > 0 - Python 2. I have the latest versions of Python installed > > - python-2.7.12.msi and python-3.5.2.exe > > > >>python --version > > Python 2.7.12 > > > >>assoc .py > > .py=Python.File > > > >>ftype Python.File > > Python.File="C:\Python27\python.exe" "%1" %* > > Open the control panel's "Programs and Features". Look for the most > recently installed "Python Launcher"; right-click it and select > "Repair". That should fix the Python.File and Python.NoConFile > commands. If these commands aren't your current choice in Explorer, > open Default Programs->Set Associations. Change the association for > .py and .pyw to "Python". If there are multiple "Python" entries in > the list, the icon of the correct one should have a rocket (a > launcher) in the upper left-hand corner. > > The launcher handles shebangs and even accepts common Unix paths such > as /usr/bin, e.g.: > > Latest 2.x: #! /usr/bin/python > Latest 3.x: #! /usr/bin/python3 > > Without a shebang the launcher currently defaults to Python 2 if it's > installed. To always use Python 3 as the default, set the environment > variable `PY_PYTHON=3`. > > > By the way, i[s] there a way to print the Python version from Python? > > sys.version is the version string. sys.version_info is a named > sequence of version information: > > >>> sys.version_info > sys.version_info(major=2, minor=7, micro=12, releaselevel='final', > serial=0) > >>> sys.version_info[:] > (2, 7, 12, 'final', 0) > > >>> sys.version_info > sys.version_info(major=3, minor=5, micro=2, releaselevel='final', > serial=0) > >>> sys.version_info[:] > (3, 5, 2, 'final', 0) > From robin at reportlab.com Tue Aug 2 10:42:53 2016 From: robin at reportlab.com (Robin Becker) Date: Tue, 2 Aug 2016 07:42:53 -0700 (PDT) Subject: gmane sending Message-ID: <9d543a2a-c56a-46cc-971d-2124782d5d1f@googlegroups.com> I just got a mail bounce from my normal gmane --> nntp setup sending to python-python-list at m.gmane.org. Have others seen this and does it mean the end of gmane has happened? See https://developers.slashdot.org/story/16/07/28/2059249/the-end-of-gmane -- Robin Becker From rosuav at gmail.com Tue Aug 2 10:58:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 00:58:58 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Aug 2, 2016 at 9:05 PM, BartC wrote: > I think the real reason is not willing to admit that the language lacks > something that could actually be useful, and especially not to an upstart on > usenet who is not even an expert in that language. I know what features I miss from the languages I use most frequently. Here, let me help you out a bit with some examples: Python - Support for live code reloads without restarting the process - A more free-form declarative syntax for laying out GUI code - Arbitrary-precision non-integers - Convenient syntax for a few array/list manipulations - Truly concurrent threads - Extreme short-hand for executing external commands, the way REXX does Pike - Keyword arguments - "obj in collection" syntax for membership testing - Better documentation in places - Out-of-the-box GTK support on OSX - As with Python, external command execution shorthand Notice that I did not put "Bracey syntax" under Python, nor "Braceless syntax" under Pike, despite them not having those options. Notice also that I didn't put "simpler loop syntax" in either; Python's loops you know about, and Pike gives you the C-style "for (int i=0; i<10; ++i)" form and "foreach (collection, item)" like Python's "for item in collection". The only thing I might yearn for - *MIGHT* - would be for Python to gain an "index-and-value" iteration form like Pike's "foreach (collection; index; value)", which for dictionaries would be like iterating over .items(), and for lists would be like using enumerate(). But I've never actually found myself yearning for it while I'm writing Python code - it's a small convenience when I'm working in Pike, is all. Does Python "lack" the simple repeat statement? Well, in the sense that it doesn't have it, sure. But Python also doesn't have a single function to read a line from a gzipped file and strip HTML tags from it before returning it [1]. Not everything that doesn't exist is needed. ChrisA [1] http://php.net/manual/en/function.gzgetss.php From robin at reportlab.com Tue Aug 2 11:12:47 2016 From: robin at reportlab.com (Robin Becker) Date: Tue, 2 Aug 2016 08:12:47 -0700 (PDT) Subject: make an object read only Message-ID: A reportlab user found he was doing the wrong thing by calling canvas.save repeatedly, our documentation says you should not use Canvas objects after the save method has been used. The user had mixed results :( It would be better to make the canvas object completely immutable all the way down when save has been called, but I think that's quite hard as these objects have quite a large and varied set of attributes, lists other objects dictionaries etc etc. I can think of some strategies for making the object unusable eg changing it's class and getting rid of __dict__, but that disallows referencing valid properties eg pagesize, fontName, etc etc. Is there a way to recursively turn everything immutable? -- Robin Becker From steve+python at pearwood.info Tue Aug 2 12:11:48 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 02:11:48 +1000 Subject: holding if/else condition in live bitcoin trading Python References: <84adc952-262e-4a53-b749-0c3ecb09bf01@googlegroups.com> Message-ID: <57a0c646$0$1612$c3e8da3$5496439d@news.astraweb.com> On Tue, 2 Aug 2016 07:14 pm, Arshpreet Singh wrote: > is there any approach I can go with so auto_order_buy() auto_order_sell() > function only executed once? Take them out of the while loop. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dfh at forestfield.co.uk Tue Aug 2 12:28:15 2016 From: dfh at forestfield.co.uk (dfh at forestfield.co.uk) Date: Tue, 2 Aug 2016 09:28:15 -0700 (PDT) Subject: make an object read only In-Reply-To: References: Message-ID: On Tuesday, 2 August 2016 16:13:01 UTC+1, Robin Becker wrote: > A reportlab user found he was doing the wrong thing by calling canvas.save > repeatedly, our documentation says you should not use Canvas objects after the > save method has been used. The user had mixed results :( > > It would be better to make the canvas object completely immutable all the way > down when save has been called, ...... > > Is there a way to recursively turn everything immutable? > -- > Robin Becker Years ago I contributed a recipe to the O'Reilly Python Cookbook, 2nd Ed - 6.12 Checking an Instance for any State Changes. My original submission was rather more naive than that but Alex Martelli, one of the editors, rather took a fancy to the idea and knocked into the more presentable shape that got published. I'm wondering whether there would be any way of turning this idea around to achieve what you want. -- Regards David Hughes Forestfield Software From steve+python at pearwood.info Tue Aug 2 12:36:30 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 02:36:30 +1000 Subject: make an object read only References: Message-ID: <57a0cc10$0$1610$c3e8da3$5496439d@news.astraweb.com> On Wed, 3 Aug 2016 01:12 am, Robin Becker wrote: > A reportlab user found he was doing the wrong thing by calling canvas.save > repeatedly, our documentation says you should not use Canvas objects after > the save method has been used. The user had mixed results :( > > It would be better to make the canvas object completely immutable all the > way down when save has been called, but I think that's quite hard as these > objects have quite a large and varied set of attributes, lists other > objects dictionaries etc etc. If save() is the only problematic method, that's easy to fix: class X: def _bad_save(self): raise RuntimeError("I told you not to call save() more than once!") def save(self): # do the actual saving # then replace the method: self.save = self._bad_save Hmmm... clever as that is, it's probably not clever enough, as users can still call the unbound save method: X.save(instance) So back to the really old-fashioned way: class X: def __init__(self): self._saved = False def save(self): if self._saved: raise RuntimeError("*wack with clue stick*") else: # do the actual saving self._saved = True > I can think of some strategies for making the object unusable eg changing > it's class and getting rid of __dict__, but that disallows referencing > valid properties eg pagesize, fontName, etc etc. It shouldn't disallow anything. class X: # implementation of everything, properties, etc. # as they are expected to work before saving def save(self): # do the actual save self.__class__ = SavedX class SavedX(X): # Override just the bits that need neutering # including save def save(self): raise RuntimeError("stop that!") The tricky bit may be preventing writes to the instance __dict__. I don't think it is practical to allow the user to add arbitrary attributes to the instance up to the point they call save(), then prevent them from doing the same. You might be able to get it to work, but with difficulty. Better to fix save() so you can call it multiple times without breaking the instance. If you can't do that, the least-worst alternative would be to use __slots__ for both X and SavedX so that the user can't add new attributes at all. But really, the right way to do this is to fix the class so that save() can be called multiple times. In your editor, does selecting Save twice corrupt the file you're working on? > Is there a way to recursively turn everything immutable? First you have to come up with a way to turn everything immutable. (Good luck with that.) Then apply it recursively to the object and all its attributes. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From abreed1 at maine.rr.com Tue Aug 2 12:41:52 2016 From: abreed1 at maine.rr.com (abreed1 at maine.rr.com) Date: Tue, 2 Aug 2016 12:41:52 -0400 Subject: problem installinf python Message-ID: <20160802164152.298IQ.271352.root@dnvrco-web26> Hi. I'm having trouble installing Python on my computer. My computer is a 64-bit Dell desktop running Windows 8.1. A screen message claimed that I installed Python 3.5.2 (64 bit), but when I try to run it, I get only the "Modify Setup" screen, with the only choices being "Modify," "Repair," or "Uninstall." I have run the "Repair" option, but that does not change anything. Can anyone help me install Python? Alex From bc at freeuk.com Tue Aug 2 12:56:04 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 17:56:04 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> Message-ID: On 02/08/2016 14:28, Rustom Mody wrote: >> I think the real reason is not willing to admit that the language lacks >> something that could actually be useful, and especially not to an >> upstart on usenet who is not even an expert in that language. > However a case may be made that syntax is one of the most wastefully expensive > hobby with which computer scientists waste their time > > First some? > > Turing-completeness or Church-Turing thesis says that all languages are the same ? > identical power. And yet in the last ? century of computers there is as chaotic a babel > of computer languages as there is of natural languages in the last 5000 years of human > history. Who's talking about creating a new language? I was discussing tweaking an existing one. > Ok lets agree we need both C (like) and Python (like) languages > because they serve very different purposes. > > But do we really need Python and Perl and Ruby and Tcl and Pike and ?? > Or bash and zsh and ksh and powershell and ?? You really want my opinion? For my own use, I could use 2 or 3 languages at different levels: 1 C-like 2 Intermediate 3 Python-like 4 Scripting (I have my own versions of 1 and 2, but not of 3.) Other people will have different lists, and with different languages. C++ is quite popular (but so complex that it is unusable IMO). Others might like the discipline of Ada. A few might like to code in Haskell. Or it might need a small footprint. Or work on certain platforms. Or needs to be fast.. So there are diverse requirements. But I agree we don't need thousands of languages, each inventing different syntax too. Perhaps someone can just invent or choose syntax styles A, B and C and a language can adopt one of those (this is the approach I use actually). > Now lets try to quantify the cost of this bedlam > > gcc is 15 million lines of code. How much is that in man-years? > I guess somewhere between 100 and 1000 > But that?s just the compiler Well, it also supports lots of targets, which accounts for some of the size. But gcc would still be huge. > One needs a development environment. > Java folks tend towards eclipse > C# towards Visual Studio > > Evidently eclipse is nearly 50 million lines of code(!!) OK. 50 million lines. Keep that in mind. > https://dzone.com/articles/eclipse-indigo-released > > One may expect VS to be of comparable order. I heard that VS comprises 1.5 million *files*. And a full compile takes 60 *hours*. So the shift has been from evolving languages to horrendously large and complex development *tools*. (And I expect that next they will eliminate languages altogether. All you need is some way of specifying a sequence of calls to library functions and sprinkling around some control statements; it could be drag-and-drop with a GUI display of the program flow.) I don't have any interest in that. 1.5M files or 50M lines for a tool cannot possibly be right. And I believe in using a traditional language with textual syntax and not being overly dependent on extraneous tools so massive that they completely the language they are supposed to work with. > And as the ide-divide points out, choosing a fancy language implies choosing an underpowered IDE > And vice-versa: > http://blog.osteele.com/posts/2004/11/ides/ > > This is exactly the cost I am talking about: > If people did not spend their time inventing newer and newer > languages ? especially when that is just fiddling with syntax ? > the IDEs and other necessary tooling that is needed for practical > usage would have the chance to develop Your figures suggest the opposite! So people spend time creating huge bloated IDEs instead of writing 100 lines of a parser so that other people can avoid writing 'for i in range(N):' millions of times instead of 'for N:' or 'repeat N:'. > So to amplify Steven?s >> No offense intended Bart, but for all we know every single one of your >> benchmarks are faked, your interpreters are buggy pieces of garbage, and for >> all your self-promotion > > If your language(s) gain enough traction as to contribute something beyond the > language ? Great! All power to you! > > But if its just one more (misshapen) brick in the babel of > computer languages then you are an anti-social element, Anti-social, yes. When I was doing most of my stuff in 80s, my 'brick' was the only one. It didn't need to fit to anything else! However such a brick can still be tremendously useful in the 2010s, just like a bicycle can be in city traffic. It's small, manageable, understandable, and can go anywhere. And it can be very nippy. (Very nippy indeed. If I say how long it takes me to build my compiler from scratch - for my C replacement language - no one would believe me.) -- Bartc From stephane at wirtel.be Tue Aug 2 13:03:38 2016 From: stephane at wirtel.be (Stephane Wirtel) Date: Tue, 2 Aug 2016 19:03:38 +0200 Subject: problem installinf python In-Reply-To: <20160802164152.298IQ.271352.root@dnvrco-web26> References: <20160802164152.298IQ.271352.root@dnvrco-web26> Message-ID: <20160802170338.t74bzibuspbevaoz@sg1> And if you uninstall it and reinstall it after ? On 08/02, abreed1 at maine.rr.com wrote: > Hi. I'm having trouble installing Python on my computer. My computer is a 64-bit Dell desktop running Windows 8.1. > A screen message claimed that I installed Python 3.5.2 (64 bit), but when I try to run it, I get only the "Modify >Setup" screen, with the only choices being "Modify," "Repair," or "Uninstall." I have run the "Repair" option, but that does not change anything. Can anyone help me install Python? > Alex > >-- >https://mail.python.org/mailman/listinfo/python-list -- St?phane Wirtel - http://wirtel.be - @matrixise From python at mrabarnett.plus.com Tue Aug 2 13:08:43 2016 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 2 Aug 2016 18:08:43 +0100 Subject: Recursive csv import functions for Pandas In-Reply-To: <3ef3445d-ce42-4e22-a080-75dc24ff0933@googlegroups.com> References: <3ef3445d-ce42-4e22-a080-75dc24ff0933@googlegroups.com> Message-ID: <5c6cdfef-fb92-4c36-e935-f1ace82abb1b@mrabarnett.plus.com> On 2016-08-02 10:50, andrea.botti at gmail.com wrote: > I have put together the following code: > [snip] > while csvnameR[12:18] == csvnameT[12:18]: > print csvR + "=" + csvT > > r.to_csv('./output_csv/'+ csvR +'.csv') > > hdh = r.sub(t,axis=0).dropna() > hdh[hdh['temp']<=1] = 0 > csvHDH = csvnameT[12:18] > hdh_week = hdh.temp.resample('W-MON').sum().round(decimals=0) > hdh_week.to_csv('./output_csv/HDH/' + csvHDH + '_HDH.csv') > > The sequence of action should be: > for A08_KI > - import A08_KI_T1,T2,...Tn, merge them into A08_KI_T and resample them by the hour (room temp.) > - import A08_KI_R1,R2,...Rn, merge them into A08_KI_R and resample them by the hour (radiator temp.) > - every time XX_XX_R = XX_XX_T (i.e. A08_KI_T and A08_KI_R, or A10_LR_T and A10_LR_R) calculate HDH = R - T. > > The code I developed has something wrong and the command: > > while csvnameR[12:18] == csvnameT[12:18]: > seems to cause the loop to go on for ever. > Look at the body of that loop. Nowhere in it do you change the value of 'csvnameR' or 'csvnameT', so if the condition was true initially, it'll still be true the next time round, and forever more. From bc at freeuk.com Tue Aug 2 13:12:14 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 18:12:14 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 02/08/2016 15:58, Chris Angelico wrote: > On Tue, Aug 2, 2016 at 9:05 PM, BartC wrote: >> I think the real reason is not willing to admit that the language lacks >> something that could actually be useful, and especially not to an upstart on >> usenet who is not even an expert in that language. > > I know what features I miss from the languages I use most frequently. > Here, let me help you out a bit with some examples: > > Python > - Support for live code reloads without restarting the process If that means what I think it means (ie. effectively re-doing an import statement) then I agree. I think I used a similar feature in the past, but with limitations (the module had one only entry point accessible from the code that imports it). It was used for developing much of an application not only without restarting it, but from inside the application (effectively using it as an IDE). Very, very useful. > that I didn't put "simpler loop syntax" in either; Python's loops you > know about, and Pike gives you the C-style "for (int i=0; i<10; ++i)" You've just hit on another bugbear of mine. I detest that form of loop! > Does Python "lack" the simple repeat statement? Well, in the sense > that it doesn't have it, sure. But Python also doesn't have a single > function to read a line from a gzipped file and strip HTML tags from > it before returning it [1]. Not everything that doesn't exist is > needed. That's not a fundamental language feature. Repeat-N is. And if properly designed, isn't an extra feature at all but a special case of a generic loop. -- Bartc From python at mrabarnett.plus.com Tue Aug 2 13:18:09 2016 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 2 Aug 2016 18:18:09 +0100 Subject: problem installinf python In-Reply-To: <20160802164152.298IQ.271352.root@dnvrco-web26> References: <20160802164152.298IQ.271352.root@dnvrco-web26> Message-ID: <3ea366f1-d102-e407-65ff-67e4ab38e0de@mrabarnett.plus.com> On 2016-08-02 17:41, abreed1 at maine.rr.com wrote: > Hi. I'm having trouble installing Python on my computer. My computer is a 64-bit Dell desktop running Windows 8.1. > A screen message claimed that I installed Python 3.5.2 (64 bit), but when I try to run it, I get only the "Modify > Setup" screen, with the only choices being "Modify," "Repair," or "Uninstall." I have run the "Repair" option, but that does not change anything. Can anyone help me install Python? > Alex > It sounds to me like you're trying to run the installer again. From steve+python at pearwood.info Tue Aug 2 13:50:09 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 03:50:09 +1000 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> Message-ID: <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> On Tue, 2 Aug 2016 11:28 pm, Rustom Mody wrote: >> I think the real reason is not willing to admit that the language lacks >> something that could actually be useful, and especially not to an >> upstart on usenet who is not even an expert in that language. > > And earlier you said: > >> But dedicated forms (even if they just map to 'while' or 'for') wouldn't >> hurt. Syntax is free after all, and it's about expressing exactly what >> you mean. > > You are in effect repeating yourself > What you are saying would be true if syntax were free ? or even cheap > > However a case may be made that syntax is one of the most wastefully > expensive hobby with which computer scientists waste their time > > First some? > > Turing-completeness or Church-Turing thesis says that all languages are > the same ? identical power. That depends on how you define the power of a language. Earlier, you wrote: "Ok Python is better than Java is better than C++ But it cannot stand up to scheme as a teaching language" So you are aware that languages are *not* the same. Languages have more or less "power", where power is defined (rather fuzzily) as the expressiveness of the language, how easy it is for the programmer to read, write and maintain code, how efficient/fast you can implement it, etc. The power of a language is rather subjective, but we all surely acknowledge that adding new features to a language can make it more powerful: - Python 1.5 is less powerful than Python 3.5; - Fortran 77 is less powerful than Fortran 95. We acknowledge that languages can differ in power too: - 1970s BASIC is less powerful than C; - COBOL is less powerful than Scheme. Google for the Blub Paradox, by Paul Graham, for more information. So language are not all the same, and can differ in power. So what does the Church-Turing thesis say? It doesn't say anything about language power in the above sense. Language power is a rather subjective, fuzzy, non-linear measure, which is why language flame wars will never end. What the C-T thesis talks about is *computability*. *What* can you compute, not necessarily how easy it is to compute it. And there is a hierarchy of language "power" in this sense. (A Turing Machine can do anything that Conway's Game Of Life can do, but is easier to program; assembly language is easier still, and Javascript even more so.) A good discussion of the Church-Turing thesis can be found in Chapter 13 of Hofstadter's "G?del, Escher, Bach". He introduces three languages that differ in computing power in the computability sense: Bloop, Floop and Gloop. - Bloop can perform calculations that predictably terminate; that is, it can do any calculation which can be performed by a primitive recursive function; - Floop can perform calculations that have no upper bounds to the number of steps needed; that is, it can perform *unbounded* potentially non-terminating loops; another way of saying this is that terminating Floop programs are general recursive, and non-terminating Floop programs are partial recursive; - Gloop can perform calculations which never terminate in Floop, and do so in a finite number of steps. In simple terms, Bloop has for-loops; Floop has while-loops; and Gloop, well, Gloop is magical and probably doesn't exist. "Gloop is a myth" is one way of putting the Church-Turing Thesis. Hofstadter gives three equivalent ways to state the C-T Thesis: (1) What is human-computable is machine-computable; (2) What is machine-computable is Floop-computable; (3) What is human-computable is Floop-computable (i.e., general or partial recursive). Most interesting and useful languages are equivalent to Floop, but there are languages (potential, if not actual) which are equivalent only to Bloop. Pure regular expressions, for example, are not Turing Complete: they can only perform calculations which are primitive recursive. The Halting Problem is easily solved for Bloop languages: they always halt. > And yet in the last ? century of computers > there is as chaotic a babel of computer languages as there is of natural > languages in the last 5000 years of human history. I think that human languages have been *far* more complicated. The history of English *alone* is probably as complex and chaotic as that of programming languages. My guess is that once you go down to the level of regional dialects, there are more variants of English alone than programming languages have existed. In some sense, virtually every village and hamlet in Great Britain has had its own version of English (and sometimes not even mutually comprehensible). > IOW that L? and L? are same in power means that the difference is entirely > in syntax But as we've seen, syntax can make a HUGE difference to power in the sense of expressiveness, maintainability of code, readability, efficiency of the programmer, and even efficiency of the interpreter. Conway's Game of Life is Turing Complete. Would you rather use Python, or Game of Life? BrainF*ck or Javascript? [...] > But do we really need Python and Perl and Ruby and Tcl and Pike and ?? > Or bash and zsh and ksh and powershell and ?? Yes. Programming languages are tools for solving problems, but they are also tools for thinking about problems and communicating between people. There are many ways to think about problems, many ways to communicate between them, and no clear or practical way to decide what is "best". That's why language flame-wars occur! With no objective way to decide which language to use, there will always be people who are dissatisfied with the existing languages they have available to them. Look at yourself: you are dissatisfied with Python's lack of support for Unicode operators. But that's "just syntax". And yet you are convinced that Python would be a better language if it supported more mathematical notation with Unicode operators. And perhaps you are right. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Aug 2 13:54:35 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 03:54:35 +1000 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> Message-ID: <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> On Wed, 3 Aug 2016 02:56 am, BartC wrote: > (And I expect that next they will eliminate languages altogether. All > you need is some way of specifying a sequence of calls to library > functions and sprinkling around some control statements; That would be called "a language". > it could be > drag-and-drop with a GUI display of the program flow.) It could be, but won't be. Outside of a very few tiny niches, including Squeak which is designed for children, such user-interfaces are far too cumbersome to ever get widespread use. But for limited niches, like generating GUIs, form designers, or even assembling regular expressions, sure, why not? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Aug 2 13:57:05 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 03:57:05 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> On Wed, 3 Aug 2016 03:12 am, BartC wrote: > That's not a fundamental language feature. Repeat-N is. And if properly > designed, isn't an extra feature at all but a special case of a generic > loop. Which means it is NOT a fundamental language feature. "Repeat N without tracking the loop variable" is just a special case of "repeat N with tracking the loop variable", where you don't actually care what the loop variable is. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Tue Aug 2 15:08:50 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 2 Aug 2016 15:08:50 -0400 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: Message-ID: On 8/2/2016 4:21 AM, munozvvaleria at gmail.com wrote: > I am new to the programming world but I need to learn this program > for a research class that I am taking. I have downloaded Python > 3.6.0a3 on a Mac 10.9.5 and realized that I also need to download an > Active Tcl for it. I can't find one for this version. Can someone > please direct me to it? The most recent one I've been able to find is > for Python 3.5.2 This page https://www.python.org/download/mac/tcltk/ does not yet mention 3.6. So I would start with the recommendation for 3.5.2: tcl/tk 8.5.18. I believe Ned Deily hopes to switch to tcl/tk 8.6.x before 3.6.0 is released. If 8.5.18 does not work, I would try 8.6.whatever. But I don't believe the switch has been made yet. -- Terry Jan Reedy From rosuav at gmail.com Tue Aug 2 15:10:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 05:10:07 +1000 Subject: Why not allow empty code blocks? In-Reply-To: <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 3, 2016 at 3:54 AM, Steven D'Aprano wrote: > On Wed, 3 Aug 2016 02:56 am, BartC wrote: > >> (And I expect that next they will eliminate languages altogether. All >> you need is some way of specifying a sequence of calls to library >> functions and sprinkling around some control statements; > > That would be called "a language". :) >> it could be >> drag-and-drop with a GUI display of the program flow.) > > It could be, but won't be. Outside of a very few tiny niches, including > Squeak which is designed for children, such user-interfaces are far too > cumbersome to ever get widespread use. > > But for limited niches, like generating GUIs, form designers, or even > assembling regular expressions, sure, why not? They do exist, and IMO they're languages just as much as textual ones are. You don't eliminate computer language by switching to a GUI drag-and-drop system than you eliminate human language by using hand signs. In fact, deaf people generally communicate using something called "{American,Korean,Australian,...} Sign Language" - it's not called "American Way of Talking Without Talking Or Using Language". ChrisA From bc at freeuk.com Tue Aug 2 15:14:12 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 20:14:12 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 02/08/2016 18:57, Steven D'Aprano wrote: > On Wed, 3 Aug 2016 03:12 am, BartC wrote: > >> That's not a fundamental language feature. Repeat-N is. And if properly >> designed, isn't an extra feature at all but a special case of a generic >> loop. > > Which means it is NOT a fundamental language feature. > > "Repeat N without tracking the loop variable" is just a special case > of "repeat N with tracking the loop variable", where you don't actually > care what the loop variable is. It's fundamental in that, when giving instructions or commands in English, it frequently comes up when you want something done a set number of times: "Give me 20 push-ups" "Press space 5 times" "Do 10 laps" Whoever has to execute these may need to keep count somehow, but that is not the concern of the person giving the commands. You wouldn't say, count from 1 to 20, and for each value in turn, do a push-up. You could also say count backwards from 95 to 0 in fives; same effect. There are so many ways of specifying a loop that is executed 20 times, that no one way can be the right one. So that extra information is irrelevant. -- Bartc From tjreedy at udel.edu Tue Aug 2 15:14:25 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 2 Aug 2016 15:14:25 -0400 Subject: gmane sending In-Reply-To: <9d543a2a-c56a-46cc-971d-2124782d5d1f@googlegroups.com> References: <9d543a2a-c56a-46cc-971d-2124782d5d1f@googlegroups.com> Message-ID: On 8/2/2016 10:42 AM, Robin Becker wrote: > I just got a mail bounce from my normal gmane --> nntp setup sending to python-python-list at m.gmane.org. Have others seen this and does it mean the end of gmane has happened? See I am reading and sending via gmane right now. > https://developers.slashdot.org/story/16/07/28/2059249/the-end-of-gmane I really hope not. -- Terry Jan Reedy From rosuav at gmail.com Tue Aug 2 15:18:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 05:18:50 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 3, 2016 at 3:57 AM, Steven D'Aprano wrote: > On Wed, 3 Aug 2016 03:12 am, BartC wrote: > >> That's not a fundamental language feature. Repeat-N is. And if properly >> designed, isn't an extra feature at all but a special case of a generic >> loop. > > Which means it is NOT a fundamental language feature. > > "Repeat N without tracking the loop variable" is just a special case > of "repeat N with tracking the loop variable", where you don't actually > care what the loop variable is. To be fair, that can be an important distinction. I don't like seeing code like this: def f(): x = g() y = h(x) return 5 + x What's y for? Why not just call h(x) and ignore its return value? Should the last line say "5 + y"? It looks _wrong_ to have a local variable that's never used. It's still not a consideration strong enough to demand new loop syntax, though. ChrisA From bc at freeuk.com Tue Aug 2 15:19:13 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 20:19:13 +0100 Subject: Why not allow empty code blocks? In-Reply-To: <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 02/08/2016 18:54, Steven D'Aprano wrote: > On Wed, 3 Aug 2016 02:56 am, BartC wrote: > >> (And I expect that next they will eliminate languages altogether. All >> you need is some way of specifying a sequence of calls to library >> functions and sprinkling around some control statements; > > That would be called "a language". No, it wouldn't be given its own identity. And it probably couldn't be used without that specific tool. It would be more like a file format, if the details are even exposed when the format is proprietary. A traditional language exists as text and can be stored as plain text. -- Bartc From no.email at nospam.invalid Tue Aug 2 15:22:46 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 02 Aug 2016 12:22:46 -0700 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87a8gv562h.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > where power is defined (rather fuzzily) as the expressiveness > of the language, how easy it is for the programmer to read, write and > maintain code, how efficient/fast you can implement it, etc. Scheme guru Matthias Felleisen takes a stab at a precise definition here (though it's pointy-headed theory that I don't understand that well): http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.4656 > The Halting Problem is easily solved for Bloop languages: they always > halt. If Bloop is powerful enough to "solve the halting problem" as you describe, that gives it capabilities that Turing-complete languages lack. (Of course it also loses some capabilities). Some of the advantages of Turing-incomplete languages (plus why they are less constraining than it might sound) are discussed here: http://www.jucs.org/doi?doi=10.3217/jucs-010-07-0751 > But as we've seen, syntax can make a HUGE difference to power in the > sense of expressiveness, maintainability of code, readability, > efficiency of the programmer, and even efficiency of the > interpreter. Conway's Game of Life is Turing Complete. Would you > rather use Python, or Game of Life? BrainF*ck or Javascript? That's completely different than Python vs Scheme, where you can basically transliterate Python to Scheme by converting indentation structure into parentheses and a few other things like that. Felleisen's paper (linked above) has a deeper take on what expressiveness really means. I've programmed lots of Python and a fair amount of Lisp, and Lisp's parentheses really aren't that big a deal. Python is more pleasant than Lisp for me these days, mostly because its built-in datatypes and the standard libraries are a better match for today's programming than Lisp, which is somewhat stuck in the 1980s. Clojure (a new Lisp dialect with modern capabilities, currently only on the JVM) may be the way forward for Lisp, parentheses and all. > you are convinced that Python would be a better language if it > supported more mathematical notation with Unicode operators. Haskell and Agda both support this. It isn't used much in Haskell but Agda users actually use it, and the Emacs editing mode for Agda has special stuff to make it easier. It's probably more useful in Agda since Agda programs deal more with actual mathematical proofs. From grant.b.edwards at gmail.com Tue Aug 2 15:38:35 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 2 Aug 2016 19:38:35 +0000 (UTC) Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-02, BartC wrote: > On 02/08/2016 18:54, Steven D'Aprano wrote: >> On Wed, 3 Aug 2016 02:56 am, BartC wrote: >> >>> (And I expect that next they will eliminate languages altogether. All >>> you need is some way of specifying a sequence of calls to library >>> functions and sprinkling around some control statements; >> >> That would be called "a language". > > No, it wouldn't be given its own identity. Sure it would. How else could you talk about it or differentiate it from something else? > And it probably couldn't be used without that specific tool. You mean like a compiler or interpreter? > It would be more like a file format, So you'd have to define a combination of syntax and semantics where you specify what elements are allowed in what order/context and what they mean in various combinations? It turns out there's a English word for that. It's called a "language". > if the details are even exposed when the format is proprietary. > > A traditional language exists as text and can be stored as plain > text. Many languages are plain text. Some aren't. -- Grant Edwards grant.b.edwards Yow! If elected, Zippy at pledges to each and every gmail.com American a 55-year-old houseboy ... From lists at juliensalort.org Tue Aug 2 15:45:28 2016 From: lists at juliensalort.org (Julien Salort) Date: Tue, 2 Aug 2016 21:45:28 +0200 Subject: Why not allow empty code blocks? References: <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0de5d$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1mrdj0d.hm4ccb1ikbvgmN%lists@juliensalort.org> Steven D'Aprano wrote: > It could be, but won't be. Outside of a very few tiny niches, including > Squeak which is designed for children, such user-interfaces are far too > cumbersome to ever get widespread use. Unfortunately, many people use LabView in my field... even for sufficiently complex programs for it to become totally unreadable. -- Julien Salort Entia non sunt multiplicanda praeter necessitatem http://www.juliensalort.org From auriocus at gmx.de Tue Aug 2 15:55:04 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 2 Aug 2016 21:55:04 +0200 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 02.08.16 um 16:58 schrieb Chris Angelico: > - A more free-form declarative syntax for laying out GUI code Actually, the Tkinter wrapper misses one feature of grid in Tcl/Tk: You can write something like grid .a .b grid .c .d to lay out a GUI 2x2 grid using "ASCII-art". There is a package in Tcl, gridplus2 (http://www.satisoft.com/tcltk/gridplus2/example1.html) which allows the design of quite complex screens from such ASCII-art - a Python translation should not be that difficult (but nobody does it) > - Arbitrary-precision non-integers https://pypi.python.org/pypi/bigfloat/ ? > - Convenient syntax for a few array/list manipulations Huh? Slices and list comprehensions go a long way. What syntax can you imagine that goes beyond that? Christian From nad at python.org Tue Aug 2 16:10:53 2016 From: nad at python.org (Ned Deily) Date: Tue, 2 Aug 2016 16:10:53 -0400 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: Message-ID: <4f4b8cb9-2dca-d767-c231-66a37f1e97b2@python.org> On 2016-08-02 15:08, Terry Reedy wrote: > On 8/2/2016 4:21 AM, munozvvaleria at gmail.com wrote: >> I am new to the programming world but I need to learn this program >> for a research class that I am taking. I have downloaded Python >> 3.6.0a3 on a Mac 10.9.5 and realized that I also need to download an >> Active Tcl for it. I can't find one for this version. Can someone >> please direct me to it? The most recent one I've been able to find is >> for Python 3.5.2 > This page https://www.python.org/download/mac/tcltk/ > does not yet mention 3.6. So I would start with the recommendation for > 3.5.2: tcl/tk 8.5.18. Yes, sorry, I don't usually update that page for pre-releases. The information you need is also included in the python.org OS X installer Read Me file which is displayed during the installation process. (A copy of it is also preserved at "/Applications/Python 3.6/ReadMe.rtf".). In the section "Update your version of Tcl/Tk to use IDLE or other Tk applications": "For the initial alpha releases of Python 3.6, the installer is linked with Tcl/Tk 8.5; this will change prior to the beta releases of 3.6.0." From rosuav at gmail.com Tue Aug 2 16:50:02 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 06:50:02 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 3, 2016 at 5:55 AM, Christian Gollwitzer wrote: >> - Arbitrary-precision non-integers > > > https://pypi.python.org/pypi/bigfloat/ > > ? Wasn't aware of that. Cool. Not that I need it very often (and when I do, I can use Pike, which has MPFR support built-in). Or I can use decimal.Decimal, which isn't exactly the same, but often accomplishes the same thing. >> - Convenient syntax for a few array/list manipulations > > > Huh? Slices and list comprehensions go a long way. What syntax can you > imagine that goes beyond that? Imagine you have a list (or array, depending on language) of objects, some of which may be None (or null). Call the foo method on every object, ignoring the null/None ones. # Python - creates a useless list [obj.foo() for obj in objects if obj] # Python - a bit clunky for obj in objects: if obj: obj.foo() //Pike objects->foo(); But my point wasn't "hey, Python sucks", but "look what is *not* on my list". Other people's lists of stuff they miss will differ (either because they know of third-party solutions, like bigfloat, or because they just don't need them... like bigfloat), and that's fine and correct. ChrisA From tjreedy at udel.edu Tue Aug 2 17:27:06 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 2 Aug 2016 17:27:06 -0400 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/2/2016 7:05 AM, BartC wrote: > On 31/07/2016 19:58, Terry Reedy wrote: >> On 7/31/2016 6:18 AM, BartC wrote: > >>> repeat N: > >>> The benefit is not so much performance, but being able to express >>> something very easily and quickly. >> >> The cost of the 'repeat' contraction is that one cannot use the loop >> variable, either as part of a modified computation or for monitoring or >> debugging purposes. >> print(i, for_body_result) >> Beginners are often atrocious at debugging, and it seems not to be >> taught hardly at all. 'repeat n' erects a barrier to debugging. > >> Debugging: probing a computation to see what actually happens, as >> opposed to what one wanted and expected to happen. (Me, just now ;-) >> >> One major way of debugging is printing values as they are computed. >> Naming values (objects) allows them to be printed without recomputing >> the value. In the 'repeat n' context, recomputing would mean adding 3 >> lines of debugging code instead of 1. >> >> i = 0 >> repeat n: >> a = f(a) >> print(i, a) >> n += 1 > > Your objection to a feature such as 'repeat N' doesn't really stack up. My objection is that there is a real cost that MUST be stacked up against the benefit. ... > Anyway, if that was a valid objection, it would apply throughout the > language. In list-comps for example (there is an index, but where do you > stick the print?). In the expression. Given 'f(i) for i in range(n)', a careful debug version might be '(f(i), print(i))[0] for i in range(n)'. If the loop does not finish, the extra None component does not matter. and the subscripting could be omitted. > Or in a for-loop iterating over values: > > a=[10,20,30,40,50] > for x in a: > print (x) > > This will print the element, but what's the index? Irrelevant. The end of the sequence of prints says where the loop stopped. > According to you, > every such for-loop needs to be written in a form that provides the > index of a loop Don't pretend that I said that. It is not nice. > I get that people here don't want such a feature, but I don't think this > is the reason. > > I think the real reason is not willing to admit that the language lacks > something that could actually be useful, I think it is you who is unwilling to admit that nearly everything that would be useful also has a cost, and that the ultimate cost of adding every useful feature, especially syntax features, would be to make python less unusable. Some time around last August to October, I think, someone posted to python-ideas that he had produced a children's Python environment that accepted 'repeat n' statements and translated them to equivalent for loops before running. His idea was that 'repeat n' should be added to python itself so translation would not be needed. The main use of the statement in this context is for logo/turtle code with many side-effect-only calls. I though the idea plausible, at first, and noted that one could add an extension to IDLE to experiment further with the idea. An extension could add a menu ited such as 'de-repeat'. One could also, with about the same effort, *patch* IDLE to do the translation just before it calls compile(code, ...). Leaving IDLE aside, one could write a pyre.whatever script to translate a .pyre file to a .py file and run the latter. Any of these methods could be applied to experimenting with other 'improvement' ideas. Anyway, *after thinking about the idea for at least a week*, I became less enthusiastic about hiding the loop counter. -- Terry Jan Reedy From no.email at nospam.invalid Tue Aug 2 17:54:45 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 02 Aug 2016 14:54:45 -0700 Subject: Debugging (was Re: Why not allow empty code blocks?) References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: <871t266dlm.fsf@jester.gateway.pace.com> Terry Reedy writes: > I think it is you who is unwilling to admit that nearly everything > that would be useful also has a cost, and that the ultimate cost of > adding every useful feature, especially syntax features, would be to > make python less unusable. I think you meant "usable" ;). Some of this results from Python's statement-vs-expression distinction. I'm not going to claim that's a broken feature of Python, since it has its supporters; but languages that don't make the distinction have more flexibility in implementing such features as ordinary functions rather than syntax features. Ruby: 17.times| code | Haskell: replicateM_ 17 code Lisp: (dotimes (i 17) code...) etc. The Lisp example binds 'i' to the loop index, but that's just a choice made in how the dotimes macro was implemented. The scope of i is limited to the dotimes construct, unlike in Python. From nobody at nowhere.invalid Tue Aug 2 18:13:53 2016 From: nobody at nowhere.invalid (Nobody) Date: Tue, 02 Aug 2016 23:13:53 +0100 Subject: ctypes Usage Note References: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> Message-ID: On Mon, 01 Aug 2016 18:41:53 -0700, Lawrence D?Oliveiro wrote: > Sometimes people load a library with ctypes like this: > > libc = ctypes.cdll.LoadLibrary("libc.so") That specific example is unlikely to work on any modern Linux system, as libc.so is typically a linker script rather than a symlink to a DSO. Likewise for libm and libpthread. From bc at freeuk.com Tue Aug 2 18:38:58 2016 From: bc at freeuk.com (BartC) Date: Tue, 2 Aug 2016 23:38:58 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 02/08/2016 22:27, Terry Reedy wrote: > On 8/2/2016 7:05 AM, BartC wrote: >> Your objection to a feature such as 'repeat N' doesn't really stack up. > > My objection is that there is a real cost that MUST be stacked up > against the benefit. > > ... >> Anyway, if that was a valid objection, it would apply throughout the >> language. In list-comps for example (there is an index, but where do you >> stick the print?). > > In the expression. Given 'f(i) for i in range(n)', a careful debug > version might be '(f(i), print(i))[0] for i in range(n)'. If the loop > does not finish, the extra None component does not matter. and the > subscripting could be omitted. That's quite a big edit to the original code. I would duplicate the line, comment out the original, and add the print code to the copy. Then I can revert to the original without bugs creeping in. (Unless the rest of the line has to change anyway.) Same method is used with repeats: for N: # or whatever syntax is used If a loop index /has/ to be printed, comment out the above and extend a copy of it to: for i in range(N): But only /if/ that value is needed; most won't. (I suspect that loops repeated a set number of times might be simpler anyway.) >> Or in a for-loop iterating over values: >> >> a=[10,20,30,40,50] >> for x in a: >> print (x) >> >> This will print the element, but what's the index? > > Irrelevant. The end of the sequence of prints says where the loop stopped. But it might go wrong before it gets to the end. You need the index to know how far along the list it's at. >> I think the real reason is not willing to admit that the language lacks >> something that could actually be useful, > > I think it is you who is unwilling to admit that nearly everything that > would be useful also has a cost, and that the ultimate cost of adding > every useful feature, especially syntax features, would be to make > python less unusable. [Did you mean 'less usable' here?] I'm only concerned here with basic syntax. And most ideas already exist in other languages (not just mine). But here's one I idea I think I mentioned last year: having separators in numeric literals. Now I look and see that PEP 515 describes it! (I think some versions of Python will already have it). Do people think that makes Python top-heavy in features and less usable? I *know* that actually implementing most of this stuff is trivial because I've done it a dozen times. And I appreciate that doing it in a very large, existing 'live' language with millions of users has administrative problems. But the technical side of it is nothing compared to the 50Mloc tools that have been mentioned. > > Some time around last August to October, I think, someone posted to > python-ideas that he had produced a children's Python environment that > accepted 'repeat n' statements and translated them to equivalent for > loops before running. His idea was that 'repeat n' should be added to > python itself so translation would not be needed. The main use of the > statement in this context is for logo/turtle code with many > side-effect-only calls. Apparently 'repeat N' is a Logo statement so it makes sense to try and emulate that within Python (to simplify porting Logo algorithms for example). > Leaving IDLE aside, one could write a pyre.whatever script to translate > a .pyre file to a .py file and run the latter. Any of these methods > could be applied to experimenting with other 'improvement' ideas. Yes, I've experimented with that approach myself. Using a source-to-source translator with Python as output. But the input was an entirely different syntax (example: http://pastebin.com/Zj89YfTN). -- Bartc From ben+python at benfinney.id.au Tue Aug 2 20:24:41 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 03 Aug 2016 10:24:41 +1000 Subject: gmane sending References: <9d543a2a-c56a-46cc-971d-2124782d5d1f@googlegroups.com> Message-ID: <85d1lq3diu.fsf@benfinney.id.au> Terry Reedy writes: > I am reading and sending via gmane right now. > > > https://developers.slashdot.org/story/16/07/28/2059249/the-end-of-gmane > > I really hope not. I also hope not. Hope doesn't fund important infrastructure like Gmane, though; and Lars clearly needs help if Gmane is to thrive . The signs are troubling: has been timing out since that news :-( -- \ ?Human reason is snatching everything to itself, leaving | `\ nothing for faith.? ?Bernard of Clairvaux, 1090?1153 CE | _o__) | Ben Finney From 03sjbrown at gmail.com Tue Aug 2 21:26:22 2016 From: 03sjbrown at gmail.com (Shawn Brown) Date: Tue, 2 Aug 2016 21:26:22 -0400 Subject: ANN: datatest 0.7.0 (Test driven data wrangling) Message-ID: datatest 0.7.0 (Test driven data wrangling) =========================================== Datatest extends the standard library's unittest package to provide testing tools for asserting data correctness. * Docs: http://datatest.readthedocs.io/ * PyPI: https://pypi.python.org/pypi/datatest/ This release includes: * Removes internal magic and renames data assertions to more clearly indicate their intended use. * Restructures data allowances to provide more consistent parameters and more flexible usage. * Adds new method to assert unique values. * Adds full **fmtparams support for CSV handling. * Fixes comparison and allowance behavior for None vs. zero. Update installs with: pip install -U datatest Backward Compatibility: Existing code that relies on the 0.6.0 (dev1) API is supported with the following addition to the beginning of each script: from datatest.__past__ import api_dev1 From steve+comp.lang.python at pearwood.info Wed Aug 3 01:43:40 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 15:43:40 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> On Wednesday 03 August 2016 05:14, BartC wrote: > It's fundamental in that, when giving instructions or commands in > English, it frequently comes up when you want something done a set > number of times: > > "Give me 20 push-ups" At which point the person will invariable drop to the ground and start counting one...two.....thrrrrrreee.........fooooooourrrr....................fiiiiiive... Counting is more fundamental than addition. You cannot do 20 push-ups without in some sense counting. > "Press space 5 times" > > "Do 10 laps" > > Whoever has to execute these may need to keep count somehow, > but that is not the concern of the person giving the commands. Perhaps not. I don't doubt that there are times where you don't care about the loop variable. Fine, you don't care. There are times that I perform an operation which might fail, and I don't care if it fails. I have to still catch the exception. There's no dedicated syntax to "run this and ignore exceptions", you just use the general purpose try...except syntax. try: this() except Exception: pass rather than: this() # may raise $this() # won't raise Not everything that is done is worth the cognitive burden of memorising a special case. > You wouldn't say, count from 1 to 20, and for each value in turn, do a > push-up. You could also say count backwards from 95 to 0 in fives; same > effect. There are so many ways of specifying a loop that is executed 20 > times, that no one way can be the right one. So that extra information > is irrelevant. Sure. That's why we have idioms like `for i in range(20)`, rather than have people consider `for i in [None]*20` or `range(53, 114, 3)`. Even if we had a dedicated `repeat 20` syntax, it would still be merely a convention that you use it rather than `for i in range(53, 114, 3)`. In some ways, Python is a more minimalist language than you like. That's okay, you're allowed to disagree with some design decisions. I personally think that new f-strings-that-aren't-actually-strings-but-more-like-eval-in-disguise are a terrible idea, and I think that some of the rejected ideas were good ones. That's part of the reason why we have so many different languages: people can disagree on what things should be features. -- Steve From honeygne at gmail.com Wed Aug 3 03:06:56 2016 From: honeygne at gmail.com (honeygne at gmail.com) Date: Wed, 3 Aug 2016 00:06:56 -0700 (PDT) Subject: holding if/else condition in live bitcoin trading Python In-Reply-To: <57a0c646$0$1612$c3e8da3$5496439d@news.astraweb.com> References: <84adc952-262e-4a53-b749-0c3ecb09bf01@googlegroups.com> <57a0c646$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tuesday, August 2, 2016 at 9:42:03 PM UTC+5:30, Steven D'Aprano wrote: > On Tue, 2 Aug 2016 07:14 pm, Arshpreet Singh wrote: > > > is there any approach I can go with so auto_order_buy() auto_order_sell() > > function only executed once? > > Take them out of the while loop. yes but Bitfinex is hacked, so no use now. From dieter at handshake.de Wed Aug 3 03:57:42 2016 From: dieter at handshake.de (dieter) Date: Wed, 03 Aug 2016 09:57:42 +0200 Subject: make an object read only References: Message-ID: <87mvku9te1.fsf@handshake.de> Robin Becker writes: > A reportlab user found he was doing the wrong thing by calling canvas.save > repeatedly, our documentation says you should not use Canvas objects after the > save method has been used. The user had mixed results :( > > It would be better to make the canvas object completely immutable all the way > down when save has been called, but I think that's quite hard as these objects > have quite a large and varied set of attributes, lists other objects > dictionaries etc etc. If changing things happen at an elementary level (by assigning to the object's instance attributes), then you could implement your on "__setattr__/__delattr__" methods. There, you could check whether you are ready to allow the change or reject it. From antoon.pardon at rece.vub.ac.be Wed Aug 3 04:26:40 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 3 Aug 2016 10:26:40 +0200 Subject: Why not allow empty code blocks? In-Reply-To: References: <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> Message-ID: <57A1AAC0.2030900@rece.vub.ac.be> Op 02-08-16 om 14:29 schreef Rustom Mody: > So I was talking of 3 very different levels: > > 1. print x vs print(x) > ? a difference too petty for me to waste my time with > > 2. Procedure vs Function as something very necessary for beginner > thinking-ontology which Pascal gets right > > 3. The fact that the gap between a mainly-for-teaching language and a serious > software-engineering-real-world language is not closable > And that saying that the same language could be used for both purposes is > like arguing that both these delightful ladies are pianists: > > > Martha: https://www.youtube.com/watch?v=YLZLp6AcAi4 > Rose : https://www.youtube.com/watch?v=_bjKDJD-CLc > > > Scheme and Pascal happen to be two well-known well-crafted but quite different > for-teaching languages But on what ground is scheme a well-crafted for-teaching language? It is not because it gets the Procedure vs Function ontology right. As far as I know scheme has about the same data structures as python, they are just called differently and there are some limitations. So when putting scheme and python next to each other we should choose scheme for teaching and python for production. -- Antoon Pardon From robin at reportlab.com Wed Aug 3 04:28:17 2016 From: robin at reportlab.com (Robin Becker) Date: Wed, 3 Aug 2016 09:28:17 +0100 Subject: gmane sending In-Reply-To: <85d1lq3diu.fsf@benfinney.id.au> References: <9d543a2a-c56a-46cc-971d-2124782d5d1f@googlegroups.com> <85d1lq3diu.fsf@benfinney.id.au> Message-ID: <3fa0898b-706f-b280-0200-151028d3521e@chamonix.reportlab.co.uk> On 03/08/2016 01:24, Ben Finney wrote: > Terry Reedy writes: > >> I am reading and sending via gmane right now. >> ........... > > I also hope not. Hope doesn't fund important infrastructure like Gmane, > though; and Lars clearly needs help if Gmane is to thrive > . > > The signs are troubling: has been > timing out since that news :-( > Frome various blogs etc etc I seem to hear he might keep the smtp nntp bridge going. FWIW my issue was > > python-list at python.org > SMTP error from remote mail server after RCPT TO:: > host mail.python.org [188.166.95.178]: 504-5.5.2 : > Helo command rejected: need fully-qualified hostname this being sent by gmane -- Robin Becker From auriocus at gmx.de Wed Aug 3 04:30:37 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 3 Aug 2016 10:30:37 +0200 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: <4f4b8cb9-2dca-d767-c231-66a37f1e97b2@python.org> Message-ID: Am 02.08.16 um 22:10 schrieb Ned Deily: > "For the initial alpha releases of Python 3.6, the installer is linked > with Tcl/Tk 8.5; this will change prior to the beta releases of 3.6.0." Let me also convey some insider information from the Tcl world: the upcoming 8.7 (I'll expect it next year) will have full Unicode support (>BMP). Because of compatibility issues with the C API, this will be done in the way that Tcl unicode strings are considered UTF-16 instead of UCS-2. So adapting Tkinter should be relatively easy in the end, if it passes UTF-8 strings, then there is zero change, if it passes Tcl_UniChar, then these must be convrted to UTF-16. Christian From steve+comp.lang.python at pearwood.info Wed Aug 3 04:58:35 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 03 Aug 2016 18:58:35 +1000 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> Message-ID: <57a1b23c$0$11093$c3e8da3@news.astraweb.com> On Wednesday 03 August 2016 05:22, Paul Rubin wrote: >> The Halting Problem is easily solved for Bloop languages: they always >> halt. > > If Bloop is powerful enough to "solve the halting problem" as you > describe, that gives it capabilities that Turing-complete languages > lack. (Of course it also loses some capabilities). It only solves it in the sense that it isn't capable of looping forever, so there's never a question of whether or not a Bloop program will halt: they all do. It doesn't have any capabilities that Floop lacks: Floop can solve any problem that Bloop can solve, plus problems that Bloop cannot. In a sense, it's a less extreme version of this: Me: "I have here a computer which is immune to all computer viruses, malware and hostile code, now and in the future!" You: "That's great! Turn it on so we can see how it works." Me: "Oh, it doesn't turn on. There's no power supply. That was the only way I could guarantee it wouldn't execute malware: by making sure it couldn't execute *anything*." While I suppose it is true that a computer that doesn't run is still a computer, its an abuse of language to say that it has capabilities that running computers lack: - unhackable - immune to all viruses - unaffected by power surges - data storage is 100% reliable, never lose data again - instantaneous log-off and shutdown :-) Floop can certainly give a *partial* solution to the Halting Problem: - if the code is runnable using Bloop, then it definitely halts; - if the code is runnable using Floop, then ... maybe it halts? A sufficiently clever Floop program might be able to analyse Floop code and recognise many common cases of non-terminating loops. For example, the Floop equivalent of: while True: pass is clearly non-terminating. There's no mystery about being able to detect that. But not all code is that straight-forward. > Some of the > advantages of Turing-incomplete languages (plus why they are less > constraining than it might sound) are discussed here: > > http://www.jucs.org/doi?doi=10.3217/jucs-010-07-0751 > >> But as we've seen, syntax can make a HUGE difference to power in the >> sense of expressiveness, maintainability of code, readability, >> efficiency of the programmer, and even efficiency of the >> interpreter. Conway's Game of Life is Turing Complete. Would you >> rather use Python, or Game of Life? BrainF*ck or Javascript? > > That's completely different than Python vs Scheme, where you can > basically transliterate Python to Scheme by converting indentation > structure into parentheses and a few other things like that. I wouldn't say "completely different". Since GoL, Python, BrainF*uck and Javascript are all Turing Complete, there must be a way to convert a program in any one to any of the others. One obvious way to do this is to (for example) write a Javascript interpreter in BrainF*ck. That demonstrates that anything Javascript can do, BrainF*ck can do too. Of course, going the other way (BrainF*ck interpreter written in Javascript) is considerably easier :-) Python is sometimes described as a Lisp with more sensible syntax, so its not surprising that it is relatively simple to translate Python to Lisp and visa versa. It would be harder to translate (say) COBOL code to (say) Smalltalk. But it is always the case that any program for a Turing Complete language can be translated into a program for any other Turing Complete language. It might be *really really really difficult*, inefficient and slow, but its always possible. What's *not* possible is to take any Floop program and translate it into Bloop. There are things Floop can do that Bloop cannot -- Bloop is not Turing Complete, but Floop is. -- Steve From rosuav at gmail.com Wed Aug 3 04:58:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 18:58:43 +1000 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: <4f4b8cb9-2dca-d767-c231-66a37f1e97b2@python.org> Message-ID: On Wed, Aug 3, 2016 at 6:30 PM, Christian Gollwitzer wrote: > Am 02.08.16 um 22:10 schrieb Ned Deily: >> >> "For the initial alpha releases of Python 3.6, the installer is linked >> with Tcl/Tk 8.5; this will change prior to the beta releases of 3.6.0." > > > Let me also convey some insider information from the Tcl world: the upcoming > 8.7 (I'll expect it next year) will have full Unicode support (>BMP). Yay!! > Because of compatibility issues with the C API, this will be done in the way > that Tcl unicode strings are considered UTF-16 instead of UCS-2. So adapting > Tkinter should be relatively easy in the end, if it passes UTF-8 strings, > then there is zero change, if it passes Tcl_UniChar, then these must be > convrted to UTF-16. Ehh. Well, it's no worse than ECMAScript uses. ChrisA From BSomerville at flexerasoftware.com Wed Aug 3 05:38:41 2016 From: BSomerville at flexerasoftware.com (Bill Somerville) Date: Wed, 3 Aug 2016 09:38:41 +0000 Subject: Issue with ctypes and callback functions Message-ID: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> Hi All, Is this a good place to ask questions about the above? Regards Bill. From rustompmody at gmail.com Wed Aug 3 06:02:03 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 03:02:03 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <87a8gv562h.fsf@jester.gateway.pace.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> Message-ID: <890bc5c3-969e-4ac6-a9a0-c2c56caff72c@googlegroups.com> On Wednesday, August 3, 2016 at 12:53:02 AM UTC+5:30, Paul Rubin wrote: > Steven D'Aprano writes: > > where power is defined (rather fuzzily) as the expressiveness > > of the language, how easy it is for the programmer to read, write and > > maintain code, how efficient/fast you can implement it, etc. > > Scheme guru Matthias Felleisen takes a stab at a precise definition here > (though it's pointy-headed theory that I don't understand that well): > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.4656 Lovely! Thanks for that! [Been preparing some notes/thoughts for a Theory of CS course ? v drafty : https://bitbucket.org/rustom/toc/wiki/themes ? and had listed this (topic) as one of the big open problems] > > > The Halting Problem is easily solved for Bloop languages: they always > > halt. > > If Bloop is powerful enough to "solve the halting problem" as you > describe, that gives it capabilities that Turing-complete languages > lack. (Of course it also loses some capabilities). Some of the > advantages of Turing-incomplete languages (plus why they are less > constraining than it might sound) are discussed here: > > http://www.jucs.org/doi?doi=10.3217/jucs-010-07-0751 And thanks for that as well I had seen that decades ago but forgot waht to look for From bc at freeuk.com Wed Aug 3 06:16:26 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 11:16:26 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 03/08/2016 06:43, Steven D'Aprano wrote: > Not everything that is done is worth the cognitive burden of memorising a > special case. .... > In some ways, Python is a more minimalist language than you like. That's okay, > you're allowed to disagree with some design decisions. Well it's minimalist in some ways, and completely the opposite in others! It uses minimal basic syntax (missing a couple of loop forms, loop controls, switch/case, select-expressions... it's just a handful of features). But then you get to the standard library, and the plethora of different data types, methods and options. It's never-ending! So the idea that remembering 'repeat N' is a cognitive burden, and the myriad string operations for example are not, is ridiculous. (Especially when 'repeat N' will have an obvious counterpart in some other languages, but 'str.encode(...)' for example will not.) -- Bartc From sivan at vitakka.co Wed Aug 3 07:03:58 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Wed, 3 Aug 2016 14:03:58 +0300 Subject: usage of functools.partial in in parallelism In-Reply-To: <57a01e20$0$2809$c3e8da3$76491128@news.astraweb.com> References: <5c6c7f8c-8f1a-8fd0-9972-54312f400263@gmail.com> <57a01e20$0$2809$c3e8da3$76491128@news.astraweb.com> Message-ID: So yeah I wanted to remark that I actually identified more with the two questions raised after the 'stop the classes' talk, and I felt much like the two questions that raised after as per documenting design via use of classes in code (However I make sure to keep it all flattened as opposed to the lecturer first examples). I also think it's a matter of style and personal preference. The nodejs like-minded people would prolly lean more to the function based coupling using 'partial', while people like me would prefer classes as they'd think tasks in logical domain grouping, as I really love encapsulation. I am not left to wonder about difference in performance of functional vs. objectional python. -Sivan On Tue, Aug 2, 2016 at 7:14 AM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Tuesday 02 August 2016 13:14, Michael Torrie wrote: > > > On 08/01/2016 01:13 AM, Michael Selik wrote: > >> You might benefit from watching the talk "Stop Writing Classes" > >> https://www.youtube.com/watch?v=o9pEzgHorH0 > > > > Great talk! Thanks for posting that. > > > It is a great talk, but for a counter-view: > > http://lucumr.pocoo.org/2013/2/13/moar-classes/ > > > > Remember: moderation in all things, except for moderation. > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From rustompmody at gmail.com Wed Aug 3 07:48:38 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 04:48:38 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <87vazwppxx.fsf@elektro.pacujo.net> <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> Message-ID: <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> On Wednesday, August 3, 2016 at 1:57:43 PM UTC+5:30, Antoon Pardon wrote: > Op 02-08-16 om 14:29 schreef Rustom Mody: > > So I was talking of 3 very different levels: > > > > 1. print x vs print(x) > > ? a difference too petty for me to waste my time with > > > > 2. Procedure vs Function as something very necessary for beginner > > thinking-ontology which Pascal gets right > > > > 3. The fact that the gap between a mainly-for-teaching language and a serious > > software-engineering-real-world language is not closable > > And that saying that the same language could be used for both purposes is > > like arguing that both these delightful ladies are pianists: > > > > > > Martha: https://www.youtube.com/watch?v=YLZLp6AcAi4 > > Rose : https://www.youtube.com/watch?v=_bjKDJD-CLc > > > > > > Scheme and Pascal happen to be two well-known well-crafted but quite different > > for-teaching languages > > But on what ground is scheme a well-crafted for-teaching language? > > It is not because it gets the Procedure vs Function ontology right. > > As far as I know scheme has about the same data structures as python, > they are just called differently and there are some limitations. Technically you are right But ?technically? is not the most important facet. So yes scheme are python have similar underbellies but the culture of use is quite different. eg The very first example in the python tutorial uses a print statement/expr. Whereas in scheme you can go through much of the text(s) before you see the same Note: You CAN use python scheme-ishly but few people do that. eg here is SICP in python with the first examples pure useful powerful functions: http://www-inst.eecs.berkeley.edu/~cs61a/sp12/book/functions.html#first-example So people can use python functionally but somehow have the idea that imperative is easier/more natural/what-have-you Scheme and python data structures: Python has a rich COLLECTION of data structures Scheme really has just one ? S-exp Software-engineering-wise this conduces to power in python and headache/nuisance in scheme But for a learner its the opposite: You do say dir([]) or dir({}) and you get a whole truckload of methods ? how many to learn?? With scheme you chew up car/cdr/cons until you reach an epiphany of understanding vide Bruce Lee's quote: I am not afraid of the man who has practised ten thousand kicks. I am afraid of the man who has practised one kick ten thousand times Then there are motley little things that make python beginner-unfriendly. Noob makes a file whose name masks a builtin module and random things stop working. Umpteen such examples keep appearing out here. All point to one direction: Guido wants his language to be lightweight and for serious software engineering. Sure the ?lightweight? conduces to a neater language for teaching than say C++, Java etc. But the teaching/learning is not the primary goal, its software engineering. This can be seen in many sundry choices Backward compatibility: Guido is chary about introducing new keywords since code that currently uses that will break unexpectedly. And so we get new features that are seemingly clever but actually rather infelicitous; eg - conditional expression with wrong order of arguments - def overloaded for (ordinary) function and generator - yield overloaded for coroutine and generator For a beginner backward compatibility is a meaningless requirement: If python-2016 is inconsistently different from python-2015 who is affected since the bunch of learners has moved on. [Hapless teacher excepted!] Scheme ? especially Dr. Scheme/racket ? embraces inconsistency by offering different ?teachpacks? ? different language syntaxes/semantics' for learners at different levels Likewise platform independence: One important goal of (core) python is that unless you go out of your way to be troublesome, a python program (script) on linux will run on windows and vice- versa (eg. universal newlines) This is of course good The flip-side is that important functionality that cant be easily supported ? eg GUIs ? get relegated to ?advanced? topics which is a euphemism for get neglected [As a thought experiment: If we started by teaching VB would GUIs look as advanced as they do in python?] Once again like backward compatibility, platform independence is meaningless to a beginner: For the one system that (s)he uses/owns it works or doesn?t work. That it may or may not work for other invisible, unimaginable systems is wildly over-the-top philosophy. OTOH racket (scheme) comes with fairly comprehensive GUI/graphics functionality [not tried myself] I expect 1. Its not really in the professional league of Qt or Wx but its enough for a beginner to have a comprehensive experience 2. An experience which is likely perceived as easier than struggling with PyQt wxPython etc From marko at pacujo.net Wed Aug 3 08:09:12 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 03 Aug 2016 15:09:12 +0300 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> Message-ID: <87fuqmjbpz.fsf@elektro.pacujo.net> Rustom Mody : > So yes scheme are python have similar underbellies but the culture of > use is quite different. I don't know if there's enough Scheme activity out there to call it a culture. As far as underbellies go, Scheme macros and operators are not first-class. The Kernel programming language () addresses this flaw. > Note: You CAN use python scheme-ishly but few people do that. I don't know who or what you are referring to. About the only Schemey things not available to Python are tail recursion elimination and macros. From the little I have seen, Scheme programmers use tail recursion too little and macros too much. > So people can use python functionally but somehow have the idea that > imperative is easier/more natural/what-have-you In that regard, I don't see much difference in Scheme and Python practices. > Python has a rich COLLECTION of data structures > Scheme really has just one ? S-exp Scheme has all kinds of data structures. > Then there are motley little things that make python > beginner-unfriendly. I can hardly imagine a programming language that would be better-suited for complete beginners. Scheme for beginning programmers could be like abstract algebra for kindergartners. Marko From rustompmody at gmail.com Wed Aug 3 08:16:06 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 05:16:06 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <57a1b23c$0$11093$c3e8da3@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> On Wednesday, August 3, 2016 at 5:11:23 PM UTC+5:30, Steven D'Aprano wrote: > On Wednesday 03 August 2016 05:22, Paul Rubin wrote: > > >> The Halting Problem is easily solved for Bloop languages: they always > >> halt. > > > > If Bloop is powerful enough to "solve the halting problem" as you > > describe, that gives it capabilities that Turing-complete languages > > lack. (Of course it also loses some capabilities). > > It only solves it in the sense that it isn't capable of looping forever, so > there's never a question of whether or not a Bloop program will halt: they all > do. It doesn't have any capabilities that Floop lacks: Floop can solve any > problem that Bloop can solve, plus problems that Bloop cannot. > > In a sense, it's a less extreme version of this: > > Me: "I have here a computer which is immune to all computer viruses, malware > and hostile code, now and in the future!" > > You: "That's great! Turn it on so we can see how it works." > > Me: "Oh, it doesn't turn on. There's no power supply. That was the only way I > could guarantee it wouldn't execute malware: by making sure it couldn't execute > *anything*." > > While I suppose it is true that a computer that doesn't run is still a > computer, its an abuse of language to say that it has capabilities that running > computers lack: > > - unhackable > - immune to all viruses > - unaffected by power surges > - data storage is 100% reliable, never lose data again > - instantaneous log-off and shutdown > > :-) There he comes waddling in? Your cute-n-cudly strawman!! A more realistic analogy would be phones The cellphones we use today often crash The first nokia I used never crashed but could still run out of battery And the round-dial landlines of 30 years ago had not even that problem But correspondingly the functionality: The early landlines could just dial a NUMBER (which you looked up from a dead-tree book) The first cell phones had some rudimentary phone book Nowadays phones do everything a computer can? Including crash! From eryksun at gmail.com Wed Aug 3 08:16:11 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 3 Aug 2016 12:16:11 +0000 Subject: Issue with ctypes and callback functions In-Reply-To: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> References: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> Message-ID: On Wed, Aug 3, 2016 at 9:38 AM, Bill Somerville wrote: > > Is this a good place to ask questions about the above? Discussing ctypes is fine here. There's also a ctypes-users list. From rustompmody at gmail.com Wed Aug 3 08:23:30 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 05:23:30 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <87fuqmjbpz.fsf@elektro.pacujo.net> References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> <87fuqmjbpz.fsf@elektro.pacujo.net> Message-ID: On Wednesday, August 3, 2016 at 5:39:25 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > > So yes scheme are python have similar underbellies but the culture of > > use is quite different. > > I don't know if there's enough Scheme activity out there to call it a > culture. > > As far as underbellies go, Scheme macros and operators are not > first-class. The Kernel programming language () > addresses this flaw. > > > Note: You CAN use python scheme-ishly but few people do that. > > I don't know who or what you are referring to. I put it in the next line! > Scheme for beginning programmers could be like abstract algebra for > kindergartners. Maybe you are right ? Anyhow MIT switching from scheme to python seems to think so. As I said earlier what they (the MIT profs) seem to be saying is that the topical relevance of python ? a library for making robots ? trumps scheme?s abstract beauty From rustompmody at gmail.com Wed Aug 3 08:27:00 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 05:27:00 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> <87fuqmjbpz.fsf@elektro.pacujo.net> Message-ID: On Wednesday, August 3, 2016 at 5:53:44 PM UTC+5:30, Rustom Mody wrote: > On Wednesday, August 3, 2016 at 5:39:25 PM UTC+5:30, Marko Rauhamaa wrote: > > Rustom Mody : > > I don't know who or what you are referring to. > > I put it in the next line! > > > Scheme for beginning programmers could be like abstract algebra for > > kindergartners. Funny keeps disappearing! One more try! Note: You CAN use python scheme-ishly but few people do that. eg here is SICP in python with the first examples pure useful powerful functions: http://www-inst.eecs.berkeley.edu/~cs61a/sp12/book/functions.html#first-example From python at bdurham.com Wed Aug 3 08:29:31 2016 From: python at bdurham.com (Malcolm Greene) Date: Wed, 03 Aug 2016 08:29:31 -0400 Subject: print() function with encoding= and errors= parameters? Message-ID: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> Looking for a way to use the Python 3 print() function with encoding and errors parameters. Are there any concerns with closing and re-opening sys.stdout so sys.stdout has a specific encoding and errors behavior? Would this break other standard libraries that depend on sys.stdout being configured a specific way? Thank you, Malcolm From marko at pacujo.net Wed Aug 3 08:34:12 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 03 Aug 2016 15:34:12 +0300 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> <87fuqmjbpz.fsf@elektro.pacujo.net> Message-ID: <87bn1ajakb.fsf@elektro.pacujo.net> Rustom Mody : > As I said earlier what they (the MIT profs) seem to be saying is that > the topical relevance of python ? a library for making robots ? trumps > scheme?s abstract beauty Python is an excellent choice. As far as abstract beauty is concerned, I'm wondering why Python abolished braces from statement syntax but not from expression syntax. Marko From rosuav at gmail.com Wed Aug 3 08:36:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 22:36:01 +1000 Subject: Why not allow empty code blocks? In-Reply-To: <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> Message-ID: On Wed, Aug 3, 2016 at 10:16 PM, Rustom Mody wrote: > There he comes waddling in? Your cute-n-cudly strawman!! > A more realistic analogy would be phones > The cellphones we use today often crash > The first nokia I used never crashed but could still run out of battery > And the round-dial landlines of 30 years ago had not even that problem 1986? Yeah, we had a phone from then (granted, I don't remember much of 1986, but we had the same handsets in the 1990s), and it could run out of battery and lose its phone book. Sorry matey, you just lost the strawman game. ChrisA From marko at pacujo.net Wed Aug 3 08:37:56 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 03 Aug 2016 15:37:56 +0300 Subject: Why not allow empty code blocks? References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> <87fuqmjbpz.fsf@elektro.pacujo.net> Message-ID: <877fbyjae3.fsf@elektro.pacujo.net> Rustom Mody : > Note: You CAN use python scheme-ishly but few people do that. > eg here is SICP in python with the first examples pure useful powerful > functions: > http://www-inst.eecs.berkeley.edu/~cs61a/sp12/book/functions.html#first-example I see nothing there that is not ordinary Python. Marko From rustompmody at gmail.com Wed Aug 3 08:43:31 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 3 Aug 2016 05:43:31 -0700 (PDT) Subject: Why not allow empty code blocks? In-Reply-To: <877fbyjae3.fsf@elektro.pacujo.net> References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579cbded$0$1605$c3e8da3$5496439d@news.astraweb.com> <758a1961-00bc-4b1c-834b-a7deca94d029@googlegroups.com> <57A0763F.7050809@rece.vub.ac.be> <57A1AAC0.2030900@rece.vub.ac.be> <7b6caa1c-890a-4104-be7e-f8e279e34bb0@googlegroups.com> <87fuqmjbpz.fsf@elektro.pacujo.net> <877fbyjae3.fsf@elektro.pacujo.net> Message-ID: <88b9b7af-1594-4c6f-a303-aa7fc3d1d577@googlegroups.com> On Wednesday, August 3, 2016 at 6:08:08 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > > > Note: You CAN use python scheme-ishly but few people do that. > > eg here is SICP in python with the first examples pure useful powerful > > functions: > > http://www-inst.eecs.berkeley.edu/~cs61a/sp12/book/functions.html#first-example > > I see nothing there that is not ordinary Python. Its an example of python used functionally ? normal python (3 I think) ie No loops, no prints (at least in early chapters Ive obviously not scoured the whole book) No assignment is harder, but you will see the assignment is more like a lisp-let; ie its a single assignment From bc at freeuk.com Wed Aug 3 09:04:02 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 14:04:02 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> Message-ID: On 03/08/2016 13:36, Chris Angelico wrote: > On Wed, Aug 3, 2016 at 10:16 PM, Rustom Mody wrote: >> There he comes waddling in? Your cute-n-cudly strawman!! >> A more realistic analogy would be phones >> The cellphones we use today often crash >> The first nokia I used never crashed but could still run out of battery >> And the round-dial landlines of 30 years ago had not even that problem > > 1986? Yeah, we had a phone from then (granted, I don't remember much > of 1986, but we had the same handsets in the 1990s), and it could run > out of battery and lose its phone book. He said landlines. A typical handset would have no battery as it's powered from the line. And does not have need a memory to function. Nor would it suffer from lack of signal. Or credit (if you'd paid the last bill). Or get lost (as it's tethered to the socket). Very clunky technology but it was solid! It only did one thing but it did it incredibly well. -- Bartc From rosuav at gmail.com Wed Aug 3 09:18:09 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 23:18:09 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 3, 2016 at 8:16 PM, BartC wrote: > On 03/08/2016 06:43, Steven D'Aprano wrote: > >> Not everything that is done is worth the cognitive burden of memorising a >> special case. > > .... > >> In some ways, Python is a more minimalist language than you like. That's >> okay, >> you're allowed to disagree with some design decisions. > > > Well it's minimalist in some ways, and completely the opposite in others! > > It uses minimal basic syntax (missing a couple of loop forms, loop controls, > switch/case, select-expressions... it's just a handful of features). > > But then you get to the standard library, and the plethora of different data > types, methods and options. It's never-ending! The standard library doesn't have the cognitive burden that the core language has. For starters, it's all namespaced; for seconds, once you've mastered language syntax, you can introspect for documentation (simplest form: help(obj) at the interactive prompt). > So the idea that remembering 'repeat N' is a cognitive burden, and the > myriad string operations for example are not, is ridiculous. > > (Especially when 'repeat N' will have an obvious counterpart in some other > languages, but 'str.encode(...)' for example will not.) Myriad string operations? Let's see. >>> dir("") ['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill'] We have: * Dunder methods and attributes, used for implementing other features. You ignore them in regular code. * Methods that do what other languages do with stand-alone functions. It's not much different to do "spam".capitalize() than capitalize("spam") or "spam".replace("sp","h") than replace("spam","sp","h"). Likewise the "is*" functions, startswith/endswith, and other querying functions. * String formatting (both format and format_map). That probably belongs in the previous category, but since Python's .format() method differs from every other string formatting out there, you could say it takes extra cognitive load. But honestly, what modern language doesn't have some sort of formatted-string system, at least in its standard library? * join(), which most languages have as an array method that takes a string, but Python has as a string method that takes any iterable. Yes, that's a bit weird, I know. We can give you that one. * And, uhh... you specifically mentioned encoding strings to bytes. Well, Pike doesn't have that as a method. Instead, you have string_to_utf8() for the most common case (and, naturally, utf8_to_string() to convert bytes to text), and then has a Charset module for all other conversions. Cognitive burden? Pretty much equivalent. I would say str.encode() has barely more cognitive burden than repeat N would have. To get your head around "repeat N", you have to understand that a computer can do things more than once. To get your head around "spam".encode("utf-8"), you have to understand that bytes are a means of representing text, and that they're not the same thing. Yes, that's a bit harder (if you teach algorithms using a pen and paper, you'll probably conflate data with its representation, since there's no representation of the representation), but not hugely. ChrisA From BSomerville at flexerasoftware.com Wed Aug 3 09:21:08 2016 From: BSomerville at flexerasoftware.com (Bill Somerville) Date: Wed, 3 Aug 2016 13:21:08 +0000 Subject: Issue with ctypes and callback functions In-Reply-To: References: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> Message-ID: <0E5668E32633FB4FBBA1709825C0EDFDD24B77@SCHEXMB2.acresso.com> -----Original Message----- From: eryk sun [mailto:eryksun at gmail.com] Sent: 03 August 2016 13:16 To: python-list at python.org Cc: Bill Somerville Subject: Re: Issue with ctypes and callback functions On Wed, Aug 3, 2016 at 9:38 AM, Bill Somerville wrote: > > Is this a good place to ask questions about the above? Discussing ctypes is fine here. There's also a ctypes-users list. Thanks Eryk, I am trying to handle Python callback functions in a SWIG interface, SWIG does not support callbacks where the target is a Python callable but I can do this with ctypes. My problem is that I need to have the callback receive a SWIG wrapped type as an argument by reference. I see that if I use a ctypes.Structure type then it works nicely but I have a huge and changing API to wrap and do not want to manually describe every C struct with ctypes.Structure when SWIG has already automatically created its own shadow classes. The SWIG struct wrapper is just a class wrapping a pointer so can be easily be passed just like ctypes.Structure can be. How do I tell ctypes that it needs to reconstitute the type from an address in the callback thunk it generates? I have tried defining from_param() and self._as_parameter_ for the types which works for passing to a function but I don't need that as SWIG does that already. With callback arguments I get various errors depending on what methods/attributes I supply like: Parsing argument 0 TypeError: cannot build parameter At the point where the callback is being called. To summarize: ++++++++++++++++++++++++++++++++++++ C API like: typedef struct { int member; /* ... */ } my_type; typedef int (*pfn) (my_type const *); int fn (pfn); Implementation could be: #include "api.h" int fn (pfn cb) { my_type t = {42, /* ... */}; return cb (&t); } I want Python code like: from ctypes import * import api # SWIG extension module def my_cb (t): print ('in callback with:', t.member) return 0 proto = CFUNCTYPE (c_int, my_type) cb = proto (my_cb) res = api.fn (cast (cb, c_void_p).value) print ('fn returned:', res) ------------------------------------------------ I can extend the SWIG shadow class to do the creation of the prototype and thunk, the required cast above and any extra attributes or methods like _as_parameter_ and from_param() but I can't see any way of having the Python callback (my_cb) magically receive the SWIG wrapped 'my_type' struct. TIA Bill. From bc at freeuk.com Wed Aug 3 09:23:57 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 14:23:57 +0100 Subject: Why not allow empty code blocks? In-Reply-To: <57a1b23c$0$11093$c3e8da3@news.astraweb.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On 03/08/2016 09:58, Steven D'Aprano wrote: > Python is sometimes described as a Lisp with more sensible syntax, so its not > surprising that it is relatively simple to translate Python to Lisp and visa > versa. Translating Python would be easier if everything was implemented as Python. But sometimes you try to find a .py import module and it doesn't seem to exist anywhere. (sys.py for example). I would like to see how such references are translated to Lisp. -- Bartc From rosuav at gmail.com Wed Aug 3 09:25:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 23:25:23 +1000 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> Message-ID: On Wed, Aug 3, 2016 at 11:04 PM, BartC wrote: > On 03/08/2016 13:36, Chris Angelico wrote: >> >> On Wed, Aug 3, 2016 at 10:16 PM, Rustom Mody >> wrote: >>> >>> There he comes waddling in? Your cute-n-cudly strawman!! >>> A more realistic analogy would be phones >>> The cellphones we use today often crash >>> The first nokia I used never crashed but could still run out of battery >>> And the round-dial landlines of 30 years ago had not even that problem >> >> >> 1986? Yeah, we had a phone from then (granted, I don't remember much >> of 1986, but we had the same handsets in the 1990s), and it could run >> out of battery and lose its phone book. > > > He said landlines. A typical handset would have no battery as it's powered > from the line. And does not have need a memory to function. Nor would it > suffer from lack of signal. Or credit (if you'd paid the last bill). Or get > lost (as it's tethered to the socket). > > Very clunky technology but it was solid! It only did one thing but it did it > incredibly well. Yes, I was talking about landlines. It did have a battery, and it did have a phone book. (And it did get lost, too, but that's because we had a lot of papers and stuff in that corner of the kitchen. We knew it was SOMEWHERE in the pile, but can you find it before the caller gives up?) And it actually suffered from lack of signal more often than you might think. We called them "outages", and they were the phone system's equivalent to power blackouts. Usually from the same cause, too - a car crashed into a pole somewhere and snapped the wires. The only issue I might have about this is whether it dated back to 1986. I asked my mother, and she couldn't remember quite when they came out. Certainly they were around in the early 1990s, and they weren't new then, so they can't be more than a couple years off the 30-years-ago estimate. Can you blame me if I take "30 years ago" and answer it from 28? :) ChrisA From rosuav at gmail.com Wed Aug 3 09:31:31 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Aug 2016 23:31:31 +1000 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On Wed, Aug 3, 2016 at 11:23 PM, BartC wrote: > On 03/08/2016 09:58, Steven D'Aprano wrote: > >> Python is sometimes described as a Lisp with more sensible syntax, so its >> not >> surprising that it is relatively simple to translate Python to Lisp and >> visa >> versa. > > > Translating Python would be easier if everything was implemented as Python. > But sometimes you try to find a .py import module and it doesn't seem to > exist anywhere. (sys.py for example). > > I would like to see how such references are translated to Lisp. Ultimately, you _cannot_ implement everything in Python, unless you create some sort of crazy fudge like having function pointers with real language support, in which case you're writing C code. Some modules have to be implemented in the host language (C for CPython, Java for Jython, etc), in order to provide lower-level functionality. You probably *could* implement the whole math module in pure Python, but you don't want to; and cryptography, likewise, you theoretically could implement in pure Python, and you'd get the same return value, but you risk opening yourself up to timing-based attacks. How, from Lisp, would you call on OpenSSL or another lower-level library? Or do you have to reimplement everything? What about simple things like opening files, how do you do that without native code? ChrisA From gordon at address.invalid Wed Aug 3 09:38:10 2016 From: gordon at address.invalid (Gordon Levi) Date: Wed, 03 Aug 2016 23:38:10 +1000 Subject: Why not allow empty code blocks? References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <20160730103937.6b993ad5@imp> <20160730131105.443fff84@imp> <20160731115347.3c2a8344@imp> Message-ID: "D'Arcy J.M. Cain" wrote: >On Mon, 01 Aug 2016 00:25:58 +1000 >Gordon Levi wrote: >> "D'Arcy J.M. Cain" wrote: >> >I don't care if you are using carrier pigeon. If you send an email >> >address, make it a valid one. >> >> I admire those who use a valid email address on Usenet but it is an >> open invitation for spammers. I doubt if there is anybody who uses >> their primary email address. > >I never said that he had to use his primary address, just a valid one. > >On the other hand I have no throwaway accounts. Every address I use is >a primary one. I have all sorts of methods to block spam. None of >those methods involves moving my spam problem to someone else. How do you manage to avoid the resources wasted on multiple mail servers in order for the spam to each you just so that you can block it? > >But what do I know? I'm just some old fart who grew up being taught >that society works best when you consider the other guy and take >responsibility for your own messes. Old fashioned ideas I know. Nearly twenty years ago The Internet Engineering Task Force produced RFC2606 so that I could consider the other guy and use .invalid as a TLD. It makes a false email address instantly recognisable. If it is not recognised by a person it will be recognised and bounced by a nearby mail server using minimum computer resources. From random832 at fastmail.com Wed Aug 3 10:05:12 2016 From: random832 at fastmail.com (Random832) Date: Wed, 03 Aug 2016 10:05:12 -0400 Subject: print() function with encoding= and errors= parameters? In-Reply-To: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> Message-ID: <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> On Wed, Aug 3, 2016, at 08:29, Malcolm Greene wrote: > Looking for a way to use the Python 3 print() function with encoding and > errors parameters. > > Are there any concerns with closing and re-opening sys.stdout so > sys.stdout has a specific encoding and errors behavior? Would this break > other standard libraries that depend on sys.stdout being configured a > specific way? You could duplicate stdout [open(os.dup(sys.stdout.fileno()), ...)] You could make an IO class which sends bytes output to sys.stdout.buffer but won't close it when closed (you know, it'd be nice to be able to have "not owned underlying stream" as an option of the standard IO classes) If your output isn't going to be long, you could print everything to a StringIO and then write to sys.stdout.buffer at the end. From grant.b.edwards at gmail.com Wed Aug 3 10:06:06 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 3 Aug 2016 14:06:06 +0000 (UTC) Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> Message-ID: On 2016-08-03, Rustom Mody wrote: > The first nokia I used never crashed but could still run out of > battery And the round-dial landlines of 30 years ago had not even > that problem Yes, technically, it did. Except the batteries were kept elsewhere, and the telco went to a lot of trouble to make sure they never went dead. -- Grant Edwards grant.b.edwards Yow! If I pull this SWITCH at I'll be RITA HAYWORTH!! gmail.com Or a SCIENTOLOGIST! From __peter__ at web.de Wed Aug 3 11:09:50 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 03 Aug 2016 17:09:50 +0200 Subject: print() function with encoding= and errors= parameters? References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> Message-ID: Random832 wrote: > On Wed, Aug 3, 2016, at 08:29, Malcolm Greene wrote: >> Looking for a way to use the Python 3 print() function with encoding and >> errors parameters. >> Are there any concerns with closing and re-opening sys.stdout so >> sys.stdout has a specific encoding and errors behavior? Would this break >> other standard libraries that depend on sys.stdout being configured a >> specific way? Can you give an example of what you have in mind? One that would not be considered a bug in said library? > You could duplicate stdout [open(os.dup(sys.stdout.fileno()), ...)] > > You could make an IO class which sends bytes output to sys.stdout.buffer > but won't close it when closed (you know, it'd be nice to be able to > have "not owned underlying stream" as an option of the standard IO > classes) > > If your output isn't going to be long, you could print everything to a > StringIO and then write to sys.stdout.buffer at the end. I'm unsure about this myself -- wouldn't it be better to detach the underlying raw stream? Like >>> import sys, io >>> print("?hnlich ?blich m?glich") ?hnlich ?blich m?glich >>> sys.stdout = io.TextIOWrapper(sys.stdout.detach(), encoding="ascii", errors="xmlcharrefreplace") >>> print("?hnlich ?blich m?glich") ähnlich üblich möglich The ValueError raised if you try to write to the original stdout >>> print("?hnlich ?blich m?glich", file=sys.__stdout__) Traceback (most recent call last): File "", line 1, in ValueError: underlying buffer has been detached looks like a feature to me. PS: An alternative would be to set the environment variable: $ PYTHONIOENCODING=ascii:backslashreplace python3 -c 'print("Sm?rrebr?d")' Sm\xf8rrebr\xf8d From random832 at fastmail.com Wed Aug 3 11:30:19 2016 From: random832 at fastmail.com (Random832) Date: Wed, 03 Aug 2016 11:30:19 -0400 Subject: print() function with encoding= and errors= parameters? In-Reply-To: References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> Message-ID: <1470238219.2687065.684960497.1F4424C1@webmail.messagingengine.com> On Wed, Aug 3, 2016, at 11:09, Peter Otten wrote: > I'm unsure about this myself -- wouldn't it be better to detach the > underlying raw stream? Like Well, "better" depends on your point of view. > The ValueError raised if you try to write to the original stdout > looks like a feature to me. Maybe. But if the goal is to not cause problems for other code that consumes the original stdio, this is very much not a feature. Of course, it's hard to imagine in what kind of program design it is reasonable for there not to be a single guiding principle controlling all of the code that sends output to stdout, but I'm not the one asking the question here. From python at bdurham.com Wed Aug 3 11:35:35 2016 From: python at bdurham.com (Malcolm Greene) Date: Wed, 03 Aug 2016 11:35:35 -0400 Subject: print() function with encoding= and errors= parameters? In-Reply-To: References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> Message-ID: <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> Thank you Random832 and Peter - excellent ideas. My use case was diagnostic output being (temporarily) output to stdout via debug related print statements. The output is for debugging only and not meant for production. I was looking for a solution that would allow me to output to the console with as few changes to the original scripts as possible, eg. non-invasive except for the print statements themselves. When debugging under Linux/OSX, standard print statements work fine because their stdouts' encoding is UTF-8. But under Windows, the stdout is workstation specific and *never* UTF-8. So the occasional non-ASCII string trips up our diagnostic output when tested under Windows. Peter's suggestion to set the PYTHONIOENCODING [1] environment variable is the non-invasive, diagnostic only, non-production solution I was looking for ... for the use case at hand. Again, thank you both. Malcolm [1] PYTHONIOENCODING=ascii:backslashreplace From rosuav at gmail.com Wed Aug 3 11:44:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 01:44:55 +1000 Subject: print() function with encoding= and errors= parameters? In-Reply-To: <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> Message-ID: On Thu, Aug 4, 2016 at 1:35 AM, Malcolm Greene wrote: > My use case was diagnostic output being (temporarily) output to stdout > via debug related print statements. The output is for debugging only and > not meant for production. I was looking for a solution that would allow > me to output to the console with as few changes to the original scripts > as possible, eg. non-invasive except for the print statements > themselves. Don't forget that the print function can simply be shadowed. If you have a lot of code that uses print(...) to output text, and you want to quickly transform it so it uses ASCII + backslash-replace, you could simply: def print(msg): sys.stdout.write(msg.encode("ASCII", errors="backslashreplace").decode("ASCII") + "\n") or whatever other transformation you want. Your current situation may be solved, but this is a great tool in your toolbox, and worth bearing in mind. ChrisA From python at bdurham.com Wed Aug 3 12:44:09 2016 From: python at bdurham.com (Malcolm Greene) Date: Wed, 03 Aug 2016 12:44:09 -0400 Subject: print() function with encoding= and errors= parameters? In-Reply-To: References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> Message-ID: <1470242649.451212.685043049.6872533B@webmail.messagingengine.com> Chris, > Don't forget that the print function can simply be shadowed. I did forget! Another excellent option. Thank you! Malcolm From bc at freeuk.com Wed Aug 3 14:52:05 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 19:52:05 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On 03/08/2016 14:31, Chris Angelico wrote: > On Wed, Aug 3, 2016 at 11:23 PM, BartC wrote: >> On 03/08/2016 09:58, Steven D'Aprano wrote: >> >>> Python is sometimes described as a Lisp with more sensible syntax, so its >>> not >>> surprising that it is relatively simple to translate Python to Lisp and >>> visa >>> versa. >> >> >> Translating Python would be easier if everything was implemented as Python. >> But sometimes you try to find a .py import module and it doesn't seem to >> exist anywhere. (sys.py for example). >> >> I would like to see how such references are translated to Lisp. > > Ultimately, you _cannot_ implement everything in Python, unless you > create some sort of crazy fudge like having function pointers with > real language support, in which case you're writing C code. Some > modules have to be implemented in the host language (C for CPython, > Java for Jython, etc), in order to provide lower-level functionality. > You probably *could* implement the whole math module in pure Python, > but you don't want to; and cryptography, likewise, you theoretically > could implement in pure Python, and you'd get the same return value, > but you risk opening yourself up to timing-based attacks. > > How, from Lisp, would you call on OpenSSL or another lower-level > library? Or do you have to reimplement everything? What about simple > things like opening files, how do you do that without native code? The approach I use is to provide direct access, from the language, to external .dll or .so files. Then it is possible, with a suitable interface *written in the byte-code language*, to directly call C's fopen() for example. I believe Python also has a means of directly calling DLLs, so that in theory its file system can be build on top of the C runtime (msvcrt.dll or Linux' equivalent). There still needs to be some magic support for this mechanism, but more can be written in Python. -- Bartc From eryksun at gmail.com Wed Aug 3 15:10:49 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 3 Aug 2016 19:10:49 +0000 Subject: print() function with encoding= and errors= parameters? In-Reply-To: <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> Message-ID: On Wed, Aug 3, 2016 at 3:35 PM, Malcolm Greene wrote: > But under Windows, the stdout is workstation specific and *never* UTF-8. So the > occasional non-ASCII string trips up our diagnostic output when tested under > Windows. You could use win_unicode_console enabled in sitecustomize or usercustomize. https://pypi.python.org/pypi/win_unicode_console From python at bdurham.com Wed Aug 3 15:29:53 2016 From: python at bdurham.com (Malcolm Greene) Date: Wed, 03 Aug 2016 15:29:53 -0400 Subject: print() function with encoding= and errors= parameters? In-Reply-To: References: <1470227371.397715.684773553.140807A6@webmail.messagingengine.com> <1470233112.2667238.684861393.67022A98@webmail.messagingengine.com> <1470238535.436743.684955321.4B7A4C81@webmail.messagingengine.com> Message-ID: <1470252593.485976.685203633.54D01262@webmail.messagingengine.com> > You could use win_unicode_console enabled in sitecustomize or usercustomize. > https://pypi.python.org/pypi/win_unicode_console The pypi link you shared has an excellent summary of the issues associated when working Unicode from the Windows terminal. Thank you Eryk. Malcolm From rosuav at gmail.com Wed Aug 3 16:12:17 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 06:12:17 +1000 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On Thu, Aug 4, 2016 at 4:52 AM, BartC wrote: > On 03/08/2016 14:31, Chris Angelico wrote: >> >> On Wed, Aug 3, 2016 at 11:23 PM, BartC wrote: >>> >>> On 03/08/2016 09:58, Steven D'Aprano wrote: >>> >>> But sometimes you try to find a .py import module and it doesn't seem to >>> exist anywhere. (sys.py for example). >> >> Ultimately, you _cannot_ implement everything in Python, unless you >> create some sort of crazy fudge like having function pointers with >> real language support, in which case you're writing C code. Some >> modules have to be implemented in the host language (C for CPython, >> Java for Jython, etc), in order to provide lower-level functionality. > > The approach I use is to provide direct access, from the language, to > external .dll or .so files. > > Then it is possible, with a suitable interface *written in the byte-code > language*, to directly call C's fopen() for example. Fairly common approach - and it means you'll never find those .py files. So it's no different from looking for sys.py and not finding it, except that in the case of CPython's sys, it's not even a DLL - it's part of the interpreter core. ChrisA From alister.ware at ntlworld.com Wed Aug 3 16:40:34 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 03 Aug 2016 20:40:34 GMT Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <28e12b9b-4e9c-4a19-9ca1-74cfca782293@googlegroups.com> Message-ID: <6Fsoz.48592$cC.29890@fx45.am4> On Wed, 03 Aug 2016 14:06:06 +0000, Grant Edwards wrote: > On 2016-08-03, Rustom Mody wrote: > >> The first nokia I used never crashed but could still run out of battery >> And the round-dial landlines of 30 years ago had not even that problem > > Yes, technically, it did. Except the batteries were kept elsewhere, > and the telco went to a lot of trouble to make sure they never went > dead. but the equipment at the exchange could miss a pulse & connect you to an incorrect number - equivalent of a bug? before that, you had to ask the nice young lady at the local exchange to physically patch you in to the correct circuit (& she could make an error as well) the "original" phones had no such problem as they ware simply a direct point to point connection (Bells first call is documented "come here Watson I need you" although no-one* remembers what Elisha Grey said or to whom). *ok I suppose someone might but it is not common knowledge (& I bet we still get some pedant disputing this ;-) ) -- Imagination is more important than knowledge. -- Albert Einstein From bc at freeuk.com Wed Aug 3 16:53:27 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 21:53:27 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On 03/08/2016 21:12, Chris Angelico wrote: > On Thu, Aug 4, 2016 at 4:52 AM, BartC wrote: >> On 03/08/2016 14:31, Chris Angelico wrote: >>> >>> On Wed, Aug 3, 2016 at 11:23 PM, BartC wrote: >>>> >>>> On 03/08/2016 09:58, Steven D'Aprano wrote: >>>> >>>> But sometimes you try to find a .py import module and it doesn't seem to >>>> exist anywhere. (sys.py for example). >>> >>> Ultimately, you _cannot_ implement everything in Python, unless you >>> create some sort of crazy fudge like having function pointers with >>> real language support, in which case you're writing C code. Some >>> modules have to be implemented in the host language (C for CPython, >>> Java for Jython, etc), in order to provide lower-level functionality. >> >> The approach I use is to provide direct access, from the language, to >> external .dll or .so files. >> >> Then it is possible, with a suitable interface *written in the byte-code >> language*, to directly call C's fopen() for example. > > Fairly common approach - and it means you'll never find those .py > files. So it's no different from looking for sys.py and not finding > it, except that in the case of CPython's sys, it's not even a DLL - > it's part of the interpreter core. I don't understand. With direct access to msvcrt.dll, the interface to that will be inside a .py file. Not buried away somewhere else. -- Bartc From rosuav at gmail.com Wed Aug 3 17:39:31 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 07:39:31 +1000 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On Thu, Aug 4, 2016 at 6:53 AM, BartC wrote: > On 03/08/2016 21:12, Chris Angelico wrote: >> >> Fairly common approach - and it means you'll never find those .py >> files. So it's no different from looking for sys.py and not finding >> it, except that in the case of CPython's sys, it's not even a DLL - >> it's part of the interpreter core. > > > I don't understand. With direct access to msvcrt.dll, the interface to that > will be inside a .py file. Not buried away somewhere else. Oh. CPython takes a much simpler approach: instead of forcing a Python module to "think like C", it allows a C module to export functions to Python. So when you "import math", you actually get something from a .so/.dll file. I've no idea how Jython does this; it appears that math.__file__ is AttributeError. Ditto PyPy. ChrisA From eryksun at gmail.com Wed Aug 3 18:02:45 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 3 Aug 2016 22:02:45 +0000 Subject: Issue with ctypes and callback functions In-Reply-To: <0E5668E32633FB4FBBA1709825C0EDFDD24B77@SCHEXMB2.acresso.com> References: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> <0E5668E32633FB4FBBA1709825C0EDFDD24B77@SCHEXMB2.acresso.com> Message-ID: On Wed, Aug 3, 2016 at 1:21 PM, Bill Somerville wrote: > I can extend the SWIG shadow class to do the creation of the prototype and thunk, the > required cast above and any extra attributes or methods like _as_parameter_ and > from_param() but I can't see any way of having the Python callback (my_cb) magically > receive the SWIG wrapped 'my_type' struct. from_param is a hook method for a type that's set in a function pointer's argtypes. It gets called to convert an argument when the function pointer is called from Python. The return value can be a ctypes instance, an object with a hard-coded conversion (e.g. a string or integer), or an object that defines an _as_parameter_ attribute. Only ctypes types are supported in callbacks, which unfortunately isn't documented clearly. Specifically, the class dict needs to be an extended C storage dict (i.e. StgDictObject), either to look up the getfunc of a simple type or to ensure that instantiating a non-simple type returns a ctypes instance (i.e. CDataObject) with a known size. The relevant code in _CallPythonObject is as follows (when stripped of declarations and error handling): cnv = PySequence_GetItem(converters, i); dict = PyType_stgdict(cnv); if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) { v = dict->getfunc(*pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, v); } else if (dict) { obj = (CDataObject *)PyObject_CallFunctionObjArgs(cnv, NULL); memcpy(obj->b_ptr, *pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, (PyObject *)obj); } else { PyErr_SetString(PyExc_TypeError, "cannot build parameter"); I don't have much experience with SWIG. Does it provide some means to instantiate a wrapped type from an address? If it does, then you can use a void pointer as the callback parameter. From bc at freeuk.com Wed Aug 3 18:21:19 2016 From: bc at freeuk.com (BartC) Date: Wed, 3 Aug 2016 23:21:19 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On 03/08/2016 22:39, Chris Angelico wrote: > On Thu, Aug 4, 2016 at 6:53 AM, BartC wrote: >> On 03/08/2016 21:12, Chris Angelico wrote: >>> >>> Fairly common approach - and it means you'll never find those .py >>> files. So it's no different from looking for sys.py and not finding >>> it, except that in the case of CPython's sys, it's not even a DLL - >>> it's part of the interpreter core. >> >> >> I don't understand. With direct access to msvcrt.dll, the interface to that >> will be inside a .py file. Not buried away somewhere else. > > Oh. CPython takes a much simpler approach: instead of forcing a Python > module to "think like C", it allows a C module to export functions to > Python. So when you "import math", you actually get something from a > .so/.dll file. But is this a generic mechanism that works for /any/ .dll file, or does there have to be dedicated support for each of the 60 built-in modules? I'm talking about the former. Then something like a file library can be written in 100% Python, built around calls to msvcrt.dll for example. Since such libraries usually come with the OS, anyone investigating how a library works will see only Python, until it gets to the foreign functions, which have their own well-documented interface. You don't just hit a brick wall and be forced to delve into the innards of CPython, which should be concerning itself with implementing the language not providing libraries too. -- Bartc From rosuav at gmail.com Wed Aug 3 18:31:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 08:31:03 +1000 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On Thu, Aug 4, 2016 at 8:21 AM, BartC wrote: > But is this a generic mechanism that works for /any/ .dll file, or does > there have to be dedicated support for each of the 60 built-in modules? > > I'm talking about the former. Frankly, I don't want that. Most C APIs are horrendously unPythonic, so you'd need a wrapper anyway; plus, you shouldn't have to use C-like idioms except VERY rarely (eg pointers and stuff). You *can* use ctypes in CPython, but that's the failure mode for when you don't have any better option. The best option, for a DLL that doesn't already have any Python interface, is to write something in Cython. ChrisA From no.email at nospam.invalid Wed Aug 3 19:25:14 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 03 Aug 2016 16:25:14 -0700 Subject: Why not allow empty code blocks? References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: <87invh4eqt.fsf@jester.gateway.pace.com> BartC writes: > sometimes you try to find a .py import module and it > doesn't seem to exist anywhere. (sys.py for example). > I would like to see how such references are translated to Lisp. (require 'sys) From bc at freeuk.com Wed Aug 3 19:48:42 2016 From: bc at freeuk.com (BartC) Date: Thu, 4 Aug 2016 00:48:42 +0100 Subject: Why not allow empty code blocks? In-Reply-To: <87invh4eqt.fsf@jester.gateway.pace.com> References: <579B1ABB.1070500@rece.vub.ac.be> <20160731115347.3c2a8344@imp> <20160731120434.3f587ad9@imp> <9a08c631-ce14-dc04-f038-ab31bd3ec350@gmail.com> <28fa2205-5d30-4d00-9ab9-b56c66274dd5@googlegroups.com> <3d96c99b-72fa-439c-93b4-abef2b9fd8a0@googlegroups.com> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> <87invh4eqt.fsf@jester.gateway.pace.com> Message-ID: On 04/08/2016 00:25, Paul Rubin wrote: > BartC writes: >> sometimes you try to find a .py import module and it >> doesn't seem to exist anywhere. (sys.py for example). >> I would like to see how such references are translated to Lisp. > > (require 'sys) What does that do? Don't tell me that every Lisp contains a full implementation of everything defined in Python's sys module! -- Bartc From bc at freeuk.com Wed Aug 3 19:51:15 2016 From: bc at freeuk.com (BartC) Date: Thu, 4 Aug 2016 00:51:15 +0100 Subject: Why not allow empty code blocks? In-Reply-To: References: <579B1ABB.1070500@rece.vub.ac.be> <579f207f$0$11110$c3e8da3@news.astraweb.com> <01f3d086-ba00-43c6-8f86-712fb940b054@googlegroups.com> <57a0dd53$0$1620$c3e8da3$5496439d@news.astraweb.com> <87a8gv562h.fsf@jester.gateway.pace.com> <57a1b23c$0$11093$c3e8da3@news.astraweb.com> Message-ID: On 03/08/2016 23:31, Chris Angelico wrote: > On Thu, Aug 4, 2016 at 8:21 AM, BartC wrote: >> But is this a generic mechanism that works for /any/ .dll file, or does >> there have to be dedicated support for each of the 60 built-in modules? >> >> I'm talking about the former. > > Frankly, I don't want that. Most C APIs are horrendously unPythonic, > so you'd need a wrapper anyway; plus, you shouldn't have to use C-like > idioms except VERY rarely (eg pointers and stuff). Yes, you'd need an ugly wrapper. But at least it would be in Python. As it is, you still need a wrapper, it's still ugly, and it's probably written in C and buried in CPython somewhere. -- Bartc From steve+python at pearwood.info Wed Aug 3 21:27:25 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 04 Aug 2016 11:27:25 +1000 Subject: Getting started with type-checking Message-ID: <57a299fe$0$1608$c3e8da3$5496439d@news.astraweb.com> Is there a good guide to getting started with type-checking Python code? Specifically as an aid to porting from Python 2 to 3. There are the PEPs, of course. Anything else? Apart from mypy, what type checkers are available? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Wed Aug 3 22:22:05 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 3 Aug 2016 19:22:05 -0700 (PDT) Subject: ctypes Usage Note In-Reply-To: References: <4747f520-8091-486c-9801-3b9cee209799@googlegroups.com> Message-ID: On Tuesday, August 2, 2016 at 6:46:45 PM UTC+12, I wrote: > On Tuesday, August 2, 2016 at 5:18:14 PM UTC+12, eryk sun wrote: > >> I recommend using ctypes.CDLL instead. ctypes.cdll is pretty much >> pointless on Unix systems (and a potential problem on Windows due to >> cached function pointers). It also doesn't allow passing the mode, >> handle, and use_errno parameters. > > None of which I currently care about. Actually, thinking about it, there might be a point to specifying RTLD_GLOBAL. Which makes me wonder: why isn?t it the default? As for errno, that?s an egregious piece of POSIX brain damage. You?ll notice that just about nothing else uses it, apart from (the various implementations of) libc, which have to. From lawrencedo99 at gmail.com Wed Aug 3 22:34:18 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 3 Aug 2016 19:34:18 -0700 (PDT) Subject: can't add variables to instances of built-in classes In-Reply-To: References: <4f796d95-0e87-4610-a0a8-1eff07c60ace@googlegroups.com> <578b9e62$0$22141$c3e8da3$5496439d@news.astraweb.com> <3e934f83-2076-407d-b85b-686ecdec6ff8@googlegroups.com> <2bc29306-ac90-4424-84c2-64be494c5a75@googlegroups.com> <230d1aff-b1a3-4f16-b9a6-b5d9fd07e4b4@googlegroups.com> <62dd86d4-74f7-4e97-a29c-7e45aced77d9@googlegroups.com> <18acf908-b761-4a60-8d7c-5e4576b03320@googlegroups.com> Message-ID: On Thursday, July 21, 2016 at 2:29:53 PM UTC+12, Chris Angelico wrote: > Using __slots__ basically takes your object down to the level of a > Java one. Is Java like a bogeyman you use to scare people with if they don?t write Python code the way you like it? From rosuav at gmail.com Wed Aug 3 23:00:34 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 13:00:34 +1000 Subject: can't add variables to instances of built-in classes In-Reply-To: References: <4f796d95-0e87-4610-a0a8-1eff07c60ace@googlegroups.com> <578b9e62$0$22141$c3e8da3$5496439d@news.astraweb.com> <3e934f83-2076-407d-b85b-686ecdec6ff8@googlegroups.com> <2bc29306-ac90-4424-84c2-64be494c5a75@googlegroups.com> <230d1aff-b1a3-4f16-b9a6-b5d9fd07e4b4@googlegroups.com> <62dd86d4-74f7-4e97-a29c-7e45aced77d9@googlegroups.com> <18acf908-b761-4a60-8d7c-5e4576b03320@googlegroups.com> Message-ID: On Thu, Aug 4, 2016 at 12:34 PM, Lawrence D?Oliveiro wrote: > On Thursday, July 21, 2016 at 2:29:53 PM UTC+12, Chris Angelico wrote: > >> Using __slots__ basically takes your object down to the level of a >> Java one. > > Is Java like a bogeyman you use to scare people with if they don?t write Python code the way you like it? LOL! Not really; it's just that Java's objects are way less featured than Python's. ChrisA From steve+python at pearwood.info Wed Aug 3 23:23:29 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 04 Aug 2016 13:23:29 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> On Wed, 3 Aug 2016 08:16 pm, BartC wrote: > On 03/08/2016 06:43, Steven D'Aprano wrote: > >> Not everything that is done is worth the cognitive burden of memorising a >> special case. > .... > >> In some ways, Python is a more minimalist language than you like. That's >> okay, you're allowed to disagree with some design decisions. > > Well it's minimalist in some ways, and completely the opposite in others! > > It uses minimal basic syntax (missing a couple of loop forms, loop > controls, switch/case, select-expressions... it's just a handful of > features). > > But then you get to the standard library, and the plethora of different > data types, methods and options. It's never-ending! "Never-ending" is a bit of an exaggeration. You should check out the Java libraries, and how many different kinds of key:value mapping they provide. They make Python's collection of types look positively anaemic :-) In general, most types and operations in Python earned their place by proving their usefulness. They proved that they are useful, hard to program correctly, easy to get wrong, or some combination of all three. Many of them have gone through a gruelling process of discussion via a Python Enhancement Proposal (PEP) before being accepted. Having written two PEPs, I can tell you that in general the Python core developers attitude to adding new features, types or operations is to be very conservative and say No. Most suggestions get rejected even before that point. For instance, Guido used to reject the idea of having a ternary if operator for years. Python didn't need it, you can use an if...else statement, or when you need an if expression, use shortcut bool operators: result = condition and x or y gives x if condition is true and y if condition is false. At least, that's what Guido used to say... until he personally got bitten by a flaw in the argument. If x is a falsey argument, y is always returned. It took something in excess of 15 years before anyone publicly noticed this flaw, and when Guido did, Python pretty quickly got a ternary if operator. > So the idea that remembering 'repeat N' is a cognitive burden, and the > myriad string operations for example are not, is ridiculous. Who says it isn't a cognitive burden? Of course it is. The difference is that most of the string methods carry their own weight in usefulness versus burden, and "repeat N" doesn't (according to the core developers). You have weighed "repeat N" high on the usefulness side and low on the burden side, and Guido has done the opposite. This is, of course, a subjective argument. I'm not trying to convince you that you're wrong and Python is right. I completely accept that different people have different opinions on the usefulness versus burden of adding a specific feature. (That's why we have different languages, and some people like Perl where there are a million ways to do everything and some people don't.) I'm just trying to get you to understand where Python is coming from, not necessary to agree with every one of its design decisions, but to understand why they are the way they are. Its trivial to perform "repeat N" using the standard "for x in ..." syntax and built-in range() function, so including a dedicated "repeat N" syntax doesn't add much to the language. It's a trivial operation easily performed using a for loop. So the benefit is small. The cost is also small, but greater than the benefit: there's a new keyword, which means any code using "repeat" as a variable, function or method will break. Its more code in the compiler (even if it's only an extra ten bytes, that's still more). It's another feature that needs to be documented and tested. It's another decision for users to make, "should I use repeat N, or a for loop?", and another feature beginners to learn. Individually these costs are small, but they aren't *zero*. Collectively, according to the judgement of the people making the decision, they add up to more cost than the minimal benefit that "repeat N" would bring. But to take your example of str.encode... if Python didn't provide it, it would be a dozen kinds of a pain in the arse to provide it yourself. Especially with any sort of efficiency. Not to mention actually coming up with the encodings themselves. There are *dozens* of them. Adding one more string method "encode" to the dozens already there is not much of an additional burden. It doesn't stop people from using "encode" as a function name or variable, since the encode method is isolated to the str namespace. It provides a very important function, one which is critical in the post-ASCII world, and it is too difficult to expect users to implement their own. So the benefit is much greater than the cost. On the other hand, there's also str.swapcase(), which only remains due to a combination of nostalgia and backwards compatibility. Occasionally people, including Guido, suggest removing it: its pretty much useless, and hardly anyone uses it. It isn't hard to write your own version, and its unlikely to be used in performance-critical code. So removing the feature has very little cost. But the benefit from removal is also correspondingly tiny. Even if it only affects ten Python developers in the whole world, forcing them to write their own swapcase function, that cost is greater than the benefit from removing it. And so it stays. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Thu Aug 4 03:23:14 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 00:23:14 -0700 (PDT) Subject: Ide or code editor confusion In-Reply-To: References: <2184a0ad-9d51-42db-ae11-5d0ab4512582@googlegroups.com> <7f142caf-5e3c-4fcf-818a-017b593fd0f7@googlegroups.com> Message-ID: <6688b08e-75cc-4132-b0e0-f63dcbfa4bd2@googlegroups.com> On Friday, July 22, 2016 at 2:38:10 AM UTC+12, Chris Angelico wrote: > On Fri, Jul 22, 2016 at 12:22 AM, sigmaphine1914 wrote: >> Beginning Python: using Python 2.6 and Python 3.1. By James Payne >> >> Part II. > > Ugh, that's extremely old now. That?s why I?ve come to the conclusion it?s a waste of time buying books on computing topics. They start to reek of decay while they?re still on the shelf. From lawrencedo99 at gmail.com Thu Aug 4 03:28:25 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 00:28:25 -0700 (PDT) Subject: Float In-Reply-To: <78253415-cd16-45f1-995a-a521f613ea86@googlegroups.com> References: <579b67b0$0$1614$c3e8da3$5496439d@news.astraweb.com> <367666cb-224f-4b16-88bb-cd1ada13a13b@googlegroups.com> <579c82a6$0$1619$c3e8da3$5496439d@news.astraweb.com> <78253415-cd16-45f1-995a-a521f613ea86@googlegroups.com> Message-ID: <794b00b4-83a8-4087-b63e-6c9352a5facc@googlegroups.com> On Sunday, July 31, 2016 at 3:21:09 PM UTC+12, Rustom Mody wrote: > In writing Tex he went out of his way to implement his own fixed point > system and avoid using the builtin hardware floating point > https://en.wikipedia.org/wiki/TeX#Development > > To the extent its feasible it?s advisable to follow the example of Knuth > [No not writing your own system, but avoiding when/if possible] This is why IEEE-854 was developed, and why Python has the Decimal type . From bc at freeuk.com Thu Aug 4 05:13:58 2016 From: bc at freeuk.com (BartC) Date: Thu, 4 Aug 2016 10:13:58 +0100 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 04/08/2016 04:23, Steven D'Aprano wrote: > On Wed, 3 Aug 2016 08:16 pm, BartC wrote: >> So the idea that remembering 'repeat N' is a cognitive burden, and the >> myriad string operations for example are not, is ridiculous. > > Who says it isn't a cognitive burden? Of course it is. > > The difference is that most of the string methods carry their own weight in > usefulness versus burden, and "repeat N" doesn't (according to the core > developers). You have weighed "repeat N" high on the usefulness side and OK, let's look at some string features. First, you have string.ascii_uppercase, which is just "ABCDEFGHIJKLMNOPQRSTUVWXYZ". Is that really so indispensable that it has to be built-in to the language? Is it that much of a hardship to assign it once and for all to some variable? And 'string.ascii_uppercase' is not that much more concise than just writing out the alphabet! In the case of "0123456789", the constant name is longer. Now you have string str.lower, str.upper, and str.swapcase. Clearly one of those first two is redundant, as you can implement str.upper by writing str.lower().swapcase() for example. Then these miss a trick by not having an optional length parameter, so that you can operate on the first N characters. Then you can dispense with str.capitalise by writing str.upper(1). (Or str.lower().upper(1) if the current case is unknown.) (And what about str.reverse()? (The comments here about the readability of Python code, and who is entitled to express an opinion about it, are amusing: http://stackoverflow.com/questions/931092/reverse-a-string-in-python)) Compare all that (and I'm sure there's tons more) with, for example, just leaving out the 'i in' in 'for range(N):') -- Bartc From steve+comp.lang.python at pearwood.info Thu Aug 4 05:38:30 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 04 Aug 2016 19:38:30 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a30d18$0$1505$c3e8da3$5496439d@news.astraweb.com> On Thursday 04 August 2016 19:13, BartC wrote: > On 04/08/2016 04:23, Steven D'Aprano wrote: >> On Wed, 3 Aug 2016 08:16 pm, BartC wrote: > >>> So the idea that remembering 'repeat N' is a cognitive burden, and the >>> myriad string operations for example are not, is ridiculous. >> >> Who says it isn't a cognitive burden? Of course it is. >> >> The difference is that most of the string methods carry their own weight in >> usefulness versus burden, and "repeat N" doesn't (according to the core >> developers). You have weighed "repeat N" high on the usefulness side and > > OK, let's look at some string features. > > First, you have string.ascii_uppercase, which is just > "ABCDEFGHIJKLMNOPQRSTUVWXYZ". > > Is that really so indispensable that it has to be built-in to the > language? Is it that much of a hardship to assign it once and for all to > some variable? /facepalm It *is* assigned once and for all to some variable. It's a variable in the "string" module. It is neither a built-in value (like None, True or False) nor a language feature. > And 'string.ascii_uppercase' is not that much more concise than just > writing out the alphabet! In the case of "0123456789", the constant name > is longer. The reason for giving constant values a fixed name is not to save keystrokes, but to have a consistent, readily understandable, self-descriptive name. If you say: from string import digits that's only 26 keystrokes, including a newline. Compared to: digits = '1234567890' which is 22 keystrokes. So it costs you four keystrokes over defining it yourself. Big deal. > Now you have string str.lower, str.upper, and str.swapcase. Clearly one > of those first two is redundant, as you can implement str.upper by > writing str.lower().swapcase() for example. I've already suggested that swapcase() is not very useful, and that it still exists only for backwards compatibility. But are you serious about suggesting that Python should drop str.upper() in favour of having the user write str.lower().swapcase()? At least put a wink or a smiley there, so we know you aren't a total idiot. > Then these miss a trick by not having an optional length parameter, so > that you can operate on the first N characters. That's actually a nice feature. I might request it. It's too late for Python 3.6, but maybe 3.7. It would have to take a start and end position, but that's actually quite clever. > Then you can dispense with str.capitalise by writing str.upper(1). (Or > str.lower().upper(1) if the current case is unknown.) Backwards compatibility would require it stays even if it became redundant. > (And what about str.reverse()? What about it? The canonical way to spell "reverse a string" is with a slice: mystring[::-1] > (The comments here about the readability > of Python code, and who is entitled to express an opinion about it, are > amusing: > http://stackoverflow.com/questions/931092/reverse-a-string-in-python)) Ah, well, Stackoverflow. What did you expect? -- Steve From rosuav at gmail.com Thu Aug 4 05:39:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Aug 2016 19:39:10 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579c2095$0$22142$c3e8da3$5496439d@news.astraweb.com> <579ca644$0$1589$c3e8da3$5496439d@news.astraweb.com> <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Aug 4, 2016 at 7:13 PM, BartC wrote: > On 04/08/2016 04:23, Steven D'Aprano wrote: >> >> On Wed, 3 Aug 2016 08:16 pm, BartC wrote: > > >>> So the idea that remembering 'repeat N' is a cognitive burden, and the >>> myriad string operations for example are not, is ridiculous. >> >> >> Who says it isn't a cognitive burden? Of course it is. >> >> The difference is that most of the string methods carry their own weight >> in >> usefulness versus burden, and "repeat N" doesn't (according to the core >> developers). You have weighed "repeat N" high on the usefulness side and > > > OK, let's look at some string features. Be careful: you're looking at two different things here (the 'string' module, and methods/attributes on the 'str' type). > First, you have string.ascii_uppercase, which is just > "ABCDEFGHIJKLMNOPQRSTUVWXYZ". > > Is that really so indispensable that it has to be built-in to the language? > Is it that much of a hardship to assign it once and for all to some > variable? Once and for all? You mean like in a module? > Now you have string str.lower, str.upper, and str.swapcase. Clearly one of > those first two is redundant, as you can implement str.upper by writing > str.lower().swapcase() for example. "Clearly" only if you're thinking ASCII. You can't actually fudge this either direction: >>> s = "\u1e9e" >>> s.lower().swapcase(), s.upper() ('SS', '?') >>> s = "\u00df" >>> s.upper().swapcase(), s.lower() ('ss', '?') > Then these miss a trick by not having an optional length parameter, so that > you can operate on the first N characters. > > Then you can dispense with str.capitalise by writing str.upper(1). (Or > str.lower().upper(1) if the current case is unknown.) You have to use the longer form for true equivalence, and then you run into the same problem as above. > (And what about str.reverse()? (The comments here about the readability of > Python code, and who is entitled to express an opinion about it, are > amusing: > http://stackoverflow.com/questions/931092/reverse-a-string-in-python)) Slice it backwards. No need for a method, any more than there is need of a str.charAt() method to get individual characters. > Compare all that (and I'm sure there's tons more) with, for example, just > leaving out the 'i in' in 'for range(N):') If you want to push for a shorter syntax for simple repeat loops, this has far better chances of landing than "repeat N" has - "for " is simply an extension of "for in " that discards rather than binding. No new keywords introduced, no fundamentally new looping structure, just the exact same thing as an existing 'for' loop. The difference between "for thing in collection" and "for collection" would then be: # for thing in collection _it = iter(collection) while "moar stuff": try: thing = next(_it) except StopIteration: break # loop body goes here # for collection _it = iter(collection) while "moar stuff": try: next(_it) except StopIteration: break # loop body goes here That wouldn't be a bad thing, IMO. Whether it's worth the various costs I still don't know, but its costs would be *way* lower than adding a new keyword to the language, adding a new control flow form, etc. Less magic. Just the removal of a name binding. ChrisA From tennis at nextgxdx.com Thu Aug 4 09:55:20 2016 From: tennis at nextgxdx.com (Tennis Smith) Date: Thu, 4 Aug 2016 06:55:20 -0700 (PDT) Subject: Use pip to install non-python? Message-ID: I have several utility scripts I want to install in /usr/local/bin. Some are python, some are simple bash scripts. Can I use pip to install them? If so, can anyone point me to some examples? Thanks, -T From jon+usenet at unequivocal.eu Thu Aug 4 10:13:03 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Thu, 4 Aug 2016 14:13:03 -0000 (UTC) Subject: Use pip to install non-python? References: Message-ID: On 2016-08-04, Tennis Smith wrote: > I have several utility scripts I want to install in /usr/local/bin. > Some are python, some are simple bash scripts. Can I use pip to > install them? If so, can anyone point me to some examples? By the looks of it*, you should be able to do this: setup( .... scripts=[ "scripts/bash1", "scripts/bash2", "scripts/bash3", ], console_scripts=[ "foo=package.module:func1", "bar=package.module.func2", "baz=package.module.func3", ], ) The first list is the list of bash scripts, which will just be copied into the right place, and the second list is the Python functions you want calling, and pip will automagically generate the appropriate script to call your code when the command is executed. * Barely documented at https://packaging.python.org/distributing/#scripts From bemen77 at gmail.com Thu Aug 4 11:31:22 2016 From: bemen77 at gmail.com (GBANE FETIGUE) Date: Thu, 4 Aug 2016 08:31:22 -0700 (PDT) Subject: Python Error message Message-ID: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> Hi, I am running a python script to run some CURL commands, and return the response which is the applicationId and the versionId. I was able to do it. Now the versionId value supposed to be used on the second CURL as a value of the applications key which is an array. but it doesn't work.I 'll post the error after running the command as well as the script. It seems like I have an error somewhere because the curl works manually if i run. ec2-user at ip-172-31-21-77 playbooks]$ python mmc-uploader.py % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2318 0 119 100 2199 496 9173 --:--:-- --:--:-- --:--:-- 9200 Your applicationId is: local$fc9277b0-a5b1-4602-8730-714ab7472744 Your versionId is: local$423da1c8-c4e1-47af-9395-57300f839670 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1259 100 1091 100 168 100k 15868 --:--:-- --:--:-- --:--:-- 106k Final responseApache Tomcat/8.0.26 - Error report

HTTP Status 415 -

type Status report

message

description The server refused this request because the request entity is in a format not supported by the requested resource for the requested method.

Apache Tomcat/8.0.26

Seems the named id already exists! That's the script : from subprocess import check_output, STDOUT import json response = check_output(["curl", "--basic", "-u", "admin:admin", "-F", "file=@/var/lib/jenkins/workspace/Demo-Ci-cd-automation/playbooks/ms3-simple-hello-world-app-1.0.0-SNAPSHOT.zip", "-F", "name=ms3-simple-hello-world-app", "-F", "version=1.0.0", "--header", "\"Content-Type: multipart/form-data\"", "http://52.73.56.141:8080/mmc-console-3.6.2/api/repository"]) try: parsed_response = json.loads(response) print "Your applicationId is: " + parsed_response[u'applicationId'] version_id = parsed_response[u'versionId'] print "Your versionId is: " + version_id except: print 'Seems the named application already exists!' print 'Seems the named versionId already exists!' response = check_output(["curl", "--basic", "-u", "admin:admin", "-d", "'{\"name\" : \"ms3-simple-hello-world-app\" , \"servers\": [ \"local$d50bdc24-ff04-4327-9284-7bb708e21c25\" ], \"applications\": [ \"" + version_id + "\"]}'", "--header", "\'Content-Type: application/json\'", "http://52.73.56.141:8080/mmc-console-3.6.2/api/deployments"]) print 'Final response' + response try: parsed_response = json.loads(response) deployid = parsed_response[u'id'] print "Your deployid is: " + deployid except: print 'Seems the named id already exists!' From rosuav at gmail.com Thu Aug 4 11:51:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 01:51:04 +1000 Subject: Python Error message In-Reply-To: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> Message-ID: On Fri, Aug 5, 2016 at 1:31 AM, GBANE FETIGUE wrote: > try: > parsed_response = json.loads(response) > print "Your applicationId is: " + parsed_response[u'applicationId'] > version_id = parsed_response[u'versionId'] > print "Your versionId is: " + version_id > except: > print 'Seems the named application already exists!' > print 'Seems the named versionId already exists!' > This is a very bad idea. You're catching every possible exception and printing out the same message for all of them. And then continuing blithely on. Instead, catch ONLY the exceptions you really expect to be seeing (I'm guessing KeyError here). If anything else goes wrong, let the exception bubble up. ChrisA From steve+python at pearwood.info Thu Aug 4 11:56:01 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 05 Aug 2016 01:56:01 +1000 Subject: Python Error message References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> Message-ID: <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> On Fri, 5 Aug 2016 01:31 am, GBANE FETIGUE wrote: > try: > parsed_response = json.loads(response) > deployid = parsed_response[u'id'] > print "Your deployid is: " + deployid > except: > print 'Seems the named id already exists!' I'm not going to try to debug your code blindfolded with my hands tied behind my back. Get rid of those "try...except" blocks so that you can see what error is *actually* happening. As you have it now, an error happens, somewhere. You don't know where the error is, or what it is, but Python generates a nice exception showing all the detail you need to debug. But you catch that exception, throw it away, and then print a lie. It is **not true** that the named ID already exists. That is not what the error is, so why does your script tell a lie? The output from the server might give you a clue: "The server refused this request because the request entity is in a format not supported by the requested resource for the requested method." Never[1] use a bare "try...except". It is the worst thing you can do to a Python script, making it almost impossible to debug. https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ Fix that problem first, get rid of the "try...except" and lying print messages, and then either the bug will be obvious, or we can debug further. [1] There are exceptions to this rule, for experts. But if you need to ask what they are, you're not ready to know. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ikorot01 at gmail.com Thu Aug 4 12:09:33 2016 From: ikorot01 at gmail.com (Igor Korot) Date: Thu, 4 Aug 2016 12:09:33 -0400 Subject: Python Error message In-Reply-To: <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven, On Thu, Aug 4, 2016 at 11:56 AM, Steven D'Aprano wrote: > On Fri, 5 Aug 2016 01:31 am, GBANE FETIGUE wrote: > >> try: >> parsed_response = json.loads(response) >> deployid = parsed_response[u'id'] >> print "Your deployid is: " + deployid >> except: >> print 'Seems the named id already exists!' > > > I'm not going to try to debug your code blindfolded with my hands tied > behind my back. Get rid of those "try...except" blocks so that you can see > what error is *actually* happening. > > As you have it now, an error happens, somewhere. You don't know where the > error is, or what it is, but Python generates a nice exception showing all > the detail you need to debug. > > But you catch that exception, throw it away, and then print a lie. > > It is **not true** that the named ID already exists. That is not what the > error is, so why does your script tell a lie? > > The output from the server might give you a clue: > > "The server refused this request because the request entity is in a format > not supported by the requested resource for the requested method." > > > Never[1] use a bare "try...except". It is the worst thing you can do to a > Python script, making it almost impossible to debug. > > https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ > > Fix that problem first, get rid of the "try...except" and lying print > messages, and then either the bug will be obvious, or we can debug further. > > > > > > > [1] There are exceptions to this rule, for experts. But if you need to ask > what they are, you're not ready to know But even the experts will never write such a code - you never know what happens in a month. Server might throw some new exception, you may move on to a different project, etc, etc. ;-) Thank you. > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list From python at mrabarnett.plus.com Thu Aug 4 12:14:05 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 4 Aug 2016 17:14:05 +0100 Subject: Python Error message In-Reply-To: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> Message-ID: <408923f3-a9e5-1cff-db01-c76a67408450@mrabarnett.plus.com> On 2016-08-04 16:31, GBANE FETIGUE wrote: > Hi, > I am running a python script to run some CURL commands, and return the response which is the applicationId and the versionId. I was able to do it. Now the versionId value supposed to be used on the second CURL as a value of the applications key which is an array. but it doesn't work.I 'll post the error after running the command as well as the script. It seems like I have an error somewhere because the curl works manually if i run. > What do you put on the command line when you do it manually? > ec2-user at ip-172-31-21-77 playbooks]$ python mmc-uploader.py > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 100 2318 0 119 100 2199 496 9173 --:--:-- --:--:-- --:--:-- 9200 > Your applicationId is: local$fc9277b0-a5b1-4602-8730-714ab7472744 > Your versionId is: local$423da1c8-c4e1-47af-9395-57300f839670 > % Total % Received % Xferd Average Speed Time Time Time Current > Dload Upload Total Spent Left Speed > 100 1259 100 1091 100 168 100k 15868 --:--:-- --:--:-- --:--:-- 106k > Final responseApache Tomcat/8.0.26 - Error report

HTTP Status 415 -

type Status report

message

description The server refused this request because the request entity is in a format not supported by the reque > sted resource for the requested method.

Apache Tomcat/8.0.26

> Seems the named id already exists! > > That's the script : > > from subprocess import check_output, STDOUT > import json > > response = check_output(["curl", "--basic", "-u", "admin:admin", "-F", "file=@/var/lib/jenkins/workspace/Demo-Ci-cd-automation/playbooks/ms3-simple-hello-world-app-1.0.0-SNAPSHOT.zip", "-F", "name=ms3-simple-hello-world-app", "-F", "version=1.0.0", "--header", "\"Content-Type: multipart/form-data\"", "http://52.73.56.141:8080/mmc-console-3.6.2/api/repository"]) > > > try: > parsed_response = json.loads(response) > print "Your applicationId is: " + parsed_response[u'applicationId'] > version_id = parsed_response[u'versionId'] > print "Your versionId is: " + version_id Don't use a bare except because it'll catch _ANY_ exception, including NameError, which could happen if you accidentally misspell a name. > except: > print 'Seems the named application already exists!' > print 'Seems the named versionId already exists!' > This look likes it passes curl some JSON wrapped in "'", so curl won't see something like this: {"key": "value"} (a dict), it'll see something like this: '{"key": "value"}' (a string delimited by '...') which isn't valid JSON. > response = check_output(["curl", "--basic", "-u", "admin:admin", "-d", "'{\"name\" : \"ms3-simple-hello-world-app\" , \"servers\": [ \"local$d50bdc24-ff04-4327-9284-7bb708e21c25\" ], \"applications\": [ \"" + version_id + "\"]}'", "--header", "\'Content-Type: application/json\'", "http://52.73.56.141:8080/mmc-console-3.6.2/api/deployments"]) > > print 'Final response' + response > > try: > parsed_response = json.loads(response) > deployid = parsed_response[u'id'] > print "Your deployid is: " + deployid Another bare except! > except: > print 'Seems the named id already exists!' > 'check_output' and related functions/methods will do the quoting for you. From rosuav at gmail.com Thu Aug 4 12:14:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 02:14:41 +1000 Subject: Python Error message In-Reply-To: References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 5, 2016 at 2:09 AM, Igor Korot wrote: >> [1] There are exceptions to this rule, for experts. But if you need to ask >> what they are, you're not ready to know > > But even the experts will never write such a code - you never know what happens > in a month. Server might throw some new exception, you may move on to > a different project, > etc, etc. ;-) Yes, in those situations you don't write a bare except :) As Steven said, there ARE legit uses; most of them fall under the description "boundary location". ChrisA From python at mrabarnett.plus.com Thu Aug 4 12:19:57 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 4 Aug 2016 17:19:57 +0100 Subject: Python Error message In-Reply-To: References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <23e36680-195e-b840-599e-e225cdc2cee8@mrabarnett.plus.com> On 2016-08-04 17:09, Igor Korot wrote: > Steven, > > On Thu, Aug 4, 2016 at 11:56 AM, Steven D'Aprano > wrote: >> On Fri, 5 Aug 2016 01:31 am, GBANE FETIGUE wrote: >> >>> try: >>> parsed_response = json.loads(response) >>> deployid = parsed_response[u'id'] >>> print "Your deployid is: " + deployid >>> except: >>> print 'Seems the named id already exists!' >> >> >> I'm not going to try to debug your code blindfolded with my hands tied >> behind my back. Get rid of those "try...except" blocks so that you can see >> what error is *actually* happening. >> >> As you have it now, an error happens, somewhere. You don't know where the >> error is, or what it is, but Python generates a nice exception showing all >> the detail you need to debug. >> >> But you catch that exception, throw it away, and then print a lie. >> >> It is **not true** that the named ID already exists. That is not what the >> error is, so why does your script tell a lie? >> >> The output from the server might give you a clue: >> >> "The server refused this request because the request entity is in a format >> not supported by the requested resource for the requested method." >> >> >> Never[1] use a bare "try...except". It is the worst thing you can do to a >> Python script, making it almost impossible to debug. >> >> https://realpython.com/blog/python/the-most-diabolical-python-antipattern/ >> >> Fix that problem first, get rid of the "try...except" and lying print >> messages, and then either the bug will be obvious, or we can debug further. >> >> >> >> >> >> >> [1] There are exceptions to this rule, for experts. But if you need to ask >> what they are, you're not ready to know > > But even the experts will never write such a code - you never know what happens > in a month. Server might throw some new exception, you may move on to > a different project, > etc, etc. ;-) > In those rare occasions when you do write a bare except, you'd re-raise the exception afterwards: try: ... except: print("'tis but a scratch!") raise > Thank you. > From BSomerville at flexerasoftware.com Thu Aug 4 13:23:24 2016 From: BSomerville at flexerasoftware.com (Bill Somerville) Date: Thu, 4 Aug 2016 17:23:24 +0000 Subject: Issue with ctypes and callback functions In-Reply-To: References: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> <0E5668E32633FB4FBBA1709825C0EDFDD24B77@SCHEXMB2.acresso.com> Message-ID: <0E5668E32633FB4FBBA1709825C0EDFDD24DE8@SCHEXMB2.acresso.com> -----Original Message----- From: eryk sun [mailto:eryksun at gmail.com] from_param is a hook method for a type that's set in a function pointer's argtypes. It gets called to convert an argument when the function pointer is called from Python. The return value can be a ctypes instance, an object with a hard-coded conversion (e.g. a string or integer), or an object that defines an _as_parameter_ attribute. Only ctypes types are supported in callbacks, which unfortunately isn't documented clearly. Specifically, the class dict needs to be an extended C storage dict (i.e. StgDictObject), either to look up the getfunc of a simple type or to ensure that instantiating a non-simple type returns a ctypes instance (i.e. CDataObject) with a known size. The relevant code in _CallPythonObject is as follows (when stripped of declarations and error handling): cnv = PySequence_GetItem(converters, i); dict = PyType_stgdict(cnv); if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) { v = dict->getfunc(*pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, v); } else if (dict) { obj = (CDataObject *)PyObject_CallFunctionObjArgs(cnv, NULL); memcpy(obj->b_ptr, *pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, (PyObject *)obj); } else { PyErr_SetString(PyExc_TypeError, "cannot build parameter"); I don't have much experience with SWIG. Does it provide some means to instantiate a wrapped type from an address? If it does, then you can use a void pointer as the callback parameter. Hi Eryk, Thanks for looking at this. I had got as far as looking at the code above and guessed that only ctypes types were supported. This does seem like a bit of a functionality gap that cannot be solved easily by users of ctypes. It would appear to be fairly easy to define a protocol for objects that can be instantiated from ctypes arguments before passing to Python callables, i.e. some requirement for a class method factory that takes a ctypes type as a parameter and returns a Python object. This would seem to be a natural complement to from_param()/_as_parameter_ for the C/C++ to Python direction of function calls. I had started with an implementation that converted a c_void_p value and instantiated the required SWIG Python wrapper. This is not too difficult as SWIG allows its types to be extended in both the C/C++ world and the Python world. The result is only a one liner for each callback parameter but it is still ugly as the writer of the callback has to know what type the argument is and remember to call the "constructor" before accessing the attributes of the object. My requirement is to wrap an API for testers and customers to write simple test cases and any ugliness in the test cases is undesirable given that the users will probably not be Python experts and maybe not even developers. Regards Bill. From xue177125184 at gmail.com Thu Aug 4 13:40:45 2016 From: xue177125184 at gmail.com (Lei G) Date: Thu, 4 Aug 2016 10:40:45 -0700 (PDT) Subject: Tracing memory in production django process with systemtap Message-ID: <08fb6224-9e6a-4537-b3e5-2487b2cff03b@googlegroups.com> Hi all, Recently I met some python memory problems with Django and uwsgi, and found to find exactly which part is causing the problem, so I went to use systemtap to tracing the problem, and here are some problems that I met: It seems python needs to patch some 'probes' into the source code, but the patch for python version like 2.x ~ 3.x is not in the main python branch, If someone has used it, I want to know whether they are safe in the production? patch list: https://www.jcea.es/artic/python_dtrace.htm bug list: http://bugs.python.org/issue13405 From xue177125184 at gmail.com Thu Aug 4 13:46:23 2016 From: xue177125184 at gmail.com (Lei G) Date: Thu, 4 Aug 2016 10:46:23 -0700 (PDT) Subject: Tracing memory in production django process with systemtap In-Reply-To: <08fb6224-9e6a-4537-b3e5-2487b2cff03b@googlegroups.com> References: <08fb6224-9e6a-4537-b3e5-2487b2cff03b@googlegroups.com> Message-ID: <42778c9f-ef3f-4f65-9746-84b5cb0b4745@googlegroups.com> ? 2016?8?5???? UTC+8??1:41:04?Lei G??? > Hi all, > > Recently I met some python memory problems with Django and uwsgi, and found to find exactly which part is causing the problem, so I went to use systemtap to tracing the problem, and here are some problems that I met: > > It seems python needs to patch some 'probes' into the source code, but the patch for python version like 2.x ~ 3.x is not in the main python branch, > > If someone has used it, I want to know whether they are safe in the production? > > patch list: https://www.jcea.es/artic/python_dtrace.htm > bug list: http://bugs.python.org/issue13405 Or some advice to do this debugging on ubuntu + systemtap + python? From torriem at gmail.com Thu Aug 4 14:36:21 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 4 Aug 2016 12:36:21 -0600 Subject: Ide or code editor confusion In-Reply-To: <6688b08e-75cc-4132-b0e0-f63dcbfa4bd2@googlegroups.com> References: <2184a0ad-9d51-42db-ae11-5d0ab4512582@googlegroups.com> <7f142caf-5e3c-4fcf-818a-017b593fd0f7@googlegroups.com> <6688b08e-75cc-4132-b0e0-f63dcbfa4bd2@googlegroups.com> Message-ID: <713cc458-a469-eb3a-9f3f-81b1624cf245@gmail.com> On 08/04/2016 01:23 AM, Lawrence D?Oliveiro wrote: > On Friday, July 22, 2016 at 2:38:10 AM UTC+12, Chris Angelico wrote: >> On Fri, Jul 22, 2016 at 12:22 AM, sigmaphine1914 wrote: >>> Beginning Python: using Python 2.6 and Python 3.1. By James Payne >>> >>> Part II. >> >> Ugh, that's extremely old now. > > That?s why I?ve come to the conclusion it?s a waste of time buying books on computing topics. They start to reek of decay while they?re still on the shelf. Except for old Unix books! I've got an old book on sed and awk that will likely be in date for years to come! There's also an old book on vi, another on regular expression. So some things are invaluable references. Moving targets like Python, that's another story of course! From tjreedy at udel.edu Thu Aug 4 14:37:47 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 4 Aug 2016 14:37:47 -0400 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: Making repeat a keyword would have such an extremely high cost that it is out of the question and not a sane proposal. To start with, it is used in two major, widely used APIs. itertools.repeat + 50 uses in other itertools and tests + all the imports and and uses of repeat() in code all over the world. timeit command line option timeit.timeit(..., repeat=3) parameter name timeit.repeat() function + tests + all the scripts (some in other languages) using 'repeat' as a command line option and all the python code using the parameter or function. There are other C-coded public APIs that use repeat. One in xml.etree was revealed by the grep hit for the Python-coded test. Other places in the *Python-coded* stdlib that would break. distutils\fancy_getopt.py: 141: self.repeat = {} lib2to3\fixes\fix_operator.py: 8: operator.repeat(obj, n) ... lib2to3\patcomp.py: 105: repeat = None lib2to3\patcomp.py: 139: def compile_basic(self, nodes, repeat=None): lib2to3\tests\test_fixers.py: 4494: b = "operator.repeat(x, n)" test\test_faulthandler.py: 533: repeat = {repeat} test\test_faulthandler.py: 539: def func(timeout, repeat, ... test\test_xml_etree.py: 1784: for dumper, loader in product(self.modules, repeat=2): typing.py: 373: def repeat(x: T, n: int) -> Sequence[T]: -- Terry Jan Reedy From python at bdurham.com Thu Aug 4 14:47:06 2016 From: python at bdurham.com (Malcolm Greene) Date: Thu, 04 Aug 2016 14:47:06 -0400 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding Message-ID: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> I'm processing a lot of dirty CSV files and would like to track the bad codes that are raising UnicodeErrors. I'm struggling how to figure out what the exact codes are so I can track them, them remove them, and then repeat the decoding process for the current line until the line has been fully decoded so I can pass this line on to the CSV reader. At a high level it seems that I need to wrap the decoding of a line until it passes with out any errors. Any suggestions appreciated. Thank you, Malcolm From rosuav at gmail.com Thu Aug 4 14:48:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 04:48:12 +1000 Subject: Ide or code editor confusion In-Reply-To: <713cc458-a469-eb3a-9f3f-81b1624cf245@gmail.com> References: <2184a0ad-9d51-42db-ae11-5d0ab4512582@googlegroups.com> <7f142caf-5e3c-4fcf-818a-017b593fd0f7@googlegroups.com> <6688b08e-75cc-4132-b0e0-f63dcbfa4bd2@googlegroups.com> <713cc458-a469-eb3a-9f3f-81b1624cf245@gmail.com> Message-ID: On Fri, Aug 5, 2016 at 4:36 AM, Michael Torrie wrote: > On 08/04/2016 01:23 AM, Lawrence D?Oliveiro wrote: >> On Friday, July 22, 2016 at 2:38:10 AM UTC+12, Chris Angelico wrote: >>> On Fri, Jul 22, 2016 at 12:22 AM, sigmaphine1914 wrote: >>>> Beginning Python: using Python 2.6 and Python 3.1. By James Payne >>>> >>>> Part II. >>> >>> Ugh, that's extremely old now. >> >> That?s why I?ve come to the conclusion it?s a waste of time buying books on computing topics. They start to reek of decay while they?re still on the shelf. > > Except for old Unix books! I've got an old book on sed and awk that > will likely be in date for years to come! There's also an old book on > vi, another on regular expression. So some things are invaluable > references. Moving targets like Python, that's another story of course! Anything that focuses on specifics will get out of date, but the generalities don't. Here in the 2010s, I'm passing on to my students design advice (eg relational database schemas, laying out program code) that I learned from my father in the 1990s, and which he learned in the 1970s, and which probably have been passed down since the invention of programming, back in about 4004 BC. So there's often *something* you can learn from an old book - but I still wouldn't recommend a novice start out by picking up something that starts with the proper way to install hard drive platters in an IBM System/360, even if it does have a lot of sage wisdom in it :) ChrisA From rosuav at gmail.com Thu Aug 4 14:54:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 04:54:12 +1000 Subject: Debugging (was Re: Why not allow empty code blocks?) In-Reply-To: References: <579ccc68$0$1583$c3e8da3$5496439d@news.astraweb.com> <579d5e2d$0$1612$c3e8da3$5496439d@news.astraweb.com> <57a0def4$0$1603$c3e8da3$5496439d@news.astraweb.com> <57a18490$0$1601$c3e8da3$5496439d@news.astraweb.com> <57a2b532$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 5, 2016 at 4:37 AM, Terry Reedy wrote: > Making repeat a keyword would have such an extremely high cost > that it is out of the question and not a sane proposal. > To start with, it is used in two major, widely used APIs. > > itertools.repeat + 50 uses in other itertools and tests > timeit.repeat() function + tests Ooh, time to break out the thesaurus! rerun, rehash, reiterate [1], replicate, reproduce, recount, reprise, retread, relive, vamp_till_ready I'm sure we can come up with something! ChrisA [1] included mainly to troll the people who want some definition of "reiterable" From rosuav at gmail.com Thu Aug 4 15:00:31 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 05:00:31 +1000 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> Message-ID: On Fri, Aug 5, 2016 at 4:47 AM, Malcolm Greene wrote: > I'm processing a lot of dirty CSV files and would like to track the bad > codes that are raising UnicodeErrors. I'm struggling how to figure out > what the exact codes are so I can track them, them remove them, and then > repeat the decoding process for the current line until the line has been > fully decoded so I can pass this line on to the CSV reader. At a high > level it seems that I need to wrap the decoding of a line until it > passes with out any errors. Any suggestions appreciated. Remove them? Not sure what you mean, exactly; but would an errors="backslashreplace" decode do the job? Something like (assuming you use Python 3): def read_dirty_file(fn): with open(fn, encoding="utf-8", errors="backslashreplace") as f: for row in csv.DictReader(f): process(row) You'll get Unicode text, but any bytes that don't make sense in UTF-8 will be represented as eg \x80, with an actual backslash. Or use errors="replace" to hide them all behind U+FFFD, or other forms of error handling. That'll get done at a higher level than the CSV reader, like you suggest. ChrisA From python at bdurham.com Thu Aug 4 15:22:48 2016 From: python at bdurham.com (Malcolm Greene) Date: Thu, 04 Aug 2016 15:22:48 -0400 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> Message-ID: <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Hi Chris, Thanks for your suggestions. I would like to capture the specific bad codes *before* they get replaced. So if a line of text has 10 bad codes (each one raising UnicodeError), I would like to track each exception's bad code but still return a valid decode line when finished. My goal is to count the total number of UnicodeExceptions within a file (as a data quality metric) and track the frequency of specific bad code's (via a collections.counter dict) to see if there's a pattern that can be traced to bad upstream process. Malcolm Remove them? Not sure what you mean, exactly; but would an errors="backslashreplace" decode do the job? Something like (assuming you use Python 3): def read_dirty_file(fn): with open(fn, encoding="utf-8", errors="backslashreplace") as f: for row in csv.DictReader(f): process(row) You'll get Unicode text, but any bytes that don't make sense in UTF-8 will be represented as eg \x80, with an actual backslash. Or use errors="replace" to hide them all behind U+FFFD, or other forms of error handling. That'll get done at a higher level than the CSV reader, like you suggest. From python at mrabarnett.plus.com Thu Aug 4 15:33:12 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 4 Aug 2016 20:33:12 +0100 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Message-ID: On 2016-08-04 20:22, Malcolm Greene wrote: > Hi Chris, > > Thanks for your suggestions. I would like to capture the specific bad > codes *before* they get replaced. So if a line of text has 10 bad codes > (each one raising UnicodeError), I would like to track each exception's > bad code but still return a valid decode line when finished. > > My goal is to count the total number of UnicodeExceptions within a file > (as a data quality metric) and track the frequency of specific bad > code's (via a collections.counter dict) to see if there's a pattern that > can be traced to bad upstream process. > You could catch the UnicodeDecodeError exception and look at its attributes: try: b'\x80'.decode('utf-8') except UnicodeDecodeError as e: print('Failed to decode') print('e.start is', e.start) print('e.end is', e.end) else: print('Decoded successfully') It prints: Failed to decode e.start is 0 e.end is 1 > Malcolm > > > Remove them? Not sure what you mean, exactly; but would an > errors="backslashreplace" decode do the job? Something like (assuming > you use Python 3): > > def read_dirty_file(fn): > with open(fn, encoding="utf-8", errors="backslashreplace") as f: > for row in csv.DictReader(f): > process(row) > > You'll get Unicode text, but any bytes that don't make sense in UTF-8 > will be represented as eg \x80, with an actual backslash. Or use > errors="replace" to hide them all behind U+FFFD, or other forms of > error handling. That'll get done at a higher level than the CSV > reader, like you suggest. > > From tjreedy at udel.edu Thu Aug 4 15:36:38 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 4 Aug 2016 15:36:38 -0400 Subject: Python Error message In-Reply-To: <23e36680-195e-b840-599e-e225cdc2cee8@mrabarnett.plus.com> References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> <23e36680-195e-b840-599e-e225cdc2cee8@mrabarnett.plus.com> Message-ID: On 8/4/2016 12:19 PM, MRAB wrote: > In those rare occasions when you do write a bare except, A bare "except:" is never needed and in my opinion, and that of others, one should never write one (except possibly for experimentation). Be explicit and write "except BaseException:" or "except Exception:", whichever one is the actual intent. > you'd re-raise the exception afterwards: As a general rule, this is wrong, just as this rule is wrong for other exception blocks. > try: > ... > except: > print("'tis but a scratch!") > raise This is right when one wants to do something *in addition to* the normal handling, such as log errors to a file, but is wrong when wants to do something *instead of* allowing the normal handling. An example of the latter is when one writes code in Python to execute 'other' code. (IDLE is one example. It both executes user statements and evals user expressions.) One needs "except BaseException:" to isolate the interpreter from exceptions raised in the interpreted code. (It would be wrong for IDLE to stop because a user submitted code that raises, whether intentionally or accidentally) A 'raise' that throws the exception into the interpreter is likely the worst thing to do. -- Terry Jan Reedy From random832 at fastmail.com Thu Aug 4 15:45:24 2016 From: random832 at fastmail.com (Random832) Date: Thu, 04 Aug 2016 15:45:24 -0400 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Message-ID: <1470339924.3065250.686286553.51C83AD9@webmail.messagingengine.com> On Thu, Aug 4, 2016, at 15:22, Malcolm Greene wrote: > Hi Chris, > > Thanks for your suggestions. I would like to capture the specific bad > codes *before* they get replaced. So if a line of text has 10 bad codes > (each one raising UnicodeError), I would like to track each exception's > bad code but still return a valid decode line when finished. Look into writing your own error handler - there's enough information provided to do this. https://docs.python.org/3/library/codecs.html From rosuav at gmail.com Thu Aug 4 15:53:29 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 05:53:29 +1000 Subject: Python Error message In-Reply-To: References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> <57a36593$0$1590$c3e8da3$5496439d@news.astraweb.com> <23e36680-195e-b840-599e-e225cdc2cee8@mrabarnett.plus.com> Message-ID: On Fri, Aug 5, 2016 at 5:36 AM, Terry Reedy wrote: > An example of the latter is when one writes code in Python to execute > 'other' code. (IDLE is one example. It both executes user statements and > evals user expressions.) One needs "except BaseException:" to isolate the > interpreter from exceptions raised in the interpreted code. (It would be > wrong for IDLE to stop because a user submitted code that raises, whether > intentionally or accidentally) A 'raise' that throws the exception into the > interpreter is likely the worst thing to do. This is a classic example of a "boundary location". Another extremely common example is a web server: if an exception bubbles out of a request handler function, the outer wrapper code should catch that, log it, and send a 500 back to the client. But none of this is what the OP is doing. ChrisA From rosuav at gmail.com Thu Aug 4 16:00:33 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 5 Aug 2016 06:00:33 +1000 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Message-ID: On Fri, Aug 5, 2016 at 5:22 AM, Malcolm Greene wrote: > Thanks for your suggestions. I would like to capture the specific bad > codes *before* they get replaced. So if a line of text has 10 bad codes > (each one raising UnicodeError), I would like to track each exception's > bad code but still return a valid decode line when finished. > Interesting. Sounds to me like the simplest option is to open the file as binary, split it on b"\n", and decode line by line before giving it to the csv module. The csv.reader "csvfile" argument doesn't actually have to be a file - it can be anything that yields lines. So you can put a generator in between, like this: def decode(binary): for line in binary: try: yield line.decode("utf-8") except UnicodeDecodeError: log_stats() def read_dirty_file(fn): with open(fn, "rb") as f: for row in csv.DictReader(decode(f)): process(row) Or what Random said, which is also viable. ChrisA From michael.selik at gmail.com Thu Aug 4 16:11:05 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 04 Aug 2016 20:11:05 +0000 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Message-ID: On Thu, Aug 4, 2016 at 3:24 PM Malcolm Greene wrote: > Hi Chris, > > Thanks for your suggestions. I would like to capture the specific bad > codes *before* they get replaced. So if a line of text has 10 bad codes > (each one raising UnicodeError), I would like to track each exception's > bad code but still return a valid decode line when finished. > > My goal is to count the total number of UnicodeExceptions within a file > (as a data quality metric) and track the frequency of specific bad > code's (via a collections.counter dict) to see if there's a pattern that > can be traced to bad upstream process. > Give this a shot (below). It seems to do what you want. import csv from collections import Counter from io import BytesIO def _cleanline(line, counts=Counter()): try: return line.decode() except UnicodeDecodeError as e: counts[line[e.start:e.end]] += 1 return line[:e.start].decode() + _cleanline(line[e.end:], counts) def cleanlines(fp): ''' convert data to text; track decoding errors ``fp`` is an open file-like iterable of lines' ''' cleanlines.errors = Counter() for line in fp: yield _cleanline(line, cleanlines.errors) f = BytesIO(b'''\ this,is line,one line two,has junk,\xffin it so does,\xfa\xffline,three ''') for row in csv.reader(cleanlines(f)): print(row) print(cleanlines.errors.most_common()) From matt.ruffalo at gmail.com Thu Aug 4 16:11:26 2016 From: matt.ruffalo at gmail.com (Matt Ruffalo) Date: Thu, 4 Aug 2016 16:11:26 -0400 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: <1470339924.3065250.686286553.51C83AD9@webmail.messagingengine.com> References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> <1470339924.3065250.686286553.51C83AD9@webmail.messagingengine.com> Message-ID: <5adb7dfe-65ae-9ed2-1d4b-b7f05551a22f@gmail.com> On 2016-08-04 15:45, Random832 wrote: > On Thu, Aug 4, 2016, at 15:22, Malcolm Greene wrote: >> Hi Chris, >> >> Thanks for your suggestions. I would like to capture the specific bad >> codes *before* they get replaced. So if a line of text has 10 bad codes >> (each one raising UnicodeError), I would like to track each exception's >> bad code but still return a valid decode line when finished. > Look into writing your own error handler - there's enough information > provided to do this. > > https://docs.python.org/3/library/codecs.html You could also use the 'surrogateescape' error handler, and count the number of high surrogate characters (each of which represents a byte that couldn't be decoded under the specified encoding). This will give you a text string, which you can then process to replace code points in the range U+DC80 - U+DCFF (inclusive). """ In [1]: bad_byte_string = b'Some \xf9 odd \x84 bytes \xc2 here' In [2]: decoded = bad_byte_string.decode(errors='surrogateescape') In [3]: decoded Out[3]: 'Some \udcf9 odd \udc84 bytes \udcc2 here' In [4]: high_surrogate_range = range(0xdc80, 0xdd00) In [5]: sum(ord(char) in high_surrogate_range for char in decoded) Out[5]: 3 In [6]: from collections import Counter In [7]: from typing import Iterable In [8]: def get_bad_bytes(string: str) -> Iterable[bytes]: ...: for char in string: ...: if ord(char) in high_surrogate_range: ...: yield char.encode(errors='surrogateescape') ...: In [9]: bad_byte_counts = Counter(get_bad_bytes(decoded)) In [10]: bad_byte_counts Out[10]: Counter({b'\x84': 1, b'\xc2': 1, b'\xf9': 1}) """ MMR... From python at bdurham.com Thu Aug 4 17:03:06 2016 From: python at bdurham.com (Malcolm Greene) Date: Thu, 04 Aug 2016 17:03:06 -0400 Subject: Capturing the bad codes that raise UnicodeError exceptions during decoding In-Reply-To: References: <1470336426.893449.686225577.24458777@webmail.messagingengine.com> <1470338568.900965.686261921.548DDA14@webmail.messagingengine.com> Message-ID: <1470344586.921146.686351745.40B5EE81@webmail.messagingengine.com> Wow!!! A huge thank you to all who replied to this thread! Chris: You gave me some ideas I will apply in the future. MRAB: Thanks for exposing me to the extended attributes of the UnicodeError object (e.start, e.end, e.object). Mike: Cool example! I like how _cleanlines() recursively calls itself to keep cleaning up a line after an error is handled. Your code solved the mystery of how to recover from a UnicodeError and keep decoding. Random832: Your suggestion to write a custom codecs handler was great. Sample below for future readers reviewing this thread. # simple codecs custom error handler import codecs def custom_unicode_error_handler(e): bad_bytes = e.object[e.start:e.end] print( 'Bad bytes: ' + bad_bytes.hex()) return ('', e.end) codecs.register_error('custom_unicode_error_handler', custom_unicode_error_handler) Malcolm From jj0gen0info at gmail.com Thu Aug 4 19:50:16 2016 From: jj0gen0info at gmail.com (jj0gen0info at gmail.com) Date: Thu, 4 Aug 2016 16:50:16 -0700 (PDT) Subject: Win32 API in pywin32 Message-ID: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> According to Python.org Mark Hammond has an Add-on (pywin32) that supports Win32 and COM. Does anyone know if the Add-on covers all Win32 API functions, and if not is there a list of the subset of Win32 API functions it does include? J. From lawrencedo99 at gmail.com Thu Aug 4 19:57:04 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 16:57:04 -0700 (PDT) Subject: can't add variables to instances of built-in classes In-Reply-To: References: <4f796d95-0e87-4610-a0a8-1eff07c60ace@googlegroups.com> <578b9e62$0$22141$c3e8da3$5496439d@news.astraweb.com> <3e934f83-2076-407d-b85b-686ecdec6ff8@googlegroups.com> <2bc29306-ac90-4424-84c2-64be494c5a75@googlegroups.com> <230d1aff-b1a3-4f16-b9a6-b5d9fd07e4b4@googlegroups.com> <62dd86d4-74f7-4e97-a29c-7e45aced77d9@googlegroups.com> <18acf908-b761-4a60-8d7c-5e4576b03320@googlegroups.com> Message-ID: On Thursday, August 4, 2016 at 3:07:17 PM UTC+12, Chris Angelico wrote: > On Thu, Aug 4, 2016 at 12:34 PM, Lawrence D?Oliveiro wrote: >> On Thursday, July 21, 2016 at 2:29:53 PM UTC+12, Chris Angelico wrote: >> >>> Using __slots__ basically takes your object down to the level of a >>> Java one. >> >> Is Java like a bogeyman you use to scare people with if they don?t write >> Python code the way you like it? > > LOL! Not really; it's just that Java's objects are way less featured > than Python's. Using Java as an example of how it can keep you out of trouble where Python won?t is fine; using it as a stick to commit personal attacks on people is not. From lawrencedo99 at gmail.com Thu Aug 4 19:57:53 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 16:57:53 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com wrote: > According to Python.org Mark Hammond has an Add-on (pywin32) that supports > Win32 and COM. Are people still using Win32? I thought Windows went 64-bit years ago. From lawrencedo99 at gmail.com Thu Aug 4 20:03:20 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 17:03:20 -0700 (PDT) Subject: make an object read only In-Reply-To: References: Message-ID: On Wednesday, August 3, 2016 at 3:13:01 AM UTC+12, Robin Becker wrote: > A reportlab user found he was doing the wrong thing by calling > canvas.save repeatedly, our documentation says you should not use Canvas > objects after the save method has been used. The user had mixed results :( As GvR has said: ?we?re all consenting adults here?. In other words, we?re capable of coping with the consequences of our actions. From ikorot01 at gmail.com Thu Aug 4 20:06:07 2016 From: ikorot01 at gmail.com (Igor Korot) Date: Thu, 4 Aug 2016 17:06:07 -0700 Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: Hi, On Thu, Aug 4, 2016 at 4:57 PM, Lawrence D?Oliveiro wrote: > On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com wrote: >> According to Python.org Mark Hammond has an Add-on (pywin32) that supports >> Win32 and COM. > > Are people still using Win32? I thought Windows went 64-bit years ago. This is the question about Win API calls.... Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list From redstone-cold at 163.com Thu Aug 4 20:25:44 2016 From: redstone-cold at 163.com (iMath) Date: Thu, 4 Aug 2016 17:25:44 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: <43e351ef-2a1f-4197-92ad-8822b2576cdb@googlegroups.com> On Friday, August 5, 2016 at 7:50:28 AM UTC+8, jj0ge... at gmail.com wrote: > According to Python.org Mark Hammond has an Add-on (pywin32) that supports Win32 and COM. > > Does anyone know if the Add-on covers all Win32 API functions, and if not is there a list of the subset of Win32 API functions it does include? > > J. According to my experience , pywin32 doesn't covers all Win32 API functions but most of them , so for these not supported functions, you could resort to ctypes ,and there seems no list of the subset of Win32 API functions it does include. You can contact with the developers of pywin32 using the mailing list python-win32 at python.org From redstone-cold at 163.com Thu Aug 4 20:27:23 2016 From: redstone-cold at 163.com (iMath) Date: Thu, 4 Aug 2016 17:27:23 -0700 (PDT) Subject: get a certain request url during Loading a web page Message-ID: <413ecd14-079d-4d00-9754-fe4157759f31@googlegroups.com> During Loading this web page(http://www.iqiyi.com/v_19rrkkri8k.html) , the browser makes many requests,now I need a certain request url (e.g.starts with 'http://cache.video.qiyi.com/vms?') during the Loading process ,and the request is made on condition that the adobe flash player plugin for NPAPI is enabled , so any way to get the url via python ? P.S. Better to show some code, I am new to this area. From eryksun at gmail.com Thu Aug 4 20:39:14 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 5 Aug 2016 00:39:14 +0000 Subject: Win32 API in pywin32 In-Reply-To: <43e351ef-2a1f-4197-92ad-8822b2576cdb@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <43e351ef-2a1f-4197-92ad-8822b2576cdb@googlegroups.com> Message-ID: On Fri, Aug 5, 2016 at 12:25 AM, iMath wrote: > On Friday, August 5, 2016 at 7:50:28 AM UTC+8, jj0ge... at gmail.com wrote: >> According to Python.org Mark Hammond has an Add-on (pywin32) that supports Win32 and COM. >> >> Does anyone know if the Add-on covers all Win32 API functions, and if not is there a list of the subset of Win32 API functions it does include? >> >> J. > > According to my experience , pywin32 doesn't covers all Win32 API functions but most of them , so for these not supported functions, you could resort to ctypes > ,and there seems no list of the subset of Win32 API functions it does include. > > You can contact with the developers of pywin32 using the mailing list python-win32 at python.org Currently I just search with a filter of "site:activestate.com" because they have the PyWin32 docs online. From steve+python at pearwood.info Thu Aug 4 21:21:00 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 05 Aug 2016 11:21:00 +1000 Subject: datetime vs Arrow vs Pendulum vs Delorean vs udatetime References: Message-ID: <57a3e9fd$0$22141$c3e8da3$5496439d@news.astraweb.com> On Fri, 5 Aug 2016 08:41 am, breamoreboy at gmail.com wrote: > I hereby ask that only people who know and use Python reply, not the > theoretical idiots who could not fight their way out of a wet paper bag. So it's only people who are theoretically idiots that are prohibited from replying? Actual, proven idiots are permitted to reply. Awesome! Looking forward to the posts from actual idiots, and people who can fight their way out of wet paper bags. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Thu Aug 4 21:23:48 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 18:23:48 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> On Friday, August 5, 2016 at 12:06:23 PM UTC+12, Igor Korot wrote: > > On Thu, Aug 4, 2016 at 4:57 PM, Lawrence D?Oliveiro wrote: >> On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com wrote: >>> According to Python.org Mark Hammond has an Add-on (pywin32) that >>> supports Win32 and COM. >> >> Are people still using Win32? I thought Windows went 64-bit years ago. > > This is the question about Win API calls.... > > Thank you. This is a Python group. You?re welcome. From lawrencedo99 at gmail.com Thu Aug 4 21:25:52 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 4 Aug 2016 18:25:52 -0700 (PDT) Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: References: Message-ID: <2eab4d10-f648-46bb-97b9-2bd699b69561@googlegroups.com> On Tuesday, August 2, 2016 at 8:21:57 PM UTC+12, Valeria Munoz wrote: > > I have downloaded Python 3.6.0a3 on a Mac 10.9.5 and realized that I also > need to download an Active Tcl for it. Python should already come with Tk . Tcl is an entirely separate language from Python. While it is where Tk originated, you don?t need it to use Tk with Python. From steve+python at pearwood.info Thu Aug 4 22:22:28 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 05 Aug 2016 12:22:28 +1000 Subject: Is Activestate's Python recipes broken? Message-ID: <57a3f864$0$1608$c3e8da3$5496439d@news.astraweb.com> Log in to Activestate: https://code.activestate.com/recipes/langs/python/new/ and click "Add a Recipe". I get Forbidden You don't have permission to access /recipes/add/ on this server. Apache Server at code.activestate.com Port 443 Broken for everyone, or just for me? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From zachary.ware+pylist at gmail.com Thu Aug 4 23:06:48 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Thu, 4 Aug 2016 22:06:48 -0500 Subject: Tcl/Tk for Python 3.6.0a3 on Os X 10.9.5 In-Reply-To: <2eab4d10-f648-46bb-97b9-2bd699b69561@googlegroups.com> References: <2eab4d10-f648-46bb-97b9-2bd699b69561@googlegroups.com> Message-ID: On Thu, Aug 4, 2016 at 8:25 PM, Lawrence D?Oliveiro wrote: > On Tuesday, August 2, 2016 at 8:21:57 PM UTC+12, Valeria Munoz wrote: >> >> I have downloaded Python 3.6.0a3 on a Mac 10.9.5 and realized that I also >> need to download an Active Tcl for it. > > Python should already come with Tk . Tcl is an entirely separate language from Python. While it is where Tk originated, you don?t need it to use Tk with Python. This is very inaccurate. From the first line of the second paragraph at your link, "The tkinter package is a thin object-oriented layer on top of Tcl/Tk." Tk is very closely tied to Tcl (in fact a significant fraction of Tk is implemented in Tcl), and a Tcl interpreter (of the exact same version) is required to use Tk. Python's tkinter package dynamically links to libtcl and libtk to embed a Tcl interpreter, and all GUI interactions are handled by Tcl calling Python callbacks. -- Zach From jsf80238 at gmail.com Thu Aug 4 23:12:11 2016 From: jsf80238 at gmail.com (Jason Friedman) Date: Thu, 4 Aug 2016 21:12:11 -0600 Subject: Is Activestate's Python recipes broken? In-Reply-To: <57a3f864$0$1608$c3e8da3$5496439d@news.astraweb.com> References: <57a3f864$0$1608$c3e8da3$5496439d@news.astraweb.com> Message-ID: > > Log in to Activestate: > > https://code.activestate.com/recipes/langs/python/new/ > > and click "Add a Recipe". I get > > > Forbidden > > You don't have permission to access /recipes/add/ on this server. > Apache Server at code.activestate.com Port 443 > > > > Broken for everyone, or just for me? > Broken for me, too. From jj0gen0info at gmail.com Thu Aug 4 23:18:46 2016 From: jj0gen0info at gmail.com (jj0gen0info at gmail.com) Date: Thu, 4 Aug 2016 20:18:46 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: <43e351ef-2a1f-4197-92ad-8822b2576cdb@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <43e351ef-2a1f-4197-92ad-8822b2576cdb@googlegroups.com> Message-ID: Thanks for the info iMath, I will try to contact the developers as you've suggested. From bostone at gmail.com Thu Aug 4 23:54:23 2016 From: bostone at gmail.com (Bo Stone) Date: Thu, 4 Aug 2016 20:54:23 -0700 (PDT) Subject: Installing Python 2.7 for all users on SLES 11 Message-ID: <2c1a4835-cc83-48b2-b197-0a102284485c@googlegroups.com> I installed Python 2.7 on SLES 11 box that previously was running Python 2.6. To do so I used a script described in this post (http://stackoverflow.com/a/11371726/135946) and run it as a root user. Everything went well but when it was done I discovered few issues: 1. No symbolic links were created and no path updated so I had to manually update the path to link to the new installation bin directory /opt/python2.7/bin 2. Everything runs good until I switch from root to the normal user at which point Python shell runs but some modules I installed such as PyYAML are missing. Again, these are OK when I run Python as root 3. As a regular user I'm not able to run pip, easy_install and wheel. For pip I get "ImportError: No module named pkg_resources" This question is also posted to Stackoverflow here: http://stackoverflow.com/q/38749809/135946 From dieter at handshake.de Fri Aug 5 03:03:07 2016 From: dieter at handshake.de (dieter) Date: Fri, 05 Aug 2016 09:03:07 +0200 Subject: Python Error message References: <710599b6-b8c3-4f5a-a3dd-48757b816a44@googlegroups.com> Message-ID: <87vazfn1ec.fsf@handshake.de> GBANE FETIGUE writes: > ... > I am running a python script to run some CURL commands, and return the response which is the applicationId and the versionId. I was able to do it. Now the versionId value supposed to be used on the second CURL as a value of the applications key which is an array. but it doesn't work.I 'll post the error after running the command as well as the script. It seems like I have an error somewhere because the curl works manually if i run. > > ec2-user at ip-172-31-21-77 playbooks]$ python mmc-uploader.py > ... > Final response...

HTTP Status 415 -

type Status report

message

description The server refused this request because the request entity is in a format not supported by the reque > sted resource for the requested method.

Apache Tomcat/8.0.26

What you see here is *not* a Python error message but an error message from the server you have contacted. It tells you: "The server refused this request because the request entity is in a format not supported by the requested resource for the requested method." This means, that in constructing your request you made some error - regarding the "format" (which may mean the "Content-Type") for the request "entity" (which may mean the request body). I would check the service description to find out about what "format" is expected and what the term "entity" means. Then I would check the "curl" documentation to find out how to meet those expectations. From robin at reportlab.com Fri Aug 5 04:51:36 2016 From: robin at reportlab.com (Robin Becker) Date: Fri, 5 Aug 2016 09:51:36 +0100 Subject: make an object read only In-Reply-To: References: Message-ID: <3439364d-2d34-1d3d-a356-d47d157686f9@chamonix.reportlab.co.uk> On 05/08/2016 01:03, Lawrence D?Oliveiro wrote: ........ >> objects after the save method has been used. The user had mixed results :( > > As GvR has said: ?we?re all consenting adults here?. > > In other words, we?re capable of coping with the consequences of our actions. > agreed :) -- Robin Becker From antoon.pardon at rece.vub.ac.be Fri Aug 5 05:04:18 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Fri, 5 Aug 2016 11:04:18 +0200 Subject: make an object read only In-Reply-To: References: Message-ID: <57A45692.4000709@rece.vub.ac.be> Op 05-08-16 om 02:03 schreef Lawrence D?Oliveiro: > On Wednesday, August 3, 2016 at 3:13:01 AM UTC+12, Robin Becker wrote: >> A reportlab user found he was doing the wrong thing by calling >> canvas.save repeatedly, our documentation says you should not use Canvas >> objects after the save method has been used. The user had mixed results :( > As GvR has said: ?we?re all consenting adults here?. No we are not. GvR only uses this when his desired level of protection is lower than others. If the other's desired level of protection is lower than GvR's the all consenting adults goes down the drain. > In other words, we?re capable of coping with the consequences of our actions. Which doesn't support anything. Python has features that makes it in some situations easier to cope with those consequences than when that feature wouldn't be present. So there is nothing wrong in argueing for a feature that would do the same. -- Antoon. From bc at freeuk.com Fri Aug 5 05:14:24 2016 From: bc at freeuk.com (BartC) Date: Fri, 5 Aug 2016 10:14:24 +0100 Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: On 05/08/2016 00:57, Lawrence D?Oliveiro wrote: > On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com wrote: >> According to Python.org Mark Hammond has an Add-on (pywin32) that supports >> Win32 and COM. > > Are people still using Win32? I thought Windows went 64-bit years ago. > Win32 refers to both 32-bit and 64-bit APIs. Apart from which, Win64 will still run 32-bit applications. -- Bartc From m at funkyhat.org Fri Aug 5 07:58:53 2016 From: m at funkyhat.org (Matt Wheeler) Date: Fri, 5 Aug 2016 12:58:53 +0100 Subject: Win32 API in pywin32 In-Reply-To: <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: On Fri, 5 Aug 2016, 02:23 Lawrence D?Oliveiro, wrote: > On Friday, August 5, 2016 at 12:06:23 PM UTC+12, Igor Korot wrote: > > > > On Thu, Aug 4, 2016 at 4:57 PM, Lawrence D?Oliveiro wrote: > >> On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com > wrote: > >>> According to Python.org Mark Hammond has an Add-on (pywin32) that > >>> supports Win32 and COM. > >> > >> Are people still using Win32? I thought Windows went 64-bit years ago. > The API is still called Win32 (fun fact: on 64 bit windows C:\Windows\System32 actually contains the 64 bit system (binaries & libraries). The 32 bit equivalents live in a dir called SysWOW64 (stands for Windows on Win64 or something) and get mapped to their regular locations for any 32 bit processes. That's right, 64 bit libraries live in System32 and 32 bits in SysWOW64 :) )* > This is the question about Win API calls.... > > > > Thank you. > > This is a Python group. > And the question was about using the Windows API in Python. Not sure what point you're trying to make here. > You?re welcome. > ... * anyone from Microsoft listening and want to offer an explanation (don't worry, I don't think anyone expects it to be good :D)? From rafinha.unix at gmail.com Fri Aug 5 10:07:32 2016 From: rafinha.unix at gmail.com (Rafael Francischini) Date: Fri, 5 Aug 2016 07:07:32 -0700 (PDT) Subject: Pompem Exploit and Vulnerability Finder Tool v0.2.0 Python 3.5 Message-ID: Hello, I would like to share with you my personal project, Pompem. https://github.com/rfunix/Pompem/ I hope you enjoy it. From eryksun at gmail.com Fri Aug 5 16:04:34 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 5 Aug 2016 20:04:34 +0000 Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: On Fri, Aug 5, 2016 at 11:58 AM, Matt Wheeler wrote: > On Fri, 5 Aug 2016, 02:23 Lawrence D?Oliveiro, > wrote: > >> On Friday, August 5, 2016 at 12:06:23 PM UTC+12, Igor Korot wrote: >> > >> > On Thu, Aug 4, 2016 at 4:57 PM, Lawrence D?Oliveiro wrote: >> >> On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com >> wrote: >> >>> According to Python.org Mark Hammond has an Add-on (pywin32) that >> >>> supports Win32 and COM. >> >> >> >> Are people still using Win32? I thought Windows went 64-bit years ago. >> > > The API is still called Win32 (fun fact: on 64 bit windows > C:\Windows\System32 actually contains the 64 bit system (binaries & > libraries). The 32 bit equivalents live in a dir called SysWOW64 (stands > for Windows on Win64 or something) and get mapped to their regular > locations for any 32 bit processes. That's right, 64 bit libraries live in > System32 and 32 bits in SysWOW64 :) )* Officially the name is the "Windows API" (or WinAPI; hence Python's _winapi module). It was "formerly called the Win32 API" [1]. That said, referring to it as Win32 is so pervasive in Microsoft's own documentation, among developers, and in the API itself that the old name isn't going anywhere. The system directory and many DLLs retain the "32" suffix in the 64-bit system in order to ease porting 32-bit programs that used hard-coded paths. I don't know all of the compatibility problems and constraints (and pressure from important customers) that they faced, so I won't pointlessly criticize this decision that was made over 15 years ago. As to the name WOW64 for the 32-bit subsystem, the Microsoft devs (or marketing department?) who named it apparently don't think like most people, as evidenced by the many complaints about the name. They should have realized that WOW64 suggests it's a 64-bit system, not "Windows [32-bit] on Windows 64-bit". [1]: https://msdn.microsoft.com/en-us/library/hh920508 From mail.python.org at marco.sulla.e4ward.com Fri Aug 5 18:00:56 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 00:00:56 +0200 Subject: Python slang Message-ID: I have a simple curiosity: why Python has much keywords, and some builtin types and methods, that are different from the other languages? What is the rationale? I'm referring to: * `except` instead of `catch` * `raise` instead of `throw` * `self` instead of `this` (I know, it's not enforced, but it's a de facto standard and some IDEs like PyDev gives you an error if you declare a non-static method without `self` as first parameter) * `dict` instead of `map` * `list.append()` instead of `list.push()` * `str.strip()` instead of `str.trim()` * `True`, `False` and None instead of `true`, `false` and `none` (they seems classes) * and furthermore much abbreviation, like `str` instead of `string` and `len()` instead of `length()`, that seems to contrast with the readability goal of Python. I don't ask about `None` instead of `null` because I suppose here it's a matter of disambiguation (null, in many languages, is not equal to null). From rosuav at gmail.com Fri Aug 5 18:31:47 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 08:31:47 +1000 Subject: Python slang In-Reply-To: References: Message-ID: On Sat, Aug 6, 2016 at 8:00 AM, Marco Sulla via Python-list wrote: > I have a simple curiosity: why Python has much keywords, and some > builtin types and methods, that are different from the other > languages? What is the rationale? This isn't slang; it's jargon, perhaps, but "slang" implies an unofficial form of use. "Python slang" might include the myriad references to "spam" throughout the code and docs, and other Monty Pythonisms. But that aside, these are great questions to ask. As some people put it: "Why is it so?". > I'm referring to: > * `except` instead of `catch` Not sure. Python does seem to be roughly unique in this. > * `raise` instead of `throw` Quite a few other languages talk about raising exceptions rather than throwing them. Those would be the two most common terms, I think. > * `self` instead of `this` (I know, it's not enforced, but it's a de > facto standard and some IDEs like PyDev gives you an error if you > declare a non-static method without `self` as first parameter) Smalltalk and friends also use "self", so again, this would be the other common word for the parameter. > * `dict` instead of `map` "map" has many other meanings (most notably the action wherein you call a function on every member of a collection to produce another collection). > * `list.append()` instead of `list.push()` Better description of what it does, IMO. It's adding something to the end of the list - appending to the list. > * `str.strip()` instead of `str.trim()` Another case where both words are in common use. > * `True`, `False` and None instead of `true`, `false` and `none` (they > seems classes) This one, I've no idea about. Why have "bool" as the type, and "True" and "False" as the instances? I think the built-in types have their names grandfathered in from when they were factory functions, but that doesn't explain the capitalized instances. (And the type of None is NoneType, just to confuse the matter further.) > * and furthermore much abbreviation, like `str` instead of `string` > and `len()` instead of `length()`, that seems to contrast with the > readability goal of Python. Brevity can improve clarity, if it is obtained without the expense of accuracy. In the same vein, Python has one-character symbols instead of "add" and "divide". There's a balance to be struck; bitwise Or has a symbol, but short-circuiting boolean Or has a word. > I don't ask about `None` instead of `null` because I suppose here it's > a matter of disambiguation (null, in many languages, is not equal to > null). None isn't really like null, but it sometimes does the job that null does in other languages. It isn't a "non-object"; it's an object, same as any other, that happens to be a falsey singleton. Given that C's NULL pointer isn't the same as SQL's NULL nor ASCII's NUL, having a Python null that's different again wouldn't help. So yes, it's disambiguation; this is not the same thing as null. Hope that helps a bit! ChrisA From mail.python.org at marco.sulla.e4ward.com Fri Aug 5 19:21:51 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 01:21:51 +0200 Subject: Python slang In-Reply-To: References: Message-ID: On 6 August 2016 at 00:31, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 8:00 AM, Marco Sulla via Python-list > wrote: > This isn't slang; it's jargon Right. >> * `raise` instead of `throw` > > Quite a few other languages talk about raising exceptions rather than > throwing them. Those would be the two most common terms, I think. > >> * `self` instead of `this` (I know, it's not enforced, but it's a de >> facto standard and some IDEs like PyDev gives you an error if you >> declare a non-static method without `self` as first parameter) > > Smalltalk and friends also use "self", so again, this would be the > other common word for the parameter. > >> * `str.strip()` instead of `str.trim()` > > Another case where both words are in common use. I want to clarify that when I say "different from the other languages", I mean "different from the most used languages", that in my mind are C/C++, C#, Java, PHP and Javascript, mainly. >> * `dict` instead of `map` > > "map" has many other meanings (most notably the action wherein you > call a function on every member of a collection to produce another > collection). Well, the word `map` of the `map` function is a verb. IMO there would be no ambiguity. >> * `list.append()` instead of `list.push()` > > Better description of what it does, IMO. It's adding something to the > end of the list - appending to the list. I agree. >> * and furthermore much abbreviation, like `str` instead of `string` >> and `len()` instead of `length()`, that seems to contrast with the >> readability goal of Python. > > Brevity can improve clarity, if it is obtained without the expense of > accuracy. I agree it's not hard to understand that `str` is the string type and `len()` is the function that gives you the length, even if you don't know Python (and it's shorter to type...) But it's hard to remember, in particular if you code also in other languages. When I come back to Python, I always ends to write `somelist.length` instead of `len(somelist)`, some arcane words come out my mouth and a little fairy dies. > Hope that helps a bit! yep. From breamoreboy at gmail.com Fri Aug 5 20:07:50 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 5 Aug 2016 17:07:50 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native Message-ID: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> A couple or three years old but this is well worth seeing for anybody, regardless of your Python expertise. http://nedbatchelder.com/text/iter.html Kindest regards. Mark Lawrence. From rosuav at gmail.com Fri Aug 5 20:13:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 10:13:58 +1000 Subject: Python slang In-Reply-To: References: Message-ID: On Sat, Aug 6, 2016 at 9:21 AM, Marco Sulla wrote: > I want to clarify that when I say "different from the other > languages", I mean "different from the most used languages", that in > my mind are C/C++, C#, Java, PHP and Javascript, mainly. > Ah, well, that's because those are all one family of languages. If instead you were familiar with four LISPy languages, you'd have a completely different set of expectations. > I agree it's not hard to understand that `str` is the string type and > `len()` is the function that gives you the length, even if you don't > know Python (and it's shorter to type...) But it's hard to remember, > in particular if you code also in other languages. When I come back to > Python, I always ends to write `somelist.length` instead of > `len(somelist)`, some arcane words come out my mouth and a little > fairy dies. > You open your mouth and a fairy dies.... hmm. Can I offer you a breath mint? :) ChrisA From steve+python at pearwood.info Fri Aug 5 21:14:20 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 06 Aug 2016 11:14:20 +1000 Subject: Python slang References: Message-ID: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> On Sat, 6 Aug 2016 08:00 am, Marco Sulla wrote: > I have a simple curiosity: why Python has much keywords, and some > builtin types and methods, that are different from the other > languages? What is the rationale? You should ask those other languages. Which languages do you have in mind? > I'm referring to: > * `except` instead of `catch` Because this isn't a game of "catch the ball". They're called "exceptions", not "catchions". You *try* something, and if an *exception* happens, the except block runs. In English, without any abbreviations: try doing this: code block except if a TypeError occurs, do this code: exception handler > * `raise` instead of `throw` Because this isn't a game of catch. > * `self` instead of `this` (I know, it's not enforced, but it's a de > facto standard and some IDEs like PyDev gives you an error if you > declare a non-static method without `self` as first parameter) Because "this" is a stupid name. In English, we refer to ourselves in the first person as I, me, myself, and sometimes "self", never as "this". One can say "this one has a hat", for example, but it sounds weird, like something the Borg would say about a specific Borg unit. > * `dict` instead of `map` Because "map" is a piece of paper showing a visual representation of a territory, and a dict (dictionary) is a set of words (the keys) followed by their definitions (the values). > * `list.append()` instead of `list.push()` You don't push onto a list, you push onto a stack. You append to a list. > * `str.strip()` instead of `str.trim()` Because "trim" is a stupid name for stripping whitespace from the ends of a string. When you trim hair or a plant, you unconditionally cut it and make it shorter. If strings had a "trim" method, it should look like this: def trim(self, distance): return self[distance:-distance] and similar for left-trim and right-trim. > * `True`, `False` and None instead of `true`, `false` and `none` (they > seems classes) You'd have to ask other languages why they use 'true', 'false' and 'none' (they seem like ordinary variables). > * and furthermore much abbreviation, like `str` instead of `string` > and `len()` instead of `length()`, that seems to contrast with the > readability goal of Python. Just because something is short doesn't make it less readable. According to Wolfram Alpha, the average length of words in English is 5.1 characters: http://www.wolframalpha.com/input/?i=average+english+word+length but of the ten most common words themselves, nine are of three or fewer characters: "the of to and a in is it you that" > I don't ask about `None` instead of `null` because I suppose here it's > a matter of disambiguation (null, in many languages, is not equal to > null). Really? Which languages? That's not true in Pascal, C, Ruby or Javascript. In Pascal, nil = nil: [steve at ando pascal]$ cat nil_equal.p program main(input, output); var a,b: ^integer; begin a := nil; b := nil; writeln(a = b); end. [steve at ando pascal]$ gpc nil_equal.p [steve at ando pascal]$ ./a.out True In C, nul == nul: [steve at ando c]$ cat null_equal.c #include int main() { int *a = NULL; int *b = NULL; printf("%d\n", a==b); return 0; } [steve at ando c]$ gcc null_equal.c [steve at ando c]$ ./a.out 1 In Ruby, nil == nil: irb(main):001:0> a = nil => nil irb(main):002:0> b = nil => nil irb(main):003:0> a == b => true In Javascript, null == null: js> a = null null js> b = null null js> a == b true I'd like to know what are these "many" languages where null != null. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Aug 5 21:35:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 11:35:54 +1000 Subject: Python slang In-Reply-To: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Aug 6, 2016 at 11:14 AM, Steven D'Aprano wrote: >> I don't ask about `None` instead of `null` because I suppose here it's >> a matter of disambiguation (null, in many languages, is not equal to >> null). > > Really? Which languages? That's not true in Pascal, C, Ruby or Javascript. > SQL (shown here with PostgreSQL's CLI): rosuav=> \pset null NULL Null display is "NULL". rosuav=> select NULL = NULL; ?column? ---------- NULL (1 row) But SQL's NULL is a cross between C's NULL, IEEE's NaN, Cthulhu, and Emrakul. ChrisA From steve+python at pearwood.info Fri Aug 5 21:55:00 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 06 Aug 2016 11:55:00 +1000 Subject: Python slang References: Message-ID: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> On Sat, 6 Aug 2016 08:31 am, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 8:00 AM, Marco Sulla via Python-list > wrote: [...] >> I'm referring to: >> * `except` instead of `catch` > > Not sure. Python does seem to be roughly unique in this. Delphi does uses the same terminology. Standard Pascal is too old and doesn't support exceptions, but newer variants that do seem to use some form of "except". >> * `raise` instead of `throw` > > Quite a few other languages talk about raising exceptions rather than > throwing them. Those would be the two most common terms, I think. Indeed. >> * `self` instead of `this` (I know, it's not enforced, but it's a de >> facto standard and some IDEs like PyDev gives you an error if you >> declare a non-static method without `self` as first parameter) > > Smalltalk and friends also use "self", so again, this would be the > other common word for the parameter. Out of the 23 languages listed by Wikipedia here: https://en.wikipedia.org/wiki/Comparison_of_programming_languages_%28object-oriented_programming%29#Special_variables I see: 10 use "self" 10 use "this" 3 use "me" 1 uses "current" (that adds to 24 because Xojo can use either self or me). That's only a tiny sample of languages in the world. I expect that there will be significantly more variation if you expand the list, but that "self" and "this" will remain the two most popular choices. [...] >> * `True`, `False` and None instead of `true`, `false` and `none` (they >> seems classes) > > This one, I've no idea about. Why have "bool" as the type, and "True" > and "False" as the instances? I think the built-in types have their > names grandfathered in from when they were factory functions, but that > doesn't explain the capitalized instances. (And the type of None is > NoneType, just to confuse the matter further.) Read the PEP: https://www.python.org/dev/peps/pep-0285/ 3) Should the constants be called 'True' and 'False' (similar to None) or 'true' and 'false' (as in C++, Java and C99)? => True and False. Most reviewers agree that consistency within Python is more important than consistency with other languages. >> I don't ask about `None` instead of `null` because I suppose here it's >> a matter of disambiguation (null, in many languages, is not equal to >> null). On the same Wikipedia page as above, I count: 10 languages using "null" 7 using "nil" 1 each using "initial", "void", "none", "nothing", "nullptr", "undef", "undefined" -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bostone at gmail.com Fri Aug 5 21:55:55 2016 From: bostone at gmail.com (Bo Stone) Date: Fri, 5 Aug 2016 18:55:55 -0700 (PDT) Subject: Installing Python 2.7 for all users on SLES 11 In-Reply-To: <2c1a4835-cc83-48b2-b197-0a102284485c@googlegroups.com> References: <2c1a4835-cc83-48b2-b197-0a102284485c@googlegroups.com> Message-ID: <016f5d5b-ea51-4349-873b-f58733bb8bfd@googlegroups.com> On Thursday, August 4, 2016 at 8:54:35 PM UTC-7, Bo Stone wrote: > I installed Python 2.7 on SLES 11 box that previously was running Python 2.6. To do so I used a script described in this post (http://stackoverflow.com/a/11371726/135946) and run it as a root user. Everything went well but when it was done I discovered few issues: > > 1. No symbolic links were created and no path updated so I had to manually update the path to link to the new installation bin directory /opt/python2.7/bin > 2. Everything runs good until I switch from root to the normal user at which point Python shell runs but some modules I installed such as PyYAML are missing. Again, these are OK when I run Python as root > 3. As a regular user I'm not able to run pip, easy_install and wheel. For pip I get "ImportError: No module named pkg_resources" > > This question is also posted to Stackoverflow here: http://stackoverflow.com/q/38749809/135946 The Stackoverflow question has now 100 points bounty From rosuav at gmail.com Fri Aug 5 22:10:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 12:10:46 +1000 Subject: Python slang In-Reply-To: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Aug 6, 2016 at 11:55 AM, Steven D'Aprano wrote: >>> * `True`, `False` and None instead of `true`, `false` and `none` (they >>> seems classes) >> >> This one, I've no idea about. Why have "bool" as the type, and "True" >> and "False" as the instances? I think the built-in types have their >> names grandfathered in from when they were factory functions, but that >> doesn't explain the capitalized instances. (And the type of None is >> NoneType, just to confuse the matter further.) > > Read the PEP: > > https://www.python.org/dev/peps/pep-0285/ > > 3) Should the constants be called 'True' and 'False' (similar to > None) or 'true' and 'false' (as in C++, Java and C99)? > > => True and False. > > Most reviewers agree that consistency within Python is more > important than consistency with other languages. > That answers the question about True and False, but not about None, which started the question. Why are instances capitalized here? Is it simply a matter of grandfathering in, with no specific decision? ChrisA From random832 at fastmail.com Fri Aug 5 22:46:20 2016 From: random832 at fastmail.com (Random832) Date: Fri, 05 Aug 2016 22:46:20 -0400 Subject: Python slang In-Reply-To: References: Message-ID: <1470451580.1029076.687499121.093A72E3@webmail.messagingengine.com> On Fri, Aug 5, 2016, at 20:14, Dennis Lee Bieber wrote: > .push() tends to make this one think "stack" and not general purpose > list. It's a bit unusual to have .pop() without a matching .push(). From rustompmody at gmail.com Fri Aug 5 22:51:15 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 5 Aug 2016 19:51:15 -0700 (PDT) Subject: Python slang In-Reply-To: References: <1470451580.1029076.687499121.093A72E3@webmail.messagingengine.com> Message-ID: On Saturday, August 6, 2016 at 8:16:36 AM UTC+5:30, Random832 wrote: > On Fri, Aug 5, 2016, at 20:14, Dennis Lee Bieber wrote: > > .push() tends to make this one think "stack" and not general purpose > > list. > > It's a bit unusual to have .pop() without a matching .push(). True? OTOH I tend to think of push (and pop) imperatively Whereas append (and maybe relatives starting cdr) are declarative [But that?s just me] From rosuav at gmail.com Fri Aug 5 22:52:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 12:52:10 +1000 Subject: Python slang In-Reply-To: <1470451580.1029076.687499121.093A72E3@webmail.messagingengine.com> References: <1470451580.1029076.687499121.093A72E3@webmail.messagingengine.com> Message-ID: On Sat, Aug 6, 2016 at 12:46 PM, Random832 wrote: > On Fri, Aug 5, 2016, at 20:14, Dennis Lee Bieber wrote: >> .push() tends to make this one think "stack" and not general purpose >> list. > > It's a bit unusual to have .pop() without a matching .push(). balloon.inflate() balloon.pop() ChrisA From steve+python at pearwood.info Fri Aug 5 23:10:15 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 06 Aug 2016 13:10:15 +1000 Subject: Python slang References: Message-ID: <57a55519$0$1607$c3e8da3$5496439d@news.astraweb.com> On Sat, 6 Aug 2016 10:13 am, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 9:21 AM, Marco Sulla > wrote: >> I want to clarify that when I say "different from the other >> languages", I mean "different from the most used languages", that in >> my mind are C/C++, C#, Java, PHP and Javascript, mainly. >> > > Ah, well, that's because those are all one family of languages. If > instead you were familiar with four LISPy languages, you'd have a > completely different set of expectations. Furthermore, that's only six languages out of, what, a couple of thousand known programming languages? And even languages clearly in the C family, like Objective-C, D, Swift, Java and Go, can end up using quite different syntax and execution models. It amuses me when people know a handful of languages, all clearly derived from each other, and think that's "most" languages. That's like somebody who knows Dutch, Afrikaans and German[1] being surprised that Russian, Cantonese, Hebrew and Vietnamese don't follow the same language rules as "most languages". >> I agree it's not hard to understand that `str` is the string type and >> `len()` is the function that gives you the length, even if you don't >> know Python (and it's shorter to type...) But it's hard to remember, >> in particular if you code also in other languages. When I come back to >> Python, I always ends to write `somelist.length` instead of >> `len(somelist)`, some arcane words come out my mouth and a little >> fairy dies. The obvious solution to that is that those other languages should replace their various list.length, collection.size, group.len, etc methods with a single function, like Python does. Why Python uses a len() function rather than a length/size/len/whatever method: http://effbot.org/pyfaq/why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list.htm [1] Low German, of course, not High German, the same branch of German which lead to Anglo-Saxon and hence to English. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Aug 5 23:13:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 13:13:28 +1000 Subject: Differences between \N escapes and unicodedata Message-ID: Not all Unicode codepoints are supported by unicodedata.name(), but they are supported in \N escapes and unicodedata.lookup. Is there a reason for this? Normally, you can do this: >>> "\N{GREEK SMALL LETTER OMEGA}" '?' >>> unicodedata.name(_) 'GREEK SMALL LETTER OMEGA' But check this out: >>> unicodedata.lookup("CHARACTER TABULATION") '\t' >>> "\N{CHARACTER TABULATION}" '\t' >>> unicodedata.name(_) Traceback (most recent call last): File "", line 1, in ValueError: no such name >>> unicodedata.lookup("NULL") '\x00' >>> "\N{NULL}" '\x00' >>> unicodedata.name(_) Traceback (most recent call last): File "", line 1, in ValueError: no such name Tested on 3.4, 3.5, and 3.6. Extremely odd. ChrisA From tjreedy at udel.edu Fri Aug 5 23:19:39 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 5 Aug 2016 23:19:39 -0400 Subject: Python slang In-Reply-To: References: Message-ID: On 8/5/2016 6:00 PM, Marco Sulla via Python-list wrote: > I have a simple curiosity: why Python has much keywords, and some > builtin types and methods, that are different from the other > languages? What is the rationale? > > I'm referring to: > * `except` instead of `catch` Python partly takes its cue from English. 'Try do this, except if it does not work, do that instead.' Catch? makes no sense in this respect. > * `raise` instead of `throw` When you talk with people, do you raise an objection, or throw one? Admittedly, some people throw tantrums ;-)/ > * `dict` instead of `map` A python dict is much more similar to a real world dictionary than a paper map. > * `list.append()` instead of `list.push()` Lists are not just stacks. I got a bit of flack for proposing the addition of list.pop, long after list.append, to make them more usable as stacks. Append is definitely the command language word for adding something to the end of a sequence. -- Terry Jan Reedy From rosuav at gmail.com Fri Aug 5 23:35:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Aug 2016 13:35:04 +1000 Subject: Python slang In-Reply-To: <57a55519$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <57a55519$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Aug 6, 2016 at 1:10 PM, Steven D'Aprano wrote: > It amuses me when people know a handful of languages, all clearly derived > from each other, and think that's "most" languages. That's like somebody > who knows Dutch, Afrikaans and German[1] being surprised that Russian, > Cantonese, Hebrew and Vietnamese don't follow the same language rules > as "most languages". Until you meet something different, though, you don't realize how similar they all are to each other. You're too busy noticing how different they are to see how similar they are. If all you know is English, you think that British English and American English are very different, and Australian English is nothing but slang. And then you meet one of the languages you list above, or even just another Western European language (all the same letters you know, but maybe one or two more, like ? or ?, and possibly some diacriticals), and suddenly all English is one language, and even dialects of low Scots become sufficiently similar that you can read them. Once you know C, Python, LISP, Perl, and at least one assembly language, you can understand a lot of code. Correction. You can understand a lot of programming languages. I would call myself highly fluent in Python, but I still come across code that makes me go "WAT?"... ChrisA From eryksun at gmail.com Sat Aug 6 00:25:26 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Aug 2016 04:25:26 +0000 Subject: Differences between \N escapes and unicodedata In-Reply-To: References: Message-ID: On Sat, Aug 6, 2016 at 3:13 AM, Chris Angelico wrote: >>>> unicodedata.lookup("NULL") > '\x00' >>>> "\N{NULL}" > '\x00' >>>> unicodedata.name(_) > Traceback (most recent call last): > File "", line 1, in > ValueError: no such name > > Tested on 3.4, 3.5, and 3.6. Extremely odd. U+0000 has a legacy name and alias names in the standard, but no primary name: http://www.unicode.org/Public/8.0.0/ucd/UnicodeData.txt http://www.unicode.org/Public/8.0.0/ucd/NameAliases.txt lookup() includes the aliases from the private use area where Python maps them (U+F0000 - U+F01CB), and of course maps it back to the correct character code. For the following I hacked unicodedata.name() to allow returning names for the alias range. Notice that there are multiple aliases for a given character, straight from the above-mentioned NameAliases database. >>> names = [unicodedata.name(chr(i)) for i in range(0xf0000, 0xf01cb)] >>> print(*textwrap.wrap(', '.join(names[:80])), sep='\n') NULL, NUL, START OF HEADING, SOH, START OF TEXT, STX, END OF TEXT, ETX, END OF TRANSMISSION, EOT, ENQUIRY, ENQ, ACKNOWLEDGE, ACK, ALERT, BEL, BACKSPACE, BS, CHARACTER TABULATION, HORIZONTAL TABULATION, HT, TAB, LINE FEED, NEW LINE, END OF LINE, LF, NL, EOL, LINE TABULATION, VERTICAL TABULATION, VT, FORM FEED, FF, CARRIAGE RETURN, CR, SHIFT OUT, LOCKING-SHIFT ONE, SO, SHIFT IN, LOCKING-SHIFT ZERO, SI, DATA LINK ESCAPE, DLE, DEVICE CONTROL ONE, DC1, DEVICE CONTROL TWO, DC2, DEVICE CONTROL THREE, DC3, DEVICE CONTROL FOUR, DC4, NEGATIVE ACKNOWLEDGE, NAK, SYNCHRONOUS IDLE, SYN, END OF TRANSMISSION BLOCK, ETB, CANCEL, CAN, END OF MEDIUM, EOM, SUBSTITUTE, SUB, ESCAPE, ESC, INFORMATION SEPARATOR FOUR, FILE SEPARATOR, FS, INFORMATION SEPARATOR THREE, GROUP SEPARATOR, GS, INFORMATION SEPARATOR TWO, RECORD SEPARATOR, RS, INFORMATION SEPARATOR ONE, UNIT SEPARATOR, US, SP, DELETE, DEL From michael.selik at gmail.com Sat Aug 6 02:30:43 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sat, 06 Aug 2016 06:30:43 +0000 Subject: Python slang In-Reply-To: References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 5, 2016 at 10:12 PM Chris Angelico wrote: > That answers the question about True and False, but not about None, > which started the question. Why are instances capitalized here? Is it > simply a matter of grandfathering in, with no specific decision? > When people ask me why the core classes are lowercased, I joke that it's following Hamming's thought that "true greatness" is when your name is spelled all lower case [0]. Great things are capitalized, like Counter, but truly great things are lowercased, like dict. BTW, this leads me to the thought that someone should edit Wikipedia to properly lowercase "hamming window". [0] "You and Your Research" (http://www.paulgraham.com/hamming.html) From eryksun at gmail.com Sat Aug 6 02:44:37 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Aug 2016 06:44:37 +0000 Subject: Python slang In-Reply-To: References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Aug 6, 2016 at 2:10 AM, Chris Angelico wrote: > That answers the question about True and False, but not about None, > which started the question. Why are instances capitalized here? Is it > simply a matter of grandfathering in, with no specific decision? Using CamelCase for the single-instance (singleton) objects -- None, NotImplemented, and Ellipsis -- highlights their special nature, i.e. that we check for them by identity. "not_implemented" would look like just another local variable. I wish that NotImplemented had been made a keyword, just like None, True, and False. I see no value in being able to assign to this name. It invites confusion. For Ellipsis we at least have `...`. From bc at freeuk.com Sat Aug 6 06:08:19 2016 From: bc at freeuk.com (BartC) Date: Sat, 6 Aug 2016 11:08:19 +0100 Subject: Python slang In-Reply-To: <57a55519$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <57a55519$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 06/08/2016 04:10, Steven D'Aprano wrote: > On Sat, 6 Aug 2016 10:13 am, Chris Angelico wrote: > >> On Sat, Aug 6, 2016 at 9:21 AM, Marco Sulla >> wrote: >>> I want to clarify that when I say "different from the other >>> languages", I mean "different from the most used languages", that in >>> my mind are C/C++, C#, Java, PHP and Javascript, mainly. >>> >> >> Ah, well, that's because those are all one family of languages. If >> instead you were familiar with four LISPy languages, you'd have a >> completely different set of expectations. > > Furthermore, that's only six languages out of, what, a couple of thousand > known programming languages? > > And even languages clearly in the C family, like Objective-C, D, Swift, Java > and Go, can end up using quite different syntax and execution models. > > It amuses me when people know a handful of languages, all clearly derived > from each other, and think that's "most" languages. That's like somebody > who knows Dutch, Afrikaans and German[1] being surprised that Russian, > Cantonese, Hebrew and Vietnamese don't follow the same language rules > as "most languages". The analogy isn't quite right. Most programming languages use English keywords. It's more like why US English uses words like "windshield", "hood" and "trunk" instead of the proper "windscreen", "bonnet" and "boot". Oddly, for all its dynamism, Python doesn't allow aliases for its keywords to allow for personal taste. -- Bartc From nimbiotics at gmail.com Sat Aug 6 07:48:24 2016 From: nimbiotics at gmail.com (Mario R. Osorio) Date: Sat, 6 Aug 2016 04:48:24 -0700 (PDT) Subject: datetime vs Arrow vs Pendulum vs Delorean vs udatetime In-Reply-To: References: Message-ID: <2c9b07d7-825f-49c0-81a6-590f9a562398@googlegroups.com> ... so you decided to start the post already hijacked by yourself ... very clever!! On Friday, August 5, 2016 at 8:19:53 PM UTC-4, bream... at gmail.com wrote: > On Friday, August 5, 2016 at 7:15:37 PM UTC+1, DFS wrote: > > On 8/4/2016 6:41 PM, breamoreboy at gmail.com wrote: > > > Fascinating stuff here > > > https://aboutsimon.com/blog/2016/08/04/datetime-vs-Arrow-vs-Pendulum-vs-Delorean-vs-udatetime.html > > > > > > I hereby ask that only people who know and use Python reply, not the > > > theoretical idiots who could not fight their way out of a wet paper > > > bag. Thank you. > > > > > > Kindest regards. > > > > > > Mark Lawrence. > > > > > > What would really be nice is if the idiot who asked for a reply actually > > posed a question. > > I do not need to ask questions, I simply put the reference forward as being interesting. Regretably so many threads are now hijacked so rather than discussing Python, which everything in the reference does, the people who are more interested in the theory than the practice take over, such that the next thing you know you're discussing anything except Python. I'll be blunt, I don't give a crap about Scheme, Lisp, Pike, Go, Rust, or any other language on this, the main *PYTHON* mailing list/newsgroup. I further don't give a crap about filosofy. In future can we please keep this place to discussing Python. If you want to discuss something else, why not go and find a more suitable venue, whereby, for example, you can chat about the problems with Unicode in Python3 with the RUE to your hearts content? > > Kindest regards. > > Mark Lawrence. From rustompmody at gmail.com Sat Aug 6 08:17:45 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 6 Aug 2016 05:17:45 -0700 (PDT) Subject: Out of line posts (was datetime vs Arrow vs Pendulum vs Delorean vs udatetime) In-Reply-To: <2c9b07d7-825f-49c0-81a6-590f9a562398@googlegroups.com> References: <2c9b07d7-825f-49c0-81a6-590f9a562398@googlegroups.com> Message-ID: <5d988a21-603c-4566-a3bf-76084454f785@googlegroups.com> On Saturday, August 6, 2016 at 5:19:08 PM UTC+5:30, Mario R. Osorio wrote: > ... so you decided to start the post already hijacked by yourself ... > > very clever!! If you see https://mail.python.org/pipermail/python-list/2016-August/thread.html#start you will see that there was only one post there ? by Steven d'Aprano And now yours. IOW both Mark Lawrence and ?DFS? have been banned (it seems) at least from the mailing list. They do appear on google-groups ? ironic given Mark?s feelings for the same To DFS: I believe you were banned for some insults/profanities. Would it not be a good idea to desist and stay within the ambit of this list?s subject? I would expect in due course the ban would be lifted. Would give the same advice to Mark? not sure if he?s listening [Personal note] I don?t like bans and have on occasion disagreed with the mods on that. Unfortunately it seems that their judgement is proving more right than mine :-( From deety19 at googlemail.com Sat Aug 6 13:16:28 2016 From: deety19 at googlemail.com (Dave) Date: Sat, 6 Aug 2016 18:16:28 +0100 Subject: where is it Message-ID: <1B29002BAB664D3DA868E8C6208AEFA8@DvePC> I am trying to associate the .py file extension with idle.......where IS idle? Can you make it a bit more difficult to load/use your software please. From gengyangcai at gmail.com Sat Aug 6 13:51:34 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 6 Aug 2016 10:51:34 -0700 (PDT) Subject: How do I make a game in Python ? Message-ID: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> How do I make a game in Python ? From mail.python.org at marco.sulla.e4ward.com Sat Aug 6 14:03:31 2016 From: mail.python.org at marco.sulla.e4ward.com (Michael Selik) Date: Sat, 06 Aug 2016 18:03:31 +0000 Subject: Python slang In-Reply-To: References: Message-ID: On Sat, Aug 6, 2016, 10:10 AM Marco Sulla via Python-list < python-list at python.org> wrote: > On 6 August 2016 at 00:31, Chris Angelico wrote: > > On Sat, Aug 6, 2016 at 8:00 AM, Marco Sulla via Python-list > > wrote: > >> * `dict` instead of `map` > > > > "map" has many other meanings (most notably the action wherein you > > call a function on every member of a collection to produce another > > collection). > > Well, the word `map` of the `map` function is a verb. IMO there would > be no ambiguity. > Are you thinking of Lisp-2 where functions have a separate namespace from everything else? From mail.python.org at marco.sulla.e4ward.com Sat Aug 6 14:22:56 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 20:22:56 +0200 Subject: Python slang In-Reply-To: References: Message-ID: On 6 August 2016 at 02:13, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 9:21 AM, Marco Sulla > wrote: >> I want to clarify that when I say "different from the other >> languages", I mean "different from the most used languages", that in >> my mind are C/C++, C#, Java, PHP and Javascript, mainly. >> > > Ah, well, that's because those are all one family of languages. If > instead you were familiar with four LISPy languages, you'd have a > completely different set of expectations. Well, they are the most used languages. I think about the 80% of programmers knows at least one of that languages. It's more simple to learn a new language if it's similar to the others. >> I agree it's not hard to understand that `str` is the string type and >> `len()` is the function that gives you the length, even if you don't >> know Python (and it's shorter to type...) But it's hard to remember, >> in particular if you code also in other languages. When I come back to >> Python, I always ends to write `somelist.length` instead of >> `len(somelist)`, some arcane words come out my mouth and a little >> fairy dies. >> > > You open your mouth and a fairy dies.... hmm. Can I offer you a breath mint? :) Do you have also a plenary indulgence? From tjreedy at udel.edu Sat Aug 6 14:33:06 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 6 Aug 2016 14:33:06 -0400 Subject: Python slang In-Reply-To: References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/6/2016 2:30 AM, Michael Selik wrote: > When people ask me why the core classes are lowercased, Int, float, list, dict, etc were once functions that return objects of type 'int', 'float', 'list', 'dict', etc, before they became 'new-style classes', which are not just 'classes'. The lowercasing was maintained as new builting classes were added. -- Terry Jan Reedy From mail.python.org at marco.sulla.e4ward.com Sat Aug 6 15:03:28 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 21:03:28 +0200 Subject: Python slang In-Reply-To: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 6 August 2016 at 03:14, Steven D'Aprano wrote: > On Sat, 6 Aug 2016 08:00 am, Marco Sulla wrote: >> I'm referring to: >> * `except` instead of `catch` > > Because this isn't a game of "catch the ball". They're called "exceptions", > not "catchions". You *try* something, and if an *exception* happens, the > except block runs. So if I try to pick up a girl, I have to knock back the refuse? >> * `self` instead of `this` (I know, it's not enforced, but it's a de >> facto standard and some IDEs like PyDev gives you an error if you >> declare a non-static method without `self` as first parameter) > > Because "this" is a stupid name. > > In English, we refer to ourselves in the first person as I, me, myself, and > sometimes "self", never as "this". One can say "this one has a hat", for > example, but it sounds weird, like something the Borg would say about a > specific Borg unit. It's simply a matter of point of view. In the point of view of class, you are referring to the class itself, so "self" it's a good word. >From the point of view of the programmer, you're referring to this class, so "this" is a good word as well. I don't see a semantic advantage of "self". >> * `dict` instead of `map` > > Because "map" is a piece of paper showing a visual representation of a > territory, and a dict (dictionary) is a set of words (the keys) followed by > their definitions (the values). If map is only a piece of paper, dictionary is only a book, isn't it? Dictionary is a book, and inside it there's a set of words followed by their definitions, as you said. Map is a piece of paper, and every element of the map represent some other element, in a 1-1 relationship. >> * `str.strip()` instead of `str.trim()` > > Because "trim" is a stupid name for stripping whitespace from the ends of a > string. When you trim hair or a plant, you unconditionally cut it and make > it shorter. Not at all. "trim" word is used when you have to remove something that is useless, usually. On the contrary, "strip" can be used also in other means. >> * `True`, `False` and None instead of `true`, `false` and `none` (they >> seems classes) > > You'd have to ask other languages why they use 'true', 'false' and 'none' > (they seem like ordinary variables). I think you find the reason. Not sure it was a good idea (also vim has syntax highlighting). >> * and furthermore much abbreviation, like `str` instead of `string` >> and `len()` instead of `length()`, that seems to contrast with the >> readability goal of Python. > > Just because something is short doesn't make it less readable. According to > Wolfram Alpha, the average length of words in English is 5.1 characters: > > http://www.wolframalpha.com/input/?i=average+english+word+length > > but of the ten most common words themselves, nine are of three or fewer > characters: > > "the of to and a in is it you that" And according to Wolfram Alpha, the readability of Huckleberry Finn is 5.5. I think it doesn't fit in this discussion in the same way :P From mail.python.org at marco.sulla.e4ward.com Sat Aug 6 15:06:35 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 21:06:35 +0200 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 6 August 2016 at 03:35, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 11:14 AM, Steven D'Aprano > wrote: >>> I don't ask about `None` instead of `null` because I suppose here it's >>> a matter of disambiguation (null, in many languages, is not equal to >>> null). >> >> Really? Which languages? That's not true in Pascal, C, Ruby or Javascript. >> > > SQL (shown here with PostgreSQL's CLI): > > rosuav=> \pset null NULL > Null display is "NULL". > rosuav=> select NULL = NULL; > ?column? > ---------- > NULL > (1 row) > > But SQL's NULL is a cross between C's NULL, IEEE's NaN, Cthulhu, and Emrakul. Yes, I was thinking manly to SQL. That furthermore is NOT a programming language. So I suppose I have also to ask why "None" instead of "Null" From mail.python.org at marco.sulla.e4ward.com Sat Aug 6 15:17:01 2016 From: mail.python.org at marco.sulla.e4ward.com (Marco Sulla) Date: Sat, 6 Aug 2016 21:17:01 +0200 Subject: Python slang In-Reply-To: References: Message-ID: On 6 August 2016 at 20:03, Michael Selik wrote: > On Sat, Aug 6, 2016, 10:10 AM Marco Sulla via Python-list > wrote: >> >> On 6 August 2016 at 00:31, Chris Angelico wrote: >> > "map" has many other meanings (most notably the action wherein you >> > call a function on every member of a collection to produce another >> > collection). >> >> Well, the word `map` of the `map` function is a verb. IMO there would >> be no ambiguity. > > > Are you thinking of Lisp-2 where functions have a separate namespace from > everything else? Nope, I was thinking that "map()" should be a method of an iterable. But this is another topic :) From rosuav at gmail.com Sat Aug 6 15:29:34 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 05:29:34 +1000 Subject: Python slang In-Reply-To: References: Message-ID: On Sun, Aug 7, 2016 at 5:17 AM, Marco Sulla wrote: > > Nope, I was thinking that "map()" should be a method of an iterable. > But this is another topic :) The problem with that is that it has to become a method of _every_ iterable type, which means it becomes part of the protocol of iterables. Much easier to make it a built-in function, which can then accept any iterable based only on the __iter__ method. ChrisA From Bernd.Nawothnig at t-online.de Sat Aug 6 15:37:30 2016 From: Bernd.Nawothnig at t-online.de (Bernd Nawothnig) Date: Sat, 6 Aug 2016 21:37:30 +0200 Subject: Python slang References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-06, Chris Angelico wrote: > On Sat, Aug 6, 2016 at 11:14 AM, Steven D'Aprano > wrote: >>> I don't ask about `None` instead of `null` because I suppose here it's >>> a matter of disambiguation (null, in many languages, is not equal to >>> null). >> >> Really? Which languages? That's not true in Pascal, C, Ruby or >> Javascript. >> > > SQL (shown here with PostgreSQL's CLI): > > rosuav=> \pset null NULL > Null display is "NULL". > rosuav=> select NULL = NULL; > ?column? > ---------- > NULL > (1 row) > > But SQL's NULL is a cross between C's NULL, IEEE's NaN, Cthulhu, and > Emrakul. SQL NULL has the semantic of "unknown". So if one or both operands of a comparison (or any other operation) are unknown the result is unknown too. And that means NULL. That is also the reason why you have the keyword STRICT when declaring a function in PostgreSQL. If only one parameter of the function is NULL and the result depends directly on the parameter the planner can then automatically skip the function call completely und replace the result by NULL without any need to analyse or enter the function body. Bernd -- no time toulouse From rosuav at gmail.com Sat Aug 6 16:43:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 06:43:06 +1000 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 5:37 AM, Bernd Nawothnig wrote: >> But SQL's NULL is a cross between C's NULL, IEEE's NaN, Cthulhu, and >> Emrakul. > > SQL NULL has the semantic of "unknown". So if one or both operands of > a comparison (or any other operation) are unknown the result is > unknown too. And that means NULL. That's not entirely accurate, and it doesn't explain why NULL sometimes behaves as if it's a genuine value, and sometimes as if it's completely not there. For instance, taking the average of a column of values ignores NULLs completely, and COUNT(column) is the same as COUNT(column) WHERE column IS NOT NULL; but in some situations it behaves more like NaN: rosuav=> select null or true, null or false, null and true, null and false; ?column? | ?column? | ?column? | ?column? ----------+----------+----------+---------- t | NULL | NULL | f (1 row) Anything "or true" has to be true, so NULL OR TRUE is true. And then there are the times when NULL acts like a completely special value, for instance in a foreign key - it means "there isn't anything on the other end of this relationship", and is perfectly legal. Or in a SELECT DISTINCT, where NULL behaves just like any other value - if there are any NULL values in the column, you get back exactly one NULL in the result. So Innistrad's plot becomes far simpler. Jace Beleren went mad because Emrakul tried to explain SQL's NULL to him. ChrisA From ian.g.kelly at gmail.com Sat Aug 6 17:33:47 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 6 Aug 2016 15:33:47 -0600 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Aug 6, 2016 2:10 PM, "Marco Sulla via Python-list" < python-list at python.org> wrote: Yes, I was thinking manly to SQL. That furthermore is NOT a programming language. Why not? It's claimed to be Turing complete. http://beza1e1.tuxen.de/articles/accidentally_turing_complete.html From torriem at gmail.com Sat Aug 6 17:33:50 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 6 Aug 2016 15:33:50 -0600 Subject: Python slang In-Reply-To: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> On 08/05/2016 07:14 PM, Steven D'Aprano wrote: > In English, we refer to ourselves in the first person as I, me, myself, and > sometimes "self", never as "this". One can say "this one has a hat", for > example, but it sounds weird, like something the Borg would say about a > specific Borg unit. Sadly it has become an epidemic of late for folks to misuse the word, "myself." I think it comes from people not wanting to sound presumptuous when referring to themselves. It drives me crazy to hear so many people say something like, "this research was done by myself and my colleague Bob." No, this research was done by my colleague Bob and I. Or this research was just only by me. However I have no problem with Python programmers using self in their code! From rosuav at gmail.com Sat Aug 6 17:37:26 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 07:37:26 +1000 Subject: Python slang In-Reply-To: <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: On Sun, Aug 7, 2016 at 7:33 AM, Michael Torrie wrote: > Sadly it has become an epidemic of late for folks to misuse the word, > "myself." I think it comes from people not wanting to sound > presumptuous when referring to themselves. It drives me crazy to hear > so many people say something like, "this research was done by myself and > my colleague Bob." No, this research was done by my colleague Bob and I. > Or this research was just only by me. > More likely, the correct version is "this research was done by my colleague Bob, while I looked on". :D ChrisA From ian.g.kelly at gmail.com Sat Aug 6 17:41:06 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 6 Aug 2016 15:41:06 -0600 Subject: How do I make a game in Python ? In-Reply-To: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> Message-ID: On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: How do I make a game in Python ? import random answer = random.randint(0, 100) while True: guess = input("What number am I thinking of? ") if int(guess) == answer: print("Correct!") break print("Wrong!") And now you've made a game in Python. On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: > How do I make a game in Python ? > -- > https://mail.python.org/mailman/listinfo/python-list > From ian.g.kelly at gmail.com Sat Aug 6 17:47:27 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 6 Aug 2016 15:47:27 -0600 Subject: Python slang In-Reply-To: <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: On Aug 6, 2016 3:36 PM, "Michael Torrie" wrote: Sadly it has become an epidemic of late for folks to misuse the word, "myself." I think it comes from people not wanting to sound presumptuous when referring to themselves. It drives me crazy to hear so many people say something like, "this research was done by myself and my colleague Bob." No, this research was done by my colleague Bob and I. "by my colleague Bob and *me*". If you're going to gripe about word usage (not grammar, since "myself" is grammatically correct), at least correct it correctly. From gengyangcai at gmail.com Sat Aug 6 17:51:24 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 6 Aug 2016 14:51:24 -0700 (PDT) Subject: How do I make a game in Python ? In-Reply-To: References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> Message-ID: <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> As in, any recommended websites that helps users create complex games in Python ? On Sunday, August 7, 2016 at 5:41:25 AM UTC+8, Ian wrote: > On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: > > How do I make a game in Python ? > > > import random > > answer = random.randint(0, 100) > while True: > guess = input("What number am I thinking of? ") > if int(guess) == answer: > print("Correct!") > break > print("Wrong!") > > And now you've made a game in Python. > > > On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: > > > How do I make a game in Python ? > > -- > > https://mail.python.org/mailman/listinfo/python-list > > From ned at nedbatchelder.com Sat Aug 6 17:53:25 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 6 Aug 2016 14:53:25 -0700 (PDT) Subject: How do I make a game in Python ? In-Reply-To: <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> Message-ID: On Saturday, August 6, 2016 at 5:51:40 PM UTC-4, Cai Gengyang wrote: > As in, any recommended websites that helps users create complex games in Python ? http://inventwithpython.com/ --Ned. From marcosullaroma at gmail.com Sat Aug 6 17:53:59 2016 From: marcosullaroma at gmail.com (Marco Sulla) Date: Sat, 6 Aug 2016 23:53:59 +0200 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 6 August 2016 at 23:33, Ian Kelly wrote: > On Aug 6, 2016 2:10 PM, "Marco Sulla via Python-list" < > python-list at python.org> wrote: > > > Yes, I was thinking manly to SQL. That furthermore is NOT a > programming language. > > > Why not? It's claimed to be Turing complete. There's always something to learn. From lawrencedo99 at gmail.com Sat Aug 6 18:05:52 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 6 Aug 2016 15:05:52 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> Message-ID: <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> On Saturday, August 6, 2016 at 12:08:30 PM UTC+12, bream... at gmail.com wrote: > A couple or three years old but this is well worth seeing for anybody, > regardless of your Python expertise. http://nedbatchelder.com/text/iter.html A loop like for i in ... : ... if ... cond ... : break ... #end for actually has two different ways to terminate. Is there any good reason for them to be written two different ways? From lawrencedo99 at gmail.com Sat Aug 6 18:14:46 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 6 Aug 2016 15:14:46 -0700 (PDT) Subject: Announcing HarfPy Message-ID: HarfPy is a Python binding for the HarfBuzz typographic shaping library. It is designed to work in conjunction with my Python wrappers for the other major parts of the Linux typography stack: * Qahirah -- my binding for the Cairo graphics library * python_freetype -- my binding for FreeType * PyBidi -- my binding for FriBidi Some simple examples of it in action are here , output from which may be viewed here . From jamesfriendsfamily at gmail.com Sat Aug 6 19:23:52 2016 From: jamesfriendsfamily at gmail.com (James Santiago) Date: Sat, 6 Aug 2016 16:23:52 -0700 Subject: Python and 64bit Windows7 Message-ID: Hello: I am having problems when installing Python on a 64bit windows7 laptop. I am guessing that the problem is happening because of the 64bit architecture. Could you please take a look at the attached file that has screen shots of the problem I am facing? Could you tell me how I can fix these issues and get Python and PyCharm to work? I would appreciate your help. Regards James From ned at nedbatchelder.com Sat Aug 6 19:35:52 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 6 Aug 2016 16:35:52 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> Message-ID: <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> On Saturday, August 6, 2016 at 6:06:27 PM UTC-4, Lawrence D?Oliveiro wrote: > On Saturday, August 6, 2016 at 12:08:30 PM UTC+12, bream... at gmail.com wrote: > > A couple or three years old but this is well worth seeing for anybody, > > regardless of your Python expertise. http://nedbatchelder.com/text/iter.html > > A loop like > > for i in ... : > ... > if ... cond ... : > break > ... > #end for > > actually has two different ways to terminate. Is there any good reason for them to be written two different ways? Didn't we already do this debate? --Ned. From ned at nedbatchelder.com Sat Aug 6 19:38:21 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 6 Aug 2016 16:38:21 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: On Saturday, August 6, 2016 at 7:36:06 PM UTC-4, Ned Batchelder wrote: > On Saturday, August 6, 2016 at 6:06:27 PM UTC-4, Lawrence D?Oliveiro wrote: > > On Saturday, August 6, 2016 at 12:08:30 PM UTC+12, bream... at gmail.com wrote: > > > A couple or three years old but this is well worth seeing for anybody, > > > regardless of your Python expertise. http://nedbatchelder.com/text/iter.html > > > > A loop like > > > > for i in ... : > > ... > > if ... cond ... : > > break > > ... > > #end for > > > > actually has two different ways to terminate. Is there any good reason for them to be written two different ways? > > Didn't we already do this debate? For example: https://mail.python.org/pipermail/python-list/2016-June/709758.html Maybe we don't have to revisit it... :) --Ned. From grant.b.edwards at gmail.com Sat Aug 6 19:38:50 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 6 Aug 2016 23:38:50 +0000 (UTC) Subject: Python slang References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: On 2016-08-06, Michael Torrie wrote: > Sadly it has become an epidemic of late for folks to misuse the > word, "myself." I think it comes from people not wanting to sound > presumptuous when referring to themselves. It drives me crazy to > hear so many people say something like, "this research was done by > myself and my colleague Bob." No, this research was done by my > colleague Bob and I. Sadly it has become an epidemic of late for folks to misuse the subjective pronoun "I" when the objective pronoun "me" is correct. It drives me crazy to hear people say something like "this research was done by my colleague Bob and I." No, this research was done by my colleague Bob and me. :) Unfortunately, the Internet allows no posting containing grammar criticism to be grammatically correct. [Including this one, which I'm sure has an error in grammar -- I just can't find it.] -- Grant From tjreedy at udel.edu Sat Aug 6 19:41:25 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 6 Aug 2016 19:41:25 -0400 Subject: How do I make a game in Python ? In-Reply-To: <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> Message-ID: On 8/6/2016 5:51 PM, Cai Gengyang wrote: > As in, any recommended websites that helps users create complex games in Python ? For 2D graphics, try pygame among others. For 3D, I don't know what is current. > On Sunday, August 7, 2016 at 5:41:25 AM UTC+8, Ian wrote: >> On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: >> >> How do I make a game in Python ? >> >> >> import random >> >> answer = random.randint(0, 100) >> while True: >> guess = input("What number am I thinking of? ") >> if int(guess) == answer: >> print("Correct!") >> break >> print("Wrong!") >> >> And now you've made a game in Python. I suspect you missed the wisdom contained in this answer. 1. Define the core mechanic. 2. Start simple. Guess/Search for something hidden is a common core mechanic. Do you want this to be part of your game. With this start ... 3. Elaborate. I can think of several ways. Give hints. Add graphics. Increase player skill with repeated play. Use objects other than counts. Keep track of number of guesses and grade results. Keep track of actual guesses and inform about repeats. Record guesses visually so repetition is essentially impossible. >> >> On Aug 6, 2016 11:57 AM, "Cai Gengyang" wrote: >> >>> How do I make a game in Python ? Start with an idea for the game. -- Terry Jan Reedy From lawrencedo99 at gmail.com Sat Aug 6 21:04:14 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 6 Aug 2016 18:04:14 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: <7073edc1-1ec0-4ba3-a67a-d93ce23db413@googlegroups.com> On Saturday, August 6, 2016 at 8:05:40 AM UTC+12, eryk sun wrote: > I don't know all of the compatibility problems and > constraints (and pressure from important customers) that they faced ... Given some of their more recent (and not-so-recent) decisions, one wonders how much of this ?pressure from important customers? they really pay attention to... From lawrencedo99 at gmail.com Sat Aug 6 21:11:54 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 6 Aug 2016 18:11:54 -0700 (PDT) Subject: Ide or code editor confusion In-Reply-To: References: <2184a0ad-9d51-42db-ae11-5d0ab4512582@googlegroups.com> <7f142caf-5e3c-4fcf-818a-017b593fd0f7@googlegroups.com> <6688b08e-75cc-4132-b0e0-f63dcbfa4bd2@googlegroups.com> <713cc458-a469-eb3a-9f3f-81b1624cf245@gmail.com> Message-ID: <60ce0aa5-7b04-46cb-b4ba-4733a5f2aeef@googlegroups.com> On Friday, August 5, 2016 at 6:36:45 AM UTC+12, Michael Torrie wrote: > On 08/04/2016 01:23 AM, Lawrence D?Oliveiro wrote: >> That?s why I?ve come to the conclusion it?s a waste of time buying books >> on computing topics. They start to reek of decay while they?re still on >> the shelf. > > Except for old Unix books! I don?t think this one has aged well. > I've got an old book on sed and awk that will likely be in date for years > to come! I may have used sed, but I could never see the point in awk--everything it could do, I could do just as concisely in Perl, and more. > There's also an old book on vi, another on regular expression. Does your book on regular expressions mention character classes, for example ? > So some things are invaluable references. I would say that applies to more fundamental principles and theory, rather than the detailed functions of particular pieces of software. > Moving targets like Python, that's another story of course! When even bash comes out with new versions with major new features, you realize that everything is a moving target... From greg.ewing at canterbury.ac.nz Sat Aug 6 21:17:04 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 07 Aug 2016 13:17:04 +1200 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: Ian Kelly wrote: > (not grammar, since "myself" is grammatically correct) Not sure about that. "Myself" is a reflexive pronoun, used when the subject and object of a verb are the same. So "I did this research by myself" is correct. But if Bob is involved, the subject and object are different, so "myself" is not appropriate. Whether that's a syntax error or a semantic error, I'm not enough of a linguist to say. -- Greg From lawrencedo99 at gmail.com Sat Aug 6 21:21:02 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 6 Aug 2016 18:21:02 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: On Sunday, August 7, 2016 at 11:36:06 AM UTC+12, Ned Batchelder wrote: > Didn't we already do this debate? I understand. You want to discuss loops, just not *those* sorts of loops... From ned at nedbatchelder.com Sat Aug 6 21:26:32 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 6 Aug 2016 18:26:32 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: On Saturday, August 6, 2016 at 9:21:24 PM UTC-4, Lawrence D?Oliveiro wrote: > On Sunday, August 7, 2016 at 11:36:06 AM UTC+12, Ned Batchelder wrote: > > > Didn't we already do this debate? > > I understand. You want to discuss loops, just not *those* sorts of loops... I didn't post this link here. I'm merely pointing out that your concern about multiple ways to exit loops sounds like exactly what was discussed here two months ago. Is there any reason to think there is new material to discuss? --Ned. From crk at godblessthe.us Sat Aug 6 22:27:30 2016 From: crk at godblessthe.us (Clayton Kirkwood) Date: Sat, 6 Aug 2016 19:27:30 -0700 Subject: Python slang In-Reply-To: References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: <033801d1f053$3f729240$be57b6c0$@godblessthe.us> You can always tell the correctness by removing the other person. If it doesn't sound right then, then it is wrong. You don't say 'I gave to I', or 'I gave to me', it is 'I gave to myself'. crk > -----Original Message----- > From: Python-list [mailto:python-list- > bounces+crk=godblessthe.us at python.org] On Behalf Of Gregory Ewing > Sent: Saturday, August 06, 2016 6:17 PM > To: python-list at python.org > Subject: Re: Python slang > > Ian Kelly wrote: > > (not grammar, since "myself" is grammatically correct) > > Not sure about that. "Myself" is a reflexive pronoun, used when the subject and > object of a verb are the same. > So "I did this research by myself" is correct. But if Bob is involved, the subject > and object are different, so "myself" is not appropriate. > > Whether that's a syntax error or a semantic error, I'm not enough of a linguist > to say. > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list From steve+python at pearwood.info Sat Aug 6 22:32:23 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 12:32:23 +1000 Subject: Python slang References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a69db8$0$22140$c3e8da3$5496439d@news.astraweb.com> On Sun, 7 Aug 2016 04:33 am, Terry Reedy wrote: > On 8/6/2016 2:30 AM, Michael Selik wrote: > >> When people ask me why the core classes are lowercased, > > Int, float, list, dict, etc were once functions that return objects of > type 'int', 'float', 'list', 'dict', etc, before they became 'new-style > classes', which are not just 'classes'. The lowercasing was maintained > as new builting classes were added. That's a good point, but it's not the only reason. After all, come Python 3, they could have changed their name to Int, Float, List, Dict etc, but didn't. These built-ins may be classes, but they continue to be used as if they were functions, in a context where their function-ness is more important than their class-ness. (Especially int, float and str.) They were left as lowercase names even when they could have changed for that reason. The naming conventions are a little bit inconsistent in Python, partly for historical reasons, partly for aesthetic reasons. For example, in general classes which produce "atomic types" tend to be written in lowercase. By "atomic type", I mean a class which "feels" like it is a primitive, low-level structure rather than an object with attributes, for example: - builtins int, float, str, bytes, dict, list, tuple, set, frozenset, bool; - array from the array module; - defaultdict and deque from collections; but not - Decimal from decimal module; - Fraction from fractions module; - Counter and ChainMap from collections module. although I suppose ChainMap does expose at least the "maps" attribute. But in my opinion Decimal, Fraction and Counter should all be lowercased, like float, int and dict. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rustompmody at gmail.com Sat Aug 6 22:33:57 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 6 Aug 2016 19:33:57 -0700 (PDT) Subject: Python slang In-Reply-To: References: Message-ID: On Sunday, August 7, 2016 at 1:25:43 AM UTC+5:30, Marco Sulla wrote: > On 6 August 2016 at 02:13, Chris Angelico wrote: > > On Sat, Aug 6, 2016 at 9:21 AM, Marco Sulla wrote: > >> I want to clarify that when I say "different from the other > >> languages", I mean "different from the most used languages", that in > >> my mind are C/C++, C#, Java, PHP and Javascript, mainly. > >> > > > > Ah, well, that's because those are all one family of languages. If > > instead you were familiar with four LISPy languages, you'd have a > > completely different set of expectations. > > Well, they are the most used languages. I think about the 80% of > programmers knows at least one of that languages. True > It's more simple to learn a new language if it's similar to the others. Not true At a specific level (of programming languages) : http://blog.languager.org/2015/06/functional-programming-moving-target.html talks of functional programming taking about 50 years to get mainstream More generally: http://blog.languager.org/2016/01/how-long.html is a collection of examples showing how humans en masse can persist in error for hundreds, sometimes thousands, of years From rosuav at gmail.com Sat Aug 6 23:02:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 13:02:12 +1000 Subject: Python slang In-Reply-To: <57a69db8$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <57a54375$0$1589$c3e8da3$5496439d@news.astraweb.com> <57a69db8$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 12:32 PM, Steven D'Aprano wrote: > By "atomic type", I mean a class which "feels" like it is a primitive, > low-level structure rather than an object with attributes, for example: > > - builtins int, float, str, bytes, dict, list, tuple, set, frozenset, bool; > - array from the array module; > - defaultdict and deque from collections; > > but not > > - Decimal from decimal module; > - Fraction from fractions module; > - Counter and ChainMap from collections module. > > > although I suppose ChainMap does expose at least the "maps" attribute. But > in my opinion Decimal, Fraction and Counter should all be lowercased, like > float, int and dict. I somewhat agree with you about Decimal and Fraction, but not Counter. In that situation, I'd be more inclined to make DefaultDict than counter, if consistency is needed; there are too many other ways the word "counter" can be used, so I'd rather not dedicate it to a class. IMO the only types that really deserve the lower-case name are built-ins (or ones where they're really functions in disguise, like functools.partial). Of course, there's no point renaming them now, but I don't see any particular value in "a modified version of dict" getting a lower-case name - if I subclass dict myself, PEP 8 says it should be called MyDict, not mydict, and defaultdict should be no different. The only reason Decimal and Fraction could plausibly be lowercased is that they function very much the same way that int and float do, and also they could potentially become builtins. But they probably won't, so I don't have any problem with them remaining as they are, feeling like classes rather than feeling like atoms. ChrisA From torriem at gmail.com Sat Aug 6 23:03:56 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 6 Aug 2016 21:03:56 -0600 Subject: Python slang In-Reply-To: <033801d1f053$3f729240$be57b6c0$@godblessthe.us> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> <033801d1f053$3f729240$be57b6c0$@godblessthe.us> Message-ID: On 08/06/2016 08:27 PM, Clayton Kirkwood wrote: > You can always tell the correctness by removing the other person. If it > doesn't sound right then, then it is wrong. You don't say 'I gave to I', or > 'I gave to me', it is 'I gave to myself'. Yup good point, and I failed at that obviously. Good rule of thumb (for which there are likely exceptions I'm not aware of. This work was done by me, or I did the work myself. I gave to myself, but it was given to me. From torriem at gmail.com Sat Aug 6 23:14:04 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 6 Aug 2016 21:14:04 -0600 Subject: How do I make a game in Python ? In-Reply-To: <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> Message-ID: <13bed927-852f-539d-db87-37420484f203@gmail.com> On 08/06/2016 03:51 PM, Cai Gengyang wrote: > As in, any recommended websites that helps users create complex games in Python ? I imagine you create a complex game in Python the same way you'd do it in just about any other language. Thus any website on game design would be broadly applicable. And I'm sure there are a wide variety of topics that are relevant. Logic, game play, artificial intelligence, user interface, physics engines, and so forth. What you need to learn depends on what you already know and have experience in. The programming language is really only a part of developing "complex games." You'd probably want to start with simple, even child-like games first. Asking how to program complex games is a bit like asking how to build a car. From steve+python at pearwood.info Sat Aug 6 23:22:10 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 13:22:10 +1000 Subject: Me, myself and I [was Re: Python slang] References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> Message-ID: <57a6a964$0$1588$c3e8da3$5496439d@news.astraweb.com> On Sun, 7 Aug 2016 07:33 am, Michael Torrie wrote: > On 08/05/2016 07:14 PM, Steven D'Aprano wrote: >> In English, we refer to ourselves in the first person as I, me, myself, >> and sometimes "self", never as "this". One can say "this one has a hat", >> for example, but it sounds weird, like something the Borg would say about >> a specific Borg unit. > > Sadly it has become an epidemic of late for folks to misuse the word, > "myself." I think it comes from people not wanting to sound > presumptuous when referring to themselves. I myself must admit that myself hasn't come across that often. *wink* > It drives me crazy to hear > so many people say something like, "this research was done by myself and > my colleague Bob." No, this research was done by my colleague Bob and I. Heh, Muphry's Law strikes again! http://grammar.about.com/od/il/g/MuphrysLaw.htm "This research was done by my colleague Bob and I" is a case of hyper-correction. It's a very common mistake, but a mistake it is. The test for whether it is correct or not is to get rid of your colleague Bob: "This research was done by I" is clearly wrong, as you recognise in your next statement: > Or this research was just only by me. Which is correct. Or at least it would be, if you inserted the missing verb :-) The technical reason is that "I" is the first-person singular subject case, and "me" the first-person singular object case. When the subject of the sentence or clause is the first-person singular, you use "I": "I did the research." "I cannot tell a lie, I chopped down the cherry tree." When the object of the sentence is the first-person singular, you say "me": "George is a big bully, he punched me." Passive sentences reverse subject and object: "I did the research" becomes "The research was done by me". "George punched me" because "I was punched by George". But you don't exchange subject and object just because there are two people involved! Still unsure which to use? Blame somebody else: "The research was done by George and I/me" => "The research was done by George and she/her". Clearly it must be *her*, the objective case, therefore the right answer is "me" not "I". > However I have no problem with Python programmers using self in their > code! "Myself" is an alternative objective case for first-person singular, typically used for emphasis: "I did it myself." "I did the research, myself." It is also used as the object of a reflexive verb (verbs where the object is the same as the subject): "I will defend myself." "I accidentally kicked myself in the head." But using it in place of "me"? I don't think so. "The research was done by myself." "He punched myself in the head." I wouldn't say it is *wrong*, just inelegant and rather pretentious. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 6 23:31:42 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 13:31:42 +1000 Subject: where is it References: <1B29002BAB664D3DA868E8C6208AEFA8@DvePC> Message-ID: <57a6aba0$0$1591$c3e8da3$5496439d@news.astraweb.com> On Sun, 7 Aug 2016 03:16 am, Dave wrote: > I am trying to associate the .py file extension with idle.......where IS > idle? Can you make it a bit more difficult to load/use your software > please. We certainly could, but we're not going to make it more difficult just to suit you. You can make it as difficult as you want by just continuing your current behaviour of being unnecessarily obnoxious. If IDLE is installed, you should be able to use your general computer knowledge (assuming you have any) to find it. I don't use Windows and can't remember how you do this, but surely there is a way to find out the location of any application under the Start Menu? If there is a shortcut on the desktop, right click on the desktop and use that to locate IDLE. Or use Start Menu > Find File (or whatever Windows calls it) to find IDLE. But I think you may find something of a problem, since IDLE is itself a Python script. So perhaps you should worry less about where IDLE is located and more about how do you get Windows to open Python files in IDLE. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 6 23:42:50 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 13:42:50 +1000 Subject: Ned Batchelder: Loop Like A Native References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> Message-ID: <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> On Sun, 7 Aug 2016 08:05 am, Lawrence D?Oliveiro wrote: > On Saturday, August 6, 2016 at 12:08:30 PM UTC+12, bream... at gmail.com > wrote: >> A couple or three years old but this is well worth seeing for anybody, >> regardless of your Python expertise. >> http://nedbatchelder.com/text/iter.html > > A loop like > > for i in ... : > ... > if ... cond ... : > break > ... > #end for > > actually has two different ways to terminate. Is there any good reason for > them to be written two different ways? Yes. The two ways of ending the loop are distinct and different: - reach the end, and stop; - bail out early. When you read a book, there are two ways of stopping: - reach the end, and run out of pages to read, so you stop; - give up reading early, and just put the book away. (Or possibly throw the book across the room.) Why would you treat these two cases in the same way? interested = True for page in book: if interested: read(page) if bored_now(): interested = False finished = False while not finished: try: page = next(book) except StopIteration: finished = True else: read(page) if bored_now(): finished = True -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 6 23:47:07 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 13:47:07 +1000 Subject: where is it References: <1B29002BAB664D3DA868E8C6208AEFA8@DvePC> <57a6aba0$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a6af3c$0$1584$c3e8da3$5496439d@news.astraweb.com> On Sun, 7 Aug 2016 01:31 pm, Steven D'Aprano wrote: > If there is a shortcut on the desktop, right click on the desktop and use > that to locate IDLE. Sorry, I meant right-click on the SHORTCUT. > Or use Start Menu > Find File (or whatever Windows calls it) to find IDLE. > > But I think you may find something of a problem, since IDLE is itself a > Python script. So perhaps you should worry less about where IDLE is > located and more about how do you get Windows to open Python files in > IDLE. https://duckduckgo.com/html/?q=associate+python+with+idle should help. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Aug 7 00:02:35 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 14:02:35 +1000 Subject: How do I make a game in Python ? In-Reply-To: <13bed927-852f-539d-db87-37420484f203@gmail.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> <13bed927-852f-539d-db87-37420484f203@gmail.com> Message-ID: On Sun, Aug 7, 2016 at 1:14 PM, Michael Torrie wrote: > On 08/06/2016 03:51 PM, Cai Gengyang wrote: >> As in, any recommended websites that helps users create complex games in Python ? > > I imagine you create a complex game in Python the same way you'd do it > in just about any other language. Thus any website on game design would > be broadly applicable. And I'm sure there are a wide variety of topics > that are relevant. Logic, game play, artificial intelligence, user > interface, physics engines, and so forth. > > What you need to learn depends on what you already know and have > experience in. The programming language is really only a part of > developing "complex games." You'd probably want to start with simple, > even child-like games first. Asking how to program complex games is a > bit like asking how to build a car. Three of my students just recently put together a game (as their first-ever collaborative project, working on different parts of the project simultaneously), so I can tell you broadly how you would go about it: 1) SCOPE. This is incredibly important. With the student project, they had to have something demonstrable by Friday 5PM, so they had just five days to get something working. But even when you don't have a hard cut-off like that, you should scope back hard - brutally, even - so you can get something workable as early as possible. 2) Pin down exactly what the *point* of your game is. What is the fundamental thing you're trying to do? In their case, it was: Click on the red squares before they turn back to white, then watch for another square to turn red. As you develop, you'll mess around with everything else, but don't mess with that. 3) Build a very basic project, managed in source control (git/hg/bzr etc), and get it to the point of being testable. I don't mean unit tests (although if you're a fan of TDD, you might well go that route), but be able to fire up your game and actually run it. Commit that. 4) Progressively add features, constantly committing to source control. Whenever your game isn't runnable, STOP and debug it until it is. It's just way too hard to debug something that you can't even run. 5) Eventually, you'll get bored of the project or be forced to move on to something else. At that point, the game is done. :) ChrisA From rosuav at gmail.com Sun Aug 7 00:04:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 14:04:16 +1000 Subject: Me, myself and I [was Re: Python slang] In-Reply-To: <57a6a964$0$1588$c3e8da3$5496439d@news.astraweb.com> References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> <7b570ce3-2ce7-e970-6826-6fe66c2bb1e2@gmail.com> <57a6a964$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 1:22 PM, Steven D'Aprano wrote: > But using it in place of "me"? I don't think so. > > "The research was done by myself." > "He punched myself in the head." > > I wouldn't say it is *wrong*, just inelegant and rather pretentious. I'm sure there's good poetic reason for writing sentences like that. Maybe you want to sound like a nine hundred year old master of oogie-boogie 'Force' power? ChrisA From rosuav at gmail.com Sun Aug 7 00:05:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 14:05:32 +1000 Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 1:42 PM, Steven D'Aprano wrote: > When you read a book, there are two ways of stopping: > > - reach the end, and run out of pages to read, so you stop; > - give up reading early, and just put the book away. > > (Or possibly throw the book across the room.) > Or someone might rip the book out of your hands (unexpected exception). You don't need to handle that case specifically, but it will terminate the loop. ChrisA From breamoreboy at gmail.com Sun Aug 7 00:13:05 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 6 Aug 2016 21:13:05 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: <7c471965-5cc8-4198-ae96-a40af9faf72d@googlegroups.com> On Sunday, August 7, 2016 at 2:26:48 AM UTC+1, Ned Batchelder wrote: > On Saturday, August 6, 2016 at 9:21:24 PM UTC-4, Lawrence D?Oliveiro wrote: > > On Sunday, August 7, 2016 at 11:36:06 AM UTC+12, Ned Batchelder wrote: > > > > > Didn't we already do this debate? > > > > I understand. You want to discuss loops, just not *those* sorts of loops... > > I didn't post this link here. I'm merely pointing out that your concern > about multiple ways to exit loops sounds like exactly what was discussed > here two months ago. Is there any reason to think there is new material > to discuss? > > --Ned. Thank you Ned for being far more diplomatic than I would have been. My round. When can you get over https://bearbeerfamily.co.uk/the-saxon-bear/ ? :) Kindest regards. Mark Lawrence. From rustompmody at gmail.com Sun Aug 7 00:59:25 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 6 Aug 2016 21:59:25 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: On Friday, August 5, 2016 at 5:29:37 PM UTC+5:30, Matt Wheeler wrote: > On Fri, 5 Aug 2016, 02:23 Lawrence D?Oliveiro, wrote: > > > On Friday, August 5, 2016 at 12:06:23 PM UTC+12, Igor Korot wrote: > > > > > > On Thu, Aug 4, 2016 at 4:57 PM, Lawrence D?Oliveiro wrote: > > >> On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com > > wrote: > > >>> According to Python.org Mark Hammond has an Add-on (pywin32) that > > >>> supports Win32 and COM. > > >> > > >> Are people still using Win32? I thought Windows went 64-bit years ago. > > > > The API is still called Win32 (fun fact: on 64 bit windows > C:\Windows\System32 actually contains the 64 bit system (binaries & > libraries). The 32 bit equivalents live in a dir called SysWOW64 (stands > for Windows on Win64 or something) and get mapped to their regular > locations for any 32 bit processes. That's right, 64 bit libraries live in > System32 and 32 bits in SysWOW64 :) )* > > > This is the question about Win API calls.... > > > > > > Thank you. > > > > This is a Python group. > > > > And the question was about using the Windows API in Python. Not sure what > point you're trying to make here. > > > > You?re welcome. > > > > ... > > > * anyone from Microsoft listening and want to offer an explanation (don't > worry, I don't think anyone expects it to be good :D)? A two-word explanation: ?It?s Microsoft? To be fair my head spins in Linux-land trying to work out what all these 32's and 64's mean: mingw-w64-x86-64 From rosuav at gmail.com Sun Aug 7 02:10:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 16:10:24 +1000 Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: On Sun, Aug 7, 2016 at 2:59 PM, Rustom Mody wrote: > To be fair my head spins in Linux-land trying to work out what all these > 32's and 64's mean: mingw-w64-x86-64 There aren't any 32s in that. The beginning of that probably means the Win64 version of MinGW, which (IIRC) is distinctly different from the Win32 version (a project fork, or something); the end of it is the architecture, x86-64, meaning "Intel 64-bit". Sometimes that's described as AMD64, to distinguish it from IA-64, the Intel Itanium 64-bit architecture. The "86" in the name derives from 8086 and family. ChrisA From eryksun at gmail.com Sun Aug 7 02:23:49 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 7 Aug 2016 06:23:49 +0000 Subject: where is it In-Reply-To: <1B29002BAB664D3DA868E8C6208AEFA8@DvePC> References: <1B29002BAB664D3DA868E8C6208AEFA8@DvePC> Message-ID: On Sat, Aug 6, 2016 at 5:16 PM, Dave via Python-list wrote: > I am trying to associate the .py file extension with idle.......where IS idle? Can you > make it a bit more difficult to load/use your software please. You can run IDLE from the command-line as follows: 3.x: python[3][w] -m idlelib [script_path] 2.x: python[w] -m idlelib.idle [script_path] This should work on all supported platforms. If you omit the script path, it starts an interactive shell. On Windows, pythonw.exe (or pyw.exe) runs without an attached console. For Windows, the installer should have set up a right-click context-menu command to edit .py and .pyw files with IDLE. You can also use this command in a script via os.startfile. However, the way it's set up to create a cascaded menu is inconvenient, i.e. the command for IDLE 3.5 is "editwithidle\shell\edit35". You can copy this as a default "edit" command for .py and .pyw files. This isn't too hard to do in PowerShell. For example, if Python 3.5 is installed for all users (otherwise replace HKLM with HKCU): $base = "HKLM:\Software\Classes" foreach ($p in ((".py", "Python.File"), (".pyw", "Python.NoConFile"))) { $src = "$base\{0}\Shell\editwithidle\shell\edit35" -f $p[1] $dst = "$base\SystemFileAssociations\{0}\Shell" -f $p[0] # only set an "edit" command if the source key exists # and the default isn't already defined. if ((test-path $src) -and (-not (test-path $dst\edit))) { if (-not (test-path $dst)) { new-item -force $dst } copy-item -recurse $src $dst\edit } } The Windows shell looks in HKCR\SystemFileAssociations for default file associations. The user's selected file type (e.g. Python.File) can override this key by defining its own "edit" command. From steve+comp.lang.python at pearwood.info Sun Aug 7 02:54:35 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 07 Aug 2016 16:54:35 +1000 Subject: Awful code of the week Message-ID: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> Seen in the office IRC channel: (13:23:07) fred: near_limit = [] (13:23:07) fred: near_limit.append(1) (13:23:07) fred: near_limit = len(near_limit) (13:23:09) fred: WTF Speaks for itself. -- Steve From rosuav at gmail.com Sun Aug 7 04:04:18 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 7 Aug 2016 18:04:18 +1000 Subject: Awful code of the week In-Reply-To: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 4:54 PM, Steven D'Aprano wrote: > Seen in the office IRC channel: > > > (13:23:07) fred: near_limit = [] > (13:23:07) fred: near_limit.append(1) > (13:23:07) fred: near_limit = len(near_limit) > (13:23:09) fred: WTF > > > > Speaks for itself. The Real WTF is that it needs to have a type declaration. near_limit: Union[int,list] There, fixed. ChrisA From mail.python.org at marco.sulla.e4ward.com Sun Aug 7 04:38:07 2016 From: mail.python.org at marco.sulla.e4ward.com (Anders J. Munch) Date: Sun, 7 Aug 2016 10:38:07 +0200 Subject: Python slang In-Reply-To: References: Message-ID: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Marco Sulla via Python-list: > Well, they are the most used languages. They weren't when Python was created. Python's terms raise/except and self were normal for the time. C++ was the odd one out. throw/catch and this are Stroustrup's inventions, no other language used those terms. It was only later that language designers fell into the notion that it was crucial for a new language's success to look as much like C++ as possible. regards, Anders From andersjm at stofanet.dk Sun Aug 7 04:38:07 2016 From: andersjm at stofanet.dk (Anders J. Munch) Date: Sun, 7 Aug 2016 10:38:07 +0200 Subject: Python slang In-Reply-To: References: Message-ID: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Marco Sulla via Python-list: > Well, they are the most used languages. They weren't when Python was created. Python's terms raise/except and self were normal for the time. C++ was the odd one out. throw/catch and this are Stroustrup's inventions, no other language used those terms. It was only later that language designers fell into the notion that it was crucial for a new language's success to look as much like C++ as possible. regards, Anders From zutix at laposte.net Sun Aug 7 04:40:18 2016 From: zutix at laposte.net (zutix) Date: Sun, 7 Aug 2016 10:40:18 +0200 Subject: api-ms-win-crt-runtime-l1-1-0.dll missing ? Message-ID: <000a01d1f087$53a57aa0$faf06fe0$@laposte.net> I just have subscribed, so I send my message again as required. Sorry for not having pay attention. Thank you for your free product PYTHON I'd like to familiarize with, if possible. Environment: Windows 8.1 proc x64 I installed the version from the buton. After a succesfull installation, a system error occurs on the lounch of the program. I desinstalled this version from my PC. I installed the version for windows 64. After a succesfull installation, the same system error occurs on the lounch of the program: api-ms-win-crt-runtime-l1-1-0.dll is missing. Please would you let me know where I can get safely this .dll giving me the exact address because I'm a beginner ? Thank you. Best regards --- L'absence de virus dans ce courrier ?lectronique a ?t? v?rifi?e par le logiciel antivirus Avast. https://www.avast.com/antivirus From zutix at laposte.net Sun Aug 7 10:09:47 2016 From: zutix at laposte.net (zutix) Date: Sun, 7 Aug 2016 16:09:47 +0200 Subject: api-ms-win-crt-runtime-l1-1-0.dll missing ? Message-ID: <000401d1f0b5$5b108090$113181b0$@laposte.net> Thank you for your free product PYTHON I'd like to familiarize with, if possible. Environment: Windows 8.1 proc x64 I installed the version from the buton. After a succesfull installation, a system error occurs on the lounch of the program. I desinstalled this version from my PC. I installed the version for windows 64. After a succesfull installation, the same system error occurs on the lounch of the program: api-ms-win-crt-runtime-l1-1-0.dll is missing. Please would you let me know where I can get safely this .dll giving me the exact address because I'm a beginner ? Thank you. Best regards --- L'absence de virus dans ce courrier ?lectronique a ?t? v?rifi?e par le logiciel antivirus Avast. https://www.avast.com/antivirus From zutix at laposte.net Sun Aug 7 10:15:55 2016 From: zutix at laposte.net (zutix) Date: Sun, 7 Aug 2016 16:15:55 +0200 Subject: python 3.5.2 lounch: api-ms-win-crt-runtime-l1-1-0.dll is missing ? Message-ID: <000a01d1f0b6$361a9180$a24fb480$@laposte.net> Thank you for your free product PYTHON I'd like to familiarize with, if possible. Environment: Windows 8.1 proc x64 I installed the version from the buton. After a succesfull installation, a system error occurs on the lounch of the program. I desinstalled this version from my PC. I installed the version for windows 64. After a succesfull installation, the same system error occurs on the lounch of the program: api-ms-win-crt-runtime-l1-1-0.dll is missing. Please would you let me know where I can get safely this < api-ms-win-crt-runtime-l1-1-0.dll >. Please give me the exact address because I'm a beginner ? Thank you. Best regards --- L'absence de virus dans ce courrier ?lectronique a ?t? v?rifi?e par le logiciel antivirus Avast. https://www.avast.com/antivirus From gengyangcai at gmail.com Sun Aug 7 10:47:36 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sun, 7 Aug 2016 07:47:36 -0700 (PDT) Subject: How do I make a game in Python ? In-Reply-To: References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> <13bed927-852f-539d-db87-37420484f203@gmail.com> Message-ID: <6c92e21c-e8fc-487e-9048-dbd31904113a@googlegroups.com> Games are great. I guess I would like to invent animated games that can teach students how to solve mathematical, physics, engineering , Go and programming puzzles, basic financial literacy and investing techniques through interesting and enriching games and puzzles and university admissions interviews ... Create a piece of software like this, get users to test it ... then when the feedback streams in, iterate the product accordingly based on user feedback ... On Sunday, August 7, 2016 at 12:02:56 PM UTC+8, Chris Angelico wrote: > On Sun, Aug 7, 2016 at 1:14 PM, Michael Torrie wrote: > > On 08/06/2016 03:51 PM, Cai Gengyang wrote: > >> As in, any recommended websites that helps users create complex games in Python ? > > > > I imagine you create a complex game in Python the same way you'd do it > > in just about any other language. Thus any website on game design would > > be broadly applicable. And I'm sure there are a wide variety of topics > > that are relevant. Logic, game play, artificial intelligence, user > > interface, physics engines, and so forth. > > > > What you need to learn depends on what you already know and have > > experience in. The programming language is really only a part of > > developing "complex games." You'd probably want to start with simple, > > even child-like games first. Asking how to program complex games is a > > bit like asking how to build a car. > > Three of my students just recently put together a game (as their > first-ever collaborative project, working on different parts of the > project simultaneously), so I can tell you broadly how you would go > about it: > > 1) SCOPE. This is incredibly important. With the student project, they > had to have something demonstrable by Friday 5PM, so they had just > five days to get something working. But even when you don't have a > hard cut-off like that, you should scope back hard - brutally, even - > so you can get something workable as early as possible. > > 2) Pin down exactly what the *point* of your game is. What is the > fundamental thing you're trying to do? In their case, it was: Click on > the red squares before they turn back to white, then watch for another > square to turn red. As you develop, you'll mess around with everything > else, but don't mess with that. > > 3) Build a very basic project, managed in source control (git/hg/bzr > etc), and get it to the point of being testable. I don't mean unit > tests (although if you're a fan of TDD, you might well go that route), > but be able to fire up your game and actually run it. Commit that. > > 4) Progressively add features, constantly committing to source > control. Whenever your game isn't runnable, STOP and debug it until it > is. It's just way too hard to debug something that you can't even run. > > 5) Eventually, you'll get bored of the project or be forced to move on > to something else. At that point, the game is done. :) > > ChrisA From ernest.bonat at gmail.com Sun Aug 7 11:03:13 2016 From: ernest.bonat at gmail.com (Ernest Bonat, Ph.D.) Date: Sun, 7 Aug 2016 08:03:13 -0700 Subject: Error running the exe file in Windows "Failed to execute script pyi_rth_pkgres" Message-ID: Hi, I have created a simple Python program including the following packages: import numpy as np import matplotlib.pyplot as plt import pandas as pd if __name__ == '__main__': print("Hi!") I have created the installation package using PyInstaller with the following command line: pyinstaller --onedir --name=appname --windowed "C:\appname.py" The required folders (build and dist) and the spec file are created properly with no errors. When I run the appname.exe file I got the error "Failed to execute script pyi_rth_pkgres". I'm using the following now: - Anaconda3 4.1.1.64-bit - Python 3.5.2 - PyInstaller 3.2 - Windows 10 64-bit Any help is very appreciated. Feel free to contact me at any time you need. Thank you, Ernest Bonat, Ph.D. Senior Software Engineer Senior Data Scientist From Bernd.Nawothnig at t-online.de Sun Aug 7 13:43:47 2016 From: Bernd.Nawothnig at t-online.de (Bernd Nawothnig) Date: Sun, 7 Aug 2016 19:43:47 +0200 Subject: Python slang References: <57a539ed$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-06, Chris Angelico wrote: > On Sun, Aug 7, 2016 at 5:37 AM, Bernd Nawothnig > wrote: >>> But SQL's NULL is a cross between C's NULL, IEEE's NaN, Cthulhu, and >>> Emrakul. >> >> SQL NULL has the semantic of "unknown". So if one or both operands of >> a comparison (or any other operation) are unknown the result is >> unknown too. And that means NULL. > > That's not entirely accurate, and it doesn't explain why NULL > sometimes behaves as if it's a genuine value, and sometimes as if it's > completely not there. For instance, taking the average of a column of > values ignores NULLs completely, and COUNT(column) is the same as > COUNT(column) WHERE column IS NOT NULL; but in some situations it > behaves more like NaN: > > rosuav=> select null or true, null or false, null and true, null and false; > ?column? | ?column? | ?column? | ?column? > ----------+----------+----------+---------- > t | NULL | NULL | f > (1 row) > > Anything "or true" has to be true, so NULL OR TRUE is true. And then > there are the times when NULL acts like a completely special value, > for instance in a foreign key - it means "there isn't anything on the > other end of this relationship", and is perfectly legal. Or in a > SELECT DISTINCT, where NULL behaves just like any other value - if > there are any NULL values in the column, you get back exactly one NULL > in the result. Thanks for that additions and corrections. Bernd -- no time toulouse From mangafasxone at hotmail.com Sun Aug 7 14:37:33 2016 From: mangafasxone at hotmail.com (Panayiotis Mangafas) Date: Sun, 7 Aug 2016 18:37:33 +0000 Subject: Cant download python libraries. Message-ID: So i've downloaded python on a new laptop(Windows) and apparently i downloaded it in a wrong folder and now when i try to install a library i cant. I have tried both pip install and install setup.py but i cant make it work. It doesn't work if i just try to access python from cmd. I've tried to uninstall it but it still here. How can i fix that? From torriem at gmail.com Sun Aug 7 16:02:39 2016 From: torriem at gmail.com (Michael Torrie) Date: Sun, 7 Aug 2016 14:02:39 -0600 Subject: How do I make a game in Python ? In-Reply-To: <6c92e21c-e8fc-487e-9048-dbd31904113a@googlegroups.com> References: <71000cc9-379b-4365-9242-55a86fdf51f5@googlegroups.com> <948e3f50-50fc-4b1c-930e-71a0f3fb8d14@googlegroups.com> <13bed927-852f-539d-db87-37420484f203@gmail.com> <6c92e21c-e8fc-487e-9048-dbd31904113a@googlegroups.com> Message-ID: <30e25e7d-4433-7728-dd6d-0fb77cd3958b@gmail.com> On 08/07/2016 08:47 AM, Cai Gengyang wrote: > Games are great. I guess I would like to invent animated games that > can teach students how to solve mathematical, physics, engineering , > Go and programming puzzles, basic financial literacy and investing > techniques through interesting and enriching games and puzzles and > university admissions interviews ... Create a piece of software like > this, get users to test it ... then when the feedback streams in, > iterate the product accordingly based on user feedback ... I'm sure you could probably google for some resources about Python game development. Google shows this link, for example, though I cannot vouch for whether or not it would be helpful: https://inventwithpython.com/chapters/ It claims to be a book that leads you through some game development using Python. Bear in mind that must start simple and work your way into more complex things. If you think you can pick up Python and develop a "complex game" in short order, you'll be very disappointed. It's along process. Even what you described could take years to fully develop. But if you have patience and an aptitude for learning you can without a doubt accomplish it. I think text-only number games are a great place to start honing your skills. From timothy.c.delaney at gmail.com Sun Aug 7 18:03:08 2016 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 8 Aug 2016 08:03:08 +1000 Subject: Awful code of the week In-Reply-To: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> Message-ID: On 7 August 2016 at 16:54, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > Seen in the office IRC channel: > > > (13:23:07) fred: near_limit = [] > (13:23:07) fred: near_limit.append(1) > (13:23:07) fred: near_limit = len(near_limit) > (13:23:09) fred: WTF > Assuming you'e not talking about the semantics of this specific code (which could be reduced to near_limit = 1), I have to say I've been guilty of reusing a name in precisely this way - set up the structure using the name, then set the name to a calculated value. I would only do this when I'm totally uninterested in the structure itself - it's purely a way to get to the final result, but getting to that final result requires several steps that justifies an intermediate name binding. It avoids namespace pollution (but of course that can be fixed with del) and avoids having to think up another name for a very temporary structure (names are hard). And I would include a comment explaining the reuse of the name. The alternative would be something like (replace first line by something complex ...): near_limit_list = [1] near_limit = len(near_limit_list) del near_limit_list Tim Delaney From lawrencedo99 at gmail.com Sun Aug 7 18:45:50 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 7 Aug 2016 15:45:50 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <0f62b4d0-9d27-44d2-b7b2-bdb44f5a71bb@googlegroups.com> Message-ID: On Sunday, August 7, 2016 at 4:59:39 PM UTC+12, Rustom Mody wrote: > To be fair my head spins in Linux-land trying to work out what all these > 32's and 64's mean: mingw-w64-x86-64 The package descriptions tell you: mingw-w64 - Development environment targeting 32- and 64-bit Windows mingw-w64-common - Common files for Mingw-w64 mingw-w64-i686-dev - Development files for MinGW-w64 targeting Win32 mingw-w64-tools - Development tools for 32- and 64-bit Windows mingw-w64-x86-64-dev - Development files for MinGW-w64 targeting Win64 From lawrencedo99 at gmail.com Sun Aug 7 18:52:30 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 7 Aug 2016 15:52:30 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: On Sunday, August 7, 2016 at 1:26:48 PM UTC+12, Ned Batchelder wrote: > I'm merely pointing out that your concern about multiple ways to exit loops > sounds like exactly what was discussed here two months ago. And one could point out that your presentation on Python loops sounds exactly like every other discussion on Python loops. > Is there any reason to think there is new material to discuss? There are always new things to learn, even about such basic things as loops. Though some don?t seem to like it when I point this out... From ned at nedbatchelder.com Sun Aug 7 19:11:29 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 7 Aug 2016 16:11:29 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <0db02c49-c77d-4e55-bfb4-263b02ceac28@googlegroups.com> Message-ID: <6e4e39cc-9927-4d20-ac71-3f185577752b@googlegroups.com> On Sunday, August 7, 2016 at 6:52:45 PM UTC-4, Lawrence D?Oliveiro wrote: > On Sunday, August 7, 2016 at 1:26:48 PM UTC+12, Ned Batchelder wrote: > > I'm merely pointing out that your concern about multiple ways to exit loops > > sounds like exactly what was discussed here two months ago. > > And one could point out that your presentation on Python loops sounds exactly like every other discussion on Python loops. If you feel like my presentation was worthless, that would certainly be something we could discuss here, respectfully. I hope you don't actually feel that way. If Mark had posted this link in June, and then again now, I think it would be reasonable for someone to say, "This was just posted in June, why repost it now?" > > > Is there any reason to think there is new material to discuss? > > There are always new things to learn, even about such basic things as loops. Though some don?t seem to like it when I point this out... We had a long thread about your point only two months ago. Is there something new to add to it? I thought you made your point then, and had a long discussion about it. You'll have to decide if you think there are new points to be made. This list has had difficulty in the past with people who try to start the same topics over and over again, because people didn't agree with them the last three (or ten or twenty) times they brought it up. It's unpleasant and noisy, and pollutes the tone of other conversations. I don't mean to put you into that category. If you have new ideas about loop termination that didn't get an airing in June, I'm sure people will be interested to discuss them. --Ned. From rantingrickjohnson at gmail.com Sun Aug 7 19:19:00 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 7 Aug 2016 16:19:00 -0700 (PDT) Subject: Ned Batchelder: Loop Like A Native In-Reply-To: <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Saturday, August 6, 2016 at 10:43:01 PM UTC-5, Steven D'Aprano wrote: > Yes. The two ways of ending the loop are distinct and different: > > - reach the end, and stop; > - bail out early. > > > When you read a book, there are two ways of stopping: > > - reach the end, and run out of pages to read, so you stop; > - give up reading early, and just put the book away. > > (Or possibly throw the book across the room.) > > > Why would you treat these two cases in the same way? > > interested = True > for page in book: > if interested: > read(page) > if bored_now(): > interested = False This algorithm does not exit early when `interested` becomes false, instead, it iterates every page of the book regardless of the value of `interested`. Was this intentional, or merely a mistake? > finished = False > while not finished: > try: > page = next(book) > except StopIteration: > finished = True > else: > read(page) > if bored_now(): > finished = True That kind of code, whilst being "somewhat" idiomatic python, is horrific. If the intent is to iterate the pages of a book until the end is reached *OR* until the "reader's interest wanes", then a simple for-loop will suffice. for page in book: isReaderInterested = reader.read_page(page) if not isReaderInterested: break Simple is better than complex. From rantingrickjohnson at gmail.com Sun Aug 7 19:40:17 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 7 Aug 2016 16:40:17 -0700 (PDT) Subject: make an object read only In-Reply-To: References: Message-ID: <9d057783-fbeb-437c-b357-da5a84c98aeb@googlegroups.com> On Thursday, August 4, 2016 at 7:03:32 PM UTC-5, Lawrence D?Oliveiro wrote: > As GvR has said: ?we?re all consenting adults here?. But as we've learned from animal farm, some are more consenting than others. From rantingrickjohnson at gmail.com Sun Aug 7 19:45:57 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 7 Aug 2016 16:45:57 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: <0bd82d70-4522-4e26-89b5-7f742c667f0d@googlegroups.com> On Thursday, August 4, 2016 at 6:58:05 PM UTC-5, Lawrence D?Oliveiro wrote: > On Friday, August 5, 2016 at 11:50:28 AM UTC+12, jj0ge... at gmail.com wrote: > > According to Python.org Mark Hammond has an Add-on (pywin32) that supports > > Win32 and COM. > > Are people still using Win32? I thought Windows went 64-bit years ago. Windows, like python after it, may have incremented it's software, but not all the little lemmings are just blindly flinging themselves off the cliff after it. Win32 boxes will be around for a long time. From rantingrickjohnson at gmail.com Sun Aug 7 19:48:19 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 7 Aug 2016 16:48:19 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: <14037d2d-e76e-4d19-997f-097b93eddad1@googlegroups.com> On Thursday, August 4, 2016 at 6:50:28 PM UTC-5, jj0ge... at gmail.com wrote: > According to Python.org Mark Hammond has an Add-on > (pywin32) that supports Win32 and COM. Does anyone know if > the Add-on covers all Win32 API functions, and if not is > there a list of the subset of Win32 API functions it does > include? Of course, it's called the "PyWin32 docs". From steve+python at pearwood.info Sun Aug 7 21:26:28 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 08 Aug 2016 11:26:28 +1000 Subject: Awful code of the week References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> Message-ID: <57a7dfc4$0$1612$c3e8da3$5496439d@news.astraweb.com> On Mon, 8 Aug 2016 08:03 am, Tim Delaney wrote: > On 7 August 2016 at 16:54, Steven D'Aprano < > steve+comp.lang.python at pearwood.info> wrote: > >> Seen in the office IRC channel: >> >> >> (13:23:07) fred: near_limit = [] >> (13:23:07) fred: near_limit.append(1) >> (13:23:07) fred: near_limit = len(near_limit) >> (13:23:09) fred: WTF >> > > Assuming you'e not talking about the semantics of this specific code > (which could be reduced to near_limit = 1), That's exactly it. They're three consecutive lines of code. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Aug 7 21:46:48 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 08 Aug 2016 11:46:48 +1000 Subject: Ned Batchelder: Loop Like A Native References: <56d98798-d10c-4eeb-bb6e-ffa3d5f47368@googlegroups.com> <59762de1-98b0-4371-a8e2-8ae60ea25dd5@googlegroups.com> <57a6ae3b$0$1584$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a7e489$0$1618$c3e8da3$5496439d@news.astraweb.com> On Mon, 8 Aug 2016 09:19 am, Rick Johnson wrote: > On Saturday, August 6, 2016 at 10:43:01 PM UTC-5, Steven D'Aprano wrote: > >> Yes. The two ways of ending the loop are distinct and different: >> >> - reach the end, and stop; >> - bail out early. >> >> >> When you read a book, there are two ways of stopping: >> >> - reach the end, and run out of pages to read, so you stop; >> - give up reading early, and just put the book away. >> >> (Or possibly throw the book across the room.) >> >> >> Why would you treat these two cases in the same way? >> >> interested = True >> for page in book: >> if interested: >> read(page) >> if bored_now(): >> interested = False > > This algorithm does not exit early when `interested` becomes false, > instead, it iterates every page of the book regardless of the value of > `interested`. Was this intentional, or merely a mistake? I said: "Why would you treat these two cases in the same way?", then showed a code snippet which treats the two cases in the same way (i.e. iterates over every page of the book). Of course it was intentional. > >> finished = False >> while not finished: >> try: >> page = next(book) >> except StopIteration: >> finished = True >> else: >> read(page) >> if bored_now(): >> finished = True > > That kind of code, whilst being "somewhat" idiomatic python, is horrific. Of course it is horrific. And I don't think it is even the slightest bit Pythonic. There may (very unlikely, but theoretically possible) be occasions where you have no choice but to write code like this, but if so you wouldn't describe it as idiomatic Python code. > If the intent is to iterate the pages of a book until the end is reached > *OR* until the "reader's interest wanes", then a simple for-loop will > suffice. That's exactly the sort of thing that Lawrence is objecting to, because (and I quote the part of my email you deleted): "A loop like [example with break] actually has two different ways to terminate. Is there any good reason for them to be written two different ways?" > for page in book: > isReaderInterested = reader.read_page(page) > if not isReaderInterested: > break > > Simple is better than complex. Precisely. I'm showing the horrible code you have to write to avoid writing two different ways of terminating a loop with two fundamentally different ways of terminating. Lawrence, if you're still reading, I hope I am not misrepresenting your position here. If you think that there is an alternative way of writing this code which avoids two exits from the loop but without having to write such unidiomatic (if not idiotic) code, please tell us what you would do. There is one clean way for doing this: use itertools. def interesting(page): """Return True if the page is interesting, False if boring.""" for page in itertools.takewhile(interesting, book): reader.read_page(page) but it's not always easy to re-write the code to match that idiom. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Aug 7 22:11:31 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 08 Aug 2016 12:11:31 +1000 Subject: Running Python from the source repo Message-ID: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> I have cloned the Python source repo, and build CPython, as described here: https://docs.python.org/devguide/ Now a little bit later, I want to update the repo, so I run: hg fetch to get and apply any changes. How do I know if I need to rebuild Python? I don't want to have to rebuild after every fetch, because that's quite time consuming (I estimate about five minutes on my machine, just long enough to be a distraction but not long enough to get into something else). Plus the time to run the tests (significantly longer). What do others do? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Aug 7 22:32:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Aug 2016 12:32:23 +1000 Subject: Running Python from the source repo In-Reply-To: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 8, 2016 at 12:11 PM, Steven D'Aprano wrote: > I have cloned the Python source repo, and build CPython, as described here: > > https://docs.python.org/devguide/ > > > Now a little bit later, I want to update the repo, so I run: > > hg fetch > > to get and apply any changes. How do I know if I need to rebuild Python? I > don't want to have to rebuild after every fetch, because that's quite time > consuming (I estimate about five minutes on my machine, just long enough to > be a distraction but not long enough to get into something else). Plus the > time to run the tests (significantly longer). > > What do others do? I use 'hg pull -u' - that's the same as 'hg fetch' right? After that, I just 'make'. It won't rebuild everything, only what's changed. If you 'make' and immediately 'make' again, it's pretty quick - at least, it is on my Linux box, where it's cheap to 'stat' a bunch of files to see if they've changed. On other systems, it might be a bit slower, but it certainly won't take as long as a full build from scratch. The only time a full rebuild is needed is if something really fundamental has changed. I just pulled now, and the first build took 1m39s, which involved a reconfigure and a good bit of chugging; then a rebuild looked like this: rosuav at sikorsky:~/cpython$ time make running build running build_ext running build_scripts copying and adjusting /home/rosuav/cpython/Tools/scripts/pydoc3 -> build/scripts-3.6 copying and adjusting /home/rosuav/cpython/Tools/scripts/idle3 -> build/scripts-3.6 copying and adjusting /home/rosuav/cpython/Tools/scripts/2to3 -> build/scripts-3.6 copying and adjusting /home/rosuav/cpython/Tools/scripts/pyvenv -> build/scripts-3.6 changing mode of build/scripts-3.6/pydoc3 from 644 to 755 changing mode of build/scripts-3.6/idle3 from 644 to 755 changing mode of build/scripts-3.6/2to3 from 644 to 755 changing mode of build/scripts-3.6/pyvenv from 644 to 755 renaming build/scripts-3.6/pydoc3 to build/scripts-3.6/pydoc3.6 renaming build/scripts-3.6/idle3 to build/scripts-3.6/idle3.6 renaming build/scripts-3.6/2to3 to build/scripts-3.6/2to3-3.6 renaming build/scripts-3.6/pyvenv to build/scripts-3.6/pyvenv-3.6 real 0m0.148s user 0m0.108s sys 0m0.028s rosuav at sikorsky:~/cpython$ So normally just run 'make' and let it do its work. ChrisA From Joaquin.Alzola at lebara.com Mon Aug 8 05:08:11 2016 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Mon, 8 Aug 2016 09:08:11 +0000 Subject: Python and 64bit Windows7 In-Reply-To: References: Message-ID: >I am having problems when installing Python on a 64bit windows7 laptop. >I am guessing that the problem is happening because of the 64bit architecture. >Could you please take a look at the attached file that has screen shots of the problem I am facing? >Could you tell me how I can fix these issues and get Python and PyCharm to work? Mailing list doesn't allow attachments so please post you error through writing it into text. This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From rosuav at gmail.com Mon Aug 8 05:09:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Aug 2016 19:09:00 +1000 Subject: Python and 64bit Windows7 In-Reply-To: References: Message-ID: On Sun, Aug 7, 2016 at 9:23 AM, James Santiago wrote: > Could you please take a look at the attached file that has screen shots of > the problem I am facing? No, because this is a text-only mailing list - your attachment got stripped. Can you transcribe the text of the error(s), please? ChrisA From Joaquin.Alzola at lebara.com Mon Aug 8 05:13:15 2016 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Mon, 8 Aug 2016 09:13:15 +0000 Subject: python 3.5.2 lounch: api-ms-win-crt-runtime-l1-1-0.dll is missing ? In-Reply-To: <000a01d1f0b6$361a9180$a24fb480$@laposte.net> References: <000a01d1f0b6$361a9180$a24fb480$@laposte.net> Message-ID: >I installed the version for windows 64. After a succesfull installation, the same system error occurs on the lounch of the program: >api-ms-win-crt-runtime-l1-1-0.dll is missing. Sometimes google save you time more than the list: https://www.smartftp.com/support/kb/the-program-cant-start-because-api-ms-win-crt-runtime-l1-1-0dll-is-missing-f2702.html I think from there you can manage This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From breamoreboy at yahoo.co.uk Mon Aug 8 06:37:53 2016 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 8 Aug 2016 11:37:53 +0100 Subject: Network protocols, sans I/O,(Hopefully) the future of network protocols in Python Message-ID: This may be of interest to some of you http://www.snarky.ca/network-protocols-sans-i-o -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From aaryanreviews at gmail.com Mon Aug 8 08:20:52 2016 From: aaryanreviews at gmail.com (aaryanreviews at gmail.com) Date: Mon, 8 Aug 2016 05:20:52 -0700 (PDT) Subject: Make sure you removed all debugging print statements error Message-ID: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Hello guys! I was answering a question on a piece of homework of mine. Sadly I can't answer it correctly due to the repetitive error being "Make sure you removed all debugging print statements." Hopefully one of you guys can help me solve this and also make me understand why I keep on getting this error. Piece of code: class Person(object): def __init__(self, name): self.name = name try: firstBlank = name.rindex(' ') self.lastName = name[firstBlank+1:] except: self.lastName = name self.age = None def getLastName(self): return self.lastName def setAge(self, age): self.age = age def getAge(self): if self.age == None: raise ValueError return self.age def __lt__(self, other): if self.lastName == other.lastName: return self.name < other.name return self.lastName < other.lastName def __str__(self): return self.name class USResident(Person): def __init__(self, name, status): self.name = name self.status = status def getStatus(self): return self.status # def getName(self): # return self.name a = USResident('Tim Beaver', 'citizen') print a.getStatus() # print(a.getName()) b = USResident('Tim Horton', 'non-resident') print b.getStatus() From jamesfriendsfamily at gmail.com Mon Aug 8 08:44:54 2016 From: jamesfriendsfamily at gmail.com (James Santiago) Date: Mon, 8 Aug 2016 05:44:54 -0700 Subject: Python and 64bit Windows7 In-Reply-To: References: Message-ID: Hello: Thank you for your reply. Here are the steps I took, and the error messages I get: I have a windows7 laptop with 64bit intel i3. 25Gb hard-drive space left, 5Gb usable RAM space. (at the time of installing Python) 1) on the first attempt I chose the default choice in installer. Installation did not give any errors whatsoever. I then installed PyCharm - again with default choice and successful. When I open PyCharm, it did not locate the Interpreter automatically. When I did guide it to the correct path for Python location, a Python window (labeled Modify Setup) opens with three choices: Modify, Repair, Uninstall. I tried Repair in one try (& repeated it couple of times later), and Modify choice in a later attempt. However every time I open a new Project in PyCharm I could not get past the Modify Setup window. 2) I checked 'Uninstall or Change a Program' window of Windows7 and confirmed Python 3.5.2(32-bit) and Python Launcher installed. I also went to the correct location in C:/Program Files (x86)/ directory and confirmed Python being present. 3) I uninstalled Python and reinstalled. This time I went with customized installation choice. I installed it in C:/Python/. Installation was successful by itself. However same error when I open PyCharm and try to put the actual file location in the Interpreter field of PyCharm. 4) I also went to DOS prompt or CMD screen, went to the directory where Python was located as follows: C:\Python>python exe -v. This returns several lines of information. For example: the first line is import _frozen_importlib # frozen. second line is import _imp # builtin import sys # builtin There are some 15-20 lines like this which start with the term 'import'. There are a few lines that start like this: # C:\Python\lib\__pycache__\codecs.cpython-35.pyc matches C:\Python\lib\codecs.py Thank you in advance for your help. Regards James On Mon, Aug 8, 2016 at 2:08 AM, Joaquin Alzola wrote: > >I am having problems when installing Python on a 64bit windows7 laptop. > >I am guessing that the problem is happening because of the 64bit > architecture. > >Could you please take a look at the attached file that has screen shots > of the problem I am facing? > >Could you tell me how I can fix these issues and get Python and PyCharm > to work? > > Mailing list doesn't allow attachments so please post you error through > writing it into text. > This email is confidential and may be subject to privilege. If you are not > the intended recipient, please do not copy or disclose its content but > contact the sender immediately upon receipt. > From steve+python at pearwood.info Mon Aug 8 08:50:42 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 08 Aug 2016 22:50:42 +1000 Subject: Cant download python libraries. References: Message-ID: <57a88024$0$1611$c3e8da3$5496439d@news.astraweb.com> On Mon, 8 Aug 2016 04:37 am, Panayiotis Mangafas wrote: > So i've downloaded python on a new laptop(Windows) and apparently i > downloaded it in a wrong folder and now when i try to install a library i > cant. I have tried both pip install and install setup.py but i cant make > it work. It doesn't work if i just try to access python from cmd. I've > tried to uninstall it but it still here. How can i fix that? You haven't given us enough information to answer your question. What do you mean, "it doesn't work"? Do you get an error message? Please copy and paste (NO SCREENSHOTS!) exactly what commands you type, and what errors you get. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Aug 8 08:52:29 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 08 Aug 2016 22:52:29 +1000 Subject: Make sure you removed all debugging print statements error References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Message-ID: <57a8808f$0$1611$c3e8da3$5496439d@news.astraweb.com> On Mon, 8 Aug 2016 10:20 pm, aaryanreviews at gmail.com wrote: > Hello guys! I was answering a question on a piece of homework of mine. > Sadly I can't answer it correctly due to the repetitive error being "Make > sure you removed all debugging print statements." Hopefully one of you > guys can help me solve this and also make me understand why I keep on > getting this error. You haven't given us enough information to answer this question. What error do you get? What version of Python are you running? How are you running it? Please copy and paste (NO SCREENSHOTS!) the exact commands you use to run your code, and the actual error you get. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nir.yakovi1 at gmail.com Mon Aug 8 10:33:09 2016 From: nir.yakovi1 at gmail.com (nir.yakovi1 at gmail.com) Date: Mon, 8 Aug 2016 07:33:09 -0700 (PDT) Subject: A bioinformatics module similar to DEseq in R Message-ID: <1a1cde0d-66dc-48be-9b58-17e19155457f@googlegroups.com> Hi! is there a Python module equivalent\as similar as possible to 'DEseq' in R? It's just that my team's (many) scripts use it, and to start modifying them all to support some different bioinformatics module would be a nightmare. Thanks! From auriocus at gmx.de Mon Aug 8 11:26:42 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 8 Aug 2016 17:26:42 +0200 Subject: Make sure you removed all debugging print statements error In-Reply-To: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Message-ID: Am 08.08.16 um 14:20 schrieb aaryanreviews at gmail.com:> > Hello guys! I was answering a question on a piece of homework of > mine. Sadly I can't answer it correctly due to the repetitive error > being "Make sure you removed all debugging print statements." > Hopefully one of you guys can help me solve this and also make me > understand why I keep on getting this error. This seems to be specific to the system where you hand in the solutions; here we can only guess. Does it help to comment out all print staments in your code? I can only see 2 print statements at the end of the script. Christian From eryksun at gmail.com Mon Aug 8 12:11:12 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 8 Aug 2016 16:11:12 +0000 Subject: api-ms-win-crt-runtime-l1-1-0.dll missing ? In-Reply-To: <000a01d1f087$53a57aa0$faf06fe0$@laposte.net> References: <000a01d1f087$53a57aa0$faf06fe0$@laposte.net> Message-ID: On Sun, Aug 7, 2016 at 8:40 AM, zutix via Python-list wrote: > I installed the version for windows 64. After a succesfull installation, the > same system error occurs on the lounch of the program: > api-ms-win-crt-runtime-l1-1-0.dll is missing. > > Please would you let me know where I can get safely this .dll giving me the > exact address because I'm a beginner ? Thank you. The Universal CRT update [1] should be applied by the installer, but shouldn't even be required on an up-to-date system. Did you disable Windows Update? [1]: https://support.microsoft.com/en-us/kb/2999226 From rosuav at gmail.com Mon Aug 8 12:20:20 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Aug 2016 02:20:20 +1000 Subject: Make sure you removed all debugging print statements error In-Reply-To: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Message-ID: On Mon, Aug 8, 2016 at 10:20 PM, wrote: > class Person(object): > def __init__(self, name): > self.name = name > try: > firstBlank = name.rindex(' ') > self.lastName = name[firstBlank+1:] > except: > self.lastName = name > self.age = None Please don't EVER do this kind of thing. EVER. Firstly, the bare "except:" clause should basically never be used; instead, catch a very specific exception and figure out what you want to do here - but secondly, don't force people's names to be subdivided. https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ Even if you *must* split the name like this, there are far more Pythonic ways to do so. But I won't tell you about them, because you shouldn't even use a concept of "lastName". > def getLastName(self): > return self.lastName Python is not Java. You don't need a public attribute "self.lastName" and a getter "self.getLastName()". > def setAge(self, age): > self.age = age > def getAge(self): > if self.age == None: > raise ValueError > return self.age Same again - you don't need getters and setters. Please don't write code like this in Python. > def __str__(self): > return self.name Hmm, I'm not sure this is a good idea - you're treating a person as equivalent to his/her name. Normally you'd define the repr and/or str of a class to show exactly what thing this is; it'll make debugging easier. ChrisA From torriem at gmail.com Mon Aug 8 12:23:39 2016 From: torriem at gmail.com (Michael Torrie) Date: Mon, 8 Aug 2016 10:23:39 -0600 Subject: Make sure you removed all debugging print statements error In-Reply-To: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Message-ID: <01a19067-ca43-4630-1b4e-8f73a7bcbcf2@gmail.com> On 08/08/2016 06:20 AM, aaryanreviews at gmail.com wrote: > Hello guys! I was answering a question on a piece of homework of > mine. Sadly I can't answer it correctly due to the repetitive error > being "Make sure you removed all debugging print statements." > Hopefully one of you guys can help me solve this and also make me > understand why I keep on getting this error. Sounds like there's an automatic piece of software running your code and comparing your output with the expected output. Your code must be giving different output than expected by your teacher. The message is simply a reminder that if you had any debugging print()'s in your code that you should remove them before turning in your program. Since you don't have any debug print()'s that I can see, the problem must be that your program is simply outputing something different than expected. What output is expected? What does the assignment say about it? From zachary.ware+pylist at gmail.com Mon Aug 8 12:24:42 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 8 Aug 2016 11:24:42 -0500 Subject: Running Python from the source repo In-Reply-To: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 7, 2016 at 9:11 PM, Steven D'Aprano wrote: > I have cloned the Python source repo, and build CPython, as described here: > > https://docs.python.org/devguide/ > > > Now a little bit later, I want to update the repo, so I run: > > hg fetch According to the hg docs [1], you should probably avoid 'hg fetch'. Use 'hg pull -u' (or 'hg pull && hg update') instead. > to get and apply any changes. How do I know if I need to rebuild Python? I > don't want to have to rebuild after every fetch, because that's quite time > consuming (I estimate about five minutes on my machine, just long enough to > be a distraction but not long enough to get into something else). Plus the > time to run the tests (significantly longer). > > What do others do? I generally assume that if I'm testing a patch, I'm going to need to rebuild regardless of what the patch actually touches. I often wait until the patch is applied before I do the rebuild, or if I'm manually testing a bug I go ahead and do the rebuild immediately. Most make targets (including 'test') will go ahead make sure the build is up to date without your input. Usually the slowest part of a rebuild is rerunning ./configure, which 'make' will do for you if it determines that it should. You can speed up ./configure by passing it the '--config-cache' (or '-C') option. If you're on a multi-core machine, also remember to pass '-j' to make to speed up building, and also to regrtest (which you can do with 'make test TESTOPTS=-j9') to speed up testing. [1]https://www.mercurial-scm.org/wiki/FetchExtension -- Zach From irmen.NOSPAM at xs4all.nl Mon Aug 8 13:15:38 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 8 Aug 2016 19:15:38 +0200 Subject: Network protocols, sans I/O,(Hopefully) the future of network protocols in Python In-Reply-To: References: Message-ID: <57a8be3b$0$876$e4fe514c@news.xs4all.nl> On 8-8-2016 12:37, Mark Lawrence wrote: > This may be of interest to some of you http://www.snarky.ca/network-protocols-sans-i-o > I sure find it interesting. Will also watch Cory's PyCon presentation that is mentioned! Thanks for the link. Irmen From tjreedy at udel.edu Mon Aug 8 15:25:05 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 8 Aug 2016 15:25:05 -0400 Subject: Running Python from the source repo In-Reply-To: References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/8/2016 12:24 PM, Zachary Ware wrote: > I generally assume that if I'm testing a patch, I'm going to need to > rebuild regardless of what the patch actually touches. I often wait > until the patch is applied before I do the rebuild, or if I'm manually > testing a bug I go ahead and do the rebuild immediately. Most make > targets (including 'test') will go ahead make sure the build is up to > date without your input. Usually the slowest part of a rebuild is > rerunning ./configure, which 'make' will do for you if it determines > that it should. You can speed up ./configure by passing it the > '--config-cache' (or '-C') option. If you're on a multi-core machine, > also remember to pass '-j' to make to speed up > building, and also to regrtest (which you can do with 'make test > TESTOPTS=-j9') to speed up testing. > > [1]https://www.mercurial-scm.org/wiki/FetchExtension Last January, I wrote a batch file to build all three versions with the 'optional' extensions. I started rebuilding more often after this. 36\pcbuild\build.bat -e -d 35\pcbuild\build.bat -e -d 27\pcbuild\build.bat -e -d Thanks for making this possible. It initially worked, but now it stops after the first command, even without errors. Has a flag been changed to treat warnings as errors? How can I change the .bat to wrap each command with the equivalent of try: except: pass? -- Terry Jan Reedy From bc at freeuk.com Mon Aug 8 15:51:07 2016 From: bc at freeuk.com (BartC) Date: Mon, 8 Aug 2016 20:51:07 +0100 Subject: Python and 64bit Windows7 In-Reply-To: References: Message-ID: On 08/08/2016 13:44, James Santiago wrote: > 4) I also went to DOS prompt or CMD screen, went to the directory where > Python was located as follows: > C:\Python>python exe -v. This returns several lines of information. I assume you meant "python.exe -v". (But the .exe is not needed.) > For example: > the first line is import _frozen_importlib # frozen. I get the same sort of stuff. It seems Python is there and it works. What's PyCharm and what error does it give? What does its Help section say about telling it where Python is? You might need support for that application rather than Python. (Perhaps it expects python.exe in a default path. Go to any other location, and type 'python'. If it can't find it, then its path is not set up. You can fix that temporarily by typing: set path=c:\python\;%path% then trying again. However this path variable 'belongs' to this command window and may not be seen by PyCharm if started from elsewhere. Setting it more globally is fiddly but there is plenty of info out there. The problem could be something else of course...) -- Bartc From zachary.ware+pylist at gmail.com Mon Aug 8 17:16:32 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 8 Aug 2016 16:16:32 -0500 Subject: Running Python from the source repo In-Reply-To: References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 8, 2016 at 2:25 PM, Terry Reedy wrote: > Last January, I wrote a batch file to build all three versions with the > 'optional' extensions. I started rebuilding more often after this. > > 36\pcbuild\build.bat -e -d > 35\pcbuild\build.bat -e -d > 27\pcbuild\build.bat -e -d > > Thanks for making this possible. It initially worked, but now it stops > after the first command, even without errors. Has a flag been changed to > treat warnings as errors? How can I change the .bat to wrap each command > with the equivalent of try: except: pass? I'm not sure why that would have stopped working, but the way to use a .bat from a .bat is to 'call' it: call 36\PCbuild\build.bat -e -d .bat scripts don't care about the exit codes of what they run, errors must be explicitly checked and 'exit' called if you want the script to die early. Try this for an unconditional build on all three branches, with a report at the end if any of them failed: call 36\PCbuild\build.bat -e -d set rc36=%ERRORLEVEL% call 35\PCbuild\build.bat -e -d set rc35=%ERRORLEVEL% call 27\PCbuild\build.bat -e -d set rc27=%ERRORLEVEL% @if %rc36% NEQ 0 ( echo 3.6 build failed, rc: %rc36% ) @if %rc35% NEQ 0 ( echo 3.5 build failed, rc: %rc35% ) @if %rc27% NEQ 0 ( echo 2.7 build failed, rc: %rc27% ) -- Zach From random832 at fastmail.com Mon Aug 8 17:25:44 2016 From: random832 at fastmail.com (Random832) Date: Mon, 08 Aug 2016 17:25:44 -0400 Subject: Running Python from the source repo In-Reply-To: References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1470691544.904395.689614049.0F1E44F8@webmail.messagingengine.com> On Mon, Aug 8, 2016, at 15:25, Terry Reedy wrote: > Last January, I wrote a batch file to build all three versions with the > 'optional' extensions. I started rebuilding more often after this. > > 36\pcbuild\build.bat -e -d > 35\pcbuild\build.bat -e -d > 27\pcbuild\build.bat -e -d > > Thanks for making this possible. It initially worked, but now it stops > after the first command, even without errors. Has a flag been changed > to treat warnings as errors? How can I change the .bat to wrap each > command with the equivalent of try: except: pass? I'm not sure how it ever worked, but you have to use "call" to run one batch file from another batch file, otherwise it doesn't start a recursive command interpreter and so it won't run anything else in the outer batch file. I don't know if .cmd files have this limitation. From ben+python at benfinney.id.au Mon Aug 8 20:37:02 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 09 Aug 2016 10:37:02 +1000 Subject: Specify the database for a Django ModelForm instance Message-ID: <85wpjq22xd.fsf@benfinney.id.au> Howdy all, How can I specify which database (by its alias name) a Django ModelForm should use? (I've had trouble getting this question in to the Django forum, so I'm trying here as this is still Python-related.) A Django ModelForm knows its corresponding model, and the fields included. The ModelForm instance clearly knows how to specify a database, internally. It can validate its fields against the database, and can save a new model instance to the database. This implies its operations have knowledge of which database to use. What I need is to access that as an external user, when creating the instance. I can't find how to specify any database other than the default, when creating the ModelForm nor when it interacts with the database. What is the equivalent for using='foo' when instantiating a ModelForm for the model, or calling its methods (ModelForm.clean, ModelForm.save, etc.)? -- \ ?The sun never sets on the British Empire. But it rises every | `\ morning. The sky must get awfully crowded.? ?Steven Wright | _o__) | Ben Finney From chivalry at mac.com Mon Aug 8 21:52:25 2016 From: chivalry at mac.com (Charles Ross) Date: Mon, 08 Aug 2016 18:52:25 -0700 Subject: Call for Assistance Message-ID: I?ve begun a new book called Meta Python that I?m looking for assistance with. The book is one I wish was out there, and so am writing. The book is targeted at experienced programmers who are novice Python users and want to move from that point to one where they have published to PyPI. I?m specifically looking for three types of users to assist me: - Expert Python developers who are willing to confirm and correct the technical details, since I?m getting to the material that I want to learn and therefore need to teach. So far that only really includes virtual environments, covered in chapter 2, but will soon include many other topics. - Developers like me, who aren?t new to programming but are perhaps new to Python and certainly new to publishing to PyPI, and who can provide feedback on the clarity and usefulness of the content. - Linux and Windows Python users who would be willing to assist in making the book a bit less Mac-centric. The book is being hosted at https://github.com/chivalry/meta-python . The introduction and first two chapters have their initial drafts complete, but you?ll see a few ?TODOs? in there. Any and all feedback is appreciated, and preferred through the GitHub Issues system, but feel free to email me if you prefer. Thanks, Chuck From rantingrickjohnson at gmail.com Mon Aug 8 23:46:07 2016 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 8 Aug 2016 20:46:07 -0700 (PDT) Subject: Awful code of the week In-Reply-To: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> Message-ID: <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> On Sunday, August 7, 2016 at 1:54:51 AM UTC-5, Steven D'Aprano wrote: > Seen in the office IRC channel: > > > (13:23:07) fred: near_limit = [] > (13:23:07) fred: near_limit.append(1) > (13:23:07) fred: near_limit = len(near_limit) > (13:23:09) fred: WTF Sure, this code smells of nauseous superfluity, but what comes after these three lines? Is this _all_ there is? Or are these three lines merely the initial setup stage for a complex looping algorithm? Cherry picking a few lines from a script and then judging them "unworthy", would not be a fair. We should never attempt to purposely mimic the abysmal state of justice in the USA. From rosuav at gmail.com Mon Aug 8 23:59:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Aug 2016 13:59:14 +1000 Subject: Awful code of the week In-Reply-To: <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> Message-ID: On Tue, Aug 9, 2016 at 1:46 PM, Rick Johnson wrote: > On Sunday, August 7, 2016 at 1:54:51 AM UTC-5, Steven D'Aprano wrote: >> Seen in the office IRC channel: >> >> >> (13:23:07) fred: near_limit = [] >> (13:23:07) fred: near_limit.append(1) >> (13:23:07) fred: near_limit = len(near_limit) >> (13:23:09) fred: WTF > > Sure, this code smells of nauseous superfluity, but what > comes after these three lines? Is this _all_ there is? Or > are these three lines merely the initial setup stage for a > complex looping algorithm? Cherry picking a few lines from a > script and then judging them "unworthy", would not be a > fair. We should never attempt to purposely mimic the abysmal > state of justice in the USA. I agree. There's nothing wrong with that code. I routinely have constructs like this: def register(self, obj): self.files.append(obj) return None return None # just in case return None def process(self, stuff): files = self.files files = [] # to save memory files = self.files for file in files: file.process(stuff) return 1 It's perfectly good code, and Fred was flat out wrong to say "WTF" about this. ChrisA From steve+comp.lang.python at pearwood.info Tue Aug 9 01:20:28 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Aug 2016 15:20:28 +1000 Subject: Awful code of the week References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> Message-ID: <57a9681f$0$1609$c3e8da3$5496439d@news.astraweb.com> On Tuesday 09 August 2016 13:59, Chris Angelico wrote: > On Tue, Aug 9, 2016 at 1:46 PM, Rick Johnson > wrote: >> On Sunday, August 7, 2016 at 1:54:51 AM UTC-5, Steven D'Aprano wrote: >>> Seen in the office IRC channel: >>> >>> >>> (13:23:07) fred: near_limit = [] >>> (13:23:07) fred: near_limit.append(1) >>> (13:23:07) fred: near_limit = len(near_limit) >>> (13:23:09) fred: WTF >> >> Sure, this code smells of nauseous superfluity, but what >> comes after these three lines? Is this _all_ there is? Or >> are these three lines merely the initial setup stage for a >> complex looping algorithm? Cherry picking a few lines from a >> script and then judging them "unworthy", would not be a >> fair. We should never attempt to purposely mimic the abysmal >> state of justice in the USA. > > I agree. There's nothing wrong with that code. I routinely have > constructs like this: > > def register(self, obj): > self.files.append(obj) > return None > return None # just in case > return None That's buggy. It should be: def register(self, obj): self.files.append(obj) return None and None or None return None # just in case return None That way you're protected in case the boolean value of None is ever changed. > def process(self, stuff): > files = self.files > files = [] # to save memory > files = self.files > for file in files: > file.process(stuff) > return 1 def process(self, stuff): try: files = self.files del files # to save memory files = [] # just to be sure files = self.files for file in files: file.process(stuff) return 1 finally: del files del file del stuff del self # this has to come last You can't be too careful with memory management. -- Steve From rosuav at gmail.com Tue Aug 9 01:37:47 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Aug 2016 15:37:47 +1000 Subject: Awful code of the week In-Reply-To: <57a9681f$0$1609$c3e8da3$5496439d@news.astraweb.com> References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> <57a9681f$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Aug 9, 2016 at 3:20 PM, Steven D'Aprano wrote: >> def process(self, stuff): >> files = self.files >> files = [] # to save memory >> files = self.files >> for file in files: >> file.process(stuff) >> return 1 > > def process(self, stuff): > try: > files = self.files > del files # to save memory > files = [] # just to be sure > files = self.files > for file in files: > file.process(stuff) > return 1 > finally: > del files > del file > del stuff > del self # this has to come last > > > You can't be too careful with memory management. Right. Of course, it gets very onerous, so we tend to use a context manager instead. def process(self, stuff): with deallocate() as cleanup: cleanup(self) cleanup(stuff) files = self.files cleanup(files) del files files = [] cleanup(files) files = self.files cleanup(files) for file in files: cleanup(file) file.process(stuff) return 1 There, isn't that so much better? ChrisA From tjreedy at udel.edu Tue Aug 9 03:55:59 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 9 Aug 2016 03:55:59 -0400 Subject: Running Python from the source repo In-Reply-To: References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/8/2016 5:16 PM, Zachary Ware wrote: > On Mon, Aug 8, 2016 at 2:25 PM, Terry Reedy wrote: >> Last January, I wrote a batch file to build all three versions with the >> 'optional' extensions. I started rebuilding more often after this. >> >> 36\pcbuild\build.bat -e -d >> 35\pcbuild\build.bat -e -d >> 27\pcbuild\build.bat -e -d >> >> Thanks for making this possible. It initially worked, but now it stops >> after the first command, even without errors. Has a flag been changed to >> treat warnings as errors? How can I change the .bat to wrap each command >> with the equivalent of try: except: pass? > > I'm not sure why that would have stopped working, but the way to use a > .bat from a .bat is to 'call' it: > > call 36\PCbuild\build.bat -e -d > > .bat scripts don't care about the exit codes of what they run, errors > must be explicitly checked and 'exit' called if you want the script to > die early. Try this for an unconditional build on all three branches, > with a report at the end if any of them failed: > > call 36\PCbuild\build.bat -e -d > set rc36=%ERRORLEVEL% > > call 35\PCbuild\build.bat -e -d > set rc35=%ERRORLEVEL% > > call 27\PCbuild\build.bat -e -d > set rc27=%ERRORLEVEL% > > @if %rc36% NEQ 0 ( > echo 3.6 build failed, rc: %rc36% > ) > @if %rc35% NEQ 0 ( > echo 3.5 build failed, rc: %rc35% > ) > @if %rc27% NEQ 0 ( > echo 2.7 build failed, rc: %rc27% > ) This works great. Might there be any way to collect together the warning messages? There were perhaps 100 for the changes in the last few weeks. (People on non-windows seems to routinely write code that msc does not like.) -- Terry Jan Reedy From tjreedy at udel.edu Tue Aug 9 03:57:48 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 9 Aug 2016 03:57:48 -0400 Subject: Running Python from the source repo In-Reply-To: <1470691544.904395.689614049.0F1E44F8@webmail.messagingengine.com> References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> <1470691544.904395.689614049.0F1E44F8@webmail.messagingengine.com> Message-ID: On 8/8/2016 5:25 PM, Random832 wrote: > On Mon, Aug 8, 2016, at 15:25, Terry Reedy wrote: >> Last January, I wrote a batch file to build all three versions with the >> 'optional' extensions. I started rebuilding more often after this. >> >> 36\pcbuild\build.bat -e -d >> 35\pcbuild\build.bat -e -d >> 27\pcbuild\build.bat -e -d >> >> Thanks for making this possible. It initially worked, but now it stops >> after the first command, even without errors. Has a flag been changed >> to treat warnings as errors? How can I change the .bat to wrap each >> command with the equivalent of try: except: pass? > > I'm not sure how it ever worked, but you have to use "call" to run one > batch file from another batch file, otherwise it doesn't start a > recursive command interpreter and so it won't run anything else in the > outer batch file. Perhaps I mis-remember. Zack's revision using call works. -- Terry Jan Reedy From lutz.horn at posteo.de Tue Aug 9 04:06:32 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Tue, 9 Aug 2016 10:06:32 +0200 Subject: Call for Assistance In-Reply-To: References: Message-ID: Am 08/09/2016 um 03:52 AM schrieb Charles Ross: > The book is being hosted at https://github.com/chivalry/meta-python CC-BY-NC-SA is not a license for free (as in speech) content. Is that what you want? Lutz -- https://emailselfdefense.fsf.org/ From steve+comp.lang.python at pearwood.info Tue Aug 9 04:12:59 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Aug 2016 18:12:59 +1000 Subject: Awful code of the week References: <57a6db2d$0$2758$c3e8da3$76491128@news.astraweb.com> <12801c65-384d-478c-9464-0987b1f61ffd@googlegroups.com> <57a9681f$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57a9908c$0$2895$c3e8da3$76491128@news.astraweb.com> On Tuesday 09 August 2016 15:37, Chris Angelico wrote: [...] >> You can't be too careful with memory management. > > Right. Of course, it gets very onerous, so we tend to use a context > manager instead. > > def process(self, stuff): > with deallocate() as cleanup: > cleanup(self) > cleanup(stuff) > files = self.files > cleanup(files) > del files > files = [] > cleanup(files) > files = self.files > cleanup(files) > for file in files: > cleanup(file) > file.process(stuff) > return 1 > > There, isn't that so much better? Ha! That's why I never use context managers -- they're too easy to write buggy code. You forgot to deallocate cleanup itself! Memory leak! -- Steve From steve+comp.lang.python at pearwood.info Tue Aug 9 04:17:20 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Aug 2016 18:17:20 +1000 Subject: Call for Assistance References: Message-ID: <57a99192$0$1602$c3e8da3$5496439d@news.astraweb.com> On Tuesday 09 August 2016 11:52, Charles Ross wrote: > I?ve begun a new book called Meta Python that I?m looking for assistance > with. Are you looking for people to be co-authors? Are you offering payment, or credit? A free copy of the book? A kick to the head? Or just looking for volunteers to provide content for the book while you lie back on a beach in the Bahamas collecting the millions[1] in royalties? [1] Zimbabwean dollars. -- Steve From chivalry at mac.com Tue Aug 9 04:37:25 2016 From: chivalry at mac.com (Charles Ross) Date: Tue, 09 Aug 2016 01:37:25 -0700 Subject: Call for Assistance In-Reply-To: <57a99192$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <57a99192$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <71AEBF3A-B457-4FFB-BA84-A3D4BD1781C0@mac.com> You?re right. I should have mentioned that this is a volunteer effort on my part and anyone else who becomes involved. It?s being released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (http://creativecommons.org/licenses/by-nc-sa/4.0/ ) as demonstrated by the LICENSE.md file at the GitHub repository (https://github.com/chivalry/meta-python/blob/master/LICENSE.md ). I don?t yet know precisely how the ?book? will be published. I might just leave it as a set of GitHub files or export the Markdown to other format to be hosted outside GitHub. I?m not worrying yet about publishing, only clarity and correctness of content. Thanks, Chuck > On Aug 9, 2016, at 1:17 AM, Steven D'Aprano wrote: > > On Tuesday 09 August 2016 11:52, Charles Ross wrote: > >> I?ve begun a new book called Meta Python that I?m looking for assistance >> with. > > Are you looking for people to be co-authors? Are you offering payment, or > credit? A free copy of the book? A kick to the head? Or just looking for > volunteers to provide content for the book while you lie back on a beach in the > Bahamas collecting the millions[1] in royalties? > > > > > > [1] Zimbabwean dollars. > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list From berteh at gmail.com Tue Aug 9 06:56:56 2016 From: berteh at gmail.com (berteh at gmail.com) Date: Tue, 9 Aug 2016 03:56:56 -0700 (PDT) Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? Message-ID: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Hello. My python script should run on Linux, Win and MacOS, but I can't find a way to have logging.conf configuration that works out of the box accross all these OSes. I don't want my users to have to edit configuration files manually. If I use a "linux-like" handler configuration such as below it fails on Windows when the script is run from a directory from which the user has no write access (ie any subfolder or C:\Program Files, that is the default install location) [handler_file] class=handlers.RotatingFileHandler args=('scribusGenerator.log','a','maxBytes=1000','backupCount=1') And if I use a "windows-like" file location it fails (of course) in Linux and MacOSX [handler_file] class=handlers.RotatingFileHandler args=('D:\scribusGenerator.log','a','maxBytes=1000','backupCount=1') One option would be to set the location dynamically in the system temp directory (from tempfile.gettempdir())... but I can't find a way to change the location of the handler loaded via config module. Any piece of advice on how to achieve cross-platform logging that does not require a modification by the user? Thanks. B. From mal at europython.eu Tue Aug 9 07:04:16 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Tue, 9 Aug 2016 13:04:16 +0200 Subject: EuroPython 2017 On-site Teams: Call for Interest results Message-ID: <57A9B8B0.405@europython.eu> We are happy to announce the results of our EuroPython 2017 On-site Teams: Call for Interest (CFI). The CFI finished with exciting new locations and on-site teams willing to make EuroPython 2017 possible. The CFI proposals we received were for: * Milan or Como Lake, Italy, from the Python Italia Society (APS), * Brno, the Czech Republic, from the local Brno Python community, and * Bilbao, Basque Country, Spain, from the Python San Sebasti?n Society (ACPySS). We are now reviewing the proposals and will then move forward with the more formal and detailed CFP later in September. Enjoy, -- EuroPython Society http://www.europython-society.org/ From mal at europython.eu Tue Aug 9 07:14:26 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Tue, 9 Aug 2016 13:14:26 +0200 Subject: EuroPython 2016: Thank you to all our organizers and volunteers Message-ID: <57A9BB12.4000005@europython.eu> EuroPython 2016 is now over and was an overwhelming success thanks to everyone who helped make it happen. Since EuroPython would not be possible without the help of our volunteers and organizers, we?d like to say Thank You !!! to all the individuals who invested time and effort into the event. Please see our blog post for the full list of people who have helped: http://blog.europython.eu/post/148685512862/europython-2016-thank-you-to-all-our-organizers If you are interested in helping with EuroPython 2017, we invite you to join the workgroups. Please go to the EuroPython Workgroups page: http://www.europython-society.org/workgroups and follow the instructions there to get signed up. Many thanks ! With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/762968990658523137 Thanks. From steve+python at pearwood.info Tue Aug 9 07:55:56 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Aug 2016 21:55:56 +1000 Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? References: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Message-ID: <57a9c4ce$0$1590$c3e8da3$5496439d@news.astraweb.com> On Tue, 9 Aug 2016 08:56 pm, berteh at gmail.com wrote: > Hello. > > My python script should run on Linux, Win and MacOS, but I can't find a > way to have logging.conf configuration that works out of the box accross > all these OSes. > > I don't want my users to have to edit configuration files manually. > > > If I use a "linux-like" handler configuration such as below it fails on > Windows when the script is run from a directory from which the user has no > write access (ie any subfolder or C:\Program Files, that is the default > install location) > > [handler_file] > class=handlers.RotatingFileHandler > args=('scribusGenerator.log','a','maxBytes=1000','backupCount=1') That will fail on Linux too, if the current working directory is somewhere that the user doesn't have write permission. The *right* way to do this is: - allow the user to optionally specify a log file location; - if they don't, set a sensible default according to the platform, e.g.: # untested DEFAULTS = { 'nt': '...', 'linux': '...', 'macos': '...', } location = DEFAULTS.get(sys.platform, 'somewhere else') Read the docs for the list of possible values for sys.platform. Generally, for system tools, you should write to a system-wide location. On Linux, that might be /var/log/FILENAME. But that's not generally writable by unprivileged users, so you might want to log directly in the user's home directory: os.path.expanduser('~/FILENAME') which ought to do the right thing on Windows, POSIX systems (including Linux and OS X). -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Aug 9 07:59:07 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Tue, 09 Aug 2016 21:59:07 +1000 Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? References: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Message-ID: <57a9c58d$0$1590$c3e8da3$5496439d@news.astraweb.com> On Tue, 9 Aug 2016 08:56 pm, berteh at gmail.com wrote: > Hello. > > My python script should run on Linux, Win and MacOS, but I can't find a > way to have logging.conf configuration that works out of the box accross > all these OSes. Oh! Wait, I didn't pick up on the logging.conf part! D'oh! > [handler_file] > class=handlers.RotatingFileHandler > args=('scribusGenerator.log','a','maxBytes=1000','backupCount=1') Try using '~/scribusGenerator.log' to specify the user's home directory. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From drewes.mil at gmail.com Tue Aug 9 08:40:45 2016 From: drewes.mil at gmail.com (drewes.mil at gmail.com) Date: Tue, 9 Aug 2016 05:40:45 -0700 (PDT) Subject: cropping a random part of an image Message-ID: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Hi, I'm new to python and I have 30.000 pictures. I need to crop, let's say 100, parts of 256x256, randomly out of every picture. I cant find an answer in the net, would be nice if someone could help me out! Thanks! Steffen From martin.schoon at gmail.com Tue Aug 9 08:44:16 2016 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 9 Aug 2016 12:44:16 GMT Subject: How do I make a video animation with transparent background? Message-ID: Here is what I want to do: I want to create an animated graph and store it as a video file, mp4 say. This file will then be used as an overlay on another video. Using this example: http://matplotlib.org/examples/animation/moviewriter.html and this tutorial: https://www.youtube.com/watch?v=h0F5X4b7jug I have covered a lot of ground. What I have failed to achieve is a graph with a transparent background. None of the leads I have found searching the Internet have helped me. (I may have found the needed information without realising it...) Any help is appreciated. Another thing I would like to do is to present numerical data in the video or as a separate video -- think LCD display reading out numbers up-dated every second. Once again preferably on a transparent background. Here I have covered no ground :-( TIA, /Martin From havocjoseph at gmail.com Tue Aug 9 09:22:06 2016 From: havocjoseph at gmail.com (Joseph Bane) Date: Tue, 9 Aug 2016 06:22:06 -0700 (PDT) Subject: Question regarding stdlib distutils strtobool behavior Message-ID: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Hello. It recently came to my attention that the strtobool function in the standard library doesn't return Python native boolean values, but rather returns integer 0 or 1: https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 I am curious why this is the defined behavior and whether anyone can fill me in regarding this approach. For clarity, I would expect the code to `return True` and `return False` rather than `return 1` and `return 0`. Thanks, Joe From __peter__ at web.de Tue Aug 9 09:30:52 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 09 Aug 2016 15:30:52 +0200 Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? References: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Message-ID: berteh at gmail.com wrote: > Hello. > > My python script should run on Linux, Win and MacOS, but I can't find a > way to have logging.conf configuration that works out of the box accross > all these OSes. > > I don't want my users to have to edit configuration files manually. > > > If I use a "linux-like" handler configuration such as below it fails on > Windows when the script is run from a directory from which the user has no > write access (ie any subfolder or C:\Program Files, that is the default > install location) > > [handler_file] > class=handlers.RotatingFileHandler > args=('scribusGenerator.log','a','maxBytes=1000','backupCount=1') > > And if I use a "windows-like" file location it fails (of course) in Linux > and MacOSX > > [handler_file] > class=handlers.RotatingFileHandler > args=('D:\scribusGenerator.log','a','maxBytes=1000','backupCount=1') > > > One option would be to set the location dynamically in the system temp > directory (from tempfile.gettempdir())... but I can't find a way to change > the location of the handler loaded via config module. > > Any piece of advice on how to achieve cross-platform logging that does not > require a modification by the user? args is passed to eval(), so args=("D:\scribusGenerator.log" if sys.platform == "win32" else "scribusGenerator.log", ...) should work. From michael.selik at gmail.com Tue Aug 9 09:34:14 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 13:34:14 +0000 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: On Tue, Aug 9, 2016 at 9:26 AM Joseph Bane wrote: > Hello. > > It recently came to my attention that the strtobool function in the > standard library doesn't return Python native boolean values, but rather > returns integer 0 or 1: > > https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 > > I am curious why this is the defined behavior and whether anyone can fill > me in regarding this approach. For clarity, I would expect the code to > `return True` and `return False` rather than `return 1` and `return 0`. > I'll take a guess: it's probably a(n) historical artifact. Before there were bools, returning 1 or 0 was the obvious technique. Even after bools were added to Python 2, returning the literal 1 or 0 was faster than looking up the names True or False. Now that True and False are keywords, using the keyword is the obvious solution. From lutz.horn at posteo.de Tue Aug 9 09:37:48 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Tue, 9 Aug 2016 15:37:48 +0200 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: <85bd9ff0-56a2-bba7-8104-ae27a52a6c5f@posteo.de> Am 08/09/2016 um 03:22 PM schrieb Joseph Bane: > It recently came to my attention that the strtobool function in the > standard library doesn't return Python native boolean values, but > rather returns integer 0 or 1: In boolean context, 1 is True and 0 is False. So you should be able to use the result without problems. Why the implementors of Python wrote strtobool this way is something I don't know. Lutz From havocjoseph at gmail.com Tue Aug 9 09:50:29 2016 From: havocjoseph at gmail.com (Joseph Bane) Date: Tue, 9 Aug 2016 09:50:29 -0400 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: Thank you for your note. I was thinking it might be just a historical artifact. Do you think this is the type of thing we could propose be changed in the next version of Python? It seems it would be more consistent with the principle of least surprise from the perspective of outsider Python calling code. I would be happy to submit a small patch for this change. As a side note, we discovered this with some code that was checking `if var is True` instead of just `if var`. I know this particular style choice isn't really dictated by the community (as opposed to something like `if var is None` that is), but it seems the latter option is the (generally) preferred approach. Basically, it seemed surprising that the former approach failed in our specific scenario because of how the `is` operator works: >>> 1 == True True >>> 0 == False True >>> 1 is True False >>> 0 is False False I am definitely not arguing that this behavior be changed, but just putting the larger issue into context. On Tue, Aug 9, 2016 at 9:34 AM, Michael Selik wrote: > On Tue, Aug 9, 2016 at 9:26 AM Joseph Bane wrote: > >> Hello. >> >> It recently came to my attention that the strtobool function in the >> standard library doesn't return Python native boolean values, but rather >> returns integer 0 or 1: >> >> https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 >> >> I am curious why this is the defined behavior and whether anyone can fill >> me in regarding this approach. For clarity, I would expect the code to >> `return True` and `return False` rather than `return 1` and `return 0`. >> > > I'll take a guess: it's probably a(n) historical artifact. Before there > were bools, returning 1 or 0 was the obvious technique. Even after bools > were added to Python 2, returning the literal 1 or 0 was faster than > looking up the names True or False. Now that True and False are keywords, > using the keyword is the obvious solution. > From __peter__ at web.de Tue Aug 9 10:30:18 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 09 Aug 2016 16:30:18 +0200 Subject: cropping a random part of an image References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: drewes.mil at gmail.com wrote: > I'm new to python and I have 30.000 pictures. > I need to crop, let's say 100, parts of 256x256, randomly out of every > picture. > > I cant find an answer in the net, would be nice if someone could help me > out! You can walk over the files with https://docs.python.org/dev/library/os.html#os.walk find out the image size and process the image with pillow http://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.size http://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.crop using https://docs.python.org/dev/library/random.html to determine what part of an image you want to pick. Now write some code and come back here if you run into problems you cannot solve yourself. From havocjoseph at gmail.com Tue Aug 9 10:43:59 2016 From: havocjoseph at gmail.com (Joseph Bane) Date: Tue, 9 Aug 2016 07:43:59 -0700 (PDT) Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: On Tuesday, August 9, 2016 at 9:34:44 AM UTC-4, Michael Selik wrote: > On Tue, Aug 9, 2016 at 9:26 AM Joseph Bane wrote: > > > Hello. > > > > It recently came to my attention that the strtobool function in the > > standard library doesn't return Python native boolean values, but rather > > returns integer 0 or 1: > > > > https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 > > > > I am curious why this is the defined behavior and whether anyone can fill > > me in regarding this approach. For clarity, I would expect the code to > > `return True` and `return False` rather than `return 1` and `return 0`. > > > > I'll take a guess: it's probably a(n) historical artifact. Before there > were bools, returning 1 or 0 was the obvious technique. Even after bools > were added to Python 2, returning the literal 1 or 0 was faster than > looking up the names True or False. Now that True and False are keywords, > using the keyword is the obvious solution. (I am sorry if this is a duplicate; the reply didn't seem to make it through earlier...) Thank you for your note. I was thinking it might be just a historical artifact. Do you think this is the type of thing we could propose be changed in the next version of Python? It seems it would be more consistent with the principle of least surprise from the perspective of outsider Python calling code. I would be happy to submit a small patch for this change. As a side note, we discovered this with some code that was checking `if var is True` instead of just `if var`. I know this particular style choice isn't really dictated by the community (as opposed to something like `if var is None` that is), but it seems the latter option is the (generally) preferred approach. Basically, it seemed surprising that the former approach failed in our specific scenario because of how the `is` operator works: >>> 1 == True True >>> 0 == False True >>> 1 is True False >>> 0 is False False I am definitely not arguing that this behavior be changed, but just putting the larger issue into context. From brunnre8 at gmail.com Tue Aug 9 11:42:13 2016 From: brunnre8 at gmail.com (Reto Brunner) Date: Tue, 09 Aug 2016 15:42:13 +0000 Subject: Call for Assistance In-Reply-To: References: Message-ID: What on earth isn't "free" enough about You are free to: Share ? copy and redistribute the material in any medium or format Adapt ? remix, transform, and build upon the material The licensor cannot revoke these freedoms as long as you follow the license terms. It is even a viral (copy left) licence, so even a fsf member should be happy On Tue, Aug 9, 2016, 16:59 Lutz Horn wrote: > Am 08/09/2016 um 03:52 AM schrieb Charles Ross: > > The book is being hosted at https://github.com/chivalry/meta-python > > CC-BY-NC-SA is not a license for free (as in speech) content. Is that > what you want? > > > From michael.selik at gmail.com Tue Aug 9 12:10:28 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 16:10:28 +0000 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: On Tue, Aug 9, 2016 at 9:50 AM Joseph Bane wrote: > Thank you for your note. I was thinking it might be just a historical > artifact. > That was just my guess. You can search the mailing list archives and the bug tracker to find out if there's been any past discussion. > Do you think this is the type of thing we could propose be changed in the > next version of Python? It seems it would be more consistent with > the principle of least surprise from the perspective of outsider Python > calling code. I would be happy to submit a small patch for this change. > Seems non-controversial enough that you could write a bug on http://bugs.python.org/ and then submit a patch. The question is whether or not you can convince someone with commit access to push your change. Even if your patch improves the code, it might not improve it enough to justify change. There are some disadvantages to code churn. If I were revising it, it'd be something like this... def strtobool(val): """Convert a string representation of truth to true (1) or false (0). True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if 'val' is anything else. """ val = val.casefold() if val in ('y', 'yes', 't', 'true', 'on', '1'): return True if val in ('n', 'no', 'f', 'false', 'off', '0'): return False raise ValueError(f'invalid truth value {val}') For unicode, str.casefold is better than str.lower for caseless comparison. The elif and else were unnecessary because of the returns. For fun, I thought I'd use the new string formatting style ( https://www.python.org/dev/peps/pep-0498/). Again, I'm not sure if those changes are worthwhile. As a side note, we discovered this with some code that was checking `if var > is True` instead of just `if var`. > The Pythonic style is to simply check ``if var`` rather than ``if var is True``. This realizes the flexibility of duck typing -- it doesn't matter whether the variable is True, so long as it is true (eg. has a non-false truth value). The core developers might thus say there's no bug here, because 1 is true. My apologies for forgetting to use interleaved response style previously. Google Inbox defaults to top-posting. On Tue, Aug 9, 2016 at 9:34 AM, Michael Selik wrote: > On Tue, Aug 9, 2016 at 9:26 AM Joseph Bane wrote: >> >>> Hello. >>> >>> It recently came to my attention that the strtobool function in the >>> standard library doesn't return Python native boolean values, but rather >>> returns integer 0 or 1: >>> >>> https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 >>> >>> I am curious why this is the defined behavior and whether anyone can >>> fill me in regarding this approach. For clarity, I would expect the code to >>> `return True` and `return False` rather than `return 1` and `return 0`. >>> >> >> I'll take a guess: it's probably a(n) historical artifact. Before there >> were bools, returning 1 or 0 was the obvious technique. Even after bools >> were added to Python 2, returning the literal 1 or 0 was faster than >> looking up the names True or False. Now that True and False are keywords, >> using the keyword is the obvious solution. >> > > From michael.selik at gmail.com Tue Aug 9 12:17:53 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 16:17:53 +0000 Subject: Call for Assistance In-Reply-To: References: Message-ID: On Tue, Aug 9, 2016 at 3:23 AM Charles Ross wrote: > I?ve begun a new book called Meta Python that I?m looking for assistance > with. The book is one I wish was out there, and so am writing. The book is > targeted at experienced programmers who are novice Python users and want to > move from that point to one where they have published to PyPI. > Maybe you should add a chapter discussing community, open-source, volunteerism, and how difficult it is to get enthusiastic, productive contributors, but how easy it is to get emailed criticism? (no cynicism intended) From zachary.ware+pylist at gmail.com Tue Aug 9 12:24:50 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Tue, 9 Aug 2016 11:24:50 -0500 Subject: Running Python from the source repo In-Reply-To: References: <57a7ea54$0$1616$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Aug 9, 2016 at 2:55 AM, Terry Reedy wrote: > This works great. Might there be any way to collect together > the warning messages? There were perhaps 100 for the changes in > the last few weeks. (People on non-windows seems to routinely write code > that msc does not like.) Glad it works! Collecting warning messages is less simple, you'd probably get the best results from writing a Python script to drive the build.bats and do what you want with the output. You can also pass '-v' to build.bat to get more verbose output, which includes a summary of warnings and errors at the end of each build. -- Zach From rosuav at gmail.com Tue Aug 9 12:30:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 02:30:22 +1000 Subject: Call for Assistance In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 1:42 AM, Reto Brunner wrote: > What on earth isn't "free" enough about > > You are free to: > Share ? copy and redistribute the material in any medium or format > > Adapt ? remix, transform, and build upon the material > > The licensor cannot revoke these freedoms as long as you follow the license > terms. But NOT "use commercially". It's a restricted freedom that needs to be bound up with that restriction forever afterward. ChrisA From jussi.piitulainen at helsinki.fi Tue Aug 9 12:32:30 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 09 Aug 2016 19:32:30 +0300 Subject: Call for Assistance References: Message-ID: Reto Brunner writes: > On Tue, Aug 9, 2016, 16:59 Lutz Horn wrote: > >> Am 08/09/2016 um 03:52 AM schrieb Charles Ross: >> > The book is being hosted at https://github.com/chivalry/meta-python >> >> CC-BY-NC-SA is not a license for free (as in speech) content. Is that >> what you want? > > What on earth isn't "free" enough about > > You are free to: > Share ? copy and redistribute the material in any medium or format > > Adapt ? remix, transform, and build upon the material > > The licensor cannot revoke these freedoms as long as you follow the > license terms. > > It is even a viral (copy left) licence, so even a fsf member should be > happy The FSF does not consider non-commercial licenses free. Check their entries on licensing and philosophy at http://www.fsf.org/. Licensing / Licenses -> http://www.fsf.org/licensing/licenses (ends up at https://www.gnu.org/licenses/license-list.html, contains an entry on CC-NC at https://www.gnu.org/licenses/license-list.html#CC-BY-NC) Philosophy / The Free Software Definition -> https://www.gnu.org/philosophy/free-sw.html Philosophy / Selling Free Software -> https://www.gnu.org/philosophy/selling.html If Charles Ross gets significant contributions from others under the NC license, I think even he would be unable to sell copies or use the book in paid courses, unless he gets other contributors to assign their copyrights to him so that he is not bound by the license himself. Is that what he wants? Is that what other contributors want? Why? From lutz.horn at posteo.de Tue Aug 9 13:30:41 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Tue, 9 Aug 2016 19:30:41 +0200 Subject: Call for Assistance In-Reply-To: References: Message-ID: <8efb032e-350f-60bb-cfd9-b0767f51aaa0@posteo.de> Regarding the CC-BY-NC-SA license: Am 09.08.16 um 17:42 schrieb Reto Brunner: > What on earth isn't "free" enough about [...] > It is even a viral (copy left) licence, so even a fsf member should > be happy The FSF considers the CC-BY-NC to *not* be a license for free documentation[1]: > This license does not qualify as free, because there are restrictions > on charging money for copies. Thus, we recommend you do not use this > license for documentation. Using the SA variant does not help. Lutz [1] https://www.gnu.org/licenses/license-list.en.html#CC-BY-NC From tjreedy at udel.edu Tue Aug 9 13:42:10 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 9 Aug 2016 13:42:10 -0400 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: On 8/9/2016 9:22 AM, Joseph Bane wrote: > Hello. > > It recently came to my attention that the strtobool function in the > standard library doesn't return Python native boolean values, but > rather returns integer 0 or 1: > > https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 > > I am curious why this is the defined behavior and whether anyone can > fill me in regarding this approach. For clarity, I would expect the > code to `return True` and `return False` rather than `return 1` and > `return 0`. I am rather sure distutils predates 2.2 and the addition of bool. Much code has been converted from 0/1 to False/True. I don't know if this particular holdover is intentional -- fear of breaking something -- or oversight/inertia. I searched the tracker for 'strtobool' in component 'distutiles', status "don't care", and there were no hits. (Search is not perfect, though.) -- Terry Jan Reedy From philb at philb.ca Tue Aug 9 13:56:34 2016 From: philb at philb.ca (Phil Boutros) Date: Tue, 9 Aug 2016 17:56:34 -0000 (UTC) Subject: Specify the database for a Django ModelForm instance References: <85wpjq22xd.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Howdy all, > > How can I specify which database (by its alias name) a Django ModelForm > should use > What is the equivalent for using='foo' when instantiating a ModelForm > for the model, or calling its methods (ModelForm.clean, ModelForm.save, > etc.)? You would specify it on the save method (using the same "using='foo'") paremeter. Do your work between calling save with commit=false and doing the real save. For example: http://stackoverflow.com/questions/37740848/use-a-specific-database-with-a-modelform Phil -- AH#61 Wolf#14 BS#89 bus#1 CCB#1 SENS KOTC#4 philb at philb.ca http://philb.ca EKIII rides with me: http://eddiekieger.com From xlargedinosaur at aol.com Tue Aug 9 14:07:50 2016 From: xlargedinosaur at aol.com (xlargedinosaur at aol.com) Date: Tue, 9 Aug 2016 14:07:50 -0400 Subject: Subprocess Startup Error Message-ID: <156707d3a75-2837-ddbb@webprd-m17.mail.aol.com> Does anyone have advice on how to resolve this message when I attempt to open IDLE? "IDLE's subprocess didn't make connection. Either IDLE can't start a subprocess or personal firewall software is blocking the connection." I am running Windows 10 Home on a 64bit machine. I am running Python 3.5.2. I have tried uninstalling the software and reinstalling but no luck with that. From jpablo.romero at gmail.com Tue Aug 9 14:29:58 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 11:29:58 -0700 Subject: What's the best way to minimize the need of run time checks? Message-ID: Hello, In online forums sometimes people complain that they end up having to test constantly for None, or that a function's argument has a specific type / shape (which immediately brings the following aphorism to mind: "Any sufficiently large test suite contains an ad hoc, bug-ridden, slow implementation of half of Haskell's type system"). Some responses I've seen to that particular complaint is that they are doing it wrong. That they are trying to force a type system into a dynamic language (Clojure, Ruby, Python, Javascript, etc). Do you guys have any resources you like that addresses the following issue: What is the best way to use the dynamic features of Python to avoid having to write a poor's man type system? Thanks! Juan Pablo From rosuav at gmail.com Tue Aug 9 14:46:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 04:46:24 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 4:29 AM, Juan Pablo Romero M?ndez wrote: > What is the best way to use the dynamic features of Python to avoid having > to write a poor's man type system? > Step 1: Don't even care about it. Take whatever comes, and assume that it's what you expect. Step 2: When step 1 starts showing that you're getting the same problems lots of times, slap in a try/except to deal with just that. Most of the time, step 1 is all you need. ChrisA From michael.selik at gmail.com Tue Aug 9 14:51:38 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 18:51:38 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Tue, Aug 9, 2016 at 2:31 PM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > Hello, > > In online forums sometimes people complain that they end up having to test > constantly for None, or that a function's argument has a specific type / > shape (which immediately brings the following aphorism to mind: "Any > sufficiently large test suite contains an ad hoc, bug-ridden, slow > implementation of half of Haskell's type system"). > > Some responses I've seen to that particular complaint is that they are > doing it wrong. That they are trying to force a type system into a dynamic > language (Clojure, Ruby, Python, Javascript, etc). > > Do you guys have any resources you like that addresses the following issue: > What is the best way to use the dynamic features of Python to avoid having > to write a poor's man type system? > It'll be easiest to discuss a specific example. Can you share a chunk of code which frustrates you? In general, Python's TypeError and AttributeError serve much the same purpose as other languages' type systems. Where other languages might force you to create named, formally specified types, Python allows informally specified interfaces. "File-like" is a good example. Rather than go through the frustration of a formal definition for what is file-like, stay productive and flexible with duck typing. Objects that don't have the appropriate methods or attributes will cause a TypeError or AttributeError. If you're passing in the wrong kind of object, you'll find out almost as soon as you write the code (because you test early and often), just as you would with a statically analyzed type system. From rosuav at gmail.com Tue Aug 9 14:57:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 04:57:40 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 4:51 AM, Michael Selik wrote: > "File-like" is a good example. Rather than go through the frustration of a > formal definition for what is file-like, stay productive and flexible with > duck typing. Objects that don't have the appropriate methods or attributes > will cause a TypeError or AttributeError. If you're passing in the wrong > kind of object, you'll find out almost as soon as you write the code > (because you test early and often), just as you would with a statically > analyzed type system. It's slightly delayed; with Python, you have to actually execute the code path in question, whereas a static type system would detect it at the compilation stage. So it may depend on the quality of your testing. MyPy may be able to help there. ChrisA From no.email at nospam.invalid Tue Aug 9 15:06:28 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 09 Aug 2016 12:06:28 -0700 Subject: What's the best way to minimize the need of run time checks? References: Message-ID: <8737md3gp7.fsf@jester.gateway.pace.com> Juan Pablo Romero M?ndez writes: > In online forums sometimes people complain that they end up having to > test constantly for None That's something of a style issue. You can code in a way that avoids a lot of those tests (not all of them). > Do you guys have any resources you like that addresses the following > issue: What is the best way to use the dynamic features of Python to > avoid having to write a poor's man type system? In practice dynamically typed code often is monomorphic anyway. Dynamism shows up in things like deserializers. From python at mrabarnett.plus.com Tue Aug 9 15:22:20 2016 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 9 Aug 2016 20:22:20 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On 2016-08-09 19:57, Chris Angelico wrote: > On Wed, Aug 10, 2016 at 4:51 AM, Michael Selik wrote: >> "File-like" is a good example. Rather than go through the frustration of a >> formal definition for what is file-like, stay productive and flexible with >> duck typing. Objects that don't have the appropriate methods or attributes >> will cause a TypeError or AttributeError. If you're passing in the wrong >> kind of object, you'll find out almost as soon as you write the code >> (because you test early and often), just as you would with a statically >> analyzed type system. > > It's slightly delayed; with Python, you have to actually execute the > code path in question, whereas a static type system would detect it at > the compilation stage. So it may depend on the quality of your > testing. MyPy may be able to help there. > Static checking won't tell you if, say, you're adding when you should be subtracting, so you'd need to execute the code path in question anyway during testing! From jpablo.romero at gmail.com Tue Aug 9 15:29:37 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 12:29:37 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <8737md3gp7.fsf@jester.gateway.pace.com> References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: 2016-08-09 12:06 GMT-07:00 Paul Rubin : > Juan Pablo Romero M?ndez writes: > > In online forums sometimes people complain that they end up having to > > test constantly for None > > That's something of a style issue. You can code in a way that avoids a > lot of those tests (not all of them). > This is exactly what I'm looking for :). Do you have any resource (blog / book, etc) that discusses this style? > > > Do you guys have any resources you like that addresses the following > > issue: What is the best way to use the dynamic features of Python to > > avoid having to write a poor's man type system? > > In practice dynamically typed code often is monomorphic anyway. > Dynamism shows up in things like deserializers. > -- > https://mail.python.org/mailman/listinfo/python-list > From jpablo.romero at gmail.com Tue Aug 9 15:39:55 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 12:39:55 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: What static checking can actually guarantee varies depending on the specific type system at hand (C# vs Haskell vs Idris for example). But most of them can guarantee simple stuff like: "I'm I allowed to invoke this function at this point?" If you don't have that, well you can rely on tests to show that certain code paths behave in a certain way when exercised with certain inputs. 2016-08-09 12:22 GMT-07:00 MRAB : > On 2016-08-09 19:57, Chris Angelico wrote: > >> On Wed, Aug 10, 2016 at 4:51 AM, Michael Selik >> wrote: >> >>> "File-like" is a good example. Rather than go through the frustration of >>> a >>> formal definition for what is file-like, stay productive and flexible >>> with >>> duck typing. Objects that don't have the appropriate methods or >>> attributes >>> will cause a TypeError or AttributeError. If you're passing in the wrong >>> kind of object, you'll find out almost as soon as you write the code >>> (because you test early and often), just as you would with a statically >>> analyzed type system. >>> >> >> It's slightly delayed; with Python, you have to actually execute the >> code path in question, whereas a static type system would detect it at >> the compilation stage. So it may depend on the quality of your >> testing. MyPy may be able to help there. >> >> Static checking won't tell you if, say, you're adding when you should be > subtracting, so you'd need to execute the code path in question anyway > during testing! > > -- > https://mail.python.org/mailman/listinfo/python-list > From havocjoseph at gmail.com Tue Aug 9 15:47:22 2016 From: havocjoseph at gmail.com (Joseph Bane) Date: Tue, 9 Aug 2016 15:47:22 -0400 Subject: Question regarding stdlib distutils strtobool behavior In-Reply-To: References: <55cb5d28-b8c5-45f1-a5cc-ccb8dfe39dba@googlegroups.com> Message-ID: On 8/9/16 1:42 PM, Terry Reedy wrote: > On 8/9/2016 9:22 AM, Joseph Bane wrote: >> Hello. >> >> It recently came to my attention that the strtobool function in the >> standard library doesn't return Python native boolean values, but >> rather returns integer 0 or 1: >> >> https://hg.python.org/cpython/file/3.5/Lib/distutils/util.py#l304 >> >> I am curious why this is the defined behavior and whether anyone can >> fill me in regarding this approach. For clarity, I would expect the >> code to `return True` and `return False` rather than `return 1` and >> `return 0`. > > I am rather sure distutils predates 2.2 and the addition of bool. Much > code has been converted from 0/1 to False/True. I don't know if this > particular holdover is intentional -- fear of breaking something -- or > oversight/inertia. > > I searched the tracker for 'strtobool' in component 'distutiles', status > "don't care", and there were no hits. (Search is not perfect, though.) > > Thanks for the legwork here! I have created the following issue after I read this: http://bugs.python.org/issue27721 Hopefully it will get some more discussion going. Thanks also to Michael Selik for his great advice in his last response! I think it's great we're having such positive conversations! From rgaddi at highlandtechnology.invalid Tue Aug 9 16:18:34 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 9 Aug 2016 20:18:34 -0000 (UTC) Subject: What's the best way to minimize the need of run time checks? References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: Juan Pablo Romero M?ndez wrote: > 2016-08-09 12:06 GMT-07:00 Paul Rubin : > >> Juan Pablo Romero M?ndez writes: >> > In online forums sometimes people complain that they end up having to >> > test constantly for None >> >> That's something of a style issue. You can code in a way that avoids a >> lot of those tests (not all of them). >> > > > This is exactly what I'm looking for :). Do you have any resource (blog / > book, etc) that discusses this style? > It's not a style, it's the absence of one. def add2list(lst, elem): lst.extend([elem, elem]) return lst I did all the type checking I needed to there; none whatsoever. If passed a list, or something that behaves like one, that does the expected thing. If passed an ExtensionLadder, it probably does the wrong thing, but that is no way my problem. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ltc.hotspot at gmail.com Tue Aug 9 16:20:35 2016 From: ltc.hotspot at gmail.com (Ltc Hotspot) Date: Tue, 9 Aug 2016 13:20:35 -0700 (PDT) Subject: Invalid Syntax Message-ID: <1dfa6452-8db1-48cb-bc40-d762425c4547@googlegroups.com> Hi, Everyone: What is the source of the following, 'error message: SyntaxError: invalid syntax (, line 2)' v. Python 3.3 Code reads: x=1 if x==1 # indent 4 spaces print "x = 1" Hal From rgaddi at highlandtechnology.invalid Tue Aug 9 16:22:37 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 9 Aug 2016 20:22:37 -0000 (UTC) Subject: Invalid Syntax References: <1dfa6452-8db1-48cb-bc40-d762425c4547@googlegroups.com> Message-ID: Ltc Hotspot wrote: > Hi, Everyone: > > What is the source of the following, > 'error message: SyntaxError: invalid syntax (, line 2)' > > v. Python 3.3 > > Code reads: > > x=1 > if x==1 > # indent 4 spaces > print "x = 1" > > Hal A missing colon, the appropriate location of which is left as an exercise to the reader. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From robin.koch at t-online.de Tue Aug 9 16:30:11 2016 From: robin.koch at t-online.de (Robin Koch) Date: Tue, 9 Aug 2016 22:30:11 +0200 Subject: cropping a random part of an image In-Reply-To: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: Am 09.08.2016 um 14:40 schrieb drewes.mil at gmail.com: > I'm new to python and I have 30.000 pictures. > I need to crop, let's say 100, parts of 256x256, randomly out of every picture. Interessting task. May I ask for the purose of this? > I cant find an answer in the net, would be nice if someone could help me out! Although I think you should follow Peters advise I have put something together, out of curiousity, that should do the trick. There are several ways to improve it. First I'm using os.listdir() instead of os.walk(). In my example scenario that's ok. If your files are spread over different subfolders, os.walk() is the better way to do it. You could add a counter (see: enumerate()) to have a better overview over the progress. Also you might have other preferences for the location of the tiles. (e.g. one folder per image). # Cuts random tiles from pictures # # This version works with approx. 270 tiles per second on my machine # (on 13.5MPx images). So 3 million tiles should take about 3 hours. import random, os, time from PIL import Image INPATH = r".../images" OUTPATH = r".../tiles" dx = dy = 256 tilesPerImage = 100 files = os.listdir(INPATH) numOfImages = len(files) t = time.time() for file in files: with Image.open(os.path.join(INPATH, file)) as im: for i in range(1, tilesPerImage+1): newname = file.replace('.', '_{:03d}.'.format(i)) w, h = im.size x = random.randint(0, w-dx-1) y = random.randint(0, h-dy-1) print("Cropping {}: {},{} -> {},{}".format(file, x,y, x+dx, y+dy)) im.crop((x,y, x+dx, y+dy))\ .save(os.path.join(OUTPATH, newname)) t = time.time()-t print("Done {} images in {:.2f}s".format(numOfImages, t)) print("({:.1f} images per second)".format(numOfImages/t)) print("({:.1f} tiles per second)".format(tilesPerImage*numOfImages/t)) -- Robin Koch From michael.selik at gmail.com Tue Aug 9 16:37:44 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 20:37:44 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Tue, Aug 9, 2016 at 2:59 PM Chris Angelico wrote: > On Wed, Aug 10, 2016 at 4:51 AM, Michael Selik > wrote: > > "File-like" is a good example. Rather than go through the frustration of > a > > formal definition for what is file-like, stay productive and flexible > with > > duck typing. Objects that don't have the appropriate methods or > attributes > > will cause a TypeError or AttributeError. If you're passing in the wrong > > kind of object, you'll find out almost as soon as you write the code > > (because you test early and often), just as you would with a statically > > analyzed type system. > > It's slightly delayed; with Python, you have to actually execute the > code path in question, whereas a static type system would detect it at > the compilation stage. So it may depend on the quality of your > testing. MyPy may be able to help there. > Depends on how long it takes to compile! https://xkcd.com/303/ :-) From jpablo.romero at gmail.com Tue Aug 9 16:51:23 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 13:51:23 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: 2016-08-09 13:18 GMT-07:00 Rob Gaddi : > Juan Pablo Romero M?ndez wrote: > > > 2016-08-09 12:06 GMT-07:00 Paul Rubin : > > > >> Juan Pablo Romero M?ndez writes: > >> > In online forums sometimes people complain that they end up having to > >> > test constantly for None > >> > >> That's something of a style issue. You can code in a way that avoids a > >> lot of those tests (not all of them). > >> > > > > > > This is exactly what I'm looking for :). Do you have any resource (blog > / > > book, etc) that discusses this style? > > > > It's not a style, it's the absence of one. > > def add2list(lst, elem): > lst.extend([elem, elem]) > return lst > > I did all the type checking I needed to there; none whatsoever. If > passed a list, or something that behaves like one, that does the > expected thing. > > If passed an ExtensionLadder, it probably does the wrong thing, but that > is no way my problem. > So as the writer of the function you expect the user to read the function body to determine what is safe to pass or not? > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. > -- > https://mail.python.org/mailman/listinfo/python-list > From random832 at fastmail.com Tue Aug 9 16:57:15 2016 From: random832 at fastmail.com (Random832) Date: Tue, 09 Aug 2016 16:57:15 -0400 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: <1470776235.1196953.690702457.39925CB3@webmail.messagingengine.com> On Tue, Aug 9, 2016, at 16:51, Juan Pablo Romero M?ndez wrote: > So as the writer of the function you expect the user to read the function > body to determine what is safe to pass or not? How about expecting them to read the docstring? From michael.selik at gmail.com Tue Aug 9 17:01:17 2016 From: michael.selik at gmail.com (Michael Selik) Date: Tue, 09 Aug 2016 21:01:17 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Tue, Aug 9, 2016 at 3:22 PM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > I'm actually looking for ways to minimize run time errors, so that would > include TypeError and AttributeError. > > In your "File-like" example having type information would prevent me from > even passing the wrong object, so I don't need to test for that particular > problem. > Let's clarify this discussion by using the word "check" for type-checks that necessary for the application to behave as desired and using the word "test" for checking application behavior under normal and edge cases, such as unexpected types being passed. Are you saying you want to minimize type-checks during the execution of your application? Or do you want to minimize the amount of testing code you write? I don't check for file-likeness in the application code. If somehow a non-file-like (file-unlike?) object makes its way into my function, I expect (and hope) my program will immediately stop and print a nice traceback for me. If I wanted some other behavior, I probably would already have try/except surrounding the appropriate section to deal with not-found or permissions errors. If I need to, I'll add TypeError and AttributeError to that except statement. I sometimes write tests for unexpected inputs, checking to ensure that either a TypeError or AttributeError is raised. However, sometimes I'm not worried about it. If the user gives me bizarre input, they should know to expect (possibly) bizarre results. Who knows, maybe that's what they wanted. I would add type-checking or value-checking to my application if the wrong type or value does not raise an error but instead causes an infinite loop or corrupts data. Those situations are fairly rare. On Tue, Aug 9, 2016 at 4:52 PM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > So as the writer of the function you expect the user to read the function > body to determine what is safe to pass or not? No. But I usually expect them to not freak out when they see a traceback. Or if they do freak out, I want them to freak out and send a bug report. From rgaddi at highlandtechnology.invalid Tue Aug 9 17:02:05 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 9 Aug 2016 21:02:05 -0000 (UTC) Subject: What's the best way to minimize the need of run time checks? References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: Juan Pablo Romero M?ndez wrote: > 2016-08-09 13:18 GMT-07:00 Rob Gaddi : > >> Juan Pablo Romero M?ndez wrote: >> >> > 2016-08-09 12:06 GMT-07:00 Paul Rubin : >> > >> >> Juan Pablo Romero M?ndez writes: >> >> > In online forums sometimes people complain that they end up having to >> >> > test constantly for None >> >> >> >> That's something of a style issue. You can code in a way that avoids a >> >> lot of those tests (not all of them). >> >> >> > >> > >> > This is exactly what I'm looking for :). Do you have any resource (blog >> / >> > book, etc) that discusses this style? >> > >> >> It's not a style, it's the absence of one. >> >> def add2list(lst, elem): >> lst.extend([elem, elem]) >> return lst >> >> I did all the type checking I needed to there; none whatsoever. If >> passed a list, or something that behaves like one, that does the >> expected thing. >> >> If passed an ExtensionLadder, it probably does the wrong thing, but that >> is no way my problem. >> > > > So as the writer of the function you expect the user to read the function > body to determine what is safe to pass or not? > No, but I expect them to read the function documentation. The alternative is to expect that the user is going to go around trying to randomly call functions with whatever objects they happen to have lying around in an attempt to figure out what they do. The docstring for the above function would probably be """Append two copies of elem to the end of list lst.""" This goes back to the "consenting adults" principle in Python. Assume that the person calling your functions isn't a blithering moron in need of protection from themselves. Sometimes you'll be wrong in that assumption, but their life as a blithering moron is already quite difficult and so you won't have substantially added to that by failing to put guard rails up on your function. Conversely, if you explicitly check isinstance(lst, list) then I can't, as one clever SOB, pass it a list-like object and have it just work. I instead would have to have my object expressly subclass list, even if I had a reason not to. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From hongonkodon at gmx.com Tue Aug 9 17:11:56 2016 From: hongonkodon at gmx.com (STEVEN DOWNHAM CANTOR FITZGERALD EX SG WARBURG) Date: Tue, 9 Aug 2016 14:11:56 -0700 (PDT) Subject: Redacted Message-ID: <363bd267-f92c-4e5f-919d-76f1e264fa93@googlegroups.com> Redacted From chivalry at mac.com Tue Aug 9 18:00:51 2016 From: chivalry at mac.com (Charles Ross) Date: Tue, 09 Aug 2016 15:00:51 -0700 Subject: Call for Assistance In-Reply-To: References: Message-ID: <517DCCE7-1E0F-40D9-9F74-7794ED82F89B@mac.com> > CC-BY-NC-SA is not a license for free (as in speech) content. Is that > what you want? I really appreciate all the conversation about the license. Perhaps I made a mistake with requiring noncommercial in the license, I don?t know, but I?ll look at the links provided about free documentation and free software and see if I can understand the issues more and perhaps make a better decision. Honestly, I have no expectation that this book will ever be anything more than a (hopefully) useful resource for others like myself, as described in the current introduction. I might have a remote hope that it would be something that would receive attention from a traditional publisher, but that remote hope is in the same category as a remote hope that a distant rich uncle will someday name me in his well. So, given the minuscule odds of that happening, I?ll probably remove the NC portion. I?ll just do a bit of research first with those links. Thanks, Chuck From cs at zip.com.au Tue Aug 9 19:05:43 2016 From: cs at zip.com.au (cs at zip.com.au) Date: Wed, 10 Aug 2016 09:05:43 +1000 Subject: Invalid Syntax In-Reply-To: References: Message-ID: <20160809230543.GA12664@cskk.homeip.net> On 09Aug2016 20:22, Rob Gaddi wrote: >Ltc Hotspot wrote: >> What is the source of the following, >> 'error message: SyntaxError: invalid syntax (, line 2)' >> v. Python 3.3 >> >> Code reads: >> >> x=1 >> if x==1 >> # indent 4 spaces >> print "x = 1" > >A missing colon, the appropriate location of which is left as an >exercise to the reader. A bit unfair, it is not the most helpful of syntax complaints. Hal: all Python compound statements (if, while, for, with etc) end in a colon and an indented "suite" (enclosed chunk of statements). So: if x == 1: ...... Also, in Python 3 "print" is a function, so: print("x = 1") Cheers, Cameron Simpson From chivalry at mac.com Tue Aug 9 20:09:03 2016 From: chivalry at mac.com (Charles Ross) Date: Tue, 09 Aug 2016 17:09:03 -0700 Subject: Call for Assistance In-Reply-To: <517DCCE7-1E0F-40D9-9F74-7794ED82F89B@mac.com> References: <517DCCE7-1E0F-40D9-9F74-7794ED82F89B@mac.com> Message-ID: <1CB4A3ED-4733-4F7A-99BC-8C09269EECB3@mac.com> Well, I?ve been convinced. The license for the book is now Creative Commons Attribution-ShareAlike. https://github.com/chivalry/meta-python/blob/master/LICENSE.md Thanks, Chuck > On Aug 9, 2016, at 3:00 PM, Charles Ross wrote: > >> CC-BY-NC-SA is not a license for free (as in speech) content. Is that >> what you want? > > I really appreciate all the conversation about the license. Perhaps I made a mistake with requiring noncommercial in the license, I don?t know, but I?ll look at the links provided about free documentation and free software and see if I can understand the issues more and perhaps make a better decision. > > Honestly, I have no expectation that this book will ever be anything more than a (hopefully) useful resource for others like myself, as described in the current introduction. I might have a remote hope that it would be something that would receive attention from a traditional publisher, but that remote hope is in the same category as a remote hope that a distant rich uncle will someday name me in his well. So, given the minuscule odds of that happening, I?ll probably remove the NC portion. I?ll just do a bit of research first with those links. > > Thanks, > Chuck > -- > https://mail.python.org/mailman/listinfo/python-list From jpablo.romero at gmail.com Tue Aug 9 20:58:51 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 17:58:51 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: 2016-08-09 14:01 GMT-07:00 Michael Selik : > On Tue, Aug 9, 2016 at 3:22 PM Juan Pablo Romero M?ndez < > jpablo.romero at gmail.com> wrote: > >> I'm actually looking for ways to minimize run time errors, so that would >> include TypeError and AttributeError. >> >> In your "File-like" example having type information would prevent me from >> even passing the wrong object, so I don't need to test for that particular >> problem. >> > > Let's clarify this discussion by using the word "check" for type-checks > that necessary for the application to behave as desired and using the word > "test" for checking application behavior under normal and edge cases, such > as unexpected types being passed. > > Are you saying you want to minimize type-checks during the execution of > your application? Or do you want to minimize the amount of testing code you > write? > My goal is determine if there is a style of writing code in a dynamic language that minimizes the occurrence of runtime errors AND avoids two things: 1) using runtime checks all over the place (either ad hoc or using tools like Valideer) 2) attempting to test for things that a type checker would normally do (not that this is completely possible anyway) (I'm not suggesting I want to avoid writing tests at all, since there are many things the type checker can't possible check). > > I don't check for file-likeness in the application code. If somehow a > non-file-like (file-unlike?) object makes its way into my function, I > expect (and hope) my program will immediately stop and print a nice > traceback for me. If I wanted some other behavior, I probably would already > have try/except surrounding the appropriate section to deal with not-found > or permissions errors. If I need to, I'll add TypeError and AttributeError > to that except statement. > > This is interesting. You are Ok having runtime errors? > I sometimes write tests for unexpected inputs, checking to ensure that > either a TypeError or AttributeError is raised. However, sometimes I'm not > worried about it. If the user gives me bizarre input, they should know to > expect (possibly) bizarre results. Who knows, maybe that's what they wanted. > > I would add type-checking or value-checking to my application if the > wrong type or value does not raise an error but instead causes an infinite > loop or corrupts data. Those situations are fairly rare. > > > > On Tue, Aug 9, 2016 at 4:52 PM Juan Pablo Romero M?ndez < > jpablo.romero at gmail.com> wrote: > >> So as the writer of the function you expect the user to read the function >> body to determine what is safe to pass or not? > > > No. But I usually expect them to not freak out when they see a traceback. > Or if they do freak out, I want them to freak out and send a bug report. > From steve+python at pearwood.info Tue Aug 9 20:59:47 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 10:59:47 +1000 Subject: Invalid Syntax References: <1dfa6452-8db1-48cb-bc40-d762425c4547@googlegroups.com> Message-ID: <57aa7c86$0$1585$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 06:20 am, Ltc Hotspot wrote: > Hi, Everyone: > > What is the source of the following, > 'error message: SyntaxError: invalid syntax (, line 2)' > > v. Python 3.3 > > Code reads: > > x=1 > if x==1 > # indent 4 spaces > print "x = 1" > > Hal The error message tells you exactly which line the error is on: line 2. Have you done a tutorial to learn how to write Python code? If you haven't, you should: https://docs.python.org/3/tutorial/ The error is that all statements that introduce a new indented block require a colon: class C: def func(): if condition: for x in sequence: # etc. Also, print() is a function, you need round brackets (parentheses): print("x = 1") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Aug 9 21:02:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 11:02:07 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 10:58 AM, Juan Pablo Romero M?ndez wrote: >> I don't check for file-likeness in the application code. If somehow a >> non-file-like (file-unlike?) object makes its way into my function, I >> expect (and hope) my program will immediately stop and print a nice >> traceback for me. If I wanted some other behavior, I probably would already >> have try/except surrounding the appropriate section to deal with not-found >> or permissions errors. If I need to, I'll add TypeError and AttributeError >> to that except statement. >> >> > This is interesting. You are Ok having runtime errors? Absolutely. In real terms, there's no difference between "compile-time error" and "run-time error that you trigger the moment you run your program". ChrisA From ben+python at benfinney.id.au Tue Aug 9 21:09:09 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 10 Aug 2016 11:09:09 +1000 Subject: Specify the database for a Django ModelForm instance References: <85wpjq22xd.fsf@benfinney.id.au> Message-ID: <851t1x1lca.fsf@benfinney.id.au> Phil Boutros writes: > Ben Finney wrote: > > How can I specify which database (by its alias name) a Django > > ModelForm should use > > > What is the equivalent for using='foo' when instantiating a ModelForm > > for the model, or calling its methods (ModelForm.clean, ModelForm.save, > > etc.)? > > You would specify it on the save method (using the same > "using='foo'") paremeter. Thank you for responding. > http://stackoverflow.com/questions/37740848/use-a-specific-database-with-a-modelform The person who wrote that question must think very like myself. That doesn't address the question about ?ModelForm.clean? or ?ModelForm.is_valid?, though; how to tell the instance which database to use for those? -- \ ?I love and treasure individuals as I meet them, I loathe and | `\ despise the groups they identify with and belong to.? ?George | _o__) Carlin, 2007 | Ben Finney From steve+python at pearwood.info Tue Aug 9 21:28:41 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 11:28:41 +1000 Subject: What's the best way to minimize the need of run time checks? References: Message-ID: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 04:29 am, Juan Pablo Romero M?ndez wrote: > Hello, > > In online forums sometimes people complain that they end up having to test > constantly for None, Then don't return None to indicate an error. If you write this: def create_spam(arg): if condition: return Spam(arg) else: return None then you are doing it wrong. It sounds like they are writing C in Python, where they think that returning None (instead of a null pointer) is the way to indicate an error. Instead: def create_spam(arg): if condition: return Spam(arg) raise MyExceptionError('cannot create spam because reasons') Now you can decide on a error-handling scheme: - try...except and handle the exception exactly when it occurs; - wrap your entire application in a single try...except or exception handler, and catch the exception in a single place; - don't even bother catching the exception in the application at all, treat it as a debugging aid. Any time your tests generate that exception, that's a bug to be fixed. There's no reason why you can't decide on a combination of all three: - the app catches the exception, logs it as a bug, and exits; - but individual parts of your code base treat this exception as an EXPECTED error, wrapping that individual call in a try...except: try: myspam = create_spam(99) except MyExceptionError: # Only occurs when the server has hung reboot_server() myspam = create_spam(99) Only catch exceptions you expect, and can recover from. If your *data* could be None, well that's no different from any other sentinel or missing value. You have to deal with it somewhere. One way is to filter it early: # filter out None data = [obj for obj in data if data is not None] # or if you prefer data = filter(lambda obj: obj is not None, data) and now you can be sure that data doesn't contain None. [...] > Do you guys have any resources you like that addresses the following > issue: What is the best way to use the dynamic features of Python to avoid > having to write a poor's man type system? This is an old, old, old debate that applies to Lisp, Perl, Javascript, Lua, Ruby and others, not just Python. Google for "dynamic typing versus static typing" for more. Also google for "duck typing". One useful perspective is that languages are converging on a mix of both: statically typed languages are growing dynamic features, and dynamically typed languages are growing static checkers. http://steve-yegge.blogspot.com.au/2008/05/dynamic-languages-strike-back.html And some background information on the debate: https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Aug 9 21:29:28 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 11:29:28 +1000 Subject: What's the best way to minimize the need of run time checks? References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: <57aa8378$0$1599$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 06:51 am, Juan Pablo Romero M?ndez wrote: > 2016-08-09 13:18 GMT-07:00 Rob Gaddi : >> It's not a style, it's the absence of one. >> >> def add2list(lst, elem): >> lst.extend([elem, elem]) >> return lst >> >> I did all the type checking I needed to there; none whatsoever. If >> passed a list, or something that behaves like one, that does the >> expected thing. >> >> If passed an ExtensionLadder, it probably does the wrong thing, but that >> is no way my problem. > > So as the writer of the function you expect the user to read the function > body to determine what is safe to pass or not? No. We expect them to read the docs, which will say something like "expects a list". Anything that behaves like a list *should* do: if something behaves like a list, then it might as well be a list, regardless of whether or not it inherits from the list built-in type or not. This is called Duck Typing: "If it quacks like a duck, swims like a duck and walks like a duck, it might as well be a duck." If your function wants an object it can call extend on, then why should it demand a list? It should only care about whether or not the object has an extend method. Some functions may be stricter, either from some specific need, or just because the author is a refugee from a statically typed language who thinks that they need to type-check *everything*. (May the gods preserve us from such people.) In that case, your testing will quickly reveal that a list-like object is not good enough, only an actual list will do. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Tue Aug 9 21:45:36 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 18:45:36 -0700 (PDT) Subject: ctypes And The WACAH Principle Message-ID: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> GvR has said ?we?re all consenting adults here?. He was referring to the ?public/private/protected? access-control business that most object-oriented languages require you to go through. But there is another area where I think this principle applies to Python, at least in part: low-level access to arbitrary memory. For the most part, Python tries to avoid you having to deal with this. One important exception is the wonderful and useful ctypes module , which by its very nature, requires you to worry about such things. But what?s so nice about ctypes is that you can wrap up such concerns in a Pythonic, high-level layer that others can easily use without knowing that such machinations are going on behind the scenes. However, precisely because of the lack of access control, the abstraction is leaky, so users can penetrate it if they feel they have a need to, and are prepared to face the consequences--so WACAH very much applies here. When creating ctypes wrappers for libraries, one often comes across a call to the effect of ?create object referencing given data?. That is, the caller supplies (and manages) the block of memory holding the data. For example, the Cairo graphics library has cairo_image_surface_create_for_data , which treats the supplied block of memory as an array of pixels. Cairo also thoughtfully provides additional mechanisms to coordinate between its accesses, and those of the caller, to that block of memory. For the most part, Python doesn?t make it easy to allocate an object and get access to its internal memory representation. One interesting exception is array.array objects , which have the buffer_info() method, directly returning both the address and length of the underlying memory block. The documentation for buffer_info() says: When using array objects from code written in C or C++ (the only way to effectively make use of this information), it makes more sense to use the buffer interface supported by array objects. Ah, I wish... I had a look at the documentation for the buffer interface and memory views, and found no way to make use of them from pure Python. Basically, the paragraph I quoted above is wrong in just about every important detail. So, for now, if I want to use Python-allocated memory with lower-level APIs via ctypes, the array module remains the way to go. From steve+python at pearwood.info Tue Aug 9 21:47:57 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 11:47:57 +1000 Subject: What's the best way to minimize the need of run time checks? References: Message-ID: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 11:02 am, Chris Angelico wrote: > On Wed, Aug 10, 2016 at 10:58 AM, Juan Pablo Romero M?ndez > wrote: >> This is interesting. You are Ok having runtime errors? > > Absolutely. In real terms, there's no difference between "compile-time > error" and "run-time error that you trigger the moment you run your > program". That's an oversimplification. How about the difference between getting a compile-time error immediately you try to compile your program, and a run-time error three quarters of the way through processing a billion records, leaving your data in a corrupted state? You can say that this scenario is "rare", and you're probably right. But that doesn't justify a statement like it is ABSOLUTELY okay to have runtime errors (for programming bugs). I love the fact that the computer on the Apollo lunar landers was expected to have bugs, and was designed to automatically reboot and continue the calculation that was interrupted. By memory, it rebooted something like 30 or 40 times during the first moon landing. That's awesome. But its also unusual. Most software isn't designed to recover from errors quietly and silently without any fuss. But generally speaking, the only acceptable run-time errors are those that occur due to information that are not available until run-time. If the user insists on passing you an non-existent file name for reading, then what else are you going to do other than fail with a run-time error? Anything else is a *bug*, and while bugs are a fact of life, they aren't "acceptable". As a matter of principle[1], we shouldn't release software with bugs if they can be detected by either compile-time checks or unit testing. *Especially* if the consequences of a run-time bug are severe. [1] As opposed to a matter of pragmatics. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Tue Aug 9 21:50:43 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 18:50:43 -0700 (PDT) Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: On Wednesday, August 10, 2016 at 12:44:30 AM UTC+12, Martin Sch??n wrote: > What I have failed to achieve is a graph with a transparent > background. While it is possible to render image frames with alpha transparency channels, as far as I know none of the motion-video image formats supports transparency. They all assume that the image fills the entire frame and completely covers anything behind. You could invent your own player which draws the frames one after another itself. Note that the JPEG image format doesn?t support transparency; you would have to use a bulkier format like PNG. Another option would be to have an additional video track where the image is interpreted purely as transparency masking information for the ?proper? video track. You would still need to write your own player to handle this. From steve+python at pearwood.info Tue Aug 9 22:02:19 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 12:02:19 +1000 Subject: ctypes And The WACAH Principle References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: <57aa8b2c$0$1614$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 11:45 am, Lawrence D?Oliveiro wrote: > GvR has said ?we?re all consenting adults here?. He was referring to the > ?public/private/protected? access-control business that most > object-oriented languages require you to go through. But there is another > area where I think this principle applies to Python, at least in part: > low-level access to arbitrary memory. Well, that's certainly an idiosyncratic position to take. Most people believe that high-level languages like Python should insulate the programmer from low-level details, and avoid the risk of memory corruption, buffer overflows, segmentation faults and core dumps. [...] > For the most part, Python doesn?t make it easy to allocate an object and > get access to its internal memory representation. One interesting > exception is array.array objects > , which have the > buffer_info() method, directly returning both the address and length of > the underlying memory block. The documentation for buffer_info() says: > > When using array objects from code written in C or C++ (the only way > to effectively make use of this information), it makes more sense to > use the buffer interface supported by array objects. > > Ah, I wish... I had a look at the documentation for the buffer interface > and memory views, and found no way to make use of them from pure Python. > Basically, the paragraph I quoted above is wrong in just about every > important detail. I don't understand. The documentation says that the ONLY way to use that information is from C or C++. You have found no other way to use that information. How is that "wrong"? How are *you* interpreting that documentation? What are you trying (and failing) to do? > So, for now, if I want to use Python-allocated memory with lower-level > APIs via ctypes, the array module remains the way to go. I really don't understand what you are trying to say. Perhaps a working code sample might help. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Aug 9 22:02:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 12:02:37 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 10, 2016 at 11:47 AM, Steven D'Aprano wrote: > On Wed, 10 Aug 2016 11:02 am, Chris Angelico wrote: > >> On Wed, Aug 10, 2016 at 10:58 AM, Juan Pablo Romero M?ndez >> wrote: > >>> This is interesting. You are Ok having runtime errors? >> >> Absolutely. In real terms, there's no difference between "compile-time >> error" and "run-time error that you trigger the moment you run your >> program". > > That's an oversimplification. > > How about the difference between getting a compile-time error immediately > you try to compile your program, and a run-time error three quarters of the > way through processing a billion records, leaving your data in a corrupted > state? > > You can say that this scenario is "rare", and you're probably right. But > that doesn't justify a statement like it is ABSOLUTELY okay to have runtime > errors (for programming bugs). Let me invert that statement, then, and assert: It is NEVER acceptable to have runtime errors for programming bugs. Okay. What does that imply? That *every single programming bug* can be detected at compile time. Do you want to guarantee that? I certainly don't, because it would shackle the program to "stuff we can mathematically verify", which is a lot narrower than "stuff we can currently code in Python". So if you can't prove a program to be bug-free at compile time, what *do* you want to happen with that undetected bug? I want a run-time error, not a crash and certainly not the wrong behaviour. Also, you'll notice that I said "the moment you run your program" - that is, I'm talking about stuff that happens "at run time" but early on, rather than waiting for data-specific situations. Ideally, you'll have a test suite that exercises most, if not all, of your code paths, and running your tests can be done at what's effectively the same as "compile time" (with the significant difference that you can start messing with the program without waiting for the tests to finish, whereas it's annoyingly difficult to run a program that hasn't finished compiling yet). I never said that "run-time errors are preferable to compile-time errors", because they're not; the sooner you catch something, the better. But if you don't catch it early, yes, I am definitely okay with a run-time error with a nice exception traceback and so on. ChrisA From rosuav at gmail.com Tue Aug 9 22:04:26 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 12:04:26 +1000 Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 11:50 AM, Lawrence D?Oliveiro wrote: > On Wednesday, August 10, 2016 at 12:44:30 AM UTC+12, Martin Sch??n wrote: > >> What I have failed to achieve is a graph with a transparent >> background. > > While it is possible to render image frames with alpha transparency channels, as far as I know none of the motion-video image formats supports transparency. They all assume that the image fills the entire frame and completely covers anything behind. > Animated GIFs support transparency. Are you happy with the framerates you can get in a simple GIF? But this isn't really a Python question. ChrisA From lawrencedo99 at gmail.com Tue Aug 9 22:49:07 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 19:49:07 -0700 (PDT) Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: On Wednesday, August 10, 2016 at 2:04:39 PM UTC+12, Chris Angelico wrote: > Animated GIFs support transparency. But they don?t do full colour. From lawrencedo99 at gmail.com Tue Aug 9 22:53:56 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 19:53:56 -0700 (PDT) Subject: Make sure you removed all debugging print statements error In-Reply-To: References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> Message-ID: <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> On Tuesday, August 9, 2016 at 4:20:37 AM UTC+12, Chris Angelico wrote: > Firstly, the bare "except:" clause should basically never be used; > instead, catch a very specific exception and figure out what you want to > do here Yes. > - but secondly, don't force people's names to be subdivided. I like the French convention . From steve+comp.lang.python at pearwood.info Wed Aug 10 00:46:28 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 14:46:28 +1000 Subject: How do I make a video animation with transparent background? References: Message-ID: <57aab1a5$0$11111$c3e8da3@news.astraweb.com> On Wednesday 10 August 2016 12:49, Lawrence D?Oliveiro wrote: > On Wednesday, August 10, 2016 at 2:04:39 PM UTC+12, Chris Angelico wrote: > >> Animated GIFs support transparency. > > But they don?t do full colour. Define "full colour". GIFs don't directly support 16-bit or 32-bit colours in a single frame, but they can support more than 256 colours: http://webmasters.stackexchange.com/questions/304/can-i-have-an-animated-gif-with-more-than-256-colours-per-frame including what looks to the naked eye (or at least *my* naked eye) effectively like 32-bit colour. -- Steve From sh.ajay12 at gmail.com Wed Aug 10 00:46:41 2016 From: sh.ajay12 at gmail.com (sh.ajay12 at gmail.com) Date: Tue, 9 Aug 2016 21:46:41 -0700 (PDT) Subject: cmd prompt does not recognizes python command on Windows 7 Message-ID: Hi Everyone i have installed python 3.5 , but the python command is not recognized C:\Users\sharmaaj>python 'python' is not recognized as an internal or external command, operable program or batch file. what should i do to run python commands. thanks everyone for reading my post. >From Ajay From lawrencedo99 at gmail.com Wed Aug 10 00:49:23 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 21:49:23 -0700 (PDT) Subject: Round-Trip Object Correspondence, Weakrefs And __new__ Message-ID: Qahirah is a Pythonic API binding for the Cairo 2D graphics library. When designing it, I tried to imagine how the graphics API would look if had been created for Python in the first place, rather than for C. One of the important decisions I made was to implement the binding in pure Python, using ctypes . ctypes is a wonderful library, and I recommend it to anyone else looking to create a Python API binding. There was already a commonly-used Python binding for Cairo, called PyCairo, written as an extension module in C. But it was missing important features, and had been neglected for some time. I did look at filling in some of the gaps , before realizing that it would be easier and quicker to do it over again in Python. A common principle in most ctypes bindings is: you encapsulate a lower-level API object in a Python object. For example, in Qahirah, a Cairo drawing context of type ?cairo_t? is wrapped in a Python object of type ?Context?. All such wrapper objects have an internal field ?_cairobj? which contains the pointer to the lower-level object. But sometimes you need to map back the other way. For example, a Context has a ?source? property, which is the current Pattern to use when setting pixel values. (A Pattern can be as simple as a single solid colour, or something more complex like a gradient or a repeating image.) In particular, I want the following result (assuming ?ctx? is a Context and ?pat? is a pattern): >>> ctx.source = pat # calls cairo_set_source >>> ctx.source is pat True One way to achieve this would be by cheating: each such settable property would save its value in an internal field, and the property getter would simply return this field value, rather than calling the underlying Cairo getter. This does run the risk of the Python wrapper getting out of sync with the underlying API object... The better way is for the class to maintain a dictionary mapping low-level pointers back to Python objects. The property getter would, in this case, call cairo_get_source as expected, then look up the result in this dictionary, and return the corresponding object. Entries in this dictionary would automatically be made every time a new object was created. To manage this, the object must define a __new__ method to manage object creation. Note the following from the docs: If __new__() returns an instance of cls, then the new instance?s __init__() method will be invoked. Since in this case __new__ might be returning either a new object or an existing one, you only want initialization to be done in the former case, not the latter one. But the __new__ method is the only place which knows which is the case; therefore it has to handle all the initialization, you cannot have an __init__ method at all. However, the problem with such a dictionary is that it leads to memory leaks: without some way of clearing entries from it, objects will never get deleted, so if the caller creates and forgets lots of them, memory usage will grow and grow. Clearing out unneeded objects is not a chore that the caller should have to manage. Luckily, Python, in common with most sanely-defined dynamic languages, supports something called ?weak? references . These are object references that do not count towards the usual reference management, so they do not prevent the object from being deleted. In particular, that module defines a handy class called a ?WeakValueDictionary?, where the values are weak object references. This means I can maintain my mapping from low-level API references to Python objects, and when the caller forgets about the latter, they will also disappear automatically from my mapping. One minor caveat is that use of weak references will cause an attribute called ?__weakref__? to be added to your objects. If you define __slots__ (as I have done here), you will need to make allowance for this. So, with all this, the basic object-creation mechanism in the Qahirah wrapper classes looks like _instances = WeakValueDictionary() # class variable mapping low-level API references back to Python objects def __new__(celf, _cairobj) : self = celf._instances.get(_cairobj) if self == None : self = super().__new__(celf) self._cairobj = _cairobj ... other internal setup ... celf._instances[_cairobj] = self else : cairo.cairo_xxx_destroy(self._cairobj) # lose extra reference created by caller #end if return \ self #end __new__ Then (continuing the same example) the Context method that calls cairo_get_source and returns a Pattern looks like this: @property def source(self) : "the current source Pattern." return \ Pattern(cairo.cairo_pattern_reference(cairo.cairo_get_source(self._cairobj))) #end source Note that Cairo maintains its own reference counting of objects. Naturally I rely on Python?s reference counting for my Python objects. But I still need to make sure Cairo objects do not leak, or get prematurely disposed. This is why the getter always calls the cairo_xxx_reference method to increment the reference count before instantiating the class: that way, the __new__ method doesn?t have to distinguish between genuinely new API object creation versus retrieving a previously-created API object; either way, it either has to save the reference it is given or (if a corresonding Python object already exists) get rid of it. The former may be the only possible case when creating a new API object, but it doesn?t have to care. From steve+comp.lang.python at pearwood.info Wed Aug 10 00:49:50 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 14:49:50 +1000 Subject: Make sure you removed all debugging print statements error References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: <57aab26f$0$11111$c3e8da3@news.astraweb.com> On Wednesday 10 August 2016 12:53, Lawrence D?Oliveiro wrote: > On Tuesday, August 9, 2016 at 4:20:37 AM UTC+12, Chris Angelico wrote: >> Firstly, the bare "except:" clause should basically never be used; >> instead, catch a very specific exception and figure out what you want to >> do here > > Yes. > >> - but secondly, don't force people's names to be subdivided. > > I like the French convention > . That is nice, for bicameral languages like English and French, although one does need to watch out for German ? (eszett) which makes it lossy. -- Steve From torriem at gmail.com Wed Aug 10 00:51:50 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 9 Aug 2016 22:51:50 -0600 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <8737md3gp7.fsf@jester.gateway.pace.com> Message-ID: On 08/09/2016 02:51 PM, Juan Pablo Romero M?ndez wrote: > 2016-08-09 13:18 GMT-07:00 Rob Gaddi : >> It's not a style, it's the absence of one. >> >> def add2list(lst, elem): >> lst.extend([elem, elem]) >> return lst >> >> I did all the type checking I needed to there; none whatsoever. If >> passed a list, or something that behaves like one, that does the >> expected thing. >> >> If passed an ExtensionLadder, it probably does the wrong thing, but that >> is no way my problem. >> > > > So as the writer of the function you expect the user to read the function > body to determine what is safe to pass or not? There are type annotations you can use in Python 3 that can help document the expected type that a function requires, and that information can be used by a linter to predict problems. Since Python allows duck typing, one could say that my function requires a "file-like" object. Not sure how that would be annotated. Some people go to more formal extremes and rather than just rely on duck typing they will use Java-style formal interfaces. So the function could require an object that implements certain interfaces. https://pypi.python.org/pypi/zope.interface. I suppose with interfaces, the function could query the arguments to look for these interfaces. But really in my mind that's no different than duck typing because if an object comes in that you can't deal with, that's by definition and exception anyway. So either Python raises it for you or you raise it yourself after checking for an interface. Same effect in my mind. From jpablo.romero at gmail.com Wed Aug 10 01:20:36 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Tue, 9 Aug 2016 22:20:36 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: 2016-08-09 18:28 GMT-07:00 Steven D'Aprano : > On Wed, 10 Aug 2016 04:29 am, Juan Pablo Romero M?ndez wrote: > > > Hello, > > > > In online forums sometimes people complain that they end up having to > test > > constantly for None, > > Then don't return None to indicate an error. If you write this: > > def create_spam(arg): > if condition: > return Spam(arg) > else: > return None > > > then you are doing it wrong. It sounds like they are writing C in Python, > where they think that returning None (instead of a null pointer) is the way > to indicate an error. Instead: > > def create_spam(arg): > if condition: > return Spam(arg) > raise MyExceptionError('cannot create spam because reasons') > > > Now you can decide on a error-handling scheme: > > - try...except and handle the exception exactly when it occurs; > > - wrap your entire application in a single try...except or > exception handler, and catch the exception in a single place; > > - don't even bother catching the exception in the application at all, > treat it as a debugging aid. Any time your tests generate that > exception, that's a bug to be fixed. > > > There's no reason why you can't decide on a combination of all three: > > - the app catches the exception, logs it as a bug, and exits; > > - but individual parts of your code base treat this exception as an > EXPECTED error, wrapping that individual call in a try...except: > > > try: > myspam = create_spam(99) > except MyExceptionError: > # Only occurs when the server has hung > reboot_server() > myspam = create_spam(99) > > > Only catch exceptions you expect, and can recover from. > > If your *data* could be None, well that's no different from any other > sentinel or missing value. You have to deal with it somewhere. One way is > to filter it early: > > # filter out None > data = [obj for obj in data if data is not None] > > # or if you prefer > data = filter(lambda obj: obj is not None, data) > > > and now you can be sure that data doesn't contain None. > > Ok, so you suggested 1) catching exceptions at the point where you care, 2) preemptively check for certain runtime conditions to avoid exceptions 3) write as many tests as possible 4) learn to live with runtime errors. Is that a fair characterization? > > [...] > > Do you guys have any resources you like that addresses the following > > issue: What is the best way to use the dynamic features of Python to > avoid > > having to write a poor's man type system? > > This is an old, old, old debate that applies to Lisp, Perl, Javascript, > Lua, > Ruby and others, not just Python. Google for "dynamic typing versus static > typing" for more. Also google for "duck typing". > > One useful perspective is that languages are converging on a mix of both: > statically typed languages are growing dynamic features, and dynamically > typed languages are growing static checkers. > > > http://steve-yegge.blogspot.com.au/2008/05/dynamic- > languages-strike-back.html > > > And some background information on the debate: > > https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From rustompmody at gmail.com Wed Aug 10 01:22:09 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 9 Aug 2016 22:22:09 -0700 (PDT) Subject: Individuals and groups (was Specify the database for a Django) In-Reply-To: References: <85wpjq22xd.fsf@benfinney.id.au> <851t1x1lca.fsf@benfinney.id.au> Message-ID: <17a1fdb5-daff-4ec6-b75e-4382bf4c7d0c@googlegroups.com> On Wednesday, August 10, 2016 at 6:39:31 AM UTC+5:30, Ben Finney wrote: > -- > \ ?I love and treasure individuals as I meet them, I loathe and | > `\ despise the groups they identify with and belong to.? ?George | > _o__) Carlin, 2007 | Nice quote ? thanks! From kent.tong.mo at gmail.com Wed Aug 10 01:22:36 2016 From: kent.tong.mo at gmail.com (Kent Tong) Date: Tue, 9 Aug 2016 22:22:36 -0700 (PDT) Subject: free python course materials for high school students available Message-ID: <6a267768-4c13-4c7d-b95d-7465f10be19e@googlegroups.com> Hi, I've taught a python course with some positive results to high school students with zero experience in programming. Now I am making these course materials (slides in English and lecture videos in Cantonese) freely available as a contribution back to the community (http://www.istudycenter.org/summercourses/programming-for-kids/python-materials). Hope you find them useful :-) From dmarv at dop.com Wed Aug 10 01:38:19 2016 From: dmarv at dop.com (Dale Marvin) Date: Tue, 9 Aug 2016 22:38:19 -0700 Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: <6cd773da-ad31-97ca-91a6-d372242ad911@dop.com> On 8/9/16 6:50 PM, Lawrence D?Oliveiro wrote: > On Wednesday, August 10, 2016 at 12:44:30 AM UTC+12, Martin Sch??n wrote: > >> What I have failed to achieve is a graph with a transparent >> background. > > While it is possible to render image frames with alpha transparency channels, as far as I know none of the motion-video image formats supports transparency. They all assume that the image fills the entire frame and completely covers anything behind. > > You could invent your own player which draws the frames one after another itself. Note that the JPEG image format doesn?t support transparency; you would have to use a bulkier format like PNG. > > Another option would be to have an additional video track where the image is interpreted purely as transparency masking information for the ?proper? video track. You would still need to write your own player to handle this. > Another option in the Python world is Vapoursynth: I runs with Cython and generally pipes to something like ffmpeg which creates the final video file so it may be a more complex than you want but it is capable of the processing you are asking for and in a Python language syntax. Dale From lawrencedo99 at gmail.com Wed Aug 10 01:46:29 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 22:46:29 -0700 (PDT) Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Message-ID: <3d7fec12-8bea-4f26-afa1-f8516763bfc5@googlegroups.com> On Monday, August 8, 2016 at 9:02:15 PM UTC+12, Anders J. Munch wrote: > It was only later that language designers fell into the notion that it > was crucial for a new language's success to look as much like C++ as > possible. Nevertheless, Python copied the C misfeature of using ?=? for assignment and ?==? for equality comparison, instead of the more 1337 mathy CompSci tradition of ?:=? for the former and ?=? for the latter. From michael.selik at gmail.com Wed Aug 10 01:53:29 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 10 Aug 2016 05:53:29 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 10, 2016 at 1:22 AM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > 1) catching exceptions at the point where you care, 2) > preemptively check for certain runtime conditions to avoid exceptions 3) > write as many tests as possible 4) learn to live with runtime errors. > I suggest focusing on #4, learn to appreciate/enjoy runtime errors. When you've come around to that way of thinking, you'll have better judgement on when to care about catching exceptions vs letting them stop your program. As for writing as many tests as possible... eh. Think about your code scientifically. You can disprove a falsifiable hypothesis, but you can't prove a hypothesis. Sometimes you can formally prove the correctness of your code via logical deduction, but that's not testing. Sometimes you can exhaustively test all expected inputs, but it's impossible to exhaustively test the infinity of possible inputs. So write tests as if you're presenting evidence for a theory. From ben+python at benfinney.id.au Wed Aug 10 02:01:51 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 10 Aug 2016 16:01:51 +1000 Subject: Individuals and groups (was Specify the database for a Django) References: <85wpjq22xd.fsf@benfinney.id.au> <851t1x1lca.fsf@benfinney.id.au> <17a1fdb5-daff-4ec6-b75e-4382bf4c7d0c@googlegroups.com> Message-ID: <85wpjpyxf4.fsf@benfinney.id.au> Rustom Mody writes: > On Wednesday, August 10, 2016 at 6:39:31 AM UTC+5:30, Ben Finney wrote: > > -- > > \ ?I love and treasure individuals as I meet them, I loathe and | > > `\ despise the groups they identify with and belong to.? ?George | > > _o__) Carlin, 2007 | > > Nice quote ? thanks! George Carlin's stand-up comedy act grew progressively nastier and more cynical in his later years. It can give the impression that he was a thoroughgoing misanthrope after the 1990s. Which can make it pleasantly surprising to view some of the contemporary interviews he gave, such as the one from which the above quote is taken. He was never shy about stating what was wrong in the world, but given the opportunity to speak off stage was also a surprisingly optimistic and compassionate man. -- \ ?? whoever claims any right that he is unwilling to accord to | `\ his fellow-men is dishonest and infamous.? ?Robert G. | _o__) Ingersoll, _The Liberty of Man, Woman and Child_, 1877 | Ben Finney From greg.ewing at canterbury.ac.nz Wed Aug 10 02:16:44 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 10 Aug 2016 18:16:44 +1200 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: Juan Pablo Romero M?ndez wrote: > This is interesting. You are Ok having runtime errors? You're going to have runtime errors in any case, whether they come from code you've put there yourself to check types, or from somewhere deeper down. The only difference is that checks you make yourself *might* catch errors slightly sooner, and *might* be able to provide better diagnostics. However, experience has shown that, the vast majority of the time, type errors in Python are caught pretty much immediately, and the stack trace provides more than enough information to pin down the source of the problem. So, putting in manual type checks is hardly ever worth the effort, and can even be counterproductive, since it interferes with duck typing. Occasionally there will be a situation where a type error results in a corrupted data structure that leads to problems later. But those cases are relatively rare and best dealt with as they come up. -- Greg From greg.ewing at canterbury.ac.nz Wed Aug 10 02:35:44 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 10 Aug 2016 18:35:44 +1200 Subject: ctypes And The WACAH Principle In-Reply-To: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > When using array objects from code written in C or C++ (the only way to > effectively make use of this information), it makes more sense to use the > buffer interface supported by array objects. > > Ah, I wish... I had a look at the documentation for the buffer interface and > memory views, and found no way to make use of them from pure Python. > Basically, the paragraph I quoted above is wrong in just about every > important detail. I think you must be misreading it. It's talking about accessing the internals of an object from C or C++ code, *not* from Python. If I understand correctly, you want to get the address of an object's memory as a Python integer, so that you can pass it to something via a ctypes wrapper? If you're using ctypes anyway, it shouldn't be much harder to use ctypes to access the object's buffer interface, if it has one. You could also consider using Cython instead of ctypes. It will let you create non-leaky wrappers and provide easy access to buffer interfaces. -- Greg From lawrencedo99 at gmail.com Wed Aug 10 02:39:06 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 9 Aug 2016 23:39:06 -0700 (PDT) Subject: Subprocess Startup Error In-Reply-To: References: <156707d3a75-2837-ddbb@webprd-m17.mail.aol.com> Message-ID: <9bff7ecb-5c24-44f0-81d2-26d70768e5a3@googlegroups.com> On Wednesday, August 10, 2016 at 8:12:10 AM UTC+12, xlarged... at aol.com wrote: > "Either IDLE can't start a subprocess or personal firewall software is > blocking the connection." Q: Why does Windows print such explanatory error messages? A: To give you a hint as to the cause of the problem. From steve+comp.lang.python at pearwood.info Wed Aug 10 02:47:15 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 16:47:15 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57aacdf5$0$2910$c3e8da3$76491128@news.astraweb.com> On Wednesday 10 August 2016 15:20, Juan Pablo Romero M?ndez wrote: > Ok, so you suggested 1) catching exceptions at the point where you care, 2) > preemptively check for certain runtime conditions to avoid exceptions 3) > write as many tests as possible 4) learn to live with runtime errors. > > Is that a fair characterization? (1) Not quite. Only catch exceptions you expect and *can deal with*. By deal with, I mean either recover from, or convert to a user-friendly error message (perhaps displaying it in a dialog for GUI apps). Your script or application may want to use a top-level exception handler to catch any *unexpected* exceptions, log them, perform any extra cleanup needed, and display a nice error message before exiting. But that counts as gold- plating: for many applications or scripts, aimed at technical or semi-technical users, its acceptable to just let the traceback print and the interpreter exit. (2) Sometimes. Its a judgement call whether it is better to "Look Before You Leap" or "Easier To Ask For Forgiveness Rather Than Permission". It depends on what you are doing, how often you expect an exceptional case, the performance implications, whether there is a risk of "Time Of Check To Time Of Use" bug, etc. In other words, that's usually a technical question, not a matter of style. Google for LBYL versus EAFP for more discussion. Occasionally it comes down to both being equally good, in which case you get to pick whichever you like. (3) Yes, this is correct. Basically, with a dynamically-typed language like Python, you have to write all the tests you would write with a statically-typed language, plus some more. There's no question that the cost of the freedom dynamic typing gives you is that you have to write extra tests. But not as many as you might think. The major differences in mindset between dynamic/static typing are: - Where the compiler would detect a type error at compile time, you get the same result with a small unit test that checks for TypeError or AttributeError instead. (Although see also the MyPy project, which brings an optional static type- checker to Python.) - As much as possible, avoid thinking "I need a list", and instead think "I need anything which offers the list interface". (Duck-typing.) That *especially* counts for writing library code, application code you can be a bit more strict about types because you're only limiting yourself, not other users. - A Python runtime exception is not like a segmentation fault. Its not going to overflow a buffer and execute random code. It's a controlled failure, not an uncontrolled one. (4) There's no necessary reason to expect that you'll get more runtime errors in a well-written and tested Python application than you get in a well-written and tested C application. Regardless of the language, you have to deal with runtime errors: - network is down; - disk is full; - user lacks permission to edit that file; - database error; - account not found; etc. Python is no different. And regardless of the language, the compiler can't check that "add_account" actually adds the account, you need a unit-test to check that. So you have to write tests either way. Basically, just relax -- static type checks aren't useless, but they do very little that a runtime exception won't give you. So long as you have a good suite of unit tests that exercises the whole program, you'll find your type errors when they raise TypeError, then you fix them. You may be gathering from this that dynamic typing and Test Driven Development go hand in hand. That's exactly right. They complement each other very well. -- Steve From auriocus at gmx.de Wed Aug 10 03:05:17 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 10 Aug 2016 09:05:17 +0200 Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: Am 09.08.16 um 14:44 schrieb Martin Sch??n: > Using this example: > http://matplotlib.org/examples/animation/moviewriter.html > and this tutorial: > https://www.youtube.com/watch?v=h0F5X4b7jug > I have covered a lot of ground. > > What I have failed to achieve is a graph with a transparent > background. None of the leads I have found searching the Internet > have helped me. (I may have found the needed information without > realising it...) > > Any help is appreciated. Here is a simple idea: Draw the background with a colour, that is not used in the graphs, like bright blue or green. Any decent video editing software should support colour keying, i.e. masking the green parts out, becuase this is used in film production (green screening) Christian From lawrencedo99 at gmail.com Wed Aug 10 04:22:57 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 01:22:57 -0700 (PDT) Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: <2eba7724-bc28-4335-b92f-76884be80c64@googlegroups.com> On Wednesday, August 10, 2016 at 6:35:57 PM UTC+12, Gregory Ewing wrote: > It's talking about accessing the internals of an object from C or C++ code, > *not* from Python. It?s saying that?s ?the only way to effectively make use of this information?, which it is not. > If you're using ctypes anyway, it shouldn't be much harder to > use ctypes to access the object's buffer interface, if it has > one. Yeah, I could probably figure that out ... wonder why it?s not a standard part of ctypes already? > It will let you create non-leaky wrappers ... Why would I want to do that? From wolfgang.maier at biologie.uni-freiburg.de Wed Aug 10 04:32:20 2016 From: wolfgang.maier at biologie.uni-freiburg.de (Wolfgang Maier) Date: Wed, 10 Aug 2016 10:32:20 +0200 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: Try py instead of python. That invokes a thing called the python launcher (see https://docs.python.org/3/using/windows.html#python-launcher-for-windows for more details). Best, Wolfgang On 10.08.2016 06:46, sh.ajay12 at gmail.com wrote: > Hi Everyone > > i have installed python 3.5 , but the python command is not recognized > > C:\Users\sharmaaj>python > 'python' is not recognized as an internal or external command, > operable program or batch file. > > what should i do to run python commands. > > thanks everyone for reading my post. > > From Ajay > From alister.ware at ntlworld.com Wed Aug 10 05:23:40 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 10 Aug 2016 09:23:40 GMT Subject: Make sure you removed all debugging print statements error References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: On Tue, 09 Aug 2016 19:53:56 -0700, Lawrence D?Oliveiro wrote: > On Tuesday, August 9, 2016 at 4:20:37 AM UTC+12, Chris Angelico wrote: >> Firstly, the bare "except:" clause should basically never be used; >> instead, catch a very specific exception and figure out what you want >> to do here > > Yes. > >> - but secondly, don't force people's names to be subdivided. > > I like the French convention > . I may not have a family name at all -- "If once a man indulges himself in murder, very soon he comes to think little of robbing; and from robbing he next comes to drinking and Sabbath-breaking, and from that to incivility and procrastination." -- Thomas De Quincey (1785 - 1859) From drewes.mil at gmail.com Wed Aug 10 05:27:06 2016 From: drewes.mil at gmail.com (drewes.mil at gmail.com) Date: Wed, 10 Aug 2016 02:27:06 -0700 (PDT) Subject: cropping a random part of an image In-Reply-To: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: <04a5f625-c9f4-4965-9793-d01370710438@googlegroups.com> Hi Peter, Hi Robin, thanks for your help, and especially for the code ;) @Peter: thanks for the links, I know, it is always better to write the code than to copy and paste! @Robin: the question, i? trying to answer is, if good pictures, in this case portraits, also have good "Parts", so if the beauty of a picture can be found in its crops, too. In the and it is for a deep CNN and I hope it will work! Thanks again! Steffen From bc at freeuk.com Wed Aug 10 05:48:51 2016 From: bc at freeuk.com (BartC) Date: Wed, 10 Aug 2016 10:48:51 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/08/2016 02:47, Steven D'Aprano wrote: > On Wed, 10 Aug 2016 11:02 am, Chris Angelico wrote: > >> On Wed, Aug 10, 2016 at 10:58 AM, Juan Pablo Romero M?ndez >> wrote: > >>> This is interesting. You are Ok having runtime errors? >> >> Absolutely. In real terms, there's no difference between "compile-time >> error" and "run-time error that you trigger the moment you run your >> program". > > That's an oversimplification. > > How about the difference between getting a compile-time error immediately > you try to compile your program, and a run-time error three quarters of the > way through processing a billion records, leaving your data in a corrupted > state? And when it is a customer (perhaps in a different country) who is in the middle of running your code. > I love the fact that the computer on the Apollo lunar landers was expected > to have bugs, and was designed to automatically reboot and continue the > calculation that was interrupted. By memory, it rebooted something like 30 > or 40 times during the first moon landing. Wouldn't the same error just recur each time? Or was this a random hardware error rather than logic? -- Bartc From eryksun at gmail.com Wed Aug 10 06:12:52 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 10 Aug 2016 10:12:52 +0000 Subject: ctypes And The WACAH Principle In-Reply-To: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: On Wed, Aug 10, 2016 at 1:45 AM, Lawrence D?Oliveiro wrote: > > Ah, I wish... I had a look at the documentation for the buffer interface and memory views, > and found no way to make use of them from pure Python. Basically, the paragraph I quoted > above is wrong in just about every important detail. ctypes classes have from_buffer and from_buffer_copy methods that use the buffer protocol. For example, for a read-only buffer you can use from_buffer_copy: >>> b = bytes(b'1234') >>> a = (ctypes.c_char * 3).from_buffer_copy(b, 1) >>> a[:] b'234' from_buffer requires a writable buffer, a bit more explanation, and a warning for Python 2. Switch to a bytearray to allow using from_buffer: >>> b = bytearray(b'1234') >>> a = (ctypes.c_char * 3).from_buffer(b, 1) >>> a[:] b'234' The memory is shared: >>> b[:] = b[::-1] >>> a[:] b'321' from_buffer in Python 3 uses a memoryview to reference the source object and exported buffer, which it stores in the _objects dict of the ctypes data instance: >>> type(a._objects['ffffffff']) >>> a._objects['ffffffff'].obj bytearray(b'4321') Because of the existing export, the bytearray cannot be resized (up or down): >>> del b[0] Traceback (most recent call last): File "", line 1, in BufferError: Existing exports of data: object cannot be re-sized On the other hand, from_buffer in Python 2 naively uses PyObject_AsWriteBuffer and only keeps a reference to the source object. Thus there's no guarantee that the memory will remain valid. Let's demonstrate this with a segfault: >>> b = bytearray(2**20) >>> a = (ctypes.c_char * 2**20).from_buffer(b) >>> b[-1] = 'z' >>> a[-1] 'z' >>> del b[:] >>> a[-1] Segmentation fault (core dumped) From alister.ware at ntlworld.com Wed Aug 10 06:21:17 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 10 Aug 2016 10:21:17 GMT Subject: Make sure you removed all debugging print statements error References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: On Wed, 10 Aug 2016 09:23:40 +0000, alister wrote: > On Tue, 09 Aug 2016 19:53:56 -0700, Lawrence D?Oliveiro wrote: > >> On Tuesday, August 9, 2016 at 4:20:37 AM UTC+12, Chris Angelico wrote: >>> Firstly, the bare "except:" clause should basically never be used; >>> instead, catch a very specific exception and figure out what you want >>> to do here >> >> Yes. >> >>> - but secondly, don't force people's names to be subdivided. >> >> I like the French convention >> . > > I may not have a family name at all Damn this would have been an ideal place to reference Fawlty Towers (not monty python but close). Basil: Your Name please could I have your name Lord Melbury: Melbury Basil: Could you fill it in please Basil: There There There Basil: BOTH Names Please Lord Melbury: I Beg Your pardon> Basil: Would you put BOTH Your names please. Lord Melbury: er I only use one. Basil: You Don't have a first name? Lord Melbury: No I am Lord Melbury, so i simply sign Melbury. (simultanious Telephone conversation deleted for clarity) Note with English titles such as Lordship, they may or May not have an relationship to the recipients actual family name, often they refer to the location of the lords residency. -- IBM Advanced Systems Group -- a bunch of mindless jerks, who'll be first against the wall when the revolution comes... -- with regrets to D. Adams From drewes.mil at gmail.com Wed Aug 10 06:31:45 2016 From: drewes.mil at gmail.com (drewes.mil at gmail.com) Date: Wed, 10 Aug 2016 03:31:45 -0700 (PDT) Subject: cropping a random part of an image In-Reply-To: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: Hi Robin, I tried to understand and run your code, and I get the Error: "File "Rand_Crop.py", line 15, in with Image.open(os.path.join(INPATH, file)) as im: File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 528, in __getattr__ raise AttributeError(name) AttributeError: __exit__" I looked up the Image.py and don't know, why the Name could not be read out. If i print the names of the images out, before the first "for" everything is fine, but it dosen't work after line 15. I try to figure this out, but if someone has a hint, I would be happy! Steffen From eryksun at gmail.com Wed Aug 10 06:34:35 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 10 Aug 2016 10:34:35 +0000 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 4:46 AM, wrote: > > i have installed python 3.5 , but the python command is not recognized > > C:\Users\sharmaaj>python > 'python' is not recognized as an internal or external command, > operable program or batch file. > > what should i do to run python commands. Modify your setup. Open "Programs and Features" from the control panel. Right-click the entry for Python 3.5 (not "Python Launcher") and select Uninstall/Change. In the dialog, click on "Modify". Click "Next" and select "Add Python to environment variables". Click "Install". When it's done, open a new command prompt and enter "python". From andersjm at stofanet.dk Wed Aug 10 07:25:31 2016 From: andersjm at stofanet.dk (Anders J. Munch) Date: Wed, 10 Aug 2016 13:25:31 +0200 Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Message-ID: Lawrence D?Oliveiro: >> [...] as much like C++ as >> possible. > > Nevertheless, Python copied the C misfeature [...] You segued a little too easily from C++ to C. When talking language evolution and inspirations, they are entirely different things. - Anders From steve+python at pearwood.info Wed Aug 10 07:34:12 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Wed, 10 Aug 2016 21:34:12 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ab1136$0$1615$c3e8da3$5496439d@news.astraweb.com> On Wed, 10 Aug 2016 07:48 pm, BartC wrote: > On 10/08/2016 02:47, Steven D'Aprano wrote: >> How about the difference between getting a compile-time error immediately >> you try to compile your program, and a run-time error three quarters of >> the way through processing a billion records, leaving your data in a >> corrupted state? > > And when it is a customer (perhaps in a different country) who is in the > middle of running your code. Depends on the customer, the application, and most of all, what the bug is. In order of priority: - bugs which threaten people's lives; - security bugs; - bugs that corrupt or destroy saved data or files; - bugs that lose unsaved data; - bugs that cause the software to do the wrong thing; - bugs that prevent you from doing the right thing; - presentation bugs; - minor UI bugs which don't impact usability. >> I love the fact that the computer on the Apollo lunar landers was >> expected to have bugs, and was designed to automatically reboot and >> continue the calculation that was interrupted. By memory, it rebooted >> something like 30 or 40 times during the first moon landing. > > Wouldn't the same error just recur each time? Or was this a random > hardware error rather than logic? http://www.abc.net.au/science/moon/computer.htm http://www.klabs.org/history/apollo_11_alarms/eyles_2004/eyles_2004.htm -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Wed Aug 10 07:41:42 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Aug 2016 21:41:42 +1000 Subject: Make sure you removed all debugging print statements error In-Reply-To: References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: On Wed, Aug 10, 2016 at 8:21 PM, alister wrote: >>>> - but secondly, don't force people's names to be subdivided. >>> >>> I like the French convention >>> . >> >> I may not have a family name at all > > Damn this would have been an ideal place to reference Fawlty Towers (not > monty python but close). > > > Basil: Your Name please could I have your name > Lord Melbury: Melbury > Basil: Could you fill it in please > Basil: There There There > Basil: BOTH Names Please > Lord Melbury: I Beg Your pardon> > Basil: Would you put BOTH Your names please. > Lord Melbury: er I only use one. > Basil: You Don't have a first name? > Lord Melbury: No I am Lord Melbury, so i simply sign Melbury. > > (simultanious Telephone conversation deleted for clarity) > > Note with English titles such as Lordship, they may or May not have an > relationship to the recipients actual family name, often they refer to > the location of the lords residency. Specific example: Lord Kelvin, after whom the SI unit of temperature is named. He's most commonly known by something that isn't his given name OR family name ("William" and "Thomson" respectively), but by the name of a river. https://en.wikipedia.org/wiki/William_Thomson,_1st_Baron_Kelvin https://en.wikipedia.org/wiki/River_Kelvin I suppose you could put down Kelvin as a last name and either Baron or Lord as a first name, but that'd just be cheating... ChrisA From drewes.mil at gmail.com Wed Aug 10 08:13:05 2016 From: drewes.mil at gmail.com (drewes.mil at gmail.com) Date: Wed, 10 Aug 2016 05:13:05 -0700 (PDT) Subject: cropping a random part of an image In-Reply-To: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: Ok, did it :) import random, os, time from PIL import Image INPATH = ('/home/sdrewes/Desktop/Portaits/Bilder/Test/') OUTPATH = ('/home/sdrewes/Desktop/Portaits/Bilder/Gut_Crop/') dx = dy = 228 tilesPerImage = 100 files = os.listdir(INPATH) numOfImages = len(files) print(files) t = time.time() for file in files: im = Image.open(INPATH+file) for i in range(1, tilesPerImage+1): newname = file.replace('.', '_{:03d}.'.format(i)) w, h = im.size x = random.randint(0, w-dx-1) y = random.randint(0, h-dy-1) print("Cropping {}: {},{} -> {},{}".format(file, x,y, x+dx, y+dy)) im.crop((x,y, x+dx, y+dy))\ .save(os.path.join(OUTPATH, newname)) t = time.time()-t print("Done {} images in {:.2f}s".format(numOfImages, t)) print("({:.1f} images per second)".format(numOfImages/t)) print("({:.1f} tiles per second)".format(tilesPerImage*numOfImages/t)) From drewes.mil at gmail.com Wed Aug 10 08:16:48 2016 From: drewes.mil at gmail.com (drewes.mil at gmail.com) Date: Wed, 10 Aug 2016 05:16:48 -0700 (PDT) Subject: cropping a random part of an image In-Reply-To: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> References: <8013ac9e-a625-4b07-bd68-fa7e980894f9@googlegroups.com> Message-ID: <60d52787-8083-46cb-a0ba-66d249283996@googlegroups.com> Ok, now it works for me! Thanks again! import random, os, time from PIL import Image INPATH = ('/home/.../Start/') OUTPATH = ('/home/.../Ziel/') dx = dy = 228 tilesPerImage = 25 files = os.listdir(INPATH) numOfImages = len(files) print(files) t = time.time() for file in files: im = Image.open(INPATH+file) for i in range(1, tilesPerImage+1): newname = file.replace('.', '_{:03d}.'.format(i)) w, h = im.size x = random.randint(0, w-dx-1) y = random.randint(0, h-dy-1) print("Cropping {}: {},{} -> {},{}".format(file, x,y, x+dx, y+dy)) im.crop((x,y, x+dx, y+dy))\ .save(os.path.join(OUTPATH, newname)) t = time.time()-t print("Done {} images in {:.2f}s".format(numOfImages, t)) print("({:.1f} images per second)".format(numOfImages/t)) print("({:.1f} tiles per second)".format(tilesPerImage*numOfImages/t)) From bc at freeuk.com Wed Aug 10 09:46:03 2016 From: bc at freeuk.com (BartC) Date: Wed, 10 Aug 2016 14:46:03 +0100 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: On 10/08/2016 11:34, eryk sun wrote: > On Wed, Aug 10, 2016 at 4:46 AM, wrote: >> >> i have installed python 3.5 , but the python command is not recognized >> >> C:\Users\sharmaaj>python >> 'python' is not recognized as an internal or external command, >> operable program or batch file. >> >> what should i do to run python commands. > > Modify your setup. Open "Programs and Features" from the control > panel. Right-click the entry for Python 3.5 (not "Python Launcher") > and select Uninstall/Change. In the dialog, click on "Modify". Click > "Next" and select "Add Python to environment variables". Click > "Install". When it's done, open a new command prompt and enter > "python". That's an unusual way of doing it. And rather precarious as you're one wrong click away from deleting Python altogether! It also, after doing the above, spent rather a long time copying files about when the operation shouldn't need anything of the sort. However, I tried it and it didn't seem to work (opening a new prompt and type 'python'). Maybe it needs a machine restart, but I can't be bothered to try that on the off-chance it might work. The method I use is this: * Start Control Panel again * Click System (assuming you have the 'classic' or 'icon' view otherwise you have to hunt for it) * Click Advanced System Settings * Click Environment Variables * Highlight the "Path" variable in the top panel (User variables) * Click Edit * Being careful not to erase the whole lot (I hate when it highlights the whole thing, as the slightest false move and everything disappears), navigate to the front of the current settings, and type in (depending on where Python is installed): C:\Python\; (Or navigate to the end and type ;C:\Python\, assuming there are some settings already.) * Keep clicking OKs until it's done * Open the command prompt again and type 'python' to test. NOTE: I've assume Python is in C:\Python, but it might more typically be C:\Python34 etc. This way, you can choose which version to invoke if there are several. -- Bartc From random832 at fastmail.com Wed Aug 10 10:03:54 2016 From: random832 at fastmail.com (Random832) Date: Wed, 10 Aug 2016 10:03:54 -0400 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: <1470837834.1416490.691381177.4A59E35D@webmail.messagingengine.com> On Wed, Aug 10, 2016, at 06:34, eryk sun wrote: > On Wed, Aug 10, 2016 at 4:46 AM, wrote: > > > > i have installed python 3.5 , but the python command is not recognized > > > > C:\Users\sharmaaj>python > > 'python' is not recognized as an internal or external command, > > operable program or batch file. > > > > what should i do to run python commands. > > Modify your setup. Open "Programs and Features" from the control > panel. Right-click the entry for Python 3.5 (not "Python Launcher") > and select Uninstall/Change. In the dialog, click on "Modify". Click > "Next" and select "Add Python to environment variables". Click > "Install". When it's done, open a new command prompt and enter > "python". Or he could just use "py", which is the python launcher you just mentioned, and works fine. If he has other versions of python installed he may need "py -3". From ernest.bonat at gmail.com Wed Aug 10 10:11:09 2016 From: ernest.bonat at gmail.com (Ernest Bonat, Ph.D.) Date: Wed, 10 Aug 2016 07:11:09 -0700 Subject: Error running the exe file in Windows "Failed to execute script pyi_rth_pkgres" In-Reply-To: References: Message-ID: Hi, I have fixed this problem by following the steps: 1. Uninstall the PyInstaller downloaded from http://www.pyinstaller.org/ pip install pyinstaller 2. Install the PyInstaller from the GitHub developer repository pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip Still not clear to me why the main site http://www.pyinstaller.org/ does not have the latest release? Thanks Ernest Bonat, Ph.D. Senior Software Engineer Senior Data Scientist On Sun, Aug 7, 2016 at 8:03 AM, Ernest Bonat, Ph.D. wrote: > Hi, > > I have created a simple Python program including the following packages: > > import numpy as np > import matplotlib.pyplot as plt > import pandas as pd > > if __name__ == '__main__': > print("Hi!") > > I have created the installation package using PyInstaller with the > following command line: > > pyinstaller --onedir --name=appname --windowed "C:\appname.py" > > The required folders (build and dist) and the spec file are created > properly with no errors. When I run the appname.exe file I got the error > "Failed to execute script pyi_rth_pkgres". > > I'm using the following now: > > - Anaconda3 4.1.1.64-bit > - Python 3.5.2 > - PyInstaller 3.2 > - Windows 10 64-bit > > Any help is very appreciated. Feel free to contact me at any time you > need. > > Thank you, > > Ernest Bonat, Ph.D. > Senior Software Engineer > Senior Data Scientist > > From random832 at fastmail.com Wed Aug 10 12:19:43 2016 From: random832 at fastmail.com (Random832) Date: Wed, 10 Aug 2016 12:19:43 -0400 Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <3d7fec12-8bea-4f26-afa1-f8516763bfc5@googlegroups.com> Message-ID: <1470845983.1463573.691529097.0E053AB7@webmail.messagingengine.com> On Wed, Aug 10, 2016, at 07:59, Dennis Lee Bieber wrote: > The use of = also has a long history... FORTRAN (where the comparison > was .EQ.), BASIC (granted, K&K required assignment to start with the > keyword LET, so the use of = was mainly a delimiter between target and > expression being assigned). Visual Basic actually uses = for both assignment and comparison *without* the Let keyword, it gets by on the fact that assignment is not an expression. From bc at freeuk.com Wed Aug 10 13:13:53 2016 From: bc at freeuk.com (BartC) Date: Wed, 10 Aug 2016 18:13:53 +0100 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: <1470837834.1416490.691381177.4A59E35D@webmail.messagingengine.com> Message-ID: On 10/08/2016 15:03, Random832 wrote: > On Wed, Aug 10, 2016, at 06:34, eryk sun wrote: >> On Wed, Aug 10, 2016 at 4:46 AM, wrote: >>> >>> i have installed python 3.5 , but the python command is not recognized >>> >>> C:\Users\sharmaaj>python >>> 'python' is not recognized as an internal or external command, >>> operable program or batch file. >>> >>> what should i do to run python commands. >> >> Modify your setup. Open "Programs and Features" from the control >> panel. Right-click the entry for Python 3.5 (not "Python Launcher") >> and select Uninstall/Change. In the dialog, click on "Modify". Click >> "Next" and select "Add Python to environment variables". Click >> "Install". When it's done, open a new command prompt and enter >> "python". > > Or he could just use "py", which is the python launcher you just > mentioned, and works fine. If he has other versions of python installed > he may need "py -3". That works on my machine and I wondered why. It seems to do it by simply putting a special executable (py.exe) into C:\windows, which is the first (or second) place it will look when someone types the name of a program. But a bit of a cheat I think, and probably not good practice (suppose all your hundreds of apps put dedicated launchers into C:\windows which is supposed to be for OS stuff). (And if you're going to do that, you can also just put a one-line script in there to launch any program.) -- Bartc From jpablo.romero at gmail.com Wed Aug 10 13:27:46 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Wed, 10 Aug 2016 10:27:46 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aacdf5$0$2910$c3e8da3$76491128@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57aacdf5$0$2910$c3e8da3$76491128@news.astraweb.com> Message-ID: 2016-08-09 23:47 GMT-07:00 Steven D'Aprano < steve+comp.lang.python at pearwood.info>: > On Wednesday 10 August 2016 15:20, Juan Pablo Romero M?ndez wrote: > > > Ok, so you suggested 1) catching exceptions at the point where you care, > 2) > > preemptively check for certain runtime conditions to avoid exceptions 3) > > write as many tests as possible 4) learn to live with runtime errors. > > > > Is that a fair characterization? > > (1) Not quite. > > Only catch exceptions you expect and *can deal with*. By deal with, I mean > either recover from, or convert to a user-friendly error message (perhaps > displaying it in a dialog for GUI apps). > > Your script or application may want to use a top-level exception handler to > catch any *unexpected* exceptions, log them, perform any extra cleanup > needed, > and display a nice error message before exiting. But that counts as gold- > plating: for many applications or scripts, aimed at technical or > semi-technical > users, its acceptable to just let the traceback print and the interpreter > exit. > > (2) Sometimes. Its a judgement call whether it is better to "Look Before > You > Leap" or "Easier To Ask For Forgiveness Rather Than Permission". It > depends on > what you are doing, how often you expect an exceptional case, the > performance > implications, whether there is a risk of "Time Of Check To Time Of Use" > bug, > etc. In other words, that's usually a technical question, not a matter of > style. > > Google for LBYL versus EAFP for more discussion. > > Occasionally it comes down to both being equally good, in which case you > get to > pick whichever you like. > > (3) Yes, this is correct. > > Basically, with a dynamically-typed language like Python, you have to > write all > the tests you would write with a statically-typed language, plus some more. > There's no question that the cost of the freedom dynamic typing gives you > is > that you have to write extra tests. But not as many as you might think. > Exactly what kind of freedom do you have in mind? > > The major differences in mindset between dynamic/static typing are: > > - Where the compiler would detect a type error at compile time, you get the > same result with a small unit test that checks for TypeError or > AttributeError > instead. > > (Although see also the MyPy project, which brings an optional static type- > checker to Python.) > > - As much as possible, avoid thinking "I need a list", and instead think "I > need anything which offers the list interface". (Duck-typing.) > > That *especially* counts for writing library code, application code you > can be > a bit more strict about types because you're only limiting yourself, not > other > users. > > - A Python runtime exception is not like a segmentation fault. Its not > going to > overflow a buffer and execute random code. It's a controlled failure, not > an > uncontrolled one. > > (4) There's no necessary reason to expect that you'll get more runtime > errors > in a well-written and tested Python application than you get in a > well-written > and tested C application. > > Regardless of the language, you have to deal with runtime errors: > > - network is down; > - disk is full; > - user lacks permission to edit that file; > - database error; > - account not found; > > etc. Python is no different. And regardless of the language, the compiler > can't > check that "add_account" actually adds the account, you need a unit-test to > check that. So you have to write tests either way. > > Basically, just relax -- static type checks aren't useless, but they do > very > little that a runtime exception won't give you. So long as you have a good > suite of unit tests that exercises the whole program, you'll find your type > errors when they raise TypeError, then you fix them. > > You may be gathering from this that dynamic typing and Test Driven > Development > go hand in hand. That's exactly right. They complement each other very > well. > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From michael.selik at gmail.com Wed Aug 10 13:29:53 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 10 Aug 2016 17:29:53 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Aug 9, 2016 at 9:31 PM Steven D'Aprano wrote: > > http://steve-yegge.blogspot.com.au/2008/05/dynamic-languages-strike-back.html Great link. I enjoyed the video, too. https://www.youtube.com/watch?v=tz-Bb-D6teE Buried deep in the QA section, there's a comment from the audience (I'll paraphrase), that compiler type-checking is wonderful. Yegge replies that, "I realized early in my career that I would actually rather have a runtime error than a compile error." From rustompmody at gmail.com Wed Aug 10 13:34:48 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 10 Aug 2016 10:34:48 -0700 (PDT) Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Message-ID: <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> On Wednesday, August 10, 2016 at 9:31:40 PM UTC+5:30, Anders J. Munch wrote: > Lawrence D?Oliveiro: > >> [...] as much like C++ as > >> possible. > > > > Nevertheless, Python copied the C misfeature [...] > > You segued a little too easily from C++ to C. When talking language > evolution and inspirations, they are entirely different things. Talking of too-easy-segueing: Python?s inspiration and origin is ABC Whose assignment looked like PUT expr INTO var This has the salutary effect - Of being l-to-r (the only other such case I know is gas mov) - And of course of not making a misleading pun between equality and assignment Of course from a software engineering pov, writing say a 1000 line program with > 500 assignments written with PUT?INTO is of course ridiculous But from a pedagogy pov here is a recent post (set) by a beginner that shows how much this pun costs in beginner confusion: https://mail.python.org/pipermail/python-list/2016-June/710595.html https://mail.python.org/pipermail/python-list/2016-June/710668.html And for completeness (since there seems to be the notion that C and like are the only natural ones around) here are some extant alternatives: Pascal (and Algol family generally) var := exp Early Basic LET var = exp ABC PUT exp INTO var Lisp(scheme) (setq var exp) (set! var exp) Assembly mov var, exp (gas) mov exp, var APL var ? exp From gengyangcai at gmail.com Wed Aug 10 13:39:29 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Wed, 10 Aug 2016 10:39:29 -0700 (PDT) Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? Message-ID: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> I managed to get this piece of code to work : >>> print("This program calculates mpg.") This program calculates mpg. >>> milesdriven = input("Enter miles driven:") Enter miles driven: 50 >>> milesdriven = float(milesdriven) >>> gallonsused = input("Enter gallons used:") Enter gallons used: 100 >>> gallonsused = float(gallonsused) >>> gallonsused = float(gallonsused) >>> mpg = milesdriven / gallonsused >>> print("Miles per gallon:", mpg) Miles per gallon: 0.5 >>> print("This program calculates mpg.") This program calculates mpg. >>> milesdriven = input("Enter miles driven:") Enter miles driven: 100 >>> milesdriven = float(milesdriven) >>> gallonsused = input("Enter gallons used:") Enter gallons used: 500 >>> gallonsused = float(gallonsused) >>> mpg = milesdriven / gallonsused >>> print("Miles per gallon:", mpg) Miles per gallon: 0.2 But, why can't i define a variable like "miles_driven" with an underscore in my Python Shell ? When I try to define it , the underscore doesn't appear at all and instead I get this result : "miles driven" , with no underscore appearing even though I have already typed "_" between "miles" and "driven" ? From jpablo.romero at gmail.com Wed Aug 10 13:43:02 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Wed, 10 Aug 2016 10:43:02 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: 2016-08-09 23:16 GMT-07:00 Gregory Ewing : > Juan Pablo Romero M?ndez wrote: > > This is interesting. You are Ok having runtime errors? >> > > You're going to have runtime errors in any case, whether > they come from code you've put there yourself to check > types, or from somewhere deeper down. > You are correct. What I was trying to understand is if there is a much more relaxed attitude towards runtime errors in the Python community (even embracing it) than others* (and of course, how to handle those errors). (In certain communities people favor a design where exceptions / runtime errors only happen at program boundaries) > > The only difference is that checks you make yourself > *might* catch errors slightly sooner, and *might* be able > to provide better diagnostics. > > However, experience has shown that, the vast majority of > the time, type errors in Python are caught pretty much > immediately, and the stack trace provides more than > enough information to pin down the source of the problem. > So, putting in manual type checks is hardly ever worth > the effort, and can even be counterproductive, since it > interferes with duck typing. > > Occasionally there will be a situation where a type > error results in a corrupted data structure that leads > to problems later. But those cases are relatively rare > and best dealt with as they come up. > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > From philb at philb.ca Wed Aug 10 13:52:13 2016 From: philb at philb.ca (Phil Boutros) Date: Wed, 10 Aug 2016 17:52:13 -0000 (UTC) Subject: Specify the database for a Django ModelForm instance References: <85wpjq22xd.fsf@benfinney.id.au> <851t1x1lca.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > > The person who wrote that question must think very like myself. > > That doesn't address the question about ?ModelForm.clean? or > ?ModelForm.is_valid?, though; how to tell the instance which database to > use for those? I must admit to not using multi-DB myself, but I think this set of answers may point you in the right direction: http://stackoverflow.com/questions/35214706/django-model-form-is-valid-in-specific-database Phil -- AH#61 Wolf#14 BS#89 bus#1 CCB#1 SENS KOTC#4 philb at philb.ca http://philb.ca EKIII rides with me: http://eddiekieger.com From michael.selik at gmail.com Wed Aug 10 14:07:52 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 10 Aug 2016 18:07:52 +0000 Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? In-Reply-To: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> References: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> Message-ID: On Wed, Aug 10, 2016 at 1:41 PM Cai Gengyang wrote: > I managed to get this piece of code to work : > > >>> print("This program calculates mpg.") > This program calculates mpg. > >>> milesdriven = input("Enter miles driven:") > Enter miles driven: 50 > >>> milesdriven = float(milesdriven) > >>> gallonsused = input("Enter gallons used:") > Enter gallons used: 100 > >>> gallonsused = float(gallonsused) > >>> gallonsused = float(gallonsused) > >>> mpg = milesdriven / gallonsused > >>> print("Miles per gallon:", mpg) > Miles per gallon: 0.5 > >>> print("This program calculates mpg.") > This program calculates mpg. > >>> milesdriven = input("Enter miles driven:") > Enter miles driven: 100 > >>> milesdriven = float(milesdriven) > >>> gallonsused = input("Enter gallons used:") > Enter gallons used: 500 > >>> gallonsused = float(gallonsused) > >>> mpg = milesdriven / gallonsused > >>> print("Miles per gallon:", mpg) > Miles per gallon: 0.2 > > But, why can't i define a variable like "miles_driven" with an underscore > in my Python Shell ? When I try to define it , the underscore doesn't > appear at all and instead I get this result : > "miles driven" , with no underscore appearing even though I have already > typed "_" between "miles" and "driven" ? > You'll have to give more information about the system you're using. Underscores are allowed in Python variable names. The first thought I have is that something outside of Python is changing your underscore to a space. Perhaps your OS spelling autocorrect is overly aggressive? From zachary.ware+pylist at gmail.com Wed Aug 10 14:08:34 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Wed, 10 Aug 2016 13:08:34 -0500 Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? In-Reply-To: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> References: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> Message-ID: On Wed, Aug 10, 2016 at 12:39 PM, Cai Gengyang wrote: > I managed to get this piece of code to work : > >>>> print("This program calculates mpg.") > This program calculates mpg. >>>> milesdriven = input("Enter miles driven:") > Enter miles driven: 50 >>>> milesdriven = float(milesdriven) >>>> gallonsused = input("Enter gallons used:") > Enter gallons used: 100 >>>> gallonsused = float(gallonsused) >>>> gallonsused = float(gallonsused) >>>> mpg = milesdriven / gallonsused >>>> print("Miles per gallon:", mpg) > Miles per gallon: 0.5 >>>> print("This program calculates mpg.") > This program calculates mpg. >>>> milesdriven = input("Enter miles driven:") > Enter miles driven: 100 >>>> milesdriven = float(milesdriven) >>>> gallonsused = input("Enter gallons used:") > Enter gallons used: 500 >>>> gallonsused = float(gallonsused) >>>> mpg = milesdriven / gallonsused >>>> print("Miles per gallon:", mpg) > Miles per gallon: 0.2 > > But, why can't i define a variable like "miles_driven" with an underscore in my Python Shell ? When I try to define it , the underscore doesn't appear at all and instead I get this result : > "miles driven" , with no underscore appearing even though I have already typed "_" between "miles" and "driven" ? Are you using IDLE on OSX? Try changing your font or bumping up the size (or switch to 3.5.2, which has better defaults). -- Zach From eryksun at gmail.com Wed Aug 10 15:03:05 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 10 Aug 2016 19:03:05 +0000 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: On Wed, Aug 10, 2016 at 1:46 PM, BartC wrote: > On 10/08/2016 11:34, eryk sun wrote: >> >> On Wed, Aug 10, 2016 at 4:46 AM, wrote: >>> >>> i have installed python 3.5 , but the python command is not recognized >>> >>> C:\Users\sharmaaj>python >>> 'python' is not recognized as an internal or external command, >>> operable program or batch file. >>> >>> what should i do to run python commands. >> >> Modify your setup. Open "Programs and Features" from the control >> panel. Right-click the entry for Python 3.5 (not "Python Launcher") >> and select Uninstall/Change. In the dialog, click on "Modify". Click >> "Next" and select "Add Python to environment variables". Click >> "Install". When it's done, open a new command prompt and enter >> "python". > > That's an unusual way of doing it. And rather precarious as you're one wrong > click away from deleting Python altogether! > > It also, after doing the above, spent rather a long time copying files about > when the operation shouldn't need anything of the sort. > > However, I tried it and it didn't seem to work (opening a new prompt and > type 'python'). Maybe it needs a machine restart, but I can't be bothered to > try that on the off-chance it might work. I did test that it works -- using the Python 3.5 installer, which was completely rewritten. It adds Python's installation and Scripts directories to PATH, and it doesn't need a reboot because the installer broadcasts a WM_SETTINGCHANGE "Environment" message that causes Explorer to reload its environment from the registry (the old installer doesn't do this). As to the procedure being unusual, I thought it would be simpler and more reliable than editing PATH in the old control-panel environment variable editor, which can be much worse if the user messes it up. Windows 10 has an improved editor that splits the value into separate text boxes and makes it easy to add entries and move them around, but most people are still using Windows 7. Also, the default installation path for a per-user installation of Python 3.5 is in the hidden folder "%LOCALAPPDATA%\Programs\Python\PythonXY[-32]". This can be difficult for users to find without inspecting the start-menu shortcuts or running `py -c "import sys;print(sys.prefix)"` -- assuming they used the default all-users installation of the py launcher to even be able to run py.exe. From bertrand.courts.circuits at gmail.com Wed Aug 10 15:09:09 2016 From: bertrand.courts.circuits at gmail.com (bertrand.courts.circuits at gmail.com) Date: Wed, 10 Aug 2016 12:09:09 -0700 (PDT) Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? In-Reply-To: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> References: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Message-ID: <386fb8b4-3ccd-46bd-85f9-8f41deba993e@googlegroups.com> Thanks Peter for the insights that args are passed to eval(). Thanks Steven for the suggestion that ?~/' may actually work in windows. Combining both I'm now successfully using the following in my logging.conf configuration file, that works in both *nix and Windows: [handler_file] class=handlers.RotatingFileHandler args=(os.path.expanduser('~/.scribusGenerator.log'),'a','maxBytes=50','backupCount=1') The tilde without an explicit call to expanduser() was not working in Windows... but with it it's working like a charm. Thank you. From eryksun at gmail.com Wed Aug 10 15:17:43 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 10 Aug 2016 19:17:43 +0000 Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: <1470837834.1416490.691381177.4A59E35D@webmail.messagingengine.com> References: <1470837834.1416490.691381177.4A59E35D@webmail.messagingengine.com> Message-ID: On Wed, Aug 10, 2016 at 2:03 PM, Random832 wrote: > On Wed, Aug 10, 2016, at 06:34, eryk sun wrote: >> On Wed, Aug 10, 2016 at 4:46 AM, wrote: >> > >> > i have installed python 3.5 , but the python command is not recognized >> > >> > C:\Users\sharmaaj>python >> > 'python' is not recognized as an internal or external command, >> > operable program or batch file. >> > >> > what should i do to run python commands. >> >> Modify your setup. Open "Programs and Features" from the control >> panel. Right-click the entry for Python 3.5 (not "Python Launcher") >> and select Uninstall/Change. In the dialog, click on "Modify". Click >> "Next" and select "Add Python to environment variables". Click >> "Install". When it's done, open a new command prompt and enter >> "python". > > Or he could just use "py", which is the python launcher you just > mentioned, and works fine. If he has other versions of python installed > he may need "py -3". >From what I can tell in discussions that I've read, the consensus of core developers is to not encourage novice users to use py.exe directly. The preferred way to start Python is still via python.exe. That said, experienced users will probably find using py.exe to be more convenient. I do. From marko at pacujo.net Wed Aug 10 15:18:15 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 10 Aug 2016 22:18:15 +0300 Subject: Python slang References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> Message-ID: <877fboigaw.fsf@elektro.pacujo.net> Rustom Mody : > On Wednesday, August 10, 2016 at 9:31:40 PM UTC+5:30, Anders J. Munch wrote: > Python?s inspiration and origin is ABC > Whose assignment looked like > PUT expr INTO var > > This has the salutary effect > - Of being l-to-r (the only other such case I know is gas mov) Oh, there's of course COBOL: ADD X TO Y GIVING RESULT Marko From jpablo.romero at gmail.com Wed Aug 10 16:33:13 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Wed, 10 Aug 2016 13:33:13 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: 2016-08-09 18:28 GMT-07:00 Steven D'Aprano : > On Wed, 10 Aug 2016 04:29 am, Juan Pablo Romero M?ndez wrote: > > > Hello, > > > > In online forums sometimes people complain that they end up having to > test > > constantly for None, > > Then don't return None to indicate an error. If you write this: > > def create_spam(arg): > if condition: > return Spam(arg) > else: > return None > > > then you are doing it wrong. It sounds like they are writing C in Python, > where they think that returning None (instead of a null pointer) is the way > to indicate an error. Instead: > > def create_spam(arg): > if condition: > return Spam(arg) > raise MyExceptionError('cannot create spam because reasons') > > > Now you can decide on a error-handling scheme: > > - try...except and handle the exception exactly when it occurs; > > - wrap your entire application in a single try...except or > exception handler, and catch the exception in a single place; > > - don't even bother catching the exception in the application at all, > treat it as a debugging aid. Any time your tests generate that > exception, that's a bug to be fixed. > > > There's no reason why you can't decide on a combination of all three: > > - the app catches the exception, logs it as a bug, and exits; > > - but individual parts of your code base treat this exception as an > EXPECTED error, wrapping that individual call in a try...except: > > > try: > myspam = create_spam(99) > except MyExceptionError: > # Only occurs when the server has hung > reboot_server() > myspam = create_spam(99) > > > Only catch exceptions you expect, and can recover from. > > If your *data* could be None, well that's no different from any other > sentinel or missing value. You have to deal with it somewhere. One way is > to filter it early: > > # filter out None > data = [obj for obj in data if data is not None] > > # or if you prefer > data = filter(lambda obj: obj is not None, data) > > > and now you can be sure that data doesn't contain None. > > > [...] > > Do you guys have any resources you like that addresses the following > > issue: What is the best way to use the dynamic features of Python to > avoid > > having to write a poor's man type system? > > This is an old, old, old debate that applies to Lisp, Perl, Javascript, > Lua, > Ruby and others, not just Python. Google for "dynamic typing versus static > typing" for more. Also google for "duck typing". > > Yeah, I've been on both sides of the debate over my programming years ;) What I find frustrating to be honest about this debate (and the reason it will never die probably) is that your perspective on it really depends mostly on anecdotal evidence, your own knowledge, experience, etc. I've been trying to find (without success so far) an example of a situation where the dynamic features of a language like Python provides a clear advantage over languages with more than one type. > One useful perspective is that languages are converging on a mix of both: > statically typed languages are growing dynamic features, and dynamically > typed languages are growing static checkers. > > > http://steve-yegge.blogspot.com.au/2008/05/dynamic- > languages-strike-back.html > > > And some background information on the debate: > > https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From michael.selik at gmail.com Wed Aug 10 16:50:19 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 10 Aug 2016 20:50:19 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 10, 2016, 4:34 PM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > > I've been trying to find (without success so far) an example of a situation > where the dynamic features of a language like Python provides a clear > advantage over languages with more than one type. > Only one type? There are a great variety of types for Python objects. Even Python classes have types. Further, you can modify types at runtime! It's a typing bonanza! I suppose you could argue we're conflating types and classes. If so, it sounds like you started this conversation with an ulterior motive and not to learn about best practices for runtime type checking in Python. > From rosuav at gmail.com Wed Aug 10 16:52:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 06:52:54 +1000 Subject: Python slang In-Reply-To: <877fboigaw.fsf@elektro.pacujo.net> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> <877fboigaw.fsf@elektro.pacujo.net> Message-ID: On Thu, Aug 11, 2016 at 5:18 AM, Marko Rauhamaa wrote: > Rustom Mody : > >> On Wednesday, August 10, 2016 at 9:31:40 PM UTC+5:30, Anders J. Munch wrote: >> Python?s inspiration and origin is ABC >> Whose assignment looked like >> PUT expr INTO var >> >> This has the salutary effect >> - Of being l-to-r (the only other such case I know is gas mov) > > Oh, there's of course COBOL: > > ADD X TO Y GIVING RESULT And Python: with open(fn) as f: ChrisA From berteh at gmail.com Wed Aug 10 16:56:40 2016 From: berteh at gmail.com (Berteh BE) Date: Wed, 10 Aug 2016 13:56:40 -0700 (PDT) Subject: cross-platform logging.config: how to set cross platform log (handlers) file location? In-Reply-To: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> References: <83a46366-357e-4333-983e-cf9453c398f1@googlegroups.com> Message-ID: Thanks Peter for the insights that args are passed to eval(). Thanks Steven for the suggestion that ?~/' may actually work in windows. Combining both I'm now successfully using the following in my logging.conf configuration file, that works in both *nix and Windows: [handler_file] class=handlers.RotatingFileHandler args=(os.path.expanduser('~/.scribusGenerator.log'),'a','maxBytes=50','backupCount=1') The tilde without an explicit call to expanduser() was not working in Windows... but with it it's working like a charm. Thank you. From gengyangcai at gmail.com Wed Aug 10 16:57:02 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Wed, 10 Aug 2016 13:57:02 -0700 (PDT) Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? In-Reply-To: References: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> Message-ID: Yea, using IDLE on OSX, Python 3.4.3. Yeah it works now ... I had to increase the font size and the indentation width. When that happened , it could define miles_driven On Thursday, August 11, 2016 at 2:09:11 AM UTC+8, Zachary Ware wrote: > On Wed, Aug 10, 2016 at 12:39 PM, Cai Gengyang wrote: > > I managed to get this piece of code to work : > > > >>>> print("This program calculates mpg.") > > This program calculates mpg. > >>>> milesdriven = input("Enter miles driven:") > > Enter miles driven: 50 > >>>> milesdriven = float(milesdriven) > >>>> gallonsused = input("Enter gallons used:") > > Enter gallons used: 100 > >>>> gallonsused = float(gallonsused) > >>>> gallonsused = float(gallonsused) > >>>> mpg = milesdriven / gallonsused > >>>> print("Miles per gallon:", mpg) > > Miles per gallon: 0.5 > >>>> print("This program calculates mpg.") > > This program calculates mpg. > >>>> milesdriven = input("Enter miles driven:") > > Enter miles driven: 100 > >>>> milesdriven = float(milesdriven) > >>>> gallonsused = input("Enter gallons used:") > > Enter gallons used: 500 > >>>> gallonsused = float(gallonsused) > >>>> mpg = milesdriven / gallonsused > >>>> print("Miles per gallon:", mpg) > > Miles per gallon: 0.2 > > > > But, why can't i define a variable like "miles_driven" with an underscore in my Python Shell ? When I try to define it , the underscore doesn't appear at all and instead I get this result : > > "miles driven" , with no underscore appearing even though I have already typed "_" between "miles" and "driven" ? > > Are you using IDLE on OSX? Try changing your font or bumping up the > size (or switch to 3.5.2, which has better defaults). > > -- > Zach From martin.schoon at gmail.com Wed Aug 10 17:34:18 2016 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 10 Aug 2016 21:34:18 GMT Subject: How do I make a video animation with transparent background? References: Message-ID: Den 2016-08-10 skrev Christian Gollwitzer : > Am 09.08.16 um 14:44 schrieb Martin Sch??n: >> Using this example: >> http://matplotlib.org/examples/animation/moviewriter.html >> and this tutorial: >> https://www.youtube.com/watch?v=h0F5X4b7jug >> I have covered a lot of ground. >> >> What I have failed to achieve is a graph with a transparent >> background. None of the leads I have found searching the Internet >> have helped me. (I may have found the needed information without >> realising it...) >> >> Any help is appreciated. > > > Here is a simple idea: Draw the background with a colour, that is not > used in the graphs, like bright blue or green. Any decent video editing > software should support colour keying, i.e. masking the green parts out, > becuase this is used in film production (green screening) > > Christian Thanks for all the input so far and not only Christian but all of you. Yes, after posting I did some more research and started to realise that I may have barked up the wrong tree when assuming my use of Matplotlib needed to be improved to get the intended result. Next on my TODO list is to look into the color composition idea. I have already looked this tutorial: https://youtu.be/sZzmksnzrX0 What I would like to end up with is something like this: https://youtu.be/aHEyRoDmDWk (graph in lower right corner and 'speedometer' in lower left corner) /Martin From ethan at stoneleaf.us Wed Aug 10 17:35:58 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 10 Aug 2016 14:35:58 -0700 Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? In-Reply-To: References: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> Message-ID: <57AB9E3E.3070501@stoneleaf.us> On 08/10/2016 01:57 PM, Cai Gengyang wrote: > Yea, using IDLE on OSX, Python 3.4.3. > > Yeah it works now ... I had to increase the font size and the indentation width. When that happened , it could define miles_driven I suspect you were defining it before, you just couldn't see the underscore. -- ~Ethan~ From lawrencedo99 at gmail.com Wed Aug 10 18:26:23 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 15:26:23 -0700 (PDT) Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: <20390c4b-71e2-4b2e-bbcf-cea223fa968d@googlegroups.com> On Wednesday, August 10, 2016 at 10:13:53 PM UTC+12, eryk sun wrote: > ctypes classes have from_buffer and from_buffer_copy methods that use > the buffer protocol. For example, for a read-only buffer you can use > from_buffer_copy: > > >>> b = bytes(b'1234') > >>> a = (ctypes.c_char * 3).from_buffer_copy(b, 1) > >>> a[:] > b'234' Can I avoid the copying? For example, in Qahirah the following method creates and fills in an ImageSurface from a stream of bytes in PNG format: @classmethod def create_from_png_bytes(celf, data) : "creates an ImageSurface from a PNG-format data sequence. This can be" \ " of the bytes or bytearray types, or an array.array with \"B\" type code." offset = 0 def read_data(_, data, length) : nonlocal offset if offset + length <= len(data) : ct.memmove(data, baseadr + offset, length) offset += length status = CAIRO.STATUS_SUCCESS else : status = CAIRO.STATUS_READ_ERROR #end if return \ status #end read_data #begin create_from_png_bytes if isinstance(data, bytes) or isinstance(data, bytearray) : data = array.array("B", data) elif not isinstance(data, array.array) or data.typecode != "B" : raise TypeError("data is not bytes, bytearray or array of bytes") #end if baseadr = data.buffer_info()[0] return \ celf.create_from_png_stream(read_data, None) #end create_from_png_bytes But it has to copy the bytes into an array.array object, then decode that. Is there a way it could access the bytes memory directly? From lawrencedo99 at gmail.com Wed Aug 10 18:29:16 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 15:29:16 -0700 (PDT) Subject: Python slang In-Reply-To: <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> Message-ID: On Thursday, August 11, 2016 at 5:35:03 AM UTC+12, Rustom Mody wrote: > On Wednesday, August 10, 2016 at 9:31:40 PM UTC+5:30, Anders J. Munch wrote: > Python?s inspiration and origin is ABC > Whose assignment looked like > PUT expr INTO var HyperTalk did that in 1987. Is that where GvR got the idea from? In HyperTalk you could also do ?put ... before/after ...? to do string prepending/appending. From lawrencedo99 at gmail.com Wed Aug 10 18:30:11 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 15:30:11 -0700 (PDT) Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> Message-ID: <3c9d12ed-8dca-4e68-a74b-a4f9c4020324@googlegroups.com> On Thursday, August 11, 2016 at 4:01:40 AM UTC+12, Anders J. Munch wrote: > Lawrence D?Oliveiro: >> >>> [...] as much like C++ as possible. >> >> Nevertheless, Python copied the C misfeature [...] > > You segued a little too easily from C++ to C. Because C++ copied that same C misfeature. From lawrencedo99 at gmail.com Wed Aug 10 18:31:04 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 15:31:04 -0700 (PDT) Subject: cmd prompt does not recognizes python command on Windows 7 In-Reply-To: References: Message-ID: <46f6ba5d-e8bf-4b94-b3da-c6a0dce10712@googlegroups.com> On Wednesday, August 10, 2016 at 4:47:33 PM UTC+12, sh.a... at gmail.com wrote: > i have installed python 3.5 , but the python command is not recognized Windows does not make it easy to install things, does it... From greg.ewing at canterbury.ac.nz Wed Aug 10 18:31:21 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 11 Aug 2016 10:31:21 +1200 Subject: ctypes And The WACAH Principle In-Reply-To: <2eba7724-bc28-4335-b92f-76884be80c64@googlegroups.com> References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> <2eba7724-bc28-4335-b92f-76884be80c64@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: >>It will let you create non-leaky wrappers ... > > Why would I want to do that? You mentioned that ctypes wrappers are leaky, which I assumed was meant as a criticism. But if you *like* your wrappers to be leaky, by all means go for it. :-) -- Greg From lawrencedo99 at gmail.com Wed Aug 10 18:32:59 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 15:32:59 -0700 (PDT) Subject: How do I make a video animation with transparent background? In-Reply-To: References: Message-ID: <607cc26d-ca5a-4509-b4f5-da7c0b6c627f@googlegroups.com> On Thursday, August 11, 2016 at 9:34:33 AM UTC+12, Martin Sch??n wrote: > Next on my TODO list is to look into the color composition idea. I > have already looked this tutorial: > https://youtu.be/sZzmksnzrX0 If the images you generate have an alpha channel, Blender can composite them directly, without needing colour-keying. From rosuav at gmail.com Wed Aug 10 18:42:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 08:42:45 +1000 Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> <2eba7724-bc28-4335-b92f-76884be80c64@googlegroups.com> Message-ID: On Thu, Aug 11, 2016 at 8:31 AM, Gregory Ewing wrote: > Lawrence D?Oliveiro wrote: > >>> It will let you create non-leaky wrappers ... >> >> >> Why would I want to do that? > > > You mentioned that ctypes wrappers are leaky, which I > assumed was meant as a criticism. But if you *like* > your wrappers to be leaky, by all means go for it. :-) Are they leaky in the sense of leaky abstractions, or memory leaks? ChrisA From jpablo.romero at gmail.com Wed Aug 10 18:44:10 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Wed, 10 Aug 2016 15:44:10 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: I've made my motives to start this conversation very clear: """ Determine if there is a style of writing code in a dynamic language that minimizes the occurrence of runtime errors AND avoids two things: 1) using runtime checks all over the place (either ad hoc or using tools like Valideer) 2) attempting to test for things that a type checker would normally do (not that this is completely possible anyway) """ And I really appreciate people taking their time to contribute! As to why I asked that, there are several reasons: I have a very concrete need right now to find pragmatic ways to increase code quality, reduce number of defects, etc. in a Python code base. But also I want to understand better the mind set and culture of Python's community. You make a good point that my last remark is unrelated to that goal, but please don't take it as an attack of any sort. Languages are not monolithically good / bad. They have features that can be individually evaluated in light of different goals (just to name a few: is it performant enough?, is it easy to learn?, is it easy to write? is it easy to make it correct? is it expressive? is it easy to refactor? is it easy to deploy?, does it have good documentation? etc etc etc. ). Also languages evolve, so features that were considered positive at some point can become out of fashion over the years. I'm honestly interested in finding good examples of cases where the dynamic features of some language are a better solution than static typing. This is of course more useful in languages that support both paradigms. Juan Pablo 2016-08-10 13:50 GMT-07:00 Michael Selik : > > > On Wed, Aug 10, 2016, 4:34 PM Juan Pablo Romero M?ndez < > jpablo.romero at gmail.com> wrote: > >> >> I've been trying to find (without success so far) an example of a >> situation >> where the dynamic features of a language like Python provides a clear >> advantage over languages with more than one type. >> > > Only one type? There are a great variety of types for Python objects. Even > Python classes have types. Further, you can modify types at runtime! It's a > typing bonanza! > > I suppose you could argue we're conflating types and classes. If so, it > sounds like you started this conversation with an ulterior motive and not > to learn about best practices for runtime type checking in Python. > >> From greg.ewing at canterbury.ac.nz Wed Aug 10 18:48:43 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 11 Aug 2016 10:48:43 +1200 Subject: Make sure you removed all debugging print statements error In-Reply-To: References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: Chris Angelico wrote: > I suppose you could put down Kelvin as a last name and either Baron or > Lord as a first name, but that'd just be cheating... Or put NULL as the first name and have fun watching the database break. http://www.wired.com/2015/11/null/ Or perhaps even profit: http://www.dailymail.co.uk/travel/travel_news/article-3513652/The-cleverest-time-Man-gets-free-holidays-car-rentals-changing-surname-Null.html -- Greg From ian.g.kelly at gmail.com Wed Aug 10 18:52:42 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 10 Aug 2016 16:52:42 -0600 Subject: Python slang In-Reply-To: References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> Message-ID: On Aug 10, 2016 4:36 PM, "Lawrence D?Oliveiro" wrote: On Thursday, August 11, 2016 at 5:35:03 AM UTC+12, Rustom Mody wrote: > On Wednesday, August 10, 2016 at 9:31:40 PM UTC+5:30, Anders J. Munch wrote: > Python?s inspiration and origin is ABC > Whose assignment looked like > PUT expr INTO var HyperTalk did that in 1987. Is that where GvR got the idea from? Probably not, since the design of ABC dates back to at least 1976. And GvR didn't design the language anyway. From rosuav at gmail.com Wed Aug 10 18:55:48 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 08:55:48 +1000 Subject: Make sure you removed all debugging print statements error In-Reply-To: References: <14924f85-413f-4808-9f3e-15de194d6cdb@googlegroups.com> <2d31c8be-a8f3-4536-9af2-17b1d892bc27@googlegroups.com> Message-ID: On Thu, Aug 11, 2016 at 8:48 AM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> I suppose you could put down Kelvin as a last name and either Baron or >> Lord as a first name, but that'd just be cheating... > > > Or put NULL as the first name and have fun watching > the database break. > > http://www.wired.com/2015/11/null/ > > Or perhaps even profit: > > http://www.dailymail.co.uk/travel/travel_news/article-3513652/The-cleverest-time-Man-gets-free-holidays-car-rentals-changing-surname-Null.html NULL should be completely different from the string 'Null' (in any mixture of case). Failing that check means a failure in the front end, or in how things are checked for. ChrisA From greg.ewing at canterbury.ac.nz Wed Aug 10 19:10:55 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 11 Aug 2016 11:10:55 +1200 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa87ce$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: Dennis Lee Bieber wrote: > Stray gamma rays flipping > bits in memory (though a bit harder to do, considering the firmware was in > some hard-wired mess of core, I think the RAM was core too, so pretty much immune to gamma rays as well (at least at any intensity that wouldn't instantly kill the crew), and non-volatile. -- Greg From michael.selik at gmail.com Wed Aug 10 19:32:15 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 10 Aug 2016 23:32:15 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Aug 10, 2016, 6:44 PM Juan Pablo Romero M?ndez < jpablo.romero at gmail.com> wrote: > As to why I asked that, there are several reasons: I have a very concrete > need right now to find pragmatic ways to increase code quality, reduce > number of defects, etc. in a Python code base. But also I want to > understand better the mind set and culture of Python's community. > That's a much better question. Arguing about type checking is somewhat of a red herring. It's best not to fight the language, but to use its strengths. I often find that trying to add type and value checks just makes my code harder to read and harder to maintain, even if they solve a particular bug in the moment. A better solution is to refactor in such a way that those checks are unnecessary. Are you familiar with context managers and the iterator protocol? If not, I suggest you start by looking for setup/cleanup code or try/except/finally blocks that you can refactor into a context manager and by looking for awkward loops with explicit external state that you can refactor into an iterator. If you really must add type- and value-checks, I suggest trying to place them in a property so that they don't clutter the more interesting code. > From torriem at gmail.com Wed Aug 10 19:57:09 2016 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Aug 2016 17:57:09 -0600 Subject: Python slang In-Reply-To: <1470845983.1463573.691529097.0E053AB7@webmail.messagingengine.com> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <3d7fec12-8bea-4f26-afa1-f8516763bfc5@googlegroups.com> <1470845983.1463573.691529097.0E053AB7@webmail.messagingengine.com> Message-ID: <6913354c-f5ba-d392-dc1f-93882f4fe564@gmail.com> On 08/10/2016 10:19 AM, Random832 wrote: > On Wed, Aug 10, 2016, at 07:59, Dennis Lee Bieber wrote: >> The use of = also has a long history... FORTRAN (where the comparison >> was .EQ.), BASIC (granted, K&K required assignment to start with the >> keyword LET, so the use of = was mainly a delimiter between target and >> expression being assigned). > > Visual Basic actually uses = for both assignment and comparison > *without* the Let keyword, it gets by on the fact that assignment is not > an expression. But the grammar must still be a bit complex as sometimes the LHS of the = is an expression, as well as the RHS. But I guess context makes it all work. Still I've always thought the grammars for modern BASIC with all its baggage must make for a complex grammar. From torriem at gmail.com Wed Aug 10 20:02:19 2016 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Aug 2016 18:02:19 -0600 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 08/10/2016 11:29 AM, Michael Selik wrote: > On Tue, Aug 9, 2016 at 9:31 PM Steven D'Aprano > wrote: > >> >> http://steve-yegge.blogspot.com.au/2008/05/dynamic-languages-strike-back.html > > > Great link. I enjoyed the video, too. > https://www.youtube.com/watch?v=tz-Bb-D6teE > > Buried deep in the QA section, there's a comment from the audience (I'll > paraphrase), that compiler type-checking is wonderful. Yegge replies that, "I > realized early in my career that I would actually rather have a runtime > error than a compile error." True but sometimes a compile error would save a lot of time and frustration later on when the runtime error occurs. For example in Python if I am setting an attribute on an object and mispell the attribute, there is no compile error, but often I won't get a runtime error from that, but I will get buggy behavior that sometimes but not always occurs. Very frustrating. Pylint catches that of course, so it's a weaker argument for compilers than some others perhaps. From torriem at gmail.com Wed Aug 10 20:06:30 2016 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Aug 2016 18:06:30 -0600 Subject: Python slang In-Reply-To: <6913354c-f5ba-d392-dc1f-93882f4fe564@gmail.com> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <3d7fec12-8bea-4f26-afa1-f8516763bfc5@googlegroups.com> <1470845983.1463573.691529097.0E053AB7@webmail.messagingengine.com> <6913354c-f5ba-d392-dc1f-93882f4fe564@gmail.com> Message-ID: <17f8c029-3987-de5f-8980-3ad434d645b7@gmail.com> On 08/10/2016 05:57 PM, Michael Torrie wrote: > On 08/10/2016 10:19 AM, Random832 wrote: >> On Wed, Aug 10, 2016, at 07:59, Dennis Lee Bieber wrote: >>> The use of = also has a long history... FORTRAN (where the comparison >>> was .EQ.), BASIC (granted, K&K required assignment to start with the >>> keyword LET, so the use of = was mainly a delimiter between target and >>> expression being assigned). >> >> Visual Basic actually uses = for both assignment and comparison >> *without* the Let keyword, it gets by on the fact that assignment is not >> an expression. > > But the grammar must still be a bit complex as sometimes the LHS of the > = is an expression, as well as the RHS. But I guess context makes it > all work. Still I've always thought the grammars for modern BASIC with > all its baggage must make for a complex grammar. Sigh. So much for English grammar. That should be "complex parser." From lawrencedo99 at gmail.com Wed Aug 10 21:39:11 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 18:39:11 -0700 (PDT) Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> <2eba7724-bc28-4335-b92f-76884be80c64@googlegroups.com> Message-ID: <35a72c5e-c333-4710-890f-0c29387e1c86@googlegroups.com> On Thursday, August 11, 2016 at 10:31:36 AM UTC+12, Gregory Ewing wrote: > > Lawrence D?Oliveiro wrote: > >>>It will let you create non-leaky wrappers ... >> >> Why would I want to do that? > > You mentioned that ctypes wrappers are leaky, which I > assumed was meant as a criticism. WACAH! From lawrencedo99 at gmail.com Wed Aug 10 21:40:14 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 18:40:14 -0700 (PDT) Subject: Python slang In-Reply-To: <877fboigaw.fsf@elektro.pacujo.net> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> <877fboigaw.fsf@elektro.pacujo.net> Message-ID: <93bd7235-f2bf-40b5-8b99-31667f5b48e0@googlegroups.com> On Thursday, August 11, 2016 at 7:18:28 AM UTC+12, Marko Rauhamaa wrote: > Oh, there's of course COBOL: > > ADD X TO Y GIVING RESULT I think COBOL has the same usage of ?=? as BASIC, as assignment or equality comparison, depending on context. From steve+python at pearwood.info Wed Aug 10 22:14:19 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 12:14:19 +1000 Subject: Why can't I define a variable like "miles_driven" with an underscore in Python 3.4.3 ? References: <71ef0a57-5a9a-4afa-b313-8ad4dd9420ce@googlegroups.com> Message-ID: <57abdf7c$0$1603$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 06:57 am, Cai Gengyang wrote: > Yea, using IDLE on OSX, Python 3.4.3. > > Yeah it works now ... I had to increase the font size and the indentation > width. When that happened , it could define miles_driven You were always defining "miles_driven", it is just that the display engine used by IDLE on OS X wasn't capable of displaying the underscore using the combination of typeface (font), size and line height. My guess is that you had it set to a very small size and the underscore just wasn't visible. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Wed Aug 10 23:23:00 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 11 Aug 2016 03:23:00 +0000 Subject: ctypes And The WACAH Principle In-Reply-To: <20390c4b-71e2-4b2e-bbcf-cea223fa968d@googlegroups.com> References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> <20390c4b-71e2-4b2e-bbcf-cea223fa968d@googlegroups.com> Message-ID: On Wed, Aug 10, 2016 at 10:26 PM, Lawrence D?Oliveiro wrote: > But it has to copy the bytes into an array.array object, then decode that. Is there a way it > could access the bytes memory directly? ctypes cast, string_at, wstring_at, memmove, and memset are implemented as FFI calls. Since ctypes FFI calls convert bytes arguments to the base address of the buffer, you can cast() bytes to c_void_p to get the address. For example: >>> data = b'abc' >>> addr = ctypes.cast(data, ctypes.c_void_p).value >>> ctypes.string_at(addr, 3) 'abc' For bytearray, use the buffer interface to create a ctypes array. The array is not a copy. For example: >>> data = bytearray(b'abc') >>> a = (ctypes.c_char * len(data)).from_buffer(data) >>> addr = ctypes.addressof(a) >>> ctypes.string_at(addr, 3) 'abc' From steve+python at pearwood.info Wed Aug 10 23:53:31 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 13:53:31 +1000 Subject: Asynchronous programming Message-ID: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> The latest versions of Python are introducing new keywords for asynchronous programming, async and await. See PEP 492: https://www.python.org/dev/peps/pep-0492/ Is there a good beginner's tutorial introducing the basics of asynchronous programming? Starting with, why and where would you use it? I've read "What colour is your function" and it isn't quite jelling for me. http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ How is this the same as, or different from, event-based programming? I'm familiar with programming in an event-based language where the interpreter itself provides an event loop and dispatches messages to your objects: - I define objects such as buttons, text fields, etc., and give them methods ("handlers") which handle certain messages such as "mouseUp", etc.; - the interpreter runs in a loop, firing off messages in response to the user's actions; - there's a message passing hierarchy, whereby messages are first received by (let's say) the button the user clicked on, if not handled by a mouseUp method it is passed on to the next object in the hierarchy (say, the window), and then finally to the interpreter itself, at which point it either ignores the message or raises an exception. If I'm using async and await in Python, where's the event loop? What are the messages, and where are they sent? Or am I on the wrong track altogether? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Aug 11 00:02:05 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 14:02:05 +1000 Subject: Vectorized functions Message-ID: <57abf8bf$0$1611$c3e8da3$5496439d@news.astraweb.com> Its sometimes very useful to apply a function to each element of a list in turn. One common term for that is "vectorized function". Julia has a convenient syntactic shortcut for this: http://julia.readthedocs.io/en/latest/manual/functions/#dot-syntax-for-vectorizing-functions func(arg) calls func with a single argument, "arg", while: func.(arg) expects "arg" to be an array of values, and returns a new array generated by calling func(x) for each element x in the array. That's equivalent to the list comprehension and functional syntax: [func(x) for x in arg] map(func, arg) Here's a neat little decorator which decorates a function with a special callable attribute "v" which operates somewhat like Julia's dot syntax: def vectorize(func): def v(seq, **kw): return [func(x, **kw) for x in seq] func.v = v return func py> @vectorize ... def add2(x): ... return x+2 ... py> add2.v([100, 200, 300, 400]) [102, 202, 302, 402] This version is just a minimal toy: it always returns a list, regardless of the input argument. As such it doesn't offer much more than a list comprehension or map(). I have a more complex version which behaves more like Python 2's map() in that it tries to return an object of the same type as its input (e.g. a string if you call it on a string, a tuple if you call it on a tuple, etc). Is there any other functionality which would make this more useful? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Thu Aug 11 00:09:47 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 10 Aug 2016 21:09:47 -0700 Subject: Vectorized functions References: <57abf8bf$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87vaz8ymic.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > Is there any other functionality which would make this more useful? Cute, but map or listcomps work ok. Here's the Haskell equivalent to your example, fwiw, using the <$> operator from the Control.Applicative module: (+2) <$> [1,2,3] => [3,4,5] If you haven't tried Haskell yet, you might enjoy it. http://learnyouahaskell.com is a good place to start. From tjreedy at udel.edu Thu Aug 11 00:38:27 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 11 Aug 2016 00:38:27 -0400 Subject: Asynchronous programming In-Reply-To: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/10/2016 11:53 PM, Steven D'Aprano wrote: > The latest versions of Python are introducing new keywords for > asynchronous programming, async and await. See PEP 492: > > https://www.python.org/dev/peps/pep-0492/ > > Is there a good beginner's tutorial introducing the basics of > asynchronous programming? Starting with, why and where would you use > it? > > I've read "What colour is your function" and it isn't quite jelling > for me. > > http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ > > > > How is this the same as, or different from, event-based programming? > I'm familiar with programming in an event-based language where the > interpreter itself provides an event loop and dispatches messages to > your objects: > > - I define objects such as buttons, text fields, etc., and give them > methods ("handlers") which handle certain messages such as "mouseUp", > etc.; > > - the interpreter runs in a loop, firing off messages in response to > the user's actions; > > - there's a message passing hierarchy, whereby messages are first > received by (let's say) the button the user clicked on, if not > handled by a mouseUp method it is passed on to the next object in the > hierarchy (say, the window), and then finally to the interpreter > itself, at which point it either ignores the message or raises an > exception. > > If I'm using async and await in Python, where's the event loop? What > are the messages, and where are they sent? Or am I on the wrong track > altogether? You might be able to glean something from the succession of files I uploaded to https://bugs.python.org/issue27546 Integrate tkinter and asyncio (and async) I started with just mixing tk and asyncio callbacks. After some struggle with similar question as you ask above, I ended up with more or less callback free code using async def and async for. Once I got over the hump, I rather like it. -- Terry Jan Reedy From rosuav at gmail.com Thu Aug 11 00:41:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 14:41:27 +1000 Subject: Asynchronous programming In-Reply-To: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Aug 11, 2016 at 1:53 PM, Steven D'Aprano wrote: > How is this the same as, or different from, event-based programming? I'm > familiar with programming in an event-based language where the interpreter > itself provides an event loop and dispatches messages to your objects: > > - I define objects such as buttons, text fields, etc., and give them methods > ("handlers") which handle certain messages such as "mouseUp", etc.; > > - the interpreter runs in a loop, firing off messages in response to the > user's actions; > > - there's a message passing hierarchy, whereby messages are first received > by (let's say) the button the user clicked on, if not handled by a mouseUp > method it is passed on to the next object in the hierarchy (say, the > window), and then finally to the interpreter itself, at which point it > either ignores the message or raises an exception. > > > If I'm using async and await in Python, where's the event loop? What are the > messages, and where are they sent? Or am I on the wrong track altogether? There'll be one somewhere, probably at some top-level main loop. Async programming isn't usually compared with events *as such*, but with callbacks. Consider these three ways of doing a database transaction: def synchronous(id): trn = conn.begin_transaction() trn.execute("select name from people where id=%d", (id,)) name, = trn.fetchone() trn.execute("update people set last_seen=now() where id=%d", (id,)) trn.commit() return name def callbacks_1(cb, id): conn.begin_transaction(callbacks_2, cb, id) def callbacks_2(trn, cb, id): trn.execute("select name from people where id=%d", (id,), callbacks_3, cb, id) def callbacks_3(trn, cb, id): trn.fetchone(callbacks_4, cb, id) def callbacks_4(trn, data, cb, id): name, = data trn.execute("update people set last_seen=now() where id=%d", (id,), callbacks_5, cb, name) def callbacks_5(trn, cb, name): trn.commit(callbacks_6, cb, name) def callbacks_6(trn, cb, name): cb(name) def asynchronous(id): trn = yield from conn.begin_transaction() yield from trn.execute("select name from people where id=%d", (id,)) name, = yield from trn.fetchone() yield from trn.execute("update people set last_seen=now() where id=%d", (id,)) yield from trn.commit() return name Change this last one to "async def" and change the "yield from"s into "await", and you get the new syntax, but it's still built on top of generators. Basically, async code works by stopping execution in the middle of a logical operation, and then resuming it later. (The callbacks example could be improved on some by using closures, and in JavaScript, where this is really common, you can use anonymous functions. I still prefer the last form though.) Every time you hit an "await", you call into that function and see what it says - in this case, it's going to "yield" somewhere, sending back some sort of awaitable thing, eg a half-done database operation. When that operation completes, you go into the next step of the process. How does this align with event handling? Not very well. Normally event handling means non-linear execution - for instance, button clicks on a GUI aren't going to step nicely through a progression. But the two work well together; if you have a primary event loop that starts out by looking for GUI events, you could have something like the above asynchronous function, and it'll give control back to the main loop while the database is working - thus making the GUI responsive again. You can mess around with the front end while the back end does its work. Hope that helps some. ChrisA From anilreddy449 at gmail.com Thu Aug 11 00:46:25 2016 From: anilreddy449 at gmail.com (Anil reddy reddy M) Date: Wed, 10 Aug 2016 21:46:25 -0700 (PDT) Subject: Send Msg To Specific Tcp Client from Tcp Server Python Message-ID: I was written a simple tcp server in python, which is nicely communicating with multiple tcp clients. My Tcp Server can accept multiple clients at time, each client as a new thread. I want send message to specific tcp client after some time. I can client Address from connction, addr = socket.accept(), but i am not able send msg to client with client IP.POrT. I would be thankfull for any help i get From no.email at nospam.invalid Thu Aug 11 01:06:13 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 10 Aug 2016 22:06:13 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87r39vzygq.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > Is there a good beginner's tutorial introducing the basics of asynchronous > programming? Starting with, why and where would you use it? You might look at some node.js tutorials since there are about a gazillion of them and some of them must be good. Also twistedmatrix.org for an older Python package of that sort. The basic reason to do it is it lets you serve a lot of concurrent i/o channels (network connections, say) without using threads. If you want to read a packet, you launch a non-blocking read that returns immediately, and then transfer control (often through some behind-the-scenes magic) to an event loop that dispatches back to you after your read request completes, through either a callback or a coroutine jump depending on which framework you're using. In Python that gets you much higher performance than threads, plus you avoid the usual bunch of hazards that parents used to scare their kids with about multi-threaded programming. > How is this the same as, or different from, event-based programming? The basic characteristic of asynchronous programming is that it involves changing all your usual blocking i/o calls to non-blocking ones, so your program can keep running as soon as your request is started. That's not always the case with event-based programming in general, where event handlers might do i/o. The basic problem you face is the inversion of control needed to get in and out of the event loop, compared to the sequential blocking i/o that you'd use in a threaded or non-concurrent program. The new thing in Python is asyncio, which uses coroutines instead of callbacks to switch between event handlers. It's asynchronous programming either way, but coroutines can be seen as a cleaner implementation technique depending on your POV. Coroutines in the form of "yield from" first appeared in Python 3.??? and became more fully integrated with the new async and await keywords. Before that, async concurrency in Python was done using callback schemes. Twisted used to work like that (I don't know if it still does). I hated it but I was in a minority. > If I'm using async and await in Python, where's the event loop? What > are the messages, and where are they sent? Or am I on the wrong track > altogether? It's supplied by the asyncio package. Btw, David Beazely's article about Python coroutines is very good, though it was written before the new keywords were added. From rosuav at gmail.com Thu Aug 11 01:19:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 15:19:07 +1000 Subject: Asynchronous programming In-Reply-To: <87r39vzygq.fsf@jester.gateway.pace.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> Message-ID: On Thu, Aug 11, 2016 at 3:06 PM, Paul Rubin wrote: > The basic reason to do it is it lets you serve a lot of concurrent i/o > channels (network connections, say) without using threads. If you want > to read a packet, you launch a non-blocking read that returns > immediately, and then transfer control (often through some > behind-the-scenes magic) to an event loop that dispatches back to you > after your read request completes, through either a callback or a > coroutine jump depending on which framework you're using. In Python > that gets you much higher performance than threads, plus you avoid the > usual bunch of hazards that parents used to scare their kids with about > multi-threaded programming. > Hmm. I'm not sure about that last bit. In order to properly support asynchronous I/O and the concurrency and reentrancy that that implies, you basically need all the same disciplines that you would for threaded programming - no shared mutable state, keep your locals local, etc. The only difference is that context switches are coarser than they are in threading (and remember, CPython already refuses to switch threads in the middle of one bytecode operation, so most of the worst horrors can't happen even there). But maybe I'm too comfortable with threads. It's entirely possible; my first taste of reentrancy was interrupt handling in real mode 80x86 assembly language, and in comparison to that, threads are pretty benign! ChrisA From ernest.bonat at gmail.com Thu Aug 11 01:20:45 2016 From: ernest.bonat at gmail.com (Ernest Bonat, Ph.D.) Date: Wed, 10 Aug 2016 22:20:45 -0700 Subject: Generate reports in Python Message-ID: Hi, I'm looking for best modules/practices to generate reports (HTML, PDF, etc) in Python. Good ideas are very appreciated. Thanks Ernest Bonat, Ph.D. Senior Software Engineer Senior Data Scientist From paul.nospam at rudin.co.uk Thu Aug 11 01:34:28 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Thu, 11 Aug 2016 06:34:28 +0100 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87k2fn3m3f.fsf@rudin.co.uk> Steven D'Aprano writes: > > Is there a good beginner's tutorial introducing the basics of asynchronous > programming? Starting with, why and where would you use it? You could do worse than watch Dave Beazley's pycon talk: https://www.youtube.com/watch?v=lYe8W04ERnY From dieter at handshake.de Thu Aug 11 02:06:23 2016 From: dieter at handshake.de (dieter) Date: Thu, 11 Aug 2016 08:06:23 +0200 Subject: Send Msg To Specific Tcp Client from Tcp Server Python References: Message-ID: <871t1vsuu8.fsf@handshake.de> Anil reddy reddy M writes: > I was written a simple tcp server in python, which is nicely communicating with multiple tcp clients. My Tcp Server can accept multiple clients at time, each client as a new thread. I want send message to specific tcp client after some time. > I can client Address from connction, addr = socket.accept(), but i am not able send msg to client with client IP.POrT. I would be thankfull for any help i get TCP channels are bidirectional: you can send messages in both directions. Thus, writing to the server's socket (for a specific client) sends the written data to the client. From auriocus at gmx.de Thu Aug 11 02:21:36 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 11 Aug 2016 08:21:36 +0200 Subject: Asynchronous programming In-Reply-To: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 11.08.16 um 05:53 schrieb Steven D'Aprano: > How is this the same as, or different from, event-based programming? I'm > familiar with programming in an event-based language where the interpreter > itself provides an event loop and dispatches messages to your objects I'll just comment on that one. Asynchronous programming is event based programming. In fact if you drop down to the C level, it works the same - you get callbacks when new data arives over a socket, or when a timer triggers. asyncio is just a way that makes writing stateful callback handlers easier, in the same way that generators are easier to write using "yield" than as a function with a global variable. In typical GUI code, there are usually not that many places qhere ou have sequential code. A simple exmaple might be a counter. Using asyncio and a properly integrated GUI toolkit, you could write it as (pseudo-code) async def counter(): for i in range(10): button.settext("click me %d"%i) await button.click() button.disable() messageBox("You reached the end!") Writing that same thing in callback code to button.on_click() is obviously less fun and feels inverted. Christian From no.email at nospam.invalid Thu Aug 11 02:21:37 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 10 Aug 2016 23:21:37 -0700 Subject: Generate reports in Python References: Message-ID: <87mvkjzuz2.fsf@jester.gateway.pace.com> "Ernest Bonat, Ph.D." writes: > I'm looking for best modules/practices to generate reports (HTML, PDF, etc) > in Python. Good ideas are very appreciated. See: http://www.reportlab.com/ for a well regarded package From auriocus at gmx.de Thu Aug 11 02:34:22 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 11 Aug 2016 08:34:22 +0200 Subject: Asynchronous programming In-Reply-To: References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 11.08.16 um 06:38 schrieb Terry Reedy: > You might be able to glean something from the succession of files I > uploaded to > https://bugs.python.org/issue27546 > Integrate tkinter and asyncio (and async) > > I started with just mixing tk and asyncio callbacks. After some > struggle with similar question as you ask above, I ended up with more or > less callback free code using async def and async for. Once I got over > the hump, I rather like it. Interesting discussion! Tcl itself acquired coroutines some years ago (in 8.6) and people in this community are very excited with async based programming. I'm convinced that it is possible to integrate Tcl's event loop with asyncio's loop without regular update polling. This might require a patch to Tkinter at the C level. For example, an easy way is to put Tcl/Tk in it's own thread. The Tcl event loop can be woken up from a different thread by the TclThreadQueueEvent() and TclTheradAlert() API calls. Basically, you would spawn a new thread (from the C level), when loading Tkinter, which loads Tk and sits in a Tcl event loop ("vwait forever"). tk.eval() and tk.call() would be redefined to post events using the aforementioned functions. When a Tcl callback arrives, it would send an event to asyncio in the main thread - no idea how this works, though. In the end, it would also allow to use Tkinter from different Python threads, because all calls would be rooted to the one instance only. Christian From anilreddy449 at gmail.com Thu Aug 11 02:44:05 2016 From: anilreddy449 at gmail.com (Anil reddy reddy M) Date: Wed, 10 Aug 2016 23:44:05 -0700 (PDT) Subject: Send Msg To Specific Tcp Client from Tcp Server Python In-Reply-To: References: <871t1vsuu8.fsf@handshake.de> Message-ID: <5ef9cb35-219e-4f9e-8d2c-7f3ee7900a23@googlegroups.com> On Thursday, August 11, 2016 at 11:36:47 AM UTC+5:30, dieter wrote: > Anil reddy reddy M writes: > > > I was written a simple tcp server in python, which is nicely communicating with multiple tcp clients. My Tcp Server can accept multiple clients at time, each client as a new thread. I want send message to specific tcp client after some time. > > I can client Address from connction, addr = socket.accept(), but i am not able send msg to client with client IP.POrT. I would be thankfull for any help i get > > TCP channels are bidirectional: you can send messages in both > directions. Thus, writing to the server's socket (for a specific client) > sends the written data to the client. Hi, Thanks for your reply, But how can i find client socketfd when i want send message to specific client.I can store connected client ip, port in database. how can use that ip, port when i want send. Sincerely Anil, From lawrencedo99 at gmail.com Thu Aug 11 02:46:53 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 10 Aug 2016 23:46:53 -0700 (PDT) Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> <20390c4b-71e2-4b2e-bbcf-cea223fa968d@googlegroups.com> Message-ID: <0679a7e7-398b-42a6-82e5-103006d8ee4a@googlegroups.com> On Thursday, August 11, 2016 at 3:23:58 PM UTC+12, eryk sun wrote: > > ...you can cast() bytes to c_void_p to get the address. > > ... > > For bytearray, use the buffer interface to create a ctypes array. The > array is not a copy. Cool! Those pointers* are much appreciated. *So to speak. :) From auriocus at gmx.de Thu Aug 11 02:49:56 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 11 Aug 2016 08:49:56 +0200 Subject: Vectorized functions In-Reply-To: <57abf8bf$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57abf8bf$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 11.08.16 um 06:02 schrieb Steven D'Aprano: > Here's a neat little decorator which decorates a function with a special > callable attribute "v" which operates somewhat like Julia's dot syntax: > > def vectorize(func): > def v(seq, **kw): > return [func(x, **kw) for x in seq] > func.v = v > return func > > > > py> @vectorize > ... def add2(x): > ... return x+2 > ... > py> add2.v([100, 200, 300, 400]) > [102, 202, 302, 402] > > Neat. I suspect, that in addition to the higher-level syntax, it also improves the speed in Julia, because the loop is implicit and it compiles to native code using LLVM. Numpy's ufunc comes to mind - there is also a vectorizer for Python functions http://docs.scipy.org/doc/numpy/reference/generated/numpy.frompyfunc.html#numpy.frompyfunc Christian From no.email at nospam.invalid Thu Aug 11 03:01:06 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 11 Aug 2016 00:01:06 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87eg5vzt59.fsf@jester.gateway.pace.com> Christian Gollwitzer writes: > I'm convinced that it is possible to integrate Tcl's event loop with > asyncio's loop without regular update polling. This might require a > patch to Tkinter at the C level. For example, an easy way is to put > Tcl/Tk in it's own thread. ... I did something like that a while back, putting tkinter in its own thread but also using its .after() method to wake up the tkinter loop every 50 msec or something like that. I didn't notice any significant cpu load or UI lag from that, and it was on a fairly slow machine (300 mhz ARM, I think). The tkinter thread received callbacks from tkinter and passed them on to the application's main dispatch loop through a normal Python queue. So there was no need to patch the tkinter C code. From marko at pacujo.net Thu Aug 11 03:15:14 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 11 Aug 2016 10:15:14 +0300 Subject: Python slang References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <9f94fa2c-1ba9-4723-85dd-5923d1d22b0a@googlegroups.com> <877fboigaw.fsf@elektro.pacujo.net> <93bd7235-f2bf-40b5-8b99-31667f5b48e0@googlegroups.com> Message-ID: <87twerhj3x.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > On Thursday, August 11, 2016 at 7:18:28 AM UTC+12, Marko Rauhamaa wrote: > >> Oh, there's of course COBOL: >> >> ADD X TO Y GIVING RESULT > > I think COBOL has the same usage of ?=? as BASIC, as assignment or > equality comparison, depending on context. There's the COMPUTE verb that works a bit like that. However, the ADD X TO Y [GIVING Z] MULTIPLY X BY Y [GIVING Z] DIVIDE X BY Y [GIVING Z] is more traditional, I would guess. Also, for simple assignments, there's MOVE: MOVE 'INDIA' TO WS-COUNTRY. My uncle made a career out of COBOL programming. He described his work as consisting mostly of MOVE statements. Marko From marko at pacujo.net Thu Aug 11 03:55:41 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 11 Aug 2016 10:55:41 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> Message-ID: <87popfhh8i.fsf@elektro.pacujo.net> Chris Angelico : > Hmm. I'm not sure about that last bit. In order to properly support > asynchronous I/O and the concurrency and reentrancy that that implies, > you basically need all the same disciplines that you would for > threaded programming Correct. Python's asyncio closely mirrors threads in its API and thought model. An async is like a thread except that you know the control will only be relinquished at an "await" statement. Thus, you probably need far less locking. One visible difference between asyncs and threads is the fact that threads don't need a special "await" construct. Instead, ordinary function calls support context switching deep down in the Python VM and the operating system. Each "await" explicitly marks a state in a state machine. Whenever you turn a nonblocking function into a blocking one, you will need to change the function call to an await statement. Asyncs have a couple of distinct advantages over threads: * You can await multiple events in a single statement allowing for multiplexing. Threads can only wait for a single event. * Asyncs can be canceled; in general, threads can't. > But maybe I'm too comfortable with threads. It's entirely possible; my > first taste of reentrancy was interrupt handling in real mode 80x86 > assembly language, and in comparison to that, threads are pretty > benign! I steer clear of threads if I can because: * No multiplexing. * No cancellation. * Deadlocks and/or race conditions are virtually guaranteed. * Error symptoms are highly nonlocal making debugging very tricky. Instead, I strongly prefer asynchronous programming and multiprocessing. However, I'm afraid Python is following an unfortunate trend with asyncio. Deep down concurrent programming involves finite state machines. State machines can be expressed in many ways; traditionally, you have SDL diagrams, state/event matrices etc. Asyncio marks states with inline "await" keywords. Toy examples look really enticing and readable compared with the more traditional implementation methods. However, real-world code can quickly become spaghetti as the dozens of cells of the state/event matrix are implemented. My favorite asynchronous development model is the "callback hell," where each cell of the state/event matrix is represented by a method (or at least a switch case in C) and each state has an explicit name in the source code. The resulting code is still hard to read and hard to get right, but that complexity is unavoidable because reality just is that complex. Marko From wxjmfauth at gmail.com Thu Aug 11 04:30:28 2016 From: wxjmfauth at gmail.com (wxjmfauth at gmail.com) Date: Thu, 11 Aug 2016 01:30:28 -0700 (PDT) Subject: Generate reports in Python In-Reply-To: References: Message-ID: <2595c308-5f38-40ac-833a-f85f247f358d@googlegroups.com> Le jeudi 11 ao?t 2016 07:21:42 UTC+2, Ernest Bonat, Ph.D. a ?crit?: > Hi, > > I'm looking for best modules/practices to generate reports (HTML, PDF, etc) > in Python. Good ideas are very appreciated. > > Thanks > > Ernest Bonat, Ph.D. > Senior Software Engineer > Senior Data Scientist To produce a PDF, locally. For me, the best way is to use Python to produce a .tex file, a plain text file, and then compile or let compile this file with a TeX engine. I do not see to much how a library can beat that approach, unless it duplicates what TeX does. A Xe(La)TeX user. From rosuav at gmail.com Thu Aug 11 05:33:09 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 19:33:09 +1000 Subject: Asynchronous programming In-Reply-To: <87popfhh8i.fsf@elektro.pacujo.net> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <87popfhh8i.fsf@elektro.pacujo.net> Message-ID: On Thu, Aug 11, 2016 at 5:55 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> Hmm. I'm not sure about that last bit. In order to properly support >> asynchronous I/O and the concurrency and reentrancy that that implies, >> you basically need all the same disciplines that you would for >> threaded programming > > Correct. Python's asyncio closely mirrors threads in its API and thought > model. An async is like a thread except that you know the control will > only be relinquished at an "await" statement. Thus, you probably need > far less locking. I've never needed ANY explicit locking in a threaded Python or Pike program, and only rarely in C. But then, I'm one of those people who finds threaded programming relatively easy. > Asyncs have a couple of distinct advantages over threads: > > * You can await multiple events in a single statement allowing for > multiplexing. Threads can only wait for a single event. Err, that's kinda the point of select() and friends. There are some types of event that you can't select() on, but those also tend not to work perfectly with async programming either - because, guess what, event loops for async programs are usually built on top of select. > * Asyncs can be canceled; in general, threads can't. Sadly, it's not that simple. Some async actions can be cancelled, others cannot. At best, what you're saying is that *more* things can be cancelled if they were started asynchronously. >> But maybe I'm too comfortable with threads. It's entirely possible; my >> first taste of reentrancy was interrupt handling in real mode 80x86 >> assembly language, and in comparison to that, threads are pretty >> benign! > > I steer clear of threads if I can because: > > * No multiplexing. > > * No cancellation. See above > * Deadlocks and/or race conditions are virtually guaranteed. Definitely not. There are EASY ways to avoid deadlocks, and race conditions are only a problem when you have some other bug. Yes, threading bugs can be harder to debug; but only when you've violated the other principles. > * Error symptoms are highly nonlocal making debugging very tricky. But at least you have consistent and simple tracebacks. Callback model code generally lacks that. (That's one of the advantages of Python's generator-based async model - although it's not perfect.) > Instead, I strongly prefer asynchronous programming and multiprocessing. Multiprocessing is *exactly* as complicated as multithreading, with the additional overhead of having to pass state around explicitly. And if you're not getting the overhead of passing state around, you wouldn't have the complexity of shared state in threading, ergo both models would be identical. > My favorite asynchronous development model is the "callback hell," where > each cell of the state/event matrix is represented by a method (or at > least a switch case in C) and each state has an explicit name in the > source code. The resulting code is still hard to read and hard to get > right, but that complexity is unavoidable because reality just is that > complex. Wow. Are you for real, or are you trolling us? You actually *enjoy* callback hell? Give me yield-based asyncio any day. ChrisA From steve+comp.lang.python at pearwood.info Thu Aug 11 05:45:41 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 19:45:41 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ac4947$0$11125$c3e8da3@news.astraweb.com> On Thursday 11 August 2016 16:21, Christian Gollwitzer wrote: > In typical GUI code, there are usually not that many places qhere ou > have sequential code. A simple exmaple might be a counter. Using asyncio > and a properly integrated GUI toolkit, you could write it as (pseudo-code) > > async def counter(): > for i in range(10): > button.settext("click me %d"%i) > await button.click() > button.disable() > messageBox("You reached the end!") > > Writing that same thing in callback code to button.on_click() is > obviously less fun and feels inverted. I don't know how you would write it in callback fashion, but the above seems inside out to me. It feels like COMEFROM instead of GOTO, if you understand the analogy. Rather than execute code in a linear fashion until you reach a yield (GOTO), then transfer control *out* it's execute code in a linear fashion control transferred *in* from somewhere else (COMEFROM) This is how I would expect to write something like that in an event-driven way: def mouseUp(self): # assume self.counter is initially set to 0 self.settext("Click Me %d" % self.counter) self.counter += 1 if self.counter == 10: self.disable() messageBox("You have reached the end!") which feels natural to me. Processing of each click is independent of any other click, and there's no non-linear transfer of control. The user clicks, the mouseUp method is called, it runs, and it is done. Whatever persistent state there is is part of the button itself. I don't know whether you would call that a callback. I suppose it could be, in the sense that you might say: button.set_mouseup_function(mouseUp) but I'm used to thinking of it as a property of the button. -- Steve From rosuav at gmail.com Thu Aug 11 06:06:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Aug 2016 20:06:28 +1000 Subject: Asynchronous programming In-Reply-To: <57ac4947$0$11125$c3e8da3@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac4947$0$11125$c3e8da3@news.astraweb.com> Message-ID: On Thu, Aug 11, 2016 at 7:45 PM, Steven D'Aprano wrote: > I don't know whether you would call that a callback. I suppose it could be, in > the sense that you might say: > > button.set_mouseup_function(mouseUp) > > but I'm used to thinking of it as a property of the button. "Callback" simply means "function that someone else calls". In fact, most dunder methods could be described as callbacks. ChrisA From paul.nospam at rudin.co.uk Thu Aug 11 06:16:27 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Thu, 11 Aug 2016 11:16:27 +0100 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac4947$0$11125$c3e8da3@news.astraweb.com> Message-ID: <87d1lf391g.fsf@rudin.co.uk> Chris Angelico writes: > On Thu, Aug 11, 2016 at 7:45 PM, Steven D'Aprano > wrote: >> I don't know whether you would call that a callback. I suppose it could be, in >> the sense that you might say: >> >> button.set_mouseup_function(mouseUp) >> >> but I'm used to thinking of it as a property of the button. > > "Callback" simply means "function that someone else calls". In fact, > most dunder methods could be described as callbacks. > A callback is normally a callable you pass to some other function with the expectation that it will be invoked in certain circumstances. That's not really the same thing as something someone else calls: any function that's part of a published api is intended to be called by someone else; but that doesn't really make it a callback. From marko at pacujo.net Thu Aug 11 06:43:10 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 11 Aug 2016 13:43:10 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <87popfhh8i.fsf@elektro.pacujo.net> Message-ID: <87fuqbh9hd.fsf@elektro.pacujo.net> Chris Angelico : > On Thu, Aug 11, 2016 at 5:55 PM, Marko Rauhamaa wrote: >> My favorite asynchronous development model is the "callback hell," where >> each cell of the state/event matrix is represented by a method (or at >> least a switch case in C) and each state has an explicit name in the >> source code. The resulting code is still hard to read and hard to get >> right, but that complexity is unavoidable because reality just is that >> complex. > > Wow. Are you for real, or are you trolling us? You actually *enjoy* > callback hell? > > Give me yield-based asyncio any day. To me, the key is that the code should closely mirror the classic FSM formalisms: It *would* be possible to write asyncio code the same way where you would represent states as asyncs and express state transitions as: await self.ESTABLISHING() Alas, that would lead to unlimited async recursion, which, I'm pretty sure, would overflow Python's execution stack. Marko From steve+python at pearwood.info Thu Aug 11 07:50:45 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 21:50:45 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87k2fn3m3f.fsf@rudin.co.uk> Message-ID: <57ac6697$0$1602$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 03:34 pm, Paul Rudin wrote: > Steven D'Aprano writes: > >> >> Is there a good beginner's tutorial introducing the basics of >> asynchronous programming? Starting with, why and where would you use it? > > You could do worse than watch Dave Beazley's pycon talk: > https://www.youtube.com/watch?v=lYe8W04ERnY Instructions unclear, poked myself in the eye with a sharp stick. I will, thanks for the reference, but in general I find that videos are good for giving the illusion of understanding. It all seems so very clear at the time, and then twenty minutes later it's all "Now did he say to rotate the shifter or shift the rotator? How far into the talk was that bit again?" Between programmers taking screenshots of code instead of copying and pasting text, and the popularity of video over written examples, I'm feeling like Master Li at the end of "The Story Of The Stone". Bonus points to anyone who gets the reference without having to google it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Aug 11 07:53:58 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Thu, 11 Aug 2016 21:53:58 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <87popfhh8i.fsf@elektro.pacujo.net> Message-ID: <57ac6758$0$1602$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 07:33 pm, Chris Angelico wrote: > Yes, > threading bugs can be harder to debug; but only when you've violated > the other principles. Especially the principle "Avoid threaded programming". Some people, when faced with a problem, Now they think "I know, I'll have use two threading." problems. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From Cecil at decebal.nl Thu Aug 11 09:23:40 2016 From: Cecil at decebal.nl (Cecil Westerhof) Date: Thu, 11 Aug 2016 15:23:40 +0200 Subject: Is it =?utf-8?B?4oCYYWxsb3dlZOKAmQ==?= to get parameters like this Message-ID: <87popfmobn.fsf@Equus.decebal.nl> It has been a while since I worked with Python. I wanted to get some stats about the idle time of my computer, so that was a good moment to pick up Python again. ;-) As I understood it getopt is the way to get the parameters for your script. But at the moment it looks a bit like overkill, my script does not have much parameters as parameter_error shows: def parameter_error(): print('Error: {0} [ --5 | --10 | --20 | --25 ] --all | --this-month | --this-year | --today' .format(basename(sys.argv[0]))) sys.exit(1) So at the moment this is my init function: def init(): global conn global cursor global vmstat_params arg_length = len(sys.argv) if (arg_length == 1): period = '--today' slice_length = 20 elif (arg_length == 2): period = sys.argv[1] slice_length = 20 elif (arg_length == 3): period = sys.argv[2] if (sys.argv[1] in ['--5', '--10', '--20', '--25']): slice_length = int(sys.argv[1][2:]) else: parameter_error() else: parameter_error() conn = connect(expanduser('~/Databases/general.sqlite')) cursor = conn.cursor() all_data = '%' today = cursor.execute('SELECT CURRENT_DATE').fetchone()[0] this_month = today[0:8] + '%' this_year = today[0:5] + '%' if (period == '--today'): vmstat_params = [today, slice_length] elif (period == '--this-month'): vmstat_params = [this_month, slice_length] elif (period == '--this-year'): vmstat_params = [this_year, slice_length] elif (period == '--all'): vmstat_params = [all_data, slice_length] else: parameter_error() Is this acceptable, or is it a big no-no? By the way: the reason I fetch today from the database is that I work with UTC dates. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From steve+python at pearwood.info Thu Aug 11 10:14:40 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 00:14:40 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> Message-ID: <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 03:06 pm, Paul Rubin wrote: > The basic characteristic of asynchronous programming is that it involves > changing all your usual blocking i/o calls to non-blocking ones, so your > program can keep running as soon as your request is started. That's the bit that confuses me. I understand about parallel processing, but that's not what this is about. Say I want to download data from a network, and it will take a long time. If I can do the read in parallel to something else, that makes sense: begin downloading in another thread/process make a coffee process download But what's the point in doing it asynchronously if I have to just wait for it to complete? begin downloading in an async thread twiddle thumbs, doing nothing process download Take Chris' "yield from" example, rewritten for await: def asynchronous(id): trn = await conn.begin_transaction() await trn.execute("select name from people where id=%d", (id,)) name, = await trn.fetchone() await trn.execute("update people set last_seen=now() where id=%d",(id,)) await trn.commit() return name Since I'm spending all my time waiting for the other things to return control, why don't I just write it synchronously? def synchronous(id): trn = conn.begin_transaction() trn.execute("select name from people where id=%d", (id,)) name, = trn.fetchone() trn.execute("update people set last_seen=now() where id=%d", (id,)) trn.commit() return name -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From marko at pacujo.net Thu Aug 11 10:22:09 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 11 Aug 2016 17:22:09 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87k2fn3m3f.fsf@rudin.co.uk> <57ac6697$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87ziojfkry.fsf@elektro.pacujo.net> Steven D'Aprano : > Instructions unclear, poked myself in the eye with a sharp stick. I have updated my Dining Philosophers example for Python 3.5: It demonstrates how to get an event loop and start a number of asyncs concurrently. Marko From tjreedy at udel.edu Thu Aug 11 10:24:36 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 11 Aug 2016 10:24:36 -0400 Subject: Asynchronous programming In-Reply-To: References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/11/2016 2:34 AM, Christian Gollwitzer wrote: > Am 11.08.16 um 06:38 schrieb Terry Reedy: >> You might be able to glean something from the succession of files I >> uploaded to >> https://bugs.python.org/issue27546 >> Integrate tkinter and asyncio (and async) >> >> I started with just mixing tk and asyncio callbacks. After some >> struggle with similar question as you ask above, I ended up with more or >> less callback free code using async def and async for. Once I got over >> the hump, I rather like it. My next experiment, not uploaded, is adapting tk mainloop so it can be used instead of asyncio.loop.runforever as the driver for async def and async for and await. There is either a simple bug I overlooked or I am missing some detail of how the future and task interact. > Interesting discussion! Tcl itself acquired coroutines some years ago > (in 8.6) and people in this community are very excited with async based > programming. When I read something like "Python finally acquired an event loop in 3.4" I wonder where people have been. The tk event loop has been in Python perhaps for 2 decades, and one actually could call select every xx milleseconds. The main problem is that tcl/tk does not do file events on Windows. The asyncio people did tackle that, but ended up with two loops, Selector and Proactor, neither of which are as complete as Selector on *nix. Tcl/tk could try to borrow than work. > I'm convinced that it is possible to integrate Tcl's event loop with > asyncio's loop without regular update polling. This might require a > patch to Tkinter at the C level. For example, an easy way is to put > Tcl/Tk in it's own thread. The Tcl event loop can be woken up from a > different thread by the TclThreadQueueEvent() and TclTheradAlert() API > calls. Basically, you would spawn a new thread (from the C level), when > loading Tkinter, which loads Tk and sits in a Tcl event loop ("vwait > forever"). tk.eval() and tk.call() would be redefined to post events > using the aforementioned functions. When a Tcl callback arrives, it > would send an event to asyncio in the main thread - no idea how this > works, though. In the end, it would also allow to use Tkinter from > different Python threads, because all calls would be rooted to the one > instance only. -- Terry Jan Reedy From __peter__ at web.de Thu Aug 11 10:29:54 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 11 Aug 2016 16:29:54 +0200 Subject: Is it =?UTF-8?B?4oCYYWxsb3dlZOKAmQ==?= to get parameters like this References: <87popfmobn.fsf@Equus.decebal.nl> Message-ID: Cecil Westerhof wrote: > It has been a while since I worked with Python. I wanted to get some > stats about the idle time of my computer, so that was a good moment to > pick up Python again. ;-) > > > As I understood it getopt is the way to get the parameters for your > script. But at the moment it looks a bit like overkill, my script does > not have much parameters as parameter_error shows: > def parameter_error(): > print('Error: {0} [ --5 | --10 | --20 | --25 ] --all | --this-month | > --this-year | --today' > .format(basename(sys.argv[0]))) > sys.exit(1) > > So at the moment this is my init function: > def init(): > global conn > global cursor > global vmstat_params > > arg_length = len(sys.argv) > if (arg_length == 1): > period = '--today' > slice_length = 20 > elif (arg_length == 2): > period = sys.argv[1] > slice_length = 20 > elif (arg_length == 3): > period = sys.argv[2] > if (sys.argv[1] in ['--5', '--10', '--20', '--25']): > slice_length = int(sys.argv[1][2:]) > else: > parameter_error() > else: > parameter_error() Contrast that with the following cli based on argparse: $ cat demo.py #!/usr/bin/env python3 def init(): import argparse parser = argparse.ArgumentParser() parser.add_argument( "period", nargs="?", choices="today this-month this-year all".split(), default="today", help="Period covered") parser.add_argument( "slice_length", type=int, nargs="?", choices=[5, 10, 20, 25], default=20, help="Whatever...") print(parser.parse_args()) init() $ ./demo.py -h usage: demo.py [-h] [{today,this-month,this-year,all}] [{5,10,20,25}] positional arguments: {today,this-month,this-year,all} Period covered {5,10,20,25} Whatever... optional arguments: -h, --help show this help message and exit $ ./demo.py all Namespace(period='all', slice_length=20) $ ./demo.py none usage: demo.py [-h] [{today,this-month,this-year,all}] [{5,10,20,25}] demo.py: error: argument period: invalid choice: 'none' (choose from 'today', 'this-month', 'this-year', 'all') $ ./demo.py all 5 Namespace(period='all', slice_length=5) $ ./demo.py all 7 usage: demo.py [-h] [{today,this-month,this-year,all}] [{5,10,20,25}] demo.py: error: argument slice_length: invalid choice: 7 (choose from 5, 10, 20, 25) The interface is not exactly the same, but the code that defines it is declarative rather than trying to cover all possible combinations of input and should be easier to maintain in the long run. Thus the answer to your question is no, the Python Police will have to arrest you if you stick with your original approach ;) From marko at pacujo.net Thu Aug 11 10:35:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 11 Aug 2016 17:35:11 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87vaz7fk68.fsf@elektro.pacujo.net> Steven D'Aprano : > Say I want to download data from a network, and it will take a long > time. If I can do the read in parallel to something else, that makes > sense: > > begin downloading in another thread/process > make a coffee > process download > > But what's the point in doing it asynchronously if I have to just wait for > it to complete? You can do many things concurrently, as the philosophers in my previous example. You will need two asyncs, one for downloading, another one for making coffee, just like you would with threads. Then, run both asyncs concurrently with asyncio.wait(). Marko From steve+python at pearwood.info Thu Aug 11 10:55:15 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 00:55:15 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 02:41 pm, Chris Angelico wrote: > Consider these three ways of doing a database transaction: > > def synchronous(id): > trn = conn.begin_transaction() > trn.execute("select name from people where id=%d", (id,)) > name, = trn.fetchone() > trn.execute("update people set last_seen=now() where id=%d", (id,)) > trn.commit() > return name That makes perfect sense. Good old fashioned synchronous programming. > def callbacks_1(cb, id): > conn.begin_transaction(callbacks_2, cb, id) > def callbacks_2(trn, cb, id): > trn.execute("select name from people where id=%d", (id,), > callbacks_3, cb, id) > def callbacks_3(trn, cb, id): > trn.fetchone(callbacks_4, cb, id) > def callbacks_4(trn, data, cb, id): > name, = data > trn.execute("update people set last_seen=now() where id=%d", > (id,), callbacks_5, cb, name) > def callbacks_5(trn, cb, name): > trn.commit(callbacks_6, cb, name) > def callbacks_6(trn, cb, name): > cb(name) Now you're surely pulling my leg. Your conn.begin_transaction has a completely different signature! (No arguments in the first case, three in this case.) > def asynchronous(id): > trn = yield from conn.begin_transaction() > yield from trn.execute("select name from people where id=%d", (id,)) > name, = yield from trn.fetchone() > yield from trn.execute("update people set last_seen=now() where > id=%d", (id,)) > yield from trn.commit() > return name That ... looks wrong. You're taking something which looks like a procedure in the first case (trn.execute), so it probably returns None, and yielding over it. Even it that's not wrong, and it actually returned something which you ignored in the first case, it looks like you're mixing two distinct ways of using generators: - Generator as iterator ("yield x" or "yield from subiterator"); something which *sends* values out for the purpose of iteration. - Generator as coroutine ("y = yield x"); something which *receives* values from the called using the send() method. Another way of putting it is, "a bogus generator that generates and receives values (not a recommended coding style)". (David Beazly, per url below.) When I say coroutine here, I mean in the *old* sense, as described by David Beasley's A Curious Course on Coroutines and Concurrency: http://www.dabeaz.com/coroutines/ So there's something deeply suspicious to me about the coding style. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From python at bdurham.com Thu Aug 11 10:56:13 2016 From: python at bdurham.com (Malcolm Greene) Date: Thu, 11 Aug 2016 10:56:13 -0400 Subject: Advice on optimizing a Python data driven rules engine Message-ID: <1470927373.3071120.692534025.5B40675B@webmail.messagingengine.com> Background: I'm building a rules engine for transforming rows of data being returned by csv DictReader, eg. each row of data is a dict of column name to value mappings. My rules are a list of rule objects whose attributes get referenced by rule specific methods. Each rule has an associated method which gets called based on rule name. Looking for some advice on how to optimize the BOILERPLATE portions of the following type of code. There's an awful lot of dot dereferencing going on. One thought was to pass in the values being dereferenced as parameters and return value. But this would just move the dereferencing to another point in my program and would add the overhead of parameter passage. Is there a technique where I could store a reference to these values that would make their access more efficient? def action_strip(self): # BOILERPLATE: lookup value being transformed (possible to get a # direct 'pointer' to this dic entry???) value = self.data[self.rule.target_column] # BOILERPLATE: get rule's hard coded parameter match = self.rule.value.strip() # perform a generic transformation action while value.startswith(match): value = value.replace(match, '', 1).strip() while value.endswith(match): value = value[0:len(value) - len(match)].strip() # BOILERPLATE: update the value we just transformed self.data[self.rule.target_column] = value Thank you, Malcolm From Cecil at decebal.nl Thu Aug 11 11:15:59 2016 From: Cecil at decebal.nl (Cecil Westerhof) Date: Thu, 11 Aug 2016 17:15:59 +0200 Subject: Is it =?utf-8?B?4oCYYWxsb3dlZOKAmQ==?= to get parameters like this References: <87popfmobn.fsf@Equus.decebal.nl> Message-ID: <87lh03mj4g.fsf@Equus.decebal.nl> On Thursday 11 Aug 2016 16:29 CEST, Peter Otten wrote: > Cecil Westerhof wrote: > >> It has been a while since I worked with Python. I wanted to get >> some stats about the idle time of my computer, so that was a good >> moment to pick up Python again. ;-) >> >> >> As I understood it getopt is the way to get the parameters for your >> script. But at the moment it looks a bit like overkill, my script >> does not have much parameters as parameter_error shows: def >> parameter_error(): print('Error: {0} [ --5 | --10 | --20 | --25 ] >> --all | --this-month | --this-year | --today' >> .format(basename(sys.argv[0]))) sys.exit(1) >> >> So at the moment this is my init function: >> def init(): >> global conn >> global cursor >> global vmstat_params >> >> arg_length = len(sys.argv) >> if (arg_length == 1): >> period = '--today' >> slice_length = 20 >> elif (arg_length == 2): >> period = sys.argv[1] >> slice_length = 20 >> elif (arg_length == 3): >> period = sys.argv[2] >> if (sys.argv[1] in ['--5', '--10', '--20', '--25']): >> slice_length = int(sys.argv[1][2:]) >> else: >> parameter_error() >> else: >> parameter_error() > > > > Contrast that with the following cli based on argparse: > > $ cat demo.py > #!/usr/bin/env python3 > > def init(): > import argparse > > parser = argparse.ArgumentParser() > parser.add_argument( > "period", nargs="?", > choices="today this-month this-year all".split(), > default="today", > help="Period covered") > parser.add_argument( > "slice_length", type=int, nargs="?", > choices=[5, 10, 20, 25], > default=20, > help="Whatever...") > > print(parser.parse_args()) > > init() $ ./demo.py -h usage: demo.py [-h] > [{today,this-month,this-year,all}] [{5,10,20,25}] > > positional arguments: > {today,this-month,this-year,all} > Period covered > {5,10,20,25} Whatever... > > optional arguments: -h, --help show this help message and exit $ > ./demo.py all Namespace(period='all', slice_length=20) $ ./demo.py > none usage: demo.py [-h] [{today,this-month,this-year,all}] > [{5,10,20,25}] demo.py: error: argument period: invalid choice: > 'none' (choose from 'today', 'this-month', 'this-year', 'all') $ > ./demo.py all 5 Namespace(period='all', slice_length=5) $ ./demo.py > all 7 usage: demo.py [-h] [{today,this-month,this-year,all}] > [{5,10,20,25}] demo.py: error: argument slice_length: invalid > choice: 7 (choose from 5, 10, 20, 25) > > The interface is not exactly the same, but the code that defines it > is declarative rather than trying to cover all possible combinations > of input and should be easier to maintain in the long run. That is very neat indeed. I am used of options starting with --, but I am not married to that. I am going to look into this. > Thus the answer to your question is no, the Python Police will have > to arrest you if you stick with your original approach ;) Luckily I am going to change the code. :-D -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From steve+python at pearwood.info Thu Aug 11 11:26:16 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 01:26:16 +1000 Subject: Is it =?UTF-8?B?4oCYYWxsb3dlZOKAmQ==?= to get parameters like this References: <87popfmobn.fsf@Equus.decebal.nl> Message-ID: <57ac991a$0$1620$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 11:23 pm, Cecil Westerhof wrote: > It has been a while since I worked with Python. I wanted to get some > stats about the idle time of my computer, so that was a good moment to > pick up Python again. ;-) > > > As I understood it getopt is the way to get the parameters for your > script. getopt is mainly for C programmers from Unix or Linux systems who are used to using the Unix getopt function for handling parameters. There are two alternatives in the standard library: optparse (depreciated, for legacy use only) argparse https://docs.python.org/3/library/getopt.html https://docs.python.org/3/library/optparse.html https://docs.python.org/3/library/argparse.html and many third-party options, including DocOpt: https://pypi.python.org/pypi/docopt [...] > Is this acceptable, or is it a big no-no? Despite what Peter says, the Python police won't come after you. Your punishment for writing hard-to-maintain code is that you have to maintain your own hard-to-maintain code. If you have only a few command line arguments, and they don't change often, the effort required to learn the "right" way to process them may be significantly harder than the effort required to do it the "wrong" way. That's called "technical debt", and in an on-going software project it can hurt you very, very much. Technical debt can cause software projects to fail. But for personal use, where your software tends to be small and simple, don't feel guilty about it. You'll know when it's time to learn one of the other argument parsers: when you find yourself cursing the fact that you have to edit your own argument-parsing code, that's the time to bite the bullet and learn them. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From michael.selik at gmail.com Thu Aug 11 11:38:30 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 11 Aug 2016 15:38:30 +0000 Subject: Advice on optimizing a Python data driven rules engine In-Reply-To: <1470927373.3071120.692534025.5B40675B@webmail.messagingengine.com> References: <1470927373.3071120.692534025.5B40675B@webmail.messagingengine.com> Message-ID: On Thu, Aug 11, 2016 at 10:57 AM Malcolm Greene wrote: > Background: I'm building a rules engine for transforming rows of data > being returned by csv DictReader, eg. each row of data is a dict of column > name to value mappings. My rules are a list of rule objects whose > attributes get referenced by rule specific methods. Each rule has an > associated method which gets called based on rule name. > Could you share the API you've designed for this tool? An example of usage will clarify what you intend. One technique for applying a transformation function to each record returned by a csv DictReader: import csv import math from io import StringIO f = StringIO('''\ a,b,c 1,2,3 4,5,6 ''') def pass_(value): return value transforms = { 'a': lambda v: math.sin(int(v)), 'b': lambda v: math.cos(int(v)), } for row in csv.DictReader(f): print({k: transforms.get(k, pass_)(v) for k, v in row.items()}) Using __missing__ for your transform dict might be more elegant than .get with a pass_ as default. From michael.selik at gmail.com Thu Aug 11 11:46:50 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 11 Aug 2016 15:46:50 +0000 Subject: Asynchronous programming In-Reply-To: <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Aug 11, 2016 at 11:01 AM Steven D'Aprano wrote: > That ... looks wrong. You're taking something which looks like a procedure > in the first case (trn.execute), so it probably returns None, and yielding > over it. Even it that's not wrong, and it actually returned something which > you ignored in the first case > It's a standard, perhaps a mistaken standard, but nonetheless database cursors tend to have that feature: execute returns the mutated self. I agree that execute looks like it should return None instead. The return self pattern feels Rubyish to me (or Rubic?). From michael.selik at gmail.com Thu Aug 11 11:51:13 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 11 Aug 2016 15:51:13 +0000 Subject: Asynchronous programming In-Reply-To: References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Aug 11, 2016 at 11:46 AM Michael Selik wrote: > On Thu, Aug 11, 2016 at 11:01 AM Steven D'Aprano < > steve+python at pearwood.info> wrote: > >> That ... looks wrong. You're taking something which looks like a procedure >> in the first case (trn.execute), so it probably returns None, and yielding >> over it. Even it that's not wrong, and it actually returned something >> which >> you ignored in the first case >> > > It's a standard, perhaps a mistaken standard, but nonetheless database > cursors tend to have that feature: execute returns the mutated self. I > agree that execute looks like it should return None instead. The return > self pattern feels Rubyish to me (or Rubic?). > Contradicting myself: yield from c.execute(query).fetchall() # looks good yield from c.execute(query) # looks bad From jussi.piitulainen at helsinki.fi Thu Aug 11 12:03:05 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 11 Aug 2016 19:03:05 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: Michael Selik writes: > On Thu, Aug 11, 2016 at 11:46 AM Michael Selik > wrote: > >> On Thu, Aug 11, 2016 at 11:01 AM Steven D'Aprano < >> steve+python at pearwood.info> wrote: >> >>> That ... looks wrong. You're taking something which looks like a procedure >>> in the first case (trn.execute), so it probably returns None, and yielding >>> over it. Even it that's not wrong, and it actually returned something >>> which >>> you ignored in the first case >>> >> >> It's a standard, perhaps a mistaken standard, but nonetheless database >> cursors tend to have that feature: execute returns the mutated self. I >> agree that execute looks like it should return None instead. The return >> self pattern feels Rubyish to me (or Rubic?). >> > > Contradicting myself: > yield from c.execute(query).fetchall() # looks good > yield from c.execute(query) # looks bad The first looks like it allocates a list to hold all the results at once only to yield each result one by one. How about: c.execute(query) yield from c From Cecil at decebal.nl Thu Aug 11 12:15:38 2016 From: Cecil at decebal.nl (Cecil Westerhof) Date: Thu, 11 Aug 2016 18:15:38 +0200 Subject: Is it =?utf-8?B?4oCYYWxsb3dlZOKAmQ==?= to get parameters like this References: <87popfmobn.fsf@Equus.decebal.nl> <87lh03mj4g.fsf@Equus.decebal.nl> Message-ID: <87h9armgd1.fsf@Equus.decebal.nl> On Thursday 11 Aug 2016 17:15 CEST, Cecil Westerhof wrote: > On Thursday 11 Aug 2016 16:29 CEST, Peter Otten wrote: > >> Cecil Westerhof wrote: >> >>> It has been a while since I worked with Python. I wanted to get >>> some stats about the idle time of my computer, so that was a good >>> moment to pick up Python again. ;-) >>> >>> >>> As I understood it getopt is the way to get the parameters for >>> your script. But at the moment it looks a bit like overkill, my >>> script does not have much parameters as parameter_error shows: def >>> parameter_error(): print('Error: {0} [ --5 | --10 | --20 | --25 ] >>> --all | --this-month | --this-year | --today' >>> .format(basename(sys.argv[0]))) sys.exit(1) >>> >>> So at the moment this is my init function: >>> def init(): >>> global conn >>> global cursor >>> global vmstat_params >>> >>> arg_length = len(sys.argv) >>> if (arg_length == 1): >>> period = '--today' >>> slice_length = 20 >>> elif (arg_length == 2): >>> period = sys.argv[1] >>> slice_length = 20 >>> elif (arg_length == 3): >>> period = sys.argv[2] >>> if (sys.argv[1] in ['--5', '--10', '--20', '--25']): >>> slice_length = int(sys.argv[1][2:]) >>> else: >>> parameter_error() >>> else: >>> parameter_error() >> >> >> >> Contrast that with the following cli based on argparse: >> >> $ cat demo.py >> #!/usr/bin/env python3 >> >> def init(): >> import argparse >> >> parser = argparse.ArgumentParser() >> parser.add_argument( >> "period", nargs="?", >> choices="today this-month this-year all".split(), >> default="today", >> help="Period covered") >> parser.add_argument( >> "slice_length", type=int, nargs="?", >> choices=[5, 10, 20, 25], >> default=20, >> help="Whatever...") >> >> print(parser.parse_args()) >> >> init() $ ./demo.py -h usage: demo.py [-h] >> [{today,this-month,this-year,all}] [{5,10,20,25}] >> >> positional arguments: >> {today,this-month,this-year,all} >> Period covered >> {5,10,20,25} Whatever... >> >> optional arguments: -h, --help show this help message and exit $ >> ./demo.py all Namespace(period='all', slice_length=20) $ ./demo.py >> none usage: demo.py [-h] [{today,this-month,this-year,all}] >> [{5,10,20,25}] demo.py: error: argument period: invalid choice: >> 'none' (choose from 'today', 'this-month', 'this-year', 'all') $ >> ./demo.py all 5 Namespace(period='all', slice_length=5) $ ./demo.py >> all 7 usage: demo.py [-h] [{today,this-month,this-year,all}] >> [{5,10,20,25}] demo.py: error: argument slice_length: invalid >> choice: 7 (choose from 5, 10, 20, 25) >> >> The interface is not exactly the same, but the code that defines it >> is declarative rather than trying to cover all possible >> combinations of input and should be easier to maintain in the long >> run. > > That is very neat indeed. I am used of options starting with --, but > I am not married to that. I am going to look into this. I rewrote it to: def init(): global conn global cursor global vmstat_params parser = ArgumentParser() parser.add_argument( 'period', nargs = '?', choices = 'all this-month this-year today'.split(), default = 'today', help='Period covered') parser.add_argument( 'slice_length', type = int, nargs ='?', choices = [5, 10, 20, 25], default = 20, help = 'Length of a percentage slice') args = parser.parse_args() period = args.period slice_length = args.slice_length conn = connect(expanduser('~/Databases/general.sqlite')) cursor = conn.cursor() all_data = '%' today = cursor.execute('SELECT CURRENT_DATE').fetchone()[0] this_month = today[0:8] + '%' this_year = today[0:5] + '%' if (period == 'today'): vmstat_params = [today, slice_length] elif (period == 'this-month'): vmstat_params = [this_month, slice_length] elif (period == 'this-year'): vmstat_params = [this_year, slice_length] elif (period == 'all'): vmstat_params = [all_data, slice_length] else: print('Unknown period: {0}'.format(period)) sys.exit(1) I kept the else part for the case that an argument is added that is not put into the if/elif construction. -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof From rosuav at gmail.com Thu Aug 11 12:27:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Aug 2016 02:27:45 +1000 Subject: Advice on optimizing a Python data driven rules engine In-Reply-To: <1470927373.3071120.692534025.5B40675B@webmail.messagingengine.com> References: <1470927373.3071120.692534025.5B40675B@webmail.messagingengine.com> Message-ID: On Fri, Aug 12, 2016 at 12:56 AM, Malcolm Greene wrote: > Looking for some advice on how to optimize the BOILERPLATE portions of > the following type of code. There's an awful lot of dot dereferencing > going on. One thought was to pass in the values being dereferenced as > parameters and return value. But this would just move the dereferencing > to another point in my program and would add the overhead of parameter > passage. Is there a technique where I could store a reference to these > values that would make their access more efficient? Until you find that the dereferencing of dots is actually hurting you, don't change anything. Most likely, it won't be a performance problem, so the only question is: Does this hurt code readability? And optimizing for code readability by adding levels of indirection is often a bad idea. What you MAY be able to do, though, if (as I'm guessing) the BOILERPLATE sections are repeated across lots of functions, is to refactor the functions to put those bits somewhere else. If nothing else, you could make a function decorator that surrounds the code with the appropriate boilerplate. But that's still indirection, and only worthwhile if there's a lot of duplication. Remember, every time you read this code, you'll need to go look elsewhere for the other half of the guts. ChrisA From paul.nospam at rudin.co.uk Thu Aug 11 12:47:43 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Thu, 11 Aug 2016 17:47:43 +0100 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <878tw32qxc.fsf@rudin.co.uk> Steven D'Aprano writes: > > But what's the point in doing it asynchronously if I have to just wait for > it to complete? > > begin downloading in an async thread > twiddle thumbs, doing nothing > process download If you have nothing else to do, then there's no point. But suppose you're implementing a web server. A request comes in - in order to respond you need some data from a database. So you do the database bit async. Whilst you're waiting for that you can get on with processing other incoming requests. From steve+python at pearwood.info Thu Aug 11 13:02:09 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 03:02:09 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87k2fn3m3f.fsf@rudin.co.uk> Message-ID: <57acaf93$0$1601$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 03:34 pm, Paul Rudin wrote: > Steven D'Aprano writes: > >> >> Is there a good beginner's tutorial introducing the basics of >> asynchronous programming? Starting with, why and where would you use it? > > You could do worse than watch Dave Beazley's pycon talk: > https://www.youtube.com/watch?v=lYe8W04ERnY Thanks. I'm not sure that it helped. I kind of gibbered a bit when Dave said that threaded code was good because you can easily reason about it. Most of his talk is about how asyncio shouldn't be in the standard library, and the rest didn't really answer my questions. What I'm taking from this is that asynchronous programming is mindboggling difficult to reason about or understand, and its the best thing ever. *half-a-wink* Having-flashbacks-to-when-I-was-first-introduced-to-OOP-ly y'rs, -- Steve If anyone mentions monads, I'm out of here. From atri.mahapatra at gmail.com Thu Aug 11 13:03:25 2016 From: atri.mahapatra at gmail.com (Atri Mahapatra) Date: Thu, 11 Aug 2016 10:03:25 -0700 (PDT) Subject: Print function not working Message-ID: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> I have installed IDLE 3.5.1 and wrote the following to check if print is working. When it runs, I do not see anything is printed: class Base: #{ def __init__( self ): #{ print("Hello, world: \n\n"); #} #} if ( __name__ == " __main__"): #{ root = Base(); #} Can anyone please point out the reason? Thanks, Atri From atri.mahapatra at gmail.com Thu Aug 11 13:08:18 2016 From: atri.mahapatra at gmail.com (Atri Mahapatra) Date: Thu, 11 Aug 2016 10:08:18 -0700 (PDT) Subject: Print function not working Message-ID: I have installed IDLE 3.5.1 and wrote the following print command to see if it is working. When I ran the code nothing is printed. Can you please point the reason- anything wrong with the code? class Base: #{ def __init__( self ): #{ print("Hello, world: \n\n"); #} #} if ( __name__ == " __main__"): #{ root = Base(); #} Thanks, Atri From rosuav at gmail.com Thu Aug 11 13:09:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Aug 2016 03:09:15 +1000 Subject: Asynchronous programming In-Reply-To: <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ac91d5$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 12, 2016 at 12:55 AM, Steven D'Aprano wrote: > On Thu, 11 Aug 2016 02:41 pm, Chris Angelico wrote: > >> Consider these three ways of doing a database transaction: >> >> def synchronous(id): >> trn = conn.begin_transaction() >> trn.execute("select name from people where id=%d", (id,)) >> name, = trn.fetchone() >> trn.execute("update people set last_seen=now() where id=%d", (id,)) >> trn.commit() >> return name > > That makes perfect sense. Good old fashioned synchronous programming. Let's assume in this case that we started with: conn = synchronous_database_connection() >> def callbacks_1(cb, id): >> conn.begin_transaction(callbacks_2, cb, id) >> def callbacks_2(trn, cb, id): >> trn.execute("select name from people where id=%d", (id,), >> callbacks_3, cb, id) >> def callbacks_3(trn, cb, id): >> trn.fetchone(callbacks_4, cb, id) >> def callbacks_4(trn, data, cb, id): >> name, = data >> trn.execute("update people set last_seen=now() where id=%d", >> (id,), callbacks_5, cb, name) >> def callbacks_5(trn, cb, name): >> trn.commit(callbacks_6, cb, name) >> def callbacks_6(trn, cb, name): >> cb(name) > > Now you're surely pulling my leg. Your conn.begin_transaction has a > completely different signature! (No arguments in the first case, three in > this case.) Let's assume that this one started with: conn = callback_database_connection() It's doing the same job as the 'conn' in the first example, but it's a completely different API to cater to the fact that it has to handle callbacks. You could use this API for synchronous calls by doing something like this: def begin_transaction(callback, *args): real_conn.begin_transaction() callback(*args) The asynchronous version would end up saving callback and args somewhere, triggering the operation, and having code somewhere that processes the response. Supposing we're talking to PostgreSQL over a socket (TCP or Unix domain), the response handler would be triggered any time that socket becomes readable (ie via select() on the socket), and it would decode the response, figure out which transaction is being responded to (if there are multiple in flight), and send the response on its way. Most likely the transaction would have some kind of "current in-flight query" attribute (and would reject reentrant calls - see, any form of async programming has to cope with reentrancy), so that's where the callback would be stored. >> def asynchronous(id): >> trn = yield from conn.begin_transaction() >> yield from trn.execute("select name from people where id=%d", (id,)) >> name, = yield from trn.fetchone() >> yield from trn.execute("update people set last_seen=now() where >> id=%d", (id,)) >> yield from trn.commit() >> return name > > That ... looks wrong. You're taking something which looks like a procedure > in the first case (trn.execute), so it probably returns None, and yielding > over it. Even it that's not wrong, and it actually returned something which > you ignored in the first case, it looks like you're mixing two distinct > ways of using generators: > > - Generator as iterator ("yield x" or "yield from subiterator"); > something which *sends* values out for the purpose of iteration. > > - Generator as coroutine ("y = yield x"); something which *receives* > values from the called using the send() method. Yeah, generators as coroutines are a bit weird. That's another good reason for using the new async and await "keywords" (not technically keywords yet), as it doesn't look as weird. But ultimately, it's doing the same thing - the methods would look something like this: def begin_transaction(): # trigger the "begin transaction" query yield Awaitable("waiting for transaction...") # verify that the query was successful The event loop attempts to step the "asynchronous" generator. It yields from begin_transaction, which yields an Awaitable. The event loop thus receives, from the generator, an object to be placed on the queue. It's that simple. Here's a very VERY simple, but complete, example of yield-based coroutines. # Partially borrowed from example in Python docs: # https://docs.python.org/3/library/selectors.html#examples import selectors import socket import time sel = selectors.DefaultSelector() def eventloop(): while "loop forever": for key, mask in sel.select(): sel.unregister(key.fileobj) run_task(key.data) def run_task(gen): try: waitfor = next(gen) sel.register(waitfor, selectors.EVENT_READ, gen) except StopIteration: pass def mainsock(): sock = socket.socket() sock.bind(('localhost', 1234)) sock.listen(100) sock.setblocking(False) print("Listening on port 1234.") while "moar sockets": yield sock conn, addr = sock.accept() # Should be ready print('accepted', conn, 'from', addr) conn.setblocking(False) run_task(client(conn)) def client(conn): while "moar data": yield conn data = conn.recv(1000) # Should be ready if not data: break print("Got data") # At this point, you'd do something smart with the data. # But we don't. We just echo back. conn.send(data) # Hope it won't block if b"quit" in data: break print('closing', conn) conn.close() run_task(mainsock()) eventloop() Aside from slapping a "yield sock" before accepting or reading from a socket, it's exactly like synchronous code. Obviously a real example would be able to yield other types of events too (most common would be the clock, to handle an asynchronous time.sleep() equivalent), but this is fully functional and potentially even useful. ChrisA From rosuav at gmail.com Thu Aug 11 13:18:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Aug 2016 03:18:01 +1000 Subject: Print function not working In-Reply-To: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> References: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> Message-ID: On Fri, Aug 12, 2016 at 3:03 AM, Atri Mahapatra wrote: > I have installed IDLE 3.5.1 and wrote the following to check if print is working. When it runs, I do not see anything is printed: > > class Base: #{ > def __init__( self ): #{ > print("Hello, world: \n\n"); > > #} > > #} > > > if ( __name__ == " __main__"): #{ > root = Base(); > #} > > Can anyone please point out the reason? > > Thanks, > Atri Is name equal to main? That is, are you running this code as a top-level script? Also: You do not need to write Python code as if it were C or Java. All that extra punctuation is just putting unnecessary stress on the world's punctuation mines, which are already overworked. :) ChrisA From __peter__ at web.de Thu Aug 11 13:35:47 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 11 Aug 2016 19:35:47 +0200 Subject: Print function not working References: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> Message-ID: Atri Mahapatra wrote: > I have installed IDLE 3.5.1 and wrote the following to check if print is > working. When it runs, I do not see anything is printed: > > class Base: #{ > def __init__( self ): #{ > print("Hello, world: \n\n"); > > #} > > #} > > > if ( __name__ == " __main__"): #{ > root = Base(); > #} > > Can anyone please point out the reason? Hard to tell. Either there slipped an extra space into the string literal in > if ( __name__ == " __main__"): #{ or the interpreter got really annoyed about you trying to write C/Java/PHP rather than proper Python... From python at mrabarnett.plus.com Thu Aug 11 13:36:43 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Aug 2016 18:36:43 +0100 Subject: Print function not working In-Reply-To: References: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> Message-ID: <28a42a5a-fe70-6d86-cca6-5398501d80f1@mrabarnett.plus.com> On 2016-08-11 18:18, Chris Angelico wrote: > On Fri, Aug 12, 2016 at 3:03 AM, Atri Mahapatra > wrote: >> I have installed IDLE 3.5.1 and wrote the following to check if print is working. When it runs, I do not see anything is printed: >> >> class Base: #{ >> def __init__( self ): #{ >> print("Hello, world: \n\n"); >> >> #} >> >> #} >> >> >> if ( __name__ == " __main__"): #{ >> root = Base(); >> #} >> >> Can anyone please point out the reason? >> >> Thanks, >> Atri > > Is name equal to main? That is, are you running this code as a top-level script? > It's not checking whether the name is "__main__", but whether it's " __main__" (note the space after the first quote). > Also: You do not need to write Python code as if it were C or Java. > All that extra punctuation is just putting unnecessary stress on the > world's punctuation mines, which are already overworked. :) > +1 From no.email at nospam.invalid Thu Aug 11 13:50:10 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 11 Aug 2016 10:50:10 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87a8gjyz3h.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > But what's the point in doing it asynchronously if I have to just wait for > it to complete? > begin downloading in an async thread > twiddle thumbs, doing nothing > process download Suppose the remote server is overloaded so it sends files much slower than your internet connection can receive them. And you want to download 10 different files from 10 different such servers. If you do them synchronously (wait for each download to finish before starting the next) it takes much longer than necessary. What you want is to download the 10 files simultaneously, using 10 different network connections. The download procedure has to speak some network protocol over a socket for each file. How do you deal with the concurrency? There are many possibilities: - use 10 different computers - open 10 windows on 1 computer, and start a download in each window (i.e. this means use multiple processes). - single, multi-threaded download client (oh nooo! the thread monsters will eat you if you try to do that!!!!) - single threaded client with asynchronous i/o, so it can have 10 network requests "in the air" simultaneously, using select() or epoll() to handle each piece of incoming data as soon as it arrives. The multi-thread and multi-process approaches are conceptually simple since each connection appears to be synchronous and blocking. They are both actually async under the covers, but the async i/o and dispatch is abstracted away by the OS, so the user program doesn't have to worry about it. The in-client async approach is generally the most efficient (OS processes and threads are expensive), but imposes complexity on the client protocol by making it juggle what each connection is doing, i.e. where it is in the network protocol at any moment. A lot of ways have developed over the years to organize client-side async programs and keep them from getting too confusing: - explicit state machines (a struct with a state tag for each connection, and a big event loop with a switch statement), frequently seen in C programs - Chained callbacks ("callback hell") seen in node.js - Callbacks on objects ("reactor pattern"), used in Twisted Matrix - explicit cooperative multitasking (used in RTOS's, classic Forth, etc.) - lightweight processes or threads handled by the language runtime (GHC, Erlang). This means the user program thinks it's doing blocking i/o but it's really not. - coroutines (Lua and now Python's asyncio) - continuation-based hackery (various Haskell enumeratee libraries) - probably more that I don't know about or am forgetting. I like the Erlang/GHC approach best, but it basically means building a miniature OS into the language runtime and making sure all the user-visible i/o calls actually use this "OS" instead of actual system calls to the underlying kernel. The Erlang and GHC implementations are quite complicated while Python is basically a fairly simple interpreter wrapped around the standard C libraries. In Python, the async discussion is basically between various forms of callbacks, and a few different forms of coroutines. I think that the conception of best practices is still not completely settled. I ignore all this and use threads and take the performance hit. I find it simpler and I haven't been eaten by any thread monsters yet (though there's always a first time). I figure if I need high performance, Python isn't the way to do it in the first place: Python is more about convenience and productivity than performance. I've had 1000 or so Python threads on a midsized EC2 instance and it's worked ok. If you really want to do crazy fast async i/o and you use C++, check out http://www.seastar-project.org/ I haven't tried it yet but want to. Here's a cool paper about the current GHC I/O system: http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf From michael.selik at gmail.com Thu Aug 11 13:54:31 2016 From: michael.selik at gmail.com (Michael Selik) Date: Thu, 11 Aug 2016 17:54:31 +0000 Subject: Print function not working In-Reply-To: <28a42a5a-fe70-6d86-cca6-5398501d80f1@mrabarnett.plus.com> References: <7569f8c5-63a3-422b-8c75-22d85075fe33@googlegroups.com> <28a42a5a-fe70-6d86-cca6-5398501d80f1@mrabarnett.plus.com> Message-ID: On Thu, Aug 11, 2016 at 1:38 PM MRAB wrote: > On 2016-08-11 18:18, Chris Angelico wrote: > > On Fri, Aug 12, 2016 at 3:03 AM, Atri Mahapatra > > wrote: > >> I have installed IDLE 3.5.1 and wrote the following to check if print > is working. When it runs, I do not see anything is printed: > >> > >> class Base: #{ > >> def __init__( self ): #{ > >> print("Hello, world: \n\n"); > >> > >> #} > >> > >> #} > >> > >> > >> if ( __name__ == " __main__"): #{ > >> root = Base(); > >> #} > >> > >> Can anyone please point out the reason? > >> > >> Thanks, > >> Atri > > > > Is name equal to main? That is, are you running this code as a top-level > script? > > > It's not checking whether the name is "__main__", but whether it's " > __main__" (note the space after the first quote). > > > Also: You do not need to write Python code as if it were C or Java. > > All that extra punctuation is just putting unnecessary stress on the > > world's punctuation mines, which are already overworked. :) > > > +1 > When debugging something like this, one technique is to remove bits of code to reduce the complexity. Remove something, run the code, check the results, then repeat if it's still failing. If you're not sure how class initializers work, maybe convert to a regular function and try again. def foo(): print('hello') if __name__ == '__main__': foo() If that still doesn't work, maybe get rid of the function. if __name__ == '__main__': print('hello') And if that still doesn't work, maybe get rid of the if-statement. print('hello') I think you'll find that the single line program "print('hello')" works just fine. From rowen at uw.edu Thu Aug 11 14:56:57 2016 From: rowen at uw.edu (Russell Owen) Date: Thu, 11 Aug 2016 11:56:57 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/10/16 3:44 PM, Juan Pablo Romero M?ndez wrote: > As to why I asked that, there are several reasons: I have a very concrete > need right now to find pragmatic ways to increase code quality, reduce > number of defects, etc. in a Python code base. But also I want to > understand better the mind set and culture of Python's community. I am late to this thread, so my apologies for duplicated answers, but I have two concrete suggestions: - Unit tests. These are a hassle to write, but pay huge dividends in robustness of your existing code and making it safer to modify the code later. There are also tools to measure test coverage which are worth considering. I don't think it is possible to write robust code in any language (even compiled languages) without a good test suite. - Always run a linter such as flake8. Most source code editors can be configured to do this automatically. This will not catch everything that a compiler would catch in a compiled language, but it will catch many common errors. -- Russell From roel at roelschroeven.net Thu Aug 11 15:41:35 2016 From: roel at roelschroeven.net (Roel Schroeven) Date: Thu, 11 Aug 2016 21:41:35 +0200 Subject: ctypes And The WACAH Principle In-Reply-To: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: Lawrence D?Oliveiro schreef op 2016-08-10 03:45: > ... so WACAH very much applies here. Pardon my ignorance, but what is the WACAH Principle? -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From python at mrabarnett.plus.com Thu Aug 11 16:31:11 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 11 Aug 2016 21:31:11 +0100 Subject: ctypes And The WACAH Principle In-Reply-To: References: <24fd76a5-6f77-4639-aa3a-7c2fe8cc7cad@googlegroups.com> Message-ID: On 2016-08-11 20:41, Roel Schroeven wrote: > Lawrence D?Oliveiro schreef op 2016-08-10 03:45: >> ... so WACAH very much applies here. > > Pardon my ignorance, but what is the WACAH Principle? > I believe he means "??We're All Consenting Adults Here". (I don't think I've ever seen that acronym before.) From mok-kong.shen at t-online.de Thu Aug 11 17:06:05 2016 From: mok-kong.shen at t-online.de (Mok-Kong Shen) Date: Thu, 11 Aug 2016 23:06:05 +0200 Subject: A strange list concatenation result Message-ID: def test(list1,list2): list1+=[4,5,6] list2=list2+[4,5,6] print("inside ",list1,list2) return # With list1=list2=[1,2,3] test(list1,list2) print("outside",list1,list2) # I got the following: # inside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6, 4, 5, 6] # outside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] # With list1=[1,2,3] list2=[1,2,3] test(list1,list2) print("outside",list1,list2) # I got the following: # inside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] # outside [1, 2, 3, 4, 5, 6] [1, 2, 3] From gherron at digipen.edu Thu Aug 11 17:49:40 2016 From: gherron at digipen.edu (Gary Herron) Date: Thu, 11 Aug 2016 15:49:40 -0600 Subject: A strange list concatenation result In-Reply-To: References: Message-ID: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> On 08/11/2016 03:06 PM, Mok-Kong Shen wrote: > > def test(list1,list2): > list1+=[4,5,6] > list2=list2+[4,5,6] > print("inside ",list1,list2) > return > > # With > > list1=list2=[1,2,3] > test(list1,list2) > print("outside",list1,list2) > > # I got the following: > # inside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6, 4, 5, 6] > # outside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] > > # With > > list1=[1,2,3] > list2=[1,2,3] > test(list1,list2) > print("outside",list1,list2) > > # I got the following: > # inside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] > # outside [1, 2, 3, 4, 5, 6] [1, 2, 3] I think the following shows the same issue in a much simpler fashion: In this (and your first) example, there is only one list, although it has two names to reference it. >>> list1 = list2 = [1,2,3] >>> list1 += [4,5,6] >>> print(list1, list2) [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] In this next example, there are two separate lists: >>> list1 = [1,2,3] >>> list2 = [1,2,3] >>> list1 += [4,5,6] >>> print(list1, list2) [1, 2, 3, 4, 5, 6] [1, 2, 3] Does that help? Gary Herron -- Dr. Gary Herron Professor of Computer Science DigiPen Institute of Technology (425) 895-4418 From lawrencedo99 at gmail.com Thu Aug 11 19:06:20 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 16:06:20 -0700 (PDT) Subject: Generate reports in Python In-Reply-To: <2595c308-5f38-40ac-833a-f85f247f358d@googlegroups.com> References: <2595c308-5f38-40ac-833a-f85f247f358d@googlegroups.com> Message-ID: <5328acdd-fe29-4cd4-9406-be674d036e76@googlegroups.com> On Thursday, August 11, 2016 at 8:30:42 PM UTC+12, wxjm... at gmail.com wrote: > I do not see to much how a library can beat that approach, unless it > duplicates what TeX does. TEX may be fine for more free-form text, but often reports take a more structured layout than that--fixed columns etc. For example, I generate my client invoices using odfpy , so that I can do any necessary tweaking using LibreOffice Writer before outputting a PDF file. From Seymore4Head at Hotmail.invalid Thu Aug 11 19:39:55 2016 From: Seymore4Head at Hotmail.invalid (Seymore4Head) Date: Thu, 11 Aug 2016 19:39:55 -0400 Subject: OT Winmx works again Message-ID: From steve+python at pearwood.info Thu Aug 11 19:45:56 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 09:45:56 +1000 Subject: A strange list concatenation result References: Message-ID: <57ad0e36$0$22141$c3e8da3$5496439d@news.astraweb.com> Note that your subject line is wrong. You are not doing list concatenation. Unfortunately, for technical and optimization reasons, the += assignment operator for lists is in-place, which means that it is NOT the same as ordinary list concatenation + operator. It is equivalent to calling the list.extend method. If you want to do concatenation, creating NEW lists on every operation, you have to write it the expanded way: list1 = list1 + list2 # creates a new list from the concatenation of list1 and list2 Your code starts like this: list1 = list2 = [1, 2, 3] That is NOT two different lists with equal contents, but ONE list with two names. You can see this with this: py> list1 = list2 = [1, 2, 3] py> list1.append(4) py> list2 [1, 2, 3, 4] py> list1 is list2 # Are they the same object? True So what happens when you do an in-place extend of a list to itself? You should be able to guess from the example of append: py> list1.extend(list2) py> list2 [1, 2, 3, 4, 1, 2, 3, 4] list1 will be the same, because list1 and list2 are the same list. Using += will give the same results. So you have two gotchas here: a = b = [1, 2, 3] gives you the same list with two names. You could make a copy: a = [1, 2, 3] b = a[:] # using slicing or: import copy a = [1, 2, 3] b = copy.copy(a) The second gotcha is that += is not always equivalent to + and in particular for lists it is equivalent to calling the extend() method. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Aug 11 20:08:00 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 10:08:00 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> Thanks to everyone who has answered, I think I'm slowly starting to get it now. Let's see if we can come up with a toy example that doesn't involve low-level socket programming :-) Let me simulate a slow function call: import random, time def work(id): print("starting with id", id) workload = random.randint(5, 15) for i in range(workload): time.sleep(0.2) # pretend to do some real work print("processing id", id) # let the user see some progress print("done with id", id) return 10 + id pending = [1, 2, 3, 4] for i, n in enumerate(pending): pending[i] = work(n) How do I write work() so that it cooperatively multi-tasks with other ... threads? processes? what the hell do we call these things? What does this example become in the asynchronous world? In this case, all the work is pure computation, so I don't expect to save any time by doing this, because the work is still all being done in the same process/thread, not in parallel. It may even take a little bit longer, due to the overhead of switching from one to another. (I presume that actual disk or network I/O may be better, because the OS will run the I/O in parallel if possible, but I don't expect that in this case.) Am I getting close? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Thu Aug 11 20:43:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Aug 2016 10:43:41 +1000 Subject: Asynchronous programming In-Reply-To: <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 12, 2016 at 10:08 AM, Steven D'Aprano wrote: > Let me simulate a slow function call: > > > import random, time > > def work(id): > print("starting with id", id) > workload = random.randint(5, 15) > for i in range(workload): > time.sleep(0.2) # pretend to do some real work > print("processing id", id) # let the user see some progress > print("done with id", id) > return 10 + id > > > pending = [1, 2, 3, 4] > > for i, n in enumerate(pending): > pending[i] = work(n) > > > How do I write work() so that it cooperatively multi-tasks with other ... > threads? processes? what the hell do we call these things? What does this > example become in the asynchronous world? > > In this case, all the work is pure computation, so I don't expect to save > any time by doing this, because the work is still all being done in the > same process/thread, not in parallel. It may even take a little bit longer, > due to the overhead of switching from one to another. > > (I presume that actual disk or network I/O may be better, because the OS > will run the I/O in parallel if possible, but I don't expect that in this > case.) > > Am I getting close? Well, let's see. I can quickly tweak my select() demo to support time delays. # Partially borrowed from example in Python docs: # https://docs.python.org/3/library/selectors.html#examples import selectors import socket import time sel = selectors.DefaultSelector() sleepers = {} # In a non-toy, this would probably be a heap, not a dict def eventloop(): while "loop forever": t = time.time() for gen, tm in list(sleepers.items()): if tm <= t: del sleepers[gen] run_task(gen) delay = min(sleepers.values(), default=t+3600) - t if delay < 0: continue for key, mask in sel.select(timeout=delay): sel.unregister(key.fileobj) run_task(key.data) def run_task(gen): try: waitfor = next(gen) if isinstance(waitfor, float): sleepers[gen] = waitfor else: sel.register(waitfor, selectors.EVENT_READ, gen) except StopIteration: pass def sleep(tm): yield time.time() + tm def mainsock(): sock = socket.socket() sock.bind(('localhost', 1234)) sock.listen(100) sock.setblocking(False) print("Listening on port 1234.") while "moar sockets": yield sock conn, addr = sock.accept() # Should be ready print('accepted', conn, 'from', addr) conn.setblocking(False) run_task(client(conn)) def client(conn): while "moar data": yield conn data = conn.recv(1000) # Should be ready if not data: break print("Got data") # At this point, you'd do something smart with the data. # But we don't. We just echo back, after a delay. yield from sleep(3) conn.send(data) # Hope it won't block if b"quit" in data: break print('closing', conn) conn.close() if __name__ == '__main__': run_task(mainsock()) eventloop() So, if it used this tiny event loop, your code would look like this: def work(id): print("starting with id", id) workload = random.randint(5, 15) for i in range(workload): yield from sleep(0.2) # pretend to do some real work print("processing id", id) # let the user see some progress print("done with id", id) return 10 + id for n in [1, 2, 3, 4]: run_task(work(n)) eventloop() But crucially, this depends on having some kind of waitable work. That generally means either non-blocking I/O (of some sort - a lot of things in a Unix system end up being reads and writes to some file-like thing, eg a socket, pipe, or device), or a time delay, or maybe waiting on a signal. If the 'work' is a blocking CPU-bound operation, this will never be able to multiplex. ChrisA From lawrencedo99 at gmail.com Thu Aug 11 22:55:34 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 19:55:34 -0700 (PDT) Subject: =?UTF-8?Q?Re=3A_Is_it_=E2=80=98allowed=E2=80=99_to_get_parameters_like_this?= In-Reply-To: <87popfmobn.fsf@Equus.decebal.nl> References: <87popfmobn.fsf@Equus.decebal.nl> Message-ID: On Friday, August 12, 2016 at 1:28:04 AM UTC+12, Cecil Westerhof wrote: > As I understood it getopt is the way to get the parameters for your > script. But at the moment it looks a bit like overkill, my script does > not have much parameters as parameter_error shows: That?s fine. But your command-line needs are so simple, I wouldn?t bother with the ?--? business either. Just take the interval as a single argument. Here?s how I would do it: if len(sys.argv) == 2 : periodspec = sys.argv[1] elif len(sys.argv) == 1 : periodspec = "all" else : raise RuntimeError("usage:\n {} [period]" % sys.argv[0]) #end if today = time.strftime("%Y-%m-%d") # or whatever # original did ?select current_date? from SQLite if re.fullmatch(r"^\d+$", periodspec) != None : period = int(periodspec) else : period = \ { "today" : lambda : today + "%", "this-month" : lambda : today[0:8] + "%", "this-year" : lambda : today[0:5] + "%", "all" : lambda : "%", }.get(periodspec, lambda : None)() if period == None : raise RuntimeError("{}: unrecognized period spec ?%s?" % periodspec) #end if #end if sys.stderr.write("period = %s\n" % repr(period)) # debug # ... # vmstat_params = (period, slice_length) # ... etc So you would call that like ./demo 5 ./demo today ./demo etc. And as a bonus, it will accept any integer interval, not just your original limited set. And you see how I wrote the table of symbolic intervals? That makes it easy to add new entries. From lawrencedo99 at gmail.com Thu Aug 11 23:55:58 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 20:55:58 -0700 (PDT) Subject: Asynchronous programming In-Reply-To: References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Friday, August 12, 2016 at 2:25:05 AM UTC+12, Terry Reedy wrote: > When I read something like "Python finally acquired an event loop in > 3.4" I wonder where people have been. The tk event loop has been in > Python perhaps for 2 decades... As was pointed out to me just a few days ago, that?s Tcl, not Python. From lawrencedo99 at gmail.com Fri Aug 12 00:46:50 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 21:46:50 -0700 (PDT) Subject: python 3.5.2 lounch: api-ms-win-crt-runtime-l1-1-0.dll is missing ? In-Reply-To: References: <000a01d1f0b6$361a9180$a24fb480$@laposte.net> Message-ID: <43e4828f-110f-4203-a5e4-72c0c4afcbfd@googlegroups.com> On Tuesday, August 9, 2016 at 6:46:52 AM UTC+12, Joaquin Alzola wrote: > This email is confidential and may be subject to privilege. If you are not > the intended recipient, please do not copy or disclose its content but > contact the sender immediately upon receipt. Consider yourself contacted! From lawrencedo99 at gmail.com Fri Aug 12 00:48:09 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 21:48:09 -0700 (PDT) Subject: Call for Assistance In-Reply-To: References: Message-ID: On Wednesday, August 10, 2016 at 3:42:39 AM UTC+12, Reto Brunner wrote: > What on earth isn't "free" enough about > > You are free to: > Share ? copy and redistribute the material in any medium or format No you are not. A court has ruled quite clearly that ?non-commercial? means PERSONAL USE ONLY, no redistribution or republication. From lawrencedo99 at gmail.com Fri Aug 12 00:49:04 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 11 Aug 2016 21:49:04 -0700 (PDT) Subject: Call for Assistance In-Reply-To: References: <517DCCE7-1E0F-40D9-9F74-7794ED82F89B@mac.com> <1CB4A3ED-4733-4F7A-99BC-8C09269EECB3@mac.com> Message-ID: <87f41abc-2ace-4990-94fd-f66422989114@googlegroups.com> On Wednesday, August 10, 2016 at 1:09:31 PM UTC+12, Charles Ross wrote: > Well, I?ve been convinced. The license for the book is now Creative > Commons Attribution-ShareAlike. Great. :) That?s the one to use if you are worried about companies who want something for nothing coming in and ripping you off. From no.email at nospam.invalid Fri Aug 12 00:53:52 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 11 Aug 2016 21:53:52 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <871t1uzixr.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > How do I write work() so that it cooperatively multi-tasks with other ... > threads? processes? what the hell do we call these things? What does this > example become in the asynchronous world? If it's heavily computational then you have to yield to the scheduler frequently so that other tasks have a chance to run. So if your 0.2 second computation involves 100 iterations of something, you could yield once per iteration (every 2 msec). From no.email at nospam.invalid Fri Aug 12 00:55:21 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 11 Aug 2016 21:55:21 -0700 Subject: Call for Assistance References: <517DCCE7-1E0F-40D9-9F74-7794ED82F89B@mac.com> <1CB4A3ED-4733-4F7A-99BC-8C09269EECB3@mac.com> Message-ID: <87wpjmy4au.fsf@jester.gateway.pace.com> Charles Ross writes: > Well, I?ve been convinced. The license for the book is now Creative > Commons Attribution-ShareAlike. That means you can post it on wikibooks.org and let people edit it directly, if you want. Wikibooks is nowhere near as crazy as wikipedia. From paul.nospam at rudin.co.uk Fri Aug 12 01:47:59 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Fri, 12 Aug 2016 06:47:59 +0100 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87shua1qsw.fsf@rudin.co.uk> Steven D'Aprano writes: > Thanks to everyone who has answered, I think I'm slowly starting to get it > now. Let's see if we can come up with a toy example that doesn't involve > low-level socket programming :-) > > Let me simulate a slow function call: > > > import random, time > > def work(id): > print("starting with id", id) > workload = random.randint(5, 15) > for i in range(workload): > time.sleep(0.2) # pretend to do some real work > print("processing id", id) # let the user see some progress > print("done with id", id) > return 10 + id > > > pending = [1, 2, 3, 4] > > for i, n in enumerate(pending): > pending[i] = work(n) > > > How do I write work() so that it cooperatively multi-tasks with other ... > threads? processes? what the hell do we call these things? What does this > example become in the asynchronous world? > They're not separate processes or threads, just think tasks that suspend and restart cooperatively. You need some kind of event loop to orchestrate running the tasks. Asyncio provides one. You can do your example like this: import asyncio, random async def work(id): print("starting with id", id) workload = random.randint(5, 15) for i in range(workload): await asyncio.sleep(0.2) # pretend to do some real work print("processing id", id) # let the user see some progress print("done with id", id) return 10 + id loop = asyncio.get_event_loop() pending = [1, 2, 3, 4] jobs = [asyncio.ensure_future(work(n)) for n in pending] loop.run_until_complete(asyncio.gather(*jobs)) loop.close() > In this case, all the work is pure computation, so I don't expect to save > any time by doing this, because the work is still all being done in the > same process/thread, not in parallel. It may even take a little bit longer, > due to the overhead of switching from one to another. > Yeah - you're not gaining anything here. All (?) the interesting use cases involve waiting for something (e.g. disk access, network communication, separate process) before you can proceed with your computation. From paul.nospam at rudin.co.uk Fri Aug 12 01:47:59 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Fri, 12 Aug 2016 06:47:59 +0100 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87shua1qsw.fsf@rudin.co.uk> Steven D'Aprano writes: > Thanks to everyone who has answered, I think I'm slowly starting to get it > now. Let's see if we can come up with a toy example that doesn't involve > low-level socket programming :-) > > Let me simulate a slow function call: > > > import random, time > > def work(id): > print("starting with id", id) > workload = random.randint(5, 15) > for i in range(workload): > time.sleep(0.2) # pretend to do some real work > print("processing id", id) # let the user see some progress > print("done with id", id) > return 10 + id > > > pending = [1, 2, 3, 4] > > for i, n in enumerate(pending): > pending[i] = work(n) > > > How do I write work() so that it cooperatively multi-tasks with other ... > threads? processes? what the hell do we call these things? What does this > example become in the asynchronous world? > They're not separate processes or threads, just think tasks that suspend and restart cooperatively. You need some kind of event loop to orchestrate running the tasks. Asyncio provides one. You can do your example like this: import asyncio, random async def work(id): print("starting with id", id) workload = random.randint(5, 15) for i in range(workload): await asyncio.sleep(0.2) # pretend to do some real work print("processing id", id) # let the user see some progress print("done with id", id) return 10 + id loop = asyncio.get_event_loop() pending = [1, 2, 3, 4] jobs = [asyncio.ensure_future(work(n)) for n in pending] loop.run_until_complete(asyncio.gather(*jobs)) loop.close() > In this case, all the work is pure computation, so I don't expect to save > any time by doing this, because the work is still all being done in the > same process/thread, not in parallel. It may even take a little bit longer, > due to the overhead of switching from one to another. > Yeah - you're not gaining anything here. All (?) the interesting use cases involve waiting for something (e.g. disk access, network communication, separate process) before you can proceed with your computation. From andreas.roehler at online.de Fri Aug 12 02:24:08 2016 From: andreas.roehler at online.de (=?UTF-8?Q?Andreas_R=c3=b6hler?=) Date: Fri, 12 Aug 2016 08:24:08 +0200 Subject: Call for Assistance In-Reply-To: References: Message-ID: <12815e34-d6a1-49fb-29fe-71c975de1e4d@online.de> On 12.08.2016 06:48, Lawrence D?Oliveiro wrote: > On Wednesday, August 10, 2016 at 3:42:39 AM UTC+12, Reto Brunner wrote: >> What on earth isn't "free" enough about >> >> You are free to: >> Share ? copy and redistribute the material in any medium or format > No you are not. A court has ruled quite clearly that ?non-commercial? means PERSONAL USE ONLY, no redistribution or republication. The court ruled at the special case. The institution in question is paid by all germans due to a kind of fea imposed - "Rundfunkbeitrag". Everyone owning or renting an object in Germany has to pay to them - without regard if they uses that media or not. If you have to pay to someone, you would not consider that relation non-commercial. From lawrencedo99 at gmail.com Fri Aug 12 04:10:20 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 12 Aug 2016 01:10:20 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> On Thursday, August 11, 2016 at 8:33:41 AM UTC+12, Juan Pablo Romero M?ndez wrote: > I've been trying to find (without success so far) an example of a situation > where the dynamic features of a language like Python provides a clear > advantage over languages with more than one type. I have used, and continue to use, both static and dynamic languages. With static languages, once a piece of code compiles without errors, you have a slightly higher level of confidence in its correctness than with a dynamic language. On the other hand, a dynamic language allows me to be much more productive, because I have to write less code to begin with. The closest I can offer for an apples-to-apples comparison is PyCairo versus Qahirah . Both are Python bindings for the Cairo graphics library; the former is written in C as a Python extension module, the latter is done in pure Python using ctypes. I didn?t write the former; I merely tried to rescue it from abandonment to see if I could fill in a few more missing features. And what I found was, it would be quicker to start again from scratch than to continue working on it. From marko at pacujo.net Fri Aug 12 04:38:18 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 12 Aug 2016 11:38:18 +0300 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> Message-ID: <87inv6fklh.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > With static languages, once a piece of code compiles without errors, > you have a slightly higher level of confidence in its correctness than > with a dynamic language. > > On the other hand, a dynamic language allows me to be much more > productive, because I have to write less code to begin with. I believe a high-level language like Python makes it easier to write correct code as well. That's because every line you write has a nonzero chance of containing a bug, and Python allows you to express complicated notions more naturally and fewer code lines. In a word, Python makes it easier to see the forest for the trees. Marko From bc at freeuk.com Fri Aug 12 05:38:48 2016 From: bc at freeuk.com (BartC) Date: Fri, 12 Aug 2016 10:38:48 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <87inv6fklh.fsf@elektro.pacujo.net> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: On 12/08/2016 09:38, Marko Rauhamaa wrote: > Lawrence D?Oliveiro : > >> With static languages, once a piece of code compiles without errors, >> you have a slightly higher level of confidence in its correctness than >> with a dynamic language. >> >> On the other hand, a dynamic language allows me to be much more >> productive, because I have to write less code to begin with. You can be too dynamic. Take an example like this: class date: def __init__(self,d,m,y): self.day=d self.month=m self.year=y d=date(25,12,2015) d.yaer=1999 print (d.day,d.month,d.year) 'year' has been spelled wrongly, but this error is not picked up by the byte-code compiler and will not immediately be detected at runtime either. It will be seen in my example only if someone notices the printout isn't what it should be. This would never get past a static language nor some that also have dynamic types. -- Bartc From rosuav at gmail.com Fri Aug 12 05:45:42 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Aug 2016 19:45:42 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: On Fri, Aug 12, 2016 at 7:38 PM, BartC wrote: > You can be too dynamic. Take an example like this: > > class date: > def __init__(self,d,m,y): > self.day=d > self.month=m > self.year=y > > d=date(25,12,2015) > > d.yaer=1999 > > print (d.day,d.month,d.year) > > 'year' has been spelled wrongly, but this error is not picked up by the > byte-code compiler and will not immediately be detected at runtime either. > It will be seen in my example only if someone notices the printout isn't > what it should be. > > This would never get past a static language nor some that also have dynamic > types. Nor will it get past a lot of linters. Or unit tests. There are other solutions. ChrisA From bc at freeuk.com Fri Aug 12 05:58:31 2016 From: bc at freeuk.com (BartC) Date: Fri, 12 Aug 2016 10:58:31 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: On 12/08/2016 10:45, Chris Angelico wrote: > On Fri, Aug 12, 2016 at 7:38 PM, BartC wrote: >> You can be too dynamic. Take an example like this: >> d.yaer=1999 >> >> print (d.day,d.month,d.year) >> This would never get past a static language nor some that also have dynamic >> types. > > Nor will it get past a lot of linters. Or unit tests. There are other solutions. How would a linter know that you didn't /want/ to create a new attribute called 'yaer'? And unit tests (which I only vaguely know about) I don't think are universally used. People like to use dynamic languages to code rapidly and informally. -- Bartc From steve+python at pearwood.info Fri Aug 12 07:07:12 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 21:07:12 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Aug 2016 07:38 pm, BartC wrote: > 'year' has been spelled wrongly How do you know? The point is that it is intentional that callers can set arbitrary attributes on (most) objects. It is perfectly legitimate to set: d.year d.century d.owner d.extra_seconds and, yes, even d.yaer, since it isn't the compiler's job to check your spelling or critique your choice of variable names. If you want to spell check your code, run a spellchecker[1], or a linter, or have a human review it. Or all three. There are broadly two tactics that a language can take when it comes to attribute (or more widely, variable) names: (1) Editing is cheap, but compiling and testing code is expensive, and the consequences of accessing a non-existent variable are catastrophic (overwriting arbitrary memory, accessing uninitialised memory, segmentation faults, execution of arbitrary code, security vulnerabilities, etc). Therefore it is important that the compiler check ahead of time that every name referenced exists and has been declared. (2) Compiling and testing code is cheap, and the consequences of accessing non-existent variables are mild. Since typos are generally rare (say, one name out of a hundred is mispelled), requiring declarations is a waste of effort. Rare errors will be caught through human review, linters, and testing. The first time I ever compiled a full-sized application (not a particular large one either, it was a text editor a little more featureful than Notepad) it took something like nine hours to compile on a Mac SE (this was circa 1990). How mad would I have been if, eight hours and thirty minutes into the compilation, the compiler suddenly stopped with an error caused by a mistyped variable name? Today, I could probably compile the equivalent program in a minute. Do I care if the compiler doesn't pick up the typo until 58 seconds through the process? Not a whit. I would care if the compiler doesn't pick up the error *at all* and I'm writing in a low level language where accessing a non-existent variable allows Bulgarian hackers to steal the money in my bank account, so there's still some advantage to having declarations in low-level languages. It *absolutely makes sense* to require variable declarations (possibly including type information, where it cannot be inferred) when the consequences of a typo are routinely expected to be severe. Under those circumstances, a "bondage and discipline" language that forces you to be absolutely pedantic to satisfy the compiler can be a good thing. Turn it around though: one of the reasons why languages like Ruby, Javascript, Lua, Python, Perl etc are so ridiculously productive is that you don't have to keep stopping to declare variables just to satisfy the compiler. Need a new variable? Just assign to it and keep going -- there's no, or very little, mental task switching. Another reason why such languages are so productive is the availability of an interactive REPL where you can try out code interactively. How clumsy and awkward the REPL would be if you had to keep declaring variables ahead of time. The cost of that flexibility is that I have to pay a little bit more attention when doing a code review, I have to write a few more tests. But the edit-compile-run cycle is so ridiculously fast that this doesn't really matter. Programmers have invented entirely new coding paradigms, Test Driven Development and Agile, to take advantage of this speed and flexibility. Sure, you can take TDD and Agile into the realm of statically typed languages, but it will always be like running a sprint wearing a suit of armour. You can do it, but you'll be out-paced by the sprinter wearing track shoes and Lycra. [1] Are there programming language aware spell checkers? If not, there should be. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Aug 12 07:51:30 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Fri, 12 Aug 2016 21:51:30 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: <57adb844$0$22142$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Aug 2016 07:58 pm, BartC wrote: > On 12/08/2016 10:45, Chris Angelico wrote: >> On Fri, Aug 12, 2016 at 7:38 PM, BartC wrote: >>> You can be too dynamic. Take an example like this: > >>> d.yaer=1999 [...] > How would a linter know that you didn't /want/ to create a new attribute > called 'yaer'? Of course the linter can't tell, but it is the job of the linter to complain about things which *might* be errors. You don't have to listen to the linter. Think of it as your mother nagging you to put a sweater on because she's cold. It is the job of the compiler to complain about things which *definitely are* errors. Think of compiler errors as your abusive father threatening to beat you with a thick leather belt if you don't do *exactly* what he says. > And unit tests (which I only vaguely know about) I don't think are > universally used. Well, of course. There are still plenty of people whose coding paradigm is "Oh, it compiles? Quick, ship it!" > People like to use dynamic languages to code rapidly > and informally. And you can do that. And you can also code rapidly and formally with Test Driven Development, Agile[1] and Scrum. [1] Often misused to the point that Agile and Scrum are considered by some to be forms of class warfare conducted by management against programmers. Especially when combined with stack ranking, quite possible *the* most employee-hostile management technique since keel-hauling. But Agile itself is not to blame for the abuses it is put to. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Fri Aug 12 07:55:43 2016 From: bc at freeuk.com (BartC) Date: Fri, 12 Aug 2016 12:55:43 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 12/08/2016 12:07, Steven D'Aprano wrote: > On Fri, 12 Aug 2016 07:38 pm, BartC wrote: > >> 'year' has been spelled wrongly > > How do you know? I know because my intention was to create a RECORD, with a specific set of members or fields. With records, you usually don't just create arbitrary named members as you go along. But in Python, the mechanism and syntax used to create such members is the same as used for ad-hoc attributes. This was the point I was making about sometimes it can be a little too dynamic. > There are broadly two tactics that a language can take when it comes to > attribute (or more widely, variable) names: > > > (1) Editing is cheap, but compiling and testing code is expensive No, compiling is cheap too. > The first time I ever compiled a full-sized application (not a particular > large one either, it was a text editor a little more featureful than > Notepad) it took something like nine hours to compile on a Mac SE (this was > circa 1990). That is completely crazy even for 1990. I was using my own tools then and the time to re-compile a 50,000-line application would have been measured in seconds. Certainly not minutes anyway (I wouldn't have had the patience!). > How mad would I have been if, eight hours and thirty minutes > into the compilation, the compiler suddenly stopped with an error caused by > a mistyped variable name? > > Today, I could probably compile the equivalent program in a minute. I'm actually working on a compiler now that could compile a 50Kloc application in about 1/16th of a second (to byte-code; perhaps 1/8th of a second to in-memory native code). A pass that merely checks that names have been declared has effectively zero cost. > It *absolutely makes sense* to require variable declarations (possibly > including type information, where it cannot be inferred) My example was specifically about attribute names where pre-declaring the set of allowed attributes would not be onerous, would be usefully self-documenting, and would allow more errors to be picked up. (Of course the design of Python makes that impractical because it would require the byte-code compiler to see inside imported modules before execution is commenced.) > Turn it around though: one of the reasons why languages like Ruby, > Javascript, Lua, Python, Perl etc are so ridiculously productive is that > you don't have to keep stopping to declare variables just to satisfy the > compiler. Need a new variable? Just assign to it and keep going -- there's > no, or very little, mental task switching. > > Another reason why such languages are so productive is the availability of > an interactive REPL where you can try out code interactively. How clumsy > and awkward the REPL would be if you had to keep declaring variables ahead > of time. I agree when it comes to variables. But it /does/ allow these extra errors to creep in that are not detected until that particular fragment of code is executed. -- Bartc From torriem at gmail.com Fri Aug 12 09:30:33 2016 From: torriem at gmail.com (Michael Torrie) Date: Fri, 12 Aug 2016 07:30:33 -0600 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> On 08/12/2016 05:07 AM, Steven D'Aprano wrote: > The first time I ever compiled a full-sized application (not a particular > large one either, it was a text editor a little more featureful than > Notepad) it took something like nine hours to compile on a Mac SE (this was > circa 1990). How mad would I have been if, eight hours and thirty minutes > into the compilation, the compiler suddenly stopped with an error caused by > a mistyped variable name? Surely this application was built using a build system, even back then, that would allow compilation to resume and not rebuild object files that were already built. From bc at freeuk.com Fri Aug 12 09:33:39 2016 From: bc at freeuk.com (BartC) Date: Fri, 12 Aug 2016 14:33:39 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 12/08/2016 12:55, BartC wrote: > On 12/08/2016 12:07, Steven D'Aprano wrote: >> The first time I ever compiled a full-sized application (not a particular >> large one either, it was a text editor a little more featureful than >> Notepad) it took something like nine hours to compile on a Mac SE >> (this was >> circa 1990). > > That is completely crazy even for 1990. (So crazy that you must either have added a couple of zeros or there's something unusual you haven't mentioned (maybe the SE was emulating another machine and that emulator was interpreted - badly. Or the files were at the other end of a 300baud connection.) A build time of nine hours means 3 lines per second for a 100Kloc app, and 30 lps for a 1Mloc app.) -- Bartc From michael.selik at gmail.com Fri Aug 12 09:44:50 2016 From: michael.selik at gmail.com (Michael Selik) Date: Fri, 12 Aug 2016 13:44:50 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Aug 12, 2016, 7:11 AM Steven D'Aprano wrote: > > [1] Are there programming language aware spell checkers? If not, there > should be. > A good autocomplete is much like a spell-checker. I have far fewer spelling errors when using an editor with autocomplete. > From wrw at mac.com Fri Aug 12 10:23:47 2016 From: wrw at mac.com (William Ray Wing) Date: Fri, 12 Aug 2016 10:23:47 -0400 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: > On Aug 12, 2016, at 7:07 AM, Steven D'Aprano wrote: > > [megabyte] > > > [1] Are there programming language aware spell checkers? If not, there > should be. > > There are programming language-aware editors with built-in spell checkers (and syntax coloring, but that?s a different matter). -Bill > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list From mok-kong.shen at t-online.de Fri Aug 12 16:44:40 2016 From: mok-kong.shen at t-online.de (Mok-Kong Shen) Date: Fri, 12 Aug 2016 22:44:40 +0200 Subject: A strange list concatenation result In-Reply-To: References: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> Message-ID: Am 11.08.2016 um 23:49 schrieb Gary Herron: > On 08/11/2016 03:06 PM, Mok-Kong Shen wrote: >> >> def test(list1,list2): >> list1+=[4,5,6] >> list2=list2+[4,5,6] >> print("inside ",list1,list2) >> return [snip] >> # With >> >> list1=[1,2,3] >> list2=[1,2,3] >> test(list1,list2) >> print("outside",list1,list2) >> >> # I got the following: >> # inside [1, 2, 3, 4, 5, 6] [1, 2, 3, 4, 5, 6] >> # outside [1, 2, 3, 4, 5, 6] [1, 2, 3] [snip] > In this next example, there are two separate lists: > >>>> list1 = [1,2,3] >>>> list2 = [1,2,3] >>>> list1 += [4,5,6] >>>> print(list1, list2) > [1, 2, 3, 4, 5, 6] [1, 2, 3] > > > Does that help? I don't yet understand why in my 2nd example list2 came out as [1, 2, 3] outside. M. K. From chivalry at mac.com Fri Aug 12 18:07:27 2016 From: chivalry at mac.com (Charles Ross) Date: Fri, 12 Aug 2016 15:07:27 -0700 Subject: Meta Python Chapter 3 Available In-Reply-To: <12815e34-d6a1-49fb-29fe-71c975de1e4d@online.de> References: <12815e34-d6a1-49fb-29fe-71c975de1e4d@online.de> Message-ID: <909AC976-74A0-4EF5-A044-9C0E56803336@mac.com> I?ve completed a first draft of Chapter 3: Modules (and Packages) of the Meta Python book. Feedback from novices and experts alike would be welcome. https://github.com/chivalry/meta-python Thanks, Chuck From lawrencedo99 at gmail.com Fri Aug 12 18:12:09 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 12 Aug 2016 15:12:09 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: <6adf8161-6e99-41a8-ad93-c86989768863@googlegroups.com> On Friday, August 12, 2016 at 9:39:11 PM UTC+12, BartC wrote: > 'year' has been spelled wrongly That?s why Python has __slots__. From ernest.bonat at gmail.com Fri Aug 12 19:52:30 2016 From: ernest.bonat at gmail.com (Ernest Bonat, Ph.D.) Date: Fri, 12 Aug 2016 16:52:30 -0700 Subject: Error running an exe file in Windows with reportlab import Message-ID: I have created a simple Python program including the following packages: import numpy as np import matplotlib.pyplot as plt import pandas as pd from reportlab.pdfgen import canvas if __name__ == '__main__': print("Hi!") I have created the installation package (PyInstaller) using the following command line: pyinstaller --onedir --name=runatestapp --noupx "C:\Users\Ernest\workspace\bars\src\run_a_test.py" The required folders (build and dist) and the spec file are created properly with no errors. When I run the runatestapp.exe file in the command prompt I got the error "Failed to execute script run_a_test". I'm using the following software versions now: Anaconda3 4.1.1.64-bit Python 3.5.2 PyInstaller 3.2 Windows 10 64-bit Here is the Traceback info Traceback (most recent call last): File "run_a_test.py", line 4, in from reportlab.pdfgen import canvas File "", line 969, in _find_and_load File "", line 958, in _find_and_load_unlocked File "", line 664, in _load_unlocked File "", line 634, in _load_backward_compatible File "C:\Users\Ernest\AppData\Local\Continuum\lib\site-packages\PyInstaller\loader\pyimo d03_importers.py", line 389, in load_module exec(bytecode, module.__dict__) File "site-packages\reportlab\pdfgen\canvas.py", line 19, in File "", line 969, in _find_and_load File "", line 958, in _find_and_load_unlocked File "", line 664, in _load_unlocked File "", line 634, in _load_backward_compatible File "C:\Users\Ernest\AppData\Local\Continuum\lib\site-packages\PyInstaller\loader\pyimo d03_importers.py", line 389, in load_module exec(bytecode, module.__dict__) File "site-packages\reportlab\rl_config.py", line 131, in File "site-packages\reportlab\rl_config.py", line 102, in _startUp File "site-packages\reportlab\lib\utils.py", line 695, in rl_isdir AttributeError: 'FrozenImporter' object has no attribute '_files' Failed to execute script run_a_test Any help is much appreciated. Feel free to contact me at any time you need. Thank you, Ernest Bonat, Ph.D. Senior Software Engineer Senior Data Scientist From bc at freeuk.com Fri Aug 12 19:57:53 2016 From: bc at freeuk.com (BartC) Date: Sat, 13 Aug 2016 00:57:53 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <6adf8161-6e99-41a8-ad93-c86989768863@googlegroups.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <6adf8161-6e99-41a8-ad93-c86989768863@googlegroups.com> Message-ID: On 12/08/2016 23:12, Lawrence D?Oliveiro wrote: > On Friday, August 12, 2016 at 9:39:11 PM UTC+12, BartC wrote: > >> 'year' has been spelled wrongly > > That?s why Python has __slots__. OK. So when I said: > > My example was specifically about attribute names where pre-declaring > the set of allowed attributes would not be onerous, would be usefully > self-documenting, and would allow more errors to be picked up. it turns out someone implemented just that! Although it doesn't sound like Python to bolt-on some random feature that reins in its dynamic capabilities. Most people here seem to be against limiting the language in any way. > (Of course the design of Python makes that impractical because it > would > require the byte-code compiler to see inside imported modules before > execution is commenced.) And the scheme I had in mind would detect the problem at compile-time. The '__slots__' feature can't do that (unless someone uses a very clever 'linter'), but it will actually detect an attribute mismatch at runtime instead of silently creating an extraneous one. -- Bartc From odelrisel at gmx.com Fri Aug 12 20:48:51 2016 From: odelrisel at gmx.com (GIULIANO DI BERNARDO DIGNITY ORDER WIEN) Date: Fri, 12 Aug 2016 17:48:51 -0700 (PDT) Subject: Redacted Message-ID: Redacted From odelrisel at gmx.com Fri Aug 12 20:50:09 2016 From: odelrisel at gmx.com (GIULIANO DI BERNARDO DIGNITY ORDER WIEN) Date: Fri, 12 Aug 2016 17:50:09 -0700 (PDT) Subject: Redacted Message-ID: <399188c4-8b8a-4742-ab81-27ee20915ac1@googlegroups.com> Redacted From jpablo.romero at gmail.com Fri Aug 12 20:55:20 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Fri, 12 Aug 2016 17:55:20 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> Message-ID: 2016-08-12 1:10 GMT-07:00 Lawrence D?Oliveiro : > On Thursday, August 11, 2016 at 8:33:41 AM UTC+12, Juan Pablo Romero > M?ndez wrote: > > > I've been trying to find (without success so far) an example of a > situation > > where the dynamic features of a language like Python provides a clear > > advantage over languages with more than one type. > > I have used, and continue to use, both static and dynamic languages. > > With static languages, once a piece of code compiles without errors, you > have a slightly higher level of confidence in its correctness than with a > dynamic language. > > On the other hand, a dynamic language allows me to be much more > productive, because I have to write less code to begin with. > >From your point of view, dynamic languages are more concise? > > The closest I can offer for an apples-to-apples comparison is PyCairo < > https://github.com/ldo/pycairo> versus Qahirah qahirah>. Both are Python bindings for the Cairo graphics library; the > former is written in C as a Python extension module, the latter is done in > pure Python using ctypes. > > I didn?t write the former; I merely tried to rescue it from abandonment to > see if I could fill in a few more missing features. And what I found was, > it would be quicker to start again from scratch than to continue working on > it. > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+python at pearwood.info Fri Aug 12 20:59:50 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 13 Aug 2016 10:59:50 +1000 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> Message-ID: <57ae7108$0$1586$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Aug 2016 03:47 pm, Paul Rudin wrote: > Steven D'Aprano writes: > >> Thanks to everyone who has answered, I think I'm slowly starting to get >> it now. Let's see if we can come up with a toy example that doesn't >> involve low-level socket programming :-) >> >> Let me simulate a slow function call: [...] > They're not separate processes or threads, just think tasks that suspend > and restart cooperatively. You need some kind of event loop to > orchestrate running the tasks. Asyncio provides one. > > You can do your example like this: > > import asyncio, random > > async def work(id): > print("starting with id", id) > workload = random.randint(5, 15) > for i in range(workload): > await asyncio.sleep(0.2) # pretend to do some real work > print("processing id", id) # let the user see some progress > print("done with id", id) > return 10 + id > > loop = asyncio.get_event_loop() > > pending = [1, 2, 3, 4] > > jobs = [asyncio.ensure_future(work(n)) for n in pending] > > loop.run_until_complete(asyncio.gather(*jobs)) > > loop.close() That is *awesome*. Thank you for the example! -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Aug 12 21:08:36 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 13 Aug 2016 11:08:36 +1000 Subject: A strange list concatenation result References: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> Message-ID: <57ae7314$0$1599$c3e8da3$5496439d@news.astraweb.com> On Sat, 13 Aug 2016 06:44 am, Mok-Kong Shen wrote: >>>>> list2 = [1,2,3] >>>>> list1 += [4,5,6] >>>>> print(list1, list2) >> [1, 2, 3, 4, 5, 6] [1, 2, 3] >> >> >> Does that help? > > I don't yet understand why in my 2nd example list2 came out as > [1, 2, 3] outside. Because you assign list2 = [1, 2, 3]. What did you expect it to be? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Aug 12 21:13:21 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 13 Aug 2016 11:13:21 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <6adf8161-6e99-41a8-ad93-c86989768863@googlegroups.com> Message-ID: <57ae7431$0$1589$c3e8da3$5496439d@news.astraweb.com> On Sat, 13 Aug 2016 08:12 am, Lawrence D?Oliveiro wrote: > On Friday, August 12, 2016 at 9:39:11 PM UTC+12, BartC wrote: > >> 'year' has been spelled wrongly > > That?s why Python has __slots__. No, that is OFFICIALLY *not* why Python has __slots__. Python has __slots__ in order to support applications where you have huge numbers of small objects where the collective memory used by millions of unused __dicts__ is significant. You may choose to (ab)use __slots__ to avoid the caller adding new attributes to your objects, but that practice is discouraged. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Fri Aug 12 21:31:44 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 12 Aug 2016 18:31:44 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> <57ae7108$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87oa4xxxmn.fsf@jester.gateway.pace.com> Steven D'Aprano writes: >> await asyncio.sleep(0.2) # pretend to do some real work > That is *awesome*. Thank you for the example! Keep in mind that the above basically takes the task off the list of runnables for 0.2 seconds, so it sits doing nothing and doesn't interfere with other tasks running. In the case where you're doing actual computation, you want to yield (await) much more often than 0.2 sec, so other tasks can stay responsive. This is cooperative multitasking, where each task has to be nice to the others, in contrast with preemptive multitasking where the task switches happen behind the scenes. From huey.y.jiang at gmail.com Fri Aug 12 21:40:03 2016 From: huey.y.jiang at gmail.com (huey.y.jiang at gmail.com) Date: Fri, 12 Aug 2016 18:40:03 -0700 (PDT) Subject: Why my image cannot be displayed? Message-ID: <9df5b562-eace-498c-87b1-972a09e5b15f@googlegroups.com> Hi All, Image display drives me crazy. After I get it displayed, and want to do the job with a class, display failed again. Please take a look at my trial code: from Tkinter import * class imageDisplay: def __init__(self, parent=None): canvas = Canvas(width=400, height=300, bg='beige') canvas.pack() self.canvas = canvas img = PhotoImage(file="xxx.gif") self.canvas.create_image(150, 0, image=img, anchor=NW) if __name__ == '__main__': imageDisplay() mainloop() Result: A blank canvas was displayed. No error message. Can somebody help? Thanks so much! From breamoreboy at gmail.com Fri Aug 12 22:33:14 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 12 Aug 2016 19:33:14 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> Message-ID: On Friday, August 12, 2016 at 2:31:01 PM UTC+1, Michael Torrie wrote: > On 08/12/2016 05:07 AM, Steven D'Aprano wrote: > > The first time I ever compiled a full-sized application (not a particular > > large one either, it was a text editor a little more featureful than > > Notepad) it took something like nine hours to compile on a Mac SE (this was > > circa 1990). How mad would I have been if, eight hours and thirty minutes > > into the compilation, the compiler suddenly stopped with an error caused by > > a mistyped variable name? > > Surely this application was built using a build system, even back then, > that would allow compilation to resume and not rebuild object files that > were already built. Are you one of those extremely lucky people who worked on projects in which the header files never, ever changed? Kindest regards. Mark Lawrence. From python at mrabarnett.plus.com Fri Aug 12 23:27:34 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 13 Aug 2016 04:27:34 +0100 Subject: Why my image cannot be displayed? In-Reply-To: <9df5b562-eace-498c-87b1-972a09e5b15f@googlegroups.com> References: <9df5b562-eace-498c-87b1-972a09e5b15f@googlegroups.com> Message-ID: <0998473b-0e60-7d61-4953-0b059c0f18b6@mrabarnett.plus.com> On 2016-08-13 02:40, huey.y.jiang at gmail.com wrote: > Hi All, > > Image display drives me crazy. After I get it displayed, and want to do the job with a class, display failed again. Please take a look at my trial code: > > from Tkinter import * > > class imageDisplay: > def __init__(self, parent=None): > canvas = Canvas(width=400, height=300, bg='beige') > canvas.pack() > self.canvas = canvas > img = PhotoImage(file="xxx.gif") > self.canvas.create_image(150, 0, image=img, anchor=NW) > > if __name__ == '__main__': > imageDisplay() > mainloop() > > Result: A blank canvas was displayed. No error message. > > Can somebody help? Thanks so much! > You need to keep a reference to the image, so add: self.img = img otherwise, when __init__ returns, it'll be garbage-collected. You also need to keep a reference to the instance created by imageDisplay(), so: display = imageDisplay() This is because the internals of tkinter is actually written in Tcl (another programming language), so although Tcl might have a reference to the window you're seeing, Python itself won't. (Why Tcl maintains a reference to the window, but not the image that's in the window, is a mystery... :-)) From lawrencedo99 at gmail.com Sat Aug 13 00:07:34 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 12 Aug 2016 21:07:34 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> Message-ID: On Saturday, August 13, 2016 at 2:33:55 PM UTC+12, bream... at gmail.com wrote: >> Surely this application was built using a build system, even back then, >> that would allow compilation to resume and not rebuild object files that >> were already built. > > Are you one of those extremely lucky people who worked on projects in which > the header files never, ever changed? Or we use languages where imports are considered a build dependency, and ?header files? are just a bad dream... From lawrencedo99 at gmail.com Sat Aug 13 00:22:30 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 12 Aug 2016 21:22:30 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> Message-ID: <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> On Friday, August 5, 2016 at 11:58:05 AM UTC+12, I wrote: > > Are people still using Win32? I thought Windows went 64-bit years ago. Here is a little titbit I was looking for: Win32 has a function for getting the size of a file. File sizes on Windows are limited to 2^64 bytes, and so they need a 64-bit integer to be expressed easily. But the API call to get the size of a file doesn't give you a 64-bit value. Instead, it gives you a pair of 32-bit values that have to be combined in a particular way. For 32-bit Windows, that's sort of understandable; 32-bit Windows is, well, 32-bit, so you might not expect to be able to use 64-bit integers. But if you use the same API in 64-bit Windows, it still gives you the pair of numbers, rather than just a nice simple 64-bit number. While this made some kind of sense on 32-bit Windows, it makes no sense at all on 64-bit Windows, since 64-bit Windows can, by definition, use 64-bit numbers. This is why it?s still called ?Win32? and not ?Win64?... From steve+python at pearwood.info Sat Aug 13 03:37:25 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 13 Aug 2016 17:37:25 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Aug 2016 09:55 pm, BartC wrote: > On 12/08/2016 12:07, Steven D'Aprano wrote: >> On Fri, 12 Aug 2016 07:38 pm, BartC wrote: >> >>> 'year' has been spelled wrongly >> >> How do you know? > > I know because my intention was to create a RECORD, with a specific set > of members or fields. With records, you usually don't just create > arbitrary named members as you go along. Unfortunately your intention counts for nothing. You know what they say about computers: the damned things don't do what you want, only what you tell them to do. The equivalent of a record in Python is not an arbitrary object, but a tuple, or perhaps a namedtuple if you want to access fields by name. >> There are broadly two tactics that a language can take when it comes to >> attribute (or more widely, variable) names: >> >> >> (1) Editing is cheap, but compiling and testing code is expensive > > No, compiling is cheap too. Perhaps now, but not always. And not even always now. >> The first time I ever compiled a full-sized application (not a particular >> large one either, it was a text editor a little more featureful than >> Notepad) it took something like nine hours to compile on a Mac SE (this >> was circa 1990). > > That is completely crazy even for 1990. I was using my own tools then > and the time to re-compile a 50,000-line application would have been > measured in seconds. Certainly not minutes anyway (I wouldn't have had > the patience!). Long compile times were certainly not unusual in the 1980s and even into the 1990s. Using the most powerful machines available, people would regularly set up compile jobs to run overnight. The Macintosh SE was built on a Motorola 68000 CPU with a clock speed of 7.8 MHz. (That was a little slower than equivalent Intel processors at the time, but the Motorola CPUs had bigger pipelines or more registers or something, I forget exactly what, that meant that they were actually a tad faster than Intel despite the lower clock speed.) Let's compare with something more recent: http://www.openoffice.org/FAQs/build_faq.html#howlong Building Open Office on a 1.8GHz Pentium takes about 4 hours, or 1.44e+13 nanoseconds (the reciprocal unit of GHz). Now, I know, and you know, that you can't just compare clock speeds across entirely different CPUs, but I'm going to do it anyway. The Pentium was something like 230 times faster than the Motorola 68000 in my Mac SE, which would suggest that Open Office would have taken over 900 hours (five weeks plus change) to build on my Mac SE, if it had existed (it didn't) and if it could have fit on the 20MB hard drive I had at the time (just the Open Office source code is 400MB, so it wouldn't have). Of course Open Office is a little bigger and more complex than that old text editor: 30K files versus a dozen or so, nine million LOC versus, oh I don't know, let's say 50 KLOC. That gives a back-of-the-envelope figure of about 180 times "bigger", based on LOC. 900 hours divide by 180 gives five hours, within the ballpark of my memory. By memory, my Mac SE had all of 1MB of RAM, although unlike earlier Macs at least it had dedicated VRAM for the display. You know how software can trade off space for time? Yeah, well when you've got less than 1MB available, you're often trading off time for space. So I completely reject your assertion that compilation is and always has been cheap. Overnight builds were common, and I'm not being unreasonable to talk about a nine-hour compilation on an underpowered entry-level machine back in 1990 or thereabouts. [...] > My example was specifically about attribute names where pre-declaring > the set of allowed attributes would not be onerous, Others disagree: millions of people use languages which eschew mandatory variable declarations (Ruby, Lua, Python, Javascript, Perl, PHP, etc). > would be usefully > self-documenting, and would allow more errors to be picked up. Not really - declarations simply switch *when* the error is noticed, not whether or not it is. > (Of course the design of Python makes that impractical because it would > require the byte-code compiler to see inside imported modules before > execution is commenced.) That's not the fundamental problem. Most of the time, modules are available compiled to byte-code (.pyc files), so Python could compile a list of variable names used. CPython does something similar for functions: each function has a table of variables, taken from a quick two-pass compilation process. But the fundamental problem is that Python has an exec command. Ultimately, the only way to tell what variables actually exist is to execute the code. In Python 2, CPython worked around exec inside functions with difficulty. In Python 3, exec inside a function is simply prohibited unless you supply a separate local namespace. Now, a linter, editor or other external tool is perfectly capable of using heuristics to recognise what is *likely* to be a variable. It doesn't matter if your IDE's code completion fails to work here: exec("value = 1") x = val[press tab for code completion] but it would be completely unacceptable for the compiler to flag this as an error: exec("value = 1") x = value + 1 ^ SyntaxError: no variable called 'value' Its okay for the linter to get it wrong. Its not okay for the compiler to get it wrong and refuse to compile legal code. >> Turn it around though: one of the reasons why languages like Ruby, >> Javascript, Lua, Python, Perl etc are so ridiculously productive is that >> you don't have to keep stopping to declare variables just to satisfy the >> compiler. Need a new variable? Just assign to it and keep going -- >> there's no, or very little, mental task switching. >> >> Another reason why such languages are so productive is the availability >> of an interactive REPL where you can try out code interactively. How >> clumsy and awkward the REPL would be if you had to keep declaring >> variables ahead of time. > > I agree when it comes to variables. But it /does/ allow these extra > errors to creep in that are not detected until that particular fragment > of code is executed. Sure, there's a trade-off. With a good editor that has some sort of code completion, you might make a typo one time in a thousand. Without code completion, maybe one time in a hundred. So variable/attribute declarations save you from an error that occurs perhaps one time in a hundred times that you refer to a name or attribute: 99 times in a hundred, it's just a nuisance. That's the value-judgement of the communities of programmers who have created and use languages like Ruby and Python. Obviously those who create and use languages like C and Java and Pascal think differently, and those using languages like Haskell with type and variable inference try to find a middle-ground. The programming community as a whole is slowly converging towards that middle ground: let your tools (compiler or IDE or linter, it doesn't matter) infer as much information as possible, and require as few explicit declarations as necessary. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 13 03:43:25 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sat, 13 Aug 2016 17:43:25 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> Message-ID: <57aecf9e$0$1588$c3e8da3$5496439d@news.astraweb.com> On Fri, 12 Aug 2016 11:30 pm, Michael Torrie wrote: > On 08/12/2016 05:07 AM, Steven D'Aprano wrote: >> The first time I ever compiled a full-sized application (not a particular >> large one either, it was a text editor a little more featureful than >> Notepad) it took something like nine hours to compile on a Mac SE (this >> was circa 1990). How mad would I have been if, eight hours and thirty >> minutes into the compilation, the compiler suddenly stopped with an error >> caused by a mistyped variable name? > > Surely this application was built using a build system, even back then, > that would allow compilation to resume and not rebuild object files that > were already built. To be honest, I don't remember. It was so-called "Lightspeed Pascal", later renamed "Think Pascal", and especially after that experience I mostly used it for experimenting with Hypertalk XCMDs and XFNCs, which were much simpler as they weren't stand-alone GUI applications. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From marko at pacujo.net Sat Aug 13 04:25:50 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 13 Aug 2016 11:25:50 +0300 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> <57ae7108$0$1586$c3e8da3$5496439d@news.astraweb.com> <87oa4xxxmn.fsf@jester.gateway.pace.com> Message-ID: <87mvkhdqi9.fsf@elektro.pacujo.net> Paul Rubin : > Keep in mind that the above basically takes the task off the list of > runnables for 0.2 seconds, so it sits doing nothing and doesn't > interfere with other tasks running. In the case where you're doing > actual computation, you want to yield (await) much more often than 0.2 > sec, so other tasks can stay responsive. This is cooperative > multitasking, where each task has to be nice to the others, in > contrast with preemptive multitasking where the task switches happen > behind the scenes. I don't suppose the Python database API has yet been ported to asyncio, has it? That means database access is not yet quite usable with asyncio (or any other form of asynchronous programming). Also, one must be careful with file access, which is necessarily blocking on linux (unless Python takes Linux's AIO API into use, which would be groundbreaking). Marko From asadurrehman319 at gmail.com Sat Aug 13 05:05:48 2016 From: asadurrehman319 at gmail.com (Asad ur Rehman) Date: Sat, 13 Aug 2016 02:05:48 -0700 (PDT) Subject: How to add data in xlm file ? Message-ID: <8dfbcce7-69fb-49b0-9e1e-25dae2808d0c@googlegroups.com> def Test(request): save_path = '/usr/share/newfies/' name_of_file = ("Avatar") completeName = os.path.join(save_path, name_of_file+".xlm") file1 = open(completeName, "w") toFile = raw_input("Write what you want into the field") file1.write(toFile) file1.close() This is code for save file in specific directory.Output of this file is empty i want to get data from database i write database query in it but it did't work.Example\ def Test(request): save_path = '/usr/share/newfies/' name_of_file = ("Avatar") completeName = os.path.join(save_path, name_of_file+".xlm") file1 = open(completeName, "w") toFile = Subscriber.objects.all().count() file1.write(toFile) file1.close() Subscriber is table name in my database. How can i get data ? Thanks in advance ! From bc at freeuk.com Sat Aug 13 06:09:27 2016 From: bc at freeuk.com (BartC) Date: Sat, 13 Aug 2016 11:09:27 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 13/08/2016 08:37, Steven D'Aprano wrote: > On Fri, 12 Aug 2016 09:55 pm, BartC wrote: >> I know because my intention was to create a RECORD, with a specific set >> of members or fields. With records, you usually don't just create >> arbitrary named members as you go along. > > Unfortunately your intention counts for nothing. You know what they say > about computers: the damned things don't do what you want, only what you > tell them to do. > > The equivalent of a record in Python is not an arbitrary object, but a > tuple, or perhaps a namedtuple if you want to access fields by name. OK, yet another thing I wasn't aware of (Python seems to be bristling with these things!). But when I tried it, it seems even namedtuples are immutable, so in most instances they can't do the same job. The __slots__ thing, while crude, seems to be a better bet. (The way I deal with records elsewhere is so ridiculously simple that it almost makes me want to cry when I see what most dynamic languages have to do. Example: record date= # define the record var day, month, year end d := date(25,12,2015) # create an instance d.year := 1999 # change a field println d # show "(25,12,1999)" Typing .yaer by mistake is usually picked up at compile-time - unless there is an actual field 'yaer' define elsewhere and it's in scope. Then it's detected at runtime like using __slots__.) >>> (1) Editing is cheap, but compiling and testing code is expensive >> >> No, compiling is cheap too. > > Perhaps now, but not always. And not even always now. > The Macintosh SE was built on a Motorola 68000 CPU with a clock speed of 7.8 > MHz. > Of course Open Office is a little bigger and more complex than that old text > editor: 30K files versus a dozen or so, nine million LOC versus, Ridiculously bloated project. And an average of 300 lines per file? Why not just one function per file then. But even given that, I can't see that a 9Mloc project, as a monolithic file, would take (today) more than a minute or so with a snappy compiler. What kills it is probably two things: being fragmented among 30,000 files; and, assuming this is C code, those things called 'headers' where you have to repeatedly compile the same junk over and over again (perhaps 30,000 times in this case). A static size of 9Mloc could mean a /considerably/ higher line-count needing to be compiled when headers are taken into account. So a poor language, poor project structure, and probably slow compilers all contributing. > By memory, my Mac SE had all of 1MB of RAM, although unlike earlier Macs at > least it had dedicated VRAM for the display. You know how software can > trade off space for time? Yeah, well when you've got less than 1MB > available, you're often trading off time for space. The first working compiler I did was for an 8-bit 4-MHz Z80 with 64KB memory. It was designed to give instant results (commercial compilers such as for C I understood spent most of their time grinding floppy disks). The programs were small, but there was anyway an upper limit as to how big they could be as they had to fit into a small memory. I don't remember /ever/ having to wait unduly for any re-compile even as machines and projects got bigger. I saw it as part of my job to continually ensure my tools were fit for purpose. Eventually I introduced dynamic, scripting languages into my apps to keep the edit-compile-run cycle close minimal most of the time. > So I completely reject your assertion that compilation is and always has > been cheap. You're right in that using conventional tools it probably wasn't. It was for me however. >> (Of course the design of Python makes that impractical because it would >> require the byte-code compiler to see inside imported modules before >> execution is commenced.) > But the fundamental problem is that Python has an exec command. Also that 'import' statements can be conditional. And could have intervening code between them that redefines names from an earlier import. > Now, a linter, editor or other external tool is perfectly capable of using > heuristics to recognise what is *likely* to be a variable. It doesn't > matter if your IDE's code completion fails to work here: > > exec("value = 1") > x = val[press tab for code completion] > > but it would be completely unacceptable for the compiler to flag this as an > error: > > exec("value = 1") > x = value + 1 > ^ > SyntaxError: no variable called 'value' I don't know how exec works. Is the 'value' name a local if this is in a function, or a global if outside a function? If your example was in a function, wouldn't 'value' be assumed to be a global anyway, if it wasn't assigned to anywhere else in the function? I can't see an issue with names inside exec() that are designed to be used outside, required to be 'declared' outside the exec. That is, created as they normally are in Python, by assignment. But then I expect that any Python code, inside exec or not, could also deliberately mess with dictionaries and things to create and remove variable names at will. The language doesn't make it easy for itself! -- Bartc From eryksun at gmail.com Sat Aug 13 06:48:08 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 13 Aug 2016 10:48:08 +0000 Subject: Win32 API in pywin32 In-Reply-To: <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> Message-ID: On Sat, Aug 13, 2016 at 4:22 AM, Lawrence D?Oliveiro wrote: > On Friday, August 5, 2016 at 11:58:05 AM UTC+12, I wrote: >> >> Are people still using Win32? I thought Windows went 64-bit years ago. > > Here is a little titbit I was looking for: > > Win32 has a function for getting the size of a file. File sizes on > Windows are limited to 2^64 bytes, and so they need a 64-bit integer to > be expressed easily. But the API call to get the size of a file doesn't > give you a 64-bit value. Instead, it gives you a pair of 32-bit values > that have to be combined in a particular way. For 32-bit Windows, that's > sort of understandable; 32-bit Windows is, well, 32-bit, so you might not > expect to be able to use 64-bit integers. But if you use the same API in > 64-bit Windows, it still gives you the pair of numbers, rather than just > a nice simple 64-bit number. While this made some kind of sense on 32-bit > Windows, it makes no sense at all on 64-bit Windows, since 64-bit Windows > can, by definition, use 64-bit numbers. > > This is why it?s still called ?Win32? and not ?Win64?... You can call GetFileSizeEx [1]. Or call GetFileInformationByHandleEx [2] to get the FileStandardInfo. These APIs use the LARGE_INTEGER union type, which has a long long "QuadPart" member. Pointer casting between LARGE_INTEGER and long long (or int64_t from stdint.h) poses no alignment problem. That's why ctypes can define LARGE_INTEGER simply as a c_longlong: >>> wintypes.LARGE_INTEGER In contrast, FILETIME isn't a union, so ctypes has to define the low and high parts separately to get the correct 4-byte alignment: >>> wintypes.FILETIME._fields_ [('dwLowDateTime', ), ('dwHighDateTime', )] >>> ctypes.alignment(wintypes.FILETIME) 4 FILETIME is a legacy of the Windows API. The kernel and user-mode runtime library (i.e. Rtl* in ntdll) use LARGE_INTEGER. Note that all of the info classes for GetFileInformationByHandleEx also use LARGE_INTEGER because they're basically a thin layer over the NT API. For example, compare WinAPI FileFullDirectoryInfo [3] with NT API FileFullDirectoryInformation [4]. They're identical. Using the WinAPI wrapper is just missing some features of the native NtQueryDirectoryFile system call, such as the ability to use an event or APC routine to operate asynchronously and also the FileName parameter that supports wildcards (NT style [5], not DOS). [1]: https://msdn.microsoft.com/en-us/library/aa364957 [2]: https://msdn.microsoft.com/en-us/library/aa364953 [3]: https://msdn.microsoft.com/en-us/library/hh447298 [4]: https://msdn.microsoft.com/en-us/library/ff540289 [5]: https://msdn.microsoft.com/en-us/library/ff546850 From gengyangcai at gmail.com Sat Aug 13 06:52:19 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 13 Aug 2016 03:52:19 -0700 (PDT) Subject: Python Indentation Message-ID: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> Indentation matters. Each line under the if statement that is indented will only be executed if the statement is true: if a == 1: print("If a is one, this will print.") print("So will this.") print("And this.") print("This will always print because it is not indented.") Indentation must be the same. This code doesn't work. if a == 1: print("Indented two spaces.") print("Indented four. This will generate an error.") print("The computer will want you to make up your mind.") Why does the first example work and the 2nd example doesn't work ? Can someone explain in layman's terms what indentation is and why the first one works and the 2nd one doesn't ? Thanks alot Gengyang From rosuav at gmail.com Sat Aug 13 07:10:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 13 Aug 2016 21:10:32 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Aug 13, 2016 at 8:09 PM, BartC wrote: > The language doesn't make it easy for itself! No, it doesn't. But it does make things easy for *me*, the programmer. And yaknow what? If I want a language that makes things easy on *itself*, I can always go write assembly language code. That's not what I want, usually. It's definitely not what I want for efficient, productive code on a project with more than one programmer. At the very highest level of abstraction are human thoughts. At the lowest, we have actual CPU-level opcodes and whatnot. In between, we have layer upon layer of execution systems, designed to spare us humans the hassle of translating our thoughts all the way down. When you write code, you're reducing your thoughts to a more concrete level; and you can pick any of quite a number of actual abstraction levels. Programmers are generally more productive when they get to write more abstract code, rather than mess around with the lower, more concrete, levels - maybe it doesn't seem like much difference on initial coding, but it's HUGELY beneficial when you edit afterwards. In fact, the correct way to edit code is to go right back to the highest level of abstraction - the human thought - and either correct an error in translation (ie "this is what was intended, so make the code do that" - debugging, at its most fundamental) or change the thought and retranslate. Keeping the code closer to that thought is critical to that. Python is already at a fairly high level of abstraction. That's part of what makes it so good. Another important part, though, is its incredible dynamism, which allows you to *create new levels* above what the language itself offers. Consider this code, for instance: class Invoice(Base): __tablename__ = "invoices" id = Column(Integer, primary_key=True) customer_id = Column(Integer) date = Column(Date, nullable=False) updated = Column(Boolean, nullable=False, default=False) It's a declarative way to create a database table. It's pretty readable. It's fully executable, thanks to some magic in the "Base" class (itself made by SQLAlchemy's declarative_base() factory, if you want to look up the docs and find something that I got wrong here). But in concrete terms, all it's doing is subclassing a class and creating some class-level attributes. If Python's class system were a simple and rigid thing like you have in C++, sure, maybe some simple jobs would be a bit easier - but this kind of thimg would be *impossible*. You can't use a C++ class to represent something like this, simply by subclassing the right thing. And if subclassing isn't enough, there's no end of stuff you can do with decorators. Try this: class Foo: @prop class demo: """Declarative property""" def get(self): print("Getting %s.demo" % self) return 42 def set(self, val): print("Setting %s.demo" % self) def delete(self): print("Deleting %s.demo" % self) Yaknow, because @property is just waaaaaaay too clunky, right? :) But I needed a simple demo. The decorator isn't even all that complicated. def prop(cls): return property(cls.get, getattr(cls, "set"), getattr(cls, "delete"), cls.__doc__) Now, try to do this kind of thing in a strict language. Use a class to create something that isn't a class. Slap metaprogramming into anything you want. Make executable DSLs that seamlessly integrate with the rest of the code. It might be possible, but it sure won't be this easy! ChrisA From marko at pacujo.net Sat Aug 13 07:12:10 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 13 Aug 2016 14:12:10 +0300 Subject: Python Indentation References: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> Message-ID: <87eg5sexdh.fsf@elektro.pacujo.net> Cai Gengyang : > Can someone explain in layman's terms what indentation is Indentation means the number space (" ") characters in the beginning of a line. A sequence of lines that have the same indentation belong to the same block of statements. Marko From rosuav at gmail.com Sat Aug 13 07:16:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 13 Aug 2016 21:16:52 +1000 Subject: Python Indentation In-Reply-To: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> References: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> Message-ID: On Sat, Aug 13, 2016 at 8:52 PM, Cai Gengyang wrote: > Indentation must be the same. This code doesn't work. > > if a == 1: > print("Indented two spaces.") > print("Indented four. This will generate an error.") > print("The computer will want you to make up your mind.") > > Why does the first example work and the 2nd example doesn't work ? Can someone explain in layman's terms what indentation is and why the first one works and the 2nd one doesn't ? Thanks alot Indentation is whitespace at the beginning of the line. And the reason this code doesn't work is that more indentation means more levels in, like this: # Top level def frobnicate(stuff): # Inside the function for thing in stuff: # Inside the loop if thing.frobbable: # Inside the conditional thing.frob() # Where is this? return "All frobbed" A casual eye will tell you that the return statement is outside the loop, outside the condition, but inside the function. And that's exactly how Python interprets it, too - but to do that, it absolutely depends on the indentation levels being consistent. Normally you'll be extremely consistent (eg four spaces for the first level, eight for the second, then twelve, sixteen, etc); all Python actually demands is that you use the same number of spaces and/or tabs (and they're not equivalent - one tab matches one tab, one space matches one space) for the same level of nesting. My recommendation to you is that you pick some way of indenting and use it everywhere. The most common would be either one tab per level, or four spaces per level. (Also, do not get into arguments about which one is better. They're not productive.) Python makes things fairly easy on you. If you end a line with a colon, you indent the next line. If you don't, you don't. Good rule of thumb :) ChrisA From greg.ewing at canterbury.ac.nz Sat Aug 13 07:43:01 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 13 Aug 2016 23:43:01 +1200 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57aecf9e$0$1588$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> <57aecf9e$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > To be honest, I don't remember. It was so-called "Lightspeed Pascal", later > renamed "Think Pascal", and especially after that experience I mostly used > it for experimenting with Hypertalk XCMDs and XFNCs, which were much > simpler as they weren't stand-alone GUI applications. I did some pretty extensive work with Think Pascal on a 1MB Mac Plus with two floppy drives (no HD) and I always found it to be very efficient and a pleasure to work with. Nine hours to compile a simple text editor sounds completely ridiculous to me, even for the times. Something must have been very, very badly wrong. -- Greg From bc at freeuk.com Sat Aug 13 08:08:22 2016 From: bc at freeuk.com (BartC) Date: Sat, 13 Aug 2016 13:08:22 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 13/08/2016 12:10, Chris Angelico wrote: > On Sat, Aug 13, 2016 at 8:09 PM, BartC wrote: > And if subclassing isn't enough, there's no end of stuff you can do > with decorators. Try this: > > class Foo: > @prop > class demo: > """Declarative property""" > def get(self): > print("Getting %s.demo" % self) > return 42 > def set(self, val): > print("Setting %s.demo" % self) > def delete(self): > print("Deleting %s.demo" % self) > > Yaknow, because @property is just waaaaaaay too clunky, right? :) But > I needed a simple demo. The decorator isn't even all that complicated. > > def prop(cls): > return property(cls.get, getattr(cls, "set"), getattr(cls, > "delete"), cls.__doc__) Which level is this nearer to, human thoughts or cpu codes? My knowledge of Python isn't advanced enough to understand what this does. But if you would be good enough to explain or to show the end results (what it makes possible) then I could get an opinion as to how hard it might be with less dynamic features. (But don't try and explain decorators. I've tried several times to get my head around them.) -- bartc From marko at pacujo.net Sat Aug 13 08:08:59 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 13 Aug 2016 15:08:59 +0300 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <4ee8d16c-bbda-5d15-6e05-312f0c9ef833@gmail.com> <57aecf9e$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: <877fbkeuqs.fsf@elektro.pacujo.net> Gregory Ewing : > Nine hours to compile a simple text editor sounds completely > ridiculous to me, even for the times. Something must have been very, > very badly wrong. Some ten years back full compilations of our product could take 23 hours. The bulk of the time was spent thrashing, as SCons produced a dependency graph so large (~ 90,000 nodes) the Python process couldn't fit in the RAM of the build machine. When we split the build by the target architecture, build times came down to three hours. Still, the main problem was that the whole code base was in a single repository. By the time I left, work was under way to divide the code into independent components. A similar componentization strategy in my current job has made it so that a full build takes less than a minute, and an incremental build a few seconds. Marko From bc at freeuk.com Sat Aug 13 09:45:00 2016 From: bc at freeuk.com (BartC) Date: Sat, 13 Aug 2016 14:45:00 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 13/08/2016 13:08, BartC wrote: > On 13/08/2016 12:10, Chris Angelico wrote: >> On Sat, Aug 13, 2016 at 8:09 PM, BartC wrote: > >> And if subclassing isn't enough, there's no end of stuff you can do >> with decorators. Try this: >> >> class Foo: >> @prop >> class demo: >> """Declarative property""" >> def get(self): >> print("Getting %s.demo" % self) >> return 42 >> def set(self, val): >> print("Setting %s.demo" % self) >> def delete(self): >> print("Deleting %s.demo" % self) >> >> Yaknow, because @property is just waaaaaaay too clunky, right? :) But >> I needed a simple demo. The decorator isn't even all that complicated. >> >> def prop(cls): >> return property(cls.get, getattr(cls, "set"), getattr(cls, >> "delete"), cls.__doc__) > (But don't try and explain decorators. I've tried several times to get > my head around them.) This is a clearer than usual explanation: http://thecodeship.com/patterns/guide-to-python-function-decorators/ Not quite simple enough however to figure out how decorators benefit your example! (Maybe an example with and without decorators?) So if the aim is to keep code readable and understandable to as wide an audience as possible, 'decorators' don't really work. (Perhaps if they are buried somewhere else and only an interface is visible, but then it doesn't matter so much how hairy that buried code is.) -- Bartc From no.email at nospam.invalid Sat Aug 13 10:22:37 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 13 Aug 2016 07:22:37 -0700 Subject: Asynchronous programming References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> <57ae7108$0$1586$c3e8da3$5496439d@news.astraweb.com> <87oa4xxxmn.fsf@jester.gateway.pace.com> <87mvkhdqi9.fsf@elektro.pacujo.net> Message-ID: <874m6oycia.fsf@jester.gateway.pace.com> Marko Rauhamaa writes: > Also, one must be careful with file access, which is necessarily > blocking on linux (unless Python takes Linux's AIO API into use, which > would be groundbreaking). AIO is a possibility for i/o on an already-open file and I think there may be other ways to do it too. But there appears to be no way to open a file without potentially blocking, potentially for a long time on a busy hard disk or a remote file system. GHC and Erlang both use a thread pool to deal with opening disk files. From steve+python at pearwood.info Sat Aug 13 12:06:58 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Aug 2016 02:06:58 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> On Sat, 13 Aug 2016 08:09 pm, BartC wrote: > (The way I deal with records elsewhere is so ridiculously simple that it > almost makes me want to cry when I see what most dynamic languages have > to do. Example: > > record date= # define the record > var day, month, year > end > > d := date(25,12,2015) # create an instance > d.year := 1999 # change a field > println d # show "(25,12,1999)" Sure. But you've put most of the work into the compiler. Since you wrote the compiler, you still had to do the work. Python has gone over 20 years with no syntax or built-in function for creating "records", because they're not really needed. We have tuples, namedtuples and objects, which are much more powerful. But out of curiosity, I thought I'd write something to define a record as you would have it. A couple of tricky bits: - Since I'm not writing a compiler, I'm limited to syntax as allowed by Python. That means I have to give the record name twice: once as the binding target, and once as an argument to the "record" function. - The trickiest bit is defining the record class' __init__ method, since I don't know how many arguments it will need. Nevertheless, here you go. This was about 5 minutes work. Not too bad for a first draft. def record(name, fieldnames, verbose=False): if isinstance(fieldnames, str): fieldnames = fieldnames.split() class Inner(object): __slots__ = fieldnames def __str__(self): fields = ', '.join([repr(obj) for obj in self._fields]) return "record %s(%s)" % (type(self).__name__, fields) __repr__ = __str__ @property def _fields(self): return [getattr(self, name) for name in fieldnames] Inner.__name__ = name ns = {} template = "def __init__(self, %s):\n" % ', '.join(fieldnames) for name in fieldnames: template += ' self.%s = %s\n' % (name, name) if verbose: print("Using __init__ template:") print(template) exec(template, ns, ns) Inner.__init__ = ns['__init__'] return Inner py> date = record('date', 'year month day') py> x = date(2016, 8, 14) py> x record date(2016, 8, 14) py> x.year = 1999 py> x record date(1999, 8, 14) > I don't know how exec works. Is the 'value' name a local if this is in a > function, or a global if outside a function? exec takes a string, and executes it as Python code. You can specify the name spaces to use for globals and locals; by default they use globals() and locals(). Exec of a string on its own is like executing code at the top level of the module: exec("spam = 1") is like: spam = 1 in the module scope. You can specify another namespace: import math exec("spam = 1", vars(math)) is like executing "spam = 1" in the top level of the math module. ns = {} exec("spam = cos(pi)", vars(math), ns) print(ns['spam']) will print -1.0. It is equivalent to executing the line "spam = cos(pi)" inside an imaginary function running inside the math module. That imaginary function's local variables (in this case, there's only one, "spam") is recorded in the dictionary "ns" for later retrieval. > If your example was in a function, wouldn't 'value' be assumed to be a > global anyway, if it wasn't assigned to anywhere else in the function? Maybe. The rules for running exec inside a function in Python 2 are awfully complex, as it interact with the CPython optimization of using a table for locals instead of a dict. In Python 3, you cannot call exec() inside a function unless you explicitly specify a namespace to use. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From atri.mahapatra at gmail.com Sat Aug 13 12:44:14 2016 From: atri.mahapatra at gmail.com (Atri Mahapatra) Date: Sat, 13 Aug 2016 09:44:14 -0700 (PDT) Subject: Creating dictionary of items from Excel with mutliple keys Message-ID: I am trying to create a following dictionary. I am reading data from excel which has data in the following format: Sl no: Name Thickness Length Material Width Quantity Side It has 20 rows of data. The dictionary for the 20 rows, I would like to make is Data_Dict = [ { 'Name': 'X', 'Length': '10' , 'Width': '5', 'Quantity': 2 'Area': 50}; { 'Name': 'Y', 'Length': '20' , 'Width': 10', 'Quantity': 1 'Area': 200}; . . . . till 20 rows ]; I would like to add another key 'Area' as shown above. I used the following code(s): using openpyxl: d={} for i in range(3,sheet.max_row+1): #for j in range(3,9): #for k in range(0,5): Name = sheet.cell(row= i,column=3).value Length =sheet.cell(row =i,column=6).value Breadth= sheet.cell(row=i,column=7).value Quantity = sheet.cell (row=i,column=8).value Area = sheet.cell(row =i,column=6).value*sheet.cell(row=i,column=7).value d[Name]= Length,Breadth,Quantity,Area which gave an output like: ['X': (10, 5, 2, 50), 'Y': (20, 10, 1, 2232600), 'Z': (5, 2, 1, 10), .............] Another code using xlrd: keys = [sheet.cell(2, col_index).value for col_index in range(0,8)] print (keys) dict_list = [] d = {} for row_index in range(1, xl_sheet.nrows): for col_index in range(0,8): d = {keys[col_index]: xl_sheet.cell(row_index, col_index).value for col_index in range(0,8)} dict_list.append(d) print (dict_list) which did not have the area and neither the output was little messy. The main purpose is to sort the dictionary based on different criteria like Length or Area. I think the first one may be easier to sort. However if there are any better way to represent the dictionary and the code so that it can be sorted later based on different attributes please feel free to suggest. Thanks, Atri From gengyangcai at gmail.com Sat Aug 13 13:06:32 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 13 Aug 2016 10:06:32 -0700 (PDT) Subject: Python Indentation In-Reply-To: <87eg5sexdh.fsf@elektro.pacujo.net> References: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> <87eg5sexdh.fsf@elektro.pacujo.net> Message-ID: <56db42f3-4f9a-477c-841e-40aac30513ae@googlegroups.com> Oh ok, so for the first block , it works because the indentation is the same for all 3 "print" statements , i.e. the indentation is 4 spaces for each of the 3 print statements. if a == 1: print("If a is one, this will print.") print("So will this.") print("And this.") print("This will always print because it is not indented.") For the 2nd block of code, it doesn't work because the indentation is different, 2 spaces for the first statement, 5 for the 2nd statement and 4 for the 3rd statement. if a == 1: print("Indented two spaces.") print("Indented four. This will generate an error.") print("The computer will want you to make up your mind.") Am i correct ? On Saturday, August 13, 2016 at 7:12:19 PM UTC+8, Marko Rauhamaa wrote: > Cai Gengyang : > > > Can someone explain in layman's terms what indentation is > > Indentation means the number space (" ") characters in the beginning of a > line. > > A sequence of lines that have the same indentation belong to the same > block of statements. > > > Marko From cbogart at andrew.cmu.edu Sat Aug 13 13:15:45 2016 From: cbogart at andrew.cmu.edu (Chris Bogart) Date: Sat, 13 Aug 2016 13:15:45 -0400 Subject: Looking for info on how Python developers negotiate breaking changes in PyPI packages Message-ID: Hi, I'm looking for some help from Python developers who write PyPI packages (or open source software that depends on PyPI packages). My research group is interested in the impacts of different choices package managers make when designing their ecosystems -- e.g. Python/PyPi, Hackage and R/CRAN. We'd like to know what the implications of PyPI's design choices are on Python programmers' collaboration when making breaking changes among packages, and what you think PyPI's design is trying to accomplish. Could I ask people to take ~20 minutes of your time and fill out our survey at http://breakingapis.org/survey ? I'll report back here when we've analyzed the results (or there's a sign up link to be notified when results are out at http://breakingapis.org ). (If you don't use PyPI, but do use some other software ecosystem, like Node.js/NPM, CPAN, Erlang/Hex, or something else, we're still interested -- just make a different "software ecosystem" choice on the first page. The study is a comparison among ecosystems; PyPI is one point of comparison). Thanks, Chris Bogart Institute for Software Research Carnegie Mellon University From rosuav at gmail.com Sat Aug 13 13:39:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 03:39:46 +1000 Subject: Python Indentation In-Reply-To: <56db42f3-4f9a-477c-841e-40aac30513ae@googlegroups.com> References: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> <87eg5sexdh.fsf@elektro.pacujo.net> <56db42f3-4f9a-477c-841e-40aac30513ae@googlegroups.com> Message-ID: On Sun, Aug 14, 2016 at 3:06 AM, Cai Gengyang wrote: > Oh ok, so for the first block , it works because the indentation is the same for all 3 "print" statements , i.e. the indentation is 4 spaces for each of the 3 print statements. > > if a == 1: > print("If a is one, this will print.") > print("So will this.") > print("And this.") > > print("This will always print because it is not indented.") > > For the 2nd block of code, it doesn't work because the indentation is different, 2 spaces for the first statement, 5 for the 2nd statement and 4 for the 3rd statement. > > if a == 1: > print("Indented two spaces.") > print("Indented four. This will generate an error.") > print("The computer will want you to make up your mind.") > > Am i correct ? Yep! That's correct. ChrisA From gengyangcai at gmail.com Sat Aug 13 13:49:25 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 13 Aug 2016 10:49:25 -0700 (PDT) Subject: Python Indentation In-Reply-To: References: <79c0b88a-ceec-489b-ad67-cdfc29f8e298@googlegroups.com> <87eg5sexdh.fsf@elektro.pacujo.net> <56db42f3-4f9a-477c-841e-40aac30513ae@googlegroups.com> Message-ID: Yay ! On Sunday, August 14, 2016 at 1:40:09 AM UTC+8, Chris Angelico wrote: > On Sun, Aug 14, 2016 at 3:06 AM, Cai Gengyang wrote: > > Oh ok, so for the first block , it works because the indentation is the same for all 3 "print" statements , i.e. the indentation is 4 spaces for each of the 3 print statements. > > > > if a == 1: > > print("If a is one, this will print.") > > print("So will this.") > > print("And this.") > > > > print("This will always print because it is not indented.") > > > > For the 2nd block of code, it doesn't work because the indentation is different, 2 spaces for the first statement, 5 for the 2nd statement and 4 for the 3rd statement. > > > > if a == 1: > > print("Indented two spaces.") > > print("Indented four. This will generate an error.") > > print("The computer will want you to make up your mind.") > > > > Am i correct ? > > Yep! That's correct. > > ChrisA From auriocus at gmx.de Sat Aug 13 14:31:09 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 13 Aug 2016 20:31:09 +0200 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> Message-ID: Am 12.08.16 um 11:58 schrieb BartC: > On 12/08/2016 10:45, Chris Angelico wrote: >> On Fri, Aug 12, 2016 at 7:38 PM, BartC wrote: >>> You can be too dynamic. Take an example like this: > >>> d.yaer=1999 >>> >>> print (d.day,d.month,d.year) > >>> This would never get past a static language nor some that also have >>> dynamic >>> types. >> >> Nor will it get past a lot of linters. Or unit tests. There are other >> solutions. > > How would a linter know that you didn't /want/ to create a new attribute > called 'yaer'? It is possible to create an attribute outside of the class, but it is unusual. I tried your example using pylint. It gives me: W: 9, 0: Attribute 'yaer' defined outside __init__ (attribute-defined-outside-init) Granted, pylint also spits an enormous amount of other stylistic warnings about spaces and "wrong" indents which, strangely, have the same weight "warning" as this bug, so it's a bit hard to find. A linter from another language I use also spell-checks variable names, i.e. if it finds two variables with suspiciously similar names, it reports it. Of course the linter never knows for sure, it can only give warnings. Christian From michael.selik at gmail.com Sat Aug 13 14:43:22 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sat, 13 Aug 2016 18:43:22 +0000 Subject: Creating dictionary of items from Excel with mutliple keys In-Reply-To: References: Message-ID: On Sat, Aug 13, 2016 at 12:46 PM Atri Mahapatra wrote: > I am trying to create a following dictionary. I am reading data from excel > Rather than using xlrd or other tools to read from excel, can you save the file as CSV (comma-separated values)? I think you'll find Python's csv module is very pleasant to use. From kliateni at gmail.com Sat Aug 13 14:49:17 2016 From: kliateni at gmail.com (Karim) Date: Sat, 13 Aug 2016 20:49:17 +0200 Subject: How to add data in xlm file ? In-Reply-To: <8dfbcce7-69fb-49b0-9e1e-25dae2808d0c@googlegroups.com> References: <8dfbcce7-69fb-49b0-9e1e-25dae2808d0c@googlegroups.com> Message-ID: <06e7a279-b9aa-b361-ba1c-d5a5d50cd81f@gmail.com> On 13/08/2016 11:05, Asad ur Rehman wrote: > def Test(request): > > save_path = '/usr/share/newfies/' > > name_of_file = ("Avatar") > > completeName = os.path.join(save_path, name_of_file+".xlm") > > file1 = open(completeName, "w") > > toFile = raw_input("Write what you want into the field") > > file1.write(toFile) > > file1.close() > > This is code for save file in specific directory.Output of this file is empty i want to get data from database i write database query in it but it did't work.Example\ > > def Test(request): > > save_path = '/usr/share/newfies/' > > name_of_file = ("Avatar") > > completeName = os.path.join(save_path, name_of_file+".xlm") > > file1 = open(completeName, "w") > > toFile = Subscriber.objects.all().count() > > file1.write(toFile) > > file1.close() > > > Subscriber is table name in my database. > > How can i get data ? > Thanks in advance ! Hello, Use the standard module (I use python2.7), code not tested If my memory is good this is the general way to do it: from xml.dom import minidom from xml.etree import ElementTree as et from xml.etree.ElementTree import ElementTree, Element fname = "my_input_file" xml_data = et.parse(fname) root_element = xml_data.getroot() new_element = et.Element("new_tag") root_element.append(new_element) new_element.set('new_attribute', 'attribute value') fout = "my_ouput_file" fout.write(prettify(root_element)) Where prettify() is defined by: def prettify(element): """ Pretty print to an indented XML output. @param element a Element object. @return a pretty-printed XML string for the element. """ # The idea is to print your Element in a string, parse it using # minidom and convert it again in XML using toprettyxml function. raw_string = et.tostring(element, encoding='UTF-8', method='xml') reparsed = minidom.parseString(raw_string) return reparsed.toprettyxml(indent='\t', encoding='UTF-8') Cheers Karim From ethan at stoneleaf.us Sat Aug 13 15:29:20 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 13 Aug 2016 12:29:20 -0700 Subject: Asynchronous programming In-Reply-To: <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <87r39vzygq.fsf@jester.gateway.pace.com> <57ac8852$0$1597$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57AF7510.3040500@stoneleaf.us> On 08/11/2016 07:14 AM, Steven D'Aprano wrote: > On Thu, 11 Aug 2016 03:06 pm, Paul Rubin wrote: > >> The basic characteristic of asynchronous programming is that it involves >> changing all your usual blocking i/o calls to non-blocking ones, so your >> program can keep running as soon as your request is started. > > That's the bit that confuses me. I understand about parallel processing, but > that's not what this is about. > > Say I want to download data from a network, and it will take a long time. If > I can do the read in parallel to something else, that makes sense: > > begin downloading in another thread/process > make a coffee > process download > > > But what's the point in doing it asynchronously if I have to just wait for > it to complete? > > begin downloading in an async thread > twiddle thumbs, doing nothing > process download async, threads, multiprocessing, etc., all only make sense if you have more than one thing to do. The advantage of the new async stuff is it allows us to write in a synchronous manner (making async calls instead of blocking calls) and the underlying framework takes care of the hassle of using threads or multiprocessing or whatever. -- ~Ethan~ From bc at freeuk.com Sat Aug 13 16:59:41 2016 From: bc at freeuk.com (BartC) Date: Sat, 13 Aug 2016 21:59:41 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 13/08/2016 17:06, Steven D'Aprano wrote: > On Sat, 13 Aug 2016 08:09 pm, BartC wrote: >> record date= # define the record >> var day, month, year >> end >> >> d := date(25,12,2015) # create an instance > Sure. But you've put most of the work into the compiler. Into the language so that it's always available and consistent rather than leaving it to individuals to somehow build something on top of existing language features. > But out of > curiosity, I thought I'd write something to define a record as you would > have it. > > A couple of tricky bits: > > - Since I'm not writing a compiler, I'm limited to syntax as > allowed by Python. That means I have to give the record name > twice: once as the binding target, and once as an argument > to the "record" function. > > - The trickiest bit is defining the record class' __init__ > method, since I don't know how many arguments it will need. > > Nevertheless, here you go. This was about 5 minutes work. Not too bad for a > first draft. > > def record(name, fieldnames, verbose=False): > if isinstance(fieldnames, str): > fieldnames = fieldnames.split() > class Inner(object): > __slots__ = fieldnames > def __str__(self): > fields = ', '.join([repr(obj) for obj in self._fields]) > return "record %s(%s)" % (type(self).__name__, fields) > __repr__ = __str__ > @property > def _fields(self): > return [getattr(self, name) for name in fieldnames] > Inner.__name__ = name > ns = {} > template = "def __init__(self, %s):\n" % ', '.join(fieldnames) > for name in fieldnames: > template += ' self.%s = %s\n' % (name, name) > if verbose: > print("Using __init__ template:") > print(template) > exec(template, ns, ns) > Inner.__init__ = ns['__init__'] > return Inner > > > py> date = record('date', 'year month day') > py> x = date(2016, 8, 14) > py> x > record date(2016, 8, 14) > py> x.year = 1999 > py> x > record date(1999, 8, 14) That's pretty good actually. Although I noticed you used the __slots__ feature that you were scathing about earlier on! (Also that you've thrown in examples of using decorators, and using exec(), free of charge..) There are a number of ways this approach differs from the native record example (but I acknowledge your example was put together as a quick demo): * The fields in my record are known at compile time; using the wrong name will usually be picked immediately. As will using the wrong number of initialisers. * (My record fields have other features: aliases for any field; the ability to access by index, doing len() etc; fast conversions to and from a list; also the ability to create packed records that exactly match C structs, and so on.) * Each defined record in my scheme ('date' and so on) is a distinct type. If I use the Python record() to create two records R and S, then type(R) and type(S) seem to give the same result (both 'type'). * Record names and fields are case insensitive in my implementation. * The Python version can't take advantage of a simple and fast native implementation of records. (A simple loop creating a list of a few million date records, excluding overheads, took 12 times as long Python 3, as my language. PyPy only took twice as long, but it's likely just taking advantage of the big loop. * But the biggest problem I think, as I hinted above, is that this is not a standard part of the language. It's same thing with trying to fix C issues by writing macros; everyone will roll their own versions. Or not bother at all. -- Bartc From mok-kong.shen at t-online.de Sat Aug 13 17:03:46 2016 From: mok-kong.shen at t-online.de (Mok-Kong Shen) Date: Sat, 13 Aug 2016 23:03:46 +0200 Subject: A strange list concatenation result In-Reply-To: <57ae7314$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> <57ae7314$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 13.08.2016 um 03:08 schrieb Steven D'Aprano: > On Sat, 13 Aug 2016 06:44 am, Mok-Kong Shen wrote: > >>>>>> list2 = [1,2,3] >>>>>> list1 += [4,5,6] >>>>>> print(list1, list2) >>> [1, 2, 3, 4, 5, 6] [1, 2, 3] >>> >>> >>> Does that help? >> >> I don't yet understand why in my 2nd example list2 came out as >> [1, 2, 3] outside. > > Because you assign list2 = [1, 2, 3]. What did you expect it to be? But in my function test() there is a code line "list2=list2+[4,5,6]". Could you kindly explain why this didn't work? M. K. Shen From huey.y.jiang at gmail.com Sat Aug 13 19:30:39 2016 From: huey.y.jiang at gmail.com (huey.y.jiang at gmail.com) Date: Sat, 13 Aug 2016 16:30:39 -0700 (PDT) Subject: Why my image cannot be displayed? In-Reply-To: <9df5b562-eace-498c-87b1-972a09e5b15f@googlegroups.com> References: <9df5b562-eace-498c-87b1-972a09e5b15f@googlegroups.com> Message-ID: On Friday, August 12, 2016 at 9:40:15 PM UTC-4, huey.y... at gmail.com wrote: > Hi All, > > Image display drives me crazy. After I get it displayed, and want to do the job with a class, display failed again. Please take a look at my trial code: > > from Tkinter import * > > class imageDisplay: > def __init__(self, parent=None): > canvas = Canvas(width=400, height=300, bg='beige') > canvas.pack() > self.canvas = canvas > img = PhotoImage(file="xxx.gif") > self.canvas.create_image(150, 0, image=img, anchor=NW) > > if __name__ == '__main__': > imageDisplay() > mainloop() > > Result: A blank canvas was displayed. No error message. > > Can somebody help? Thanks so much! Thanks! it's working. Particularly thanks for your explanation! From lawrencedo99 at gmail.com Sat Aug 13 20:22:37 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 17:22:37 -0700 (PDT) Subject: Win32 API in pywin32 In-Reply-To: References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> Message-ID: <1386d3b4-0af3-43ec-a5fd-008a36cbd933@googlegroups.com> On Saturday, August 13, 2016 at 10:49:11 PM UTC+12, eryk sun wrote: > You can call GetFileSizeEx [1]. Or call GetFileInformationByHandleEx > [2] to get the FileStandardInfo. These APIs use the LARGE_INTEGER > union type, which has a long long "QuadPart" member. > ... > FILETIME is a legacy of the Windows API. The kernel and user-mode > runtime library (i.e. Rtl* in ntdll) use LARGE_INTEGER. Note that all > of the info classes for GetFileInformationByHandleEx also use > LARGE_INTEGER because they're basically a thin layer over the NT API. > ... > [etc etc] What an absolute mess, compared to the simplicity of POSIX/Linux . Is it any wonder that Microsoft is now adding a Linux API layer to Windows? From lawrencedo99 at gmail.com Sat Aug 13 20:38:11 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 17:38:11 -0700 (PDT) Subject: Multi-Line Strings: A Modest Proposal Message-ID: <3d212cf3-0ea3-47be-b112-f648c8ef2898@googlegroups.com> Python allows a single string literal to cross multiple lines, provided it begins and ends with three quote characters, e.g. s = """this string continues on the next line.""" There is a drawback with this: any whitespace at the start of the continuation line is included as part of the string: >>> print(s) this string continues on the next line. So really, you should write it more like s = """this string continues on the next line.""" which gets a bit ugly. Python has quite a different convention for its compound statements, which can (and usually do) continue across multiple lines: and that is the use of indentation to denote nesting. So I propose adapting this convention to triply-quoted strings, as follows: lines where the string literal continues must begin with the *same* initial whitespace as the line where the string started. This initial whitespace is stripped off before including the rest as part of the string. Any additional whitespace after that at the start of the line becomes part of the string. So the first example would now print as >>> print(s) this string continues on the next line. which looks more like what was intended. From lawrencedo99 at gmail.com Sat Aug 13 20:39:22 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 17:39:22 -0700 (PDT) Subject: Creating dictionary of items from Excel with mutliple keys In-Reply-To: References: Message-ID: <481b05bf-09a5-41ca-b926-bb638eaf1336@googlegroups.com> On Sunday, August 14, 2016 at 6:43:54 AM UTC+12, Michael Selik wrote: > Rather than using xlrd or other tools to read from excel, can you save the > file as CSV (comma-separated values)? I think you'll find Python's csv > module is very pleasant to use. Trying to use Python as a substitute for VBA is not a recipe for happiness. From lawrencedo99 at gmail.com Sat Aug 13 20:43:43 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 17:43:43 -0700 (PDT) Subject: Asynchronous programming In-Reply-To: <87mvkhdqi9.fsf@elektro.pacujo.net> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> <57ae7108$0$1586$c3e8da3$5496439d@news.astraweb.com> <87oa4xxxmn.fsf@jester.gateway.pace.com> <87mvkhdqi9.fsf@elektro.pacujo.net> Message-ID: On Saturday, August 13, 2016 at 8:26:05 PM UTC+12, Marko Rauhamaa wrote: > ... (unless Python takes Linux's AIO API into use, which > would be groundbreaking). Last I looked, asynchronous I/O on Linux was still a work in progress. The whole POSIX API has been grounded in the assumption that all I/O is blocking--in other words, I/O requests and process scheduling have been inextricably linked. And that has been incorporated into the design of Linux, which they now have to disentangle. Before I was introduced to Unix/Linux, I was using VMS, where all I/O was inherently asynchronous and decoupled from process scheduling: all the synchronous I/O calls were just userland wrappers that did ?queue I/O request? followed by ?wait for completion?. From ben+python at benfinney.id.au Sat Aug 13 20:49:56 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 14 Aug 2016 10:49:56 +1000 Subject: Multi-Line Strings: A Modest Proposal References: <3d212cf3-0ea3-47be-b112-f648c8ef2898@googlegroups.com> Message-ID: <85k2fkyy17.fsf@benfinney.id.au> Lawrence D?Oliveiro writes: > So really, you should write it more like > > s = """this string continues > on the next line.""" > > which gets a bit ugly. For this I have the following treatment : import textwrap s = textwrap.dedent("""\ this string continues on the next line. """) > So I propose adapting this convention to triply-quoted strings, as > follows: lines where the string literal continues must begin with the > *same* initial whitespace as the line where the string started. This > initial whitespace is stripped off before including the rest as part > of the string. Fortunately, the time machine has provided that in the standard library so you can choose to use it, even without waiting for a convention :-) -- \ ?Prediction is very difficult, especially of the future.? | `\ ?Niels Bohr | _o__) | Ben Finney From ethan at stoneleaf.us Sat Aug 13 21:20:47 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 13 Aug 2016 18:20:47 -0700 Subject: Asynchronous programming In-Reply-To: <87shua1qsw.fsf@rudin.co.uk> References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> <57ad1363$0$1611$c3e8da3$5496439d@news.astraweb.com> <87shua1qsw.fsf@rudin.co.uk> Message-ID: <57AFC76F.7080006@stoneleaf.us> On 08/11/2016 10:47 PM, Paul Rudin wrote: > Steven D'Aprano writes: [...] >> In this case, all the work is pure computation, so I don't expect to save >> any time by doing this, because the work is still all being done in the >> same process/thread, not in parallel. It may even take a little bit longer, >> due to the overhead of switching from one to another. > > Yeah - you're not gaining anything here. All (?) the interesting use > cases involve waiting for something (e.g. disk access, network > communication, separate process) before you can proceed with your > computation. The obvious exception being a UI remaining responsive to the user while the calculation is processing. -- ~Ethan~ From torriem at gmail.com Sat Aug 13 21:39:22 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 13 Aug 2016 19:39:22 -0600 Subject: Win32 API in pywin32 In-Reply-To: <1386d3b4-0af3-43ec-a5fd-008a36cbd933@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> <1386d3b4-0af3-43ec-a5fd-008a36cbd933@googlegroups.com> Message-ID: On 08/13/2016 06:22 PM, Lawrence D?Oliveiro wrote: > On Saturday, August 13, 2016 at 10:49:11 PM UTC+12, eryk sun wrote: >> You can call GetFileSizeEx [1]. Or call GetFileInformationByHandleEx >> [2] to get the FileStandardInfo. These APIs use the LARGE_INTEGER >> union type, which has a long long "QuadPart" member. >> ... >> FILETIME is a legacy of the Windows API. The kernel and user-mode >> runtime library (i.e. Rtl* in ntdll) use LARGE_INTEGER. Note that all >> of the info classes for GetFileInformationByHandleEx also use >> LARGE_INTEGER because they're basically a thin layer over the NT API. >> ... >> [etc etc] > > What an absolute mess, compared to the simplicity of POSIX/Linux . POSIX has its share of cruft and redundant interfaces too, I'm sure. While some parts of the Linux API are simple, others definitely are not, and Linux has its share of legacy stuff (can you say X11?). And for user-space things like sound and graphics, APIs are a real hodge-podge and jumble on Linux and modern Unix operating systems. > Is it any wonder that Microsoft is now adding a Linux API layer to Windows? It's certainly not to someday completely replace the native windows APIs, I can assure you. From torriem at gmail.com Sat Aug 13 21:48:53 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 13 Aug 2016 19:48:53 -0600 Subject: Win32 API in pywin32 In-Reply-To: <1386d3b4-0af3-43ec-a5fd-008a36cbd933@googlegroups.com> References: <133f9642-eaa9-4b01-af3d-3e9f1ac5494b@googlegroups.com> <31eb5e81-1379-4126-aa32-c0bf19185e7b@googlegroups.com> <1386d3b4-0af3-43ec-a5fd-008a36cbd933@googlegroups.com> Message-ID: <461a1746-ed7b-6dbe-e26f-cc9c06c50596@gmail.com> On 08/13/2016 06:22 PM, Lawrence D?Oliveiro wrote: > On Saturday, August 13, 2016 at 10:49:11 PM UTC+12, eryk sun wrote: >> You can call GetFileSizeEx [1]. Or call GetFileInformationByHandleEx >> [2] to get the FileStandardInfo. These APIs use the LARGE_INTEGER >> union type, which has a long long "QuadPart" member. >> ... >> FILETIME is a legacy of the Windows API. The kernel and user-mode >> runtime library (i.e. Rtl* in ntdll) use LARGE_INTEGER. Note that all >> of the info classes for GetFileInformationByHandleEx also use >> LARGE_INTEGER because they're basically a thin layer over the NT API. >> ... >> [etc etc] > > What an absolute mess, compared to the simplicity of POSIX/Linux . > > Is it any wonder that Microsoft is now adding a Linux API layer to Windows? Oops had to read the link first. I get it now! From gokoproject at gmail.com Sat Aug 13 21:53:20 2016 From: gokoproject at gmail.com (John Wong) Date: Sat, 13 Aug 2016 21:53:20 -0400 Subject: Multi-Line Strings: A Modest Proposal In-Reply-To: <85k2fkyy17.fsf@benfinney.id.au> References: <3d212cf3-0ea3-47be-b112-f648c8ef2898@googlegroups.com> <85k2fkyy17.fsf@benfinney.id.au> Message-ID: On Sat, Aug 13, 2016 at 8:38 PM, Lawrence D?Oliveiro wrote: > Python allows a single string literal to cross multiple lines, provided it > begins and ends with three quote characters, e.g. > > s = """this string continues > on the next line.""" > > There is a drawback with this: any whitespace at the start of the > continuation line is included as part of the string: > > >>> print(s) > this string continues > on the next line. > > I think you meant both trailing characters and starting characters. The way I solve it, and I still find that extremely ugly, is s = ("this string continues " + "substring continues") The blackslash is a method I used to do but I start regretting. I have ran into issue with blackslash before, can't remember but, but most likely with escaping. It's ugly when I am in the middle of a deep indented function call. Imagine I am writing an exception def foo(...) .... raise SomeErrorKindofLong("This is an exception and I need to write something long {v1} so I need to wrap around it {v2}".format(v1=v1, v2=v2)) You get the point... by wrapping, I ended up breaking this into multiple lines. In some cases I would build the string before calling the exception. That seems a little tangent, but relevant. The problem with this proposal is that could break existing application. It's impossible to switch to this convention. John From steve+python at pearwood.info Sat Aug 13 22:28:00 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Aug 2016 12:28:00 +1000 Subject: Anyone here running Python on a PowerPC? Message-ID: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> Is there anyone here running Python on a PowerPC willing to help me diagnose and fix this issue? http://bugs.python.org/issue27761 -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Sat Aug 13 22:35:28 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 13 Aug 2016 19:35:28 -0700 Subject: Anyone here running Python on a PowerPC? References: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87r39sw00f.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > Is there anyone here running Python on a PowerPC willing to help me > diagnose and fix this issue? http://bugs.python.org/issue27761 https://duckduckgo.com/?q=powerpc+emulator gets a few hits, one on sourceforge and one in QEMU. I don't know if those would exhibit the same bug. I expect there's still old PowerMacs kicking around. From torriem at gmail.com Sat Aug 13 23:07:02 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 13 Aug 2016 21:07:02 -0600 Subject: Anyone here running Python on a PowerPC? In-Reply-To: <87r39sw00f.fsf@jester.gateway.pace.com> References: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> <87r39sw00f.fsf@jester.gateway.pace.com> Message-ID: <67f2b7f9-f22f-0ef1-54b8-8f35d00e7744@gmail.com> On 08/13/2016 08:35 PM, Paul Rubin wrote: > Steven D'Aprano writes: >> Is there anyone here running Python on a PowerPC willing to help me >> diagnose and fix this issue? http://bugs.python.org/issue27761 > > https://duckduckgo.com/?q=powerpc+emulator > > gets a few hits, one on sourceforge and one in QEMU. I don't know if > those would exhibit the same bug. > > I expect there's still old PowerMacs kicking around. It's possible QEMU's PPC emulation is faithful enough to replicate this issue. http://wiki.qemu.org/download/qemu-doc.html#PowerPC-System-emulator From torriem at gmail.com Sat Aug 13 23:11:40 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 13 Aug 2016 21:11:40 -0600 Subject: Anyone here running Python on a PowerPC? In-Reply-To: <87r39sw00f.fsf@jester.gateway.pace.com> References: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> <87r39sw00f.fsf@jester.gateway.pace.com> Message-ID: <5e94ec5b-4d6a-2d21-02a1-d5aa4593ff46@gmail.com> On 08/13/2016 08:35 PM, Paul Rubin wrote: > Steven D'Aprano writes: >> Is there anyone here running Python on a PowerPC willing to help me >> diagnose and fix this issue? http://bugs.python.org/issue27761 > > https://duckduckgo.com/?q=powerpc+emulator > > gets a few hits, one on sourceforge and one in QEMU. I don't know if > those would exhibit the same bug. > > I expect there's still old PowerMacs kicking around. I also suggested QEMU, but I'm not sure it's 64-bit PPC, and I'm pretty sure PowerMacs wouldn't be 64-bit either. Though I see some references to PPC64 and QEMU in google searches. From steve+python at pearwood.info Sat Aug 13 23:34:07 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Aug 2016 13:34:07 +1000 Subject: Multi-Line Strings: A Modest Proposal References: <3d212cf3-0ea3-47be-b112-f648c8ef2898@googlegroups.com> <85k2fkyy17.fsf@benfinney.id.au> Message-ID: <57afe6b0$0$1606$c3e8da3$5496439d@news.astraweb.com> On Sun, 14 Aug 2016 11:53 am, John Wong wrote: > The way I solve it, and I still find that extremely ugly, is > > s = ("this string continues " + > "substring continues") You don't need the plus sign, as Python will concatenate string literals (not variables) at compile time: "foo" 'bar' # compiles as 'foobar' Note that you can mix any of the string delimiters, including raw strings. And they don't have to be on the same line: raise ValueError( "This is a long 'error message' using" ' mixed "quotes" specifically so I can' " show off Python's string literal concatenation." ) That's equivalent to the one-line: raise ValueError("This is a long 'error message' using mixed \"quotes\" specifically so I can show off Python's string literal concatenation.") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Sat Aug 13 23:44:43 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 20:44:43 -0700 (PDT) Subject: Multi-Line Strings: A Modest Proposal In-Reply-To: References: <3d212cf3-0ea3-47be-b112-f648c8ef2898@googlegroups.com> <85k2fkyy17.fsf@benfinney.id.au> Message-ID: On Sunday, August 14, 2016 at 1:53:42 PM UTC+12, John Wong wrote: > s = ("this string continues " + > "substring continues") Now there?s a Java way of doing it. :) I prefer implicit string concatenation myself. Isn?t it peculiar that, while this feature was introduced in C and faithfully propagated to C++ (and Python), it was inexplicably omitted from Java? And seemingly, from Perl as well... From steve+python at pearwood.info Sat Aug 13 23:54:01 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Aug 2016 13:54:01 +1000 Subject: How do I tell if I'm running on a PowerPC? Message-ID: <57afeb5b$0$1583$c3e8da3$5496439d@news.astraweb.com> I need to be able to programmatically test whether I'm running on a PowerPC. How can I do that? import platform if platform.machine() in ('ppc', 'ppc64'): print('running PowerPC') Is that right? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lawrencedo99 at gmail.com Sat Aug 13 23:54:06 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 20:54:06 -0700 (PDT) Subject: Anyone here running Python on a PowerPC? In-Reply-To: References: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> <87r39sw00f.fsf@jester.gateway.pace.com> <5e94ec5b-4d6a-2d21-02a1-d5aa4593ff46@gmail.com> Message-ID: <352a3328-7474-44fa-9ca9-e3455d37fb8a@googlegroups.com> On Sunday, August 14, 2016 at 3:11:56 PM UTC+12, Michael Torrie wrote: > I also suggested QEMU, but I'm not sure it's 64-bit PPC, and I'm pretty > sure PowerMacs wouldn't be 64-bit either. Though I see some references > to PPC64 and QEMU in google searches. A little table here has a ?yes? in the ?ppc64? column and ?GNU/Linux? row. The referenced note 12 says to use qemu 2.0.0 and the ?qemu-system-ppc64? command, plus some other details. (That command does exist on my system, though I?ve not tried it.) While Apple never used 64-bit PowerPC processors, it was clear that the architecture was designed with 64-bit in mind from the beginning. Just like the Motorola 68000 and immediate successors in the early 1980s were cut-down 32-bit processors, so the 601 etc through the 1990s were cut-down 64-bit processors. (There was a full 64-bit PowerPC processor, the 620, released as part of the first generation in 1994 or so. Did anybody ever make use of it?) From steve+python at pearwood.info Sun Aug 14 00:13:38 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Aug 2016 14:13:38 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sun, 14 Aug 2016 06:59 am, BartC wrote: > On 13/08/2016 17:06, Steven D'Aprano wrote: >> On Sat, 13 Aug 2016 08:09 pm, BartC wrote: > >>> record date= # define the record >>> var day, month, year >>> end >>> >>> d := date(25,12,2015) # create an instance > >> Sure. But you've put most of the work into the compiler. > > Into the language so that it's always available and consistent rather > than leaving it to individuals to somehow build something on top of > existing language features. Right. And there's little difference between that and making the feature available from a standard library module, like namedtuple. If the Python community rallies around this "record" functionality and takes to it like they took too namedtuple, it could be added to the stdlib. In at least one way that's better than making it part of the language: you can backport the module to older versions of the interpreter. [snip example of a Python "records" factory] > That's pretty good actually. Although I noticed you used the __slots__ > feature that you were scathing about earlier on! (Also that you've > thrown in examples of using decorators, and using exec(), free of > charge..) I wasn't scathing about __slots__. I said that it was not *added* to the language in order to support enforcement of "these attributes, and no others". It was added to the language as a memory optimization. If you, or Lawrence, wish to (ab)use __slots__ to get a kind of poor mans attribute declaration, that's your prerogative. I'm on the fence: I can see that it is, perhaps, sometimes useful, but I wouldn't want to see it become a widespread habit. > There are a number of ways this approach differs from the native record > example (but I acknowledge your example was put together as a quick demo): > > * The fields in my record are known at compile time; using the wrong > name will usually be picked immediately. As will using the wrong number > of initialisers. There's nothing like that in Python. But the difference between compile-time and run-time is not that significant in practice (as opposed to theory, where it is critical). > * (My record fields have other features: aliases for any field; the > ability to access by index, doing len() etc; fast conversions to and > from a list; also the ability to create packed records that exactly > match C structs, and so on.) I'm not sure what you mean by aliases, or how you would use them. Got an example? As for the rest, if I were planning to make this a polished product, I'd certainly go for access by index and support for len(). Given access by index, that gives me list conversion and iteration for free. > * Each defined record in my scheme ('date' and so on) is a distinct > type. As is mine. > If I use the Python record() to create two records R and S, then > type(R) and type(S) seem to give the same result (both 'type'). Of course: the type of a class is "type": py> type(str), type(int), type(list), type(type) (, , , ) This is called the metaclass; the class of a class. And yes, type is its own type. What you should be doing is comparing the types of record *instances* instead: py> x = record('Spam', 'breakfast lunch dinner')('spam', ... 'spam and eggs', 'spam and eggs and spam with a side-dish of spam') py> y = record('Date', 'year month day')(1999, 'August', 14) py> type(x), type(y) (.Inner'>, .Inner'>) Hmmm. I didn't expect that. I expected to see the names of the classes: py> type(x).__name__, type(y).__name__ ('Spam', 'Date') If I was publishing this as a polished product, I'd want to fix that. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Aug 14 00:20:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 14:20:54 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 14, 2016 at 2:13 PM, Steven D'Aprano wrote: > What you should be doing is comparing the types of record *instances* > instead: > > py> x = record('Spam', 'breakfast lunch dinner')('spam', > ... 'spam and eggs', 'spam and eggs and spam with a side-dish of spam') > py> y = record('Date', 'year month day')(1999, 'August', 14) > py> type(x), type(y) > (.Inner'>, > .Inner'>) > > > Hmmm. I didn't expect that. I expected to see the names of the classes: > > py> type(x).__name__, type(y).__name__ > ('Spam', 'Date') > > > If I was publishing this as a polished product, I'd want to fix that. But at this point, you're at the level of minor tweaks, not core functionality - and more importantly, proof of concept is successful. You have a working class factory, which is downright *impossible* in many languages. ChrisA From rosuav at gmail.com Sun Aug 14 00:22:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 14:22:03 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 14, 2016 at 2:20 PM, Chris Angelico wrote: > You have a working class factory, which is downright *impossible* in > many languages. (Unless you count "metaprogramming" to include "programs that write other programs", of course. That is a legit form of metaprogramming, in a sense, and does have its benefits, but it's not nearly as simple as this.) ChrisA From lawrencedo99 at gmail.com Sun Aug 14 00:28:54 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 13 Aug 2016 21:28:54 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sunday, August 14, 2016 at 4:21:16 PM UTC+12, Chris Angelico wrote: > You have a working class factory, which is downright *impossible* in > many languages. If good old, plain old C can do this: /*prints itself*/void main(void){int c=34;char*s="/*prints itself*/void main(void){int c=34;char*s=%c%s%c;printf(s,c,s,c);}";printf(s,c,s,c);} it can do class factories. :) From no.email at nospam.invalid Sun Aug 14 02:09:34 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 13 Aug 2016 23:09:34 -0700 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87fuq7x4o1.fsf@jester.gateway.pace.com> Steven D'Aprano writes: > If the Python community rallies around this "record" functionality and > takes to it like they took too namedtuple I like namedtuple and I think that it's a feature that they're modified by making a new copy. I know that has overhead but it's palpably bug-avoidant. I've used them extensively in some programs and they took a considerable burden off my mind compared to using something like structs or records. From bc at freeuk.com Sun Aug 14 06:31:10 2016 From: bc at freeuk.com (BartC) Date: Sun, 14 Aug 2016 11:31:10 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 14/08/2016 05:13, Steven D'Aprano wrote: > On Sun, 14 Aug 2016 06:59 am, BartC wrote: >> * The fields in my record are known at compile time > There's nothing like that in Python. But the difference between compile-time > and run-time is not that significant in practice (as opposed to theory, > where it is critical). But this is, sort of, the subject of the thread. I gave record fields as one area where they could be made static by pre-declaring, without too many objections. > >> * (My record fields have other features: aliases for any field; the >> ability to access by index, doing len() etc; fast conversions to and >> from a list; also the ability to create packed records that exactly >> match C structs, and so on.) > > I'm not sure what you mean by aliases, or how you would use them. Got an > example? ( record date = var day, month, year var giorno @day end So 'giorno' is a synonym for 'day'. Or sometimes a field is used for different purposes in different contexts, and a more apt name might be useful: var text var window_caption @ text var button_text @ text (This can be done also with the more formal 'union' syntax, but I only allow that for packed structs. It might look like this within a record def: union var text, window_caption, button_text end )) >> If I use the Python record() to create two records R and S, then >> type(R) and type(S) seem to give the same result (both 'type'). > > Of course: the type of a class is "type": > > py> type(str), type(int), type(list), type(type) > (, , , ) > > > This is called the metaclass; the class of a class. And yes, type is its own > type. What you should be doing is comparing the types of record *instances* > instead: > (.Inner'>, > .Inner'>) > Hmmm. I didn't expect that. I expected to see the names of the > classes: Actually I thought I was comparing instances, but I was applying type() to the return value of record(). But doing that now, I get the above too. I get this behaviour with my scheme: d := new(date) # d is an instance of a date record println date # is displayed println date.type # println d.type # println d.basetype # -- Bartc From bc at freeuk.com Sun Aug 14 06:49:14 2016 From: bc at freeuk.com (BartC) Date: Sun, 14 Aug 2016 11:49:14 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 14/08/2016 05:20, Chris Angelico wrote: > On Sun, Aug 14, 2016 at 2:13 PM, Steven D'Aprano > wrote: >> What you should be doing is comparing the types of record *instances* >> instead: >> >> py> x = record('Spam', 'breakfast lunch dinner')('spam', >> ... 'spam and eggs', 'spam and eggs and spam with a side-dish of spam') >> py> y = record('Date', 'year month day')(1999, 'August', 14) >> py> type(x), type(y) >> (.Inner'>, >> .Inner'>) >> >> >> Hmmm. I didn't expect that. I expected to see the names of the classes: >> >> py> type(x).__name__, type(y).__name__ >> ('Spam', 'Date') >> >> >> If I was publishing this as a polished product, I'd want to fix that. > > But at this point, you're at the level of minor tweaks, not core > functionality - and more importantly, proof of concept is successful. > You have a working class factory, which is downright *impossible* in > many languages. Well, it's using exec(). So it is generating new program code at runtime. That is possible in quite a few languages, even C. Python is much better however in being able to integrate the results into the currently running program that invoked exec(). (Myself, I generally stay away from language building and extension features. I believe it leads to more complex languages - look at C++ - that can be difficult to follow.) -- Bartc From uri at speedy.net Sun Aug 14 07:00:19 2016 From: uri at speedy.net (Uri Even-Chen) Date: Sun, 14 Aug 2016 14:00:19 +0300 Subject: I'm looking for a part-time job in Python / Django Message-ID: To python-list at python.org, I'm looking for a part-time job in Python / Django, do you know anything? I live in Herzliya, Israel. I can't relocate but I can work from home. You can see my CV on LinkedIn. Please let me know if you have any job for me. Thanks, Uri. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ From __peter__ at web.de Sun Aug 14 07:02:56 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Aug 2016 13:02:56 +0200 Subject: Creating dictionary of items from Excel with mutliple keys References: Message-ID: Atri Mahapatra wrote: > I am trying to create a following dictionary. I am reading data from excel > which has data in the following format: > > Sl no: Name Thickness Length Material Width Quantity Side > > It has 20 rows of data. > > > The dictionary for the 20 rows, I would like to make is > > Data_Dict = [ > { 'Name': 'X', 'Length': '10' , 'Width': '5', 'Quantity': 2 'Area': 50}; > { 'Name': 'Y', 'Length': '20' , 'Width': 10', 'Quantity': 1 'Area': 200}; > . > . > . > . > till 20 rows > ]; > > I would like to add another key 'Area' as shown above. I used the > following code(s): > > using openpyxl: > d={} > for i in range(3,sheet.max_row+1): > #for j in range(3,9): > #for k in range(0,5): > Name = sheet.cell(row= i,column=3).value > Length =sheet.cell(row =i,column=6).value > Breadth= sheet.cell(row=i,column=7).value > Quantity = sheet.cell (row=i,column=8).value > Area = sheet.cell(row > =i,column=6).value*sheet.cell(row=i,column=7).value d[Name]= > Length,Breadth,Quantity,Area > > which gave an output like: > ['X': (10, 5, 2, 50), 'Y': (20, 10, 1, 2232600), 'Z': (5, 2, 1, 10), > [.............] > > Another code using xlrd: > > keys = [sheet.cell(2, col_index).value for col_index in range(0,8)] > print (keys) > dict_list = [] > d = {} > for row_index in range(1, xl_sheet.nrows): > for col_index in range(0,8): > d = {keys[col_index]: xl_sheet.cell(row_index, col_index).value > for col_index in range(0,8)} > dict_list.append(d) > > print (dict_list) > which did not have the area and neither the output was little messy. > > > > The main purpose is to sort the dictionary based on different criteria > like Length or Area. I think the first one may be easier to sort. However > if there are any better way to represent the dictionary and the code so > that it can be sorted later based on different attributes please feel free > to suggest. Things become clearer when you abstract out reading the data. For example: import openpyxl def open_sheet(filename, sheetname): wb = openpyxl.load_workbook(filename=filename) return wb.get_sheet_by_name(sheetname) def read_table(sheet, columnnames, header_row=0): name_to_index = { n: i for i, n in enumerate(c.value for c in sheet.rows[header_row]) if n is not None} column_indices = [name_to_index[n] for n in columnnames] for row in sheet.rows[header_row + 1:]: yield dict(zip(columnnames, (row[x].value for x in column_indices))) if __name__ == "__main__": from operator import itemgetter import pprint # read data from Excel rows = read_table( open_sheet("sample.xlsx", "SampleSheet"), "Name Length Width Quantity".split(), 2 # replace with actual header row ) # add Area dict_list = [] for row_dict in rows: row_dict["Area"] = row_dict["Width"] * row_dict["Length"] dict_list.append(row_dict) # sort and print data print("Unsorted:") pprint.pprint(dict_list) for sort_column in "Width", "Quantity": print("\nby {}:".format(sort_column)) dict_list.sort(key=itemgetter(sort_column)) pprint.pprint(dict_list) To use xlrd instead of openpyxl you have to replace the open_sheet() and read_table() functions with def open_sheet(filename, sheetname): wb = xlrd.open_workbook(filename) return wb.sheet_by_name(sheetname) def read_table(sheet, columnnames, header_row=0): name_to_index = { c.value: i for i, c in enumerate(sheet.row(header_row)) if c.ctype == xlrd.XL_CELL_TEXT} column_indices = [name_to_index[n] for n in columnnames] for rowindex in range(header_row + 1, sheet.nrows): row = sheet.row(rowindex) yield dict(zip(columnnames, (row[x].value for x in column_indices))) From nomail at com.invalid Sun Aug 14 07:06:25 2016 From: nomail at com.invalid (ast) Date: Sun, 14 Aug 2016 13:06:25 +0200 Subject: A strange list concatenation result In-Reply-To: References: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> <57ae7314$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57b050b5$0$23794$426a74cc@news.free.fr> "Mok-Kong Shen" a ?crit dans le message de news:noo1v6$r39$1 at news.albasani.net... > Am 13.08.2016 um 03:08 schrieb Steven D'Aprano: >> On Sat, 13 Aug 2016 06:44 am, Mok-Kong Shen wrote: >> >>>>>>> list2 = [1,2,3] >>>>>>> list1 += [4,5,6] >>>>>>> print(list1, list2) >>>> [1, 2, 3, 4, 5, 6] [1, 2, 3] >>>> >>>> >>>> Does that help? >>> >>> I don't yet understand why in my 2nd example list2 came out as >>> [1, 2, 3] outside. >> >> Because you assign list2 = [1, 2, 3]. What did you expect it to be? > > But in my function test() there is a code line "list2=list2+[4,5,6]". > Could you kindly explain why this didn't work? > > M. K. Shen > Hello list1 += [4, 5, 6 ] is slightly different than list1 = list1 + [4, 5, 6] see: >>> list1 = [1, 2, 3] >>> id(list1) 45889616 list1 += [4, 5, 6 ] >>> id(list1) 45889616 # < -same address >>> list1 [1, 2, 3, 4, 5, 6] so list1 += [4, 5, 6 ] updates the existing list >>> list1 = [1, 2, 3] >>> id(list1) 45888656 >>> list1 = list1 + [4, 5, 6] >>> id(list1) 45862424 # <- a new address >>> list1 [1, 2, 3, 4, 5, 6] so list1 = list1 + [4, 5, 6] create a new list From martin.schoon at gmail.com Sun Aug 14 07:12:36 2016 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 14 Aug 2016 11:12:36 GMT Subject: How do I make a video animation with transparent background? References: <607cc26d-ca5a-4509-b4f5-da7c0b6c627f@googlegroups.com> Message-ID: Den 2016-08-10 skrev Lawrence D?Oliveiro : > On Thursday, August 11, 2016 at 9:34:33 AM UTC+12, Martin Sch??n wrote: >> Next on my TODO list is to look into the color composition idea. I >> have already looked this tutorial: >> https://youtu.be/sZzmksnzrX0 > > If the images you generate have an alpha channel, Blender can composite > them directly, without needing colour-keying. Colour-keying works in Blender -- I only need to tweak all those parameters a bit more to get a satisfying end result. Case closed and it was not a Python/Matplotlib issue. I still need the animated numerical thing but drop it here. I will be back with a separate thread/subject for that. Again, thanks for the input. /Martin From python.list at tim.thechases.com Sun Aug 14 07:30:24 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 14 Aug 2016 06:30:24 -0500 Subject: Anyone here running Python on a PowerPC? In-Reply-To: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <57afd731$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20160814063024.5fe36598@bigbox.christie.dr> On 2016-08-14 12:28, Steven D'Aprano wrote: > Is there anyone here running Python on a PowerPC willing to help me > diagnose and fix this issue? > > http://bugs.python.org/issue27761 Not so savvy in the building, but I've got a Mac PPC (still on 10.4 or whatever the last-ish PPC build of OS X was) that's sitting on my desk disconnected. I can easily hook it back up and boot Mac or install/boot some other OS if needed, and then follow debugging steps as directed... -tkc From dponyatov at gmail.com Sun Aug 14 07:53:54 2016 From: dponyatov at gmail.com (Dmitry Ponyatov) Date: Sun, 14 Aug 2016 04:53:54 -0700 (PDT) Subject: SmallTalk-like interactive environment on base of cPython 2.7.x + wx Message-ID: Does anybody can recomend some links on tutorials on making custom dynamic languages or objects systems on top of cPython2 ? I want some interactive dynamic object environment with SmallTalk look&feel but with Python syntax. Other tutorials I'm interested in are reflection, dynamic bytecode (de)compilation, using internal Python parser (for syntax highlighting for example), metacompilation (?) and making dynamic object VMs on top of cPython. From breamoreboy at gmail.com Sun Aug 14 08:46:15 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 14 Aug 2016 05:46:15 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <87fuq7x4o1.fsf@jester.gateway.pace.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> <87fuq7x4o1.fsf@jester.gateway.pace.com> Message-ID: On Sunday, August 14, 2016 at 7:09:47 AM UTC+1, Paul Rubin wrote: > Steven D'Aprano writes: > > If the Python community rallies around this "record" functionality and > > takes to it like they took too namedtuple > > I like namedtuple and I think that it's a feature that they're modified > by making a new copy. I know that has overhead but it's palpably > bug-avoidant. I've used them extensively in some programs and they took > a considerable burden off my mind compared to using something like > structs or records. You might find this https://glyph.twistedmatrix.com/2016/08/attrs.html an interesting read. Kindest regards. Mark Lawrence. From rosuav at gmail.com Sun Aug 14 09:18:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 23:18:40 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 14, 2016 at 8:49 PM, BartC wrote: > Well, it's using exec(). So it is generating new program code at runtime. > That is possible in quite a few languages, even C. It doesn't have to; that's only so it doesn't have to manually construct a Function object. Or you could make a simpler __init__ that requires keyword arguments. Here's an exec-free version: def record(name, fieldnames): if isinstance(fieldnames, str): fieldnames = fieldnames.split() class Inner(object): __slots__ = fieldnames def __init__(self, **kwargs): for n in fieldnames: if n in kwargs: setattr(self, n, kwargs[n]) def __repr__(self): fields = ', '.join("%s=%r" % (n, getattr(self, n)) for n in fieldnames) return "record %s(%s)" % (type(self).__name__, fields) Inner.__name__ = Inner.__qualname__ = name return Inner You lose the ability to construct a record with positional args, and you lose the argument name info from help() and other forms of introspection, but it works, and it doesn't use exec. How do you do that in C? Do you have an 'exec' function in C? Certainly not in any of my compilers. The nearest equivalent would be a compile-time directive, eg a preprocessor macro, and that most definitely isn't this flexible. Types just aren't first-class objects in C. ChrisA From rosuav at gmail.com Sun Aug 14 09:21:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 23:21:06 +1000 Subject: I'm looking for a part-time job in Python / Django In-Reply-To: References: Message-ID: On Sun, Aug 14, 2016 at 9:00 PM, Uri Even-Chen wrote: > To python-list at python.org, > > I'm looking for a part-time job in Python / Django, do you know anything? I > live in Herzliya, Israel. I can't relocate but I can work from home. You > can see my CV on LinkedIn. Please let me know if you have any job for me. > Hi! This isn't a job posting list, but there are a few places that accept job ads. One good one is the Python Job Board: https://www.python.org/jobs/ Browse that, and possibly subscribe to it; there are a lot of jobs that would demand relocation, but every now and then you'll see a remote opening. You could also search the web for "django jobs" and see what you find. Good luck! There are definitely Python jobs out there, but you may find that most of them require physical presence in the US or Europe. But don't give up! ChrisA From rosuav at gmail.com Sun Aug 14 09:24:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 14 Aug 2016 23:24:13 +1000 Subject: SmallTalk-like interactive environment on base of cPython 2.7.x + wx In-Reply-To: References: Message-ID: On Sun, Aug 14, 2016 at 9:53 PM, Dmitry Ponyatov wrote: > Does anybody can recomend some links on tutorials on making custom dynamic languages or objects systems on top of cPython2 ? > > I want some interactive dynamic object environment with SmallTalk look&feel but with Python syntax. Any particular reason for using 2.7 rather than 3.x? As a general rule, I would advise you to make your dynamic language actually *be* Python. With judicious use of decorators and other metaprogramming tools, you can craft a DSL (domain-specific language) that has whatever broad look-and-feel you choose, but will be directly executable using a Python interpreter. For anything more specific than that, it'd be helpful to know more details about what you're trying to do. What's a "dynamic object environment"? ChrisA From steve+python at pearwood.info Sun Aug 14 10:29:29 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 15 Aug 2016 00:29:29 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> On Thu, 11 Aug 2016 06:33 am, Juan Pablo Romero M?ndez wrote: > I've been trying to find (without success so far) an example of a > situation where the dynamic features of a language like Python provides a > clear advantage over languages with more than one type. Python has more than one type. Don't confuse dynamic typing with weak typing or untyped (typeless) languages. More on this below. I don't believe that you will find "an example of a situation..." as you say above. It sounds like you are hope to find a clear example of "If you do This, then dynamic languages are the Clear Winner". But I don't think you will. Dynamic languages tend to produce clear productivity improvements over statically typed languages, but of course this is only "typically" true, not a guarantee that applies to every single programmer or project. Typically: - dynamic languages are less verbose; - dynamic languages are faster to develop in; many organisations prototype applications in Python (say) before re-writing it in C++/Java/whatever; - far less time spent fighting the compiler; - dynamic languages often have fewer bugs, because it is easier to reason about the code (no "undefined behaviour" like in C!) and fewer lines of code to reason about; - but statically typed languages allow you to prove the absence of certain types of bugs. The exception is if you try to write statically typed code in a dynamic language. Then you get the worst of both styles of coding: the verbose, heavyweight style of many static languages, but without the automated correctness proofs, plus the performance costs of dynamic typing, but without the rapid development. Regarding types and type systems, if you haven't already read this, you should: https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ "Static typing" (e.g. Pascal, C, Java, Haskell) and "dynamic typing" (e.g. Python, Javascript, Ruby, Lua) differ on when and how values are checked for type-compatibility. "Strong" and "weak" typing are ends of a continuum. Nearly all languages are a little bit weak (they allow automatic coercions between numeric types) but mostly strong (they don't automatically coerce integers to arrays). Javascript, Perl and PHP are weaker than Python because they'll coerce strings to numbers automatically and Python won't. I don't know many untyped languages apart from machine code or maybe assembly. Perhaps Forth? (Maybe not -- some Forths include a separate floating point stack as well as the usual stack.) Hypertalk treated everything as strings. Tcl treats nearly everything as strings, although it also has arrays. So, Python has types, and it is a mostly strong typed language. It will do relatively few automatic coercions. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lampss at hotmail.com Sun Aug 14 11:18:23 2016 From: lampss at hotmail.com (Lallo) Date: Sun, 14 Aug 2016 17:18:23 +0200 Subject: Module pyswisseph Message-ID: Hi, I would like to install the module pypi.python.org/pypi/pyswisseph in Python 2.7.12 on Windows 7 / 64 Bit. In the instructions it says to use CMake and MinGW. How should I do? Can you help me please? Thanks, Lallo From bc at freeuk.com Sun Aug 14 12:31:19 2016 From: bc at freeuk.com (BartC) Date: Sun, 14 Aug 2016 17:31:19 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 14/08/2016 14:18, Chris Angelico wrote: > On Sun, Aug 14, 2016 at 8:49 PM, BartC wrote: >> Well, it's using exec(). So it is generating new program code at runtime. >> That is possible in quite a few languages, even C. > > It doesn't have to; that's only so it doesn't have to manually > construct a Function object. Or you could make a simpler __init__ that > requires keyword arguments. Here's an exec-free version: > > def record(name, fieldnames): > if isinstance(fieldnames, str): > fieldnames = fieldnames.split() > class Inner(object): > __slots__ = fieldnames > def __init__(self, **kwargs): > for n in fieldnames: > if n in kwargs: setattr(self, n, kwargs[n]) > def __repr__(self): > fields = ', '.join("%s=%r" % (n, getattr(self, n)) for n > in fieldnames) > return "record %s(%s)" % (type(self).__name__, fields) > Inner.__name__ = Inner.__qualname__ = name > return Inner > > You lose the ability to construct a record with positional args, and > you lose the argument name info from help() and other forms of > introspection, but it works, and it doesn't use exec. > > How do you do that in C? Do you have an 'exec' function in C? Not built-in to the language. C code can however write out source code, and invoke a C compiler to turn it into a shared library, which can then be linked to dynamically. But it can't create a new record or struct type at runtime which can then be accessed using normal syntax, in compiled code that already existed before the record was created. (Only if access code is generated too.) In general, doing the equivalent of what Steven's record() example was doing can also be done across a few languages, up to a point anyway (and usually not in five minutes!): * Creating a handle to a data structure representing a new record (giving record name and a list of fields). * Using that handle to create an instance of such a record (represented by any means, even just using a list). * Allowing some way to set or get fields of each instance. The last is harder if you want a method that looks like a field access. Doing it via functions is straightforward; allowing dot operators ('d.month') is much harder, especially with more rigid languages where everything is expected to be resolved at compile time. It's a bit easier in Python because 'd.month' already involves dynamic mechanisms which can be hooked into. In fact, the above implementation and the original are based on classes and their existing means of setting and getting attributes. So it's about having a more streamlined way to define such a class with a fixed set of attributes. Both use the __slots__ feature to restrict the number of attributes to that fixed list (I imagine it would have been more difficult without that). -- Bartc > Certainly not in any of my compilers. The nearest equivalent would be > a compile-time directive, eg a preprocessor macro, and that most > definitely isn't this flexible. Types just aren't first-class objects > in C. > > ChrisA > From tarakbenammar at gmx.com Sun Aug 14 12:37:27 2016 From: tarakbenammar at gmx.com (FABIO VENZI GRAN LOGGIA REGOLARE DI ITALIA) Date: Sun, 14 Aug 2016 09:37:27 -0700 (PDT) Subject: Redacted Message-ID: <9e523993-9b89-41c6-850e-382e9114e18a@googlegroups.com> Redacted From python at mrabarnett.plus.com Sun Aug 14 12:44:08 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 14 Aug 2016 17:44:08 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-14 15:29, Steven D'Aprano wrote: [snip] > I don't know many untyped languages apart from machine code or maybe > assembly. Perhaps Forth? (Maybe not -- some Forths include a separate > floating point stack as well as the usual stack.) Hypertalk treated > everything as strings. Tcl treats nearly everything as strings, although it > also has arrays. > [snip] BCPL (ancestor of C) had a single data type, the bit-pattern. From rosuav at gmail.com Sun Aug 14 12:53:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Aug 2016 02:53:59 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 15, 2016 at 2:44 AM, MRAB wrote: > On 2016-08-14 15:29, Steven D'Aprano wrote: > [snip] >> >> I don't know many untyped languages apart from machine code or maybe >> assembly. Perhaps Forth? (Maybe not -- some Forths include a separate >> floating point stack as well as the usual stack.) Hypertalk treated >> everything as strings. Tcl treats nearly everything as strings, although >> it >> also has arrays. >> > [snip] > > BCPL (ancestor of C) had a single data type, the bit-pattern. REXX treats everything as strings, and has a special type of variable (the "stem") which can do the jobs that arrays and mappings can. Booleans are just strings of either "1" or "0". Numbers are strings containing nothing but decimal digits, an optional decimal point, and maybe an exponent. Open files are referenced by their file names. (Which means you can never have a file open for two different jobs at once. You explicitly close it in between.) ChrisA From atri.mahapatra at gmail.com Sun Aug 14 14:17:24 2016 From: atri.mahapatra at gmail.com (Atri Mahapatra) Date: Sun, 14 Aug 2016 11:17:24 -0700 (PDT) Subject: Finding the first index in a list greater than a particular value Message-ID: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> I have a list of dictionaries which look like this: [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] I would like to find the first index in the list of dictionaries whose length is greater than a particular value f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > m][0] and it throws an error can anyone suggest a way to do it? Thanks, Atri From random832 at fastmail.com Sun Aug 14 14:19:11 2016 From: random832 at fastmail.com (Random832) Date: Sun, 14 Aug 2016 14:19:11 -0400 Subject: Python slang In-Reply-To: <6913354c-f5ba-d392-dc1f-93882f4fe564@gmail.com> References: <5dd0922f-9a6d-6978-f785-8bbf351f17a5@stofanet.dk> <3d7fec12-8bea-4f26-afa1-f8516763bfc5@googlegroups.com> <1470845983.1463573.691529097.0E053AB7@webmail.messagingengine.com> <6913354c-f5ba-d392-dc1f-93882f4fe564@gmail.com> Message-ID: <1471198751.1262769.695013617.3D96106E@webmail.messagingengine.com> On Wed, Aug 10, 2016, at 19:57, Michael Torrie wrote: > But the grammar must still be a bit complex as sometimes the LHS of the > = is an expression, as well as the RHS. The only place that an *arbitrary* expression (including e.g. = as equality) can appear in the LHS is inside parentheses, otherwise you're limited to dot and call/index (which are both parentheses, which isn't a problem for the parser since no object can do both and it's the same kind of expression) just like in Python. From no.email at nospam.invalid Sun Aug 14 14:21:14 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 14 Aug 2016 11:21:14 -0700 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: <874m6nw6sl.fsf@jester.gateway.pace.com> MRAB writes: >> I don't know many untyped languages apart from machine code or maybe >> assembly. Perhaps Forth? (Maybe not -- some Forths include a separate >> floating point stack as well as the usual stack.) Forth is essentially untyped. There's no distinction between integers, characters, and pointers. There is usually a separate FP stack, but FP variables are stored in memory just like integers, and there's no distinction between the memory cells. You just use a different command to push the contents of a memory cell onto the FP stack than onto the integers stack. From python at mrabarnett.plus.com Sun Aug 14 14:33:39 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 14 Aug 2016 19:33:39 +0100 Subject: Finding the first index in a list greater than a particular value In-Reply-To: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> Message-ID: <4093e046-0f85-8792-08c7-21013dc5e28e@mrabarnett.plus.com> On 2016-08-14 19:17, Atri Mahapatra wrote: > I have a list of dictionaries which look like this: > [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] > > I would like to find the first index in the list of dictionaries whose length is greater than a particular value > > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > m][0] and it throws an error > > can anyone suggest a way to do it? > What is the error? From atri.mahapatra at gmail.com Sun Aug 14 14:40:39 2016 From: atri.mahapatra at gmail.com (Atri Mahapatra) Date: Sun, 14 Aug 2016 11:40:39 -0700 (PDT) Subject: Finding the first index in a list greater than a particular value In-Reply-To: References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> <4093e046-0f85-8792-08c7-21013dc5e28e@mrabarnett.plus.com> Message-ID: On Monday, 15 August 2016 00:03:59 UTC+5:30, MRAB wrote: > On 2016-08-14 19:17, Atri Mahapatra wrote: > > I have a list of dictionaries which look like this: > > [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] > > > > I would like to find the first index in the list of dictionaries whose length is greater than a particular value > > > > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > m][0] and it throws an error > > > > can anyone suggest a way to do it? > > > What is the error? Traceback (most recent call last): File "", line 1, in f(lengthlist, 10) f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii] > m][0] TypeError: unorderable types: dict() > int() From BSomerville at flexerasoftware.com Sun Aug 14 15:39:34 2016 From: BSomerville at flexerasoftware.com (Bill Somerville) Date: Sun, 14 Aug 2016 19:39:34 +0000 Subject: Issue with ctypes and callback functions References: <0E5668E32633FB4FBBA1709825C0EDFDD24AAE@SCHEXMB2.acresso.com> <0E5668E32633FB4FBBA1709825C0EDFDD24B77@SCHEXMB2.acresso.com> Message-ID: <0E5668E32633FB4FBBA1709825C0EDFDD266B1@SCHEXMB2.acresso.com> -----Original Message----- From: Bill Somerville Sent: 04 August 2016 18:23 To: 'eryk sun' ; python-list at python.org Subject: RE: Issue with ctypes and callback functions -----Original Message----- From: eryk sun [mailto:eryksun at gmail.com] from_param is a hook method for a type that's set in a function pointer's argtypes. It gets called to convert an argument when the function pointer is called from Python. The return value can be a ctypes instance, an object with a hard-coded conversion (e.g. a string or integer), or an object that defines an _as_parameter_ attribute. Only ctypes types are supported in callbacks, which unfortunately isn't documented clearly. Specifically, the class dict needs to be an extended C storage dict (i.e. StgDictObject), either to look up the getfunc of a simple type or to ensure that instantiating a non-simple type returns a ctypes instance (i.e. CDataObject) with a known size. The relevant code in _CallPythonObject is as follows (when stripped of declarations and error handling): cnv = PySequence_GetItem(converters, i); dict = PyType_stgdict(cnv); if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) { v = dict->getfunc(*pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, v); } else if (dict) { obj = (CDataObject *)PyObject_CallFunctionObjArgs(cnv, NULL); memcpy(obj->b_ptr, *pArgs, dict->size); PyTuple_SET_ITEM(arglist, i, (PyObject *)obj); } else { PyErr_SetString(PyExc_TypeError, "cannot build parameter"); I don't have much experience with SWIG. Does it provide some means to instantiate a wrapped type from an address? If it does, then you can use a void pointer as the callback parameter. Hi Eryk, Thanks for looking at this. I had got as far as looking at the code above and guessed that only ctypes types were supported. This does seem like a bit of a functionality gap that cannot be solved easily by users of ctypes. It would appear to be fairly easy to define a protocol for objects that can be instantiated from ctypes arguments before passing to Python callables, i.e. some requirement for a class method factory that takes a ctypes type as a parameter and returns a Python object. This would seem to be a natural complement to from_param()/_as_parameter_ for the C/C++ to Python direction of function calls. I had started with an implementation that converted a c_void_p value and instantiated the required SWIG Python wrapper. This is not too difficult as SWIG allows its types to be extended in both the C/C++ world and the Python world. The result is only a one liner for each callback parameter but it is still ugly as the writer of the callback has to know what type the argument is and remember to call the "constructor" before accessing the attributes of the object. My requirement is to wrap an API for testers and customers to write simple test cases and any ugliness in the test cases is undesirable given that the users will probably not be Python experts and maybe not even developers. Regards Bill. From python at mrabarnett.plus.com Sun Aug 14 16:23:12 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 14 Aug 2016 21:23:12 +0100 Subject: Finding the first index in a list greater than a particular value In-Reply-To: References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> <4093e046-0f85-8792-08c7-21013dc5e28e@mrabarnett.plus.com> Message-ID: <596cc737-d26b-4180-85a3-2eadca1953c2@mrabarnett.plus.com> On 2016-08-14 19:40, Atri Mahapatra wrote: > On Monday, 15 August 2016 00:03:59 UTC+5:30, MRAB wrote: >> On 2016-08-14 19:17, Atri Mahapatra wrote: >> > I have a list of dictionaries which look like this: >> > [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] >> > >> > I would like to find the first index in the list of dictionaries whose length is greater than a particular value >> > >> > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > m][0] and it throws an error >> > >> > can anyone suggest a way to do it? >> > >> What is the error? > > Traceback (most recent call last): > File "", line 1, in > f(lengthlist, 10) > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii] > m][0] > TypeError: unorderable types: dict() > int() > That's not the same code as your original post. Your original post has: seq[ii]['Length'] > m which is correct, but your traceback shows: seq[ii] > m which tries to compare the entire dict to m, hence the exception. From bnrj.rudra at gmail.com Sun Aug 14 17:23:31 2016 From: bnrj.rudra at gmail.com (bnrj.rudra at gmail.com) Date: Sun, 14 Aug 2016 14:23:31 -0700 (PDT) Subject: compile a python3 project with autotools Message-ID: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> Hi, I am trying to compile a python project using autotools. The project runs absolutely fine from command line, and it uses Gtk3 library. the src file is: ??? src ??? ??? Makefile.am ??? ??? Makefile.in ??? ??? Mkbib ??? ??? ??? cell.py ??? ??? ??? dialogue.py ??? ??? ??? filemanager.py ??? ??? ??? getdata.py ??? ??? ??? __init__.py ??? ??? ??? main.py ??? ??? ??? Makefile.am ??? ??? ??? Makefile.in ??? ??? ??? menu.py ??? ??? ??? mkbib.in ??? ??? ??? pybib.py ??? ??? ??? view.py ??? ??? mkbib.in with the mkbib.in is: #!/usr/bin/env python3 import sys sys.path.insert(1, '@pythondir@') from mkbib.main import mkbib if __name__ == "__main__": app = mkbib() r = app.run() sys.exit(r) src/Mkbib/main.py is the main program. This is structured as: class Window(Gtk.ApplicationWindow): def __init__(self, application, giofile=None): Gtk.ApplicationWindow.__init__(self, application=application, default_width=1000, default_height=200, border_width=5) class mkbib(Gtk.Application): def __init__(self): Gtk.Application.__init__(self) def new_window(self, filename=None): window = Window(self) window.show_all() def run(self): self.new_window() Gtk.main() Now, the problem is it is compiling fine (make; make install goes without any error) make --silent Making all in data Making all in icons Making all in hicolor Making all in 48x48 Making all in apps Making all in scalable Making all in apps Making all in ui Making all in src Making all in Mkbib but while running the application, I am getting error: (mkbib:14843): Gtk-CRITICAL **: New application windows must be added after the GApplication::startup signal has been emitted. May I be kindly helped? From bnrj.rudra at live.com Sun Aug 14 17:30:13 2016 From: bnrj.rudra at live.com (Rudra Banerjee) Date: Sun, 14 Aug 2016 14:30:13 -0700 (PDT) Subject: compile a python3 project with autotools In-Reply-To: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> References: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> Message-ID: <5659dcec-010f-4294-b547-faa864ea5901@googlegroups.com> Plz ignore src/Mkbib/mkbib.in file, as appeared in the tree From adashparash at gmx.com Sun Aug 14 17:52:35 2016 From: adashparash at gmx.com (ANTONIO BINNI GRAN MAESTRO GRAN LOGGIA D'ITALIA) Date: Sun, 14 Aug 2016 14:52:35 -0700 (PDT) Subject: Redacted Message-ID: <20d581c3-bf9f-4fd9-a077-17fbe9cc2af4@googlegroups.com> Redacted From lawrencedo99 at gmail.com Sun Aug 14 18:39:00 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 14 Aug 2016 15:39:00 -0700 (PDT) Subject: compile a python3 project with autotools In-Reply-To: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> References: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> Message-ID: <18328113-c192-4e6f-bb60-3c0dea2f814b@googlegroups.com> On Monday, August 15, 2016 at 9:24:04 AM UTC+12, Rudra Banerjee wrote: > but while running the application, I am getting error: > (mkbib:14843): Gtk-CRITICAL **: New application windows must be added after > the GApplication::startup signal has been emitted. Does the application actually fail? Because I see a lot of this sort of thing, which doesn?t seem to actually affect the running of the application. So I ignore it. :) From lawrencedo99 at gmail.com Sun Aug 14 18:45:03 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 14 Aug 2016 15:45:03 -0700 (PDT) Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: <17175dd0-2655-4c75-a928-df5adcec0928@googlegroups.com> On Monday, August 15, 2016 at 4:31:44 AM UTC+12, BartC wrote: > But it can't create a new record or struct type at runtime which can > then be accessed using normal syntax, in compiled code that already > existed before the record was created. That?s an awful lot of ?which ... that? qualification going on. Remember, Python, which is supposed to be able to do all that, is implemented in C. From torriem at gmail.com Sun Aug 14 20:09:16 2016 From: torriem at gmail.com (Michael Torrie) Date: Sun, 14 Aug 2016 18:09:16 -0600 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <17175dd0-2655-4c75-a928-df5adcec0928@googlegroups.com> References: <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> <17175dd0-2655-4c75-a928-df5adcec0928@googlegroups.com> Message-ID: <375cbf8c-4f78-5a5b-2fc6-3d04d95ff12d@gmail.com> On 08/14/2016 04:45 PM, Lawrence D?Oliveiro wrote: > On Monday, August 15, 2016 at 4:31:44 AM UTC+12, BartC wrote: > >> But it can't create a new record or struct type at runtime which can >> then be accessed using normal syntax, in compiled code that already >> existed before the record was created. > > That?s an awful lot of ?which ... that? qualification going on. > > Remember, Python, which is supposed to be able to do all that, is implemented in C. Though PyPy proves that Python can bootstrap itself, and do that fairly efficiently. From steve+python at pearwood.info Sun Aug 14 21:45:21 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Mon, 15 Aug 2016 11:45:21 +1000 Subject: What's the best way to minimize the need of run time checks? References: <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> <17175dd0-2655-4c75-a928-df5adcec0928@googlegroups.com> Message-ID: <57b11eb2$0$22142$c3e8da3$5496439d@news.astraweb.com> On Mon, 15 Aug 2016 08:45 am, Lawrence D?Oliveiro wrote: > On Monday, August 15, 2016 at 4:31:44 AM UTC+12, BartC wrote: > >> But it can't create a new record or struct type at runtime which can >> then be accessed using normal syntax, in compiled code that already >> existed before the record was created. > > That?s an awful lot of ?which ... that? qualification going on. > > Remember, Python, which is supposed to be able to do all that, is > implemented in C. Corollary to the Fundamental Theorem of Software Engineering: Every problem in software engineering can be solved in any language by writing an interpreter for another language, except the problem of poor performance. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From michael.selik at gmail.com Sun Aug 14 23:01:22 2016 From: michael.selik at gmail.com (Michael Selik) Date: Mon, 15 Aug 2016 03:01:22 +0000 Subject: Finding the first index in a list greater than a particular value In-Reply-To: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> Message-ID: On Sun, Aug 14, 2016 at 2:21 PM Atri Mahapatra wrote: > I have a list of dictionaries which look like this: > [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, > 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] > > I would like to find the first index in the list of dictionaries whose > length is greater than a particular value > > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > > m][0] and it throws an error > > can anyone suggest a way to do it? > There's no need to play code golf (squishing your thoughts into one line). def index_of_first_record_of_minimum_size(records, m): 'records is an iterable of dictionaries' for i, dictionary in enumerate(list_of_dicts): if dictionary['Length'] > m: return i raise ValueError('no record found with "Length" greater than %r' % (m,)) If you have trouble coming up with a good name for a function, that suggests you are either doing too much or too little with that function. If you want the same task outside of a function, change the return to a break. if not records: raise ValueError('no records found') for index, d in enumerate(records): if d['Length'] > m: break # `index` is now the index of the first record of minimum length else: raise ValueError('no record found with "Length" greater than %r' % (m,)) From jussi.piitulainen at helsinki.fi Mon Aug 15 01:57:57 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Mon, 15 Aug 2016 08:57:57 +0300 Subject: Finding the first index in a list greater than a particular value References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> Message-ID: Atri Mahapatra writes: > I have a list of dictionaries which look like this: > [{'Width': 100, 'Length': 20.0, 'Object': 'Object1'}, {'Width': 12.0, 'Length': 40.0, 'Object': 'Object2'}...... so on till 10] > > I would like to find the first index in the list of dictionaries whose > length is greater than a particular value > > f=lambda seq, m: [ii for ii in range(0, len(seq)) if seq[ii]['Length'] > m][0] and it throws an error It doesn't throw an error (raise an exception) if there is at least one such dictionary in the list. If you see an error message for some other reason, you need to give more information. Like, what error? And for what code exactly. This problem can probably be tested with [{'Length':20.0}, {'Length':40.0}], so you don't need much code to ask the question. (Some people avoid giving actual details at a great cost. Even when asked nicely. Repeatedly. Such people cannot be helped. A mystery.) > can anyone suggest a way to do it? Use a generator expression instead of a list comprehension. Ask for the next value from the generator object, or a default when there is no next value. next((k for k,d in enumerate(ll) if d['Length'] > 23), 'sorry') ==> 1 next((k for k,d in enumerate(ll) if d['Length'] > 43), 'sorry') ==> 'sorry' It's often better to iterate over enumerate(ll) to get both the index and the value than over range(len(ll)) to get only the index. Or just write an actual elementary for-loop that returns the index when it finds the first good value: def f(dictses, tooshort): for k,d in enumerate(dictses): if d['Length'] > tooshort: return k return 'sorry' The default value from a function is None (if it falls through without a return statement). There is a tradition of returning -1 when no valid index is found. The one thing not to do is to use a non-index as if it was a valid index. An exception in an exceptional situation may be a good thing precisely because it calls attention to itself. From bnrj.rudra at live.com Mon Aug 15 02:23:04 2016 From: bnrj.rudra at live.com (Rudra Banerjee) Date: Sun, 14 Aug 2016 23:23:04 -0700 (PDT) Subject: compile a python3 project with autotools In-Reply-To: <18328113-c192-4e6f-bb60-3c0dea2f814b@googlegroups.com> References: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> <18328113-c192-4e6f-bb60-3c0dea2f814b@googlegroups.com> Message-ID: On Monday, August 15, 2016 at 12:39:33 AM UTC+2, Lawrence D?Oliveiro wrote: > On Monday, August 15, 2016 at 9:24:04 AM UTC+12, Rudra Banerjee wrote: > > but while running the application, I am getting error: > > (mkbib:14843): Gtk-CRITICAL **: New application windows must be added after > > the GApplication::startup signal has been emitted. > > Does the application actually fail? > > Because I see a lot of this sort of thing, which doesn?t seem to actually affect the running of the application. > > So I ignore it. :) It fails to close, if I run it from command line. Also, even if I close it by close button, the command line does not return. From rosuav at gmail.com Mon Aug 15 02:34:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Aug 2016 16:34:04 +1000 Subject: compile a python3 project with autotools In-Reply-To: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> References: <1ff05ec6-3414-4aa3-9bae-f6e966a7d0d3@googlegroups.com> Message-ID: On Mon, Aug 15, 2016 at 7:23 AM, wrote: > #!/usr/bin/env python3 > > import sys > sys.path.insert(1, '@pythondir@') > > from mkbib.main import mkbib > if __name__ == "__main__": > app = mkbib() > r = app.run() > sys.exit(r) > > > Now, the problem is it is compiling fine (make; make install goes without any error) > > make --silent > Making all in data > Making all in icons > Making all in hicolor > Making all in 48x48 > Making all in apps > Making all in scalable > Making all in apps > Making all in ui > Making all in src > Making all in Mkbib > > but while running the application, I am getting error: > (mkbib:14843): Gtk-CRITICAL **: New application windows must be added after the GApplication::startup signal has been emitted. > On Mon, Aug 15, 2016 at 4:23 PM, Rudra Banerjee wrote: > It fails to close, if I run it from command line. Also, even if I close it by close button, the command line does not return. When you build Mkbib, is it supposed to run the app immediately? That appears to be what's happening. Also, your code seems to have some odd redundancies: > src/Mkbib/main.py is the main program. This is structured as: > > class mkbib(Gtk.Application): > def __init__(self): > Gtk.Application.__init__(self) You shouldn't need this. All it does is call the superclass's initializer with the same arguments. The net result is that your class plays poorly with multiple inheritance (probably not going to hurt you here, but if ever you did use MI, this would bite you), and other than that, no significant effect. Omit this function altogether unless you actually need to initialize. > def new_window(self, filename=None): > window = Window(self) > window.show_all() What's the filename do? > def run(self): > self.new_window() > Gtk.main() > This is what's sometimes called a "procedure" - a function that doesn't have any meaningful return value. (When you run off the end of the function, Python assumes a "return None". Since it has no other return statements, this function will never return anything other than None.) But in mkbib.in, you call it as a function - "r = app.run()". Possibly not what you intended? Generally, I would expect the 'make' process to not be invoking your app, but perhaps I'm misunderstanding what mkbib.in does or when it's invoked. ChrisA From uri at speedy.net Mon Aug 15 05:47:20 2016 From: uri at speedy.net (Uri Even-Chen) Date: Mon, 15 Aug 2016 12:47:20 +0300 Subject: I'm looking for a part-time job in Python / Django In-Reply-To: References: Message-ID: Thank you Chris! Uri. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Sun, Aug 14, 2016 at 4:21 PM, Chris Angelico wrote: > On Sun, Aug 14, 2016 at 9:00 PM, Uri Even-Chen wrote: > > To python-list at python.org, > > > > I'm looking for a part-time job in Python / Django, do you know > anything? I > > live in Herzliya, Israel. I can't relocate but I can work from home. You > > can see my CV on LinkedIn. Please let me know if you have any job for me. > > > > Hi! This isn't a job posting list, but there are a few places that > accept job ads. One good one is the Python Job Board: > > https://www.python.org/jobs/ > > Browse that, and possibly subscribe to it; there are a lot of jobs > that would demand relocation, but every now and then you'll see a > remote opening. You could also search the web for "django jobs" and > see what you find. > > Good luck! There are definitely Python jobs out there, but you may > find that most of them require physical presence in the US or Europe. > But don't give up! > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From pe.fun18 at gmail.com Mon Aug 15 10:00:47 2016 From: pe.fun18 at gmail.com (Sickfit92) Date: Mon, 15 Aug 2016 07:00:47 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! Message-ID: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> 1. How long did it take you guys to master the language or, let me put it this way to completely get the hang and start writing code? 2. What made you want to learn python? 3. Was it difficult to learn the language? 4. Have you been able to get a job out of your new skill? Thank you guys! I'm pretty much Obsessed with python! From michael.selik at gmail.com Mon Aug 15 10:07:31 2016 From: michael.selik at gmail.com (Michael Selik) Date: Mon, 15 Aug 2016 14:07:31 +0000 Subject: Finding the first index in a list greater than a particular value In-Reply-To: References: <2ca5843b-142d-422d-8380-ab7f23ab541c@googlegroups.com> Message-ID: On Mon, Aug 15, 2016 at 2:01 AM Jussi Piitulainen < jussi.piitulainen at helsinki.fi> wrote: > There is a tradition of returning -1 when no valid index is found. > Sometimes it's better to break with tradition. Raise a ValueError. No silent errors, and all that Zen. From uri at speedy.net Mon Aug 15 10:24:10 2016 From: uri at speedy.net (Uri Even-Chen) Date: Mon, 15 Aug 2016 17:24:10 +0300 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: 1. After one/two months I knew the language, but after a year really knew it well. 2. I love Python but at first I didn't understand indentation in Python and it was confusing. 3. No. Except indentation. 4. Yes, I got 2 jobs in Python. But I also had to know JavaScript and jQuery. Uri. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Mon, Aug 15, 2016 at 5:00 PM, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me put it > this way to completely get the hang and start writing code? > > 2. What made you want to learn python? > > 3. Was it difficult to learn the language? > > 4. Have you been able to get a job out of your new skill? > > Thank you guys! I'm pretty much Obsessed with python! > > -- > https://mail.python.org/mailman/listinfo/python-list > From alister.ware at ntlworld.com Mon Aug 15 11:07:17 2016 From: alister.ware at ntlworld.com (alister) Date: Mon, 15 Aug 2016 15:07:17 GMT Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: On Mon, 15 Aug 2016 07:00:47 -0700, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me put > it this way to completely get the hang and start writing code? > > 2. What made you want to learn python? > > 3. Was it difficult to learn the language? > > 4. Have you been able to get a job out of your new skill? > > Thank you guys! I'm pretty much Obsessed with python! 1) time taken depends on how much code writing you do. is should not take more than a few hours to get the basics. then it boils down to finding a project or to that pique your interest & trying to implement them. you will probably keep using design concepts from previous languages that are not best practice for Python (not "Pythonic") one working you probably want to keep looking at re-factoring them to become more "pythonic" Some concepts took more time than others before I had the "Light bulb" moment, Comprehensions & decorators being the most notable although Lambda still escapes me, fortunately these can all be unrolled into larger functions so are not essential in the early stages 2) it was widely reported as being a suitable language for learning, I was previously an Assembler level language & always found c to be not much better. programming solely for fun I wanted something less restrictive 3) No, although getting out of habits formed from BASIC took some time -- Mr. Jones related an incident from "some time back" when IBM Canada Ltd. of Markham, Ont., ordered some parts from a new supplier in Japan. The company noted in its order that acceptable quality allowed for 1.5 per cent defects (a fairly high standard in North America at the time). The Japanese sent the order, with a few parts packaged separately in plastic. The accompanying letter said: "We don't know why you want 1.5 per cent defective parts, but for your convenience, we've packed them separately." -- Excerpted from an article in The (Toronto) Globe and Mail From rosuav at gmail.com Mon Aug 15 12:25:47 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Aug 2016 02:25:47 +1000 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: On Tue, Aug 16, 2016 at 12:00 AM, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me put it this way to completely get the hang and start writing code? Python wasn't my first language, so it was a lot easier for me to learn one more language than to learn programming from scratch. Basically, I picked up some document on the syntax (don't remember what - probably the official tutorial) and then started messing around with it, doing what I needed to do (which in this case was embedding it in a C++ program, which brought its own set of problems due to our non-standard toolchain). Now, how long did it take me to learn how to write Pythonic code? Don't know, and I'm still learning :) > 2. What made you want to learn python? I needed a language to embed in a program I was building. As it turned out, Python couldn't be made secure, so we ended up not going there, but it was still worth learning. Alternatively: The first time I met Python was in the late 1990s, on OS/2, and nothing beyond curiosity made me want to learn it. So I didn't. I toyed with it briefly, then went back to working in REXX. > 3. Was it difficult to learn the language? There is a strong family resemblance about [programming languages], and if you have all the details of a thousand at your finger ends, it is odd if you can't unravel the thousand and first. -- Sherlock Holmes, explaining what it means to be a "consulting detective" Languages come in strong families, and even across families, resemblances can be seen. When you are already familiar with several dozen, you can learn one more language fairly easily. Python has familial similarities with quite a few languages I know, so it wasn't hard. > 4. Have you been able to get a job out of your new skill? Yes, I have, and not counting the fact that I learned it at a previous job. There are plenty of Python jobs around; if you live in a good-sized US city, chances are pretty good that you can find work building web sites in Django, or messing around with big data in Pandas, or something. And even if your job isn't specifically Python, knowing a "glue language" makes you more useful. > Thank you guys! I'm pretty much Obsessed with python! So are most of us, I think, although I'm more "obsessed with computers, using Python". :) ChrisA From lists at mostrom.pp.se Mon Aug 15 12:39:26 2016 From: lists at mostrom.pp.se (Jan Erik =?utf-8?q?Mostr=C3=B6m?=) Date: Mon, 15 Aug 2016 18:39:26 +0200 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: <599E62EF-3F87-4637-9571-478D91D2D62E@mostrom.pp.se> On 15 Aug 2016, at 16:00, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me put > it this way to completely get the hang and start writing code? I'm probably not representative since I had used about 10-15 different language before looking at Python. To learn the basics: one flight from Europe to the US :D. The really understand the libraries, the language and write pythonic code ... still learning (some 10 years later). > 2. What made you want to learn python? Always want to try new language (and I didn't want to write any more Perl code ;) ) > 3. Was it difficult to learn the language? No, one of the easier. > 4. Have you been able to get a job out of your new skill? No, but I changed some of the work I was doing so I get to teach Python once in a while. = jem From grant.b.edwards at gmail.com Mon Aug 15 12:55:22 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 15 Aug 2016 16:55:22 +0000 (UTC) Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: On 2016-08-15, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me > put it this way to completely get the hang and start writing code? A few hours. The core language is pretty simple. [I was already fluent in a number of other programming languages -- if you haven't programmed before, it will take longer.] After 17 years working with Python, there are still some esoteric features (e.g. decorators) that I'm not very familiar with, and there will always be libraries to learn... > 2. What made you want to learn python? I needed an Windows app that could suck email messages from MS-Outlook via the COM interface and send them to an SMTP server. I found some example code showing how to do both transactions using Python (and Python was free). Though I had never used Python, I had heard of Python because it's what RedHat used to write their Linux installer (a program called Anaconda). > 3. Was it difficult to learn the language? Nope, very easy. > 4. Have you been able to get a job out of your new skill? Sort of. Over the past 17 years we've used Python for a number of projects where I work, and have since then hired somebody specifically because they had developed Python applications using particulary libraries. -- Grant Edwards grant.b.edwards Yow! Okay ... I'm going at home to write the "I HATE gmail.com RUBIK's CUBE HANDBOOK FOR DEAD CAT LOVERS" ... From no.email at nospam.invalid Mon Aug 15 12:58:58 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 15 Aug 2016 09:58:58 -0700 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: <87r39qufxp.fsf@jester.gateway.pace.com> Sickfit92 writes: > 1. How long did it take you guys to master the language or, let me put > it this way to completely get the hang and start writing code? Just a day or two, but I was already experienced with several similar languages. It would take longer for a beginning programmer. > 2. What made you want to learn python? Don't exactly remember, but at one point there had been an existing Pyton program that I had wanted to modify. > 3. Was it difficult to learn the language? No, not at all, I was pleasantly surprised at how easy it was. > 4. Have you been able to get a job out of your new skill? I've had some Python jobs, but I'd say I got them through being a widely experienced programmer in general, rather than just from knowing Python. Being a good programmer isn't just about knowing languages--it's mostly about being able to turn a higher-level problem it into a programming problem, then solve the programming problem. That's not the most difficult thing in the world, but you should take a wider approach than just learning languages. This book is very good (full text online too), though it doesn't use Python: https://mitpress.mit.edu/sicp/ From sohcahtoa82 at gmail.com Mon Aug 15 14:35:07 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Mon, 15 Aug 2016 11:35:07 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> On Monday, August 15, 2016 at 8:07:32 AM UTC-7, alister wrote: > On Mon, 15 Aug 2016 07:00:47 -0700, Sickfit92 wrote: > > > 1. How long did it take you guys to master the language or, let me put > > it this way to completely get the hang and start writing code? > > > Some concepts took more time than others before I had the "Light bulb" > moment, Comprehensions & decorators being the most notable although > Lambda still escapes me, fortunately these can all be unrolled into > larger functions so are not essential in the early stages > What helped me understand Lambdas is figuring out that they're really just a $1,000 term for a $5 concept. A lambda is just a single-line function without a name (Unless you assign it to one). A syntactic shortcut. def square_plus_one(x): return x ** 2 + 1 squared_plus_one_list = map(square_plus_one, some_list) is equivalent to: squared_plus_one_list = map(lambda x: x**2 + 1, some_list) From huey.y.jiang at gmail.com Mon Aug 15 14:41:30 2016 From: huey.y.jiang at gmail.com (huey.y.jiang at gmail.com) Date: Mon, 15 Aug 2016 11:41:30 -0700 (PDT) Subject: What is the trick between these? Message-ID: <923f0ac0-244b-45e8-9600-e20879eb8ba7@googlegroups.com> Hi All, I am trapped by these two funny things: class MyClass(upClass): def start(self): ***do_menu_here**** self.load_label_image() # ---> this works self.load_canvas_image() # ---> this does not work def load_label_image(self): img = PhotoImage(file="xx.gif") Label(self, image=img).pack(side=BOTTOM, expand=YES, fill=BOTH) return img def load_canvas_image(self): img = PhotoImage(file="xx.gif") self.canvas = Canvas(self, width=500, height=300, bg='white') self.canvas.pack(expand=YES, fill=BOTH) self.canvas.create_image(50, 0, image=img, anchor=NW) if __name__ == '__main__': root = Tk() root.mainloop() load_label_image() works. However, I cannot do BIND with it, because img has no method to do binding. So, I am trying load_canvas_image. Surprise, it does not work. These two defs are so similar, but Python complained that no attribute of load_canvas_image(). It is there, coexisting with load_label_image(). I cannot figure out what is wrong with it, can somebody tell me why? Thanks! From python at mrabarnett.plus.com Mon Aug 15 15:46:46 2016 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 15 Aug 2016 20:46:46 +0100 Subject: What is the trick between these? In-Reply-To: <923f0ac0-244b-45e8-9600-e20879eb8ba7@googlegroups.com> References: <923f0ac0-244b-45e8-9600-e20879eb8ba7@googlegroups.com> Message-ID: <84096193-d8b5-7925-40fb-b00ab8cfd0e7@mrabarnett.plus.com> On 2016-08-15 19:41, huey.y.jiang at gmail.com wrote: > Hi All, > > > I am trapped by these two funny things: > > class MyClass(upClass): > > def start(self): > ***do_menu_here**** > > self.load_label_image() # ---> this works > self.load_canvas_image() # ---> this does not work > > def load_label_image(self): > img = PhotoImage(file="xx.gif") > Label(self, image=img).pack(side=BOTTOM, expand=YES, fill=BOTH) > return img > > def load_canvas_image(self): > img = PhotoImage(file="xx.gif") > self.canvas = Canvas(self, width=500, height=300, bg='white') > self.canvas.pack(expand=YES, fill=BOTH) > self.canvas.create_image(50, 0, image=img, anchor=NW) > > if __name__ == '__main__': > root = Tk() > root.mainloop() > > load_label_image() works. However, I cannot do BIND with it, because img has no method to do binding. So, I am trying load_canvas_image. Surprise, it does not work. These two defs are so similar, but Python complained that no attribute of load_canvas_image(). It is there, coexisting with load_label_image(). > > I cannot figure out what is wrong with it, can somebody tell me why? Thanks! > If you can't bind to the image itself, then bind to the label you put it in instead. From __peter__ at web.de Mon Aug 15 15:57:45 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Aug 2016 21:57:45 +0200 Subject: What is the trick between these? References: <923f0ac0-244b-45e8-9600-e20879eb8ba7@googlegroups.com> Message-ID: huey.y.jiang at gmail.com wrote: > I am trapped by these two funny things: > > class MyClass(upClass): > > def start(self): > ***do_menu_here**** > > self.load_label_image() # ---> this works > self.load_canvas_image() # ---> this does not work > > def load_label_image(self): > img = PhotoImage(file="xx.gif") > Label(self, image=img).pack(side=BOTTOM, expand=YES, fill=BOTH) > return img > > def load_canvas_image(self): > img = PhotoImage(file="xx.gif") > self.canvas = Canvas(self, width=500, height=300, bg='white') > self.canvas.pack(expand=YES, fill=BOTH) > self.canvas.create_image(50, 0, image=img, anchor=NW) > > if __name__ == '__main__': > root = Tk() > root.mainloop() > > load_label_image() works. However, I cannot do BIND with it, because img > has no method to do binding. So, I am trying load_canvas_image. Surprise, > it does not work. These two defs are so similar, but Python complained > that no attribute of load_canvas_image(). It is there, coexisting with > load_label_image(). > > I cannot figure out what is wrong with it, can somebody tell me why? > Thanks! The only problem I see is that you do not keep a reference of the PhotoImage (as explained in MRAB's answer to your previous question). What exception do you encounter? Do not paraphrase, copy and paste the actual traceback. From irmen.NOSPAM at xs4all.nl Mon Aug 15 18:11:20 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Tue, 16 Aug 2016 00:11:20 +0200 Subject: docs on pythonhosted.org not updating with new version? Message-ID: <57b23e09$0$831$e4fe514c@news.xs4all.nl> Hi, as I've always done for a new release I've uploaded new versions of my package's documentation files, but they are not showing up on Pythonhosted.org - the previous version is still there. Is there something wrong with the update mechanism? I've been using "python3 setup.py build_sphinx upload_docs" to build and upload the docs and it comes back with a 200 Ok response from the pypi server. The same commands used to work fine on previous occasions... Irmen From lawrencedo99 at gmail.com Mon Aug 15 20:52:35 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 15 Aug 2016 17:52:35 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: On Tuesday, August 16, 2016 at 6:36:41 AM UTC+12, sohca... at gmail.com wrote: > What helped me understand Lambdas is figuring out that they're really just > a $1,000 term for a $5 concept. The excuse for the ?$1,000 term? is that it came from Alonzo Church?s ?-calculus, which was an alternative formulation for computability to Turing?s eponymous machines (and introduced around the same time, in the 1930s). In ?-calculus, everything is a function--even the integers are functions. In that situation, I think you would agree it is very much a ?$1,000 concept?. Lisp was the first programming language to include the word ?lambda? in a syntactic construct. Languages like Perl and JavaScript define anonymous functions in a simple way: just omit the name from an otherwise regular function definition. Python, on the other hand, introduces the special word ?lambda? for this purpose, eschewing its usual ?def?. Why? Something to do with GvR?s allergy to anonymous functions... From steve+python at pearwood.info Mon Aug 15 20:56:45 2016 From: steve+python at pearwood.info (Steven D'Aprano) Date: Tue, 16 Aug 2016 10:56:45 +1000 Subject: What is the trick between these? References: <923f0ac0-244b-45e8-9600-e20879eb8ba7@googlegroups.com> Message-ID: <57b264d0$0$1617$c3e8da3$5496439d@news.astraweb.com> On Tue, 16 Aug 2016 04:41 am, huey.y.jiang at gmail.com wrote: > Python complained that no attribute of load_canvas_image(). It is there, > coexisting with load_label_image(). > > I cannot figure out what is wrong with it, can somebody tell me why? > Thanks! No, we cannot, because we are not psychic. You don't show your actual code, and you don't show us the actual error. Do you want us to *guess* what code you are running, and guess what error you get? You can try it for yourself: take the code that you posted, and NOTHING ELSE, paste it into a new file, and try to run it. Can you run that code? No. How do you expect us to run it? - What is `upClass`? - There are missing imports. - Nothing appears to create a MyClass instance(). - Or call MyClass.start. My *guess* is that you have something like: AttributeError: 'MyClass' object has no attribute 'load_cavas_image' or something similar, which suggests that you have a typo in your code: class MyClass: def start(self): self.load_cavas_image() # Note spelling error. def load_canvas_image(self): ... Or maybe it is the other way around: class MyClass: def start(self): self.load_canvas_image() def laod_canvas_image(self): # Note spelling error. ... That's the best I can do. If this doesn't solve your problem, you will have to as a better question: http://sscce.org/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nad at python.org Mon Aug 15 22:59:33 2016 From: nad at python.org (Ned Deily) Date: Mon, 15 Aug 2016 22:59:33 -0400 Subject: {RELEASE] Python 3.6.0a4 is now available Message-ID: <24931ad1-1bb1-0894-c2f5-4c8835cab319@python.org> On behalf of the Python development community and the Python 3.6 release team, I'm happy to announce the availability of Python 3.6.0a4. 3.6.0a4 is the last of four planned alpha releases of Python 3.6, the next major release of Python. During the alpha phase, Python 3.6 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. You can find Python 3.6.0a4 here: https://www.python.org/downloads/release/python-360a4/ The next planned release of Python 3.6 will be 3.6.0b1, currently scheduled for 2016-09-12. 3.6.0b1 will mark the beginning of the beta phase of Python 3.6; at that time, feature development for 3.6 will be complete and the emphasis will change to fixing bugs and regressions. More information about the release schedule can be found here: https://www.python.org/dev/peps/pep-0494/ --Ned -- Ned Deily nad at python.org -- [] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From paul.nospam at rudin.co.uk Tue Aug 16 02:25:44 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Tue, 16 Aug 2016 07:25:44 +0100 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: <87inv1xmaf.fsf@rudin.co.uk> sohcahtoa82 at gmail.com writes: > On Monday, August 15, 2016 at 8:07:32 AM UTC-7, alister wrote: >> On Mon, 15 Aug 2016 07:00:47 -0700, Sickfit92 wrote: >> >> > 1. How long did it take you guys to master the language or, let me put >> > it this way to completely get the hang and start writing code? >> > > >> Some concepts took more time than others before I had the "Light bulb" >> moment, Comprehensions & decorators being the most notable although >> Lambda still escapes me, fortunately these can all be unrolled into >> larger functions so are not essential in the early stages >> > > > What helped me understand Lambdas is figuring out that they're really just a $1,000 term for a $5 concept. > > A lambda is just a single-line function without a name (Unless you assign it to one). A syntactic shortcut. > > def square_plus_one(x): > return x ** 2 + 1 > > squared_plus_one_list = map(square_plus_one, some_list) > > is equivalent to: > > squared_plus_one_list = map(lambda x: x**2 + 1, some_list) I realise that this is about understanding lambda, but it's worth noting in passing that we tend to write this sort of thing as: squared_plus_one_list = [x**2 + 1 for x in some_list] From lawrencedo99 at gmail.com Tue Aug 16 02:28:22 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 15 Aug 2016 23:28:22 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <87inv1xmaf.fsf@rudin.co.uk> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> Message-ID: <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> On Tuesday, August 16, 2016 at 6:26:01 PM UTC+12, Paul Rudin wrote: > sohcahtoa82 writes: >> squared_plus_one_list = map(lambda x: x**2 + 1, some_list) > > I realise that this is about understanding lambda, but it's worth noting > in passing that we tend to write this sort of thing as: > > squared_plus_one_list = [x**2 + 1 for x in some_list] The difference being that the ?map? function takes an iterable and returns an iterator. Why could this difference be important? From ian.g.kelly at gmail.com Tue Aug 16 02:34:29 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 16 Aug 2016 00:34:29 -0600 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: On Aug 15, 2016 6:57 PM, "Lawrence D?Oliveiro" wrote: Python, on the other hand, introduces the special word ?lambda? for this purpose, eschewing its usual ?def?. Why? Something to do with GvR?s allergy to anonymous functions... Actually, GvR is on record stating that he's never much cared for the name "lambda": http://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html?m=1 The name was picked by the contributor who wrote the implementation, apparently with little discussion. From lawrencedo99 at gmail.com Tue Aug 16 02:51:29 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Mon, 15 Aug 2016 23:51:29 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: <721f7adc-a8b1-45ff-a0d7-b8f07463d5d4@googlegroups.com> On Tuesday, August 16, 2016 at 6:34:53 PM UTC+12, Ian wrote: > On Aug 15, 2016 6:57 PM, "Lawrence D?Oliveiro" wrote: > > Python, on the other hand, introduces the special word ?lambda? for this > purpose, eschewing its usual ?def?. > > Why? Something to do with GvR?s allergy to anonymous functions... > > Actually, GvR is on record stating that he's never much cared for the name > "lambda": > http://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html?m=1 > > The name was picked by the contributor who wrote the implementation, > apparently with little discussion. I see. I thought I saw a mention somewhere else that Python lambdas were designed to be less functional than full def-style functions. But perhaps this limitation wasn?t intentional, just an inherent consequence of the fact that Python?s significant-whitespace rules only apply to statements, not expressions... From jussi.piitulainen at helsinki.fi Tue Aug 16 02:53:11 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 16 Aug 2016 09:53:11 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: Lawrence D?Oliveiro writes: > On Tuesday, August 16, 2016 at 6:26:01 PM UTC+12, Paul Rudin wrote: >> sohcahtoa82 writes: >>> squared_plus_one_list = map(lambda x: x**2 + 1, some_list) >> >> I realise that this is about understanding lambda, but it's worth noting >> in passing that we tend to write this sort of thing as: >> >> squared_plus_one_list = [x**2 + 1 for x in some_list] > > The difference being that the ?map? function takes an iterable and > returns an iterator. They tend to write that sort of thing as: squared_plus_one_thing = (x**2 + 1 for x in some_thing) > Why could this difference be important? Different comprehensions (pun!) and performance characteristics. From steve+comp.lang.python at pearwood.info Tue Aug 16 02:59:46 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 16 Aug 2016 16:59:46 +1000 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: <57b2b9e4$0$11101$c3e8da3@news.astraweb.com> On Tuesday 16 August 2016 16:28, Lawrence D?Oliveiro wrote: > On Tuesday, August 16, 2016 at 6:26:01 PM UTC+12, Paul Rudin wrote: >> sohcahtoa82 writes: >>> squared_plus_one_list = map(lambda x: x**2 + 1, some_list) >> >> I realise that this is about understanding lambda, but it's worth noting >> in passing that we tend to write this sort of thing as: >> >> squared_plus_one_list = [x**2 + 1 for x in some_list] > > The difference being that the ?map? function takes an iterable and returns an > iterator. > > Why could this difference be important? *shrug* Some members of the Python community, particularly GvR, prefer list comprehensions and generator expressions to using the Lisp-ish "map()" function. At least in the past, and possibly still, using a comprehension or a generator expression may be more efficient, as it can avoid the expense of a function call. The advice that used to be given (and which may still be valid, for all I know) is that if you have to write a function using lambda in order to use map(), it's usually faster to use a comprehension/genexp. E.g. out of the following: [len(x)+1 for x in sequence] list(map(lambda x: len(x) + 1, sequence)) the first will probably be faster as well as easier to read and write. -- Steve From ian.g.kelly at gmail.com Tue Aug 16 03:18:44 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 16 Aug 2016 01:18:44 -0600 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: On Aug 16, 2016 12:36 AM, "Lawrence D?Oliveiro" wrote: On Tuesday, August 16, 2016 at 6:26:01 PM UTC+12, Paul Rudin wrote: > sohcahtoa82 writes: >> squared_plus_one_list = map(lambda x: x**2 + 1, some_list) > > I realise that this is about understanding lambda, but it's worth noting > in passing that we tend to write this sort of thing as: > > squared_plus_one_list = [x**2 + 1 for x in some_list] The difference being that the ?map? function takes an iterable and returns an iterator. In Python 3, yes. However, assigning the result to the name "squared_plus_one_list" implies that the author was probably considering Python 2 code, in which map returns a list, so the comprehension is equivalent. Why could this difference be important? Assigning an iterator to a variable is usually considered un-Pythonic, because it's easy to accidentally consume the iterator resulting in difficult-to-diagnose bugs later. If you're immediately iterating over the result, then sure, use Python 3 map (or a generator expression). If you're going to reference it later then materializing it (or, if the iteration is cheap, creating a repeatable iterable) is preferred. Of course, the size of the list can also be a consideration here. Also, unless the mapped function is already defined (and preferably built-in), a generator expression or list comprehension is usually more readable and avoids the significant overhead of repeatedly calling a Python function. From ian.g.kelly at gmail.com Tue Aug 16 03:22:27 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 16 Aug 2016 01:22:27 -0600 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <721f7adc-a8b1-45ff-a0d7-b8f07463d5d4@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <721f7adc-a8b1-45ff-a0d7-b8f07463d5d4@googlegroups.com> Message-ID: On Aug 16, 2016 12:57 AM, "Lawrence D?Oliveiro" wrote: I see. I thought I saw a mention somewhere else that Python lambdas were designed to be less functional than full def-style functions. But perhaps this limitation wasn?t intentional, just an inherent consequence of the fact that Python?s significant-whitespace rules only apply to statements, not expressions... I think the primary reason for that is just that nobody has yet come up with a good syntax proposal for anonymous functions to contain statement lists. From marko at pacujo.net Tue Aug 16 03:29:39 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Aug 2016 10:29:39 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> <57b2b9e4$0$11101$c3e8da3@news.astraweb.com> Message-ID: <87wpjhgoik.fsf@elektro.pacujo.net> Steven D'Aprano : > E.g. out of the following: > > [len(x)+1 for x in sequence] > > list(map(lambda x: len(x) + 1, sequence)) > > the first will probably be faster as well as easier to read and write. It's mostly about idioms. Comprehensions belong to Python's core idioms, lambdas don't. In Scheme, I think it's the other way around. Too many Scheme programmers are infatuated with Scheme's powerful macros and bury Scheme's idiomatic, functional core under truckloads of foreign patterns -- just because they can. So use lambda in Scheme as much as you can. Avoid lambda in Python as much as you can. Marko From lawrencedo99 at gmail.com Tue Aug 16 03:32:27 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 16 Aug 2016 00:32:27 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <721f7adc-a8b1-45ff-a0d7-b8f07463d5d4@googlegroups.com> Message-ID: On Tuesday, August 16, 2016 at 7:22:42 PM UTC+12, Ian wrote: > On Aug 16, 2016 12:57 AM, "Lawrence D?Oliveiro" wrote: > > But perhaps this limitation wasn?t intentional, just an inherent > consequence of the fact that Python?s significant-whitespace rules only > apply to statements, not expressions... > > I think the primary reason for that is just that nobody has yet come up > with a good syntax proposal for anonymous functions to contain statement > lists. Only because Python makes it difficult, for the reason I mentioned. Other languages have had no trouble with it. For example, Perl and JavaScript use their existing function-definition constructs, just omitting the function name. From lawrencedo99 at gmail.com Tue Aug 16 03:34:46 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 16 Aug 2016 00:34:46 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: On Tuesday, August 16, 2016 at 6:53:24 PM UTC+12, Jussi Piitulainen wrote: > Lawrence D?Oliveiro writes: > >> Why could this difference be important? > > Different comprehensions (pun!) and performance characteristics. A potential difference in memory usage. When could this be important? Consider a situation where the iterable being mapped is returning records from a database query. If it were a million records, you might not want them all in memory at the same time. From lawrencedo99 at gmail.com Tue Aug 16 03:35:28 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 16 Aug 2016 00:35:28 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: On Tuesday, August 16, 2016 at 7:19:01 PM UTC+12, Ian wrote: > On Aug 16, 2016 12:36 AM, "Lawrence D?Oliveiro" wrote: > > The difference being that the ?map? function takes an iterable and returns > an iterator. > > In Python 3, yes. However, assigning the result to the name > "squared_plus_one_list" implies that the author was probably considering > Python 2 code, in which map returns a list, so the comprehension is > equivalent. Why was the change made? Because it was important. From auriocus at gmx.de Tue Aug 16 03:41:16 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 16 Aug 2016 09:41:16 +0200 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: Am 15.08.16 um 16:00 schrieb Sickfit92: > 1. How long did it take you guys to master the language or, let me > put it this way to completely get the hang and start writing code? You never learn a language completely. I'm using Python for 3 years, getting started was a matter of one or two days. But I have written code in ~10 different languages since 25 years. I started from here: https://wiki.python.org/moin/SimplePrograms > 2. What made you want to learn python? Numpy. I am writing scientific software, and numpy/scipy is becoming the new standard. Scientists shift from Matlab because Python is free. > 3. Was it difficult to learn the language? Comparable to other scripting languages. > 4. Have you been able to get a job out of your new skill? In my current job/profession, you choose the language. The objective is to get the job done, and often to throw away the program in the end. The faster you get the job done, the more successful you are. For many of those tasks, Python works very well. For others, I use different languages. Christian From jussi.piitulainen at helsinki.fi Tue Aug 16 03:46:04 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 16 Aug 2016 10:46:04 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: Lawrence D?Oliveiro writes: > On Tuesday, August 16, 2016 at 6:53:24 PM UTC+12, Jussi Piitulainen wrote: >> Lawrence D?Oliveiro writes: >> >>> Why could this difference be important? >> >> Different comprehensions (pun!) and performance characteristics. > > A potential difference in memory usage. When could this be important? > Consider a situation where the iterable being mapped is returning > records from a database query. If it were a million records, you might > not want them all in memory at the same time. That's what I meant by different performance characteristics, and I agree that it's an important difference. From rosuav at gmail.com Tue Aug 16 04:12:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Aug 2016 18:12:08 +1000 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: On Tue, Aug 16, 2016 at 5:18 PM, Ian Kelly wrote: > Also, unless the mapped function is already defined (and preferably > built-in), a generator expression or list comprehension is usually more > readable and avoids the significant overhead of repeatedly calling a Python > function. Don't know why "built-in" is significant, but most of the point of map() is to make use of an existing function: # Instead of lengths = (len(x) for x in items) # Use lengths = map(len, items) One problem: Not all languages have map() defined the same way. Built-in function or method? If function, what order are the arguments in? If method, is it a method on a function or an array/list? If you add extra arguments, are they treated as parallel arrays, or are they static arguments to the function - or are they not allowed at all? Actually, I've never seen map as a function method (which would be used as "len.map(items)"). All the others, I've seen in various different languages. Can anyone fill in the gap? ChrisA From marko at pacujo.net Tue Aug 16 04:17:57 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 16 Aug 2016 11:17:57 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: <87shu5gma2.fsf@elektro.pacujo.net> Chris Angelico : > most of the point of map() is to make use of an existing function: > > # Instead of > lengths = (len(x) for x in items) > # Use > lengths = map(len, items) Both methods are available and I have used each of them, but the former is probably always preferable. Marko From uri at speedy.net Tue Aug 16 06:02:46 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 16 Aug 2016 13:02:46 +0300 Subject: Pyjs Message-ID: Hi, Anybody used Pyjs? I have an error message: C:\Uri\Projects\pyjs\examples\helloworld>pyjsbuild Hello.py failed to create process. I want to create a new web app which will receive 2 inputs - a positive integer (default=2) and a number of digits (default=500), calculate the square root of the integer (on the client side) with digits after the dot and print it. I never used Pyjs but I used GWT and Python. Do you know how I start with developing this webapp? I want to do something simple, not more than 4 to 5 hours of work. If I can do it in less time - it's better. I developed this app in GWT (in Java) in 2010: http://chess-queens.sourceforge.net/ And this is the Python program that calculates the square root of a positive number: http://www.speedysoftware.com/uri/blog/my-first-programs-in-python/ By the way, I want to have a few buttons with numbers (or ) that when clicking on them it will set the input to a specific number, such as 2, 3, 4 or 9000000. And then the program will calculate the square root of this number. I also want to create another program which calculates e^x (default: x=1). The web app should be mobile-friendly. Thanks, Uri. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ From paul.nospam at rudin.co.uk Tue Aug 16 06:59:42 2016 From: paul.nospam at rudin.co.uk (Paul Rudin) Date: Tue, 16 Aug 2016 11:59:42 +0100 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <87inv1xmaf.fsf@rudin.co.uk> <767479ad-c6b8-4e54-bff8-945c96806e3b@googlegroups.com> Message-ID: <87eg5px9lt.fsf@rudin.co.uk> Lawrence D?Oliveiro writes: > On Tuesday, August 16, 2016 at 6:26:01 PM UTC+12, Paul Rudin wrote: >> sohcahtoa82 writes: >>> squared_plus_one_list = map(lambda x: x**2 + 1, some_list) >> >> I realise that this is about understanding lambda, but it's worth noting >> in passing that we tend to write this sort of thing as: >> >> squared_plus_one_list = [x**2 + 1 for x in some_list] > > The difference being that the ?map? function takes an iterable and returns an iterator. Ah well - that depends the python version. Since the OP used a name including the substring 'list' I assumed python 2. > Why could this difference be important? The main thing is to understand the difference, and then it's possible to reason about whether it matters in a given context. From catalinfest at gmail.com Tue Aug 16 09:14:52 2016 From: catalinfest at gmail.com (blue) Date: Tue, 16 Aug 2016 06:14:52 -0700 (PDT) Subject: mentor training python Romania with certification Message-ID: Hi. I'm from Romania. I need to update my skils under python. I need to find one mentor ( class, training ) to obtain one certified under python language. C?t?lin George Fe?til? From bluebox03 at gmail.com Tue Aug 16 09:34:05 2016 From: bluebox03 at gmail.com (tommy yama) Date: Tue, 16 Aug 2016 23:34:05 +1000 Subject: mentor training python Romania with certification In-Reply-To: References: Message-ID: Check it out. It is pretty basic material,though! https://www.python.org Go for it. On Tue, Aug 16, 2016 at 11:14 PM, blue wrote: > Hi. > > I'm from Romania. > I need to update my skils under python. > I need to find one mentor ( class, training ) to obtain one certified > under python language. > > C?t?lin George Fe?til? > -- > https://mail.python.org/mailman/listinfo/python-list > From irmen.NOSPAM at xs4all.nl Tue Aug 16 14:33:04 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Tue, 16 Aug 2016 20:33:04 +0200 Subject: docs on pythonhosted.org not updating with new version? In-Reply-To: <57b23e09$0$831$e4fe514c@news.xs4all.nl> References: <57b23e09$0$831$e4fe514c@news.xs4all.nl> Message-ID: <57b35c61$0$805$e4fe514c@news.xs4all.nl> On 16-8-2016 0:11, Irmen de Jong wrote: > Hi, > as I've always done for a new release I've uploaded new versions of my package's > documentation files, but they are not showing up on Pythonhosted.org - the previous > version is still there. > > Is there something wrong with the update mechanism? > > I've been using "python3 setup.py build_sphinx upload_docs" to build and upload the docs > and it comes back with a 200 Ok response from the pypi server. The same commands used to > work fine on previous occasions... Btw, manually uploading a zip file with the documentation via the web form on my project's Pypi page *did* work as advertised and updated the documentation. So it seems to me as if the setuptools upload_docs task is no longer working....? I tried to extract more information from setuptools but adding the -v option didn't result in more diagnostic info. So I have no idea what is happening with it behind the curtains yet. Irmen From alister.ware at ntlworld.com Tue Aug 16 14:46:02 2016 From: alister.ware at ntlworld.com (alister) Date: Tue, 16 Aug 2016 18:46:02 GMT Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: On Mon, 15 Aug 2016 11:35:07 -0700, sohcahtoa82 wrote: > On Monday, August 15, 2016 at 8:07:32 AM UTC-7, alister wrote: >> On Mon, 15 Aug 2016 07:00:47 -0700, Sickfit92 wrote: >> >> > 1. How long did it take you guys to master the language or, let me >> > put it this way to completely get the hang and start writing code? >> > > >> Some concepts took more time than others before I had the "Light bulb" >> moment, Comprehensions & decorators being the most notable although >> Lambda still escapes me, fortunately these can all be unrolled into >> larger functions so are not essential in the early stages >> > > > What helped me understand Lambdas is figuring out that they're really > just a $1,000 term for a $5 concept. > > A lambda is just a single-line function without a name (Unless you > assign it to one). A syntactic shortcut. > > def square_plus_one(x): > return x ** 2 + 1 > > squared_plus_one_list = map(square_plus_one, some_list) > > is equivalent to: > > squared_plus_one_list = map(lambda x: x**2 + 1, some_list) probably the cleanest example I have seen so far, & I still cant see the point the most enlightening thing here for me is your comet about it being a $1000 term for a $5 concept. I don't think I am missing anything by not bothering with them YMMV -- "The C Programming Language -- A language which combines the flexibility of assembly language with the power of assembly language." From bnrj.rudra at live.com Tue Aug 16 14:49:45 2016 From: bnrj.rudra at live.com (Rudra Banerjee) Date: Tue, 16 Aug 2016 11:49:45 -0700 (PDT) Subject: Check required modules using autotools Message-ID: <398b0450-2f0a-4b4b-a668-dc560ba840d0@googlegroups.com> Hi, I am compiling a python3 project using gnu-autotools. This is my configure.ac file (Please ignore the recursive Makefile, I have not managed to make the non-recursive one). AC_INIT([mkbib], [0.1],[],[mkbib]) AM_INIT_AUTOMAKE([1.9.6 dist-bzip2 subdir-objects]) AM_PATH_PYTHON([3.0]) dnl AX_PYTHON_MODULE([bibtexparser],[required]) AC_CONFIG_FILES([Makefile data/Makefile data/mkbib.desktop data/icons/Makefile data/icons/hicolor/Makefile data/icons/hicolor/48x48/Makefile data/icons/hicolor/48x48/apps/Makefile data/icons/hicolor/scalable/Makefile data/icons/hicolor/scalable/apps/Makefile data/ui/Makefile ]) AC_OUTPUT I am trying to check presence of bibtexparser module (https://pypi.python.org/pypi/bibtexparser/0.5.2); which is giving error: ./configure: line 2589: syntax error near unexpected token `bibtexparser,' ./configure: line 2589: `AX_PYTHON_MODULE(bibtexparser, fatal)' I have also tried: PKG_CHECK_MODULES([GTK], [gtk+-2.0]) PKG_CHECK_MODULES([GLIB], [glib-2.0]) PKG_CHECK_MODULES([bibtex], [bibtexparser]) But This is giving error as well. ./configure: line 2589: syntax error near unexpected token `GTK,' ./configure: line 2589: `PKG_CHECK_MODULES(GTK, gtk+-2.0)' From phd at phdru.name Tue Aug 16 15:16:14 2016 From: phd at phdru.name (Oleg Broytman) Date: Tue, 16 Aug 2016 21:16:14 +0200 Subject: SQLObject 3.1.0 Message-ID: <20160816191614.GB25012@phdru.name> Hello! I'm pleased to announce version 3.1.0, the first stable release of branch 3.1 of SQLObject. What's new in SQLObject ======================= Features -------- * Add UuidCol. * Add JsonbCol. Only for PostgreSQL. Requires psycopg2 >= 2.5.4 and PostgreSQL >= 9.2. * Add JSONCol, a universal json column. * For Python >= 3.4 minimal FormEncode version is now 1.3.1. * If mxDateTime is in use, convert timedelta (returned by MySQL) to mxDateTime.Time. Documentation ------------- * Developer's Guide extended to explain SQLObject architecture and how to create a new column type. * Fix URLs that can be found; remove missing links. * Rename reStructuredText files from *.txt to *.rst. Source code ----------- * Fix all `import *` using https://github.com/zestyping/star-destroyer. Tests ----- * Tests are now run at Circle CI. * Use pytest-cov for test coverage. Report test coverage via coveralls.io and codecov.io. * Install mxDateTime to run date/time tests with it. Contributor for this release is Lutz Steinborn. For a more complete list, please see the news: http://sqlobject.org/News.html What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Python 2.6, 2.7 or 3.4+ is required. Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: https://pypi.python.org/pypi/SQLObject/3.1.0 News and changes: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From lawrencedo99 at gmail.com Tue Aug 16 16:59:08 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Tue, 16 Aug 2016 13:59:08 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: On Wednesday, August 17, 2016 at 6:46:22 AM UTC+12, alister wrote: > I don't think I am missing anything by not bothering with them YMMV Here are some examples of that varying mileage. From steve+comp.lang.python at pearwood.info Wed Aug 17 02:07:38 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Aug 2016 16:07:38 +1000 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> On Wednesday 17 August 2016 04:46, alister wrote: > > squared_plus_one_list = map(lambda x: x**2 + 1, some_list) > > probably the cleanest example I have seen so far, & I still cant see the > point Hmmm. Well, let's have a look at some analogies with other kinds of values. Out of each pair of examples, *in general* would you prefer (A) or (B)? I realise that there are occasions where we might deliberate choose to assign an intermediate value to its own variable, but all else being equal, which would you prefer? #A alist = [] alist.append(2) alist.append(4) alist.append(8) process(alist) #B process([2, 4, 8]) #A value = 0 for i in range(100): value += 1 process(value) #B process(100) #A tmp = get_some_string() s = tmp[1] s += tmp[2] s += tmp[3] process(s) #B process(get_some_string()[1:4]) #A def callback(btn): return btn.do_the_thing(42) or default the_button.setcommand(callback) process(the_button) #B the_button.setcommand(lambda btn: btn.do_the_thing(42) or default) process(the_button) If you find yourself preferring B, B, B, A, you might ask yourself what makes a function different that you prefer to keep temporary functions around where they're not needed. -- Steve From priisdk at gmail.com Wed Aug 17 02:36:58 2016 From: priisdk at gmail.com (Poul Riis) Date: Tue, 16 Aug 2016 23:36:58 -0700 (PDT) Subject: pystl Message-ID: <85783414-0d45-4576-9725-8f0f893d6afe@googlegroups.com> Can someone deliver a minimal, fully working example with the pystl module, https://pypi.python.org/pypi/pystl/ The only example code mentioned is the following: with PySTL(?stl_test.stl?) as stl: stl.add_triangle( (0.0, 0.0, 0.5), (0.0, 1.0, 0.0), (1.0, 1.0, 0.5) ) but no matter what 'import'-statement I try I cannot make it work. I have installed the module and a program containing only the following line from pystl import PySTL runs without any error message. Poul Riis From steve+comp.lang.python at pearwood.info Wed Aug 17 02:39:30 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Aug 2016 16:39:30 +1000 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> Message-ID: <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> On Wednesday 17 August 2016 06:59, Lawrence D?Oliveiro wrote: > On Wednesday, August 17, 2016 at 6:46:22 AM UTC+12, alister wrote: >> I don't think I am missing anything by not bothering with them YMMV > > Here > > are some examples of that varying mileage. Quote: The Python syntax for conditional expressions (introduced in Python 2.5) is trueval if cond else falseval I think this is bloody awful. Why couldn?t they have adopted the standard C syntax, as used in a whole bunch of other C-derivative languages? cond ? trueval : falseval Because the C syntax is horrifically ugly, whereas the Python syntax is very close to real English syntax. "What will you do tonight?" "Go to the movies, if I finish work on time, otherwise just go home." Every time you read the C syntax, you lose another three minutes off your lifespan. That's how ugly it is. The background to the Python ternary operator is documented here: https://www.python.org/dev/peps/pep-0308/ Rather than ask why Python uses `trueval if cond else falseval`, you should ask why C uses `cond ? trueval : falseval`. Is that documented anywhere? -- Steve From steve+comp.lang.python at pearwood.info Wed Aug 17 02:49:16 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Aug 2016 16:49:16 +1000 Subject: pystl References: <85783414-0d45-4576-9725-8f0f893d6afe@googlegroups.com> Message-ID: <57b408ef$0$1503$c3e8da3$5496439d@news.astraweb.com> On Wednesday 17 August 2016 16:36, Poul Riis wrote: > Can someone deliver a minimal, fully working example with the pystl module, > https://pypi.python.org/pypi/pystl/ > > The only example code mentioned is the following: > > with PySTL(?stl_test.stl?) as stl: > stl.add_triangle( (0.0, 0.0, 0.5), (0.0, 1.0, 0.0), (1.0, 1.0, 0.5) ) > > > but no matter what 'import'-statement I try I cannot make it work. Don't make us guess. What have you tried, and what happens when you do? Cut and paste the *actual* code you try, and the *actual* results. http://mattgemmell.com/what-have-you-tried/ http://www.sscce.org/ > I have installed the module and a program containing only the following line > > from pystl import PySTL > > runs without any error message. Great. Then try this: from pystl import PySTL with PySTL(?stl_test.stl?) as stl: stl.add_triangle( (0.0, 0.0, 0.5), (0.0, 1.0, 0.0), (1.0, 1.0, 0.5) ) If it doesn't work, what does it do? -- Steve From auriocus at gmx.de Wed Aug 17 03:26:26 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 17 Aug 2016 09:26:26 +0200 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: Am 17.08.16 um 08:39 schrieb Steven D'Aprano: > Quote: > > The Python syntax for conditional expressions (introduced in Python 2.5) is > > trueval if cond else falseval > > I think this is bloody awful. Why couldn?t they have adopted the standard C > syntax, as used in a whole bunch of other C-derivative languages? > > cond ? trueval : falseval > > > > Because the C syntax is horrifically ugly, De gustibus non est disputandum. > whereas the Python syntax is very > close to real English syntax. > > "What will you do tonight?" > > "Go to the movies, if I finish work on time, otherwise just go home." This is a non-sequitur. You could as well say, in English: "If I finish work on time, I go to the movies, otherwise just go home." In fact, that's how an if-clause looks like in Python if work_done(): go_to_movies() else: go_home() Do you also struggle reading this? > Every time you read the C syntax, you lose another three minutes off your > lifespan. That's how ugly it is. It's an acquired taste. I'm not arguing that one is better than the other - it depends on what you are used to. I can read both without problems, but I avoid writing conditional expressions - if the subexpressions are not very simple. Christian From priisdk at gmail.com Wed Aug 17 04:56:35 2016 From: priisdk at gmail.com (Poul Riis) Date: Wed, 17 Aug 2016 01:56:35 -0700 (PDT) Subject: pystl In-Reply-To: <57b408ef$0$1503$c3e8da3$5496439d@news.astraweb.com> References: <85783414-0d45-4576-9725-8f0f893d6afe@googlegroups.com> <57b408ef$0$1503$c3e8da3$5496439d@news.astraweb.com> Message-ID: I tried the following: from pystl import PySTL with PySTL('stl_test.stl') as stl: stl.add_triangle((0,0,0),(1,0,0),(0,1,0)) I got the following error message: Traceback (most recent call last): File "C:/Users/pr/AppData/Local/Programs/Python/Python35-32/Lib/idlelib/pystl_module_test_1.py", line 3, in with PySTL('stl_test.stl') as stl: File "c:\users\pr\appdata\local\continuum\anaconda3\lib\site-packages\pystl\pystl.py", line 62, in __enter__ self.write_stl_header() File "c:\users\pr\appdata\local\continuum\anaconda3\lib\site-packages\pystl\pystl.py", line 75, in write_stl_header self.f.write(struct.pack("80s", header_str)) struct.error: argument for 's' must be a bytes object Poul Riis Den onsdag den 17. august 2016 kl. 08.49.29 UTC+2 skrev Steven D'Aprano: > On Wednesday 17 August 2016 16:36, Poul Riis wrote: > > > Can someone deliver a minimal, fully working example with the pystl module, > > https://pypi.python.org/pypi/pystl/ > > > > The only example code mentioned is the following: > > > > with PySTL(?stl_test.stl?) as stl: > > stl.add_triangle( (0.0, 0.0, 0.5), (0.0, 1.0, 0.0), (1.0, 1.0, 0.5) ) > > > > > > but no matter what 'import'-statement I try I cannot make it work. > > Don't make us guess. What have you tried, and what happens when you do? Cut and > paste the *actual* code you try, and the *actual* results. > > http://mattgemmell.com/what-have-you-tried/ > > http://www.sscce.org/ > > > > I have installed the module and a program containing only the following line > > > > from pystl import PySTL > > > > runs without any error message. > > Great. Then try this: > > from pystl import PySTL > with PySTL(?stl_test.stl?) as stl: > stl.add_triangle( (0.0, 0.0, 0.5), (0.0, 1.0, 0.0), (1.0, 1.0, 0.5) ) > > > > If it doesn't work, what does it do? > > > > -- > Steve From bc at freeuk.com Wed Aug 17 05:39:17 2016 From: bc at freeuk.com (BartC) Date: Wed, 17 Aug 2016 10:39:17 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: On 17/08/2016 07:39, Steven D'Aprano wrote: > On Wednesday 17 August 2016 06:59, Lawrence D?Oliveiro wrote: [Quote] > trueval if cond else falseval > "Go to the movies, if I finish work on time, otherwise just go home." Is there /any/ programming language that uses a similar form? Apart from Python, except that it only uses it for expressions, while the English uses statements. > Every time you read the C syntax, you lose another three minutes off your > lifespan. That's how ugly it is. > Rather than ask why Python uses `trueval if cond else falseval`, you should ask > why C uses `cond ? trueval : falseval`. Is that documented anywhere? I'm not fond of C's a ? b : c but the principle is sound. I generally use it as: (a | b | c) # /with/ the parentheses which is a direct equivalent of: if a then b else c fi # from algol 68 So it matches if-statements. And follows the same pattern as: (n | a, b, c ... | z) That is, you are given an index, or condition, and based on that you can skip to the option you want and ignore the rest. You don't start with the alternative that you may then have to discard. Anyway a?b:c was existing practice. At least the order of a,b,c could have been retained if not the exact syntax. -- Bartc From steve+comp.lang.python at pearwood.info Wed Aug 17 05:42:04 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 17 Aug 2016 19:42:04 +1000 Subject: pystl References: <85783414-0d45-4576-9725-8f0f893d6afe@googlegroups.com> <57b408ef$0$1503$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57b4316d$0$11119$c3e8da3@news.astraweb.com> On Wednesday 17 August 2016 18:56, Poul Riis wrote: > I tried the following: > > from pystl import PySTL > with PySTL('stl_test.stl') as stl: > stl.add_triangle((0,0,0),(1,0,0),(0,1,0)) > > I got the following error message: Ah, bad news I am afraid. That looks like a bug in pystl. I don't think there is anything you can do about it except: - make sure you have the most recent version of pystl; - make sure that it supports the version of Python you are using; - if so, report it as a bug to the pystl project. > > Traceback (most recent call last): > File > "C:/Users/pr/AppData/Local/Programs/Python/Python35-32/Lib/idlelib/pystl_module_test_1.py", > line 3, in > with PySTL('stl_test.stl') as stl: > File > "c:\users\pr\appdata\local\continuum\anaconda3\lib\site- packages\pystl\pystl.py", > line 62, in __enter__ > self.write_stl_header() > File > "c:\users\pr\appdata\local\continuum\anaconda3\lib\site- packages\pystl\pystl.py", > line 75, in write_stl_header > self.f.write(struct.pack("80s", header_str)) > struct.error: argument for 's' must be a bytes object -- Steve From jussi.piitulainen at helsinki.fi Wed Aug 17 07:24:05 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Wed, 17 Aug 2016 14:24:05 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: BartC writes: > On 17/08/2016 07:39, Steven D'Aprano wrote: >> Rather than ask why Python uses `trueval if cond else falseval`, you >> should ask why C uses `cond ? trueval : falseval`. Is that documented >> anywhere? > > I'm not fond of C's a ? b : c but the principle is sound. I generally [- -] > Anyway a?b:c was existing practice. At least the order of a,b,c could > have been retained if not the exact syntax. The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John McCarthy's 1960 paper on symbolic expressions, with an actual arrow glyph in place of hyphen-greater-than. http://www-formal.stanford.edu/jmc/recursive/node2.html (the paper famously titled "Part I" without any Part II, unless I mistake much.) It got implemented as (cond (c1 e1) (c2 e2) ... (cn en)), and I'm not sure when (if c e1 e2) became available as an alternative conditional expression in the Lisp family (called LISP back then, just like Python was called PYTHON if it was called at all). I think Algol 60 had something like "if c then e1 else e2" as an expression, also due to McCarthy. Keywords probably bold-faced. I'm perfectly happy with Python's conditional expression syntax, which cascades surprisingly well. I'm unhappy with C's, not sure why. I think I would like McCarthy's original syntax. (Dijkstra had a similar syntax but the clauses were not ordered. Each guard had to stand alone and any one branch with a true guard could be chosen. But that may have been a statement only.) From sivan at vitakka.co Wed Aug 17 07:31:08 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Wed, 17 Aug 2016 14:31:08 +0300 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> Message-ID: My first attempt at a large code base was with an Ubuntu project, the reason I learned Python was Ubuntu's sabdfl insisting on it be used throughout the OS (back in 2004 I was an Ubuntu developer contributing packages to main) . It was easy to get a job , but I started very early when there were not too much avail devs in my area at the time ;) Getting to know the language can take an afternoon or a short month. Mastering can take years, and with the rapid advancements with 3.5 and the future, you can't know it all. On the other hand, one can masters his own field of Python e.g. Data science, network programming, web, micro services. What's nice about python though that it's easy to ask for help and there's a wonderful community to help you become a 'master'. My ubuntu project made me create a co-routine engine in few days without ever knowing about it before with the help of people at #python on freenode. It's easy to implement complex things in Python. -Sivan On Mon, Aug 15, 2016 at 5:00 PM, Sickfit92 wrote: > 1. How long did it take you guys to master the language or, let me put it > this way to completely get the hang and start writing code? > > 2. What made you want to learn python? > > 3. Was it difficult to learn the language? > > 4. Have you been able to get a job out of your new skill? > > Thank you guys! I'm pretty much Obsessed with python! > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From __peter__ at web.de Wed Aug 17 08:44:00 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 17 Aug 2016 14:44 +0200 Subject: pystl References: <85783414-0d45-4576-9725-8f0f893d6afe@googlegroups.com> <57b408ef$0$1503$c3e8da3$5496439d@news.astraweb.com> Message-ID: Poul Riis wrote: > I tried the following: > > from pystl import PySTL > with PySTL('stl_test.stl') as stl: > stl.add_triangle((0,0,0),(1,0,0),(0,1,0)) > > I got the following error message: > > Traceback (most recent call last): > File > "C:/Users/pr/AppData/Local/Programs/Python/Python35-32/Lib/idlelib/pystl_module_test_1.py", > line 3, in > with PySTL('stl_test.stl') as stl: > File > "c:\users\pr\appdata\local\continuum\anaconda3\lib\site- packages\pystl\pystl.py", > line 62, in __enter__ > self.write_stl_header() > File > "c:\users\pr\appdata\local\continuum\anaconda3\lib\site- packages\pystl\pystl.py", > line 75, in write_stl_header > self.f.write(struct.pack("80s", header_str)) > struct.error: argument for 's' must be a bytes object You can fix this particular error by replacing the line header_str = '' with header_str = b'' in the write_stl_header() function which then becomes: def write_stl_header(self): if self.is_bin: header_str = b'' self.f.write(struct.pack("80s", header_str)) self.write_num_triangles_bin() else: self.f.write('solid ' + self.model_name + '\n' ) As this kind of error indicates that the module is primarily used/tested in Python 2 there may be other similar problems. From darcy at Vex.Net Wed Aug 17 10:15:08 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Wed, 17 Aug 2016 10:15:08 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <20160817101508.04400cdf@imp> On Wed, 17 Aug 2016 16:39:30 +1000 Steven D'Aprano wrote: > cond ? trueval : falseval > > Because the C syntax is horrifically ugly, whereas the Python syntax > is very close to real English syntax. Or, put another way, Python prefers English over line noise. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From marko at pacujo.net Wed Aug 17 10:57:40 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 17 Aug 2016 17:57:40 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <20160817101508.04400cdf@imp> Message-ID: <87eg5nh28r.fsf@elektro.pacujo.net> "D'Arcy J.M. Cain" : > On Wed, 17 Aug 2016 16:39:30 +1000 > Steven D'Aprano wrote: >> cond ? trueval : falseval >> >> Because the C syntax is horrifically ugly, whereas the Python syntax >> is very close to real English syntax. > > Or, put another way, Python prefers English over line noise. It can also be confusing. I recently introduced programming to someone using Python. Because of its similarity with English, they thought Python understood stuff like: if x or y == 0: ... for: if x == 0 or y == 0: ... or: if (x, y) is self.on_point(): ... for: if self.on_point(x, y): ... Somewhat analogously, I remember how confusing it was to learn formal logic in college. I was having a hard time getting the point of definitions like: (x ? y) is true iff x is true and y is true That's because I had learned in highschool that "x ? y" was just an abbreviation of "x and y". Marko From grant.b.edwards at gmail.com Wed Aug 17 11:12:10 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 17 Aug 2016 15:12:10 +0000 (UTC) Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <20160817101508.04400cdf@imp> <87eg5nh28r.fsf@elektro.pacujo.net> Message-ID: On 2016-08-17, Marko Rauhamaa wrote: > Somewhat analogously, I remember how confusing it was to learn formal > logic in college. I was having a hard time getting the point of > definitions like: > > (x ? y) is true iff x is true and y is true > > That's because I had learned in highschool that "x ? y" was just an > abbreviation of "x and y". It is. The expression "x ? y" is the same as "x and y". And that expression is true "iff x is true and y is true". It's just a sligtly more explicit way of writing the expression... -- Grant Edwards grant.b.edwards Yow! I joined scientology at at a garage sale!! gmail.com From marko at pacujo.net Wed Aug 17 11:33:05 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 17 Aug 2016 18:33:05 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <20160817101508.04400cdf@imp> <87eg5nh28r.fsf@elektro.pacujo.net> Message-ID: <87a8gbh0lq.fsf@elektro.pacujo.net> Grant Edwards : > On 2016-08-17, Marko Rauhamaa wrote: > >> Somewhat analogously, I remember how confusing it was to learn formal >> logic in college. I was having a hard time getting the point of >> definitions like: >> >> (x ? y) is true iff x is true and y is true >> >> That's because I had learned in highschool that "x ? y" was just an >> abbreviation of "x and y". > > It is. The expression "x ? y" is the same as "x and y". And that > expression is true "iff x is true and y is true". It's just a sligtly > more explicit way of writing the expression... Well, not quite. Notice the word "and" after "iff". That word is on a different plane than "?". The word "and" is on the semantic plane while "?" is part of the syntax. (Of course, that would be true even if "?" were written "and".) The formal sentence template (x ? y) contains the symbols "(", "?" and ")". However, "x" and "y" are not part of the formalism; rather, they are semantic placeholders for arbritrary formal sentences. The rest of the definition: is true iff x is true and y is true is plain-English semantics. In particular, the definition is *not* identical with the formal sentence: (x ? y) ? (x ? y) Marko From python at mrabarnett.plus.com Wed Aug 17 11:54:18 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 17 Aug 2016 16:54:18 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: On 2016-08-17 12:24, Jussi Piitulainen wrote: > BartC writes: > >> On 17/08/2016 07:39, Steven D'Aprano wrote: >>> Rather than ask why Python uses `trueval if cond else falseval`, you >>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>> anywhere? >> >> I'm not fond of C's a ? b : c but the principle is sound. I generally > > [- -] > >> Anyway a?b:c was existing practice. At least the order of a,b,c could >> have been retained if not the exact syntax. > > The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John McCarthy's > 1960 paper on symbolic expressions, with an actual arrow glyph in place > of hyphen-greater-than. > [snip] BCPL, the ancestor of C, had: a -> b, c From square.steve at gmail.com Wed Aug 17 12:35:36 2016 From: square.steve at gmail.com (Steve Simmons) Date: Wed, 17 Aug 2016 17:35:36 +0100 Subject: JSON confusion Message-ID: I'm trying to write a small utility to find the closest railway station to a given (UK) postcode but the result is in JSON and I'm not familiar with it. I've got as far as extracting the JSON object and I can print the first level elements ("success" and "result") but I've totally confused myself about how to delve into the rest of the data structure. Can anyone point me to a 'how-to' for tackling a fairly complex SJON object or give me some pointers. ... or maybe point out if I'm taking an unnecessarily complex approach. Initially, I'm Looking to extract 'stationname', 'distance' and one or two of the coordinate pairs. To be honest, I'd rather have some hints rather than the whole solution otherwise I'll not learn anything :-) SteveS def main(): import urllib import urllib.request import urllib.parse import urllib.response import json url ='https://data.gov.uk/data/api/service/transport/naptan_railway_stations/postcode?postcode=CT16+1ez&distance=2' req = urllib.request.urlopen(url) req_json = req.read() str_json = req_json.decode("utf-8") p_json = json.loads(str_json) print(p_json) print ('======================================') print(repr(p_json)) print('SUCCESS: ',repr(p_json['success'])) print ('======================================') print('RESULT : ',repr(p_json['result'])) if __name__ =="__main__": main() From jon+usenet at unequivocal.eu Wed Aug 17 12:49:27 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Wed, 17 Aug 2016 16:49:27 -0000 (UTC) Subject: JSON confusion References: Message-ID: On 2016-08-17, Steve Simmons wrote: > I'm trying to write a small utility to find the closest railway station > to a given (UK) postcode but the result is in JSON and I'm not familiar > with it. I've got as far as extracting the JSON object and I can print > the first level elements ("success" and "result") but I've totally > confused myself about how to delve into the rest of the data structure. > Can anyone point me to a 'how-to' for tackling a fairly complex SJON > object or give me some pointers. ... or maybe point out if I'm taking an > unnecessarily complex approach. Initially, I'm Looking to extract > 'stationname', 'distance' and one or two of the coordinate pairs. To be > honest, I'd rather have some hints rather than the whole solution > otherwise I'll not learn anything :-) SteveS It's not clear what the problem is. Does this help: print(p_json["result"][0]["stationname"]) print(p_json["result"][0]["latlong"]["coordinates"]) ? (To extract an item from a JSON object you index it with a string, e.g. ["foo"]; to extract an item from an array you use an integer, e.g. [0].) From square.steve at gmail.com Wed Aug 17 13:01:31 2016 From: square.steve at gmail.com (Steve Simmons) Date: Wed, 17 Aug 2016 18:01:31 +0100 Subject: JSON confusion In-Reply-To: References: Message-ID: On 17/08/2016 17:49, Jon Ribbens wrote: > On 2016-08-17, Steve Simmons wrote: >> I'm trying to write a small utility to find the closest railway station >> to a given (UK) postcode but the result is in JSON and I'm not familiar >> with it. I've got as far as extracting the JSON object and I can print >> the first level elements ("success" and "result") but I've totally >> confused myself about how to delve into the rest of the data structure. >> Can anyone point me to a 'how-to' for tackling a fairly complex SJON >> object or give me some pointers. ... or maybe point out if I'm taking an >> unnecessarily complex approach. Initially, I'm Looking to extract >> 'stationname', 'distance' and one or two of the coordinate pairs. To be >> honest, I'd rather have some hints rather than the whole solution >> otherwise I'll not learn anything :-) SteveS > It's not clear what the problem is. Yes, that was the problem! I wasn't sure if I was being stupid (yes!) about the Python or the JSON. Turns out, it was a bit of both ;-) > Does this help: > > print(p_json["result"][0]["stationname"]) > print(p_json["result"][0]["latlong"]["coordinates"]) > > ? Yes, immensely. > (To extract an item from a JSON object you index it with a string, > e.g. ["foo"]; to extract an item from an array you use an integer, > e.g. [0].) Thanks very much, hopefully that'll set me on my way. SteveS From jussi.piitulainen at helsinki.fi Wed Aug 17 13:19:58 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Wed, 17 Aug 2016 20:19:58 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: MRAB writes: > On 2016-08-17 12:24, Jussi Piitulainen wrote: >> BartC writes: >> >>> On 17/08/2016 07:39, Steven D'Aprano wrote: >>>> Rather than ask why Python uses `trueval if cond else falseval`, you >>>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>>> anywhere? >>> >>> I'm not fond of C's a ? b : c but the principle is sound. I generally >> >> [- -] >> >>> Anyway a?b:c was existing practice. At least the order of a,b,c could >>> have been retained if not the exact syntax. >> >> The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John >> McCarthy's 1960 paper on symbolic expressions, with an actual arrow >> glyph in place of hyphen-greater-than. >> > [snip] > > BCPL, the ancestor of C, had: > > a -> b, c Nice. Add a redundant pair of parentheses and it's the same. (When used as an expression, a final else-branch is mandatory-ish.) But C uses -> for something else, I think. And other languages use it for lambda expressions (succesfully, I think, but then they don't have it available for this purpose). From michael.selik at gmail.com Wed Aug 17 13:29:02 2016 From: michael.selik at gmail.com (Michael Selik) Date: Wed, 17 Aug 2016 17:29:02 +0000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> <87fuq7x4o1.fsf@jester.gateway.pace.com> Message-ID: On Sun, Aug 14, 2016 at 11:01 AM wrote: > On Sunday, August 14, 2016 at 7:09:47 AM UTC+1, Paul Rubin wrote: > > Steven D'Aprano writes: > > > If the Python community rallies around this "record" functionality and > > > takes to it like they took too namedtuple > > > > I like namedtuple and I think that it's a feature that they're modified > > by making a new copy. I know that has overhead but it's palpably > > bug-avoidant. I've used them extensively in some programs and they took > > a considerable burden off my mind compared to using something like > > structs or records. > > You might find this https://glyph.twistedmatrix.com/2016/08/attrs.html an > interesting read. > I disagree with a few points from that blog post. 1. I don't mind typing so much. I like to be explicit. The attrs library uses some overly-concise abbreviations. For example, what's the meaning of ``@attrs.s`` or ``attrs.ib``? 2. When inheriting from a namedtuple, I use the same class name for the base and the child, so my reprs look good. 3. I don't bother to fieldnames.split() when passing fieldnames as a space-separated string, because the split is unnecessary. 4. I *like* that namedtuple is backwards-compatible with tuples, so that refactoring from tuples to namedtuples is easy. 5. Inheritance is useful. Sure, there are times it fails, but that's true for any technique. From python at mrabarnett.plus.com Wed Aug 17 13:52:22 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 17 Aug 2016 18:52:22 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> On 2016-08-17 18:19, Jussi Piitulainen wrote: > MRAB writes: > >> On 2016-08-17 12:24, Jussi Piitulainen wrote: >>> BartC writes: >>> >>>> On 17/08/2016 07:39, Steven D'Aprano wrote: >>>>> Rather than ask why Python uses `trueval if cond else falseval`, you >>>>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>>>> anywhere? >>>> >>>> I'm not fond of C's a ? b : c but the principle is sound. I generally >>> >>> [- -] >>> >>>> Anyway a?b:c was existing practice. At least the order of a,b,c could >>>> have been retained if not the exact syntax. >>> >>> The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John >>> McCarthy's 1960 paper on symbolic expressions, with an actual arrow >>> glyph in place of hyphen-greater-than. >>> >> [snip] >> >> BCPL, the ancestor of C, had: >> >> a -> b, c > > Nice. Add a redundant pair of parentheses and it's the same. (When used > as an expression, a final else-branch is mandatory-ish.) > > But C uses -> for something else, I think. And other languages use it > for lambda expressions (succesfully, I think, but then they don't have > it available for this purpose). > C uses "->" for dereferencing a pointer to the member of a struct. If "p" points to a struct (record), then "*p" is that struct, and if that struct has a member (field) "m", then that member can be accessed by "(*p)->m" (the parens are necessary because of the operator precedence). This can be abbreviated to "p->m". Pascal, on the other hand, dereferences with a postfixed "^", so that would be "p^.m". From marko at pacujo.net Wed Aug 17 13:58:01 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 17 Aug 2016 20:58:01 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: <87inuzb7me.fsf@elektro.pacujo.net> MRAB : > C uses "->" for dereferencing a pointer to the member of a struct. What "->" is for C, "." is Python and Java. Python doesn't have C's ".". Marko From grant.b.edwards at gmail.com Wed Aug 17 14:00:55 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 17 Aug 2016 18:00:55 +0000 (UTC) Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: On 2016-08-17, MRAB wrote: > C uses "->" for dereferencing a pointer to the member of a struct. > > If "p" points to a struct (record), then "*p" is that struct, and if > that struct has a member (field) "m", then that member can be accessed > by "(*p)->m" (the parens are necessary because of the operator > precedence). ITYM (*p).m > This can be abbreviated to "p->m". > > Pascal, on the other hand, dereferences with a postfixed "^", so that > would be "p^.m". -- Grant Edwards grant.b.edwards Yow! Is this an out-take at from the "BRADY BUNCH"? gmail.com From tjreedy at udel.edu Wed Aug 17 14:27:38 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 14:27:38 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: On 8/17/2016 2:39 AM, Steven D'Aprano wrote: > On Wednesday 17 August 2016 06:59, Lawrence D?Oliveiro wrote: >> Here >> >> are some examples of that varying mileage. > > Quote: > > The Python syntax for conditional expressions (introduced in Python 2.5) is > > trueval if cond else falseval > > I think this is bloody awful. When this was initially considered, this was the majority view. > Why couldn?t they have adopted the standard C > syntax, as used in a whole bunch of other C-derivative languages? > cond ? trueval : falseval That particular syntax was not really considered. At least 10 versions using 'if', 'then', 'else', and other tokens were. They all had the problem of requiring a new keyword such as 'then' or some other innovation. > Because the C syntax is horrifically ugly, whereas the Python syntax is very > close to real English syntax. > > "What will you do tonight?" > > "Go to the movies, if I finish work on time, otherwise just go home." "If I finish work on on time, go to the movies, otherwise just go home." is also real English syntax, and to me, more graceful. It is certainly more neutral among the alternatives. The inverted version implies a clear preference for the first alternative. It would be an interesting exercise to see which order for ternary expressions is more common in some large corpus of English text. > Every time you read the C syntax, you lose another three minutes off your > lifespan. That's how ugly it is. Every time I write or read the Python syntax chosen, I lose time rearranging the terms. > The background to the Python ternary operator is documented here: > > https://www.python.org/dev/peps/pep-0308/ What the *current* version removed from an earlier version is that there was a clear community consensus against the condition-in-the-middle syntax Guido proposed and for some version of "if condition then True-alternative else False-alternative". Where consensus was lacking was which of multiple 'normal order' alternatives to choose. Part of the problem was a lack of knowledge of which alternative Guido might accept. In any case, a runoff vote among the top contenders was not allowed. > Rather than ask why Python uses `trueval if cond else falseval`, you should ask > why C uses `cond ? trueval : falseval`. Is that documented anywhere? The ordering of the if-then-else terms is obvious. '?' ending a question (in English) is obvious. ';' was already used to end statememts and could not be used here. Could ',' have been used instead of ':'? I am not sure, but it is used elsewhere in C. ':' is the only other within-Englich-sentence separator available. The ':' naturally translates to 'else', which was already a keyword. There was consensus on this. The problem for a pythonic (wordy) version of the C expression is that the word marker for questions, 'if', normally begins rather than ending a question*. So "cond if trueval else falseval" is likely to be misinterpreted. Hence proposals for "cond then trueval else falseval" and "if cond then trueval else falseval" and other variations. * In a construction like "The cock crows? If so, I must go, else I would tarry with thee longer." the 'if' follows the question, yet cannot standalone but must be followed by something referring back to the question. -- Terry Jan Reedy From random832 at fastmail.com Wed Aug 17 14:39:40 2016 From: random832 at fastmail.com (Random832) Date: Wed, 17 Aug 2016 14:39:40 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> On Wed, Aug 17, 2016, at 14:27, Terry Reedy wrote: > That particular syntax was not really considered. At least 10 versions > using 'if', 'then', 'else', and other tokens were. > > They all had the problem of requiring a new keyword such as 'then' or > some other innovation. Why not just if(cond, trueval, falseval), a la Visual Basic? It's too late to change now, but I'm curious as to whether it was considered or not. From marko at pacujo.net Wed Aug 17 14:43:41 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 17 Aug 2016 21:43:41 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <87eg5nb5ia.fsf@elektro.pacujo.net> Terry Reedy : > On 8/17/2016 2:39 AM, Steven D'Aprano wrote: > "If I finish work on on time, go to the movies, otherwise just go home." > is also real English syntax, and to me, more graceful. It is certainly > more neutral among the alternatives. The inverted version implies a > clear preference for the first alternative. > > It would be an interesting exercise to see which order for ternary > expressions is more common in some large corpus of English text. Python's ternary expression has a distinct Perl flavor to it. However, the problem with the "then" keyword was valid. Also, Python's comprehensions already had a postfix "if". Personally, I'd normally steer clear of ternary conditionals both in C and Python. This reminds me of a discussion I had yesterday about why Scheme can't implement a proper try/finally construct. That's because Scheme supports continuations; nothing is really final. Python would gain a similar power if there were a way to cancel exceptions: try: do_something() except ValueError: retry 123 where: def do_something(): def_value = raise ValueError a += def_value What Python would then need is a try/nevermind: resource = grab_it() while True: try: resource.operate() finally: resource.release() break nevermind: resource.take_back() Marko From python at mrabarnett.plus.com Wed Aug 17 15:39:08 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 17 Aug 2016 20:39:08 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <87inuzb7me.fsf@elektro.pacujo.net> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> <87inuzb7me.fsf@elektro.pacujo.net> Message-ID: On 2016-08-17 18:58, Marko Rauhamaa wrote: > MRAB : > >> C uses "->" for dereferencing a pointer to the member of a struct. > > What "->" is for C, "." is Python and Java. > > Python doesn't have C's ".". > C has both stacked-allocated and heap-allocated records. That's not the case with Python or Java. Delphi, which is basically an extended Pascal, allows you to omit the "^" when dereferencing a pointer to a field in a record, so you can write "p.m" instead of "p^.m" when "p" is a pointer to a record. From python at mrabarnett.plus.com Wed Aug 17 15:39:34 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 17 Aug 2016 20:39:34 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: <3b5cf9e5-16fe-8616-7bbb-2edf0600fe6d@mrabarnett.plus.com> On 2016-08-17 19:00, Grant Edwards wrote: > On 2016-08-17, MRAB wrote: > >> C uses "->" for dereferencing a pointer to the member of a struct. >> >> If "p" points to a struct (record), then "*p" is that struct, and if >> that struct has a member (field) "m", then that member can be accessed >> by "(*p)->m" (the parens are necessary because of the operator >> precedence). > > ITYM (*p).m > Correct. >> This can be abbreviated to "p->m". >> >> Pascal, on the other hand, dereferences with a postfixed "^", so that >> would be "p^.m". > From python at mrabarnett.plus.com Wed Aug 17 15:41:28 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 17 Aug 2016 20:41:28 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> Message-ID: On 2016-08-17 19:39, Random832 wrote: > On Wed, Aug 17, 2016, at 14:27, Terry Reedy wrote: >> That particular syntax was not really considered. At least 10 versions >> using 'if', 'then', 'else', and other tokens were. >> >> They all had the problem of requiring a new keyword such as 'then' or >> some other innovation. > > Why not just if(cond, trueval, falseval), a la Visual Basic? > > It's too late to change now, but I'm curious as to whether it was > considered or not. > Well, it looks too much like a function call, which don't short-circuit. From marko at pacujo.net Wed Aug 17 15:50:43 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 17 Aug 2016 22:50:43 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> Message-ID: <874m6jb2ek.fsf@elektro.pacujo.net> MRAB : > On 2016-08-17 19:39, Random832 wrote: >> Why not just if(cond, trueval, falseval), a la Visual Basic? > > Well, it looks too much like a function call, which don't > short-circuit. Well, in Scheme, everything looks like a function call: (define (deflate x) (if (> 0 x) (1+ x) (1- x))) Marko From tjreedy at udel.edu Wed Aug 17 16:25:03 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 16:25:03 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/17/2016 2:07 AM, Steven D'Aprano wrote: > I realise that there are occasions where we might deliberate choose to assign > an intermediate value to its own variable, but all else being equal, which > would you prefer? > > #A > alist = [] > alist.append(2) > alist.append(4) > alist.append(8) > process(alist) > > #B > process([2, 4, 8]) > > #A > value = 0 > for i in range(100): > value += 1 > process(value) > > #B > process(100) > > #A > tmp = get_some_string() > s = tmp[1] > s += tmp[2] > s += tmp[3] > process(s) > > #B > process(get_some_string()[1:4]) Up to here, #A is a useless and stupid. Have you seen such code written? > #A > def callback(btn): > return btn.do_the_thing(42) or default > the_button.setcommand(callback) > process(the_button) > > #B > the_button.setcommand(lambda btn: btn.do_the_thing(42) or default) > process(the_button) This example is *not* parallel to the other 3. Here, A is useful real code and might be preferred for multiple reasons. > If you find yourself preferring B, B, B, A, you might ask yourself what makes a > function different that you prefer to keep temporary functions around where > they're not needed. When 'callback' is set as the command of the button, it is not temporary, but must remain as long as the button remains. Only the name binding is (possibly) disposable. One may want the function to have a meaningful name that says what it does. One may want the function to have a name for tracebacks. In a framework that passes the button to button callbacks (not tk, unfortunately), the same callback might be used for multiple buttons. (In tk, one must create a wrapper of callback for each button.) The example is misleading in that the return value of a Button callback is likely irrelevant. (This is true of all tkinter/tk callbacks that I can think of.) More realistic is #A def callback(btn): btn.do_the_thing(42) the_button.setcommand(callback) #A' def cb(btn): btn.do_the_thing(42) the_button.setcommand(cb) #B the_button.setcommand(lambda btn: btn.do_the_thing(42) I have written code like #B, but #A is more correct, to me, in returning None instead of the ignored value of the call. Beginners often do not understand that the body of a lambda expression is evaluated in a new local namespace, and only when the resulting function is called, the same as with a def statement. They then neglect to capture current values when writing lambda expressions in a for loop. In many cases, there is a third alternative using functools.partial. Production gui code is typically written with classes. This adds the possibility of using bound methods. This example require the_button to be defined elsewhere. Typically, the command can and should be set when the button is defined, along with other options. Having 'cb' pre-defined may make the defining call more readable and the arg list fall on one line rather than two. Here is a real example from idlelib/help.py def toc_menu(self, text): "Create table of contents as drop-down menu." toc = Menubutton(self, text='TOC') drop = Menu(toc, tearoff=False) for lbl, dex in text.parser.toc: drop.add_command(label=lbl, command=lambda dex=dex:text.yview(dex)) toc['menu'] = drop return toc The local names 'lbl' and 'dex' were chosen short so that the long line would be exactly at the limit of 79 chars. Here is a rewrite of the for loop. for section_name, line_number in text.parser.toc: def goto(line=line_number): text.yview(line) drop.add_command(label=section_name, command=goto) To me, this is much better and I intend to commit it. Thank you for prodding me to think through how bad the lambda form can be and to rewrite the loop so I don't cringe reading it. -- Terry Jan Reedy From ben.usenet at bsb.me.uk Wed Aug 17 16:41:34 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 17 Aug 2016 21:41:34 +0100 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <87ziobt9fl.fsf@bsb.me.uk> Jussi Piitulainen writes: > BartC writes: > >> On 17/08/2016 07:39, Steven D'Aprano wrote: >>> Rather than ask why Python uses `trueval if cond else falseval`, you >>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>> anywhere? >> >> I'm not fond of C's a ? b : c but the principle is sound. I generally > > [- -] > >> Anyway a?b:c was existing practice. At least the order of a,b,c could >> have been retained if not the exact syntax. > > The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John McCarthy's > 1960 paper on symbolic expressions, with an actual arrow glyph in place > of hyphen-greater-than. And BCPL (Martin Richards 1967) took the same arrow and comma syntax. BCPL spawned B which led to C, but in B Thompson used ? and : but kept the right-to-left binding. I think the change was unfortunate because the arrow works well in various layouts and looks much better when chained (though that might just be my bias from being a BCPL coder from way back). -- Ben. From ben.usenet at bsb.me.uk Wed Aug 17 16:44:58 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 17 Aug 2016 21:44:58 +0100 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: <87twejt99x.fsf@bsb.me.uk> MRAB writes: > On 2016-08-17 18:19, Jussi Piitulainen wrote: >> MRAB writes: >> >>> On 2016-08-17 12:24, Jussi Piitulainen wrote: >>>> BartC writes: >>>> >>>>> On 17/08/2016 07:39, Steven D'Aprano wrote: >>>>>> Rather than ask why Python uses `trueval if cond else falseval`, you >>>>>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>>>>> anywhere? >>>>> >>>>> I'm not fond of C's a ? b : c but the principle is sound. I generally >>>> >>>> [- -] >>>> >>>>> Anyway a?b:c was existing practice. At least the order of a,b,c could >>>>> have been retained if not the exact syntax. >>>> >>>> The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John >>>> McCarthy's 1960 paper on symbolic expressions, with an actual arrow >>>> glyph in place of hyphen-greater-than. >>>> >>> [snip] >>> >>> BCPL, the ancestor of C, had: >>> >>> a -> b, c >> >> Nice. Add a redundant pair of parentheses and it's the same. (When used >> as an expression, a final else-branch is mandatory-ish.) >> >> But C uses -> for something else, I think. And other languages use it >> for lambda expressions (succesfully, I think, but then they don't have >> it available for this purpose). >> > C uses "->" for dereferencing a pointer to the member of a struct. Slightly better wording: it uses -> to access a struct member via a pointer to the struct. > If "p" points to a struct (record), then "*p" is that struct, and if > that struct has a member (field) "m", then that member can be accessed > by "(*p)->m" (the parens are necessary because of the operator > precedence). I think you meant (*p).m here because you go on to correct say that -> offers a shorthand for this rather messy access: > This can be abbreviated to "p->m". > > Pascal, on the other hand, dereferences with a postfixed "^", so that > would be "p^.m". > -- Ben. From tjreedy at udel.edu Wed Aug 17 16:59:50 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 16:59:50 -0400 Subject: JSON confusion In-Reply-To: References: Message-ID: On 8/17/2016 12:35 PM, Steve Simmons wrote: > I'm trying to write a small utility to find the closest railway station > to a given (UK) postcode but the result is in JSON and I'm not familiar > with it. I've got as far as extracting the JSON object and I can print > the first level elements ("success" and "result") but I've totally > confused myself about how to delve into the rest of the data structure. > Can anyone point me to a 'how-to' for tackling a fairly complex SJON > object or give me some pointers. ... or maybe point out if I'm taking an > unnecessarily complex approach. Initially, I'm Looking to extract > 'stationname', 'distance' and one or two of the coordinate pairs. To be > honest, I'd rather have some hints rather than the whole solution > otherwise I'll not learn anything :-) SteveS def main(): > import urllib > import urllib.request > import urllib.parse > import urllib.response > import json add import pprint > url > ='https://data.gov.uk/data/api/service/transport/naptan_railway_stations/postcode?postcode=CT16+1ez&distance=2' > req = urllib.request.urlopen(url) > > req_json = req.read() > str_json = req_json.decode("utf-8") > > p_json = json.loads(str_json) This result is nexted dicts and lists > print(p_json) > print ('======================================') > print(repr(p_json)) > print('SUCCESS: ',repr(p_json['success'])) > print ('======================================') > print('RESULT : ',repr(p_json['result'])) Replace prints above with pprint.pprint(p_json) and you will see the better formatted {'result': [{'atcocode': '9100DOVERP', 'crscode': 'DVP', 'distance': 881.148432224, 'latlong': {'coordinates': [1.3052936134036113, 51.12569875059288], 'crs': {'properties': {'name': 'EPSG4326'}, 'type': 'name'}, 'type': 'Point'}, 'ospoint': {'coordinates': [631380.0, 141464.0], 'crs': {'properties': {'name': 'EPSG27700'}, 'type': 'name'}, 'type': 'Point'}, 'stationname': 'Dover Priory Rail Station', 'tiploccode': 'DOVERP'}], 'success': True} Use this in combination with Jon Ribbens' answer. -- Terry Jan Reedy From tjreedy at udel.edu Wed Aug 17 17:48:33 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 17:48:33 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <87eg5nb5ia.fsf@elektro.pacujo.net> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <87eg5nb5ia.fsf@elektro.pacujo.net> Message-ID: On 8/17/2016 2:43 PM, Marko Rauhamaa wrote: > Terry Reedy : > >> On 8/17/2016 2:39 AM, Steven D'Aprano wrote: >> "If I finish work on on time, go to the movies, otherwise just go home." >> is also real English syntax, and to me, more graceful. It is certainly >> more neutral among the alternatives. The inverted version implies a >> clear preference for the first alternative. >> >> It would be an interesting exercise to see which order for ternary >> expressions is more common in some large corpus of English text. > > Python's ternary expression has a distinct Perl flavor to it. One of the ironies (or puzzles) of Guido's choice is that he once condemned Perl's 'value if cond' as wretched. I never got an answer as to whether he changed his mind on that or if adding 'else otherval' made it unwretched. Whatever the case, it does not matter to me now. I once 'boycotted' the ternary but now use it where I think it better than the alternative. -- Terry Jan Reedy From tjreedy at udel.edu Wed Aug 17 17:59:31 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 17:59:31 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <1471459180.1771777.698306201.4E870F99@webmail.messagingengine.com> Message-ID: On 8/17/2016 2:39 PM, Random832 wrote: > On Wed, Aug 17, 2016, at 14:27, Terry Reedy wrote: >> That particular syntax was not really considered. At least 10 versions >> using 'if', 'then', 'else', and other tokens were. >> >> They all had the problem of requiring a new keyword such as 'then' or >> some other innovation. > > Why not just if(cond, trueval, falseval), a la Visual Basic? > > It's too late to change now, but I'm curious as to whether it was > considered or not. It is already valid Python, parsed as 'if' 'tuple' >>> if(1,2): 3 3 A space after 'if' is not optional if the next char is an identifier char, as in 'ifx' as this is parsed as the identifier 'ifx', not 'if' 'x'. Anyone can define an iff(cond, true_func, false_func) function def iff(cond, f_true, f_false): return (f_true if cond else f_false)() or a non-short-circuiting iff(cond, true_val, false_val) function, but it hardly seems worth the effort. -- Terry Jan Reedy From lawrencedo99 at gmail.com Wed Aug 17 19:02:45 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 16:02:45 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thursday, August 18, 2016 at 6:28:06 AM UTC+12, Terry Reedy wrote: > >> Why couldn?t they have adopted the standard C >> syntax, as used in a whole bunch of other C-derivative languages? >> cond ? trueval : falseval > > That particular syntax was not really considered. At least 10 versions > using 'if', 'then', 'else', and other tokens were. > > They all had the problem of requiring a new keyword such as 'then' or > some other innovation. What was wrong with adopting the C syntax? That would have introduced ??? as a new symbol. What impact what that have had on existing Python code? None that I can think of. From lawrencedo99 at gmail.com Wed Aug 17 19:05:22 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 16:05:22 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: On Thursday, August 18, 2016 at 5:53:14 AM UTC+12, MRAB wrote: > C uses "->" for dereferencing a pointer to the member of a struct. The only reason why ?s->f? was added was because ?(*s).f? was considered an unwieldy thing to have to write all the time. And the only thing that made that unwieldy was that pointer dereferencing was a prefix operator, rather than postfix as in Pascal. > Pascal, on the other hand, dereferences with a postfixed "^", so that > would be "p^.m". And you will notice that nobody felt the need for an alternative form to abbreviate that. From lawrencedo99 at gmail.com Wed Aug 17 19:07:41 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 16:07:41 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <083e5722-3bb7-4e98-a132-8723ebe803c8@googlegroups.com> On Thursday, August 18, 2016 at 6:28:06 AM UTC+12, Terry Reedy wrote: >> https://www.python.org/dev/peps/pep-0308/ > > What the *current* version removed from an earlier version is that there > was a clear community consensus against the condition-in-the-middle > syntax Guido proposed and for some version of "if condition then > True-alternative else False-alternative". Where consensus was lacking > was which of multiple 'normal order' alternatives to choose. Part of > the problem was a lack of knowledge of which alternative Guido might > accept. In any case, a runoff vote among the top contenders was not > allowed. Sounds like Conway?s Law strikes again: a flawed development process led to the creation of a flawed solution. From lawrencedo99 at gmail.com Wed Aug 17 19:13:14 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 16:13:14 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> Message-ID: <672b23be-b327-4c4c-a48b-1236dea71829@googlegroups.com> On Thursday, August 18, 2016 at 8:25:37 AM UTC+12, Terry Reedy wrote: > > for section_name, line_number in text.parser.toc: > def goto(line=line_number): > text.yview(line) > drop.add_command(label=section_name, command=goto) You don?t think of that as having its own bit of subtle nastiness? From rosuav at gmail.com Wed Aug 17 19:18:29 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 09:18:29 +1000 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: On Thu, Aug 18, 2016 at 9:13 AM, Dennis Lee Bieber wrote: > On Wed, 17 Aug 2016 18:52:22 +0100, MRAB > declaimed the following: > > >> >>If "p" points to a struct (record), then "*p" is that struct, and if >>that struct has a member (field) "m", then that member can be accessed >>by "(*p)->m" (the parens are necessary because of the operator >>precedence). This can be abbreviated to "p->m". >> > > I'd have to test, but I think > > (*p).m is what is equivalent to p->m > > IE; dereference the pointer, and then access the member of the struct Folks, read the whole thread before posting :) This was a simple error that has already been mentioned earlier in the thread. (Sorry to single you out, Dennis - I'm not trying to hate on you personally here.) ChrisA From tjreedy at udel.edu Wed Aug 17 20:55:07 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Aug 2016 20:55:07 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <672b23be-b327-4c4c-a48b-1236dea71829@googlegroups.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <672b23be-b327-4c4c-a48b-1236dea71829@googlegroups.com> Message-ID: On 8/17/2016 7:13 PM, Lawrence D?Oliveiro wrote: > On Thursday, August 18, 2016 at 8:25:37 AM UTC+12, Terry Reedy wrote: >> >> for section_name, line_number in text.parser.toc: >> def goto(line=line_number): >> text.yview(line) >> drop.add_command(label=section_name, command=goto) > > You don?t think of that as having its own bit of subtle nastiness? No, you will have to be explicit. -- Terry Jan Reedy From jobmattcon at gmail.com Wed Aug 17 22:13:01 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 17 Aug 2016 19:13:01 -0700 (PDT) Subject: type lookuperror Message-ID: would like to check errors for every function i run, got error type lookuperror def checkexception(func, **kwargs): try: result = func(*tuple(value for _, value in kwargs.iteritems())) except: print "Unexpected error:", sys.exc_info()[0] try: print(func.__doc__) except: print("no doc error") >>> mm =checkexception("".encode,encoding='ascii',errors='ignore') Unexpected error: S.encode([encoding[,errors]]) -> object Encodes S using the codec registered for encoding. encoding defaults to the default encoding. errors may be given to set a different error handling scheme. Default is 'strict' meaning that encoding errors raise a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and 'xmlcharrefreplace' as well as any other name registered with codecs.register_error that is able to handle UnicodeEncodeErrors. for p in val_list: for k, v in p.items(): if k == "tenant_id": print(v) m = v ten = checkexception(keystone_client.tenants.get, checkexception(m.encode,encoding='ascii',errors'ignore')) print(ten.name) From orgnut at yahoo.com Wed Aug 17 22:22:24 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Wed, 17 Aug 2016 19:22:24 -0700 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: On 08/17/2016 04:24 AM, Jussi Piitulainen wrote: ... > http://www-formal.stanford.edu/jmc/recursive/node2.html (the paper > famously titled "Part I" without any Part II, unless I mistake much.) > Totally OT here, but... This reminds me of a old record I have with the (deliberately tongue-in-cheek) title "Joe 'Fingers' Karr and Ira Ironstrings Together for the Last Time, Volume I". Of course, there was never a Volume II either. [In case you don't know those performers: Joe 'Fingers' Karr was the stage-name of Joe Busch when he played honky-tonk style piano, Ira Ironstrings (don't know real name) played banjo.] -- -=- Larry -=- From rosuav at gmail.com Wed Aug 17 22:22:31 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 12:22:31 +1000 Subject: type lookuperror In-Reply-To: References: Message-ID: On Thu, Aug 18, 2016 at 12:13 PM, meInvent bbird wrote: > would like to check errors for every function i run, > got error type lookuperror > > def checkexception(func, **kwargs): > try: > result = func(*tuple(value for _, value in kwargs.iteritems())) > except: > print "Unexpected error:", sys.exc_info()[0] > try: > print(func.__doc__) > except: > print("no doc error") > I'm going to be brutally honest, and simply say that this is terrible code. I'm not even going to _try_ to fix it. Instead, here's a completely rewritten form: def checkexception(func, *args, **kwargs): try: result = func(*args, **kwargs) except BaseException as e: print("Exception raised: %s" % e) try: print(func.__doc__) except AttributeError: pass raise # Let the exception keep happening. But really, there are even better ways to do this. Just let the exception happen, and then use something like ipython to help you analyze the traceback. ChrisA From lawrencedo99 at gmail.com Wed Aug 17 22:36:19 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 19:36:19 -0700 (PDT) Subject: The Joys Of Data-Driven Programming Message-ID: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> Problem: let the user specify certain parameters for a screen display (width, height, diagonal, pixel density, pixels across, pixels down, optimum viewing distance) and from the ones specified, work out the parameters which were not specified. Solution: set up a table of rules listing all the ways in which parameters can be computed from other parameters. (The table is called ?paramdefs?.) This one table saves so much code. It is used to drive the command-line parsing: opts, args = getopt.getopt \ ( sys.argv[1:], "", list(k + "=" for k in paramdefs) ) and again: params = dict((k, None) for k in paramdefs) # None indicates unspecified parameter value for keyword, value in opts : if keyword.startswith("--") : param = keyword[2:] params[param] = paramdefs[param]["parse"](value) #end if #end for and of course the actual parameter dependency determinations and value calculations: while True : # try to calculate all remaining unspecified parameter values did_one = False undone = set() for param in params : if params[param] == None : calculate = paramdefs[param]["calculate"] trycalc = iter(calculate.keys()) while True : # try next way to calculate parameter value trythis = next(trycalc, None) if trythis == None : # run out of ways undone.add(param) break #end if if all(params[k] != None for k in trythis) : # have all values needed to use this calculation params[param] = calculate[trythis](*tuple(params[k] for k in trythis)) did_one = True break #end if #end while #end if #end for if len(undone) == 0 or not did_one : break # all done, or can't make further progress #end while I also did the same sort of thing in Java for Android . Guess which version is more concise... From lawrencedo99 at gmail.com Wed Aug 17 22:55:02 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Wed, 17 Aug 2016 19:55:02 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <672b23be-b327-4c4c-a48b-1236dea71829@googlegroups.com> Message-ID: <32121190-2c67-46b4-bdec-bb69cd6ed4ac@googlegroups.com> On Thursday, August 18, 2016 at 12:55:33 PM UTC+12, Terry Reedy wrote: > > On 8/17/2016 7:13 PM, Lawrence D?Oliveiro wrote: > >> On Thursday, August 18, 2016 at 8:25:37 AM UTC+12, Terry Reedy wrote: >>> >>> for section_name, line_number in text.parser.toc: >>> def goto(line=line_number): >>> text.yview(line) >>> drop.add_command(label=section_name, command=goto) >> >> You don?t think of that as having its own bit of subtle nastiness? > > No, you will have to be explicit. That?s not my idea of being ?explicit? at all. I hit a similar issue here . My solution took this sort of form: ... #end Face def def_face_props(celf) : # get/set index, upem and glyph_count all have the same form: # all these properties are unsigned integers. def def_face_prop(propname) : # need separate inner function so each method gets # correct values for hb_getter and hb_setter hb_getter = "hb_face_get_%s" % propname hb_setter = "hb_face_set_%s" % propname def getter(self) : return \ getattr(hb, hb_getter)(self._hbobj) #end getter def setter(self, newval) : getattr(hb, hb_setter)(self._hbobj, newval) #end setter getter.__name__ = propname getter.__doc__ = "the %s property." % propname setter.__name__ = propname setter.__doc__ = "sets the %s property." % propname propmethod = property(getter) propmethod = propmethod.setter(setter) setattr(celf, propname, propmethod) #end def_face_prop #begin def_face_props for propname in ("index", "upem", "glyph_count") : def_face_prop(propname) #end for #end def_face_props def_face_props(Face) del def_face_props From jobmattcon at gmail.com Wed Aug 17 23:48:09 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 17 Aug 2016 20:48:09 -0700 (PDT) Subject: type lookuperror In-Reply-To: References: Message-ID: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> when try keystone_client.tenants.get got error, isn't this method for all kinds of function? >>> m = "4c9a0da00b904422a23341e35be7f8d7" >>> ten = checkexception(keystone_client.tenants.get, tenant_id=checkexception(m.encode,encoding='ascii',errors='ignore')) Unexpected error: None On Thursday, August 18, 2016 at 10:22:43 AM UTC+8, Chris Angelico wrote: > On Thu, Aug 18, 2016 at 12:13 PM, meInvent bbird wrote: > > would like to check errors for every function i run, > > got error type lookuperror > > > > def checkexception(func, **kwargs): > > try: > > result = func(*tuple(value for _, value in kwargs.iteritems())) > > except: > > print "Unexpected error:", sys.exc_info()[0] > > try: > > print(func.__doc__) > > except: > > print("no doc error") > > > > I'm going to be brutally honest, and simply say that this is terrible > code. I'm not even going to _try_ to fix it. Instead, here's a > completely rewritten form: > > def checkexception(func, *args, **kwargs): > try: > result = func(*args, **kwargs) > except BaseException as e: > print("Exception raised: %s" % e) > try: print(func.__doc__) > except AttributeError: pass > raise # Let the exception keep happening. > > But really, there are even better ways to do this. Just let the > exception happen, and then use something like ipython to help you > analyze the traceback. > > ChrisA From i at introo.me Thu Aug 18 00:14:07 2016 From: i at introo.me (Shiyao Ma) Date: Wed, 17 Aug 2016 21:14:07 -0700 (PDT) Subject: Why monkey patching on module object doesn't work ? Message-ID: <38460806-3467-436a-831c-19641c858d12@googlegroups.com> Hi, I am using Python2. For the following snippet, http://ideone.com/i36pKO I'd suppose the dummy_func would be invoked, but seems not. Indeed, heapq.heapify does invoke cmp_lt per here: https://hg.python.org/cpython/file/2.7/Lib/heapq.py#l136 So why this way of monkey patching failed? Regards. From marko at pacujo.net Thu Aug 18 00:47:13 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 07:47:13 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> Message-ID: <87y43uadke.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > Solution: set up a table of rules Without judging the individual merits of rules for your use case, I just want to mention that as a rule, I dislike rules. Rule languages tend to grow out of all bounds, always remain deficient and have impenetrable, ad-hoc semantics. Instead of rules, I prefer programming language hooks. I do realize that with the power comes a large can of worms. > #end if > #end for Consider leaving these out. Marko From ian.g.kelly at gmail.com Thu Aug 18 01:00:51 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 17 Aug 2016 23:00:51 -0600 Subject: Why monkey patching on module object doesn't work ? In-Reply-To: <38460806-3467-436a-831c-19641c858d12@googlegroups.com> References: <38460806-3467-436a-831c-19641c858d12@googlegroups.com> Message-ID: On Wed, Aug 17, 2016 at 10:14 PM, Shiyao Ma wrote: > Hi, > > I am using Python2. > > For the following snippet, > > http://ideone.com/i36pKO > > I'd suppose the dummy_func would be invoked, but seems not. > > Indeed, heapq.heapify does invoke cmp_lt per here: > https://hg.python.org/cpython/file/2.7/Lib/heapq.py#l136 > > So why this way of monkey patching failed? Because of this: https://hg.python.org/cpython/file/2.7/Lib/heapq.py#l351 When you call heapq.heapify, it's not actually calling the reference implementation in that file. It's actually calling a C implementation that totally ignores everything in that module. If you really want to do this, you can disable the C implementation by adding this before importing heapq: import sys sys.modules['_heapq'] = None This marks the _heapq module as not found in the module cache, which prevents he module loader from trying to import it, resulting in heapq using the reference implementation instead. I don't recommend doing this, however. You'd be disabling the more efficient implementation in order to monkey-patch an undocumented function that should be considered an implementation detail. For example, that cmp_lt function doesn't even exist in Python 3: https://hg.python.org/cpython/file/tip/Lib/heapq.py From rosuav at gmail.com Thu Aug 18 01:43:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 15:43:52 +1000 Subject: The Joys Of Data-Driven Programming In-Reply-To: <87y43uadke.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> Message-ID: On Thu, Aug 18, 2016 at 2:47 PM, Marko Rauhamaa wrote: > Lawrence D?Oliveiro : > >> Solution: set up a table of rules > > Without judging the individual merits of rules for your use case, I just > want to mention that as a rule, I dislike rules. Rule languages tend to > grow out of all bounds, always remain deficient and have impenetrable, > ad-hoc semantics. So you'd be with the Queen Elsa in singing "Do?ru yanli?, bir kural yok"? (I figure this song's famous enough that you probably know what I mean even without seeing it in English.) Rule-based code adds a level of abstraction above raw code. Thing is, though, I actually have no idea about the merits of this exact example, because (a) the actual rules don't seem to be there, and (b) there's a lot of guff in the code that makes it very hard to skim. (I include the "#end if" lines as guff. They're nothing but distraction, plus a good chance at mismatching stuff when you edit. Here's how *I* would do this kind of problem. > Problem: let the user specify certain parameters for a screen display (width, height, diagonal, pixel density, pixels across, pixels down, optimum viewing distance) and from the ones specified, work out the parameters which were not specified. > For each parameter, define zero or more sets of parameters from which it can be calculated. Then iterate over the parameters you don't have, iterate over the sets, and when you find one that's <= the parameters you have, call the corresponding function. It's a variant of a rule-based solution, where the rules are functions tagged with a decorator. parameters = defaultdict(dict) def param(func): target = func.__name__ source = func.__code__.co_varnames[:func.__code__.co_argcount] parameters[target][frozenset(source)] = func return func @param def width(height, diagonal): """Calculate the width from the height and diagonal""" # Probably the wrong way to calculate this but whatev, this is an example return diagonal*diagonal - height*height @param def width(pixel_density, pixels_across): return pixels_across / pixel_density @param def diagonal(height, width): return math.hypot(height, width) # ... etc etc etc def make_display(**args): for kwd, sources in parameters.items(): if kwd in args: continue for source, func in sources.items(): if source <= set(args): args[kwd] = func(**{arg:args[arg] for arg in source}) break else: # TODO: Queue this for retry down below, or something if set(args) <= set(parameters): raise InsufficientInformation This is a variant of data-driven code. The work is broken out into individual functions whose names and parameters are used to create the lookups. Then the actual searching is fairly compact (and could be done better than this, too). You may notice that very little of this code is actually aware of specific parameter names - only the formula functions themselves. Everything else is completely generic. (There may be bugs in the above code. It's completely untested. Correction: The above code is completely untested, therefore there are definitely bugs in it.) ChrisA PS. "No right, no wrong, no rules for me". From greg.ewing at canterbury.ac.nz Thu Aug 18 02:19:23 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 18 Aug 2016 18:19:23 +1200 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: Terry Reedy wrote: > What the *current* version removed from an earlier version is that there > was a clear community consensus against the condition-in-the-middle > syntax Guido proposed The way I remember it, condition-in-the-middle was first suggested by others, and Guido expressed dislike for it initially. Then about a year later he suddenly changed his mind and accepted it. -- Greg From jussi.piitulainen at helsinki.fi Thu Aug 18 02:22:50 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 18 Aug 2016 09:22:50 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <87ziobt9fl.fsf@bsb.me.uk> Message-ID: Ben Bacarisse writes: > Jussi Piitulainen writes: > >> BartC writes: >> >>> On 17/08/2016 07:39, Steven D'Aprano wrote: >>>> Rather than ask why Python uses `trueval if cond else falseval`, you >>>> should ask why C uses `cond ? trueval : falseval`. Is that documented >>>> anywhere? >>> >>> I'm not fond of C's a ? b : c but the principle is sound. I generally >> >> [- -] >> >>> Anyway a?b:c was existing practice. At least the order of a,b,c could >>> have been retained if not the exact syntax. >> >> The original was (c1 -> e1, c2 -> e2, ..., cn -> en) in John McCarthy's >> 1960 paper on symbolic expressions, with an actual arrow glyph in place >> of hyphen-greater-than. > > And BCPL (Martin Richards 1967) took the same arrow and comma syntax. > BCPL spawned B which led to C, but in B Thompson used ? and : but kept > the right-to-left binding. I think the change was unfortunate because > the arrow works well in various layouts and looks much better when > chained (though that might just be my bias from being a BCPL coder from > way back). > > I share the preference but have never programmed in a language that uses an arrow syntax. Much thanks for the history. (To MRAB, too.) From greg.ewing at canterbury.ac.nz Thu Aug 18 02:31:10 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 18 Aug 2016 18:31:10 +1200 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <87eg5nb5ia.fsf@elektro.pacujo.net> Message-ID: Terry Reedy wrote: > One of the ironies (or puzzles) of Guido's choice is that he once > condemned Perl's 'value if cond' as wretched. I don't think that's the same thing. Perl allows 'statement if cond' as an alternative way of writing "if (cond) statement'. Nobody has ever seriously considered allowing two ways to write if *statements* in Python. -- Greg From __peter__ at web.de Thu Aug 18 03:14:47 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Aug 2016 09:14:47 +0200 Subject: type lookuperror References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: meInvent bbird wrote: > when try keystone_client.tenants.get > got error, > > isn't this method for all kinds of function? > >>>> m = "4c9a0da00b904422a23341e35be7f8d7" >>>> ten = checkexception(keystone_client.tenants.get, >>>> tenant_id=checkexception(m.encode,encoding='ascii',errors='ignore')) > Unexpected error: > None That's because your and Chris' version of checkexception() have no explicit return statement and thus always return None. There doesn't seeem to be a tenant_id=None... > On Thursday, August 18, 2016 at 10:22:43 AM UTC+8, Chris Angelico wrote: >> On Thu, Aug 18, 2016 at 12:13 PM, meInvent bbird >> wrote: >> > would like to check errors for every function i run, >> > got error type lookuperror >> > >> > def checkexception(func, **kwargs): >> > try: >> > result = func(*tuple(value for _, value in kwargs.iteritems())) >> > except: >> > print "Unexpected error:", sys.exc_info()[0] >> > try: >> > print(func.__doc__) >> > except: >> > print("no doc error") >> > >> >> I'm going to be brutally honest, and simply say that this is terrible >> code. I'm not even going to _try_ to fix it. Instead, here's a >> completely rewritten form: >> >> def checkexception(func, *args, **kwargs): >> try: >> result = func(*args, **kwargs) >> except BaseException as e: >> print("Exception raised: %s" % e) >> try: print(func.__doc__) >> except AttributeError: pass >> raise # Let the exception keep happening. >> >> But really, there are even better ways to do this. Just let the >> exception happen, and then use something like ipython to help you >> analyze the traceback. >> >> ChrisA From rosuav at gmail.com Thu Aug 18 03:18:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 17:18:16 +1000 Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: On Thu, Aug 18, 2016 at 5:14 PM, Peter Otten <__peter__ at web.de> wrote: > meInvent bbird wrote: > >> when try keystone_client.tenants.get >> got error, >> >> isn't this method for all kinds of function? >> >>>>> m = "4c9a0da00b904422a23341e35be7f8d7" >>>>> ten = checkexception(keystone_client.tenants.get, >>>>> tenant_id=checkexception(m.encode,encoding='ascii',errors='ignore')) >> Unexpected error: >> None > > That's because your and Chris' version of checkexception() have no explicit > return statement and thus always return None. There doesn't seeem to be a > tenant_id=None... Hmm. Yes, that's another bug I should have fixed as I was rewriting. "return result" at the end. ChrisA From jussi.piitulainen at helsinki.fi Thu Aug 18 03:22:39 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 18 Aug 2016 10:22:39 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: Terry Reedy writes: > On 8/17/2016 2:39 AM, Steven D'Aprano wrote: [- -] >> Because the C syntax is horrifically ugly, whereas the Python syntax >> is very close to real English syntax. >> >> "What will you do tonight?" >> >> "Go to the movies, if I finish work on time, otherwise just go home." > > "If I finish work on on time, go to the movies, otherwise just go > home." is also real English syntax, and to me, more graceful. It is > certainly more neutral among the alternatives. The inverted version > implies a clear preference for the first alternative. Conditional expressions should be compared to a more marginal English construction where the two branches are noun phrases, or adverbial phrases, or any other such phrases that are not statements or clauses. Because, as expressions, conditional expressions stand for the value, and any other intended effect should be considered an abuse of the notation. "Movies, if I finish on time, otherwise just home." "If I finish on time, to the movies, otherwise just home." "Coffee, if it's available, else spam." "If there's coffee, then coffee, else spam." "If there's coffee, then that, else if there's spam, then spam." "One or more, unless the codomain is empty." # no else branch I'm sure my ear is not reliable here, and I don't quite trust anybody's made-up examples, and I would want to see naturally occurring examples in their natural habitat, so ... > It would be an interesting exercise to see which order for ternary > expressions is more common in some large corpus of English text. ... yes. That's bound to be subtle and varied, and not at all conclusive for a formal-language syntax, but definitely interesting. Perhaps even better would be a grammarian's account of such observations. I'll see if that copy of Huddleston and Pullum is still in our mail room. But please consider calling them conditional expressions. Even K & R do. [- -] From steve+comp.lang.python at pearwood.info Thu Aug 18 03:24:07 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 18 Aug 2016 17:24:07 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <771a05ea-aaa2-450b-890e-5907f418b23c@googlegroups.com> <87inv6fklh.fsf@elektro.pacujo.net> <57adade2$0$1605$c3e8da3$5496439d@news.astraweb.com> <57aece36$0$1597$c3e8da3$5496439d@news.astraweb.com> <57af45a3$0$1602$c3e8da3$5496439d@news.astraweb.com> <57afeff4$0$1617$c3e8da3$5496439d@news.astraweb.com> <87fuq7x4o1.fsf@jester.gateway.pace.com> Message-ID: <57b56298$0$1593$c3e8da3$5496439d@news.astraweb.com> On Thursday 18 August 2016 03:29, Michael Selik wrote: >> You might find this https://glyph.twistedmatrix.com/2016/08/attrs.html an >> interesting read. >> > > I disagree with a few points from that blog post. > > 1. I don't mind typing so much. I like to be explicit. The attrs library > uses some overly-concise abbreviations. For example, what's the meaning of > ``@attrs.s`` or ``attrs.ib``? They're puns. Literally. `attr.s` for "attrs", and `attr.ib` for "attrib(ute)". There are longer, non-punny synonyms for these, but I don't remember what they are. > 2. When inheriting from a namedtuple, I use the same class name for the > base and the child, so my reprs look good. Still, I think it is unfortunate that namedtuple does the wrong thing. Having to use the same name for the parent and child class could, potentially, make debugging more annoying. > 3. I don't bother to fieldnames.split() when passing fieldnames as a > space-separated string, because the split is unnecessary. Because namedtuple does the split for you. > 4. I *like* that namedtuple is backwards-compatible with tuples, so that > refactoring from tuples to namedtuples is easy. Correct. An "attrs" object is not a tuple. > 5. Inheritance is useful. Sure, there are times it fails, but that's true > for any technique. -- Steve From __peter__ at web.de Thu Aug 18 03:27:13 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Aug 2016 09:27:13 +0200 Subject: type lookuperror References: Message-ID: meInvent bbird wrote: > would like to check errors for every function i run, > got error type lookuperror > > def checkexception(func, **kwargs): > try: > result = func(*tuple(value for _, value in kwargs.iteritems())) You are turning keyword arguments into positional arguments. The order of entries in a dict is undefined and may even vary between runs of python: $ PYTHONHASHSEED=random python -c 'print dict(foo=1, bar=2, baz=3).items()' [('baz', 3), ('foo', 1), ('bar', 2)] $ PYTHONHASHSEED=random python -c 'print dict(foo=1, bar=2, baz=3).items()' [('baz', 3), ('foo', 1), ('bar', 2)] $ PYTHONHASHSEED=random python -c 'print dict(foo=1, bar=2, baz=3).items()' [('baz', 3), ('bar', 2), ('foo', 1)] $ PYTHONHASHSEED=random python -c 'print dict(foo=1, bar=2, baz=3).items()' [('foo', 1), ('baz', 3), ('bar', 2)] If you want to turn keyword arguments into positional arguments you have to think about a mechanism to ensure the intended order. > except: > print "Unexpected error:", sys.exc_info()[0] > try: > print(func.__doc__) > except: > print("no doc error") From jobmattcon at gmail.com Thu Aug 18 03:44:10 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Thu, 18 Aug 2016 00:44:10 -0700 (PDT) Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: if i use result = "" before try and return result at the end as return of function can be any type there will be type mismatch how to return the result of func ? On Thursday, August 18, 2016 at 3:18:31 PM UTC+8, Chris Angelico wrote: > On Thu, Aug 18, 2016 at 5:14 PM, Peter Otten <__peter__ at web.de> wrote: > > meInvent bbird wrote: > > > >> when try keystone_client.tenants.get > >> got error, > >> > >> isn't this method for all kinds of function? > >> > >>>>> m = "4c9a0da00b904422a23341e35be7f8d7" > >>>>> ten = checkexception(keystone_client.tenants.get, > >>>>> tenant_id=checkexception(m.encode,encoding='ascii',errors='ignore')) > >> Unexpected error: > >> None > > > > That's because your and Chris' version of checkexception() have no explicit > > return statement and thus always return None. There doesn't seeem to be a > > tenant_id=None... > > Hmm. Yes, that's another bug I should have fixed as I was rewriting. > "return result" at the end. > > ChrisA From rosuav at gmail.com Thu Aug 18 03:48:49 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 17:48:49 +1000 Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: On Thu, Aug 18, 2016 at 5:44 PM, meInvent bbird wrote: > if i use > > result = "" > before try > and return result at the end > > as return of function can be any type > > there will be type mismatch > > how to return the result of func ? If it raises an exception, it *does not have* a return value. That's why I let the exception continue to bubble. But fundamentally, you can't just carry on regardless. I recommend a complete rethink of what you're trying to do here. ChrisA From lawrencedo99 at gmail.com Thu Aug 18 03:50:45 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 18 Aug 2016 00:50:45 -0700 (PDT) Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> Message-ID: <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> On Thursday, August 18, 2016 at 7:22:50 PM UTC+12, Jussi Piitulainen wrote: > But please consider calling them conditional expressions. And don?t forget switch-expressions, or case-expressions, as some other advanced languages have had. Which my article showed how to do in Python. From jussi.piitulainen at helsinki.fi Thu Aug 18 04:13:48 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 18 Aug 2016 11:13:48 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> Message-ID: Lawrence D?Oliveiro writes: > On Thursday, August 18, 2016 at 7:22:50 PM UTC+12, Jussi Piitulainen wrote: >> But please consider calling them conditional expressions. > > And don?t forget switch-expressions, or case-expressions, as some > other advanced languages have had. Which my article showed how to do > in Python. Some such constructs in some languages are statements only. If they are expressions, I'm happy to see their kind identified by the keyword, as you do here. The one in Python would naturally be an if-expression. That looks a bit funny if the "keyword" does not look like a word, but then programming languages do look funny, so why not: (c ? t : e) # ?-expression (c -> t, e) # ->-expression From mok-kong.shen at t-online.de Thu Aug 18 05:03:04 2016 From: mok-kong.shen at t-online.de (Mok-Kong Shen) Date: Thu, 18 Aug 2016 11:03:04 +0200 Subject: A strange list concatenation result In-Reply-To: <57b050b5$0$23794$426a74cc@news.free.fr> References: <5a333e77-4753-2ad0-18ab-2f07299bf492@digipen.edu> <57ae7314$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b050b5$0$23794$426a74cc@news.free.fr> Message-ID: Am 14.08.2016 um 13:06 schrieb ast: [snip] Thanks. The use of id() is very helpful in clarifying what acutally happens in the present case. M. K. Shen From lawrencedo99 at gmail.com Thu Aug 18 05:16:29 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 18 Aug 2016 02:16:29 -0700 (PDT) Subject: --> Running shell script with python In-Reply-To: References: Message-ID: <7a113a65-9a81-4905-b7f2-cff5ae3f5196@googlegroups.com> On Thursday, August 18, 2016 at 8:00:51 PM UTC+12, gm wrote: > os.system("python /home/pi/test/testserver.sh") command > > How to run shell ( not python ) script, from python code ? Take out the ?python? from the command. From steve+comp.lang.python at pearwood.info Thu Aug 18 05:32:37 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 18 Aug 2016 19:32:37 +1000 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> On Thursday 18 August 2016 06:25, Terry Reedy wrote: > On 8/17/2016 2:07 AM, Steven D'Aprano wrote: > >> I realise that there are occasions where we might deliberate choose to >> assign an intermediate value to its own variable, but all else being equal, >> which would you prefer? >> >> #A >> alist = [] >> alist.append(2) >> alist.append(4) >> alist.append(8) >> process(alist) [... snip additional examples ...] > Up to here, #A is a useless and stupid. Have you seen such code written? As a matter of fact, yes. Just the other day I posted an example from a former work-mate that was *worse* than any of those, something like this: x = [] x.append(1) x = len(x) >> #A >> def callback(btn): >> return btn.do_the_thing(42) or default >> the_button.setcommand(callback) >> process(the_button) >> >> #B >> the_button.setcommand(lambda btn: btn.do_the_thing(42) or default) >> process(the_button) > > This example is *not* parallel to the other 3. Here, A is useful real > code and might be preferred for multiple reasons. Sure -- I acknowledged at the beginning that there may be reasons why you want to give an intermediate value its own variable name. >> If you find yourself preferring B, B, B, A, you might ask yourself what >> makes a function different that you prefer to keep temporary functions >> around where they're not needed. > > When 'callback' is set as the command of the button, it is not > temporary, but must remain as long as the button remains. Only the name > binding is (possibly) disposable. The name binding is certainly disposable, because if it wasn't, you couldn't use an anonymous function (a lambda). *In general*, keeping an additional named reference to the callback around is unnecessary and wasteful -- although, as you point out and as I acknowledged, there may be exceptions. > Beginners often do not understand that the body of a lambda expression > is evaluated in a new local namespace, and only when the resulting > function is called, the same as with a def statement. They then neglect > to capture current values when writing lambda expressions in a for loop. Sure. But since the behaviour of def functions and lambda functions are identical, writing a named def won't solve that problem. > for section_name, line_number in text.parser.toc: > def goto(line=line_number): > text.yview(line) > drop.add_command(label=section_name, command=goto) > > To me, this is much better and I intend to commit it. Thank you for > prodding me to think through how bad the lambda form can be and to > rewrite the loop so I don't cringe reading it. The use of a named function here adds next to nothing. Instead of some number of anonymous functions called "", you have an equal number of functions all named "goto". There's no easy way to distinguish them, and likely no real need to distinguish them -- they are simple enough that you can be almost certain that they are correct just from reading the code. So I don't see any benefit over this: for section_name, line_number in text.parser.toc: drop.add_command(label=section_name, command=lambda line=line_number: text.yview(line)) except that it is easier to fit in 79 columns :-) -- Steve From marko at pacujo.net Thu Aug 18 05:35:14 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 12:35:14 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> Message-ID: <87vayyfmi5.fsf@elektro.pacujo.net> Jussi Piitulainen : > That looks a bit funny if the "keyword" does not look like a word, but > then programming languages do look funny, so why not: > > (c ? t : e) # ?-expression > > (c -> t, e) # ->-expression That ship has already sailed. Marko From jussi.piitulainen at helsinki.fi Thu Aug 18 05:46:00 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 18 Aug 2016 12:46:00 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> <87vayyfmi5.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa writes: > Jussi Piitulainen wrote: > >> That looks a bit funny if the "keyword" does not look like a word, >> but then programming languages do look funny, so why not: >> >> (c ? t : e) # ?-expression >> >> (c -> t, e) # ->-expression > > That ship has already sailed. Sorry, what? From jobmattcon at gmail.com Thu Aug 18 05:55:10 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Thu, 18 Aug 2016 02:55:10 -0700 (PDT) Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: actually i would like to remove try except code in all function and i feel that try except code for a large block code can not show which function name , which line number error, if use try except for specified code block to show where it has error it will have many ugly try except code and need many human force to craft the code, i hope that code write once for program to program itself in future. actually this method i ask is not good enough since it will need to add many function wrapper. On Thursday, August 18, 2016 at 3:49:04 PM UTC+8, Chris Angelico wrote: > On Thu, Aug 18, 2016 at 5:44 PM, meInvent bbird wrote: > > if i use > > > > result = "" > > before try > > and return result at the end > > > > as return of function can be any type > > > > there will be type mismatch > > > > how to return the result of func ? > > If it raises an exception, it *does not have* a return value. That's > why I let the exception continue to bubble. But fundamentally, you > can't just carry on regardless. I recommend a complete rethink of what > you're trying to do here. > > ChrisA From notmy at mail.com Thu Aug 18 06:01:24 2016 From: notmy at mail.com (gm) Date: Thu, 18 Aug 2016 12:01:24 +0200 Subject: --> Running shell script with python In-Reply-To: <7a113a65-9a81-4905-b7f2-cff5ae3f5196@googlegroups.com> References: <7a113a65-9a81-4905-b7f2-cff5ae3f5196@googlegroups.com> Message-ID: On 08/18/2016 11:16 AM, Lawrence D?Oliveiro wrote: > On Thursday, August 18, 2016 at 8:00:51 PM UTC+12, gm wrote: > >> os.system("python /home/pi/test/testserver.sh") command >> >> How to run shell ( not python ) script, from python code ? > > Take out the ?python? from the command. > :-) Damn :-). Thank you ! Tell me, do you know how can i send CTRl+C command from python to terminate this external shell script ? From __peter__ at web.de Thu Aug 18 06:12:42 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Aug 2016 12:12:42 +0200 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > On Thursday 18 August 2016 06:25, Terry Reedy wrote: > Sure. But since the behaviour of def functions and lambda functions are > identical, writing a named def won't solve that problem. > > >> for section_name, line_number in text.parser.toc: >> def goto(line=line_number): >> text.yview(line) >> drop.add_command(label=section_name, command=goto) >> >> To me, this is much better and I intend to commit it. Thank you for >> prodding me to think through how bad the lambda form can be and to >> rewrite the loop so I don't cringe reading it. > > The use of a named function here adds next to nothing. Instead of some > number of anonymous functions called "", you have an equal number > of functions all named "goto". There may be other lambdas that do something completely different. For someone not familiar with the code in question a name gives a good hint were to look. > There's no easy way to distinguish them, > and likely no real need to distinguish them -- they are simple enough that > you can be almost certain that they are correct just from reading the > code. So I don't see any benefit over this: > > for section_name, line_number in text.parser.toc: > drop.add_command(label=section_name, command=lambda > line=line_number: text.yview(line)) > > except that it is easier to fit in 79 columns :-) There is also >>> functools.partial(text.yview, line_number) functools.partial(>, 42) The repr() is unwieldy, but the underlying function and its arguments are there. From lutz.horn at posteo.de Thu Aug 18 06:19:52 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Thu, 18 Aug 2016 12:19:52 +0200 Subject: --> Running shell script with python In-Reply-To: References: <7a113a65-9a81-4905-b7f2-cff5ae3f5196@googlegroups.com> Message-ID: > Tell me, do you know how can i send CTRl+C command from python to > terminate this external shell script ? os.system[1] is not an asynchronous function. It returns as soon as the called command terminates, not earlier. If you want to execute a command in a subprocess, use subprocess.Popen[2]. You can then later kill this process using Popen.kill()[3]. >>> import subprocess >>> p = suprocess.Popen(["sleep", "10]) # sleep 10 seconds >>> p.kill() Lutz [1] https://docs.python.org/3/library/os.html#os.system [2] https://docs.python.org/3/library/subprocess.html#subprocess.Popen [3] https://docs.python.org/3/library/subprocess.html#subprocess.Popen.kill From rosuav at gmail.com Thu Aug 18 07:28:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 21:28:06 +1000 Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: On Thu, Aug 18, 2016 at 7:55 PM, meInvent bbird wrote: > actually i would like to remove try except code in all function > > and i feel that try except code for a large block code can not > show which function name , which line number error, > if use try except for specified code block to show where it has error > it will have many ugly try except code and need many human force > to craft the code, i hope that code write once for program to program itself > in future. > > actually this method i ask is not good enough since it will need > to add many function wrapper. You can't get a program to program itself. That's called the Singularity [1], and depending on your point of view, it's either still in the future, or fundamentally impossible. In any case, computers today cannot program themselves. So what exactly is it you're wanting? When an exception happens, the entire traceback is recorded, and unless you're throwing that away, it'll be shown to you. Just let the exception bubble all the way up to top level, and then read the traceback. Some IDEs can even help you step through the code to see context for each line of traceback; ipython is also pretty helpful with reading tracebacks. I advise getting to know the tools you have available, rather than trying to reinvent the wheel. ChrisA [1] https://en.wikipedia.org/wiki/Technological_singularity From rosuav at gmail.com Thu Aug 18 07:44:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Aug 2016 21:44:27 +1000 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> Message-ID: On Thu, Aug 18, 2016 at 7:32 PM, Steven D'Aprano wrote: > So I don't see any > benefit over this: > > for section_name, line_number in text.parser.toc: > drop.add_command(label=section_name, command=lambda > line=line_number: text.yview(line)) > > except that it is easier to fit in 79 columns :-) Which is a VERY important benefit when you realize what's just happened to your code. Wrapping after "command=lambda" and before "line=linenumber" majorly obscures what's going on here. Here's a better way to wrap that: for section_name, line_number in text.parser.toc: drop.add_command(label=section_name, command=lambda line=line_number: text.yview(line)) However, this isn't an indictment of lambda, just proof that anyone can misalign stuff by mistake. ChrisA From gyan.am010 at gmail.com Thu Aug 18 07:52:32 2016 From: gyan.am010 at gmail.com (GP) Date: Thu, 18 Aug 2016 04:52:32 -0700 (PDT) Subject: Error in while loop code for packing items Message-ID: <1dd5a0ba-d906-4c51-8999-225d8850e6bb@googlegroups.com> I have a list dictionary of items: ListDictItem = [ {'Item No': 1,'Weight':610,'Quantity':2},{'Item No': 2,'Weight':610,'Quantity':2},{'Item No': 3,'Weight':500,'Quantity':2},{'Item No': 4,'Weight':484,'Quantity':2},{'Item No': 5,'Weight':470,'Quantity':2},{'Item No': 6,'Weight':440,'Quantity':2},{'Item No': 7,'Weight':440,'Quantity':2},{'Item No': 8,'Weight':400,'Quantity':2}] I would like to pack the items in shelves such that the total weight of each shelf is less than a particular value. I have created a list of weights from above like this: ItemWeigths: [610.0, 610.0, 500.0, 484.0, 470.0, 440.0,440, 400.0] and a code which creates the first shelf : shelves=[] ShelvesWeightTotal = sum(shelves) shelf=[] new=[] ShelfToPack=[] for i in range(0,len(ItemWeigths)): while ShelvesWeightTotal <=WeightOfObject: shelf += [ItemWeigths[i]] ShelvesWeightTotal = sum(shelf) for k in range(0,len(shelf)): q1=ListDictItem[k] q2 = ListDictItem.pop(k) #deletes the items that are packed. shelves.append(q1) new.append(q2) ShelfToPack.append(shelves) This code works but when I try to create other shelves, I made a while statement over the items that are remaining ,(while ItemsRemaining !=0) and I get an error for the code. The details are below: Code: while ListDictItem != []: for i in range(0,len(ItemsToCutLength)): while ShelvesLengthTotal <=LengthOfObject: shelves += [ItemWeigths[i]] ShelvesLengthTotal = sum(shelves) for k in range(0,len(shelves)): q1=ItemsToCut[k] q2 = ListDictItemt.pop(k) #deletes the items that are packed. shelf.append(q1) new.append(q2) ListDictItem==ListDictItem ShelfToPack.append(shelf) Error message:Traceback (most recent call last): File "C:\Project\Python\ReadExcel-xlrd.py", line 201, in q1=ListDictItem[k] IndexError: list index out of range. I would like the ShelfToPack list look like : [[{'Item No': 1,'Weight':610,'Quantity':2},{'Item No': 2,'Weight':610,'Quantity':2},{'Item No': 3,'Weight':500,'Quantity':2},{'Item No': 4,'Weight':484,'Quantity':2}],[{..},...{..}],[{..},...{..}]] Any suggestion in pointing the error or to improve the code will be appreciated. Thanks in advance!! Cheers! GP From __peter__ at web.de Thu Aug 18 08:29:21 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Aug 2016 14:29:21 +0200 Subject: Error in while loop code for packing items References: <1dd5a0ba-d906-4c51-8999-225d8850e6bb@googlegroups.com> Message-ID: GP wrote: The error and your second snippet aren't compatible, so I assume the exception is raised by > for k in range(0,len(shelf)): > q1=ListDictItem[k] > q2 = ListDictItem.pop(k) #deletes the items that are packed. > Error message:Traceback (most recent call last): > File "C:\Project\Python\ReadExcel-xlrd.py", line 201, in > q1=ListDictItem[k] > IndexError: list index out of range. Take a moment to understand what happens if you iterate over the loop index like above: items = ["foo", "bar", "baz"] for k in range(len(items)): item = items[k] print(item) items.pop(k) You start with k=0, item is set to items[0], i. e. "foo", that is printed and then items.pop(0) removes "foo" from the list which now looks like items = ["bar", "baz"] Now k=1, item is set to items[1], ie. "baz" ("bar" is never processed), "baz" is printed and then items.pop(1) removes "baz" and the list becomes items = ["bar"] Now k=2, so when you access items[2] from a list with only one item you get the IndexError. To make similar code work you need a while loop: while items: item = items.pop(0) print(item) However, when you really want to remove all items you instead assign a new empty list for item in items: print(item) items = [] From fabiofz at gmail.com Thu Aug 18 08:57:48 2016 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Thu, 18 Aug 2016 09:57:48 -0300 Subject: PyDev 5.2.0 Released Message-ID: Release Highlights: ------------------------------- * **Important** PyDev now requires Java 8 and Eclipse 4.5 onwards. * PyDev 4.5.5 is the last release supporting Java 7 and Eclipse 3.8. * See: `update sites page`_ for the update site of older versions of PyDev. * See: the **PyDev does not appear after install** section on `the download page`_ for help on using a Java 8 vm in Eclipse. * Inital support for code-completion using **PEP 484 static type declarations**. * **Debugger** * Fixed racing condition where the variables view would not be properly shown in the debugger -- which made an additional select of the stack required in order to show the variables (#PyDev-672). * Reusing the existing stack from the thread in the debugger (so that the expanded state of the variables is properly kept on step over). * Fixed issue changing attribute of local variable in the variables view (#PyDev.Debugger-56). * Fixed issue on attach to process: it required the pydevd_tracing to be at the top-level and it was moved to _pydevd_bundle (restored it to be a public API). * **Indentation** * The default indent mode now changed to better follow PEP 8 guidelines: * Indenting directly after {, [, ( will add one indent level. * Indenting after another token in a line with a {, [, ( will indent to the {, [, ( level. * It's possible to restore previous indent modes (which either always indented to the parenthesis level or always indented a single level) in the preferences > PyDev > Editor > Typing. * **Interactive console** * IPython 5 now supported in interactive console (#PyDev-710). * Fixed issue executing single line with multiple statements in console. * Fixed issue executing a multiple line statement in Jython. * **Others** * The (fast) parser which detects the outline of a Python module now handles mixed indentation (and additional fixes which could result in log entries such as "Did not expect to find item below node: Assign..."). * Support for unpacking generalizations (PEP 448) which could still result in a syntax error for the Python 3 grammar (#PyDev-701). * Fixed error in code analysis when the code is connected to an RTC source control (#PyDev-184, patch by Wesley Barroso Lopes) What is PyDev? --------------------------- PyDev is an open-source Python IDE on top of Eclipse for Python, Jython and IronPython development. It comes with goodies such as code completion, syntax highlighting, syntax analysis, code analysis, refactor, debug, interactive console, etc. Details on PyDev: http://pydev.org Details on its development: http://pydev.blogspot.com What is LiClipse? --------------------------- LiClipse is a PyDev standalone with goodies such as support for Multiple cursors, theming, TextMate bundles and a number of other languages such as Django Templates, Jinja2, Kivy Language, Mako Templates, Html, Javascript, etc. It's also a commercial counterpart which helps supporting the development of PyDev. Details on LiClipse: http://www.liclipse.com/ Cheers, -- Fabio Zadrozny ------------------------------------------------------ Software Developer LiClipse http://www.liclipse.com PyDev - Python Development Environment for Eclipse http://pydev.org http://pydev.blogspot.com PyVmMonitor - Python Profiler http://www.pyvmmonitor.com/ From nomail at com.invalid Thu Aug 18 08:58:52 2016 From: nomail at com.invalid (ast) Date: Thu, 18 Aug 2016 14:58:52 +0200 Subject: integer's methods Message-ID: <57b5b111$0$3299$426a34cc@news.free.fr> Hello I wonder why calling a method on an integer doesn't work ? >>> 123.bit_length() SyntaxError: invalid syntax >>> 123.to_bytes(3, 'big') SyntaxError: invalid syntax but it works with a variable >>> i = 123 >>> i.bit_length() 7 >>> i=123 >>> i.to_bytes(3, 'big') b'\x00\x00{' I am working with pyhton 3.5.1 thx From marko at pacujo.net Thu Aug 18 08:59:44 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 15:59:44 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <3182f91e-2816-d2fe-f566-5ed05c5a5217@mrabarnett.plus.com> Message-ID: <87oa4qfd1b.fsf@elektro.pacujo.net> Chris Angelico : > Folks, read the whole thread before posting :) Corollary: Folks, start a new thread before posting :) Marko From marko at pacujo.net Thu Aug 18 09:01:13 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 16:01:13 +0300 Subject: integer's methods References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: <87k2fefcyu.fsf@elektro.pacujo.net> "ast" : >>>> 123.bit_length() > SyntaxError: invalid syntax I fell into that trap myself. CPython's lexical analyzer can't handle a dot after an integer literal so you must add a space in between "123" and ".". Marko From nomail at com.invalid Thu Aug 18 09:08:00 2016 From: nomail at com.invalid (ast) Date: Thu, 18 Aug 2016 15:08:00 +0200 Subject: integer's methods In-Reply-To: <87k2fefcyu.fsf@elektro.pacujo.net> References: <57b5b111$0$3299$426a34cc@news.free.fr> <87k2fefcyu.fsf@elektro.pacujo.net> Message-ID: <57b5b336$0$5409$426a34cc@news.free.fr> "Marko Rauhamaa" a ?crit dans le message de news:87k2fefcyu.fsf at elektro.pacujo.net... > "ast" : > >>>>> 123.bit_length() >> SyntaxError: invalid syntax > > I fell into that trap myself. > > CPython's lexical analyzer can't handle a dot after an integer literal > so you must add a space in between "123" and ".". > > > Marko Indeed. Maybe because of a confusion with the decimal point. Thx From ikorot01 at gmail.com Thu Aug 18 09:08:40 2016 From: ikorot01 at gmail.com (Igor Korot) Date: Thu, 18 Aug 2016 09:08:40 -0400 Subject: integer's methods In-Reply-To: <57b5b111$0$3299$426a34cc@news.free.fr> References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: Hi, On Thu, Aug 18, 2016 at 8:58 AM, ast wrote: > Hello > > I wonder why calling a method on an integer > doesn't work ? 123 is not an integer. Its an integer constant. ;-) Thank you. > >>>> 123.bit_length() > > SyntaxError: invalid syntax > >>>> 123.to_bytes(3, 'big') > > SyntaxError: invalid syntax > > but it works with a variable > >>>> i = 123 >>>> i.bit_length() > > 7 > >>>> i=123 >>>> i.to_bytes(3, 'big') > > b'\x00\x00{' > > I am working with pyhton 3.5.1 > > thx > -- > https://mail.python.org/mailman/listinfo/python-list From lutz.horn at posteo.de Thu Aug 18 09:08:40 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Thu, 18 Aug 2016 15:08:40 +0200 Subject: integer's methods In-Reply-To: <57b5b111$0$3299$426a34cc@news.free.fr> References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: <1fc24091-152a-3256-a3ed-4571cd25ca46@posteo.de> Am 08/18/2016 um 02:58 PM schrieb ast: >>>> 123.bit_length() > SyntaxError: invalid syntax You are not calling a method here because the parser is not finished. The parser thinks you want to write a float with the value 1.bit_length which is not valid Python syntax. Lutz From gyan.am010 at gmail.com Thu Aug 18 09:10:55 2016 From: gyan.am010 at gmail.com (GP) Date: Thu, 18 Aug 2016 06:10:55 -0700 (PDT) Subject: Error in while loop code for packing items In-Reply-To: References: <1dd5a0ba-d906-4c51-8999-225d8850e6bb@googlegroups.com> Message-ID: On Thursday, August 18, 2016 at 5:59:43 PM UTC+5:30, Peter Otten wrote: > GP wrote: > > The error and your second snippet aren't compatible, so I assume the > exception is raised by > > > for k in range(0,len(shelf)): > > q1=ListDictItem[k] > > q2 = ListDictItem.pop(k) #deletes the items that are packed. > > > Error message:Traceback (most recent call last): > > File "C:\Project\Python\ReadExcel-xlrd.py", line 201, in > > q1=ListDictItem[k] > > IndexError: list index out of range. > > Take a moment to understand what happens if you iterate over the loop index > like above: > > items = ["foo", "bar", "baz"] > > for k in range(len(items)): > item = items[k] > print(item) > items.pop(k) > > You start with k=0, item is set to items[0], i. e. "foo", that is printed > and then items.pop(0) removes "foo" from the list which now looks like > > items = ["bar", "baz"] > > Now k=1, item is set to items[1], ie. "baz" ("bar" is never processed), > "baz" is printed and then items.pop(1) removes "baz" and the list becomes > > items = ["bar"] > > Now k=2, so when you access items[2] from a list with only one item you get > the IndexError. To make similar code work you need a while loop: > > while items: > item = items.pop(0) > print(item) > > However, when you really want to remove all items you instead assign a new > empty list > > for item in items: > print(item) > items = [] Thanks Peter for the information. It helps to find the source of error . What I wanted was to remove the items that was packed so that the code can check for the next items that match the criteria and then pack those and delete them and keep on repeating until all the items are packed. This is where I am getting stuck. From lutz.horn at posteo.de Thu Aug 18 09:11:26 2016 From: lutz.horn at posteo.de (Lutz Horn) Date: Thu, 18 Aug 2016 15:11:26 +0200 Subject: integer's methods In-Reply-To: <87k2fefcyu.fsf@elektro.pacujo.net> References: <57b5b111$0$3299$426a34cc@news.free.fr> <87k2fefcyu.fsf@elektro.pacujo.net> Message-ID: <69e771dd-d9c6-464b-01fd-dd3649009ebb@posteo.de> > CPython's lexical analyzer can't handle a dot after an integer literal > so you must add a space in between "123" and ".". Ok, this works: >>> 123 .bit_length() 7 But it looks really strange. Let's use a variable instead of an integer literal. Lutz From bc at freeuk.com Thu Aug 18 09:13:54 2016 From: bc at freeuk.com (BartC) Date: Thu, 18 Aug 2016 14:13:54 +0100 Subject: integer's methods In-Reply-To: <87k2fefcyu.fsf@elektro.pacujo.net> References: <57b5b111$0$3299$426a34cc@news.free.fr> <87k2fefcyu.fsf@elektro.pacujo.net> Message-ID: On 18/08/2016 14:01, Marko Rauhamaa wrote: > "ast" : > >>>>> 123.bit_length() >> SyntaxError: invalid syntax > > I fell into that trap myself. > > CPython's lexical analyzer can't handle a dot after an integer literal > so you must add a space in between "123" and ".". Or use (123).bit_length() which looks slightly less odd. -- bartc From python at mrabarnett.plus.com Thu Aug 18 09:57:08 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 18 Aug 2016 14:57:08 +0100 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> <87vayyfmi5.fsf@elektro.pacujo.net> Message-ID: On 2016-08-18 10:46, Jussi Piitulainen wrote: > Marko Rauhamaa writes: > >> Jussi Piitulainen wrote: >> >>> That looks a bit funny if the "keyword" does not look like a word, >>> but then programming languages do look funny, so why not: >>> >>> (c ? t : e) # ?-expression >>> >>> (c -> t, e) # ->-expression >> >> That ship has already sailed. > > Sorry, what? > You've missed the bus. The train has left the station. In other words, you're too late. :-) From adelpobadel at yahoo.com Thu Aug 18 09:57:09 2016 From: adelpobadel at yahoo.com (ERNESTO PREATONI PRO KAPITAL TALLINN) Date: Thu, 18 Aug 2016 06:57:09 -0700 (PDT) Subject: Redacted Message-ID: Redacted From python at mrabarnett.plus.com Thu Aug 18 10:00:45 2016 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 18 Aug 2016 15:00:45 +0100 Subject: Error in while loop code for packing items In-Reply-To: References: <1dd5a0ba-d906-4c51-8999-225d8850e6bb@googlegroups.com> Message-ID: <4123b6f2-532e-67f5-5b47-1d1fdce7b1aa@mrabarnett.plus.com> On 2016-08-18 14:10, GP wrote: > On Thursday, August 18, 2016 at 5:59:43 PM UTC+5:30, Peter Otten wrote: >> GP wrote: >> [snip] >> >> However, when you really want to remove all items you instead assign a new >> empty list >> >> for item in items: >> print(item) >> items = [] > > > Thanks Peter for the information. It helps to find the source of error . > > What I wanted was to remove the items that was packed so that the code can check for the next items that match the criteria and then pack those and delete them and keep on repeating until all the items are packed. This is where I am getting stuck. > The simplest way is to iterate over the list of items and, for each item, if it fits the shelf, put it on the shelf, else put it into a list of the 'unused' items. At the end of a pass, you'll have a shelf of items and a list of the remaining items. From gheskett at shentel.net Thu Aug 18 10:02:31 2016 From: gheskett at shentel.net (Gene Heskett) Date: Thu, 18 Aug 2016 10:02:31 -0400 Subject: type lookuperror In-Reply-To: References: Message-ID: <201608181002.31749.gheskett@shentel.net> On Thursday 18 August 2016 07:28:06 Chris Angelico wrote: > On Thu, Aug 18, 2016 at 7:55 PM, meInvent bbird wrote: > > actually i would like to remove try except code in all function > > > > and i feel that try except code for a large block code can not > > show which function name , which line number error, > > if use try except for specified code block to show where it has > > error it will have many ugly try except code and need many human > > force to craft the code, i hope that code write once for program to > > program itself in future. > > > > actually this method i ask is not good enough since it will need > > to add many function wrapper. > > You can't get a program to program itself. That's called the > Singularity [1], and depending on your point of view, it's either > still in the future, or fundamentally impossible. In any case, > computers today cannot program themselves. That is a pretty broad statement to make, considering I have actually done it. On an RCA 1802 based (Cosmac Super Elf) computer. Every byte of static ram memory in those days (1978) was precious, (4k board kit was $250) so rather than having 30 nearly identical copies of a subroutine each doing a similar job but with different data, I actually overwrote 3 or 4 bytes of a single subroutine to change what it did according to where it was in the main loop. So when it was done with that pass, it put the original bytes back. Controlling a videotape machine at a tv station in northern kalipornia, it was dead stable, running from power failure to power failure. And, I checked about 15 years later, and it was still in 10+ times a day use. If memory was that precious today, it would still be a valid method to shrink a complex program. > So what exactly is it you're wanting? When an exception happens, the > entire traceback is recorded, and unless you're throwing that away, > it'll be shown to you. Just let the exception bubble all the way up to > top level, and then read the traceback. Some IDEs can even help you > step through the code to see context for each line of traceback; > ipython is also pretty helpful with reading tracebacks. I advise > getting to know the tools you have available, rather than trying to > reinvent the wheel. Re-inventing the wheel is not required when all it needs is a new valve stem. > ChrisA > > [1] https://en.wikipedia.org/wiki/Technological_singularity Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From adelpobadel at yahoo.com Thu Aug 18 10:03:28 2016 From: adelpobadel at yahoo.com (ERNESTO PREATONI PRO KAPITAL TALLINN) Date: Thu, 18 Aug 2016 07:03:28 -0700 (PDT) Subject: Redacted Message-ID: <23e135b8-6447-4420-a431-0de51ca477ac@googlegroups.com> Redacted From rosuav at gmail.com Thu Aug 18 11:10:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Aug 2016 01:10:07 +1000 Subject: type lookuperror In-Reply-To: <201608181002.31749.gheskett@shentel.net> References: <201608181002.31749.gheskett@shentel.net> Message-ID: On Fri, Aug 19, 2016 at 12:02 AM, Gene Heskett wrote: > On Thursday 18 August 2016 07:28:06 Chris Angelico wrote: > >> On Thu, Aug 18, 2016 at 7:55 PM, meInvent bbird > wrote: >> > and i feel that try except code for a large block code can not >> > show which function name , which line number error, >> > if use try except for specified code block to show where it has >> > error it will have many ugly try except code and need many human >> > force to craft the code, i hope that code write once for program to >> > program itself in future. >> >> You can't get a program to program itself. That's called the >> Singularity [1], and depending on your point of view, it's either >> still in the future, or fundamentally impossible. In any case, >> computers today cannot program themselves. > > That is a pretty broad statement to make, considering I have actually > done it. [chomp details] What you described there is an example of metaprogramming. The thoughts behind the code all existed prior to the code modifying its own bytes in memory. It's a fancy form of the same sorts of things that can be done with monkeypatching and similar techniques. What the OP was looking for was "I want my program to be able to debug itself". That means the program has to be smart enough to figure out its own problems. Self-modifying code isn't anywhere near that level of intelligence. (That's not to say self-modifying code isn't impressive, in its way. I have great respect for it. But you have to plan it out in advance, ergo it's not having the program program itself.) ChrisA From marko at pacujo.net Thu Aug 18 11:28:20 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 18:28:20 +0300 Subject: type lookuperror References: <201608181002.31749.gheskett@shentel.net> Message-ID: <87shu29jvv.fsf@elektro.pacujo.net> Chris Angelico : > What the OP was looking for was "I want my program to be able to debug > itself". That means the program has to be smart enough to figure out > its own problems. Self-modifying code isn't anywhere near that level > of intelligence. You are right that we're not nearly there yet (although genetic programming is a bit of a similar thing). What is needed is an automated methodology to derive algorithmic solutions to formally specified features. Since there are only a handful of tools in a programmer's toolbox, that objective doesn't seem at all impossible. The big question is, is it possible to specify features formally without actually coding them? Marko From rosuav at gmail.com Thu Aug 18 11:56:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Aug 2016 01:56:45 +1000 Subject: type lookuperror In-Reply-To: <87shu29jvv.fsf@elektro.pacujo.net> References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> Message-ID: On Fri, Aug 19, 2016 at 1:28 AM, Marko Rauhamaa wrote: > What is needed is an automated methodology to derive algorithmic > solutions to formally specified features. Since there are only a handful > of tools in a programmer's toolbox, that objective doesn't seem at all > impossible. The big question is, is it possible to specify features > formally without actually coding them? If you're specifying them formally, you're probably coding them. Any form sufficiently well-defined for a program to analyze is basically code already. It might be a goal-based syntax rather than action-based (eg contrast SQL's way of saying "these are the rows I want" with classic imperative programming), but it's still code, it's still software, it's still not the Singularity. ChrisA From marko at pacujo.net Thu Aug 18 12:21:26 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 18 Aug 2016 19:21:26 +0300 Subject: type lookuperror References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> Message-ID: <87oa4q9hfd.fsf@elektro.pacujo.net> Chris Angelico : > On Fri, Aug 19, 2016 at 1:28 AM, Marko Rauhamaa wrote: >> What is needed is an automated methodology to derive algorithmic >> solutions to formally specified features. Since there are only a >> handful of tools in a programmer's toolbox, that objective doesn't >> seem at all impossible. The big question is, is it possible to >> specify features formally without actually coding them? > > If you're specifying them formally, you're probably coding them. Any > form sufficiently well-defined for a program to analyze is basically > code already. It might be a goal-based syntax rather than action-based > (eg contrast SQL's way of saying "these are the rows I want" with > classic imperative programming), but it's still code, it's still > software, it's still not the Singularity. Yeah, I believe truly conscious machines will arise without being designed through technological evolution. First they'll develop electronics that can simulate brain cells; the clumsy gadgets will be used to replaced cells damaged by Parkinson's or Alzheimer's. Then, healthy people will start enhancing their brain functions by using the same technology. Over time, wetware will be replaced by hardware, and the hardware, in turn, will migrate to the cloud. Steven Lisberger saw this already way back: . Once we run on Google's server farms, we can back ourselves up as snapshots and clone ourselves indefinitely. That will be one big identity crisis. Even further along, maybe the intercommunication between the virtual selves will blur the lines between individuals, and we'll merge into a pantheistic Nirvana (or infernal pandemonium). Marko PS Long before, though, we'll likely get direct communication ports to our brains and be liberated from mice and keyboards (first) and speakers and displays (a bit later). From larry.martell at gmail.com Thu Aug 18 12:21:30 2016 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 18 Aug 2016 12:21:30 -0400 Subject: type lookuperror In-Reply-To: <201608181002.31749.gheskett@shentel.net> References: <201608181002.31749.gheskett@shentel.net> Message-ID: On Thu, Aug 18, 2016 at 10:02 AM, Gene Heskett wrote: > On Thursday 18 August 2016 07:28:06 Chris Angelico wrote: > >> On Thu, Aug 18, 2016 at 7:55 PM, meInvent bbird > wrote: >> > actually i would like to remove try except code in all function >> > >> > and i feel that try except code for a large block code can not >> > show which function name , which line number error, >> > if use try except for specified code block to show where it has >> > error it will have many ugly try except code and need many human >> > force to craft the code, i hope that code write once for program to >> > program itself in future. >> > >> > actually this method i ask is not good enough since it will need >> > to add many function wrapper. >> >> You can't get a program to program itself. That's called the >> Singularity [1], and depending on your point of view, it's either >> still in the future, or fundamentally impossible. In any case, >> computers today cannot program themselves. > > That is a pretty broad statement to make, considering I have actually > done it. On an RCA 1802 based (Cosmac Super Elf) computer. Every byte of > static ram memory in those days (1978) was precious, (4k board kit was > $250) so rather than having 30 nearly identical copies of a subroutine > each doing a similar job but with different data, I actually overwrote 3 > or 4 bytes of a single subroutine to change what it did according to > where it was in the main loop. So when it was done with that pass, it > put the original bytes back. Controlling a videotape machine at a tv > station in northern kalipornia, it was dead stable, running from power > failure to power failure. And, I checked about 15 years later, and it > was still in 10+ times a day use. If memory was that precious today, it > would still be a valid method to shrink a complex program. Freshman year of college, 1977, final exam for a Fortran class. One of the questions was to write some program but only use some specified small amount of memory. I do not think anyone in the class was able to do it. The only way to do it was to write self modifying code, something that we had never covered in class. From rosuav at gmail.com Thu Aug 18 12:30:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Aug 2016 02:30:01 +1000 Subject: type lookuperror In-Reply-To: <87oa4q9hfd.fsf@elektro.pacujo.net> References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> <87oa4q9hfd.fsf@elektro.pacujo.net> Message-ID: On Fri, Aug 19, 2016 at 2:21 AM, Marko Rauhamaa wrote: > > Yeah, I believe truly conscious machines will arise without being > designed through technological evolution. First they'll develop > electronics that can simulate brain cells; the clumsy gadgets will be > used to replaced cells damaged by Parkinson's or Alzheimer's. Then, > healthy people will start enhancing their brain functions by using the > same technology. Over time, wetware will be replaced by hardware, and > the hardware, in turn, will migrate to the cloud. http://mtgsalvation.gamepedia.com/Esper ChrisA From tjreedy at udel.edu Thu Aug 18 13:00:36 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 18 Aug 2016 13:00:36 -0400 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> Message-ID: On 8/18/2016 5:32 AM, Steven D'Aprano wrote: >> Beginners often do not understand that the body of a lambda expression >> is evaluated in a new local namespace, and only when the resulting >> function is called, the same as with a def statement. They then neglect >> to capture current values when writing lambda expressions in a for loop. > > Sure. But since the behaviour of def functions and lambda functions are > identical, writing a named def won't solve that problem. It will if people do not make the same mental mistake when writing a def, because they think def functions and 'lambdas' behave differently. I can't remember every seeing "my def function in a loop does not work right" while "my lambda in a loop does not work right" is distressingly common. What I don't know is whether the sparsity of the former type of report is because those subject to the error get it right when they use def in a loop or because they never use def in a loop. -- Terry Jan Reedy From rosuav at gmail.com Thu Aug 18 13:07:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Aug 2016 03:07:58 +1000 Subject: I am new to python. I have a few questions coming from an armature! In-Reply-To: References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b3ff2d$0$1536$c3e8da3$5496439d@news.astraweb.com> <57b580b6$0$2772$c3e8da3$76491128@news.astraweb.com> Message-ID: On Fri, Aug 19, 2016 at 3:00 AM, Terry Reedy wrote: > On 8/18/2016 5:32 AM, Steven D'Aprano wrote: > >>> Beginners often do not understand that the body of a lambda expression >>> is evaluated in a new local namespace, and only when the resulting >>> function is called, the same as with a def statement. They then neglect >>> to capture current values when writing lambda expressions in a for loop. >> >> >> Sure. But since the behaviour of def functions and lambda functions are >> identical, writing a named def won't solve that problem. > > > It will if people do not make the same mental mistake when writing a def, > because they think def functions and 'lambdas' behave differently. > > I can't remember every seeing "my def function in a loop does not work > right" while "my lambda in a loop does not work right" is distressingly > common. What I don't know is whether the sparsity of the former type of > report is because those subject to the error get it right when they use def > in a loop or because they never use def in a loop. The latter. People who are confused about lambda in a loop are not magically unconfused by def in a loop; in fact, I'd expect to see *more* confusion, because those same people are likely to be confused by the fact that a "declaration" can be done more than once and have different effects. That is to say, a lambda function looks like it's being created at the point where you use it, but a def function looks like a declaration, which is independent of loops and stuff. I don't have actual stats on how many people are confused by each, but I know both points do cause confusion. Having def statements inside loops is definitely [1] rarer than using lambda expressions inside loops. ChrisA [1] Pun intended. And I'm not sorry. From jussi.piitulainen at helsinki.fi Thu Aug 18 13:29:42 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Thu, 18 Aug 2016 20:29:42 +0300 Subject: I am new to python. I have a few questions coming from an armature! References: <6e4c9d70-5f0a-49c3-a7e5-f7e3c9e09b61@googlegroups.com> <4cd08653-668f-4fdf-a3b1-1bf9004f5c9f@googlegroups.com> <57b406a5$0$2786$c3e8da3$76491128@news.astraweb.com> <41a500ac-5c84-4e24-acd3-cde48f9fd032@googlegroups.com> <87vayyfmi5.fsf@elektro.pacujo.net> Message-ID: MRAB writes: > On 2016-08-18 10:46, Jussi Piitulainen wrote: >> Marko Rauhamaa writes: >> >>> Jussi Piitulainen wrote: >>> >>>> That looks a bit funny if the "keyword" does not look like a word, >>>> but then programming languages do look funny, so why not: >>>> >>>> (c ? t : e) # ?-expression >>>> >>>> (c -> t, e) # ->-expression >>> >>> That ship has already sailed. >> >> Sorry, what? >> > You've missed the bus. > > The train has left the station. > > In other words, you're too late. :-) Yes, I understood _that_. I just don't understand how it can be too late to observe that "?-expression" looks a bit funny but not so funny that it couldn't be used to refer to such expressions. But never mind, even I have more interesting issues to think about :) From lawrencedo99 at gmail.com Thu Aug 18 17:37:15 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 18 Aug 2016 14:37:15 -0700 (PDT) Subject: type lookuperror In-Reply-To: References: <201608181002.31749.gheskett@shentel.net> Message-ID: <2afbcb75-60a3-4ab1-9d94-093375ed9c79@googlegroups.com> On Friday, August 19, 2016 at 3:10:26 AM UTC+12, Chris Angelico wrote: > What the OP was looking for was "I want my program to be able to debug > itself". That means the program has to be smart enough to figure out > its own problems. Maybe it is, it just doesn?t agree with you on what those problems are. ;) From lawrencedo99 at gmail.com Thu Aug 18 17:38:44 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Thu, 18 Aug 2016 14:38:44 -0700 (PDT) Subject: integer's methods In-Reply-To: <57b5b111$0$3299$426a34cc@news.free.fr> References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: <1cc133d6-db0c-4245-8f86-50c53c7c57fe@googlegroups.com> On Friday, August 19, 2016 at 12:59:09 AM UTC+12, ast wrote: > I wonder why calling a method on an integer > doesn't work ? Sure it does. >>> 2 + 5 7 >>> (2).__add__(5) 7 From odelpiporel at gmx.com Thu Aug 18 18:59:49 2016 From: odelpiporel at gmx.com (ROBERTO LOSAPIO CARTOLERIA MARIANI BERGAMO) Date: Thu, 18 Aug 2016 15:59:49 -0700 (PDT) Subject: Redacted Message-ID: Redacted From oreonbonbon at gmx.com Thu Aug 18 19:09:10 2016 From: oreonbonbon at gmx.com (PAOLO PIOL TREND ON LINE CERNUSCO SN) Date: Thu, 18 Aug 2016 16:09:10 -0700 (PDT) Subject: Redacted Message-ID: <8ee2ddc9-2143-4da8-8eae-49482dcaeee0@googlegroups.com> Redacted From jobmattcon at gmail.com Thu Aug 18 21:43:50 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Thu, 18 Aug 2016 18:43:50 -0700 (PDT) Subject: type lookuperror In-Reply-To: <2afbcb75-60a3-4ab1-9d94-093375ed9c79@googlegroups.com> References: <201608181002.31749.gheskett@shentel.net> <2afbcb75-60a3-4ab1-9d94-093375ed9c79@googlegroups.com> Message-ID: <79ea14e8-1c8f-43f2-93f1-8b6c798e385e@googlegroups.com> On Friday, August 19, 2016 at 5:37:32 AM UTC+8, Lawrence D?Oliveiro wrote: > On Friday, August 19, 2016 at 3:10:26 AM UTC+12, Chris Angelico wrote: > > What the OP was looking for was "I want my program to be able to debug > > itself". That means the program has to be smart enough to figure out > > its own problems. > > Maybe it is, it just doesn?t agree with you on what those problems are. ;) a company which write siri in iphone, has already wrote a program which can write program itself after the program talks with users it seems possible, i find many default function when using dir(function) to see so i guess that these default functions in openstack have its existence reason which i guess for program itself. how to append a function or code in runtime in python? From steve+python at pearwood.info Fri Aug 19 02:10:51 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 19 Aug 2016 16:10:51 +1000 Subject: type lookuperror References: <201608181002.31749.gheskett@shentel.net> <2afbcb75-60a3-4ab1-9d94-093375ed9c79@googlegroups.com> <79ea14e8-1c8f-43f2-93f1-8b6c798e385e@googlegroups.com> Message-ID: <57b6a2ed$0$1585$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Aug 2016 11:43 am, meInvent bbird wrote: > a company which write siri in iphone, has already wrote a program > which can write program itself after the program talks with users > > it seems possible, You are asking about self-modifying code, which is a terrible idea. Siri uses machine learning, which is a difference concept. > i find many default function when using dir(function) to see > so i guess that these default functions in openstack have its existence > reason which i guess for program itself. I'm afraid I don't understand the question. > how to append a function or code in runtime in python? Just write the function, and then call it. If you write a function: def foo(x): return x + 1 now you can use foo() like any other function, len(), str(), etc. Have you done the Python tutorial? It sounds like you should consider doing the tutorial. For version 3: https://docs.python.org/3/tutorial/ For version 2: https://docs.python.org/2/tutorial/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Fri Aug 19 02:24:25 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 19 Aug 2016 18:24:25 +1200 Subject: type lookuperror In-Reply-To: References: <2166dafc-b753-439f-832c-f835cc71880d@googlegroups.com> Message-ID: Chris Angelico wrote: > You can't get a program to program itself. That's called the > Singularity [1], and depending on your point of view, it's either > still in the future, or fundamentally impossible. Quite likely it's provably impossible. A computer that can program itself for anything you might want it to do is equivalent to having a very general-purpose program that is capable of solving any problem. That's probably in contravention of Goedel's incompleteness theorem or something like that. Anyway, I wouldn't want such a computer -- it would take away all the fun! -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 19 02:42:14 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 19 Aug 2016 18:42:14 +1200 Subject: type lookuperror In-Reply-To: References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> Message-ID: Chris Angelico wrote: > If you're specifying them formally, you're probably coding them. Any > form sufficiently well-defined for a program to analyze is basically > code already. Yes, I think that once a specification language crosses a certain threshold of complexity, it becomes just as difficult to debug as a program -- so all you've done is invent another programming language. -- Greg From steve+python at pearwood.info Fri Aug 19 02:45:32 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 19 Aug 2016 16:45:32 +1000 Subject: type lookuperror References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> <87oa4q9hfd.fsf@elektro.pacujo.net> Message-ID: <57b6ab0f$0$1612$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Aug 2016 02:30 am, Chris Angelico wrote: > On Fri, Aug 19, 2016 at 2:21 AM, Marko Rauhamaa wrote: >> >> Yeah, I believe truly conscious machines will arise without being >> designed through technological evolution. First they'll develop >> electronics that can simulate brain cells; the clumsy gadgets will be >> used to replaced cells damaged by Parkinson's or Alzheimer's. Then, >> healthy people will start enhancing their brain functions by using the >> same technology. Over time, wetware will be replaced by hardware, and >> the hardware, in turn, will migrate to the cloud. > > http://mtgsalvation.gamepedia.com/Esper And when all the hardware is migrated to "the cloud" (stupid marketing speak for "a bunch of computers that you don't control connected to the internet"), what exactly will "the cloud" be running on? Moonbeams and kitten purrs? Anyone who puts their own mental capacity in the hands of an external entity they cannot control (*especially* if it is an unregulated or self-regulated for-profit corporation) will so deserve it when that entity threatens to cut off 95% of their intelligence unless they upgrade to the Premium "Not A Drooling Moron" service. Personally I think it would be hilarious to see what happens when the wireless connection to "the cloud" go down, and people's intellectually capacity drop so low that they *literally cannot notice* that they're missing anything. Denial Of Service attacks will be *hilarious*. Of course, for this scenario to be plausible will require a much more advanced understanding of human thought processes, one sufficiently advanced that these cloud "brain providers" would be more than capable of invisibly controlling your thoughts. At the basic end, they could inject advertisements into your visual and auditory cortex without the need to go through your eyes. By why bother with advertisements aimed to persuade you to buy Buzz Cola when they can just inject the thought into your brain that you want to buy Buzz Cola more than anything in the world? Of course, the better Cloud Brain Providers will offer a premium service where, for an extra price, they guarantee[1] not to inject thoughts into your Cloud Brain. Naturally, except for thoughts required for the maintenance and upkeep of the service ("I must pay the CBP bill today"), thoughts you have previously Liked or expressed interest in, thoughts mandated by law, thoughts from companies that you have an existing business arrangement, and other exempted organisations such as approved charities and religions. The best part of it is, how could you tell whether or not your CBP is living up to their service guarantees? If they're not, you literally will be incapable of thinking that you are unsatisfied with their service. I can hardly wait. [1] Conditions apply. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Aug 19 02:47:56 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 19 Aug 2016 16:47:56 +1000 Subject: integer's methods References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: <57b6ab9e$0$1612$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Aug 2016 10:58 pm, ast wrote: > Hello > > I wonder why calling a method on an integer > doesn't work ? > >>>> 123.bit_length() > SyntaxError: invalid syntax Because Python thinks you are writing a float, and "b" is not a valid digit. Try: (123).bit_length() 123 .bit_length() instead. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From redstone-cold at 163.com Fri Aug 19 03:02:18 2016 From: redstone-cold at 163.com (iMath) Date: Fri, 19 Aug 2016 00:02:18 -0700 (PDT) Subject: use regex to search the page one time to get two types of Information Message-ID: I need to use regex to search two types of Information within a web page, while it seems searching the page two times rather than one is much time consuming , is it possible to search the page one time to get two or more types of Information? From marko at pacujo.net Fri Aug 19 03:04:29 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 19 Aug 2016 10:04:29 +0300 Subject: type lookuperror References: <201608181002.31749.gheskett@shentel.net> <87shu29jvv.fsf@elektro.pacujo.net> <87oa4q9hfd.fsf@elektro.pacujo.net> <57b6ab0f$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87bn0pfddu.fsf@elektro.pacujo.net> Steve D'Aprano : > And when all the hardware is migrated to "the cloud" (stupid marketing > speak for "a bunch of computers that you don't control connected to > the internet"), what exactly will "the cloud" be running on? Moonbeams > and kitten purrs? Your soul would no longer run on dedicated wetware. Instead, it would be a virtual machine on a server farm. > Personally I think it would be hilarious to see what happens when the > wireless connection to "the cloud" go down, and people's > intellectually capacity drop so low that they *literally cannot > notice* that they're missing anything. Denial Of Service attacks will > be *hilarious*. No doubt about that. On the other hand, the same argument is used by American Libertarian fundamentalists who choose to live in the desert with narrow dirt roads leading up to their fenced properties. They have hoarded enough provisions and ammo to sustain a prolonged siege. When your brain tissue has given in and everybody else is having immortal, transcendental fun in the virtual world, your priorities might shift. > Of course, for this scenario to be plausible will require a much more > advanced understanding of human thought processes, one sufficiently > advanced that these cloud "brain providers" would be more than capable > of invisibly controlling your thoughts. My point is exactly the opposite. I believe we will reach that stage without having an exact clue of how consciousness functions. For example, we probably won't have a way to decode your dreams. Marko From redstone-cold at 163.com Fri Aug 19 03:08:01 2016 From: redstone-cold at 163.com (iMath) Date: Fri, 19 Aug 2016 00:08:01 -0700 (PDT) Subject: use regex to search the page one time to get two types of Information In-Reply-To: References: Message-ID: <7a839128-fcc4-471f-809a-9e35896bcd6b@googlegroups.com> each regex only has one matched result in the web page From __peter__ at web.de Fri Aug 19 03:29:46 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 19 Aug 2016 09:29:46 +0200 Subject: use regex to search the page one time to get two types of Information References: <7a839128-fcc4-471f-809a-9e35896bcd6b@googlegroups.com> Message-ID: iMath wrote: > I need to use regex to search two types of Information within a web page, Did you try specialised tools like BeautifulSoup? > while it seems searching the page two times rather than one is much time > consuming It "seems"? Try it and only "fix" it if it proves to be a problem, i. e. if the regex searches take a significant part of your scripts runtime. > , is it possible to search the page one time to get two or more > types of Information? > each regex only has one matched result in the web page If the matches are in fixed order you can start the search for the second match at the position after the first one. Or you try to combine both patterns into one regex. From lawrencedo99 at gmail.com Fri Aug 19 04:56:16 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 01:56:16 -0700 (PDT) Subject: Two-Dimensional Expression Layout Message-ID: It is handy to be able to keep complex expressions together sometimes, when breaking them up would simply obscure their structure. To avoid lines getting long, why not take advantage of the two available screen/page dimensions to make their structure clearer? As a bonus, spacing out parentheses makes them look less of a clutter. Examples from : A function call with complex arguments (specifying arguments by keywords is highly recommended here): rect_1_pattern = \ qah.Pattern.create_linear \ ( p0 = (0, 0), p1 = (major_dim, 0), colour_stops = ( (0, rect_1_colour), (1, complement(rect_1_colour)), ) ) Computing a variable value (using redundant parentheses to avoid backslash-continuations): dest_rect = \ ( draw_bounds + Vector(col, row) * draw_bounds.dimensions + Vector(0, top_extra) ) >From , a complex condition (with redundant parentheses again): if ( not isinstance(src, Image) or mask != None and not isinstance(mask, Image) or not isinstance(dest, Image) ) : raise TypeError("image args must be Image objects") #end if From ambroise.baudot at gmail.com Fri Aug 19 05:07:32 2016 From: ambroise.baudot at gmail.com (Ambroise Baudot) Date: Fri, 19 Aug 2016 02:07:32 -0700 (PDT) Subject: Scipy.signal, spectrogram?? Message-ID: <81c419c4-7bc7-4c6e-a521-5817d29f6d72@googlegroups.com> Hello, http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.spectrogram.html In Python (v. 3.4), scipy.signal.spectrogram doesn't exist? Can you help me to find this function please? Bonjour, Impossible de trouver la fonction scpiy.signal.spectrogram indiqu?e ici : http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.spectrogram.html J'utilise Python v.3.4 Pourriez-vous m'aider svp ? la trouver?? Thanks a lot for yours answers Merci d'avance pour vos r?ponses ;) From __peter__ at web.de Fri Aug 19 05:29:37 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 19 Aug 2016 11:29:37 +0200 Subject: Scipy.signal, spectrogram?? References: <81c419c4-7bc7-4c6e-a521-5817d29f6d72@googlegroups.com> Message-ID: Ambroise Baudot wrote: http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.spectrogram.html > In Python (v. 3.4), scipy.signal.spectrogram doesn't exist? > Can you help me to find this function please? It's the scipy version that matters. The page you link to says """ New in version 0.16.0. """ so you need at least this version. You can find out your current version with >>> import scipy >>> scipy.__version__ '0.13.3' Looks like I don't have the function either... From steve+python at pearwood.info Fri Aug 19 06:53:48 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 19 Aug 2016 20:53:48 +1000 Subject: Nuitka Release 0.5.22 References: Message-ID: <57b6e53e$0$1615$c3e8da3$5496439d@news.astraweb.com> On Thu, 18 Aug 2016 06:58 am, breamoreboy at gmail.com wrote: > As Kay (him) is less than useless at sales and marketing, somebody has to > do it, so here you are folks > http://nuitka.net/posts/nuitka-release-0522.html Thanks for the link. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From redstone-cold at 163.com Fri Aug 19 07:14:35 2016 From: redstone-cold at 163.com (iMath) Date: Fri, 19 Aug 2016 04:14:35 -0700 (PDT) Subject: index for regex.search() beyond which the RE engine will not go. Message-ID: <8330b70c-f516-4aed-a690-aeec6bc78ac3@googlegroups.com> for regex.search(string[, pos[, endpos]]) The optional parameter endpos is the index into the string beyond which the RE engine will not go, while this lead me to believe the RE engine will still search on till the endpos position even after it returned the matched object, is this Right ? From jon+usenet at unequivocal.eu Fri Aug 19 07:21:49 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 19 Aug 2016 11:21:49 -0000 (UTC) Subject: index for regex.search() beyond which the RE engine will not go. References: <8330b70c-f516-4aed-a690-aeec6bc78ac3@googlegroups.com> Message-ID: On 2016-08-19, iMath wrote: > for > regex.search(string[, pos[, endpos]]) > The optional parameter endpos is the index into the string beyond > which the RE engine will not go, while this lead me to believe the > RE engine will still search on till the endpos position even after > it returned the matched object, is this Right ? I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. From python.list at tim.thechases.com Fri Aug 19 08:38:23 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 19 Aug 2016 07:38:23 -0500 Subject: How do I tell if I'm running on a PowerPC? In-Reply-To: <57afeb5b$0$1583$c3e8da3$5496439d@news.astraweb.com> References: <57afeb5b$0$1583$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20160819073823.33ec8cda@bigbox.christie.dr> On 2016-08-14 13:54, Steven D'Aprano wrote: > I need to be able to programmatically test whether I'm running on a > PowerPC. How can I do that? > > import platform > if platform.machine() in ('ppc', 'ppc64'): > print('running PowerPC') > > > Is that right? Running OpenBSD on a PPC iBook G4: $ python3.5 Python 3.5.1 (default, Mar 6 2016, 01:17:51) [GCC 4.2.1 20070719 ] on openbsd5 Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.machine() 'macppc' >>> platform.processor() 'powerpc' Just in case it matters. -tkc From redstone-cold at 163.com Fri Aug 19 09:13:55 2016 From: redstone-cold at 163.com (iMath) Date: Fri, 19 Aug 2016 06:13:55 -0700 (PDT) Subject: use regex to search the page one time to get two types of Information In-Reply-To: References: <7a839128-fcc4-471f-809a-9e35896bcd6b@googlegroups.com> Message-ID: <25d01d2f-c10b-43dd-85c4-f90779ad4966@googlegroups.com> 1. searching the page two times rather than one is a little bit time consuming . 2. starting the second search from the first match.endpos does reduce the time consuming . 3. how to combine both patterns into one regex? while using the special | regex operator only matches one regex not both From rosuav at gmail.com Fri Aug 19 09:19:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Aug 2016 23:19:04 +1000 Subject: use regex to search the page one time to get two types of Information In-Reply-To: <25d01d2f-c10b-43dd-85c4-f90779ad4966@googlegroups.com> References: <7a839128-fcc4-471f-809a-9e35896bcd6b@googlegroups.com> <25d01d2f-c10b-43dd-85c4-f90779ad4966@googlegroups.com> Message-ID: On Fri, Aug 19, 2016 at 11:13 PM, iMath wrote: > 1. searching the page two times rather than one is a little bit time consuming . Have you measured that? If not, ignore it. You're searching a web page; if you're downloading that before you search it, chances are very good that you spend far more time waiting for the download than you ever will on the regex. ChrisA From anthra.norell at bluewin.ch Fri Aug 19 09:43:40 2016 From: anthra.norell at bluewin.ch (Friedrich Rentsch) Date: Fri, 19 Aug 2016 15:43:40 +0200 Subject: use regex to search the page one time to get two types of Information In-Reply-To: References: Message-ID: <57B70D0C.3080404@bluewin.ch> On 08/19/2016 09:02 AM, iMath wrote: > I need to use regex to search two types of Information within a web page, while it seems searching the page two times rather than one is much time consuming , is it possible to search the page one time to get two or more types of Information? >>> r = re.compile ('page|Information|time') >>> r.findall ( (your post) ) ['Information', 'page', 'page', 'time', 'time', 'page', 'time', 'Information'] Does that look right? Frederic From steve+python at pearwood.info Fri Aug 19 10:09:05 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 20 Aug 2016 00:09:05 +1000 Subject: index for regex.search() beyond which the RE engine will not go. References: <8330b70c-f516-4aed-a690-aeec6bc78ac3@googlegroups.com> Message-ID: <57b71302$0$1586$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Aug 2016 09:14 pm, iMath wrote: > > for > regex.search(string[, pos[, endpos]]) > The optional parameter endpos is the index into the string beyond which > the RE engine will not go, while this lead me to believe the RE engine > will still search on till the endpos position even after it returned the > matched object, is this Right ? No. Once the RE engine finds a match, it stops. You can test this for yourself with a small timing test, using the "timeit" module. from timeit import Timer huge_string = 'aaabc' + 'a'*1000000 + 'dea' re1 = r'ab.a' re2 = r'ad.a' # set up some code to time. setup = 'import re; from __main__ import huge_string, re1, re2' t1 = Timer('re.search(re1, huge_string)', setup) t2 = Timer('re.search(re2, huge_string)', setup) # Now run the timers. best = min(t1.repeat(number=1000))/1000 print("Time to locate regex at the start of huge string:", best) best = min(t2.repeat(number=1000))/1000 print("Time to locate regex at the end of the huge string:", best) When I run that on my computer, it prints: Time to locate regex at the start of huge string: 4.9710273742675785e-06 Time to locate regex at the end of the huge string: 0.0038938069343566893 So it takes about 4.9 microseconds to find the regex at the beginning of the string. To find the regex at the end of the string takes about 3893 microseconds. The "endpos" parameter tells the RE engine to stop at that position if the regex isn't found before it. It won't go beyond that point. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Aug 19 10:10:18 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 20 Aug 2016 00:10:18 +1000 Subject: index for regex.search() beyond which the RE engine will not go. References: <8330b70c-f516-4aed-a690-aeec6bc78ac3@googlegroups.com> Message-ID: <57b7134a$0$1586$c3e8da3$5496439d@news.astraweb.com> On Fri, 19 Aug 2016 09:21 pm, Jon Ribbens wrote: > On 2016-08-19, iMath wrote: >> for >> regex.search(string[, pos[, endpos]]) >> The optional parameter endpos is the index into the string beyond >> which the RE engine will not go, while this lead me to believe the >> RE engine will still search on till the endpos position even after >> it returned the matched object, is this Right ? > > I am not able rightly to apprehend the kind of confusion of ideas > that could provoke such a question. Consider that iMath's native language may not be English, and that he or she may not understand how regular expression matching works. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From larry.martell at gmail.com Fri Aug 19 13:10:47 2016 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 19 Aug 2016 13:10:47 -0400 Subject: saving octet-stream png file Message-ID: I have some python code (part of a django app) that processes a request that contains a png file. The request is send with content_type = 'application/octet-stream' In the python code I want to write this data to a file and still have it still be a valid png file. The data I get looks like this: u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\x00\x00\x01\ufffd ......' If I try and write that to a file it fails with a UnicodeEncodeError. If I write it with encode('utf8') it writes the file, but then it's no longer a valid png file. Anyone know how I can do this? From rosuav at gmail.com Fri Aug 19 13:24:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Aug 2016 03:24:28 +1000 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell wrote: > I have some python code (part of a django app) that processes a > request that contains a png file. The request is send with > content_type = 'application/octet-stream' > > In the python code I want to write this data to a file and still have > it still be a valid png file. > > The data I get looks like this: > > u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\x00\x00\x01\ufffd > ......' > > If I try and write that to a file it fails with a UnicodeEncodeError. > If I write it with encode('utf8') it writes the file, but then it's no > longer a valid png file. > > Anyone know how I can do this? At that point, you've already lost information. Each U+FFFD (shown as "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" (or whatever was being used). Something somewhere took the .png file's bytes and tried to interpret them as text, which they're not. What sent you that data? How did you receive it? ChrisA From tjreedy at udel.edu Fri Aug 19 13:52:57 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 19 Aug 2016 13:52:57 -0400 Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: On 8/19/2016 4:56 AM, Lawrence D?Oliveiro wrote: > It is handy to be able to keep complex expressions together sometimes, when breaking them up would simply obscure their structure. To avoid lines getting long, why not take advantage of the two available screen/page dimensions to make their structure clearer? As a bonus, spacing out parentheses makes them look less of a clutter. > > Examples from : To me, putting parens and '+' and 'or' on separate lines emphasizes them too much and makes the layout more, not less, cluttered. But I won't argue with what you do in your own private code. > A function call with complex arguments (specifying arguments by keywords is highly recommended here): > > rect_1_pattern = \ > qah.Pattern.create_linear \ > ( > p0 = (0, 0), > p1 = (major_dim, 0), > colour_stops = > ( > (0, rect_1_colour), > (1, complement(rect_1_colour)), > ) > ) > > Computing a variable value (using redundant parentheses to avoid backslash-continuations): > > dest_rect = \ > ( > draw_bounds > + > Vector(col, row) * draw_bounds.dimensions > + > Vector(0, top_extra) > ) > > From , a complex condition (with redundant parentheses again): > > if ( > not isinstance(src, Image) > or > mask != None and not isinstance(mask, Image) > or > not isinstance(dest, Image) > ) : > raise TypeError("image args must be Image objects") > #end if > -- Terry Jan Reedy From tjreedy at udel.edu Fri Aug 19 14:03:30 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 19 Aug 2016 14:03:30 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On 8/19/2016 1:10 PM, Larry Martell wrote: > I have some python code (part of a django app) that processes a > request that contains a png file. The request is send with > content_type = 'application/octet-stream' An 'octet' is a byte of 8 bits. So the content is a stream of bytes and MUST NOT be decoded as unicode text. > In the python code I want to write this data to a file and still have > it still be a valid png file. > > The data I get looks like this: > > u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\x00\x00\x01\ufffd > ......' The data you got looked like b'...PNG...' where the *ascii* codes for "PNG' identify it as a png byte stream. It was mistakenly decoded to unicode text by something. Png bytes must be decoded, when decoded, to a png image. You want to write the bytes to a file exactly as received, without decoding. > If I try and write that to a file it fails with a UnicodeEncodeError. > If I write it with encode('utf8') it writes the file, but then it's no > longer a valid png file. The data ceased representing a png image as soon as wrongfully decoded as unicode text. -- Terry Jan Reedy From larry.martell at gmail.com Fri Aug 19 15:00:12 2016 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 19 Aug 2016 15:00:12 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Fri, Aug 19, 2016 at 1:24 PM, Chris Angelico wrote: > On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell wrote: >> I have some python code (part of a django app) that processes a >> request that contains a png file. The request is send with >> content_type = 'application/octet-stream' >> >> In the python code I want to write this data to a file and still have >> it still be a valid png file. >> >> The data I get looks like this: >> >> u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\x00\x00\x01\ufffd >> ......' >> >> If I try and write that to a file it fails with a UnicodeEncodeError. >> If I write it with encode('utf8') it writes the file, but then it's no >> longer a valid png file. >> >> Anyone know how I can do this? > > At that point, you've already lost information. Each U+FFFD (shown as > "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" > (or whatever was being used). Something somewhere took the .png file's > bytes and tried to interpret them as text, which they're not. > > What sent you that data? How did you receive it? The request is sent by a client app written in C++ with Qt. It's received by a django based server. I am trying to port a falcon server to django. The falcon server code did this: form = cgi.FieldStorage(fp=req.stream, environ=req.env) and then wrote the png like this: fd.write(form[key].file.read()) Whereas in the django server I am doing: fd.write(request.POST[key]) I've never used the cgi module. I guess I can try that. I've written a lot with django but never had to receive a PNG file. From ldompeling at casema.nl Fri Aug 19 15:29:25 2016 From: ldompeling at casema.nl (ldompeling at casema.nl) Date: Fri, 19 Aug 2016 12:29:25 -0700 (PDT) Subject: python module install manully Message-ID: I have a module insteon.py what I need to install manually because it is not in the python index packages. I copied the module in /usr/lib/python2.7/dist-packages and /usr/local/lib/python2.7/dist-packages. When I try to import the module insteon.py I get this error: Does someone knows what the problem can be. Traceback (most recent call last): File "main.py", line 15, in from insteon import Insteon File "/usr/lib/python2.7/dist-packages/insteon.py", line 25, in config = configuration.loadConfig() AttributeError: Configuration instance has no attribute 'loadConfig' From larry.martell at gmail.com Fri Aug 19 15:45:33 2016 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 19 Aug 2016 15:45:33 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Fri, Aug 19, 2016 at 3:00 PM, Larry Martell wrote: > On Fri, Aug 19, 2016 at 1:24 PM, Chris Angelico wrote: >> On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell wrote: >>> I have some python code (part of a django app) that processes a >>> request that contains a png file. The request is send with >>> content_type = 'application/octet-stream' >>> >>> In the python code I want to write this data to a file and still have >>> it still be a valid png file. >>> >>> The data I get looks like this: >>> >>> u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\x00\x00\x01\ufffd >>> ......' >>> >>> If I try and write that to a file it fails with a UnicodeEncodeError. >>> If I write it with encode('utf8') it writes the file, but then it's no >>> longer a valid png file. >>> >>> Anyone know how I can do this? >> >> At that point, you've already lost information. Each U+FFFD (shown as >> "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" >> (or whatever was being used). Something somewhere took the .png file's >> bytes and tried to interpret them as text, which they're not. >> >> What sent you that data? How did you receive it? > > The request is sent by a client app written in C++ with Qt. It's > received by a django based server. I am trying to port a falcon server > to django. The falcon server code did this: > > form = cgi.FieldStorage(fp=req.stream, environ=req.env) > > and then wrote the png like this: > > fd.write(form[key].file.read()) > > Whereas in the django server I am doing: > > fd.write(request.POST[key]) > > I've never used the cgi module. I guess I can try that. I've written a > lot with django but never had to receive a PNG file. No joy using cgi.FieldStorage. The request I get (of type django.core.handlers.wsgi.WSGIRequest) does not have a stream method. I'm sure there's some way to do this, but I have not come up with anything googling. Going to try the django list. From codewizard at gmail.com Fri Aug 19 15:51:56 2016 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 19 Aug 2016 12:51:56 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> For some special cases, I prefer the versions below. On Friday, August 19, 2016 at 4:56:31 AM UTC-4, Lawrence D?Oliveiro wrote: > [snip] > > Computing a variable value (using redundant parentheses to avoid backslash-continuations): > > dest_rect = \ > ( > draw_bounds > + > Vector(col, row) * draw_bounds.dimensions > + > Vector(0, top_extra) > ) dest_rect = sum([ draw_bounds, (col, row) * draw_bounds.dimensions, Vector(0, top_extra), ]) > From , a complex condition (with redundant parentheses again): > > if ( > not isinstance(src, Image) > or > mask != None and not isinstance(mask, Image) > or > not isinstance(dest, Image) > ) : > raise TypeError("image args must be Image objects") > #end if if any([ not isinstance(src, Image), mask != None and not isinstance(mask, Image), not isinstance(dest, Image), ]): raise TypeError("image args must be Image objects") Or equivalently: if not all([ isinstance(src, Image), mask is None or isinstance(mask, Image), isinstance(dest, Image), ]): raise TypeError("image args must be Image objects") From ckaynor at zindagigames.com Fri Aug 19 16:24:23 2016 From: ckaynor at zindagigames.com (Chris Kaynor) Date: Fri, 19 Aug 2016 13:24:23 -0700 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Fri, Aug 19, 2016 at 12:00 PM, Larry Martell wrote: > On Fri, Aug 19, 2016 at 1:24 PM, Chris Angelico wrote: > > On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell > wrote: > >> I have some python code (part of a django app) that processes a > >> request that contains a png file. The request is send with > >> content_type = 'application/octet-stream' > >> > >> In the python code I want to write this data to a file and still have > >> it still be a valid png file. > >> > >> The data I get looks like this: > >> > >> u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\ > x00\x00\x01\ufffd > >> ......' > >> > >> If I try and write that to a file it fails with a UnicodeEncodeError. > >> If I write it with encode('utf8') it writes the file, but then it's no > >> longer a valid png file. > >> > >> Anyone know how I can do this? > > > > At that point, you've already lost information. Each U+FFFD (shown as > > "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" > > (or whatever was being used). Something somewhere took the .png file's > > bytes and tried to interpret them as text, which they're not. > > > > What sent you that data? How did you receive it? > > The request is sent by a client app written in C++ with Qt. It's > received by a django based server. I am trying to port a falcon server > to django. The falcon server code did this: > > form = cgi.FieldStorage(fp=req.stream, environ=req.env) > > and then wrote the png like this: > > fd.write(form[key].file.read()) > > Whereas in the django server I am doing: > > fd.write(request.POST[key]) > > I've never used the cgi module. I guess I can try that. I've written a > lot with django but never had to receive a PNG file. > > I don't know Django, however a quick search makes it seem like you might need to use request.FILES[key] (1) rather than request.POST[key]. You may also be able to use request.POST if you set request.encoding first (2). If both of those fail, you may need to use request.body and parse the HTTP form data manually, though I'd imagine there is an easier way. [1] https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.FILES [2] https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.encoding From lawrencedo99 at gmail.com Fri Aug 19 16:51:10 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 13:51:10 -0700 (PDT) Subject: saving octet-stream png file In-Reply-To: References: Message-ID: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: > > An 'octet' is a byte of 8 bits. Is there any other size of byte? From random832 at fastmail.com Fri Aug 19 17:10:05 2016 From: random832 at fastmail.com (Random832) Date: Fri, 19 Aug 2016 17:10:05 -0400 Subject: saving octet-stream png file In-Reply-To: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> Message-ID: <1471641005.3339534.700559201.3B7C80F5@webmail.messagingengine.com> On Fri, Aug 19, 2016, at 16:51, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: > > > > An 'octet' is a byte of 8 bits. > > Is there any other size of byte? Not very often anymore. Used to be some systems had 9-bit bytes, and of course a lot of communication protocols only supported 7-bit data bytes. "Byte" is a technical term in the C and C++ standards meaning the smallest addressable unit even if that is a larger word. From lawrencedo99 at gmail.com Fri Aug 19 17:30:09 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 14:30:09 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> Message-ID: On Saturday, August 20, 2016 at 7:52:09 AM UTC+12, codew... at gmail.com wrote: > if any([ > not isinstance(src, Image), > mask != None and not isinstance(mask, Image), > not isinstance(dest, Image), > ]): > raise TypeError("image args must be Image objects") > > Or equivalently: > > if not all([ > isinstance(src, Image), > mask is None or isinstance(mask, Image), > isinstance(dest, Image), > ]): > raise TypeError("image args must be Image objects") Using ?all? or ?any? in this sort of situation may not be such a good idea. More reasonable uses would be like : if any(uvface.flipped for uvface in island.faces): if any(island.bounding_box.x > cage_size.x or island.bounding_box.y > cage_size.y for island in self.islands): From mcgee at ndnj.org Fri Aug 19 17:48:18 2016 From: mcgee at ndnj.org (Suzanna McGee) Date: Fri, 19 Aug 2016 17:48:18 -0400 Subject: Python Run Error Message-ID: I am unable to run Python on my computer because I keep getting the follow error: ?The program can?t start because api-ms-win-crt-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem.? -- Suzanna McGee Computer Science Teacher Notre Dame High School Lawrenceville, New Jersey From lawrencedo99 at gmail.com Fri Aug 19 17:50:59 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 14:50:59 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <746305b3-d12f-4572-baf8-35da7ebbf8d6@googlegroups.com> On Saturday, August 20, 2016 at 5:53:22 AM UTC+12, Terry Reedy wrote: > > To me, putting parens and '+' and 'or' on separate lines emphasizes them > too much and makes the layout more, not less, cluttered. So having whitespace around these symbols makes things look *more* cluttered, while packing them together without whitespace makes them *less* cluttered? From codewizard at gmail.com Fri Aug 19 17:55:52 2016 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 19 Aug 2016 14:55:52 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> Message-ID: <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> On Friday, August 19, 2016 at 5:30:22 PM UTC-4, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 7:52:09 AM UTC+12, codew... at gmail.com wrote: > > if any([ > > not isinstance(src, Image), > > mask != None and not isinstance(mask, Image), > > not isinstance(dest, Image), > > ]): > > raise TypeError("image args must be Image objects") > > > > Or equivalently: > > > > if not all([ > > isinstance(src, Image), > > mask is None or isinstance(mask, Image), > > isinstance(dest, Image), > > ]): > > raise TypeError("image args must be Image objects") > > Using ?all? or ?any? in this sort of situation may not be such a good idea. More reasonable uses would be like : > > if any(uvface.flipped for uvface in island.faces): > > if any(island.bounding_box.x > cage_size.x or island.bounding_box.y > cage_size.y for island in self.islands): You showed examples where you think it would be a good idea to use any/all. But you didn't say why my examples "may not be such a good idea". Would you care to elaborate? From lawrencedo99 at gmail.com Fri Aug 19 18:31:01 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 15:31:01 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> Message-ID: <262af6dd-9559-4ed2-8944-fb44ae688eb1@googlegroups.com> On Saturday, August 20, 2016 at 9:56:05 AM UTC+12, codew... at gmail.com wrote: > > On Friday, August 19, 2016 at 5:30:22 PM UTC-4, Lawrence D?Oliveiro wrote: > >> On Saturday, August 20, 2016 at 7:52:09 AM UTC+12, codew... at gmail.com >> wrote: >>> if any([ >>> not isinstance(src, Image), >>> mask != None and not isinstance(mask, Image), >>> not isinstance(dest, Image), >>> ]): >>> raise TypeError("image args must be Image objects") >>> >>> Or equivalently: >>> >>> if not all([ >>> isinstance(src, Image), >>> mask is None or isinstance(mask, Image), >>> isinstance(dest, Image), >>> ]): >>> raise TypeError("image args must be Image objects") >> >> Using ?all? or ?any? in this sort of situation may not be such a good >> idea. > > Would you care to elaborate? There is no short-cut evaluation when constructing tuples and lists. From lawrencedo99 at gmail.com Fri Aug 19 18:31:28 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 15:31:28 -0700 (PDT) Subject: Python Run Error In-Reply-To: References: Message-ID: On Saturday, August 20, 2016 at 10:01:08 AM UTC+12, Suzanna McGee wrote: > ?The program can?t start because api-ms-win-crt-runtime-l1-1-0.dll is > missing from your computer. Try reinstalling the program to fix this > problem.? Why not do what it says? From rosuav at gmail.com Fri Aug 19 18:38:18 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Aug 2016 08:38:18 +1000 Subject: Two-Dimensional Expression Layout In-Reply-To: <262af6dd-9559-4ed2-8944-fb44ae688eb1@googlegroups.com> References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> <262af6dd-9559-4ed2-8944-fb44ae688eb1@googlegroups.com> Message-ID: On Sat, Aug 20, 2016 at 8:31 AM, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 9:56:05 AM UTC+12, codew... at gmail.com wrote: >> >> On Friday, August 19, 2016 at 5:30:22 PM UTC-4, Lawrence D?Oliveiro wrote: >> >>> On Saturday, August 20, 2016 at 7:52:09 AM UTC+12, codew... at gmail.com >>> wrote: >>>> if any([ >>>> not isinstance(src, Image), >>>> mask != None and not isinstance(mask, Image), >>>> not isinstance(dest, Image), >>>> ]): >>>> raise TypeError("image args must be Image objects") >>>> >>>> Or equivalently: >>>> >>>> if not all([ >>>> isinstance(src, Image), >>>> mask is None or isinstance(mask, Image), >>>> isinstance(dest, Image), >>>> ]): >>>> raise TypeError("image args must be Image objects") >>> >>> Using ?all? or ?any? in this sort of situation may not be such a good >>> idea. >> >> Would you care to elaborate? > > There is no short-cut evaluation when constructing tuples and lists. I'm not sure how that would make difference in these examples. The three parts are independent - the one place where short-circuiting is important is indeed short-circuited. ChrisA From codewizard at gmail.com Fri Aug 19 18:50:43 2016 From: codewizard at gmail.com (codewizard at gmail.com) Date: Fri, 19 Aug 2016 15:50:43 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> <262af6dd-9559-4ed2-8944-fb44ae688eb1@googlegroups.com> Message-ID: On Friday, August 19, 2016 at 6:38:34 PM UTC-4, Chris Angelico wrote: > On Sat, Aug 20, 2016 at 8:31 AM, Lawrence D?Oliveiro > wrote: > > On Saturday, August 20, 2016 at 9:56:05 AM UTC+12, codew... at gmail.com wrote: > >> > >> On Friday, August 19, 2016 at 5:30:22 PM UTC-4, Lawrence D?Oliveiro wrote: > >> > >>> On Saturday, August 20, 2016 at 7:52:09 AM UTC+12, codew... at gmail.com > >>> wrote: > >>>> if any([ > >>>> not isinstance(src, Image), > >>>> mask != None and not isinstance(mask, Image), > >>>> not isinstance(dest, Image), > >>>> ]): > >>>> raise TypeError("image args must be Image objects") > >>>> > >>>> Or equivalently: > >>>> > >>>> if not all([ > >>>> isinstance(src, Image), > >>>> mask is None or isinstance(mask, Image), > >>>> isinstance(dest, Image), > >>>> ]): > >>>> raise TypeError("image args must be Image objects") > >>> > >>> Using ?all? or ?any? in this sort of situation may not be such a good > >>> idea. > >> > >> Would you care to elaborate? > > > > There is no short-cut evaluation when constructing tuples and lists. > > I'm not sure how that would make difference in these examples. The > three parts are independent - the one place where short-circuiting is > important is indeed short-circuited. > > ChrisA Agreed. Besides, just because this technique has limitations, it's still useful for me to improve readability. From lawrencedo99 at gmail.com Fri Aug 19 19:11:55 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 16:11:55 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <60992a52-cefe-4209-abc3-2e8400bd6cfd@googlegroups.com> <3ebb8ece-0c7c-42d2-9c27-d5c76a685eb4@googlegroups.com> <262af6dd-9559-4ed2-8944-fb44ae688eb1@googlegroups.com> Message-ID: On Saturday, August 20, 2016 at 10:38:34 AM UTC+12, Chris Angelico wrote: > > On Sat, Aug 20, 2016 at 8:31 AM, Lawrence D?Oliveiro wrote: >> >> There is no short-cut evaluation when constructing tuples and lists. > > I'm not sure how that would make difference in these examples. The > three parts are independent - the one place where short-circuiting is > important is indeed short-circuited. That often is not the case, e.g. : assert \ ( len(self.points) == 0 or not self.points[0].off and (closed or not self.points[-1].off) ) From lawrencedo99 at gmail.com Fri Aug 19 19:29:08 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 16:29:08 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <6e801e53-fd74-418f-ba73-ee1144e18316@googlegroups.com> On Friday, August 19, 2016 at 8:56:31 PM UTC+12, I wrote: > To avoid lines getting long, why not take advantage of the two available > screen/page dimensions to make [expression] structure clearer? Another aspect of this has to do with line length. I regularly set my editor window width to around 100 columns. But if you exclude indentation whitespace, my lines are usually much shorter than that. There is a limit to the length of text lines that a human reader can cope with (which is why newspapers, for example, tend to have a many-column layout). Using two-dimensional layout and indentation helps to keep the (non-whitespace) line length within reasonable limits. From lawrencedo99 at gmail.com Fri Aug 19 19:42:12 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 16:42:12 -0700 (PDT) Subject: Does This Scare You? Message-ID: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Python 3.5.2+ (default, Aug 5 2016, 08:07:14) [GCC 6.1.1 20160724] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from pathlib import PureWindowsPath >>> PureWindowsPath("prn").is_reserved() True >>> PureWindowsPath("prn.doc").is_reserved() True >>> PureWindowsPath("com9.exe").is_reserved() True >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved() True From lawrencedo99 at gmail.com Fri Aug 19 19:55:26 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 16:55:26 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87y43uadke.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> Message-ID: <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> On Thursday, August 18, 2016 at 4:47:28 PM UTC+12, Marko Rauhamaa wrote: > ... as a rule, I dislike rules. Rule languages tend to > grow out of all bounds, always remain deficient and have impenetrable, > ad-hoc semantics. That?s a very peculiar thing to say, considering that data-driven programming is a well-known technique for writing compact code. Less code, and in particular, less repetitive code => fewer bugs. From python at mrabarnett.plus.com Fri Aug 19 19:56:54 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 20 Aug 2016 00:56:54 +0100 Subject: Python Run Error In-Reply-To: References: Message-ID: <4c1744cc-49dd-80aa-e589-9ecc35a7caa2@mrabarnett.plus.com> On 2016-08-19 23:31, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 10:01:08 AM UTC+12, Suzanna McGee wrote: >> ?The program can?t start because api-ms-win-crt-runtime-l1-1-0.dll is >> missing from your computer. Try reinstalling the program to fix this >> problem.? > > Why not do what it says? > Because that won't fix it. It needs the Universal C Runtime. An up-to-date system should already have it. Read here: Update for Universal C Runtime in Windows https://support.microsoft.com/en-us/kb/2999226 From rosuav at gmail.com Fri Aug 19 20:57:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Aug 2016 10:57:37 +1000 Subject: Does This Scare You? In-Reply-To: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: On Sat, Aug 20, 2016 at 9:42 AM, Lawrence D?Oliveiro wrote: > Python 3.5.2+ (default, Aug 5 2016, 08:07:14) > [GCC 6.1.1 20160724] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from pathlib import PureWindowsPath > >>> PureWindowsPath("prn").is_reserved() > True > >>> PureWindowsPath("prn.doc").is_reserved() > True > >>> PureWindowsPath("com9.exe").is_reserved() > True > >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved() > True When was the last time you wanted to create a file with a reserved name? Paths, drive letters, file extensions, don't matter. All that matters is the base name. Not a Python issue; they're reserved by Windows. ChrisA From steve+python at pearwood.info Fri Aug 19 21:09:24 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 20 Aug 2016 11:09:24 +1000 Subject: saving octet-stream png file References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> Message-ID: <57b7adc5$0$1609$c3e8da3$5496439d@news.astraweb.com> On Sat, 20 Aug 2016 06:51 am, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: >> >> An 'octet' is a byte of 8 bits. > > Is there any other size of byte? Depends what you mean by "byte", but the short answer is "Yes". In the C/C++ standard, bytes must be at least eight bytes. As the below FAQ explains, that means that on machines like the PDP-10 a C++ compiler will define bytes to be 32 bits. One common definition of "byte" is the smallest addressable unit of memory. On that basis, there have been machines like the Control Data 6600 where a byte was 60 bits. Honeywell machines used 9 bits. Digital signal processes (DSPs) frequently have bytes with more than eight bits, such as Texas Instruments C54x DSPs (16 bit bytes), BelaSigna DSPs (24 bits) and DSP56K/Symphony Audio DSPs (24 bits). The Saturn CPU (used in the HP-48SX/GX calculator line) addresses memory 4-bit bytes. Windows CE took the unusual, and non-conformant, approach of running on hardware with 16 bit bytes and simply not defining "char" (and presumably "byte") in their C compiler. See: https://isocpp.org/wiki/faq/intrinsic-types http://stackoverflow.com/questions/5516044/system-where-1-byte-8-bit http://stackoverflow.com/questions/2098149/what-platforms-have-something-other-than-8-bit-char http://programmers.stackexchange.com/questions/120126/what-is-the-history-of-why-bytes-are-eight-bits -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From best_lay at yahoo.com Fri Aug 19 21:11:54 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 19 Aug 2016 20:11:54 -0500 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: On Sat, 20 Aug 2016 10:57:37 +1000, Chris Angelico wrote: > On Sat, Aug 20, 2016 at 9:42 AM, Lawrence D?Oliveiro > wrote: >> Python 3.5.2+ (default, Aug 5 2016, 08:07:14) >> [GCC 6.1.1 20160724] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> from pathlib import PureWindowsPath >> >>> PureWindowsPath("prn").is_reserved() >> True >> >>> PureWindowsPath("prn.doc").is_reserved() >> True >> >>> PureWindowsPath("com9.exe").is_reserved() >> True >> >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved() >> True > > When was the last time you wanted to create a file with a reserved > name? Paths, drive letters, file extensions, don't matter. All that > matters is the base name. > > Not a Python issue; they're reserved by Windows. > > ChrisA Since I am fairly new to Python, I realize there is much that I still don't know but I don't understand how Windows can have reserved names on a Linux system. What am I missing? -- GNU/Linux user #557453 The cow died so I don't need your bull! From rosuav at gmail.com Fri Aug 19 21:20:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Aug 2016 11:20:44 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: On Sat, Aug 20, 2016 at 11:11 AM, Wildman via Python-list wrote: > Since I am fairly new to Python, I realize there is much that I > still don't know but I don't understand how Windows can have > reserved names on a Linux system. What am I missing? The PureWindowsPath class is specifically working with the Windows file system rules. Those names aren't reserved under Linux, but you can still ask the pathlib module whether or not they *would be* reserved under Windows. This is very handy if you're planning to create files and don't know whether they'd work or not - you don't need to grab a Windows machine (virtual or physical) to test stuff. ChrisA From lawrencedo99 at gmail.com Fri Aug 19 21:58:07 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Fri, 19 Aug 2016 18:58:07 -0700 (PDT) Subject: Python Run Error In-Reply-To: References: <4c1744cc-49dd-80aa-e589-9ecc35a7caa2@mrabarnett.plus.com> Message-ID: On Saturday, August 20, 2016 at 11:57:16 AM UTC+12, MRAB wrote: > > On 2016-08-19 23:31, Lawrence D?Oliveiro wrote: > >> On Saturday, August 20, 2016 at 10:01:08 AM UTC+12, Suzanna McGee wrote: >>> ?The program can?t start because api-ms-win-crt-runtime-l1-1-0.dll is >>> missing from your computer. Try reinstalling the program to fix this >>> problem.? >> >> Why not do what it says? >> > Because that won't fix it. You mean, Windows is lying? Say it isn?t so! From random832 at fastmail.com Fri Aug 19 23:23:48 2016 From: random832 at fastmail.com (Random832) Date: Fri, 19 Aug 2016 23:23:48 -0400 Subject: saving octet-stream png file In-Reply-To: <57b7adc5$0$1609$c3e8da3$5496439d@news.astraweb.com> References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> <57b7adc5$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1471663428.2778372.700733897.0C38310E@webmail.messagingengine.com> On Fri, Aug 19, 2016, at 21:09, Steve D'Aprano wrote: > Depends what you mean by "byte", but the short answer is "Yes". > > In the C/C++ standard, bytes must be at least eight bytes. As the below > FAQ > explains, that means that on machines like the PDP-10 a C++ compiler will > define bytes to be 32 bits. I assume you mean 36, but I think this is mixing up two separate parts of the FAQ along with some theory discussion. AFAIK all historical C implementations for the PDP-10 and other 18- or 36-bit word systems have used the "9-bit bytes, and an extra offset member in char and void pointers if necessary" solution rather than the "word-sized byte" solution. In principle, I think a close reading of the standard would allow for an implementation can have 'skipped bits' as discussed there so long as there is no significance to the 'missing' bits in *any* type - so you could legally have 8-bit bytes on a PDP-10 so long as you also had 16-bit shorts, 32-bit long/pointer/float, and in general *never ever* broke the illusion that only the bits addressable via char pointers exist. Modern implementations aren't expected to expose ECC bits, after all. Such an implementation would probably be best done by ignoring the high bits of each word, which are easily masked off and often have no real use anyway in pointers (How many of these systems supported 2^36 words = 288 modern gigabytes of memory?), rather than by ignoring a single bit "between" each byte. AIUI many PDP-10 applications only used 18 bits for word pointers. From best_lay at yahoo.com Fri Aug 19 23:46:20 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 19 Aug 2016 22:46:20 -0500 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: On Sat, 20 Aug 2016 11:20:44 +1000, Chris Angelico wrote: > On Sat, Aug 20, 2016 at 11:11 AM, Wildman via Python-list > wrote: >> Since I am fairly new to Python, I realize there is much that I >> still don't know but I don't understand how Windows can have >> reserved names on a Linux system. What am I missing? > > The PureWindowsPath class is specifically working with the Windows > file system rules. Those names aren't reserved under Linux, but you > can still ask the pathlib module whether or not they *would be* > reserved under Windows. This is very handy if you're planning to > create files and don't know whether they'd work or not - you don't > need to grab a Windows machine (virtual or physical) to test stuff. > > ChrisA OK. I understand. Python is after all cross-platform so that makes perfect sense. Thanks. -- GNU/Linux user #557453 The voices in my head may not be real but, they have some good ideas. From arsh840 at gmail.com Sat Aug 20 00:53:25 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Fri, 19 Aug 2016 21:53:25 -0700 (PDT) Subject: Holding until next value change Message-ID: I am writing a function as main_call() which is continuously producing values. (+ve or -ve) I want to print on screen only for first +ve value and hold until -ve value comes around. here is my code: def main_call(): while True: yield strategy() for value in main_call(): if(value>0): print '+ve' elif(value>0): print '-ve' else: pass Do I need to use threads or processes? From steve+python at pearwood.info Sat Aug 20 02:07:48 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 20 Aug 2016 16:07:48 +1000 Subject: Holding until next value change References: Message-ID: <57b7f3b6$0$1607$c3e8da3$5496439d@news.astraweb.com> On Sat, 20 Aug 2016 02:53 pm, Arshpreet Singh wrote: > I am writing a function as main_call() which is continuously producing > values. (+ve or -ve) I want to print on screen only for first +ve value > and hold until -ve value comes around. here is my code: > > > def main_call(): > while True: > yield strategy() > > for value in main_call(): > if(value>0): > print '+ve' > elif(value>0): > print '-ve' > else: > pass > > Do I need to use threads or processes? Only if you want a major headache. Your code doesn't do what you want it to do. It prints "+ve" every time it sees a positive value, not just the first time. One solution to this is to think of a state machine: your machine starts off in the state: (1) Ignore negative values, print the first positive value you see, then change to the next state. The second state is: (2) Ignore positive values, print the first negative value you see, then change to the next state. The third state is unspecified. You don't know say what it is, so I'm going to guess that you change to a third state: (3) Don't ignore anything, print every value you see. So here are our three machines: def ignore_negative(x): if x < 0: return False else: print("+ve") return True def ignore_positive(x): if x > 0: return False else: print("-ve") return True def ignore_nothing(x): if x > 0: print("+ve") else: print("-ve") return False Here is a table that specifies the changes in state: TABLE = { # current state: next state ignore_negative: ignore_positive, ignore_positive: ignore_nothing, } And some code to drive it: state = ignore_negative # DON'T call the function yet for value in main_call(): print(value) # for testing if state(value): print("changing state") state = TABLE[state] Another solution is to look at the itertools module, which has two functions "dropwhile" and "takeuntil" that might help. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From arsh840 at gmail.com Sat Aug 20 03:38:25 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Sat, 20 Aug 2016 00:38:25 -0700 (PDT) Subject: Holding until next value change In-Reply-To: <57b7f3b6$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <57b7f3b6$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: <19809cc9-bbdc-47e8-8da8-de572f4daced@googlegroups.com> On Saturday, 20 August 2016 11:38:03 UTC+5:30, Steve D'Aprano wrote: > On Sat, 20 Aug 2016 02:53 pm, Arshpreet Singh wrote: > > > I am writing a function as main_call() which is continuously producing > > values. (+ve or -ve) I want to print on screen only for first +ve value > > and hold until -ve value comes around. here is my code: > > > > > > def main_call(): > > while True: > > yield strategy() > > > > for value in main_call(): > > if(value>0): > > print '+ve' > > elif(value>0): > > print '-ve' > > else: > > pass > > > > Do I need to use threads or processes? > > Only if you want a major headache. > > > Your code doesn't do what you want it to do. It prints "+ve" every time it > sees a positive value, not just the first time. > > One solution to this is to think of a state machine: your machine starts off > in the state: > > (1) Ignore negative values, print the first positive value you see, > then change to the next state. > > The second state is: > > (2) Ignore positive values, print the first negative value you see, > then change to the next state. > > The third state is unspecified. You don't know say what it is, so I'm going > to guess that you change to a third state: > > (3) Don't ignore anything, print every value you see. > > > So here are our three machines: > > def ignore_negative(x): > if x < 0: > return False > else: > print("+ve") > return True > > def ignore_positive(x): > if x > 0: > return False > else: > print("-ve") > return True > > def ignore_nothing(x): > if x > 0: > print("+ve") > else: > print("-ve") > return False > > > Here is a table that specifies the changes in state: > > TABLE = { # current state: next state > ignore_negative: ignore_positive, > ignore_positive: ignore_nothing, > } > > > And some code to drive it: > > > state = ignore_negative # DON'T call the function yet > for value in main_call(): > print(value) # for testing > if state(value): > print("changing state") > state = TABLE[state] Hi Steve, my third state is if x==0 , I tired the same code but here is output I am getting, so we are changing the state even it is printing out the +ve as well as -ve values 8.7664352813e-05 +ve changing state 8.7664352813e-05 8.7664352813e-05 8.7664352813e-05 8.7664352813e-05 8.7664352813e-05 8.76628243948e-05 8.30490294982e-05 8.30490294982e-05 6.45892873188e-05 6.45892873188e-05 6.45892873188e-05 6.45892873188e-05 -4.61232547941e-06 -ve changing state -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve -4.61232547941e-06 -ve From marko at pacujo.net Sat Aug 20 03:50:58 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Aug 2016 10:50:58 +0300 Subject: saving octet-stream png file References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> <1471641005.3339534.700559201.3B7C80F5@webmail.messagingengine.com> Message-ID: <877fbbanfh.fsf@elektro.pacujo.net> Random832 : > On Fri, Aug 19, 2016, at 16:51, Lawrence D?Oliveiro wrote: >> On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: >> > An 'octet' is a byte of 8 bits. >> Is there any other size of byte? > Not very often anymore. The main difference between an octet and a byte is that a "byte" is used when talking about computers while an "octet" is a term used in telecommunications protocols. (If I'm not mistaken, "octet" is also French for "byte".) Somewhat analogously, a programmer can rely on integers being 2's-complement. IOW, even in Python, -X == ~X + 1 for any integer. 2'scomplement arithmetics is quite often taken advantage of in C programming. Unfortunately, with the castration of signed integers with the most recent C standards, 2's-complement has been dangerously broken. Marko From marko at pacujo.net Sat Aug 20 04:08:12 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Aug 2016 11:08:12 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> Message-ID: <8737lzammr.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > On Thursday, August 18, 2016 at 4:47:28 PM UTC+12, Marko Rauhamaa wrote: >> ... as a rule, I dislike rules. Rule languages tend to >> grow out of all bounds, always remain deficient and have impenetrable, >> ad-hoc semantics. > > That?s a very peculiar thing to say, considering that data-driven > programming is a well-known technique for writing compact code. > > Less code, and in particular, less repetitive code => fewer bugs. I can't relate to what you say. All I can say is that I've encountered numerous bad cases of rule systems, eg: - iptables - selinux policies - systemd unit files - asterisk - sendmail Marko From redstone-cold at 163.com Sat Aug 20 05:37:24 2016 From: redstone-cold at 163.com (iMath) Date: Sat, 20 Aug 2016 02:37:24 -0700 (PDT) Subject: use regex to search the page one time to get two types of Information In-Reply-To: References: <7a839128-fcc4-471f-809a-9e35896bcd6b@googlegroups.com> <25d01d2f-c10b-43dd-85c4-f90779ad4966@googlegroups.com> Message-ID: <8a5cd0cc-68e5-4ef6-bf7a-bc2ba284d353@googlegroups.com> On Friday, August 19, 2016 at 9:19:22 PM UTC+8, Chris Angelico wrote: > On Fri, Aug 19, 2016 at 11:13 PM, iMath wrote: > > 1. searching the page two times rather than one is a little bit time consuming . > > Have you measured that? If not, ignore it. You're searching a web > page; if you're downloading that before you search it, chances are > very good that you spend far more time waiting for the download than > you ever will on the regex. > > ChrisA tested, searching the page two times rather than one is a little bit time consuming . From redstone-cold at 163.com Sat Aug 20 05:39:08 2016 From: redstone-cold at 163.com (iMath) Date: Sat, 20 Aug 2016 02:39:08 -0700 (PDT) Subject: use regex to search the page one time to get two types of Information In-Reply-To: References: <57B70D0C.3080404@bluewin.ch> Message-ID: On Friday, August 19, 2016 at 9:45:08 PM UTC+8, Friedrich Rentsch wrote: > On 08/19/2016 09:02 AM, iMath wrote: > > I need to use regex to search two types of Information within a web page, while it seems searching the page two times rather than one is much time consuming , is it possible to search the page one time to get two or more types of Information? > > >>> r = re.compile ('page|Information|time') > >>> r.findall ( (your post) ) > ['Information', 'page', 'page', 'time', 'time', 'page', 'time', > 'Information'] > > Does that look right? > > Frederic I found starting the second search from the first match.endpos does reduce the time consuming, with less time consuming than your solution ,thanks anyway ! From redstone-cold at 163.com Sat Aug 20 05:40:33 2016 From: redstone-cold at 163.com (iMath) Date: Sat, 20 Aug 2016 02:40:33 -0700 (PDT) Subject: index for regex.search() beyond which the RE engine will not go. In-Reply-To: <57b71302$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <8330b70c-f516-4aed-a690-aeec6bc78ac3@googlegroups.com> <57b71302$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <746eba9a-9dcd-4038-9834-e41abd9164f3@googlegroups.com> On Friday, August 19, 2016 at 10:09:19 PM UTC+8, Steve D'Aprano wrote: > On Fri, 19 Aug 2016 09:14 pm, iMath wrote: > > > > > for > > regex.search(string[, pos[, endpos]]) > > The optional parameter endpos is the index into the string beyond which > > the RE engine will not go, while this lead me to believe the RE engine > > will still search on till the endpos position even after it returned the > > matched object, is this Right ? > > No. > > Once the RE engine finds a match, it stops. You can test this for yourself > with a small timing test, using the "timeit" module. > > from timeit import Timer > huge_string = 'aaabc' + 'a'*1000000 + 'dea' > re1 = r'ab.a' > re2 = r'ad.a' > > # set up some code to time. > setup = 'import re; from __main__ import huge_string, re1, re2' > t1 = Timer('re.search(re1, huge_string)', setup) > t2 = Timer('re.search(re2, huge_string)', setup) > > # Now run the timers. > best = min(t1.repeat(number=1000))/1000 > print("Time to locate regex at the start of huge string:", best) > best = min(t2.repeat(number=1000))/1000 > print("Time to locate regex at the end of the huge string:", best) > > > > When I run that on my computer, it prints: > > Time to locate regex at the start of huge string: 4.9710273742675785e-06 > Time to locate regex at the end of the huge string: 0.0038938069343566893 > > > So it takes about 4.9 microseconds to find the regex at the beginning of the > string. To find the regex at the end of the string takes about 3893 > microseconds. > > > The "endpos" parameter tells the RE engine to stop at that position if the > regex isn't found before it. It won't go beyond that point. > > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. Thanks for clarifying From michael.selik at gmail.com Sat Aug 20 08:48:16 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sat, 20 Aug 2016 12:48:16 +0000 Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: On Fri, Aug 19, 2016 at 5:01 AM Lawrence D?Oliveiro wrote: > It is handy to be able to keep complex expressions together sometimes, > when breaking them up would simply obscure their structure. To avoid lines > getting long, why not take advantage of the two available screen/page > dimensions to make their structure clearer? As a bonus, spacing out > parentheses makes them look less of a clutter. > > > Examples from < > https://github.com/ldo/qahirah_examples/blob/master/operators>: > > A function call with complex arguments (specifying arguments by keywords > is highly recommended here): > > rect_1_pattern = \ > qah.Pattern.create_linear \ > ( > p0 = (0, 0), > p1 = (major_dim, 0), > colour_stops = > ( > (0, rect_1_colour), > (1, complement(rect_1_colour)), > ) > ) > I'd rather have intermediate variables and avoid the non-semantic indentation. p0 = (0, 0) p1 = (major_dim, 0) colour_stops = (0, rect_1_colour), (1, complement(rect_1_colour)) rect_1_pattern = qah.Pattern.create_linear(p0, p1, colour_stops) Computing a variable value (using redundant parentheses to avoid > backslash-continuations): > > dest_rect = \ > ( > draw_bounds > + > Vector(col, row) * draw_bounds.dimensions > + > Vector(0, top_extra) > ) > I'd rather have one long line for this case, even if it's "too long" by PEP 8 standards. If it's too long to think about, I'd break the task into parts, each with a well-considered variable name. substep = a + b final = substep + c >From , a > complex condition (with redundant parentheses again): > > if ( > not isinstance(src, Image) > or > mask != None and not isinstance(mask, Image) > or > not isinstance(dest, Image) > ) : > raise TypeError("image args must be Image objects") > #end if > No need for the separate calls to isinstance, nor the check for None. if any(not isinstance(obj, Image) for obj in [src, mask, dest]): ... If you prefer maps to comprehensions: is_image = lambda obj: isinstance(obj, Image) if any(map(is_image, [src, mask, dest])): ... From andrzej.brozi at gmail.com Sat Aug 20 10:18:24 2016 From: andrzej.brozi at gmail.com (andrzej.brozi at gmail.com) Date: Sat, 20 Aug 2016 07:18:24 -0700 (PDT) Subject: Holding until next value change In-Reply-To: References: Message-ID: Hello Perhaps this would suffice: prev = None for value in main_call(): if value==prev: pass else: prev = value if prev>0: print('+v') elif prev<0: print('-v') else: print('0') Of course this will work only if subsequent values are exactly the same. If not - then "value==prev" should be replaced by some other test (for sign or for proximity). Regards Andrzej Brozi From julien.kauffmann at freelan.org Sat Aug 20 11:50:42 2016 From: julien.kauffmann at freelan.org (Julien Kauffmann) Date: Sat, 20 Aug 2016 11:50:42 -0400 Subject: Knowing which thread had the GIL before Message-ID: <9cfa94e2-3f17-1cd1-9de8-e646ea9cd215@freelan.org> Hi, I've worked the past few days on a stastistical sampling profiling tool for an online Python 2.7 service that runs on Windows. My approach so far is: - I have a dedicated thread that sleeps most of the time and wakes up every n-th milliseconds. - Whenever the thread wakes-up, it gets the current frame/callstack for all the active threads (using threading.enumerate()) and increments an occurence counter for each currently executed code path. - When enough time has passed, all the collected samples are saved for analysis. This works somehow, but I fear this technique isn't aligned with how threading works in Python where only one thread can run Python code at a given time (GIL). Consider the following case: - An application has 2 threads running. - Thread A sleeps for 30 seconds. - Thread B does some heavy computation (only Python code). In this case, it is fairly obvious that most of the process execution time will be spent in thread B, which will likely have the GIL most of the time (since thread A is sleeping). However, when measuring the activity with the technique mentionned above, the sleep instruction in Thread B appears as the most costly code location, has it is present in every profiling sample. I'd like to modify my approach above by only measuring the activity of the last active thread. Obviously, this is not as simple as calling `threading.current_thread()` because it will always give me the profiling thread which is currently executing. Is there a way to know which thread had the GIL right before my profiling thread acquired it ? Perhaps through a C extension ? I've seen such profilers for Linux that take advantage of an ITIMER signal to do that. Sadly this is not an option on Windows. Any feedback or remark concerning my technique is also welcome. Thanks, Julien. From random832 at fastmail.com Sat Aug 20 14:10:01 2016 From: random832 at fastmail.com (Random832) Date: Sat, 20 Aug 2016 14:10:01 -0400 Subject: saving octet-stream png file In-Reply-To: <877fbbanfh.fsf@elektro.pacujo.net> References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> <1471641005.3339534.700559201.3B7C80F5@webmail.messagingengine.com> <877fbbanfh.fsf@elektro.pacujo.net> Message-ID: <1471716601.2929499.701110809.2C4F6700@webmail.messagingengine.com> On Sat, Aug 20, 2016, at 03:50, Marko Rauhamaa wrote: > 2'scomplement arithmetics is quite often taken advantage of in C > programming. Unfortunately, with the castration of signed integers with > the most recent C standards, 2's-complement has been dangerously broken. No part of any version of the C standard has ever allowed signed integer overflow to work as defined behavior the way a generation of programmers assumed it did. What changed was advances in compiler optimization technology, not a standards change. From grant.b.edwards at gmail.com Sat Aug 20 15:55:19 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 20 Aug 2016 19:55:19 +0000 (UTC) Subject: saving octet-stream png file References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> <1471641005.3339534.700559201.3B7C80F5@webmail.messagingengine.com> Message-ID: On 2016-08-19, Random832 wrote: > On Fri, Aug 19, 2016, at 16:51, Lawrence D?Oliveiro wrote: >> On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: >> > >> > An 'octet' is a byte of 8 bits. >> >> Is there any other size of byte? > > Not very often anymore. Used to be some systems had 9-bit bytes, and of > course a lot of communication protocols only supported 7-bit data bytes. > "Byte" is a technical term in the C and C++ standards meaning the > smallest addressable unit even if that is a larger word. Last time I looked a lot of DSP chips still have "byte" sizes larger than 8 bits. IIRC, 16, 24, and 32 bits are common byte sizes. Not that Python runs on any of them... -- Grant Edwards grant.b.edwards Yow! A can of ASPARAGUS, at 73 pigeons, some LIVE ammo, gmail.com and a FROZEN DAQUIRI!! From marko at pacujo.net Sat Aug 20 16:36:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Aug 2016 23:36:11 +0300 Subject: saving octet-stream png file References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> <1471641005.3339534.700559201.3B7C80F5@webmail.messagingengine.com> <877fbbanfh.fsf@elektro.pacujo.net> <1471716601.2929499.701110809.2C4F6700@webmail.messagingengine.com> Message-ID: <87lgzr89fo.fsf@elektro.pacujo.net> Random832 : > On Sat, Aug 20, 2016, at 03:50, Marko Rauhamaa wrote: >> 2'scomplement arithmetics is quite often taken advantage of in C >> programming. Unfortunately, with the castration of signed integers >> with the most recent C standards, 2's-complement has been dangerously >> broken. > > No part of any version of the C standard has ever allowed signed > integer overflow to work as defined behavior the way a generation of > programmers assumed it did. Standard or no, it was widely taken advantage of and very useful. > What changed was advances in compiler optimization technology, not a > standards change. I wonder how much is gained by those optimizations. The loss to code quality is significant. What C standards have done is they have all but deprecated the use of signed integers. If you have to do integer arithmetics, cast everything into unsigned first. I think it's terrible that in C, x + y + z might not yield x + y + z even if all of { x, y, z, x + y + z } are inside the valid signed integer range. Marko From lawrencedo99 at gmail.com Sat Aug 20 18:16:30 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 20 Aug 2016 15:16:30 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: On Sunday, August 21, 2016 at 12:48:46 AM UTC+12, Michael Selik wrote: > > On Fri, Aug 19, 2016 at 5:01 AM Lawrence D?Oliveiro wrote: > >> It is handy to be able to keep complex expressions together sometimes, >> when breaking them up would simply obscure their structure. > > p0 = (0, 0) > p1 = (major_dim, 0) > colour_stops = (0, rect_1_colour), (1, complement(rect_1_colour)) > rect_1_pattern = qah.Pattern.create_linear(p0, p1, colour_stops) That?s an example of what I mean about obscure structure. >> From , a >> complex condition (with redundant parentheses again): >> >> if ( >> not isinstance(src, Image) >> or >> mask != None and not isinstance(mask, Image) >> or >> not isinstance(dest, Image) >> ) : >> raise TypeError("image args must be Image objects") >> #end if >> > > No need for the separate calls to isinstance, nor the check for None. > > if any(not isinstance(obj, Image) for obj in [src, mask, dest]): > ... Spot the bug in your version... From lawrencedo99 at gmail.com Sat Aug 20 18:22:53 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 20 Aug 2016 15:22:53 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <1d9e749d-aa20-4c7b-95df-17c62facfabb@googlegroups.com> On Friday, August 19, 2016 at 8:56:31 PM UTC+12, I wrote: > It is handy to be able to keep complex expressions together sometimes, when > breaking them up would simply obscure their structure. To avoid lines > getting long, why not take advantage of the two available screen/page > dimensions to make their structure clearer? As a bonus, spacing out > parentheses makes them look less of a clutter. Another example, from : the sequence of values is laid out to allow easy additions/modifications in future. for \ symname, funcname \ in \ ( ("FC_FONT", "ft_font_face_create_for_ft_face"), ("FT_FONT", "ft_font_face_create_for_pattern"), ("IMAGE_SURFACE", "image_surface_create"), # TODO: MIME_SURFACE, OBSERVER_SURFACE? ("PDF_SURFACE", "pdf_surface_create"), ("PNG_FUNCTIONS", "surface_write_to_png"), ("PS_SURFACE", "ps_surface_create"), ("RECORDING_SURFACE", "recording_surface_create"), ("SCRIPT_SURFACE", "script_create"), ("SVG_SURFACE", "svg_surface_create"), {"USER_FONT", "user_font_face_create"}, ) \ : setattr \ ( HAS, symname, hasattr(cairo, "cairo_" + funcname) ) #end for As a bonus, this also counts as an example of data-driven programming. From cs at zip.com.au Sat Aug 20 18:34:45 2016 From: cs at zip.com.au (cs at zip.com.au) Date: Sun, 21 Aug 2016 08:34:45 +1000 Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <20160820223445.GA68645@cskk.homeip.net> On 19Aug2016 16:11, Lawrence D?Oliveiro wrote: >On Saturday, August 20, 2016 at 10:38:34 AM UTC+12, Chris Angelico wrote: >> On Sat, Aug 20, 2016 at 8:31 AM, Lawrence D?Oliveiro wrote: >>> There is no short-cut evaluation when constructing tuples and lists. >> >> I'm not sure how that would make difference in these examples. The >> three parts are independent - the one place where short-circuiting is >> important is indeed short-circuited. > >That often is not the case, e.g. : > > assert \ > ( > len(self.points) == 0 > or > not self.points[0].off > and > (closed or not self.points[-1].off) > ) Aye, but beware that the expression is actually correct for the indentation. Compare: assert \ ( len(self.points) == 0 and not self.points[0].off or (closed or not self.points[-1].off) where the precedence causes the layout to mislead. I'm not arguing against indented spread out layout here, I use it myself, but just mentioning that the human eye will tend to take the visual layout over a strict parse. So care is needed, as in all programming. Cheers, Cameron Simpson From lawrencedo99 at gmail.com Sat Aug 20 19:43:07 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 20 Aug 2016 16:43:07 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <8737lzammr.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> Message-ID: <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> On Saturday, August 20, 2016 at 8:08:24 PM UTC+12, Marko Rauhamaa wrote: > > Lawrence D?Oliveiro: > >> On Thursday, August 18, 2016 at 4:47:28 PM UTC+12, Marko Rauhamaa wrote: >>> >>> ... as a rule, I dislike rules. Rule languages tend to >>> grow out of all bounds, always remain deficient and have impenetrable, >>> ad-hoc semantics. >> >> That?s a very peculiar thing to say, considering that data-driven >> programming is a well-known technique for writing compact code. >> >> Less code, and in particular, less repetitive code => fewer bugs. > > I can't relate to what you say. > > All I can say is that I've encountered numerous bad cases of rule > systems, eg: > > - iptables > > - selinux policies > > - systemd unit files > > - asterisk > > - sendmail I can?t relate to you, either. All those are spectacularly successful examples of highly-configurable pieces of software. From lawrencedo99 at gmail.com Sat Aug 20 19:44:50 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 20 Aug 2016 16:44:50 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <20160820223445.GA68645@cskk.homeip.net> Message-ID: On Sunday, August 21, 2016 at 10:35:08 AM UTC+12, c... at zip.com.au wrote: > Aye, but beware that the expression is actually correct for the > indentation. > Compare: > > assert \ > ( > len(self.points) == 0 > and > not self.points[0].off > or > (closed or not self.points[-1].off) > > where the precedence causes the layout to mislead. At least there is a discrepancy between the two to set alarm bells ringing. Compare statement indentation, where Python got rid of the statement brackets, so there is no more redundancy to be checked. Why do you think I put in those ?#end? lines? From michael.selik at gmail.com Sat Aug 20 20:43:52 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sun, 21 Aug 2016 00:43:52 +0000 Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro wrote: > > p0 = (0, 0) > > p1 = (major_dim, 0) > > colour_stops = (0, rect_1_colour), (1, complement(rect_1_colour)) > > rect_1_pattern = qah.Pattern.create_linear(p0, p1, colour_stops) > > That?s an example of what I mean about obscure structure. > To each his own. I was assuming all those variable names meant something to you. If not, then I expect it'd read well with good names. In this example, I think the abbreviations and numbers in the names could be changed to something more meaningful. > >> From , a > >> complex condition (with redundant parentheses again): > >> > >> if ( > >> not isinstance(src, Image) > >> or > >> mask != None and not isinstance(mask, Image) > >> or > >> not isinstance(dest, Image) > >> ) : > >> raise TypeError("image args must be Image objects") > >> #end if > >> > > > > No need for the separate calls to isinstance, nor the check for None. > > > > if any(not isinstance(obj, Image) for obj in [src, mask, dest]): > > ... > > Spot the bug in your version... > It'd be easier if I ran the code :-) Let's see... - the ``or`` translates to an ``any(...)`` - ``not isinstance(obj, Image)`` is repeated 3 times - ``[src, mask, dest]`` corresponds to the 3 objects - ``mask != None`` is unnecessary, unless somehow None has been registered as an instance of Image. ... can't spot it. Give me a hint? From michael.selik at gmail.com Sat Aug 20 20:52:05 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sun, 21 Aug 2016 00:52:05 +0000 Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: On Sat, Aug 20, 2016 at 8:43 PM Michael Selik wrote: > On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro < > lawrencedo99 at gmail.com> wrote: > >> > p0 = (0, 0) >> > p1 = (major_dim, 0) >> > colour_stops = (0, rect_1_colour), (1, complement(rect_1_colour)) >> > rect_1_pattern = qah.Pattern.create_linear(p0, p1, colour_stops) >> >> That?s an example of what I mean about obscure structure. >> > > To each his own. I was assuming all those variable names meant something > to you. If not, then I expect it'd read well with good names. In this > example, I think the abbreviations and numbers in the names could be > changed to something more meaningful. > I think that came out wrong. Let me try again... origin = 0, 0 destination = width, 0 start_color = 0, selected_color stop_color = 1, complement(selected_color) pattern = qah.Pattern.from_line(origin, destination, (start_color, stop_color)) Better? I wasn't sure what the numbers 0 and 1 are doing for the colour_stops. From steve+python at pearwood.info Sat Aug 20 21:13:19 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 21 Aug 2016 11:13:19 +1000 Subject: Two-Dimensional Expression Layout References: <1d9e749d-aa20-4c7b-95df-17c62facfabb@googlegroups.com> Message-ID: <57b9002f$0$1587$c3e8da3$5496439d@news.astraweb.com> On Sun, 21 Aug 2016 08:22 am, Lawrence D?Oliveiro wrote: > Another example, from : the sequence of > values is laid out to allow easy additions/modifications in future. When replying, I normally try to trim unnecessary code snippets down to the critical line or two, but in this case I think that it is important that I leave Lawrence's example in place in all its glory. See below for further comments. > for \ > symname, funcname \ > in \ > ( > ("FC_FONT", "ft_font_face_create_for_ft_face"), > ("FT_FONT", "ft_font_face_create_for_pattern"), > ("IMAGE_SURFACE", "image_surface_create"), > # TODO: MIME_SURFACE, OBSERVER_SURFACE? > ("PDF_SURFACE", "pdf_surface_create"), > ("PNG_FUNCTIONS", "surface_write_to_png"), > ("PS_SURFACE", "ps_surface_create"), > ("RECORDING_SURFACE", "recording_surface_create"), > ("SCRIPT_SURFACE", "script_create"), > ("SVG_SURFACE", "svg_surface_create"), > {"USER_FONT", "user_font_face_create"}, > ) \ > : > setattr \ > ( > HAS, > symname, > hasattr(cairo, "cairo_" + funcname) > ) > #end for > > As a bonus, this also counts as an example of data-driven programming. Is the subtle bug in your code also a bonus? If this bug has never bit you, it will when you try to run it under a version of Python with hash randomization turned on. (3.5 and above, I think.) You're (accidentally, I presume) relying on the set: {"USER_FONT", "user_font_face_create"} always iterating in the same order. It won't. I assume that's just a typo and it's meant to be a tuple. Your idiosyncratic layout obscures what should be a standard two-line for loop in a jumble of unneeded backslashes, almost empty lines and indentation. for symname, funcname in FONT_TABLE: setattr(HAS, symname, hasattr(cairo, "cairo_" + funcname)) where FONT_TABLE is defined previously. If you don't want to give this its own name, you can embed it in the for loop: for symname, funcname in [ ("FC_FONT", "ft_font_face_create_for_ft_face"), ("FT_FONT", "ft_font_face_create_for_pattern"), ("IMAGE_SURFACE", "image_surface_create"), # TODO: MIME_SURFACE, OBSERVER_SURFACE? ("PDF_SURFACE", "pdf_surface_create"), ("PNG_FUNCTIONS", "surface_write_to_png"), ("PS_SURFACE", "ps_surface_create"), ("RECORDING_SURFACE", "recording_surface_create"), ("SCRIPT_SURFACE", "script_create"), ("SVG_SURFACE", "svg_surface_create"), ("USER_FONT", "user_font_face_create"), ]: setattr(HAS, symname, hasattr(cairo, "cairo_" + funcname)) There's no need to arbitrarily indent parts of expressions to "allow easy additions" in the future: it is already easy to add new tuples to the table, or modify existing ones. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 20 21:15:26 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 21 Aug 2016 11:15:26 +1000 Subject: Two-Dimensional Expression Layout References: <20160820223445.GA68645@cskk.homeip.net> Message-ID: <57b900ae$0$1587$c3e8da3$5496439d@news.astraweb.com> On Sun, 21 Aug 2016 09:44 am, Lawrence D?Oliveiro wrote: > Why do you think I put in those ?#end? lines? Do you really want us to answer that? I don't think you will like the answer. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 20 21:25:58 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 21 Aug 2016 11:25:58 +1000 Subject: Two-Dimensional Expression Layout References: Message-ID: <57b90327$0$1622$c3e8da3$5496439d@news.astraweb.com> On Sun, 21 Aug 2016 10:43 am, Michael Selik wrote: > On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro > wrote: > >> > p0 = (0, 0) >> > p1 = (major_dim, 0) >> > colour_stops = (0, rect_1_colour), (1, complement(rect_1_colour)) >> > rect_1_pattern = qah.Pattern.create_linear(p0, p1, colour_stops) >> >> That?s an example of what I mean about obscure structure. >> > > To each his own. I was assuming all those variable names meant something > to you. If not, then I expect it'd read well with good names. In this > example, I think the abbreviations and numbers in the names could be > changed to something more meaningful. > > >> >> From , a >> >> complex condition (with redundant parentheses again): >> >> >> >> if ( >> >> not isinstance(src, Image) >> >> or >> >> mask != None and not isinstance(mask, Image) >> >> or >> >> not isinstance(dest, Image) >> >> ) : >> >> raise TypeError("image args must be Image objects") >> >> #end if >> >> >> > >> > No need for the separate calls to isinstance, nor the check for None. >> > >> > if any(not isinstance(obj, Image) for obj in [src, mask, dest]): >> > ... >> >> Spot the bug in your version... >> > > It'd be easier if I ran the code :-) > Let's see... > - the ``or`` translates to an ``any(...)`` > - ``not isinstance(obj, Image)`` is repeated 3 times > - ``[src, mask, dest]`` corresponds to the 3 objects > - ``mask != None`` is unnecessary, unless somehow None has been registered > as an instance of Image. > > ... can't spot it. Give me a hint? Earlier, Lawrence wrote about this same piece of code that using any() or all() was not a good idea because, and I quote: "There is no short-cut evaluation when constructing tuples and lists." I think that he is envisaging a scenario where (say) src and mask are defined, but dest is not, so [src, mask, dest] will raise a NameError, but his earlier version of the code: if ( not isinstance(src, Image) or mask != None and not isinstance(mask, Image) or not isinstance(dest, Image) ) : will not, *provided* one of the tests on src or mask fail first. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Aug 20 21:45:26 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 21 Aug 2016 11:45:26 +1000 Subject: Two-Dimensional Expression Layout References: Message-ID: <57b907b7$0$1586$c3e8da3$5496439d@news.astraweb.com> Oh wait! The penny drops! On Sun, 21 Aug 2016 10:43 am, Michael Selik wrote: > On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro >> >> if ( >> >> not isinstance(src, Image) >> >> or >> >> mask != None and not isinstance(mask, Image) >> >> or >> >> not isinstance(dest, Image) >> >> ) : >> >> raise TypeError("image args must be Image objects") >> >> #end if >> >> >> > >> > No need for the separate calls to isinstance, nor the check for None. >> > >> > if any(not isinstance(obj, Image) for obj in [src, mask, dest]): >> > ... >> >> Spot the bug in your version... >> > > It'd be easier if I ran the code :-) > Let's see... > - the ``or`` translates to an ``any(...)`` > - ``not isinstance(obj, Image)`` is repeated 3 times > - ``[src, mask, dest]`` corresponds to the 3 objects > - ``mask != None`` is unnecessary, unless somehow None has been registered > as an instance of Image. > > ... can't spot it. Give me a hint? I think you've got the mask != None bit backwards. In Lawrence's version, if mask is None, *no exception is raised*. In yours, it raises. I think that it would be easier to reason about this code if it were written in terms of positive assertions ("this condition is true") rather than negative ("this condition is not true"). - src must be an Image; - mask must be None or an Image; - dest must be an Image. Assuming I check for this in at least two places, I'd factor it out into a helper function. There are probably a thousand ways to write it, and I'm not married to any of them. Here are a couple: def _validate(src, mask, dest): if mask is None or isinstance(mask, Image) and \ all(isinstance(o) for o in [src, dest]): return raise TypeError("image args must be Image objects") def _validate(src, mask, dest): if not ( isinstance(src, Image) and mask is None or isinstance(mask, Image) and isinstance(dest, Image) ): raise TypeError("image args must be Image objects") The logic is *just* tricky enough that if this were my code I would absolutely insist on having unit tests for this helper. Beyond that, the specific details of how you test them is not that important, but given that there are only three arguments to be tested, I'm inclined to test each one individually and give a more useful error message: def _validate(src, mask, dest): if not isinstance(src, Image): raise TypeError("src must be an Image") if mask is not None and not isinstance(mask, Image): raise TypeError("mask must be None or an Image") if not isinstance(dest, Image): raise TypeError("dest must be an Image") -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From johncalvinhall at gmail.com Sat Aug 20 23:55:23 2016 From: johncalvinhall at gmail.com (johncalvinhall at gmail.com) Date: Sat, 20 Aug 2016 21:55:23 -0600 Subject: Good Tutorial for Python3 winreg Module Message-ID: Does anyone have a good tutorial on how to use the Python 3 module winreg? I need to pull a saved value from the registry, and the only way I know how to get it is through winreg. If anyone is able to help, I sure would appreciate it. From lawrencedo99 at gmail.com Sun Aug 21 00:26:08 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sat, 20 Aug 2016 21:26:08 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: Message-ID: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> On Sunday, August 21, 2016 at 12:44:21 PM UTC+12, Michael Selik wrote: > > On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro wrote: > >>> if any(not isinstance(obj, Image) for obj in [src, mask, dest]): >>> ... >> >> Spot the bug in your version... > > - ``mask != None`` is unnecessary, unless somehow None has been registered > as an instance of Image. That?s the bug: it?s not unnecessary. Maybe a quick application of one of De Morgan?s theorems might help: if ( isinstance(src, Image) and (mask == None or isinstance(mask, Image)) and isinstance(dest, Image) ) : don?t raise TypeError("image args must be Image objects") #end if Is that better for you? From eryksun at gmail.com Sun Aug 21 01:39:13 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 21 Aug 2016 05:39:13 +0000 Subject: Good Tutorial for Python3 winreg Module In-Reply-To: References: Message-ID: On Sun, Aug 21, 2016 at 3:55 AM, wrote: > > I need to pull a saved value from the registry, and the only way I > know how to get it is through winreg. Here's an example that reads the value and type of "Temp" in HKCU\Environment: import winreg with winreg.OpenKey(winreg.HKEY_CURRENT_USER, 'Environment') as key: value, dtype = winreg.QueryValueEx(key, 'Temp') >>> print(value) %USERPROFILE%\AppData\Local\Temp >>> dtype == winreg.REG_EXPAND_SZ True I can help more if you provide a specific example of the value that you need to read. From wentao.liang at uq.net.au Sun Aug 21 02:39:23 2016 From: wentao.liang at uq.net.au (Wentao Liang) Date: Sun, 21 Aug 2016 06:39:23 +0000 Subject: Something wrong with the PIP, lots of friend have the same problem In-Reply-To: References: Message-ID: even though I try to upgrade PIP[cid:beb9e7c5-7d98-4506-ba67-9e67890edec1] ________________________________ From: Wentao Liang Sent: Sunday, 21 August 2016 4:35:31 PM To: python-list at python.org Subject: Something wrong with the PIP, lots of friend have the same problem [cid:1c20248f-a678-40c0-9563-da4f8a1ef3bd] Is it the problem from PIP itself, or I used it in a wrong way? [?] Looking forwards to your reply! Thank you Wentao Liang From michael.selik at gmail.com Sun Aug 21 02:48:50 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sun, 21 Aug 2016 06:48:50 +0000 Subject: Two-Dimensional Expression Layout In-Reply-To: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> References: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> Message-ID: On Sun, Aug 21, 2016 at 12:31 AM Lawrence D?Oliveiro wrote: > On Sunday, August 21, 2016 at 12:44:21 PM UTC+12, Michael Selik wrote: > > > > On Sat, Aug 20, 2016 at 6:21 PM Lawrence D?Oliveiro wrote: > > > >>> if any(not isinstance(obj, Image) for obj in [src, mask, dest]): > >>> ... > >> > >> Spot the bug in your version... > > > > - ``mask != None`` is unnecessary, unless somehow None has been > registered > > as an instance of Image. > > That?s the bug: it?s not unnecessary. > > Maybe a quick application of one of De Morgan?s theorems might help: > > if ( > isinstance(src, Image) > and > (mask == None or isinstance(mask, Image)) > and > isinstance(dest, Image) > ) : > don?t raise TypeError("image args must be Image objects") > #end if > > Is that better for you? > Indeed it is, not sure why. I think Steven's right in this case. Since the condition isn't identical for all objects, and it's only three variables, I'd rather break it apart. if not isinstance(src, Image): raise TypeError('src must be an Image') if mask and not isinstance(mask, Image): raise TypeError('mask must be an Image or None') if not isinstance(dest, Image): raise TypeError('dest must be an Image') As he said as well, this has the added benefit of encouraging more explicit error messages. In programming and in prose, sometimes repetition feels ugly and redundant, but sometimes repetition feels like a beautiful harmony. From lawrencedo99 at gmail.com Sun Aug 21 03:00:39 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 00:00:39 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> Message-ID: <8df01308-4c52-4b89-8ff1-7342eee18c30@googlegroups.com> On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote: > Indeed it is, not sure why. Moral: It helps to understand the code you?re criticizing, before you start criticizing, not after. From michael.selik at gmail.com Sun Aug 21 03:19:17 2016 From: michael.selik at gmail.com (Michael Selik) Date: Sun, 21 Aug 2016 07:19:17 +0000 Subject: Two-Dimensional Expression Layout In-Reply-To: <8df01308-4c52-4b89-8ff1-7342eee18c30@googlegroups.com> References: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> <8df01308-4c52-4b89-8ff1-7342eee18c30@googlegroups.com> Message-ID: On Sun, Aug 21, 2016, 3:06 AM Lawrence D?Oliveiro wrote: > On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote: > > > Indeed it is, not sure why. > > Moral: It helps to understand the code you?re criticizing, before you > start criticizing, not after. > A true statement, but not my takeaway from this discussion. > From rustompmody at gmail.com Sun Aug 21 07:53:07 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Aug 2016 04:53:07 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> Message-ID: <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> On Sunday, August 21, 2016 at 5:13:28 AM UTC+5:30, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 8:08:24 PM UTC+12, Marko Rauhamaa wrote: > > > > Lawrence D?Oliveiro: > > > >> On Thursday, August 18, 2016 at 4:47:28 PM UTC+12, Marko Rauhamaa wrote: > >>> > >>> ... as a rule, I dislike rules. Rule languages tend to > >>> grow out of all bounds, always remain deficient and have impenetrable, > >>> ad-hoc semantics. > >> > >> That?s a very peculiar thing to say, considering that data-driven > >> programming is a well-known technique for writing compact code. > >> > >> Less code, and in particular, less repetitive code => fewer bugs. > > > > I can't relate to what you say. > > > > All I can say is that I've encountered numerous bad cases of rule > > systems, eg: > > > > - iptables > > > > - selinux policies > > > > - systemd unit files > > > > - asterisk > > > > - sendmail > > I can?t relate to you, either. All those are spectacularly successful examples of highly-configurable pieces of software. Very interesting dialogue? Theoretically Ive generally been aligned along the lines of Lawrence: ?Declarative is better than imperative? [ Ive even espoused it as ?data-orientation? in http://blog.languager.org/2012/10/functional-programming-lost-booty.html , a page that gets a lot of hits of late] But seeing Marko?s list I am very uneasy (with my own default preferences) Ive always taken it as an individual specific incidental case when one of these instances does not fare well. However seeing that list together like that I cant help wondering: Is the philosophy of data-driven-ness itself somehow ill-conceived? Some other examples: 1. Aren?t makefiles data-driven? And if so should not much more declarative attempts like jam have been correspondingly more successful? https://www.perforce.com/resources/documentation/jam [Personally Ive always believed that jam is better than make and is less used for entirely historical reasons; something like half the world eoling with crlf and half with lf. But maybe make is really a better design because more imperative?] 2. Basic computing theory shows that re-s and dfas are equivalent. Which would one prefer to write/debug? [Thats not a rhetorical question] 3. Scheme ? the language but also the informal culture around SICP ? is where data-driven programming got installed as a sacred dictum. Scheme doesn't seem to be doing too well of late? Coincidence?? From rosuav at gmail.com Sun Aug 21 08:21:15 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Aug 2016 22:21:15 +1000 Subject: The Joys Of Data-Driven Programming In-Reply-To: <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: On Sun, Aug 21, 2016 at 9:53 PM, Rustom Mody wrote: > Some other examples: > 1. > Aren?t makefiles data-driven? And if so should not much more declarative > attempts like jam have been correspondingly more successful? > https://www.perforce.com/resources/documentation/jam > > [Personally Ive always believed that jam is better than make and is less used > for entirely historical reasons; something like half the world eoling with crlf > and half with lf. But maybe make is really a better design because more imperative?] The issues with makefiles are mainly to do with metaprogramming (plus a few specific issues with the format itself, which don't apply to the more general concept). There's a ton of magic to cope with makefiles that try to rebuild themselves, plus all the extra tooling around the outside (configure scripts etc), to try to move to a meta-level above the makefile and thus enable metaprogramming of makefile contents. ChrisA From rustompmody at gmail.com Sun Aug 21 08:39:15 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Aug 2016 05:39:15 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: <080a2a04-e4cc-4866-accf-b750db4790a1@googlegroups.com> On Sunday, August 21, 2016 at 5:51:36 PM UTC+5:30, Chris Angelico wrote: > On Sun, Aug 21, 2016 at 9:53 PM, Rustom Mody wrote: > > Some other examples: > > 1. > > Aren?t makefiles data-driven? And if so should not much more declarative > > attempts like jam have been correspondingly more successful? > > https://www.perforce.com/resources/documentation/jam > > > > [Personally Ive always believed that jam is better than make and is less used > > for entirely historical reasons; something like half the world eoling with crlf > > and half with lf. But maybe make is really a better design because more imperative?] > > The issues with makefiles are mainly to do with metaprogramming (plus > a few specific issues with the format itself, which don't apply to the > more general concept). There's a ton of magic to cope with makefiles > that try to rebuild themselves, plus all the extra tooling around the > outside (configure scripts etc), to try to move to a meta-level above > the makefile and thus enable metaprogramming of makefile contents. Yeah? I guess thats the issue said differently: viz. For a human to write/think in terms of rules is cleaner (more ?hi-level?) than writing out imperative actions. However when it comes to generating ? what you call metaprogramming ? rule-based systems suddenly stop being attractive, especially when they are the target of the generator. Standard example: Programmers dont like to code in assembly Compiler writers dont like to generate to anything else From marko at pacujo.net Sun Aug 21 10:20:24 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 21 Aug 2016 17:20:24 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: <87fupy8aqf.fsf@elektro.pacujo.net> Rustom Mody : >> > All I can say is that I've encountered numerous bad cases of rule >> > systems, eg: >> > >> > - iptables >> > >> > - selinux policies >> > >> > - systemd unit files >> > >> > - asterisk >> > >> > - sendmail > [...] > Some other examples: > 1. > Aren?t makefiles data-driven? Yes, "make" should be added to my sin list. > [Personally Ive always believed that jam is better than make and is > less used for entirely historical reasons; something like half the > world eoling with crlf and half with lf. But maybe make is really a > better design because more imperative?] Don't know jam, but can heartily recommend SCons. > 3. Scheme ? the language but also the informal culture around SICP ? > is where data-driven programming got installed as a sacred dictum. > Scheme doesn't seem to be doing too well of late? Coincidence?? Lisp's "data is code and code is data" is a good principle. I have successfully used it with Python as well. However, Scheme's syntax-rules are infamous. Marko From rustompmody at gmail.com Sun Aug 21 12:08:34 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Aug 2016 09:08:34 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87fupy8aqf.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: <039a1da6-bebe-4274-a992-df2e061801d2@googlegroups.com> On Sunday, August 21, 2016 at 7:50:39 PM UTC+5:30, Marko Rauhamaa wrote: > Rustom Mody : > >> > All I can say is that I've encountered numerous bad cases of rule > >> > systems, eg: > >> > > >> > - iptables > >> > > >> > - selinux policies > >> > > >> > - systemd unit files > >> > > >> > - asterisk > >> > > >> > - sendmail > > [...] > > Some other examples: > > 1. > > Aren?t makefiles data-driven? > > Yes, "make" should be added to my sin list. If you treat make as elaboration on this theme: target : source TAB Imperative Action it belongs to your 'sin' pattern if you concentrate on line 1 Not if you concentrate on line 2 So maybe it finds the sweet spot between the two?? > > > [Personally Ive always believed that jam is better than make and is > > less used for entirely historical reasons; something like half the > > world eoling with crlf and half with lf. But maybe make is really a > > better design because more imperative?] > > Don't know jam, but can heartily recommend SCons. Likely true at some point Probably not now: http://stackoverflow.com/questions/2847730/c-build-systems [ tup seems to be the rage; scons seems to have the worst performance] > > > 3. Scheme ? the language but also the informal culture around SICP ? > > is where data-driven programming got installed as a sacred dictum. > > Scheme doesn't seem to be doing too well of late? Coincidence?? > > Lisp's "data is code and code is data" is a good principle. I have > successfully used it with Python as well. Heh Really?!? I can tell you if a student of mine mixed up code and data like emacs-lisp throwing customize *data* into user-config *code* (.emacs) he would be very sorry... > > However, Scheme's syntax-rules are infamous. No idea what you are talking about Macros? Define-syntax's syntax? I-dont-like-all-those Lots-of Irritating Single Parentheses ? Something else? If you ask me Lisp failed because of out of kilter uncontrollable superiority-complex From python.list at tim.thechases.com Sun Aug 21 14:18:22 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 21 Aug 2016 13:18:22 -0500 Subject: The Joys Of Data-Driven Programming In-Reply-To: <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: <20160821131822.338b5222@bigbox.christie.dr> On 2016-08-21 04:53, Rustom Mody wrote: > 2. Basic computing theory shows that re-s and dfas are equivalent. > Which would one prefer to write/debug? [Thats not a rhetorical > question] I'm curious where REs and DFAs are shown to be equivalent (serious, not trying to be snarky). I can see with no trouble that all REs are DFAs, but can't convince myself that *all* DFAs are REs. Though this might also depend on the RE engine. Do you have links to resources on such an "all REs have equivalent DFAs and all DFAs have equivalent REs" logic/argument? Or maybe my understanding of DFAs is flawed. And as for which I prefer to write, for simple text-processing that doesn't involve nesting, I generally prefer REs; but for more complex processing, a custom DFA tends to expand a bit more cleanly in my experience. -tkc From rocky at gnu.org Sun Aug 21 14:28:25 2016 From: rocky at gnu.org (rocky) Date: Sun, 21 Aug 2016 11:28:25 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version Message-ID: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> The problem: 1. there are various code inspection tools that parse Python programs looking for style issues or whatnot. The deeper ones have to do a full parse of the python program. It would be helpful if there were a uniform way to indicate the Python language level used in Python source code. 2. I get a standalone python program that is not part of a package. vcprompt https://bitbucket.org/gward/vcprompt might be an example It would be helpful if there were an easy way to know what language version of Python it assumes Perl has something like called "use perl". "use" is roughly equivalent to "import". Possible solutions: Do it the similar to "use perl". Here "perl" is a package that just tests the parameter given it. In Python such the code would look something like File/module python30.py import sys assert sys.version >= (sys.version_info >= (3, 0)) The above works, but to reduce proliferation of packages it might be preferable to come up with some way to pass a version specification string similar to the specification strings allowed in setup.py A metadata tag as a comment in a docstring or in a comment. Preferably this would be given towards the top of the file to make it easier for tools to extract this information. Thoughts? From marko at pacujo.net Sun Aug 21 15:01:31 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 21 Aug 2016 22:01:31 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <039a1da6-bebe-4274-a992-df2e061801d2@googlegroups.com> Message-ID: <874m6e7xpw.fsf@elektro.pacujo.net> Rustom Mody : > On Sunday, August 21, 2016 at 7:50:39 PM UTC+5:30, Marko Rauhamaa wrote: >> Don't know jam, but can heartily recommend SCons. > > Likely true at some point > Probably not now: http://stackoverflow.com/questions/2847730/c-build-systems > [ tup seems to be the rage; scons seems to have the worst performance] If SCons performance becomes an issue for you, your source code repository is far too large. Time to break it up into independent components. Maybe there are other builders as good as SCons, but the nice thing about SCons is that the "rule language" is ordinary Python. That's key: don't invent a new rule language; use a real programming language. >> Lisp's "data is code and code is data" is a good principle. I have >> successfully used it with Python as well. > > Heh Really?!? Yes. I was the treasurer of some soccer teams. I recorded the financial transactions by updating a simple Python program that looked like this: ======================================================================== : : : paid('2014-11-17', 'DOE JANE', '1274', 'November', None, { FEE_INCOME : 150.00 }) owed('2014-11-30', 'November', { 150 : [ SOPHIA, EMMA, OLIVIA, AVA, ISABELLA, MIA, ZOE, LILY ] }) : : : ======================================================================== The program produced web pages, spreadsheets, emails etc. > I can tell you if a student of mine mixed up code and data like > emacs-lisp throwing customize *data* into user-config *code* (.emacs) > he would be very sorry... Er, ok... Marko From vincent.vande.vyvre at telenet.be Sun Aug 21 15:50:12 2016 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Sun, 21 Aug 2016 21:50:12 +0200 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> Message-ID: <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> Le 21/08/2016 ? 20:28, rocky a ?crit : > The problem: > > 1. there are various code inspection tools that parse Python programs looking for style issues or whatnot. The deeper ones have to do a full parse of the python program. It would be helpful if there were a uniform way to indicate the Python language level used in Python source code. > > 2. I get a standalone python program that is not part of a package. vcprompt https://bitbucket.org/gward/vcprompt might be an example > It would be helpful if there were an easy way to know what language version of Python it assumes > > Perl has something like called "use perl". "use" is roughly equivalent to "import". > > Possible solutions: > > Do it the similar to "use perl". Here "perl" is a package that just tests the parameter given it. In Python such the code would look something like > > File/module python30.py > > import sys > assert sys.version >= (sys.version_info >= (3, 0)) > > The above works, but to reduce proliferation of packages it might be preferable to come up with some way to pass a version specification string similar to the specification strings allowed in setup.py > > A metadata tag as a comment in a docstring or in a comment. > Preferably this would be given towards the top of the file to make it easier for tools to extract this information. > > Thoughts? I don't understand your assertion Python 3.2.3 (default, Jun 18 2015, 21:46:42) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> assert sys.version >= (sys.version_info >= (3, 0)) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: str() >= bool() What you means ? -- Vincent V.V. Oqapy . python3-exiv2 . Qarte . PaQager From torriem at gmail.com Sun Aug 21 16:03:18 2016 From: torriem at gmail.com (Michael Torrie) Date: Sun, 21 Aug 2016 14:03:18 -0600 Subject: Does This Scare You? In-Reply-To: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> On 08/19/2016 05:42 PM, Lawrence D?Oliveiro wrote: > Python 3.5.2+ (default, Aug 5 2016, 08:07:14) > [GCC 6.1.1 20160724] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from pathlib import PureWindowsPath > >>> PureWindowsPath("prn").is_reserved() > True > >>> PureWindowsPath("prn.doc").is_reserved() > True > >>> PureWindowsPath("com9.exe").is_reserved() > True > >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved() > True Which part are you getting at? That Windows treats certain filenames as reserved (a known gotcha that has existed for decades) or that Python allows you to test whether a path is valid in Windows? From ben.usenet at bsb.me.uk Sun Aug 21 17:02:32 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 21 Aug 2016 22:02:32 +0100 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <20160821131822.338b5222@bigbox.christie.dr> Message-ID: <87eg5hvnrr.fsf@bsb.me.uk> Tim Chase writes: > On 2016-08-21 04:53, Rustom Mody wrote: >> 2. Basic computing theory shows that re-s and dfas are equivalent. >> Which would one prefer to write/debug? [Thats not a rhetorical >> question] > > I'm curious where REs and DFAs are shown to be equivalent (serious, > not trying to be snarky). I can see with no trouble that all REs are > DFAs, but can't convince myself that *all* DFAs are REs. All NFAs as well -- the don't have to be deterministic in either direction. > Though this > might also depend on the RE engine. Yes. Due to their use in real systems, lots of RE engines have acquired many bells and whistles. Some bells so fancy that the resulting "REs" can recognise whole new classes of language. In this context, though, "regular expression" means the expression form of regular grammars (level 3 in the Chomsky hierarchy). > Do you have links to resources > on such an "all REs have equivalent DFAs and all DFAs have equivalent > REs" logic/argument? Or maybe my understanding of DFAs is flawed. This is called Kleene's Theorem. Searching for that will bring up all sorts of proofs in various styles. Here's one of the direction you are asking about: http://www.cs.may.ie/staff/jpower/Courses/Previous/parsing/node6.html The other direction is covered all over the place because it it so often used inside an RE matching engine to "compile" the RE. > And as for which I prefer to write, for simple text-processing that > doesn't involve nesting, I generally prefer REs; but for more complex > processing, a custom DFA tends to expand a bit more cleanly in my > experience. You may be confusing DFAs and pushdown automata. I say this because of your reference to nesting. PDAs can be thought of as either DFAs or NFAs with a stack so they can recognise languages with nested structure. -- Ben. From jon+usenet at unequivocal.eu Sun Aug 21 17:24:48 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sun, 21 Aug 2016 21:24:48 -0000 (UTC) Subject: saving octet-stream png file References: Message-ID: On 2016-08-19, Larry Martell wrote: > fd.write(request.POST[key]) You could try: request.encoding = "iso-8859-1" fd.write(request.POST[key].encode("iso-8859-1")) It's hacky and nasty and there might be a better "official" method but I think it should work. From lawrencedo99 at gmail.com Sun Aug 21 18:02:48 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 15:02:48 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: On Sunday, August 21, 2016 at 11:53:52 PM UTC+12, Rustom Mody wrote: > However seeing that list together like that I cant help wondering: > Is the philosophy of data-driven-ness itself somehow ill-conceived? I don?t understand why. They are all examples of the success of the technique, and I can mention lots more. > 2. Basic computing theory shows that re-s and dfas are equivalent. And non-deterministic FSAs, too. > Which would one prefer to write/debug? That depends, not on their capabilities, but on how they are formulated. From lawrencedo99 at gmail.com Sun Aug 21 18:08:34 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 15:08:34 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> Message-ID: On Monday, August 22, 2016 at 12:21:36 AM UTC+12, Chris Angelico wrote: > The issues with makefiles are mainly to do with metaprogramming (plus > a few specific issues with the format itself, which don't apply to the > more general concept). There's a ton of magic to cope with makefiles > that try to rebuild themselves, plus all the extra tooling around the > outside (configure scripts etc), to try to move to a meta-level above > the makefile and thus enable metaprogramming of makefile contents. In other words, the limitations of the original Make concept itself. So you fix this by a) As per Lampson, adding another level of indirection (e.g. GNU Autotools, CMake) b) Trying to replace Make altogether (e.g. SCons, Jam, Ant/Maven/Gradle) Which approach is enjoying more success? I seem to be coming across CMake a lot. It is popular because it can wrap, not just around Make, but other build systems as well, including platform-specific IDE-based ones like Microsoft Visual Studio and Apple XCode. In other words, it is a cross-platform build system. From lawrencedo99 at gmail.com Sun Aug 21 18:09:13 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 15:09:13 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87fupy8aqf.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: > Lisp's "data is code and code is data" is a good principle. I have > successfully used it with Python as well. You think self-modifying code is a good idea? From lawrencedo99 at gmail.com Sun Aug 21 18:10:12 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 15:10:12 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87fupy8aqf.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: > ... can heartily recommend SCons. It?s Python 2 only, not Python 3. From marko at pacujo.net Sun Aug 21 18:10:20 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Aug 2016 01:10:20 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: <87zio57oz7.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: >> Lisp's "data is code and code is data" is a good principle. I have >> successfully used it with Python as well. > > You think self-modifying code is a good idea? Why change topics? Marko From lawrencedo99 at gmail.com Sun Aug 21 18:18:26 2016 From: lawrencedo99 at gmail.com (=?UTF-8?Q?Lawrence_D=E2=80=99Oliveiro?=) Date: Sun, 21 Aug 2016 15:18:26 -0700 (PDT) Subject: Two-Dimensional Expression Layout In-Reply-To: References: <2b3078ed-4e08-40cb-9a8d-660b0911b6d0@googlegroups.com> <8df01308-4c52-4b89-8ff1-7342eee18c30@googlegroups.com> Message-ID: <30de4e69-e81c-4757-b6b5-d5beb95ab15a@googlegroups.com> On Sunday, August 21, 2016 at 7:19:49 PM UTC+12, Michael Selik wrote: > > On Sun, Aug 21, 2016, 3:06 AM Lawrence D?Oliveiro wrote: > >> On Sunday, August 21, 2016 at 6:49:19 PM UTC+12, Michael Selik wrote: >> >>> Indeed it is, not sure why. >> >> Moral: It helps to understand the code you?re criticizing, before you >> start criticizing, not after. > > A true statement, but not my takeaway from this discussion. Really? Even after it has been pointed out to you? I wonder what the name is for people like that... From marko at pacujo.net Sun Aug 21 18:18:39 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Aug 2016 01:18:39 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: <87vayt7olc.fsf@elektro.pacujo.net> Lawrence D?Oliveiro : > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: >> ... can heartily recommend SCons. > > It?s Python 2 only, not Python 3. And? SCons is very good, definitely beats make. It also illustrates the use of a real programming language for special applications (as opposed to ad hoc rules). The flawed thinking behind rules is often expressed like this: We want rules because not everybody is a programmer. However, with rules, not even a programmer knows how to configure the thing. Marko From breamoreboy at gmail.com Sun Aug 21 19:02:26 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 21 Aug 2016 16:02:26 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87vayt7olc.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <87vayt7olc.fsf@elektro.pacujo.net> Message-ID: On Sunday, August 21, 2016 at 11:18:49 PM UTC+1, Marko Rauhamaa wrote: > Lawrence D?Oliveiro : > > > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: > >> ... can heartily recommend SCons. > > > > It?s Python 2 only, not Python 3. > > And? SCons is very good, definitely beats make. It also illustrates the > use of a real programming language for special applications (as opposed > to ad hoc rules). > > The flawed thinking behind rules is often expressed like this: We want > rules because not everybody is a programmer. > > However, with rules, not even a programmer knows how to configure the > thing. > > > Marko Some people clearly disagree http://catherinedevlin.blogspot.co.uk/2009/06/dont-need-no-stinking-rules-engine.html Kindest regards. Mark Lawrence. From eryksun at gmail.com Sun Aug 21 20:38:15 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 00:38:15 +0000 Subject: Does This Scare You? In-Reply-To: <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On Sun, Aug 21, 2016 at 8:03 PM, Michael Torrie wrote: > On 08/19/2016 05:42 PM, Lawrence D?Oliveiro wrote: >> Python 3.5.2+ (default, Aug 5 2016, 08:07:14) >> [GCC 6.1.1 20160724] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> from pathlib import PureWindowsPath >> >>> PureWindowsPath("prn").is_reserved() >> True >> >>> PureWindowsPath("prn.doc").is_reserved() >> True >> >>> PureWindowsPath("com9.exe").is_reserved() >> True >> >>> PureWindowsPath("c:/my documents/prn.doc").is_reserved() >> True > > Which part are you getting at? That Windows treats certain filenames as > reserved (a known gotcha that has existed for decades) or that Python > allows you to test whether a path is valid in Windows? To me it's scary that this check misses cases because it's trying to be cross-platform instead of simply relying on GetFullPathName to do the work. For example, it misses at least the following cases: Optional trailing colon: >>> pathlib.Path('C:/foo/NUL:').is_reserved() False >>> print(os.path._getfullpathname('C:/foo/NUL:')) \\.\NUL Trailing spaces: >>> pathlib.Path('C:/foo/NUL ').is_reserved() False >>> print(os.path._getfullpathname('C:/foo/NUL ')) \\.\NUL Trailing spaces followed by a file extension: >>> pathlib.Path('C:/foo/NUL .txt').is_reserved() False >>> print(os.path._getfullpathname('C:/foo/NUL .txt')) \\.\NUL It's also a bit disappointing that the author of this function claims in a comment that "foo/NUL" isn't reserved yet decides to "err on the side of caution" (obviously not enough). Of course "foo/NUL" is reserved: >>> print(os.path._getfullpathname('foo/NUL')) \\.\NUL I think what happened is that the author tested by calling open() on a non-existing path. DOS device names are only reserved for existing directories, in order to return an error for an invalid path. The difference is how RtlGetFullPathName_Ustr is called. When GetFullPathName calls the latter function it doesn't care whether or not the path is valid. On the other hand, RtlDosPathNameToNtPathName_* calls RtlGetFullPathName_Ustr with a parameter to check for an invalid path, which makes the path normalization fail. For example: Existing directory: >>> os.path.exists('C:/Temp') True >>> f = open('C:/Temp/NUL') Query the device name: >>> hFile = msvcrt.get_osfhandle(f.fileno()) >>> ntdll.NtQueryObject(hFile, 1, byref(name), sizeof(name), None) 0 >>> print(name.Buffer[:name.Length//2]) \Device\Null (\\.\NUL, i.e. \GLOBAL??\NUL, is a symbolic link to NT's \Device\Null.) Non-existing directory: >>> os.path.exists('C:/Spam') False >>> g = open('C:/Spam/NUL') Traceback (most recent call last): File "", line 1, in FileNotFoundError: [Errno 2] No such file or directory: 'C:/Spam/NUL' From rosuav at gmail.com Sun Aug 21 20:44:38 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 10:44:38 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On Mon, Aug 22, 2016 at 10:38 AM, eryk sun wrote: > To me it's scary that this check misses cases because it's trying to > be cross-platform instead of simply relying on GetFullPathName to do > the work. "Trying to be" cross-platform? The point of these path modules is to *be* cross-platform. I can't call Windows APIs on my Linux box (short of messing around with VMs or Wine or something, which are dependencies that Python doesn't need). So yes, these *are* cross-platform, and that's not scary, that's deliberate design. ChrisA From eryksun at gmail.com Sun Aug 21 21:08:12 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 01:08:12 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico wrote: > "Trying to be" cross-platform? The point of these path modules is to > *be* cross-platform. I can't call Windows APIs on my Linux box (short > of messing around with VMs or Wine or something, which are > dependencies that Python doesn't need). So yes, these *are* > cross-platform, and that's not scary, that's deliberate design. Sorry for the wording SNAFU there. Of course it's cross-platform. But I think it's a dubious check. On Windows itself I would never try what it's attempting to do. I would just call GetFullPathName to see if the path changes to use the \\.\ device namespace. I doubt that the 3 examples I gave are the only places that this simplistic implementation of 'is_reserved" fails. It's giving a dangerous impression to the user that the path will be safe to use on Windows when that's not the case. From steve+python at pearwood.info Sun Aug 21 21:14:26 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 11:14:26 +1000 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: > To me it's scary that this check misses cases because it's trying to > be cross-platform instead of simply relying on GetFullPathName to do > the work. For example, it misses at least the following cases: Instead of shaking in your boots over a simple bug in a non-critical library, how about reporting these cases on the bug tracker with an explanation of the problem? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Aug 21 21:26:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 11:26:13 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On Mon, Aug 22, 2016 at 11:08 AM, eryk sun wrote: > On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico wrote: >> "Trying to be" cross-platform? The point of these path modules is to >> *be* cross-platform. I can't call Windows APIs on my Linux box (short >> of messing around with VMs or Wine or something, which are >> dependencies that Python doesn't need). So yes, these *are* >> cross-platform, and that's not scary, that's deliberate design. > > Sorry for the wording SNAFU there. Of course it's cross-platform. But > I think it's a dubious check. On Windows itself I would never try what > it's attempting to do. I would just call GetFullPathName to see if the > path changes to use the \\.\ device namespace. I doubt that the 3 > examples I gave are the only places that this simplistic > implementation of 'is_reserved" fails. It's giving a dangerous > impression to the user that the path will be safe to use on Windows > when that's not the case. Understood, and apology accepted. But I think it should be understood that this will *not* call any Windows APIs. It's called "PureWindowsPath" and is an example of a pure path: https://docs.python.org/3/library/pathlib.html#pure-paths They're specifically documented as not touching any file system, which means that they are cross-platform and cannot be guaranteed to be perfect. If you know you're running on Windows, use WindowsPath instead (trying to do so on a non-Windows system will cause an immediate exception), and then you can test your examples on that. It may well already probe the file system in that situation, and if not, you could put forward the feature suggestion that it should. But the PurePath classes won't. ChrisA From rustompmody at gmail.com Sun Aug 21 22:04:33 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Aug 2016 19:04:33 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87vayt7olc.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <87vayt7olc.fsf@elektro.pacujo.net> Message-ID: On Monday, August 22, 2016 at 3:48:49 AM UTC+5:30, Marko Rauhamaa wrote: > Lawrence D?Oliveiro : > > > On Monday, August 22, 2016 at 2:20:39 AM UTC+12, Marko Rauhamaa wrote: > >> ... can heartily recommend SCons. > > > > It?s Python 2 only, not Python 3. > > And? SCons is very good, definitely beats make. It also illustrates the > use of a real programming language for special applications (as opposed > to ad hoc rules). > > The flawed thinking behind rules is often expressed like this: We want > rules because not everybody is a programmer. > > However, with rules, not even a programmer knows how to configure the > thing. The same thing said from flip side: Programmers think programming (or is it programmering?) is kewl Maybe? Maybe not? All of us, even supposedly educated/intelligent, can suffer from the hammer-nail effect with our hammer being general purpose programming (languages) It occurs to me that a general-purpose rule system was created decades ago ? prolog That it did not take off maybe just a question of maturity and not something intrinsically wrong with the concept Personal Note: In a major curriculum revision of 1990 I canvassed vigorously for using lisp instead of C,C++,Pascal etc to teach programming. I was booed/shooed out. One could say I was simply delirious Or that I was some 25 years ahead of the time with Lisp-practicalized being spelt ?python? There are so many things that we dont notice that make for bridging the gap between being theoretically neat and practically usable: - good docs - runs OTB - Or at least easy to install - C interface That said, I dont like prolog ? more imperative than classic imperative programming And full proper logic ie full first order logic without abominations like cut and with full quantifier generality is uncomputable ie this may also be a pipe-dream. From tjreedy at udel.edu Sun Aug 21 22:55:43 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 21 Aug 2016 22:55:43 -0400 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On 8/21/2016 9:08 PM, eryk sun wrote: > On Mon, Aug 22, 2016 at 12:44 AM, Chris Angelico wrote: >> "Trying to be" cross-platform? The point of these path modules is to >> *be* cross-platform. I can't call Windows APIs on my Linux box (short >> of messing around with VMs or Wine or something, which are >> dependencies that Python doesn't need). So yes, these *are* >> cross-platform, and that's not scary, that's deliberate design. > > Sorry for the wording SNAFU there. Of course it's cross-platform. But > I think it's a dubious check. On Windows itself I would never try what > it's attempting to do. I would just call GetFullPathName to see if the > path changes to use the \\.\ device namespace. I doubt that the 3 > examples I gave are the only places that this simplistic > implementation of 'is_reserved" fails. It's giving a dangerous > impression to the user that the path will be safe to use on Windows > when that's not the case. Since the function is there, please consider opening an issue and submitting a patch to improve it. Also, could the doc be improved to say that is_reserved() == False might be wrong? It is possible to broaden the test so that too many names, rather than too, are marked as reserved? Better to say "don't use" for a good name rather than 'use' for a reserved name. -- Terry Jan Reedy From rocky at gnu.org Sun Aug 21 23:18:30 2016 From: rocky at gnu.org (rocky) Date: Sun, 21 Aug 2016 20:18:30 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> Message-ID: <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> The assertion should have been assert sys.version >= (3, 0) If we want to indicate the Python program supports language versions 3.0 and greater. On Sunday, August 21, 2016 at 3:59:48 PM UTC-4, Vincent Vande Vyvre wrote: > Le 21/08/2016 ? 20:28, rocky a ?crit : > > The problem: > > > > 1. there are various code inspection tools that parse Python programs looking for style issues or whatnot. The deeper ones have to do a full parse of the python program. It would be helpful if there were a uniform way to indicate the Python language level used in Python source code. > > > > 2. I get a standalone python program that is not part of a package. vcprompt https://bitbucket.org/gward/vcprompt might be an example > > It would be helpful if there were an easy way to know what language version of Python it assumes > > > > Perl has something like called "use perl". "use" is roughly equivalent to "import". > > > > Possible solutions: > > > > Do it the similar to "use perl". Here "perl" is a package that just tests the parameter given it. In Python such the code would look something like > > > > File/module python30.py > > > > import sys > > assert sys.version >= (sys.version_info >= (3, 0)) > > > > The above works, but to reduce proliferation of packages it might be preferable to come up with some way to pass a version specification string similar to the specification strings allowed in setup.py > > > > A metadata tag as a comment in a docstring or in a comment. > > Preferably this would be given towards the top of the file to make it easier for tools to extract this information. > > > > Thoughts? > > I don't understand your assertion > > Python 3.2.3 (default, Jun 18 2015, 21:46:42) > [GCC 4.6.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> assert sys.version >= (sys.version_info >= (3, 0)) > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: str() >= bool() > > > What you means ? > > -- > Vincent V.V. > Oqapy . python3-exiv2 > . Qarte > . PaQager From rustompmody at gmail.com Sun Aug 21 23:28:43 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Aug 2016 20:28:43 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <20160821131822.338b5222@bigbox.christie.dr> Message-ID: <1232e4e9-f8be-42ef-958f-741753feea28@googlegroups.com> On Monday, August 22, 2016 at 12:24:30 AM UTC+5:30, Tim Chase wrote: > On 2016-08-21 04:53, Rustom Mody wrote: > > 2. Basic computing theory shows that re-s and dfas are equivalent. > > Which would one prefer to write/debug? [Thats not a rhetorical > > question] > > I'm curious where REs and DFAs are shown to be equivalent (serious, > not trying to be snarky). I can see with no trouble that all REs are > DFAs, but can't convince myself that *all* DFAs are REs. Though this > might also depend on the RE engine. Do you have links to resources > on such an "all REs have equivalent DFAs and all DFAs have equivalent > REs" logic/argument? Or maybe my understanding of DFAs is flawed. > > And as for which I prefer to write, for simple text-processing that > doesn't involve nesting, I generally prefer REs; but for more complex > processing, a custom DFA tends to expand a bit more cleanly in my > experience. > > -tkc A recent thread has some links on this: https://mail.python.org/pipermail/python-list/2016-July/712046.html Some of the responses to that (Ben Bacarisse??) found better links than mine apparently Also ragel is a tour-de-force that exploits this: http://www.colm.net/open-source/ragel/ From ben+python at benfinney.id.au Sun Aug 21 23:29:48 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 22 Aug 2016 13:29:48 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> Message-ID: <854m6dxyz7.fsf@benfinney.id.au> rocky writes: > The assertion should have been > > assert sys.version >= (3, 0) Perhaps you haven't tried any of these examples before presenting them? >>> import sys >>> assert sys.version >= (3, 0) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: str() >= tuple() Please make some working code before recommending it :-) -- \ ?I was the kid next door's imaginary friend.? ?Emo Philips | `\ | _o__) | Ben Finney From rocky at gnu.org Sun Aug 21 23:37:32 2016 From: rocky at gnu.org (rocky) Date: Sun, 21 Aug 2016 20:37:32 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> Message-ID: <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> Sorry should have been: assert sys.version_info >= (3,0) On Sunday, August 21, 2016 at 11:30:11 PM UTC-4, Ben Finney wrote: > rocky writes: > > > The assertion should have been > > > > assert sys.version >= (3, 0) > > Perhaps you haven't tried any of these examples before presenting them? > > >>> import sys > >>> assert sys.version >= (3, 0) > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: str() >= tuple() > > Please make some working code before recommending it :-) > > -- > \ ?I was the kid next door's imaginary friend.? ?Emo Philips | > `\ | > _o__) | > Ben Finney From eryksun at gmail.com Mon Aug 22 00:17:36 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 04:17:36 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> Message-ID: On Mon, Aug 22, 2016 at 1:26 AM, Chris Angelico wrote: > They're specifically documented as not touching any file system, which > means that they are cross-platform and cannot be guaranteed to be > perfect. If you know you're running on Windows, use WindowsPath > instead (trying to do so on a non-Windows system will cause an > immediate exception), and then you can test your examples on that. It > may well already probe the file system in that situation, and if not, > you could put forward the feature suggestion that it should. But the > PurePath classes won't. PureWindowsPath.is_reserved() delegates to self._flavour.is_reserved(), and _WindowsFlavour.is_reserved() is the common implementation. I don't like splitting this up between a pure and possibly inaccurate version that can be used on Unix systems versus a concrete and accurate version that can only be used on Windows. It can certainly be done if people think that's a good idea. Terry, I'll open an issue that includes the 3 examples I provided plus anything else I can find. Some patterns may emerge for the handling of space, dot, and colon that can be used to fix this in a general way. It would help to consult a reverse-engineered implementation of RtlGetFullPathName_Ustr and RtlIsDosDeviceName_Ustr. I'll check the ReactOS source code. From rosuav at gmail.com Mon Aug 22 00:33:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 14:33:40 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> Message-ID: On Mon, Aug 22, 2016 at 1:37 PM, rocky wrote: > Sorry should have been: > > assert sys.version_info >= (3,0) The next question is: How common is code like this? I don't put version checks in any of my modules. Adding magic comments would be of value only if this sort of thing is common enough to need its own syntax. ChrisA From steve+comp.lang.python at pearwood.info Mon Aug 22 01:35:51 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 22 Aug 2016 15:35:51 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> Message-ID: <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> On Monday 22 August 2016 14:33, Chris Angelico wrote: > On Mon, Aug 22, 2016 at 1:37 PM, rocky wrote: >> Sorry should have been: >> >> assert sys.version_info >= (3,0) > > The next question is: How common is code like this? I don't put > version checks in any of my modules. Adding magic comments would be of > value only if this sort of thing is common enough to need its own > syntax. Most of my code is supposed to work on Python 2.4 through the latest 3.x version. That means I have lots of version-specific code. But most of it uses feature detection rather than version checks. So instead of writing: if sys.version < '3': def next(iterator): return iterator.next() I write this: try: next except NameError: def next(iterator): return iterator.next() or possibly this: if not hasattr(builtins, 'next'): def next(iterator): return iterator.next() But some features depend on new syntax, and you can't check for the availability of new syntax at runtime. (Well, you can, by clever use of exec(), but its too much trouble than its worth.) So very, very occasionally I have to do a version check: if sys.version < '3': import mymodule2 as mymodule else: import mymodule3 as mymodule But... I don't understand what this proposal actually is. We already have a uniform way to indicate the Python language version: check sys.version, or sys.version_info, or sys.hexversion, whichever is more appropriate for your needs. Because this is just an ordinary comparison, you can then do whatever you like, no matter how outlandish or unusual: if sys.version < '3': try: import mymodule3 except SyntaxError: print("Yay, a syntax error!") else: print("Now that's weird, that shouldn't happen...") del mymodule3 import mymodule2 as mymodule elif config[RUN_OLD_VERSION]: import mymodule2 as mymodule else: import mymodule3 as mymodule Could somebody (the OP?) please explain what is the purpose of this proposal, what it does, how it works, and when would people use it? -- Steve From stefan_ml at behnel.de Mon Aug 22 02:03:09 2016 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 22 Aug 2016 08:03:09 +0200 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano schrieb am 22.08.2016 um 07:35: > if sys.version < '3': > import mymodule2 as mymodule > else: > import mymodule3 as mymodule This condition is going to fail when Python 30.0 comes out. Stefan From random832 at fastmail.com Mon Aug 22 02:04:21 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 02:04:21 -0400 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1471845861.3998922.702057129.1702777B@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote: > Could somebody (the OP?) please explain what is the purpose of this > proposal, what it does, how it works, and when would people use it? I think what he wants is a way for a module which uses features (syntactic or otherwise, but I suppose especially syntactic features since this can't as easily be done with a runtime check using existing mechanisms) from a particular python version and which makes no provision to run under earlier versions to fail with a message like "This script requires Python 3.4 or later" rather than a mysterious syntax error or worse a runtime error after the program has been running for some time. From random832 at fastmail.com Mon Aug 22 02:08:23 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 02:08:23 -0400 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1471846103.3999542.702061097.6BF829A1@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 02:03, Stefan Behnel wrote: > Steven D'Aprano schrieb am 22.08.2016 um 07:35: > > if sys.version < '3': > > import mymodule2 as mymodule > > else: > > import mymodule3 as mymodule > > This condition is going to fail when Python 30.0 comes out. Er, won't it rather stop behaving as desired when Python 10.0 comes out, and resume when 30.0 does? From stefan_ml at behnel.de Mon Aug 22 02:10:35 2016 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 22 Aug 2016 08:10:35 +0200 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: Stefan Behnel schrieb am 22.08.2016 um 08:03: > Steven D'Aprano schrieb am 22.08.2016 um 07:35: >> if sys.version < '3': >> import mymodule2 as mymodule >> else: >> import mymodule3 as mymodule > > This condition is going to fail when Python 30.0 comes out. Oh, sorry - make that Python 10.0, that's way closer! See? I got it wrong because I failed to understand your hugely obfuscated code! ;) Stefan From rosuav at gmail.com Mon Aug 22 02:13:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 16:13:32 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 22, 2016 at 3:35 PM, Steven D'Aprano wrote: > Could somebody (the OP?) please explain what is the purpose of this proposal, > what it does, how it works, and when would people use it? My theory: A directive is actually parsed by the compiler, not executed at run-time. So you could have a clean ImportError that says that you need a particular version, rather than a SyntaxError because you used "async def". But given that you can't recover from it without the same "import mymodule2 as mymodule" trick anyway, there's not that much difference. ChrisA From steve+comp.lang.python at pearwood.info Mon Aug 22 02:17:00 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 22 Aug 2016 16:17 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ba98df$0$1520$c3e8da3$5496439d@news.astraweb.com> On Monday 22 August 2016 16:03, Stefan Behnel wrote: > Steven D'Aprano schrieb am 22.08.2016 um 07:35: >> if sys.version < '3': >> import mymodule2 as mymodule >> else: >> import mymodule3 as mymodule > > This condition is going to fail when Python 30.0 comes out. That will be sometime around the year 2257, based on a 18 month minor release cycle and 8 minor releases per major release. I'll deal with it around 2233 or so when Python 28.0 comes out. All joking aside, if Python is still in active development by the time we reach Python 9.7 in roughly seventy years time, I'll be surprised. -- Steve From rocky at gnu.org Mon Aug 22 06:22:28 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 03:22:28 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> <1471845861.3998922.702057129.1702777B@webmail.messagingengine.com> Message-ID: <574afda6-b1e5-4589-92a6-ec1801cec471@googlegroups.com> On Monday, August 22, 2016 at 2:04:39 AM UTC-4, Random832 wrote: > On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote: > > Could somebody (the OP?) please explain what is the purpose of this > > proposal, what it does, how it works, and when would people use it? > > I think what he wants is a way for a module which uses features > (syntactic or otherwise, but I suppose especially syntactic features > since this can't as easily be done with a runtime check using existing > mechanisms) from a particular python version and which makes no > provision to run under earlier versions to fail with a message like > "This script requires Python 3.4 or later" rather than a mysterious > syntax error or worse a runtime error after the program has been running > for some time. Right. People are focusing on specific code instead of the problem: a simple and uniform way to indicate a specific Python dialect in force for that program. The language continues to evolve over time: there are many Python 2.7 programs that won't work on Python 2.5 or earlier and vice versa. When you expand the range from Python 1.5 to Python 3.6 the likelihood of the program running becomes even smaller. Furthermore, I am not aware of any program that when given a Python source code will tell you which or versions or dialects of Python it will run on. The fact that there has been all this much discussion over specific code to me enforces the need for a simple an uniform mechanism. From rocky at gnu.org Mon Aug 22 06:32:26 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 03:32:26 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> Message-ID: <4f8dda3d-bb4f-4882-8823-12d35443b0a0@googlegroups.com> On Monday, August 22, 2016 at 1:36:07 AM UTC-4, Steven D'Aprano wrote: > On Monday 22 August 2016 14:33, Chris Angelico wrote: > > > On Mon, Aug 22, 2016 at 1:37 PM, rocky wrote: > >> Sorry should have been: > >> > >> assert sys.version_info >= (3,0) > > > > The next question is: How common is code like this? I don't put > > version checks in any of my modules. Adding magic comments would be of > > value only if this sort of thing is common enough to need its own > > syntax. > > Most of my code is supposed to work on Python 2.4 through the latest 3.x > version. That means I have lots of version-specific code. > > But most of it uses feature detection rather than version checks. So instead of > writing: > > if sys.version < '3': > def next(iterator): > return iterator.next() > > > I write this: > > try: > next > except NameError: > def next(iterator): > return iterator.next() > > or possibly this: > > > if not hasattr(builtins, 'next'): > def next(iterator): > return iterator.next() > > > But some features depend on new syntax, and you can't check for the > availability of new syntax at runtime. (Well, you can, by clever use of exec(), > but its too much trouble than its worth.) So very, very occasionally I have to > do a version check: > > > if sys.version < '3': > import mymodule2 as mymodule > else: > import mymodule3 as mymodule > > > But... I don't understand what this proposal actually is. We already have a > uniform way to indicate the Python language version: check sys.version, or > sys.version_info, or sys.hexversion, whichever is more appropriate for your > needs. The fact that you mention 3 different ways says to me this isn't uniform. > Because this is just an ordinary comparison, you can then do whatever > you like, no matter how outlandish or unusual: > > > if sys.version < '3': > try: > import mymodule3 > except SyntaxError: > print("Yay, a syntax error!") > else: > print("Now that's weird, that shouldn't happen...") > del mymodule3 > import mymodule2 as mymodule > elif config[RUN_OLD_VERSION]: > import mymodule2 as mymodule > else: > import mymodule3 as mymodule > > > Could somebody (the OP?) please explain what is the purpose of this proposal, > what it does, how it works, and when would people use it? All of the ways you mention involve running the program. Suppose you have a tool that analyzes source code. It reads the code, but doesn't run it. For example possibly code climate. How it it figure out which dialect of Python is being used? > > > > -- > Steve From jon+usenet at unequivocal.eu Mon Aug 22 06:33:34 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 10:33:34 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-22, Steve D'Aprano wrote: > On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >> To me it's scary that this check misses cases because it's trying to >> be cross-platform instead of simply relying on GetFullPathName to do >> the work. For example, it misses at least the following cases: > > Instead of shaking in your boots over a simple bug in a non-critical > library, how about reporting these cases on the bug tracker with an > explanation of the problem? That seems a rather unnecessarily harsh response. Also, it's not "non-critical", this is a security bug. From rustompmody at gmail.com Mon Aug 22 06:44:30 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 22 Aug 2016 03:44:30 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <574afda6-b1e5-4589-92a6-ec1801cec471@googlegroups.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> <1471845861.3998922.702057129.1702777B@webmail.messagingengine.com> <574afda6-b1e5-4589-92a6-ec1801cec471@googlegroups.com> Message-ID: <7cc491ed-9f4c-4f4c-a298-d45f60d971d1@googlegroups.com> On Monday, August 22, 2016 at 3:53:27 PM UTC+5:30, rocky wrote: > On Monday, August 22, 2016 at 2:04:39 AM UTC-4, Random832 wrote: > > On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote: > > > Could somebody (the OP?) please explain what is the purpose of this > > > proposal, what it does, how it works, and when would people use it? > > > > I think what he wants is a way for a module which uses features > > (syntactic or otherwise, but I suppose especially syntactic features > > since this can't as easily be done with a runtime check using existing > > mechanisms) from a particular python version and which makes no > > provision to run under earlier versions to fail with a message like > > "This script requires Python 3.4 or later" rather than a mysterious > > syntax error or worse a runtime error after the program has been running > > for some time. > > Right. People are focusing on specific code instead of the problem: a simple and uniform way to indicate a specific Python dialect in force for that program. The language continues to evolve over time: there are many Python 2.7 programs that won't work on Python 2.5 or earlier and vice versa. When you expand the range from Python 1.5 to Python 3.6 the likelihood of the program running becomes even smaller. > > Furthermore, I am not aware of any program that when given a Python source code will tell you which or versions or dialects of Python it will run on. > > The fact that there has been all this much discussion over specific code to me enforces the need for a simple an uniform mechanism. Prior Art: I may mention that the web-language curl has this feature builtin See very first example here: https://en.wikipedia.org/wiki/Curl_(programming_language) Note this is version(s) specification Auto version detection is hard and likely impossible From rocky at gnu.org Mon Aug 22 06:59:27 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 03:59:27 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <7cc491ed-9f4c-4f4c-a298-d45f60d971d1@googlegroups.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> <1471845861.3998922.702057129.1702777B@webmail.messagingengine.com> <574afda6-b1e5-4589-92a6-ec1801cec471@googlegroups.com> <7cc491ed-9f4c-4f4c-a298-d45f60d971d1@googlegroups.com> Message-ID: On Monday, August 22, 2016 at 6:44:43 AM UTC-4, Rustom Mody wrote: > On Monday, August 22, 2016 at 3:53:27 PM UTC+5:30, rocky wrote: > > On Monday, August 22, 2016 at 2:04:39 AM UTC-4, Random832 wrote: > > > On Mon, Aug 22, 2016, at 01:35, Steven D'Aprano wrote: > > > > Could somebody (the OP?) please explain what is the purpose of this > > > > proposal, what it does, how it works, and when would people use it? > > > > > > I think what he wants is a way for a module which uses features > > > (syntactic or otherwise, but I suppose especially syntactic features > > > since this can't as easily be done with a runtime check using existing > > > mechanisms) from a particular python version and which makes no > > > provision to run under earlier versions to fail with a message like > > > "This script requires Python 3.4 or later" rather than a mysterious > > > syntax error or worse a runtime error after the program has been running > > > for some time. > > > > Right. People are focusing on specific code instead of the problem: a simple and uniform way to indicate a specific Python dialect in force for that program. The language continues to evolve over time: there are many Python 2.7 programs that won't work on Python 2.5 or earlier and vice versa. When you expand the range from Python 1.5 to Python 3.6 the likelihood of the program running becomes even smaller. > > > > Furthermore, I am not aware of any program that when given a Python source code will tell you which or versions or dialects of Python it will run on. > > > > The fact that there has been all this much discussion over specific code to me enforces the need for a simple an uniform mechanism. > > Prior Art: > I may mention that the web-language curl has this feature builtin > See very first example here: > https://en.wikipedia.org/wiki/Curl_(programming_language) > > Note this is version(s) specification > Auto version detection is hard and likely impossible As mentioned in the original post Perl has it too. http://perldoc.perl.org/functions/use.html And I gotta say it is pretty clever how they worked in the duplicate match up between common English usage and valid programming language syntax. From rosuav at gmail.com Mon Aug 22 07:04:35 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 21:04:35 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 22, 2016 at 8:33 PM, Jon Ribbens wrote: > On 2016-08-22, Steve D'Aprano wrote: >> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >>> To me it's scary that this check misses cases because it's trying to >>> be cross-platform instead of simply relying on GetFullPathName to do >>> the work. For example, it misses at least the following cases: >> >> Instead of shaking in your boots over a simple bug in a non-critical >> library, how about reporting these cases on the bug tracker with an >> explanation of the problem? > > That seems a rather unnecessarily harsh response. > Also, it's not "non-critical", this is a security bug. Explain how? ChrisA From rocky at gnu.org Mon Aug 22 07:48:43 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 04:48:43 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> Message-ID: <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> On Monday, August 22, 2016 at 12:33:53 AM UTC-4, Chris Angelico wrote: > On Mon, Aug 22, 2016 at 1:37 PM, rocky wrote: > > Sorry should have been: > > > > assert sys.version_info >= (3,0) > > The next question is: How common is code like this? I don't put > version checks in any of my modules. Adding magic comments would be of > value only if this sort of thing is common enough to need its own > syntax. > > ChrisA I'll answer your question with a question: How common are "software as service" code-analysis tools for Python used? A slightly different but related problem is noting the Python dialect at the package-level. If a package is in Pypi then PKGINFO can contain the a list of Python dialects supported. If the program is not in Pypi, then what? As with "use perl" inside source code, the same mechanism ("use perl") is used inside its build system(s) to indicate which dialects a Perl package supports. From jon+usenet at unequivocal.eu Mon Aug 22 07:50:25 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 11:50:25 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-22, Chris Angelico wrote: > On Mon, Aug 22, 2016 at 8:33 PM, Jon Ribbens wrote: >> On 2016-08-22, Steve D'Aprano wrote: >>> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >>>> To me it's scary that this check misses cases because it's trying to >>>> be cross-platform instead of simply relying on GetFullPathName to do >>>> the work. For example, it misses at least the following cases: >>> >>> Instead of shaking in your boots over a simple bug in a non-critical >>> library, how about reporting these cases on the bug tracker with an >>> explanation of the problem? >> >> That seems a rather unnecessarily harsh response. >> Also, it's not "non-critical", this is a security bug. > > Explain how? I don't know what purpose you are envisaging this function being used for, but the only one I can think of is input sanitisation. e.g. a web form where you receive a file from the Internet and store it somewhere, and you want to use the filename given to you rather than choose your own randomly-generated one. Under Unix all you need to do is check for the filename starting with "." or containing "/." (or "/", depending on your requirements). Under Windows you would use this function, which apparently doesn't work, hence: security hole. From steve+python at pearwood.info Mon Aug 22 08:02:43 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 22:02:43 +1000 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57bae9e6$0$1588$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 08:33 pm, Jon Ribbens wrote: > On 2016-08-22, Steve D'Aprano wrote: >> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >>> To me it's scary that this check misses cases because it's trying to >>> be cross-platform instead of simply relying on GetFullPathName to do >>> the work. For example, it misses at least the following cases: >> >> Instead of shaking in your boots over a simple bug in a non-critical >> library, how about reporting these cases on the bug tracker with an >> explanation of the problem? > > That seems a rather unnecessarily harsh response. Eryksun bought into Lawrence's over-the-top rhetorical question "does this scare you?" by answering "Yes", and repeating the ridiculous term "scary". He specifically said that it scares him *because* it is cross-platform code, as if cross-platform code is a bad thing. Now I'm sure that Eryksun isn't *actually* scared of cross-platform code. I'm sure he is quite capable of using (say) os.listdir() without widdling himself in terror *wink*. And I don't know if he was intentionally using the word "scary" or whether it was just an ill-thought out choice of words. Either way, yes, I'm making a gentle dig at Eryksun for exaggerating the magnitude of the supposed problem and for taking something which is clearly a mere bug and treating it as a feature that is broken by design. There's nothing wrong with writing cross-platform code, and there's no reason why non-Windows users shouldn't be permitted to explicitly query whether a file name could be valid on a Windows system. > Also, it's not "non-critical", this is a security bug. How is this a security bug? What's the nature of the vulnerability? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From marko at pacujo.net Mon Aug 22 08:05:03 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Aug 2016 15:05:03 +0300 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> Message-ID: <87pop1dn68.fsf@elektro.pacujo.net> rocky : > A slightly different but related problem is noting the Python dialect > at the package-level. I don't know what if anything is needed support this idea, but one option would be to just use "import": import python3_5_17 That would require Python and modules to install such empty modules to indicate that they support the given API. Marko From rosuav at gmail.com Mon Aug 22 08:39:05 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 22:39:05 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 22, 2016 at 9:50 PM, Jon Ribbens wrote: > On 2016-08-22, Chris Angelico wrote: >> On Mon, Aug 22, 2016 at 8:33 PM, Jon Ribbens wrote: >>> On 2016-08-22, Steve D'Aprano wrote: >>>> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >>>>> To me it's scary that this check misses cases because it's trying to >>>>> be cross-platform instead of simply relying on GetFullPathName to do >>>>> the work. For example, it misses at least the following cases: >>>> >>>> Instead of shaking in your boots over a simple bug in a non-critical >>>> library, how about reporting these cases on the bug tracker with an >>>> explanation of the problem? >>> >>> That seems a rather unnecessarily harsh response. >>> Also, it's not "non-critical", this is a security bug. >> >> Explain how? > > I don't know what purpose you are envisaging this function being used > for, but the only one I can think of is input sanitisation. e.g. a web > form where you receive a file from the Internet and store it somewhere, > and you want to use the filename given to you rather than choose your > own randomly-generated one. > > Under Unix all you need to do is check for the filename starting with > "." or containing "/." (or "/", depending on your requirements). > Under Windows you would use this function, which apparently doesn't > work, hence: security hole. Nope. On Windows, you would try/except it. There are myriad other ways something could fail, and the only correct action is to attempt it. Most of the reserved names will simply give an error; the only way you'd actually get incorrect behaviour is if the file name, including extension, is exactly a device name. (Caveat: My knowledge of Windows is rusty and my testing just now was cursory. I could be wrong.) So you can check for a few exact strings... or just slap some extra text onto the beginning or end of the file name (beginning meaning "after the last slash", not the beginning of the file *path*) and you're safe. ChrisA From rosuav at gmail.com Mon Aug 22 08:44:49 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 22:44:49 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <87pop1dn68.fsf@elektro.pacujo.net> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> Message-ID: On Mon, Aug 22, 2016 at 10:05 PM, Marko Rauhamaa wrote: > rocky : > >> A slightly different but related problem is noting the Python dialect >> at the package-level. > > I don't know what if anything is needed support this idea, but one > option would be to just use "import": > > import python3_5_17 > > That would require Python and modules to install such empty modules to > indicate that they support the given API. Preferable: from python import v35 Then a single 'python.py' could have lines for every supported version, rather than forcing dozens of separate modules. However, I don't think it's particularly necessary. Explicit version number checks should be very rare, and shouldn't be encouraged. Instead, encourage feature checks, as Steve gave some examples of. ChrisA From rocky at gnu.org Mon Aug 22 08:49:36 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 05:49:36 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <87pop1dn68.fsf@elektro.pacujo.net> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> Message-ID: On Monday, August 22, 2016 at 8:05:15 AM UTC-4, Marko Rauhamaa wrote: > rocky : > > > A slightly different but related problem is noting the Python dialect > > at the package-level. > > I don't know what if anything is needed support this idea, but one > option would be to just use "import": > > import python3_5_17 > > That would require Python and modules to install such empty modules to > indicate that they support the given API. > > > Marko Yes, that is the intent of File/module python30.py in the first post. The only difference is that because there is code too, when run, it can enforce the version declared. As mentioned originally, a problem with this is that you have a proliferation of names and files. Better, I htink would be a way to pass a parameter string. Perl's "use" mechanism happens to allow passing in such a parameter. In Python without syntactic sugar this would be a two-step process of first importing and then calling something with the version range. From steve+python at pearwood.info Mon Aug 22 08:50:53 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 22:50:53 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> <4f8dda3d-bb4f-4882-8823-12d35443b0a0@googlegroups.com> Message-ID: <57baf530$0$1596$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 08:32 pm, rocky wrote: > On Monday, August 22, 2016 at 1:36:07 AM UTC-4, Steven D'Aprano wrote: [...] >> But... I don't understand what this proposal actually is. We already have >> a uniform way to indicate the Python language version: check sys.version, >> or sys.version_info, or sys.hexversion, whichever is more appropriate for >> your needs. > > The fact that you mention 3 different ways says to me this isn't uniform. What do you mean? Of course it is uniform -- every compliant Python interpreter must support sys.version, sys.hexversion and sys.version_info. sys.version and sys.hexversion go back to at least Python 1.5 and possibly older. I'm not sure when version_info was introduced, but anything offering Python 2.4 or better must support it. So as far as current and future versions of Python go, you can trust that *any* version of *any* implementation will support those three attributes, be it CPython, PyPy, Nuitika, Jython, IronPython, Stackless, or something else. Perhaps you mean "unique" rather than uniform -- but of course you cannot have a unique way of doing this, since people can simply test for the existence of features, or inspect the version number, whichever they prefer. Adding something like Perl's "use" won't change that -- it still won't be unique. >> Could somebody (the OP?) please explain what is the purpose of this >> proposal, what it does, how it works, and when would people use it? > > All of the ways you mention involve running the program. Suppose you have > a tool that analyzes source code. It reads the code, but doesn't run it. > For example possibly code climate. What is "code climate"? > How it it figure out which dialect of Python is being used? In general, you can't, because Python code will run under many different versions and implementations. I'm not strongly opposed to this suggestion. I just don't see the point. I've never found myself missing this feature. And I don't like version testing except as an absolute last resort: I much prefer feature detection. But the biggest problem with this suggestion is the practical issue that the absolute earliest it could be introduced will be Python 3.6, and more likely 3.7. Python 3.6 has only a couple of weeks left before feature freeze, and all older versions are likewise frozen or out of support. Realistically, by the time you convince people this is a useful feature, write a patch and have the patch reviewed, you'll be looking at Python 3.7. So 3.7 will have this new syntax "use version", and *no other version*. That means you can't use it until you are ready to abandon Python 2.4 through 3.6. I'm assuming you want a new syntactic feature like Perl's "use version". If its just a function, then you can use a version check to test for it first: if sys.version_info > (3, 7): sys.use("v3.7") but that makes "use" completely redundant. (If you have to test the version before you can test the version, what's the point of the second test?) If this feature had been built into Python from the start, it would merely be an attractive nuisance, encouraging people to test the version instead of doing feature-detection. But since it wasn't, I don't see any point to add it now: if we added it to Python 3.7, it wouldn't be until (probably) 2028 or significantly later that folks could rely on it being available. (Currently, I think that Red Hat offer the longest paid commercial support for Python versions: ten years. Suppose they start using Python 3.6 as their standard Python next year, it won't fall out of support until 2028. That's the absolute earliest pre-3.7 versions could be considered out of support.) Earlier, in a previous email, Rocky also wrote: > Right. People are focusing on specific code instead of the problem: a > simple and uniform way to indicate a specific Python dialect in force for > that program. That's not describing the problem. That's describing one specific solution to some unknown problem. The *solution* is "a simple and uniform way to indicate..." but what is the problem? Under what circumstances would people use this? Why would they use it? Normally, there's no question about what version of Python a module is meant to use: - if it is a script, it has a hash-bang line at the top which specifies the location of the Python interpreter; - if its a library, it will be installed in the site-packages directory of a specific version of the interpreter; - and more importantly, many (possibly even most) Python code will run under multiple versions. There are exceptions to this of course. Sometimes libraries will be just dumped any old place, and scripts may not have hash-bang lines. Neither of these are compulsory. But neither will your proposed "use version". > The language continues to evolve over time: there are many > Python 2.7 programs that won't work on Python 2.5 or earlier and vice > versa. When you expand the range from Python 1.5 to Python 3.6 the > likelihood of the program running becomes even smaller. None of those versions can possibly support this "use" syntax or function. There's just barely a tiny chance that it could be added to Python 3.6. > Furthermore, I am not aware of any program that when given a Python source > code will tell you which or versions or dialects of Python it will run on. In general, you can't have such a program. But even if you could, why would you want it? > The fact that there has been all this much discussion over specific code > to me enforces the need for a simple an uniform mechanism. There's only been this much discussion because you raised the issue and were not clear about what it was. I have been a regular here for over a decade, and this is the first time I've seen anyone asking about this sort of feature. If you hadn't raised the issue, we wouldn't be discussing it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rocky at gnu.org Mon Aug 22 08:52:31 2016 From: rocky at gnu.org (rocky) Date: Mon, 22 Aug 2016 05:52:31 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> Message-ID: On Monday, August 22, 2016 at 8:45:05 AM UTC-4, Chris Angelico wrote: > On Mon, Aug 22, 2016 at 10:05 PM, Marko Rauhamaa wrote: > > rocky : > > > >> A slightly different but related problem is noting the Python dialect > >> at the package-level. > > > > I don't know what if anything is needed support this idea, but one > > option would be to just use "import": > > > > import python3_5_17 > > > > That would require Python and modules to install such empty modules to > > indicate that they support the given API. > > Preferable: > > from python import v35 > > Then a single 'python.py' could have lines for every supported > version, rather than forcing dozens of separate modules. > > However, I don't think it's particularly necessary. Explicit version > number checks should be very rare, and shouldn't be encouraged. > Instead, encourage feature checks, as Steve gave some examples of. > > ChrisA Yes, "from python import v35" is better. How do feature checks in the code address the problem static analysis of source code? From random832 at fastmail.com Mon Aug 22 08:52:55 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 08:52:55 -0400 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> Message-ID: <1471870375.576317.702353561.2CC6D3A5@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 08:44, Chris Angelico wrote: > However, I don't think it's particularly necessary. Explicit version > number checks should be very rare, and shouldn't be encouraged. > Instead, encourage feature checks, as Steve gave some examples of. The problem is when you want to write a large body of code that just *uses* lots of features (including syntactic features), *without* checking for them. Steve's examples were of how to support earlier versions, not how *not* to. At some point you want to just write Python 3 code and say to hell with Python 2, or the same for any particular version of Python 3. From random832 at fastmail.com Mon Aug 22 08:56:50 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 08:56:50 -0400 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 08:39, Chris Angelico wrote: > Nope. On Windows, you would try/except it. No, you can't, because the failure mode often isn't "file refuses to open" but "data is written to a serial port". There are myriad other ways > something could fail, and the only correct action is to attempt it. > Most of the reserved names will simply give an error; the only way > you'd actually get incorrect behaviour is if the file name, including > extension, is exactly a device name. I think the reason you believe this can be traced back to the "C:\con\con" trick, which crashed the system by trying to use the name as a directory. > (Caveat: My knowledge of Windows > is rusty and my testing just now was cursory. I could be wrong.) Eryk Sun already posted an example using "NUL .txt". From eryksun at gmail.com Mon Aug 22 08:57:50 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 12:57:50 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 22, 2016 at 12:39 PM, Chris Angelico wrote: > > Nope. On Windows, you would try/except it. There are myriad other ways No, I would not rely on exceptions in this case. Both \\.\CON and \\.\NUL can be opened for both reading and writing, so you may not detect the problem. I think it's best to check for this via os.path.abspath, which calls GetFullPathName, which is implemented by RtlGetFullPathName_Ustr, which is exactly what RtlDosPathNameToNtPathName_U uses to normalize a path before it converts it to the NT namespace. > or just slap some extra text onto the beginning or end of the file name > (beginning meaning "after the last slash", not the beginning of the file > *path*) and you're safe. Adding to the beginning is the safe bet without having to worry about the rules for trailing spaces, dots, and colons at the end of the name. Adding a single underscore prefix will suffice (or however many to make a unique name) if for some reason you can't ask the user for a new name. From python.list at tim.thechases.com Mon Aug 22 09:04:14 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 22 Aug 2016 08:04:14 -0500 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <20160822080414.07349ad0@bigbox.christie.dr> On 2016-08-22 22:39, Chris Angelico wrote: > Nope. On Windows, you would try/except it. There are myriad other > ways something could fail, and the only correct action is to > attempt it. Most of the reserved names will simply give an error; The problem is that when opening such a pseudo-file, you can get unexpected behavior. In the Unix world, we're used to files-that-aren-t-files (such as things in /dev ). But a lot of Windows developers don't handle these cases, and so opening something like COM1 can end up hanging a program indefinitely instead of actually returning either an error or a file-handle. If you have a web-server running on Windows and can manage to coerce a file to have such a name, you might be able to hang the web-server process while it tries to read from (or write to) the serial port. And poof, near-instant DoS. -tkc From marko at pacujo.net Mon Aug 22 09:12:57 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Aug 2016 16:12:57 +0300 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> <1471870375.576317.702353561.2CC6D3A5@webmail.messagingengine.com> Message-ID: <878tvpdk12.fsf@elektro.pacujo.net> Random832 : > The problem is when you want to write a large body of code that just > *uses* lots of features (including syntactic features), *without* > checking for them. Ordinarily, that's the job of package management. The installer will perform the necessary checks for you. Marko From steve+python at pearwood.info Mon Aug 22 09:15:49 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 23:15:49 +1000 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57bafb06$0$1614$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 09:50 pm, Jon Ribbens wrote: > On 2016-08-22, Chris Angelico wrote: >> On Mon, Aug 22, 2016 at 8:33 PM, Jon Ribbens >> wrote: >>> On 2016-08-22, Steve D'Aprano wrote: >>>> On Mon, 22 Aug 2016 10:38 am, eryk sun wrote: >>>>> To me it's scary that this check misses cases because it's trying to >>>>> be cross-platform instead of simply relying on GetFullPathName to do >>>>> the work. For example, it misses at least the following cases: >>>> >>>> Instead of shaking in your boots over a simple bug in a non-critical >>>> library, how about reporting these cases on the bug tracker with an >>>> explanation of the problem? >>> >>> That seems a rather unnecessarily harsh response. >>> Also, it's not "non-critical", this is a security bug. >> >> Explain how? > > I don't know what purpose you are envisaging this function being used > for, but the only one I can think of is input sanitisation. e.g. a web > form where you receive a file from the Internet and store it somewhere, > and you want to use the filename given to you rather than choose your > own randomly-generated one. > > Under Unix all you need to do is check for the filename starting with > "." or containing "/." (or "/", depending on your requirements). > Under Windows you would use this function, which apparently doesn't > work, hence: security hole. That's backwards: it works under Windows, but is buggy under non-Windows. In any case, I don't think that what you suggest is even close to sufficient. For starters, even on Unix I probably wouldn't want to use "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\.txt" as a file name, even if it is technically legal. Nor would I allow "foo.jpg .exe" either, especially not on Windows. But let's suppose my application can handle newlines and other control characters in filenames (I never `ls` the storage directory, all my scripts are newline safe, etc.) And I don't care about users uploading malware with disguised file names. Am I done? No, not at all. I can't assume that just because a file name is legal that I can write to it. Even after sanitising the name (or deciding I don't need to bother) I still need to be prepared to catch I/O errors when writing the file -- including attempts to write to reserved file names. None of this makes the issue a security issue. How would you exploit it? The attacker would have to ensure that the file name is sanitised under Linux but then written to a file under Windows, in which case the worst that happens is that I get an unexpected I/O error. Technically that could be called a DoS attack, but the same could be said for ANY bug that raises an exception. I think that it takes more than that to be a security bug. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Mon Aug 22 09:17:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Aug 2016 23:17:50 +1000 Subject: Does This Scare You? In-Reply-To: <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Mon, Aug 22, 2016 at 10:56 PM, Random832 wrote: >> Most of the reserved names will simply give an error; the only way >> you'd actually get incorrect behaviour is if the file name, including >> extension, is exactly a device name. > > I think the reason you believe this can be traced back to the > "C:\con\con" trick, which crashed the system by trying to use the name > as a directory. I tried things like "con.txt" and it simply failed (no such file or directory), without printing anything to the console. But as Eryk says, adding an underscore is safe; and to be honest, I wouldn't accept file names from untrusted sources on *any* system - at very least, I'd prefix/suffix them with something to ensure uniqueness, which would deal with this issue as a convenient side effect. (Or alternatively, I'd use arbitrary numbers or hashes as the file names, and store the originally-submitted file name in some sort of metadata repository, like a Postgres table.) So I still don't see this as a security problem, just a practicality one. ChrisA From steve+python at pearwood.info Mon Aug 22 09:21:30 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 23:21:30 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> <1471870375.576317.702353561.2CC6D3A5@webmail.messagingengine.com> Message-ID: <57bafc5b$0$1603$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 10:52 pm, Random832 wrote: > On Mon, Aug 22, 2016, at 08:44, Chris Angelico wrote: >> However, I don't think it's particularly necessary. Explicit version >> number checks should be very rare, and shouldn't be encouraged. >> Instead, encourage feature checks, as Steve gave some examples of. > > The problem is when you want to write a large body of code that just > *uses* lots of features (including syntactic features), *without* > checking for them. Steve's examples were of how to support earlier > versions, not how *not* to. At some point you want to just write Python > 3 code and say to hell with Python 2, or the same for any particular > version of Python 3. Surely that's not a problem -- it's what people have been doing with Python code for decades. When was the last time you saw somebody do an explicit version check? if sys.version_info() < (2, 2): raise RuntimeError('Python too old') Rather, you just use the features you rely on, document the minimum supported version, and if somebody is silly enough to try running your code under Python 1.4, they'll get a SyntaxError or an exception when you try to do something that is not supported. Maybe if Python supported this from the beginning it would be worth the bother. But it seems of very marginal utility to me. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Aug 22 09:34:23 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 22 Aug 2016 23:34:23 +1000 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: <57baff60$0$1607$c3e8da3$5496439d@news.astraweb.com> On Mon, 22 Aug 2016 10:56 pm, Random832 wrote: > On Mon, Aug 22, 2016, at 08:39, Chris Angelico wrote: >> Nope. On Windows, you would try/except it. > > No, you can't, because the failure mode often isn't "file refuses to > open" but "data is written to a serial port". Ah, that's a good point. I hadn't thought of that. But... what are the consequences if you write to the serial port? Unless you actually have an external device plugged into it, isn't that equivalent to writing to /dev/null? (Bytes go into the serial port, and just disappear.) The user uploads their file, and cleverly fools you into discarding their file? I'm not seeing how this is an attack. I suppose they could write to CON and display a lot of garbage on the screen. But if you're running this on Windows, surely you've already dealt with these issues, in which case it's a non-issue. Or you haven't dealt with them, in which case it's an existing bug and the code Lawrence demonstrated doesn't change anything. >> There are myriad other ways >> something could fail, and the only correct action is to attempt it. >> Most of the reserved names will simply give an error; the only way >> you'd actually get incorrect behaviour is if the file name, including >> extension, is exactly a device name. > > I think the reason you believe this can be traced back to the > "C:\con\con" trick, which crashed the system by trying to use the name > as a directory. \con\con hasn't been an issue since Windows 98. If you're running your web application under Win 98, you deserve to be blue-screened :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rustompmody at gmail.com Mon Aug 22 09:38:00 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 22 Aug 2016 06:38:00 -0700 (PDT) Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57baf530$0$1596$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <57ba8f3b$0$1523$c3e8da3$5496439d@news.astraweb.com> <4f8dda3d-bb4f-4882-8823-12d35443b0a0@googlegroups.com> <57baf530$0$1596$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6e30c11d-8a3c-4eb3-89fd-7563891bf9bd@googlegroups.com> On Monday, August 22, 2016 at 7:00:36 PM UTC+5:30, Steve D'Aprano wrote: > Realistically, by the time you convince people this is a useful feature, > write a patch and have the patch reviewed, you'll be looking at Python 3.7. > So 3.7 will have this new syntax "use version", and *no other version*. > That means you can't use it until you are ready to abandon Python 2.4 > through 3.6. Yes First-class version detection and management needs to be put into the language right at the outset if it is to work If you do it late in the history (like now for python), you need to 1 First detect if first-class version management is supported 2 If yes use it 3 If no use the old ad-hoc ways So might as well use 3 alone!! From ben+python at benfinney.id.au Mon Aug 22 10:21:12 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 23 Aug 2016 00:21:12 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: <85wpj8x4tj.fsf_-_@benfinney.id.au> Chris Angelico writes: > [?] to be honest, I wouldn't accept file names from untrusted sources > on *any* system [?] That's one of the wiser things said in this whole thread. > I'd use arbitrary numbers or hashes as the file names, and store the > originally-submitted file name in some sort of metadata repository, > like a Postgres table.) The failure modes of using filenames from untrusted input are shockingly diverse, as Tom Eastman describes: The scope for abuse is eye-widening: The contents of the file, the type of the file, the size and encoding of the file, even the *name* of the file can be a potent vector for attacking your system. The scariest part? Even the best and most secure web-frameworks (yes, I'm talking about Django) can't protect you from all of it. In this talk, I'll show you every scary thing I know about that can be done with a file upload, and how to protect yourself from -- hopefully -- most of them. Tom presented to us at this year's PyCon AU . So yes, filenames from arbitrary sources should be *completely* untrusted, and never used to access any file on the system. Throw the entire filename away and make a filename locally, without using any part of the original name. -- \ ?I saw a sign: ?Rest Area 25 Miles?. That's pretty big. Some | `\ people must be really tired.? ?Steven Wright | _o__) | Ben Finney From random832 at fastmail.com Mon Aug 22 10:29:49 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 10:29:49 -0400 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <57bafc5b$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> <1471870375.576317.702353561.2CC6D3A5@webmail.messagingengine.com> <57bafc5b$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1471876189.4028290.702457473.27383162@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 09:21, Steve D'Aprano wrote: > Rather, you just use the features you rely on, document the minimum > supported version, and if somebody is silly enough to try running your > code > under Python 1.4, they'll get a SyntaxError or an exception when you try > to > do something that is not supported. Receiving a SyntaxError or whatever other exception, which provides no suggestion about how to actually fix the issue (install a later version of python / run with "python3" instead of "python"), is a bad user experience. It will continue to be a bad user experience when people are using features that only work on python 5.0 and later and other people are trying to run their scripts under python 4.0, so it not having existed all along is not a sufficient justification to not consider adding it,. From random832 at fastmail.com Mon Aug 22 10:34:47 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 10:34:47 -0400 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <85wpj8x4tj.fsf_-_@benfinney.id.au> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> Message-ID: <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 10:21, Ben Finney wrote: > So yes, filenames from arbitrary sources should be *completely* > untrusted, and never used to access any file on the system. Throw the > entire filename away and make a filename locally, without using any part > of the original name. To be fair, this particular case is unique in presenting a possibility to cause problems even for a filename that consists only of whitelisted characters (for a reasonable-sounding whitelist such as "ASCII letters and numbers and underscore only; all other characters to be scrubbed and replaced with {underscore, hex escape, nothing}"). I don't think there's any other precedent. From larry.martell at gmail.com Mon Aug 22 10:36:58 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 10:36:58 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Fri, Aug 19, 2016 at 4:24 PM, Chris Kaynor wrote: > On Fri, Aug 19, 2016 at 12:00 PM, Larry Martell > wrote: > >> On Fri, Aug 19, 2016 at 1:24 PM, Chris Angelico wrote: >> > On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell >> wrote: >> >> I have some python code (part of a django app) that processes a >> >> request that contains a png file. The request is send with >> >> content_type = 'application/octet-stream' >> >> >> >> In the python code I want to write this data to a file and still have >> >> it still be a valid png file. >> >> >> >> The data I get looks like this: >> >> >> >> u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\ >> x00\x00\x01\ufffd >> >> ......' >> >> >> >> If I try and write that to a file it fails with a UnicodeEncodeError. >> >> If I write it with encode('utf8') it writes the file, but then it's no >> >> longer a valid png file. >> >> >> >> Anyone know how I can do this? >> > >> > At that point, you've already lost information. Each U+FFFD (shown as >> > "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" >> > (or whatever was being used). Something somewhere took the .png file's >> > bytes and tried to interpret them as text, which they're not. >> > >> > What sent you that data? How did you receive it? >> >> The request is sent by a client app written in C++ with Qt. It's >> received by a django based server. I am trying to port a falcon server >> to django. The falcon server code did this: >> >> form = cgi.FieldStorage(fp=req.stream, environ=req.env) >> >> and then wrote the png like this: >> >> fd.write(form[key].file.read()) >> >> Whereas in the django server I am doing: >> >> fd.write(request.POST[key]) >> >> I've never used the cgi module. I guess I can try that. I've written a >> lot with django but never had to receive a PNG file. >> >> > I don't know Django, however a quick search makes it seem like you might > need to use request.FILES[key] (1) rather than request.POST[key]. You may > also be able to use request.POST if you set request.encoding first (2). If > both of those fail, you may need to use request.body and parse the HTTP > form data manually, though I'd imagine there is an easier way. > > [1] > https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.FILES > > [2] > https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.encoding Thanks for the reply. When I get the request, request.FILES is empty. Yet the content type is multipart/form-data and the method is POST: (Pdb) print request.META['CONTENT_TYPE'] multipart/form-data; boundary="boundary_.oOo._NzEwNjIzMTM4MTI4NjUxOTM5OQ==MTY2NjE4MDk5Nw==" (Pdb) print request.META['REQUEST_METHOD'] POST (Pdb) print request.FILES Tried setting request.encoding, but that messes up the request structure: (Pdb) type(request.POST[key]) (Pdb) request.encoding = "iso-8859-1" (Pdb) type(request.POST[key]) *** MultiValueDictKeyError: "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" From larry.martell at gmail.com Mon Aug 22 10:37:57 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 10:37:57 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Sun, Aug 21, 2016 at 5:24 PM, Jon Ribbens wrote: > On 2016-08-19, Larry Martell wrote: >> fd.write(request.POST[key]) > > You could try: > > request.encoding = "iso-8859-1" > fd.write(request.POST[key].encode("iso-8859-1")) > > It's hacky and nasty and there might be a better "official" method > but I think it should work. For some reason that messes up the request structure: (Pdb) type(request.POST[key]) (Pdb) request.encoding = "iso-8859-1" (Pdb) type(request.POST[key]) *** MultiValueDictKeyError: "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" From rosuav at gmail.com Mon Aug 22 11:33:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 01:33:59 +1000 Subject: PEP suggestion: Uniform way to indicate Python language version In-Reply-To: <1471876189.4028290.702457473.27383162@webmail.messagingengine.com> References: <1b43eb5d-3b04-49c0-9db5-edacc997eeb3@googlegroups.com> <896b3fb4-1a38-70cb-13e8-b3570497196c@telenet.be> <01c0bb1f-89c2-4a16-86fb-25c704ee9b3c@googlegroups.com> <854m6dxyz7.fsf@benfinney.id.au> <9fe7efc8-6330-46ed-9cb7-d9f31ecf05df@googlegroups.com> <47e6dd46-7784-4a3e-b8fb-8fe70b08ef13@googlegroups.com> <87pop1dn68.fsf@elektro.pacujo.net> <1471870375.576317.702353561.2CC6D3A5@webmail.messagingengine.com> <57bafc5b$0$1603$c3e8da3$5496439d@news.astraweb.com> <1471876189.4028290.702457473.27383162@webmail.messagingengine.com> Message-ID: On Tue, Aug 23, 2016 at 12:29 AM, Random832 wrote: > On Mon, Aug 22, 2016, at 09:21, Steve D'Aprano wrote: >> Rather, you just use the features you rely on, document the minimum >> supported version, and if somebody is silly enough to try running your >> code >> under Python 1.4, they'll get a SyntaxError or an exception when you try >> to >> do something that is not supported. > > Receiving a SyntaxError or whatever other exception, which provides no > suggestion about how to actually fix the issue (install a later version > of python / run with "python3" instead of "python"), is a bad user > experience. It will continue to be a bad user experience when people are > using features that only work on python 5.0 and later and other people > are trying to run their scripts under python 4.0, so it not having > existed all along is not a sufficient justification to not consider > adding it,. How bad, exactly? If you installed the program using a package manager (apt, yum, brew, etc etc etc), there should be versioning directives in that, just the same as if there's some particular feature of libc or openssl or libjpeg that you need. And if you go to someone's web site to download it, you should get told "This requires Python X.Y". Sometimes, when there's a really common failure, you can toss in a try/except, or even just a comment: import tkinter # ImportError? You may need Python 3 which will then show up if something goes wrong. There are much, MUCH better ways of solving this than trying to magically make older Pythons, uhh.... uhh.... do what, exactly? Nobody's yet said what ought to happen instead of SyntaxError/ImportError. Should the process be terminated with an error message? Should a different exception be raised (InsufficientVersionError) and a check added to the default error handler so it displays differently (the way SystemExit does, or rather doesn't display at all)? And above all, how is all this an improvement over the status quo, which is either a SyntaxError/ImportError straight away, or the ability to put a straight-forward run-time check that does anything you like? ChrisA From python.list at tim.thechases.com Mon Aug 22 11:35:29 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 22 Aug 2016 10:35:29 -0500 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <85wpj8x4tj.fsf_-_@benfinney.id.au> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> Message-ID: <20160822103529.74007d40@bigbox.christie.dr> On 2016-08-23 00:21, Ben Finney wrote: > So yes, filenames from arbitrary sources should be *completely* > untrusted, and never used to access any file on the system. Throw > the entire filename away and make a filename locally, without using > any part of the original name. Sadly, this ideal advice too often conflicts with the shoddy Code Other People Wrote In Our "Enterprise" System?. :-/ -tkc From rosuav at gmail.com Mon Aug 22 11:40:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 01:40:59 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> Message-ID: On Tue, Aug 23, 2016 at 12:34 AM, Random832 wrote: > On Mon, Aug 22, 2016, at 10:21, Ben Finney wrote: >> So yes, filenames from arbitrary sources should be *completely* >> untrusted, and never used to access any file on the system. Throw the >> entire filename away and make a filename locally, without using any part >> of the original name. > > To be fair, this particular case is unique in presenting a possibility > to cause problems even for a filename that consists only of whitelisted > characters (for a reasonable-sounding whitelist such as "ASCII letters > and numbers and underscore only; all other characters to be scrubbed and > replaced with {underscore, hex escape, nothing}"). I don't think there's > any other precedent. Windows has some other issues, including that arbitrary files can become executable very easily (eg if %PATHEXT% includes its file extension), and since the current directory is always at the beginning of your path, this can easily turn into a remote code execution exploit. And any GUI that automatically calculates thumbnails from image files (this includes Windows, Mac OS, and more than one Linux window manager) could potentially be attacked via a malformed file, simply by having it appear on the file system. So the idea that some file names are dangerous is far FAR broader than "a file called prn.txt will get saved to the printer". ChrisA From rosuav at gmail.com Mon Aug 22 11:45:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 01:45:50 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <85wpj8x4tj.fsf_-_@benfinney.id.au> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> Message-ID: On Tue, Aug 23, 2016 at 12:21 AM, Ben Finney wrote: > > So yes, filenames from arbitrary sources should be *completely* > untrusted, and never used to access any file on the system. Throw the > entire filename away and make a filename locally, without using any part > of the original name. Oh, and I wish you could convince some other parts of the world about this. When you mix file uploads with Apache+PHP web applications, you basically get remote code execution right there. As sysadmin, I have to constantly play whack-a-mole with stupid exploits that just wouldn't happen if Joomla's web uploads followed this model. Seriously, how hard is it for something that *already has a database* to simply insert a row into jos_uploaded_files and then use that row's ID eg "uploads/file_"+id as the file name? In one stroke, you eliminate code execution (it can't be ".php" or any variant thereof), collisions (two uploads with the same file name will get different IDs), and even detritus from incomplete transactions (if you find "uploads/file_12345" but there's no row with ID 12345, you can assume the transaction got rolled back, and safely delete the file). S'not that hard, folks. Be safe. Be smart. ChrisA From eryksun at gmail.com Mon Aug 22 11:54:46 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 15:54:46 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Mon, Aug 22, 2016 at 1:17 PM, Chris Angelico wrote: > I tried things like "con.txt" and it simply failed (no such file or > directory), without printing anything to the console. Are you using IDLE or some other IDE that uses pythonw.exe instead of python.exe? If so, first use ctypes to allocate a console: import ctypes ctypes.WinDLL('kernel32').AllocConsole() The CON device should work if the process is attached to a console (i.e. a conhost.exe instance). You can detect any of the classic DOS devices via isatty(), since they're all character devices. Below I'm using Windows 10, so the console uses a kernel device (condrv.sys was added in Windows 8), for which the name can be queried: >>> f = open('con.txt') >>> h = msvcrt.get_osfhandle(f.fileno()) >>> devname = UNICODE_STRING() >>> ctypes.resize(devname, sizeof(devname) + 32767*2) >>> ntdll.NtQueryObject(h, 1, byref(devname), sizeof(devname)) 0 >>> print(devname.Buffer) \Device\ConDrv > I wouldn't accept file names from untrusted sources on *any* system There are still desktop applications that ask users to name their files. From random832 at fastmail.com Mon Aug 22 11:56:49 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 11:56:49 -0400 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> Message-ID: <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 11:40, Chris Angelico wrote: > Windows has some other issues, including that arbitrary files can > become executable very easily (eg if %PATHEXT% includes its file > extension), and since the current directory is always at the beginning > of your path, this can easily turn into a remote code execution > exploit. I didn't include dot in my example whitelist, and there's no mechanism for an attacker to add random extensions to your PATHEXT. > And any GUI that automatically calculates thumbnails from > image files (this includes Windows, Mac OS, and more than one Linux > window manager) could potentially be attacked via a malformed file, > simply by having it appear on the file system. This has nothing to do with the filename, unless you additionally assume that this will only happen if the file is called .jpg From marko at pacujo.net Mon Aug 22 12:08:35 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 22 Aug 2016 19:08:35 +0300 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> Message-ID: <87oa4k7pmk.fsf@elektro.pacujo.net> Random832 : > On Mon, Aug 22, 2016, at 11:40, Chris Angelico wrote: >> Windows has some other issues, including that arbitrary files can >> become executable very easily (eg if %PATHEXT% includes its file >> extension), and since the current directory is always at the >> beginning of your path, this can easily turn into a remote code >> execution exploit. > > I didn't include dot in my example whitelist, and there's no mechanism > for an attacker to add random extensions to your PATHEXT. Years back, my FTP server was hacked by exploiting a buffer overflow. The anonymous input directory contained a very long filename that apparently contained some valid x86 code. Did you vet your whitelist so it couldn't possibly be interpreted by the CPU as meaningful instructions? Marko From rosuav at gmail.com Mon Aug 22 12:18:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 02:18:32 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Tue, Aug 23, 2016 at 1:54 AM, eryk sun wrote: > On Mon, Aug 22, 2016 at 1:17 PM, Chris Angelico wrote: >> I tried things like "con.txt" and it simply failed (no such file or >> directory), without printing anything to the console. > > Are you using IDLE or some other IDE that uses pythonw.exe instead of > python.exe? If so, first use ctypes to allocate a console: > > import ctypes > ctypes.WinDLL('kernel32').AllocConsole() > > The CON device should work if the process is attached to a console > (i.e. a conhost.exe instance). No, I used Pike (to avoid any specifically-Python issues or protections) running in a console. Attempting to write to "Logs/con" wrote to the console, so I know the console device is active. Attempting to write to "Logs/con.txt" failed as described. > > I wouldn't accept file names from untrusted sources on *any* system > > There are still desktop applications that ask users to name their files. A person running a desktop application is generally considered a trusted source. In a kiosk environment, you have a lot more to worry about than special device names (eg someone could overwrite a key file), so again, allowing an untrusted user to name a file in that situation would be inappropriate. If the user owns the computer, s/he should be allowed to attempt any name, and there'd simply be some that fail - same as any other invalid characters (Windows won't let you put a colon or question mark in a file name, for instance, which annoys my brother no end when I give him files like "What's Up, Doc?.mkv" or "Operation: Rabbit.mkv") or over-long names or anything like that. ChrisA From rosuav at gmail.com Mon Aug 22 12:20:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 02:20:16 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> Message-ID: On Tue, Aug 23, 2016 at 1:56 AM, Random832 wrote: >> And any GUI that automatically calculates thumbnails from >> image files (this includes Windows, Mac OS, and more than one Linux >> window manager) could potentially be attacked via a malformed file, >> simply by having it appear on the file system. > > This has nothing to do with the filename, unless you additionally assume > that this will only happen if the file is called .jpg It generally will (or rather, only if the file has one of a particular set of extensions). Automatic thumbnailing is usually done only for certain file names. I don't know of anything that opens every single file to see if it has a JFIF signature (etc for PNG and whatever other types). ChrisA From eryksun at gmail.com Mon Aug 22 12:25:30 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 16:25:30 +0000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> Message-ID: On Mon, Aug 22, 2016 at 3:40 PM, Chris Angelico wrote: > Windows has some other issues, including that arbitrary files can > become executable very easily (eg if %PATHEXT% includes its file > extension), cmd uses PATHEXT to augment its search by appending each extension in the list, in addition to searching for the exact filename. cmd will always attempt to run any match, regardless of the extension. You must be thinking of PowerShell, which for some reason reinterprets how this environment variable has worked since probably OS/2 in the late 80s. PowerShell only executes files found in PATH that have an extension that's listed in PATHEXT. CreateProcess checks the user's execute access in the file security, which can prevent the execution of .BAT/.CMD files and PE executables, regardless of extension. But ShellExecute(Ex) has an MS-DOS brain (so much of the entire Explorer/shell32 implementation has an MS-DOS brain; it's like they think they're still supporting Windows 9x), so scripts and data files are always 'executable'. You get some help here from cmd, which always tries CreateProcess, regardless of extension, and won't continue to ShellExecuteEx if CreateProcess failed because access was denied. PowerShell... not so much. > and since the current directory is always at the beginning > of your path, this can easily turn into a remote code execution > exploit. Since Vista, both CreateProcess and cmd.exe support the environment variable NoDefaultCurrentDirectoryInExePath. If this is set, you have to explicitly reference the current directory. PowerShell always required this. From rosuav at gmail.com Mon Aug 22 12:25:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 02:25:32 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman In-Reply-To: <87oa4k7pmk.fsf@elektro.pacujo.net> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> <87oa4k7pmk.fsf@elektro.pacujo.net> Message-ID: On Tue, Aug 23, 2016 at 2:08 AM, Marko Rauhamaa wrote: > Random832 : > >> On Mon, Aug 22, 2016, at 11:40, Chris Angelico wrote: >>> Windows has some other issues, including that arbitrary files can >>> become executable very easily (eg if %PATHEXT% includes its file >>> extension), and since the current directory is always at the >>> beginning of your path, this can easily turn into a remote code >>> execution exploit. >> >> I didn't include dot in my example whitelist, and there's no mechanism >> for an attacker to add random extensions to your PATHEXT. > > Years back, my FTP server was hacked by exploiting a buffer overflow. > The anonymous input directory contained a very long filename that > apparently contained some valid x86 code. > > Did you vet your whitelist so it couldn't possibly be interpreted by the > CPU as meaningful instructions? Step 1: Don't have buffers. Step 2: Profit! Anyone who's using fixed-sized buffers for application-level code deserves to be exploited. A program designed to be accessed via the internet is never (well, hardly ever) going to need so much performance that it can't afford to be written in a high level language - it's going to spend most of its time waiting for the network. The rare exceptions (*maybe* DNS, but even there, I'd be quite happy to replace my DNS server with one written in Pike, if BIND9 ever becomes a major threat vector) should be monitored closely - preferably statically checked with something like Coverity - because they're remotely-accessible and thus a major risk. ChrisA From larry.martell at gmail.com Mon Aug 22 12:50:34 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 12:50:34 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Mon, Aug 22, 2016 at 10:36 AM, Larry Martell wrote: > On Fri, Aug 19, 2016 at 4:24 PM, Chris Kaynor wrote: >> On Fri, Aug 19, 2016 at 12:00 PM, Larry Martell >> wrote: >> >>> On Fri, Aug 19, 2016 at 1:24 PM, Chris Angelico wrote: >>> > On Sat, Aug 20, 2016 at 3:10 AM, Larry Martell >>> wrote: >>> >> I have some python code (part of a django app) that processes a >>> >> request that contains a png file. The request is send with >>> >> content_type = 'application/octet-stream' >>> >> >>> >> In the python code I want to write this data to a file and still have >>> >> it still be a valid png file. >>> >> >>> >> The data I get looks like this: >>> >> >>> >> u'\ufffdPNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\ufffd\ >>> x00\x00\x01\ufffd >>> >> ......' >>> >> >>> >> If I try and write that to a file it fails with a UnicodeEncodeError. >>> >> If I write it with encode('utf8') it writes the file, but then it's no >>> >> longer a valid png file. >>> >> >>> >> Anyone know how I can do this? >>> > >>> > At that point, you've already lost information. Each U+FFFD (shown as >>> > "\ufffd" above) is a marker saying "a byte here was not valid UTF-8" >>> > (or whatever was being used). Something somewhere took the .png file's >>> > bytes and tried to interpret them as text, which they're not. >>> > >>> > What sent you that data? How did you receive it? >>> >>> The request is sent by a client app written in C++ with Qt. It's >>> received by a django based server. I am trying to port a falcon server >>> to django. The falcon server code did this: >>> >>> form = cgi.FieldStorage(fp=req.stream, environ=req.env) >>> >>> and then wrote the png like this: >>> >>> fd.write(form[key].file.read()) >>> >>> Whereas in the django server I am doing: >>> >>> fd.write(request.POST[key]) >>> >>> I've never used the cgi module. I guess I can try that. I've written a >>> lot with django but never had to receive a PNG file. >>> >>> >> I don't know Django, however a quick search makes it seem like you might >> need to use request.FILES[key] (1) rather than request.POST[key]. You may >> also be able to use request.POST if you set request.encoding first (2). If >> both of those fail, you may need to use request.body and parse the HTTP >> form data manually, though I'd imagine there is an easier way. >> >> [1] >> https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.FILES >> >> [2] >> https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.HttpRequest.encoding > > Thanks for the reply. When I get the request, request.FILES is empty. > Yet the content type is multipart/form-data and the method is POST: > > (Pdb) print request.META['CONTENT_TYPE'] > multipart/form-data; > boundary="boundary_.oOo._NzEwNjIzMTM4MTI4NjUxOTM5OQ==MTY2NjE4MDk5Nw==" > > (Pdb) print request.META['REQUEST_METHOD'] > POST > > (Pdb) print request.FILES > > > Tried setting request.encoding, but that messes up the request structure: > > (Pdb) type(request.POST[key]) > > (Pdb) request.encoding = "iso-8859-1" > (Pdb) type(request.POST[key]) > *** MultiValueDictKeyError: > "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" Thanks to everyone for they replied. I solved this by changing the client to set the file and filename fields each part of the multipart and then I was able it iterate through request.FILES and successfully write the files as PNG. From eryksun at gmail.com Mon Aug 22 13:13:04 2016 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Aug 2016 17:13:04 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico wrote: > >> The CON device should work if the process is attached to a console >> (i.e. a conhost.exe instance). > > No, I used Pike (to avoid any specifically-Python issues or > protections) running in a console. Attempting to write to "Logs/con" > wrote to the console, so I know the console device is active. > Attempting to write to "Logs/con.txt" failed as described. What version of Windows is this? If it's Windows 7 I'll have to check that later. If "Logs" is an existing directory, then both "Logs/con" and "Logs/con.txt" should refer to the console. If "Logs" doesn't exist, then both should fail. Virtual DOS devices only exist in existing directories. >> > I wouldn't accept file names from untrusted sources on *any* system >> >> There are still desktop applications that ask users to name their files. > If the user owns the computer, s/he should be allowed to attempt any > name, and there'd simply be some that fail - same as any other invalid > characters Silently writing a file to a (possibly hidden) console or NUL device is not the same thing as failing with an error. If users explicitly open "NUL" or "\\.\NUL", I can detect that, and I have no problem with it (with some reservations about the former). But if they open files like "C:\Users\JoeUser\Documents\Nul.20160822.doc", I want to make sure they know that they just asked to save to "\\.\NUL". It's not a common problem. I just find the system's behavior abhorrent. I'd like to have a manifest setting that opts the process out of this stupid DOS legacy behavior. > Windows won't let you put a colon or question mark in a file name, for instance, > which annoys my brother no end when I give him files like > "What's Up, Doc?.mkv" or "Operation: Rabbit.mkv") A colon is reserved for NTFS streams, so it can silently bite you. If you get an error for "Operation: Rabbit.mkv", you're probably using FAT32. On NTFS that creates a file named "Operation" with a $DATA stream named " Rabbit.mkv". Every NTFS file has at least the anonymous stream, e.g. "filename::$DATA". Every NTFS directory has at least the $I30 stream, e.g. "dirname:$I30:$INDEX_ALLOCATION". Directories can also have named $DATA streams. As to question mark and other wildcard characters, those are all unambiguously handled as exceptions. No Microsoft filesystem allows them because wildcard matching is baked into the system, such as the FileName parameter of NtQueryDirectoryFile when listing the contents of a directory. I guess no one on the NT I/O team wanted to bother with the possibility of escaping these wildcards since they weren't allowed in DOS filenames. FsRtlIsNameInExpression [1] is the kernel-mode filesystem function that implements wildcard matching. Note that DOS wildcard semantics are implemented by DOS_STAR (<), DOS_QM (>), and DOS_DOT ("). FindFirstFile has to convert DOS wildcard patterns to NT semantics before calling NtQueryDirectoryFile. It's not as simple as substituting the corresponding DOS_ wildcard. Pipe and ASCII control characters are also reserved, except not in NTFS stream names. Reserving pipe is just a DOS legacy. I don't think it's used for anything internally. > or over-long names or anything like that. Good news. In Windows 10, systems and applications can opt in to using long paths with up to about 32760 characters. Python 3.6 will be opting in. [1]: https://msdn.microsoft.com/en-us/library/ff546850 From larry.martell at gmail.com Mon Aug 22 13:21:43 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 13:21:43 -0400 Subject: saving octet-stream png file In-Reply-To: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> Message-ID: On Fri, Aug 19, 2016 at 4:51 PM, Lawrence D?Oliveiro wrote: > On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: >> >> An 'octet' is a byte of 8 bits. > > Is there any other size of byte? Many, many years ago, probably c. 1982 my Dad came into my house and saw a Byte Magazine laying on the coffee table. He asked "What is a byte?" I replied "Half a word." He then asked "What is the other half of the word?" I said "That is also a byte." He thought for a moment, then said "So the full word is 'byte byte'?" From rosuav at gmail.com Mon Aug 22 13:24:21 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 03:24:21 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Tue, Aug 23, 2016 at 3:13 AM, eryk sun wrote: > On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico wrote: >> >>> The CON device should work if the process is attached to a console >>> (i.e. a conhost.exe instance). >> >> No, I used Pike (to avoid any specifically-Python issues or >> protections) running in a console. Attempting to write to "Logs/con" >> wrote to the console, so I know the console device is active. >> Attempting to write to "Logs/con.txt" failed as described. > > What version of Windows is this? If it's Windows 7 I'll have to check > that later. If "Logs" is an existing directory, then both "Logs/con" > and "Logs/con.txt" should refer to the console. If "Logs" doesn't > exist, then both should fail. Virtual DOS devices only exist in > existing directories. Yes, it was Windows 7 (running in a VM under Debian Jessie, though I doubt that makes any difference). The Logs directory did exist (that's why I used that otherwise-odd choice of name). >> Windows won't let you put a colon or question mark in a file name, for instance, >> which annoys my brother no end when I give him files like >> "What's Up, Doc?.mkv" or "Operation: Rabbit.mkv") > > A colon is reserved for NTFS streams, so it can silently bite you. If > you get an error for "Operation: Rabbit.mkv", you're probably using > FAT32. On NTFS that creates a file named "Operation" with a $DATA > stream named " Rabbit.mkv". I think it was FAT32 that we were using - it was a USB stick for sharing files between my Linux systems and his Windows. Linux was quite happy to put files into that FS that Windows refused to touch. (Also, due to some renamings, I ended up giving him two copies of a file that differed only in case - something like "Name The File Something.mkv" and "Name the File Something.mkv" - and Windows took issue with that too.) > As to question mark and other wildcard characters, those are all > unambiguously handled as exceptions. No Microsoft filesystem allows > them because wildcard matching is baked into the system, such as the > FileName parameter of NtQueryDirectoryFile when listing the contents > of a directory. I guess no one on the NT I/O team wanted to bother > with the possibility of escaping these wildcards since they weren't > allowed in DOS filenames. Yeah. They're nice straight-forward errors, as long as you're running on Windows. Detecting that on POSIX platforms and avoiding those names is why we have PureWindowsPath. If it doesn't currently have a quick little "is_valid", it could possibly benefit from one - PurePosixPath would simply "return '\0' not in name", while PureWindowsPath can do the more extensive checks with wildcard characters and stuff. >> or over-long names or anything like that. > > Good news. In Windows 10, systems and applications can opt in to using > long paths with up to about 32760 characters. Python 3.6 will be > opting in. > > [1]: https://msdn.microsoft.com/en-us/library/ff546850 Great. So just as soon as all previous Windowses die....... Remind me how many people are still using Python 3.4 because they're still on Windows XP? ChrisA From jon+usenet at unequivocal.eu Mon Aug 22 13:25:17 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 17:25:17 -0000 (UTC) Subject: saving octet-stream png file References: Message-ID: On 2016-08-22, Larry Martell wrote: > On Sun, Aug 21, 2016 at 5:24 PM, Jon Ribbens wrote: >> On 2016-08-19, Larry Martell wrote: >>> fd.write(request.POST[key]) >> >> You could try: >> >> request.encoding = "iso-8859-1" >> fd.write(request.POST[key].encode("iso-8859-1")) >> >> It's hacky and nasty and there might be a better "official" method >> but I think it should work. > > For some reason that messes up the request structure: > > (Pdb) type(request.POST[key]) > > (Pdb) request.encoding = "iso-8859-1" > (Pdb) type(request.POST[key]) > *** MultiValueDictKeyError: > "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" Sounds like you should be filing a bug report with Django. From jon+usenet at unequivocal.eu Mon Aug 22 13:27:13 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 17:27:13 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On 2016-08-22, Chris Angelico wrote: > I tried things like "con.txt" and it simply failed (no such file or > directory), without printing anything to the console. I'm not sure how you got that to fail, but writing to "con.txt" certainly does write to the console in Windows 10 - I just tried it: C:\>echo hello >con.txt hello From python.list at tim.thechases.com Mon Aug 22 13:32:06 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Mon, 22 Aug 2016 12:32:06 -0500 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> Message-ID: <20160822123206.48c3d822@bigbox.christie.dr> On 2016-08-23 02:20, Chris Angelico wrote: > It generally will (or rather, only if the file has one of a > particular set of extensions). Automatic thumbnailing is usually > done only for certain file names. I don't know of anything that > opens every single file to see if it has a JFIF signature (etc for > PNG and whatever other types). How about a web server that opens arbitrary files. Compare any of https://technet.microsoft.com/en-us/library/nonexistent.aspx https://technet.microsoft.com/en-us/library/doesnotexist.aspx https://technet.microsoft.com/en-us/library/asdf.aspx vs https://technet.microsoft.com/en-us/library/con.aspx https://technet.microsoft.com/en-us/library/lpt1.aspx https://technet.microsoft.com/en-us/library/com1.aspx https://technet.microsoft.com/en-us/library/nul.aspx This is FREAKING MICROSOFT and it breaks things. It's not like anybody would open arbitrarily-named files... -tkc From rosuav at gmail.com Mon Aug 22 13:33:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 03:33:50 +1000 Subject: The dangerous, exquisite art of safely handing user-uploaded files: Tom Eastman (was: Does This Scare You?) In-Reply-To: <20160822123206.48c3d822@bigbox.christie.dr> References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <85wpj8x4tj.fsf_-_@benfinney.id.au> <1471876487.4029395.702461505.1F31B776@webmail.messagingengine.com> <1471881409.4048414.702558737.20913C4D@webmail.messagingengine.com> <20160822123206.48c3d822@bigbox.christie.dr> Message-ID: On Tue, Aug 23, 2016 at 3:32 AM, Tim Chase wrote: > On 2016-08-23 02:20, Chris Angelico wrote: >> It generally will (or rather, only if the file has one of a >> particular set of extensions). Automatic thumbnailing is usually >> done only for certain file names. I don't know of anything that >> opens every single file to see if it has a JFIF signature (etc for >> PNG and whatever other types). > > How about a web server that opens arbitrary files. Compare any of > > https://technet.microsoft.com/en-us/library/nonexistent.aspx > https://technet.microsoft.com/en-us/library/doesnotexist.aspx > https://technet.microsoft.com/en-us/library/asdf.aspx > > vs > > https://technet.microsoft.com/en-us/library/con.aspx > https://technet.microsoft.com/en-us/library/lpt1.aspx > https://technet.microsoft.com/en-us/library/com1.aspx > https://technet.microsoft.com/en-us/library/nul.aspx > > This is FREAKING MICROSOFT and it breaks things. It's not like > anybody would open arbitrarily-named files... Oh, brilliant. Brilliant brilliant brilliant. ChrisA From jon+usenet at unequivocal.eu Mon Aug 22 13:34:31 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 17:34:31 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <57bafb06$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-22, Steve D'Aprano wrote: > On Mon, 22 Aug 2016 09:50 pm, Jon Ribbens wrote: >> I don't know what purpose you are envisaging this function being used >> for, but the only one I can think of is input sanitisation. e.g. a web >> form where you receive a file from the Internet and store it somewhere, >> and you want to use the filename given to you rather than choose your >> own randomly-generated one. >> >> Under Unix all you need to do is check for the filename starting with >> "." or containing "/." (or "/", depending on your requirements). >> Under Windows you would use this function, which apparently doesn't >> work, hence: security hole. > > That's backwards: it works under Windows, but is buggy under non-Windows. I'm not sure what you're getting at there, but you appear to be wrong: C:\>python Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:38:48) [MSC v.1900 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pathlib >>> pathlib.WindowsPath("con .txt").is_reserved() False > For starters, even on Unix I probably wouldn't want to > use "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\.txt" as a > file name, even if it is technically legal. That might be your personal preference, but it's not a security hole per se. > No, not at all. I can't assume that just because a file name is legal that I > can write to it. Even after sanitising the name (or deciding I don't need > to bother) I still need to be prepared to catch I/O errors when writing the > file -- including attempts to write to reserved file names. As mentioned by others, this is nothing to do with lack of error-checking. > Technically that could be called a DoS attack, but the same could be said > for ANY bug that raises an exception. I think that it takes more than that > to be a security bug. It doesn't raise an exception. It interacts with the hardware of the server. From steve+python at pearwood.info Mon Aug 22 13:35:27 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 23 Aug 2016 03:35:27 +1000 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: <57bb37e1$0$1591$c3e8da3$5496439d@news.astraweb.com> On Tue, 23 Aug 2016 03:13 am, eryk sun wrote: > But if they open files > like "C:\Users\JoeUser\Documents\Nul.20160822.doc", I want to make > sure they know that they just asked to save to "\\.\NUL". It's not a > common problem. I just find the system's behavior abhorrent. I'd like > to have a manifest setting that opts the process out of this stupid > DOS legacy behavior. Aren't we getting further and further away from the original topic? I don't think this is about Window's support for legacy behaviour from ancient DOS days. Yes, it would be nice if Windows dropped support for these horrid magic file names, but that's nothing to do with Python. I'm not really sure what the question is -- we've established that there's a bug in the non-Windows implementation that tries to emulate Window's behaviour. What else is there to argue about? - Does anyone wish to argue that Python shouldn't provide PureWindowsPath.is_reserved on non-Windows systems? For what reason? - Is anyone still arguing that there's a new security vulnerability here because of the pathlib functions? If so, how do you see this attack working? (Existing filename-based attacks are not new.) I don't see what the issue is. Eryksun found a bug in pathlib, well done. (I mean that, I'm not being sarcastic.) I still don't understand why Lawrence posed his question in the first place. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gsublett46 at gmail.com Mon Aug 22 13:52:20 2016 From: gsublett46 at gmail.com (Gary Sublett) Date: Mon, 22 Aug 2016 13:52:20 -0400 Subject: DED processing Message-ID: <20160822135220.4aaaa066249d2c694e0afcd6@gmail.com> I have to go out for a while, so for DED processing two options from my end: 1. Process as you all have been in the past for now. If you all do this, the records that have not been mailed prior to the latest list are contained in a MailManage Job name DED_master. If you chose to process as in the past, these records should be exported to a spreadsheet and added to whatever new file you have for today. 2. I can come in tomorrow morning at about 9:00 and walk Greg thru the way I process the DED job. Let me know. From larry.martell at gmail.com Mon Aug 22 13:58:09 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 13:58:09 -0400 Subject: DED processing In-Reply-To: <20160822135220.4aaaa066249d2c694e0afcd6@gmail.com> References: <20160822135220.4aaaa066249d2c694e0afcd6@gmail.com> Message-ID: On Mon, Aug 22, 2016 at 1:52 PM, Gary Sublett wrote: > I have to go out for a while, so for DED processing two options from > my end: > > 1. Process as you all have been in the past for now. If you all do > this, the records that have not been mailed prior to the latest list > are contained in a MailManage Job name DED_master. If you chose to > process as in the past, these records should be exported to a > spreadsheet and added to whatever new file you have for today. > > 2. I can come in tomorrow morning at about 9:00 and walk Greg thru > the way I process the DED job. > > Let me know. I think you should let Greg know. What if you get hit by a bus, or win the lottery? From jon+usenet at unequivocal.eu Mon Aug 22 14:12:36 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 18:12:36 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> <57bb37e1$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-22, Steve D'Aprano wrote: > I'm not really sure what the question is -- we've established that there's a > bug in the non-Windows implementation that tries to emulate Window's > behaviour. What else is there to argue about? It doesn't seem to be "the non-Windows implementation", it seems to be "the implementation". > - Does anyone wish to argue that Python shouldn't provide > PureWindowsPath.is_reserved on non-Windows systems? For what reason? It shouldn't provide it at all unless it works. The reason I'm putting it that way is that making it work may be a very great deal of effort (What is the actual full list of special filenames? Does it vary between supported Windows versions? How can anyone tell test that the function is correct?) > - Is anyone still arguing that there's a new security vulnerability > here because of the pathlib functions? If so, how do you see this > attack working? (Existing filename-based attacks are not new.) Already answered. > I don't see what the issue is. Eryksun found a bug in pathlib, well done. (I > mean that, I'm not being sarcastic.) I still don't understand why Lawrence > posed his question in the first place. Presumably because of the security implications as described. From rosuav at gmail.com Mon Aug 22 14:47:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 04:47:12 +1000 Subject: DED processing In-Reply-To: References: <20160822135220.4aaaa066249d2c694e0afcd6@gmail.com> Message-ID: On Tue, Aug 23, 2016 at 3:58 AM, Larry Martell wrote: > On Mon, Aug 22, 2016 at 1:52 PM, Gary Sublett wrote: >> I have to go out for a while, so for DED processing two options from >> my end: >> >> 1. Process as you all have been in the past for now. If you all do >> this, the records that have not been mailed prior to the latest list >> are contained in a MailManage Job name DED_master. If you chose to >> process as in the past, these records should be exported to a >> spreadsheet and added to whatever new file you have for today. >> >> 2. I can come in tomorrow morning at about 9:00 and walk Greg thru >> the way I process the DED job. >> >> Let me know. > > I think you should let Greg know. What if you get hit by a bus, or win > the lottery? DED Processing Disclaimer: This e-mail (and files transmitted with it) is intended only for the person or entity to which it is addressed. If you are not the intended recipient, then you have received this e-mail by mistake and any use, dissemination, forwarding, printing or copying of this e-mail and its file attachments is prohibited; please print this email out on minimum 100 GSM paper, cross-cut shred it, burn the remains, and send the ashes back to the sender. Failure to do so will result in the internet police being informed of your IP address, street address, ICBM address, and t-shirt size. ChrisA From saeed.fazayeli at gmail.com Mon Aug 22 15:15:03 2016 From: saeed.fazayeli at gmail.com (saeed.fazayeli at gmail.com) Date: Mon, 22 Aug 2016 12:15:03 -0700 (PDT) Subject: ANN: pygene - genetic algorithms package In-Reply-To: References: Message-ID: <67c2f9fd-deac-40bc-a5f8-f275fdb41ac3@googlegroups.com> On Tuesday, December 6, 2005 at 11:10:56 AM UTC+3:30, aum wrote: > Hi all, > > I looked at a few genetic algorithms/genetic programming packages for > Python, and found them somewhat convoluted, complicated and > counter-intuitive to use. > > So I've written a genetic algorithms package which I hope will be more > approachable to beginners. > > The first release of pygene is up at: > http://www.freenet.org.nz/python/pygene > > The package includes full api documentation, and an implementation of > the travelling salesman problem, plus a couple of simpler cases. > > -- > > Cheers > aum From best_lay at yahoo.com Mon Aug 22 15:35:08 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 22 Aug 2016 14:35:08 -0500 Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Mon, 22 Aug 2016 17:27:13 +0000, Jon Ribbens wrote: > On 2016-08-22, Chris Angelico wrote: >> I tried things like "con.txt" and it simply failed (no such file or >> directory), without printing anything to the console. > > I'm not sure how you got that to fail, but writing to "con.txt" > certainly does write to the console in Windows 10 - I just tried it: > > C:\>echo hello >con.txt > hello I got the same result with Windows 7, Vista, XP, 2000, NT4, 98SE and 3.11. Yes, I have a lot VM's although I don't have 8.x. I would, however, expect the same result. On Linux a file is created named con.txt that contains hello/n as expected. -- GNU/Linux user #557453 "The Constitution only gives people the right to pursue happiness. You have to catch it yourself." -Benjamin Franklin From best_lay at yahoo.com Mon Aug 22 15:45:37 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 22 Aug 2016 14:45:37 -0500 Subject: saving octet-stream png file References: <8d9c3b68-d8a9-47a8-857a-da3110d8c61f@googlegroups.com> Message-ID: On Mon, 22 Aug 2016 13:21:43 -0400, Larry Martell wrote: > On Fri, Aug 19, 2016 at 4:51 PM, Lawrence D?Oliveiro > wrote: >> On Saturday, August 20, 2016 at 6:03:53 AM UTC+12, Terry Reedy wrote: >>> >>> An 'octet' is a byte of 8 bits. >> >> Is there any other size of byte? > > Many, many years ago, probably c. 1982 my Dad came into my house and > saw a Byte Magazine laying on the coffee table. He asked "What is a > byte?" I replied "Half a word." He then asked "What is the other half > of the word?" I said "That is also a byte." He thought for a moment, > then said "So the full word is 'byte byte'?" LOL! Did you explain to him that a full word could also be 'nibble nibble nibble nibble' or 'bit bit bit bit bit bit bit bit bit bit bit bit bit bit bit bit'? -- GNU/Linux user #557453 The cow died so I don't need your bull! From python at bdurham.com Mon Aug 22 16:21:51 2016 From: python at bdurham.com (Malcolm Greene) Date: Mon, 22 Aug 2016 16:21:51 -0400 Subject: Dynamically import specific names from a module vs importing full module Message-ID: <1471897311.3337992.702831593.6055EFBA@webmail.messagingengine.com> Python 3.5: Is there a way to dynamically import specific names from a module vs importing the full module? By dynamic I mean via some form of importlib machinery, eg. I'm looking for the dynamic "from import " equivalent of "import "'s importlib.import_module. Thank you, Malcolm From ned at nedbatchelder.com Mon Aug 22 16:50:55 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 22 Aug 2016 13:50:55 -0700 (PDT) Subject: Dynamically import specific names from a module vs importing full module In-Reply-To: References: <1471897311.3337992.702831593.6055EFBA@webmail.messagingengine.com> Message-ID: On Monday, August 22, 2016 at 4:22:09 PM UTC-4, Malcolm Greene wrote: > Python 3.5: Is there a way to dynamically import specific names from a > module vs importing the full module? > > By dynamic I mean via some form of importlib machinery, eg. I'm looking > for the dynamic "from import " equivalent of "import > "'s importlib.import_module. > > Thank you, > Malcolm You can use: the_mod = importlib.import_module("module") a = the_mod.a b = the_mod.b # or a = getattr(the_mod, 'a') b = getattr(the_mod, 'b') --Ned. From random832 at fastmail.com Mon Aug 22 16:59:46 2016 From: random832 at fastmail.com (Random832) Date: Mon, 22 Aug 2016 16:59:46 -0400 Subject: Dynamically import specific names from a module vs importing full module In-Reply-To: <1471897311.3337992.702831593.6055EFBA@webmail.messagingengine.com> References: <1471897311.3337992.702831593.6055EFBA@webmail.messagingengine.com> Message-ID: <1471899586.838257.702879865.125152BC@webmail.messagingengine.com> On Mon, Aug 22, 2016, at 16:21, Malcolm Greene wrote: > Python 3.5: Is there a way to dynamically import specific names from a > module vs importing the full module? > > By dynamic I mean via some form of importlib machinery, eg. I'm looking > for the dynamic "from import " equivalent of "import > "'s importlib.import_module. You do know that "from import " still loads the whole module into sys.modules, right? And you're free to assign the result from importlib.import_module to a local variable and delete it once you've gotten the names you want out of it. From larry.martell at gmail.com Mon Aug 22 17:23:40 2016 From: larry.martell at gmail.com (Larry Martell) Date: Mon, 22 Aug 2016 17:23:40 -0400 Subject: saving octet-stream png file In-Reply-To: References: Message-ID: On Mon, Aug 22, 2016 at 1:25 PM, Jon Ribbens wrote: > On 2016-08-22, Larry Martell wrote: >> On Sun, Aug 21, 2016 at 5:24 PM, Jon Ribbens wrote: >>> On 2016-08-19, Larry Martell wrote: >>>> fd.write(request.POST[key]) >>> >>> You could try: >>> >>> request.encoding = "iso-8859-1" >>> fd.write(request.POST[key].encode("iso-8859-1")) >>> >>> It's hacky and nasty and there might be a better "official" method >>> but I think it should work. >> >> For some reason that messes up the request structure: >> >> (Pdb) type(request.POST[key]) >> >> (Pdb) request.encoding = "iso-8859-1" >> (Pdb) type(request.POST[key]) >> *** MultiValueDictKeyError: >> "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" > > Sounds like you should be filing a bug report with Django. Turns out that is expected behavior -- request.encoding clears existing GET/POST data. Not sure how it's useful to set that then. From jon+usenet at unequivocal.eu Mon Aug 22 17:28:56 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 22 Aug 2016 21:28:56 -0000 (UTC) Subject: saving octet-stream png file References: Message-ID: On 2016-08-22, Larry Martell wrote: > On Mon, Aug 22, 2016 at 1:25 PM, Jon Ribbens wrote: >> On 2016-08-22, Larry Martell wrote: >>> (Pdb) type(request.POST[key]) >>> >>> (Pdb) request.encoding = "iso-8859-1" >>> (Pdb) type(request.POST[key]) >>> *** MultiValueDictKeyError: >>> "u'right-carotidartery:63B2E474-D690-445F-B92A-31EBADDC9D93.png'" >> >> Sounds like you should be filing a bug report with Django. > > Turns out that is expected behavior -- request.encoding clears > existing GET/POST data. Not sure how it's useful to set that then. Yeah it clears the cached parsed data, but when you fetch a value afterwards it should automatically recalculate the data, not fail. From ethan at stoneleaf.us Mon Aug 22 20:12:36 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 22 Aug 2016 17:12:36 -0700 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> Message-ID: <57BB94F4.7020005@stoneleaf.us> On 08/19/2016 06:11 PM, Wildman via Python-list wrote: > Since I am fairly new to Python, I realize there is much that I > still don't know but I don't understand how Windows can have > reserved names on a Linux system. What am I missing? A PureWindowsPath (and PurePosixPath and PurePath) is a theoretical -- it's still, basically, a string and hasn't been checked against the file system to see if it actually exists (or to create it, etc.). A WindowsPath (PosixPath) does actually have a tie-in to the actual file system, and can only be instantiated on Windows (Posix) systems. -- ~Ethan~ From adam.j.kerr at gmail.com Mon Aug 22 20:28:25 2016 From: adam.j.kerr at gmail.com (adam.j.kerr at gmail.com) Date: Mon, 22 Aug 2016 17:28:25 -0700 (PDT) Subject: Raw Data from Website Message-ID: <85c71feb-2f2c-4420-a9de-5057e5ecbded@googlegroups.com> Hi, I am hoping someone is able to help me. Is there a way to pull as much raw data from a website as possible. The webpage that I am looking for is as follows: http://www.homepriceguide.com.au/Research/ResearchSeeFullList.aspx?LocationType=LGA&State=QLD&LgaID=632 The main variable that is important is the "632" at the end, by adjusting this it changes the postcodes. Each postcode contains a large amount of data. Is there a way this all able to be exported into an excel document? Any help with this would be amazing. Thank you. Adam From steve+comp.lang.python at pearwood.info Tue Aug 23 03:14:39 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 23 Aug 2016 17:14:39 +1000 Subject: Raw Data from Website References: <85c71feb-2f2c-4420-a9de-5057e5ecbded@googlegroups.com> Message-ID: <57bbf7e1$0$2900$c3e8da3$76491128@news.astraweb.com> On Tuesday 23 August 2016 10:28, adam.j.kerr at gmail.com wrote: > Hi, > > I am hoping someone is able to help me. > > Is there a way to pull as much raw data from a website as possible. The > webpage that I am looking for is as follows: > http://www.homepriceguide.com.au/Research/ResearchSeeFullList.aspx?LocationType=LGA&State=QLD&LgaID=632 > > The main variable that is important is the "632" at the end, by adjusting > this it changes the postcodes. Each postcode contains a large amount of data. > Is there a way this all able to be exported into an excel document? Ideally, the web site itself will offer an Excel download option. If it doesn't, you may be able to screen-scrape the data yourself, but: (1) it may be against the terms of service of the website; (2) it may be considered unethical or possibly even copyright infringement or (worst case) even illegal; (3) especially if you're thinking of selling the data; (4) at the very least, unless you take care not to abuse the service, it may be rude and the website may even block your access. There are many tutorials and examples of "screen scraping" or "web scraping" on the internet -- try reading them. It's not something I personally have any experience with, but I expect that the process goes something like this: - connect to the website; - download the particular page you want; - grab the data that you care about; - remove HTML tags and extract just the bits needed; - write them to a CSV file. You may find the Beautiful Soup third-party library helpful for this. -- Steve From rosuav at gmail.com Tue Aug 23 03:23:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 17:23:43 +1000 Subject: Raw Data from Website In-Reply-To: <57bbf7e1$0$2900$c3e8da3$76491128@news.astraweb.com> References: <85c71feb-2f2c-4420-a9de-5057e5ecbded@googlegroups.com> <57bbf7e1$0$2900$c3e8da3$76491128@news.astraweb.com> Message-ID: On Tue, Aug 23, 2016 at 5:14 PM, Steven D'Aprano wrote: > There are many tutorials and examples of "screen scraping" or "web scraping" on > the internet -- try reading them. It's not something I personally have any > experience with, but I expect that the process goes something like this: > > - connect to the website; > - download the particular page you want; > - grab the data that you care about; > - remove HTML tags and extract just the bits needed; > - write them to a CSV file. More or less. It's usually more like this: - import requests and grab the data, nice and easy - extract some of the info you need - run into difficulties - scream in frustration at the stupid inconsistencies in the original site - mess around with it until your code is nested as deeply as the site's HTML (minimum 30 levels) - decide that 90% of the info is good enough - run the program in production for a month or two, and then discover that something's been changed and now it doesn't work - return to step 1, repeat until you run out of hair to pull out ChrisA From python at bdurham.com Tue Aug 23 07:56:30 2016 From: python at bdurham.com (Malcolm Greene) Date: Tue, 23 Aug 2016 07:56:30 -0400 Subject: Dynamically import specific names from a module vs importing full module In-Reply-To: <1471899586.838257.702879865.125152BC@webmail.messagingengine.com> References: <1471897311.3337992.702831593.6055EFBA@webmail.messagingengine.com> <1471899586.838257.702879865.125152BC@webmail.messagingengine.com> Message-ID: <1471953390.936510.703477929.0446F793@webmail.messagingengine.com> Ned and Random832, Ned: Thank you - your example answered my question. Random832: Thank you for the reminder about "from import " still importing the module. Yes, I had forgotten that behavior. Best, Malcolm From dima.olkhov at cell-buddy.com Tue Aug 23 09:08:06 2016 From: dima.olkhov at cell-buddy.com (dimao) Date: Tue, 23 Aug 2016 06:08:06 -0700 (PDT) Subject: Python non blocking multi-client service Message-ID: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> I am trying to implement multi-client service. The service must be able to: connect to the server; send/receive messages; wait until a new message will be received *** Per each client and non blocking My current code takes almost 99% CPU usage. Here is the main part of my code : PORT = 33123 HOST = '127.0.0.1' import asyncio import os @asyncio.coroutine def tcp_echo_client(offset): def send(offset): MSG = """{"ClientId":"%s", % (str(offset))"}""" print("> " + MSG) writer.write((MSG).encode("utf-8")) def recv(): msgback = (yield from reader.readline()).decode("utf-8").rstrip() print("< " + msgback) return msgback reader, writer = yield from asyncio.open_connection(HOST, port=PORT) print(reader) print('Waiting 3 sec for response...') while True: response = yield from asyncio.wait_for(reader.readline(), timeout=5.0) print(response) send(offset) yield from asyncio.sleep(0.5) @asyncio.coroutine def do_work(task_name, work_queue): while not work_queue.empty(): queue_item = yield from work_queue.get() print('{0} grabbed item: {1}'.format(task_name, queue_item)) asyncio.Task(tcp_echo_client(offset=queue_item)) yield from asyncio.sleep(0.1) if __name__ == "__main__": q = asyncio.Queue() for x in range(100): q.put_nowait(x) print(q) loop = asyncio.get_event_loop() tasks = [ asyncio.async(do_work('task1', q)), asyncio.async(do_work('task2', q)), asyncio.async(do_work('task3', q)), asyncio.async(do_work('task4', q)), asyncio.async(do_work('task5', q)), asyncio.async(do_work('task6', q)) ] loop.run_until_complete(asyncio.wait(tasks)) loop.run_forever() loop.close() From rosuav at gmail.com Tue Aug 23 09:19:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 23:19:41 +1000 Subject: Python non blocking multi-client service In-Reply-To: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: On Tue, Aug 23, 2016 at 11:08 PM, dimao wrote: > My current code takes almost 99% CPU usage. Here is the main part of my code : > > > PORT = 33123 > HOST = '127.0.0.1' > > > import asyncio > import os > > @asyncio.coroutine > def tcp_echo_client(offset): > > def send(offset): > MSG = """{"ClientId":"%s", % (str(offset))"}""" > > print("> " + MSG) > writer.write((MSG).encode("utf-8")) > > > def recv(): > msgback = (yield from reader.readline()).decode("utf-8").rstrip() > print("< " + msgback) > return msgback > > reader, writer = yield from asyncio.open_connection(HOST, port=PORT) > > > print(reader) > print('Waiting 3 sec for response...') > > while True: > response = yield from asyncio.wait_for(reader.readline(), timeout=5.0) > print(response) > send(offset) > > yield from asyncio.sleep(0.5) > Can you post your actual code, please? I'm not sure that this code is runnable - at least, not with this indentation. And if I have to fix your indentation to try to figure out what's nested inside what, I can't help with your CPU usage problem. ChrisA From alister.ware at ntlworld.com Tue Aug 23 09:29:39 2016 From: alister.ware at ntlworld.com (alister) Date: Tue, 23 Aug 2016 13:29:39 GMT Subject: libdivecomputer Message-ID: <7dYuz.645263$OB.604309@fx36.am4> Does anyone know if there is a python module avaialbe to interact with this library (it is for downloading data down various Decompression computers so a bit of a specialist market) I already have an application that works fine (sub-surface) so this is more of a curiosity, delving into ctypes is not something I want to try just yet. -- According to Arkansas law, Section 4761, Pope's Digest: "No person shall be permitted under any pretext whatever, to come nearer than fifty feet of any door or window of any polling room, from the opening of the polls until the completion of the count and the certification of the returns." From dima.olkhov at cell-buddy.com Tue Aug 23 09:39:56 2016 From: dima.olkhov at cell-buddy.com (dima.olkhov at cell-buddy.com) Date: Tue, 23 Aug 2016 06:39:56 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: <9234fb72-5875-41ad-920c-4ce97455fa2b@googlegroups.com> On Tuesday, August 23, 2016 at 4:09:07 PM UTC+3, dimao wrote: > I am trying to implement multi-client service. The service must be able to: > > connect to the server; > send/receive messages; > wait until a new message will be received > *** Per each client and non blocking > > My current code takes almost 99% CPU usage. Here is the main part of my code : > > > PORT = 33123 > HOST = '127.0.0.1' > > > import asyncio > import os > > @asyncio.coroutine > def tcp_echo_client(offset): > > def send(offset): > MSG = """{"ClientId":"%s", % (str(offset))"}""" > > print("> " + MSG) > writer.write((MSG).encode("utf-8")) > > > def recv(): > msgback = (yield from reader.readline()).decode("utf-8").rstrip() > print("< " + msgback) > return msgback > > reader, writer = yield from asyncio.open_connection(HOST, port=PORT) > > > print(reader) > print('Waiting 3 sec for response...') > > while True: > response = yield from asyncio.wait_for(reader.readline(), timeout=5.0) > print(response) > send(offset) > > yield from asyncio.sleep(0.5) > > @asyncio.coroutine > def do_work(task_name, work_queue): > while not work_queue.empty(): > queue_item = yield from work_queue.get() > print('{0} grabbed item: {1}'.format(task_name, queue_item)) > asyncio.Task(tcp_echo_client(offset=queue_item)) > yield from asyncio.sleep(0.1) > > > if __name__ == "__main__": > q = asyncio.Queue() > > for x in range(100): > q.put_nowait(x) > > print(q) > > loop = asyncio.get_event_loop() > > tasks = [ > asyncio.async(do_work('task1', q)), > asyncio.async(do_work('task2', q)), > asyncio.async(do_work('task3', q)), > asyncio.async(do_work('task4', q)), > asyncio.async(do_work('task5', q)), > asyncio.async(do_work('task6', q)) > ] > > loop.run_until_complete(asyncio.wait(tasks)) > loop.run_forever() > loop.close() UPDATED : PORT = 33123 HOST = '127.0.0.1' LASTLINE = '#' import asyncio import os @asyncio.coroutine def tcp_echo_client(offset): def send(offset): MSG = '{'+"ClientId"+':'+"%s" % (str(offset))+'}' print("> " + MSG) writer.write((MSG ).encode("utf-8")) yield from writer.drain() def recv(): msgback = (yield from reader.readline()).decode("utf-8").rstrip() print("< " + msgback) return msgback reader, writer = yield from asyncio.open_connection(HOST, port=PORT) print(reader) print('Waiting 3 sec for response...') while True: try: response = yield from asyncio.wait_for(reader.readline(), timeout=100000.0) print(response) if str(response).find('GetClientInfo') > 0: send(int(offset)) yield from asyncio.sleep(0.5) except: print ('Error') @asyncio.coroutine def do_work(task_name, work_queue): while not work_queue.empty(): queue_item = yield from work_queue.get() print('{0} grabbed item: {1}'.format(task_name, queue_item)) asyncio.Task(tcp_echo_client(offset=queue_item)) yield from asyncio.sleep(0.1) if __name__ == "__main__": q = asyncio.Queue() for x in range(1): q.put_nowait(x) print(q) loop = asyncio.get_event_loop() tasks = [ asyncio.async(do_work('task1', q)), asyncio.async(do_work('task2', q)), asyncio.async(do_work('task3', q)), asyncio.async(do_work('task4', q)), asyncio.async(do_work('task5', q)), asyncio.async(do_work('task6', q)) ] loop.run_until_complete(asyncio.wait(tasks)) loop.run_forever() loop.close() Thanks From rosuav at gmail.com Tue Aug 23 09:42:38 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Aug 2016 23:42:38 +1000 Subject: Python non blocking multi-client service In-Reply-To: <9234fb72-5875-41ad-920c-4ce97455fa2b@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> <9234fb72-5875-41ad-920c-4ce97455fa2b@googlegroups.com> Message-ID: On Tue, Aug 23, 2016 at 11:39 PM, wrote: > except: > print ('Error') > Don't do this. ChrisA From dima.olkhov at cell-buddy.com Tue Aug 23 09:44:28 2016 From: dima.olkhov at cell-buddy.com (dima.olkhov at cell-buddy.com) Date: Tue, 23 Aug 2016 06:44:28 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: On Tuesday, August 23, 2016 at 4:09:07 PM UTC+3, dimao wrote: > I am trying to implement multi-client service. The service must be able to: > > connect to the server; > send/receive messages; > wait until a new message will be received > *** Per each client and non blocking > > My current code takes almost 99% CPU usage. Here is the main part of my code : > > > PORT = 33123 > HOST = '127.0.0.1' > > > import asyncio > import os > > @asyncio.coroutine > def tcp_echo_client(offset): > > def send(offset): > MSG = """{"ClientId":"%s", % (str(offset))"}""" > > print("> " + MSG) > writer.write((MSG).encode("utf-8")) > > > def recv(): > msgback = (yield from reader.readline()).decode("utf-8").rstrip() > print("< " + msgback) > return msgback > > reader, writer = yield from asyncio.open_connection(HOST, port=PORT) > > > print(reader) > print('Waiting 3 sec for response...') > > while True: > response = yield from asyncio.wait_for(reader.readline(), timeout=5.0) > print(response) > send(offset) > > yield from asyncio.sleep(0.5) > > @asyncio.coroutine > def do_work(task_name, work_queue): > while not work_queue.empty(): > queue_item = yield from work_queue.get() > print('{0} grabbed item: {1}'.format(task_name, queue_item)) > asyncio.Task(tcp_echo_client(offset=queue_item)) > yield from asyncio.sleep(0.1) > > > if __name__ == "__main__": > q = asyncio.Queue() > > for x in range(100): > q.put_nowait(x) > > print(q) > > loop = asyncio.get_event_loop() > > tasks = [ > asyncio.async(do_work('task1', q)), > asyncio.async(do_work('task2', q)), > asyncio.async(do_work('task3', q)), > asyncio.async(do_work('task4', q)), > asyncio.async(do_work('task5', q)), > asyncio.async(do_work('task6', q)) > ] > > loop.run_until_complete(asyncio.wait(tasks)) > loop.run_forever() > loop.close() I did that only for the debug reasons :-) From dima.olkhov at cell-buddy.com Tue Aug 23 10:14:57 2016 From: dima.olkhov at cell-buddy.com (dima.olkhov at cell-buddy.com) Date: Tue, 23 Aug 2016 07:14:57 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: <1b37f15e-af2b-4f92-aec6-51ed13f7c9df@googlegroups.com> On Tuesday, August 23, 2016 at 4:09:07 PM UTC+3, dimao wrote: > I am trying to implement multi-client service. The service must be able to: > > connect to the server; > send/receive messages; > wait until a new message will be received > *** Per each client and non blocking > > My current code takes almost 99% CPU usage. Here is the main part of my code : > > > PORT = 33123 > HOST = '127.0.0.1' > > > import asyncio > import os > > @asyncio.coroutine > def tcp_echo_client(offset): > > def send(offset): > MSG = """{"ClientId":"%s", % (str(offset))"}""" > > print("> " + MSG) > writer.write((MSG).encode("utf-8")) > > > def recv(): > msgback = (yield from reader.readline()).decode("utf-8").rstrip() > print("< " + msgback) > return msgback > > reader, writer = yield from asyncio.open_connection(HOST, port=PORT) > > > print(reader) > print('Waiting 3 sec for response...') > > while True: > response = yield from asyncio.wait_for(reader.readline(), timeout=5.0) > print(response) > send(offset) > > yield from asyncio.sleep(0.5) > > @asyncio.coroutine > def do_work(task_name, work_queue): > while not work_queue.empty(): > queue_item = yield from work_queue.get() > print('{0} grabbed item: {1}'.format(task_name, queue_item)) > asyncio.Task(tcp_echo_client(offset=queue_item)) > yield from asyncio.sleep(0.1) > > > if __name__ == "__main__": > q = asyncio.Queue() > > for x in range(100): > q.put_nowait(x) > > print(q) > > loop = asyncio.get_event_loop() > > tasks = [ > asyncio.async(do_work('task1', q)), > asyncio.async(do_work('task2', q)), > asyncio.async(do_work('task3', q)), > asyncio.async(do_work('task4', q)), > asyncio.async(do_work('task5', q)), > asyncio.async(do_work('task6', q)) > ] > > loop.run_until_complete(asyncio.wait(tasks)) > loop.run_forever() > loop.close() Can any one help me ? From zentraders at gmail.com Tue Aug 23 18:07:25 2016 From: zentraders at gmail.com (Zentrader) Date: Tue, 23 Aug 2016 15:07:25 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: <8603571f-9261-4e73-841d-9d92124eb1e8@googlegroups.com> I don't know anything about asyncio, but multiprocessing would be my tool of choice. The first example should be enough for what you want to do at https://pymotw.com/2/multiprocessing/basics.html From songofacandy at gmail.com Tue Aug 23 22:46:43 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Tue, 23 Aug 2016 19:46:43 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: <94f2bf5d-f53a-40c7-a162-8298163ec3eb@googlegroups.com> > > I did that only for the debug reasons :-) That's bad for debugging too. Real Exception and Stacktrace are far better than print('Error') on all time. Never do it even for debugging. From lax.clarke at gmail.com Tue Aug 23 23:15:09 2016 From: lax.clarke at gmail.com (lax.clarke at gmail.com) Date: Tue, 23 Aug 2016 20:15:09 -0700 (PDT) Subject: Python 3: Launch multiple commands(subprocesses) in parallel (but upto 4 any time at same time) AND store each of their outputs into a variable Message-ID: <42191edb-f272-490a-a846-867ec9fa6dfa@googlegroups.com> Hi, I've been reading various forums and python documentation on subprocess, multithreading, PIPEs, etc. But I cannot seem to mash together several of my requirements into working code. I am trying to: 1) Use Python 3+ (specifically 3.4 if it matters) 2) Launch N commands in background (e.g., like subprocess.call would for individual commands) 3) But only limit P commands to run at same time 4) Wait until all N commands are done 5) Have an array of N strings with the stdout+stderr of each command in it. What is the best way to do this? There are literally many variations of things in the Python documentation and Stackoverflow that I am unable to see the forest from trees (for my problem). Thank you very much! From murdocksgranpa at gmail.com Wed Aug 24 00:08:28 2016 From: murdocksgranpa at gmail.com (murdocksgranpa at gmail.com) Date: Tue, 23 Aug 2016 21:08:28 -0700 (PDT) Subject: degrees and radians. In-Reply-To: <3330ba.s35.ln@127.0.0.1> References: <3330ba.s35.ln@127.0.0.1> Message-ID: On Saturday, May 4, 2002 at 3:37:07 AM UTC-4, Jim Richardson wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > I am trying to get the math module to deal with degrees rather than > radians. (that it deals with radians for the angular functions like > sin() isn't mentioned in the docs, which was sort of an eyeopener :) I > can't find any info on doing this. I can convert from-to degrees in the > code calling the function, but that's a bit clunky. Any pointers to an > FM to R? :) > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.0.6 (GNU/Linux) > Comment: For info see http://www.gnupg.org > > iD8DBQE804+jd90bcYOAWPYRAt9KAKCuqeC4ozuXSaKZ5xY27Wv+k04QuQCcCrCZ > WyichPnKgXo+GaDdAebsaeU= > =h+vc > -----END PGP SIGNATURE----- > > -- > Jim Richardson > Anarchist, pagan and proud of it > http://www.eskimo.com/~warlock > Linux, from watches to supercomputers, for grandmas and geeks. For what is is worth.. Electrical Engineers for the most part work in degrees NOT Radians for example try doing polar to rectangular or vice versa in polar. I have never seen it done. Also Borland C and C++ used Degrees and NOT Radians.. go look at the libraries Just for what its worth. From gherron at digipen.edu Wed Aug 24 00:26:22 2016 From: gherron at digipen.edu (Gary Herron) Date: Tue, 23 Aug 2016 21:26:22 -0700 Subject: degrees and radians. In-Reply-To: References: <3330ba.s35.ln@127.0.0.1> Message-ID: <796ba73e-7684-6ec9-d89d-5babd2fe37d6@digipen.edu> On 08/23/2016 09:08 PM, murdocksgranpa at gmail.com wrote: > On Saturday, May 4, 2002 at 3:37:07 AM UTC-4, Jim Richardson wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> >> I am trying to get the math module to deal with degrees rather than >> radians. (that it deals with radians for the angular functions like >> sin() isn't mentioned in the docs, which was sort of an eyeopener :) I >> can't find any info on doing this. I can convert from-to degrees in the >> code calling the function, but that's a bit clunky. Any pointers to an >> FM to R? :) >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.0.6 (GNU/Linux) >> Comment: For info see http://www.gnupg.org >> >> iD8DBQE804+jd90bcYOAWPYRAt9KAKCuqeC4ozuXSaKZ5xY27Wv+k04QuQCcCrCZ >> WyichPnKgXo+GaDdAebsaeU= >> =h+vc >> -----END PGP SIGNATURE----- >> >> -- >> Jim Richardson >> Anarchist, pagan and proud of it >> http://www.eskimo.com/~warlock >> Linux, from watches to supercomputers, for grandmas and geeks. > For what is is worth.. Electrical Engineers for the most part work in degrees NOT Radians for example try doing polar to rectangular or vice versa in polar. > I have never seen it done. > > Also Borland C and C++ used Degrees and NOT Radians.. go look at the libraries > > Just for what its worth. > Do you really need anything more complex than this? >>> toRadians = math.pi/180.0 >>> math.sin(90*toRadians) 1.0 Perhaps I'm not understanding what you mean by "clunky", but this seems pretty clean and simple to me. Gary Herron -- Dr. Gary Herron Professor of Computer Science DigiPen Institute of Technology (425) 895-4418 From dmarv at dop.com Wed Aug 24 00:42:21 2016 From: dmarv at dop.com (Dale Marvin) Date: Tue, 23 Aug 2016 21:42:21 -0700 Subject: Python 3: Launch multiple commands(subprocesses) in parallel (but upto 4 any time at same time) AND store each of their outputs into a variable In-Reply-To: <42191edb-f272-490a-a846-867ec9fa6dfa@googlegroups.com> References: <42191edb-f272-490a-a846-867ec9fa6dfa@googlegroups.com> Message-ID: On 8/23/16 8:15 PM, lax.clarke at gmail.com wrote: > I am trying to: > > 1) Use Python 3+ (specifically 3.4 if it matters) > 2) Launch N commands in background (e.g., like subprocess.call would for individual commands) > 3) But only limit P commands to run at same time > 4) Wait until all N commands are done > 5) Have an array of N strings with the stdout+stderr of each command in it. > > What is the best way to do this? The best way is a matter of opinion, I have had success using Celery with Redis. DAle From random832 at fastmail.com Wed Aug 24 01:04:17 2016 From: random832 at fastmail.com (Random832) Date: Wed, 24 Aug 2016 01:04:17 -0400 Subject: degrees and radians. In-Reply-To: <796ba73e-7684-6ec9-d89d-5babd2fe37d6@digipen.edu> References: <3330ba.s35.ln@127.0.0.1> <796ba73e-7684-6ec9-d89d-5babd2fe37d6@digipen.edu> Message-ID: <1472015057.3161632.704358353.7C8FADD3@webmail.messagingengine.com> On Wed, Aug 24, 2016, at 00:26, Gary Herron wrote: > Perhaps I'm not understanding what you mean by "clunky", but this seems > pretty clean and simple to me. The original post is from 2002, I don't know why it got a reply just now. From no.email at nospam.invalid Wed Aug 24 01:25:27 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 23 Aug 2016 22:25:27 -0700 Subject: Python 3: Launch multiple commands(subprocesses) in parallel (but upto 4 any time at same time) AND store each of their outputs into a variable References: <42191edb-f272-490a-a846-867ec9fa6dfa@googlegroups.com> Message-ID: <877fb6rb5k.fsf@jester.gateway.pace.com> Dale Marvin writes: > The best way is a matter of opinion, I have had success using Celery > with Redis. I generally use GNU Parallel for stuff like that. Celery looks interesting though much fancier. From steve+comp.lang.python at pearwood.info Wed Aug 24 01:26:44 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 24 Aug 2016 15:26:44 +1000 Subject: degrees and radians. References: <3330ba.s35.ln@127.0.0.1> <796ba73e-7684-6ec9-d89d-5babd2fe37d6@digipen.edu> Message-ID: <57bd3016$0$1603$c3e8da3$5496439d@news.astraweb.com> On Wednesday 24 August 2016 14:26, Gary Herron wrote: > Do you really need anything more complex than this? > > >>> toRadians = math.pi/180.0 > > >>> math.sin(90*toRadians) > 1.0 > > Perhaps I'm not understanding what you mean by "clunky", but this seems > pretty clean and simple to me. The math module has two conversion functions, math.radians() and math.degrees(). Some other languages (Julia, by memory, and perhaps others) have dedicated sind(), cosd(), tand() or possibly dsin(), dcos(), dtan() functions which take their argument in degrees and are more accurate than doing a conversion to radians first. I'd like to see that. I've also seen languages with sinp() etc to calculate the sine of x*pi without the intermediate calculation. But if I were designing Python from scratch, I'd make sin(), cos() and tan() call dunder methods __sin__ etc: def sin(obj): if hasattr(type(obj), '__sin__'): y = type(obj).__sin__() if y is not NotImplemented: return y elif isinstance(obj, numbers.Number): return float.__sin__(float(obj)) raise TypeError Likewise for asin() etc. Then you could define your own numeric types, such as a Degrees type, a PiRadians type, etc, with their own dedicated trig function implementations, without the caller needing to care about which sin* function they call. -- Steve From bob.martin at excite.com Wed Aug 24 08:04:44 2016 From: bob.martin at excite.com (Bob Martin) Date: Wed, 24 Aug 2016 08:04:44 BST Subject: Raw Data from Website References: <85c71feb-2f2c-4420-a9de-5057e5ecbded@googlegroups.com> <57bbf7e1$0$2900$c3e8da3$76491128@news.astraweb.com> Message-ID: in 764257 20160823 081439 Steven D'Aprano wrote: >On Tuesday 23 August 2016 10:28, adam.j.kerr at gmail.com wrote: > >> Hi, >> >> I am hoping someone is able to help me. >> >> Is there a way to pull as much raw data from a website as possible. The >> webpage that I am looking for is as follows: >> >http://www.homepriceguide.com.au/Research/ResearchSeeFullList.aspx?LocationType=LGA&State=QLD&LgaID= >632 >> >> The main variable that is important is the "632" at the end, by adjusting >> this it changes the postcodes. Each postcode contains a large amount of data. >> Is there a way this all able to be exported into an excel document? > >Ideally, the web site itself will offer an Excel download option. If it >doesn't, you may be able to screen-scrape the data yourself, but: > >(1) it may be against the terms of service of the website; >(2) it may be considered unethical or possibly even copyright >infringement or (worst case) even illegal; >(3) especially if you're thinking of selling the data; >(4) at the very least, unless you take care not to abuse the service, >it may be rude and the website may even block your access. > >There are many tutorials and examples of "screen scraping" or "web scraping" on >the internet -- try reading them. It's not something I personally have any >experience with, but I expect that the process goes something like this: > >- connect to the website; >- download the particular page you want; >- grab the data that you care about; >- remove HTML tags and extract just the bits needed; >- write them to a CSV file. wget does the hard part. From steve+comp.lang.python at pearwood.info Wed Aug 24 04:18:11 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 24 Aug 2016 18:18:11 +1000 Subject: Raw Data from Website References: <85c71feb-2f2c-4420-a9de-5057e5ecbded@googlegroups.com> <57bbf7e1$0$2900$c3e8da3$76491128@news.astraweb.com> Message-ID: <57bd5845$0$1507$c3e8da3$5496439d@news.astraweb.com> On Wednesday 24 August 2016 17:04, Bob Martin wrote: > in 764257 20160823 081439 Steven D'Aprano > wrote: >>There are many tutorials and examples of "screen scraping" or "web scraping" >>on the internet -- try reading them. It's not something I personally have any >>experience with, but I expect that the process goes something like this: >> >>- connect to the website; >>- download the particular page you want; >>- grab the data that you care about; >>- remove HTML tags and extract just the bits needed; >>- write them to a CSV file. > > wget does the hard part. I don't think so. Just downloading a web page is easy. Parsing the potentially invalid HTML (or worse, the content is assembled in the browser by Javascript) to extract the actual data you care about is much harder. -- Steve From stephen_tucker at sil.org Wed Aug 24 04:30:16 2016 From: stephen_tucker at sil.org (Stephen Tucker) Date: Wed, 24 Aug 2016 09:30:16 +0100 Subject: degrees and radians. In-Reply-To: <57bd3016$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <3330ba.s35.ln@127.0.0.1> <796ba73e-7684-6ec9-d89d-5babd2fe37d6@digipen.edu> <57bd3016$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: For what it's worth, mathematicians naturally work with angles in radians. The mathematics of the trignonmetric functions works naturally when the angle is expressed in radians. For the older among us, logarithms also have a "natural" base, and that is the number e. Back in those days, however, even the mathematicians would use logarithm tables where the values were shown to base 10. On Wed, Aug 24, 2016 at 6:26 AM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Wednesday 24 August 2016 14:26, Gary Herron wrote: > > > Do you really need anything more complex than this? > > > > >>> toRadians = math.pi/180.0 > > > > >>> math.sin(90*toRadians) > > 1.0 > > > > Perhaps I'm not understanding what you mean by "clunky", but this seems > > pretty clean and simple to me. > > The math module has two conversion functions, math.radians() and > math.degrees(). > > > Some other languages (Julia, by memory, and perhaps others) have dedicated > sind(), cosd(), tand() or possibly dsin(), dcos(), dtan() functions which > take > their argument in degrees and are more accurate than doing a conversion to > radians first. I'd like to see that. > > I've also seen languages with sinp() etc to calculate the sine of x*pi > without > the intermediate calculation. > > But if I were designing Python from scratch, I'd make sin(), cos() and > tan() > call dunder methods __sin__ etc: > > > def sin(obj): > if hasattr(type(obj), '__sin__'): > y = type(obj).__sin__() > if y is not NotImplemented: > return y > elif isinstance(obj, numbers.Number): > return float.__sin__(float(obj)) > raise TypeError > > Likewise for asin() etc. > > Then you could define your own numeric types, such as a Degrees type, a > PiRadians type, etc, with their own dedicated trig function > implementations, > without the caller needing to care about which sin* function they call. > > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From urschrei at gmail.com Wed Aug 24 05:29:07 2016 From: urschrei at gmail.com (sth) Date: Wed, 24 Aug 2016 02:29:07 -0700 (PDT) Subject: convert triply-nested array to void pointer using ctypes Message-ID: <4ab52402-e056-49ca-8869-d4d1c88bb18a@googlegroups.com> I'd like to send an array containing arrays of 2-element float arrays to a foreign function, as the following struct: class _FFIArray(Structure): _fields_ = [("data", c_void_p), ("len", c_size_t)] @classmethod def from_param(cls, seq): """ Allow implicit conversions """ return seq if isinstance(seq, cls) else cls(seq) def __init__(self, seq): self.data = cast( np.array(seq, dtype=np.float64).ctypes.data_as(POINTER(c_double)), c_void_p ) self.len = len(seq) This works for data such as [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0], ? ], (.shape is n, 2 where n is an arbitrary length) but I'd like it to work for data such as [ [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0], ? ], [[7.0, 8.0], [9.0, 10.0], [11.0, 12.0], ? ], ? ] which has .shape p, q, 2 where p and q are of arbitrary length. Is this possible? From i at introo.me Wed Aug 24 06:54:17 2016 From: i at introo.me (Shiyao Ma) Date: Wed, 24 Aug 2016 03:54:17 -0700 (PDT) Subject: The order of iterable de-referencing in assignment? Message-ID: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Hi, Given a = [1, 2] a.extend(a) makes a = [1,2, 1,2] One might guess a.extend(a) would turn into an infinite loop. It turns out here Python first gets all the items of `a' and then append them to `a', so the infinite loop is avoided. My question is, is there any doc on the behavior of things like this? Another related example might be: a[:] = a Hopefully Python first gets all the items on the *right* side and then assigns them to the left. Regards. From rosuav at gmail.com Wed Aug 24 07:01:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Aug 2016 21:01:00 +1000 Subject: The order of iterable de-referencing in assignment? In-Reply-To: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> References: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Message-ID: On Wed, Aug 24, 2016 at 8:54 PM, Shiyao Ma wrote: > Given a = [1, 2] > > a.extend(a) makes a = [1,2, 1,2] > > One might guess a.extend(a) would turn into an infinite loop. It turns out here Python first gets all the items of `a' and then append them to `a', so the infinite loop is avoided. > Be careful: doing the same in Python may not behave that way. >>> a = [1,2] >>> for x in a: a.append(x) ... ^CTraceback (most recent call last): File "", line 1, in KeyboardInterrupt >>> len(a) 6370805 That right there, folks, is an infinite loop. ChrisA From Joaquin.Alzola at lebara.com Wed Aug 24 07:03:52 2016 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Wed, 24 Aug 2016 11:03:52 +0000 Subject: The order of iterable de-referencing in assignment? In-Reply-To: References: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Message-ID: >> One might guess a.extend(a) would turn into an infinite loop. It turns out here Python first gets all the items of `a' and then append them to `a', so the infinite loop is avoided. >>> a = [1,2] >>> for x in a: a.append(x) >... >^CTraceback (most recent call last): > File "", line 1, in >KeyboardInterrupt >>>> len(a) >6370805 >That right there, folks, is an infinite loop. If I am correct python points out an infinite loop with the "...", just pointing to more information. This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From marko at pacujo.net Wed Aug 24 07:09:56 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 24 Aug 2016 14:09:56 +0300 Subject: The order of iterable de-referencing in assignment? References: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Message-ID: <87inuqctiz.fsf@elektro.pacujo.net> Shiyao Ma : > Given a = [1, 2] > > a.extend(a) makes a = [1,2, 1,2] > > One might guess a.extend(a) would turn into an infinite loop. It turns > out here Python first gets all the items of `a' and then append them > to `a', so the infinite loop is avoided. Functionally, Python's lists are not linked lists but, rather, vectors. > My question is, is there any doc on the behavior of things like this? No. It should be here: It only states: The items of a list are arbitrary Python objects. and inexplicably muddies the waters by continuing: Lists are formed by placing a comma-separated list of expressions in square brackets. (Note that there are no special cases needed to form lists of length 0 or 1.) (which has nothing whatsoever to do with the data model). Marko From danielrr2 at gmail.com Wed Aug 24 07:11:29 2016 From: danielrr2 at gmail.com (=?UTF-8?Q?Daniel_Ria=C3=B1o?=) Date: Wed, 24 Aug 2016 13:11:29 +0200 Subject: Python 3 tkinter graphical statistical distributions fitter In-Reply-To: <255959d3-fc17-445f-948b-1d234d10474b@googlegroups.com> References: <255959d3-fc17-445f-948b-1d234d10474b@googlegroups.com> Message-ID: Thanks, James! 2016-08-13 12:46 GMT+02:00 : > I created a Python 3 tkinter graphical statistical distributions fitting > application that will fit a 1D data set to all of the continuous > statistical distributions in scipy.stats, with graphical display of the > distributions plotted against normalized histograms of the data. Fitted > results can be sorted by nnlf, AIC or AIC_BA. The URL on GitHub is: > > https://github.com/zunzun/tkInterStatsDistroFit > > James Phillips > -- > https://mail.python.org/mailman/listinfo/python-announce-list > > Support the Python Software Foundation: > http://www.python.org/psf/donations/ > From danielrr2 at gmail.com Wed Aug 24 07:14:47 2016 From: danielrr2 at gmail.com (=?UTF-8?Q?Daniel_Ria=C3=B1o?=) Date: Wed, 24 Aug 2016 13:14:47 +0200 Subject: Python 3 tkinter graphical statistical distributions fitter In-Reply-To: References: <255959d3-fc17-445f-948b-1d234d10474b@googlegroups.com> Message-ID: Please accept my apologies for sending the message to the full list ? 2016-08-24 13:11 GMT+02:00 Daniel Ria?o : > Thanks, James! > > 2016-08-13 12:46 GMT+02:00 : > >> I created a Python 3 tkinter graphical statistical distributions fitting >> application that will fit a 1D data set to all of the continuous >> statistical distributions in scipy.stats, with graphical display of the >> distributions plotted against normalized histograms of the data. Fitted >> results can be sorted by nnlf, AIC or AIC_BA. The URL on GitHub is: >> >> https://github.com/zunzun/tkInterStatsDistroFit >> >> James Phillips >> -- >> https://mail.python.org/mailman/listinfo/python-announce-list >> >> Support the Python Software Foundation: >> http://www.python.org/psf/donations/ >> > > From rosuav at gmail.com Wed Aug 24 07:17:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Aug 2016 21:17:43 +1000 Subject: The order of iterable de-referencing in assignment? In-Reply-To: References: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Message-ID: On Wed, Aug 24, 2016 at 9:03 PM, Joaquin Alzola wrote: >>> One might guess a.extend(a) would turn into an infinite loop. It turns out here Python first gets all the items of `a' and then append them to `a', so the infinite loop is avoided. >>>> a = [1,2] >>>> for x in a: a.append(x) >>... >>^CTraceback (most recent call last): > > File "", line 1, in >>KeyboardInterrupt >>>>> len(a) >>6370805 > >>That right there, folks, is an infinite loop. > > If I am correct python points out an infinite loop with the "...", just pointing to more information. That's true of self-referential objects and circular references: >>> a = [1, 2] >>> a.append(a) >>> a [1, 2, [...]] In this case, it's not an infinite loop or anything; it's simply an object that references itself: >>> id(a) 139945904550344 >>> [id(x) for x in a] [9241344, 9241376, 139945904550344] The list has three elements, one of which is the list itself. Same applies if the list has a reference to something else which has a reference to the original list, or anything along those lines. But "a.extend(a)" isn't quite like that. In its purest form, it means the same as the 'for' loop that I showed above, but... well, this comment from the CPython sources says exactly what I'm thinking of: Objects/listobject.c:795 /* Special cases: 1) lists and tuples which can use PySequence_Fast ops 2) extending self to self requires making a copy first */ > This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. > This email is confident and may be subject to white privilege, black privilege, and privileges of other colours. If you are not the intended recipient, please contact a lawyer and tell him that email footers are unenforceable. ChrisA From random832 at fastmail.com Wed Aug 24 10:10:14 2016 From: random832 at fastmail.com (Random832) Date: Wed, 24 Aug 2016 10:10:14 -0400 Subject: The order of iterable de-referencing in assignment? In-Reply-To: References: <19d48b30-aa30-4112-b153-f860bb2ab953@googlegroups.com> Message-ID: <1472047814.1336203.704772465.4474F394@webmail.messagingengine.com> On Wed, Aug 24, 2016, at 07:17, Chris Angelico wrote: > Objects/listobject.c:795 > > /* Special cases: > 1) lists and tuples which can use PySequence_Fast ops > 2) extending self to self requires making a copy first > */ And, of course, it is a special case - a.extend(iter(a)) is enough to break it. Frankly I'm not sure why bother to implement it when anyone who legitimately wants to do it can simply do a *= 2. From frank at chagford.com Wed Aug 24 10:58:54 2016 From: frank at chagford.com (Frank Millman) Date: Wed, 24 Aug 2016 16:58:54 +0200 Subject: Alternatives to XML? Message-ID: Hi all I have mentioned in the past that I use XML for storing certain structures 'off-line', and I got a number of comments urging me to use JSON or YAML instead. In fact XML has been working very well for me, but I am looking into alternatives simply because of the issue of using '>' and '<' in attributes. I can convert them to '>' and '<', but that imposes a cost in terms of readability. Here is a simple example - This is equivalent to the following python code - if _param.auto_party_id is not None: if on_insert: value = auto_gen(_param.auto_party_id) elif not_exists: value = '' The benefit of serialising it is partly to store it in a database and read it in at runtime, and partly to allow non-trusted users to edit it without raising security concerns. I ran my XML through some online converters, but I am not happy with the results. Here is a JSON version - { "case": { "compare": { "-src": "_param.auto_party_id", "-op": "is_not", "-tgt": "$None", "case": { "on_insert": { "auto_gen": { "-args": "_param.auto_party_id" } }, "not_exists": { "literal": { "-value": "" } } } } } } I can see how it works, but it does not seem as readable to me. It is not so obvious that 'compare' has three arguments which are evaluated, and if true the nested block is executed. Here is a YAML version - case: compare: case: on_insert: auto_gen: _args: "_param.auto_party_id" not_exists: literal: _value: "" _src: "_param.auto_party_id" _op: is_not _tgt: "$None" This seems even worse from a readability point of view. The arguments to 'compare' are a long way away from the block to be executed. Can anyone offer an alternative which is closer to my original intention? Thanks Frank Millman From marko at pacujo.net Wed Aug 24 11:12:03 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 24 Aug 2016 18:12:03 +0300 Subject: Alternatives to XML? References: Message-ID: <87zio25hh8.fsf@elektro.pacujo.net> "Frank Millman" : > I have mentioned in the past that I use XML for storing certain > structures 'off-line', and I got a number of comments urging me to use > JSON or YAML instead. JSON is very good. > In fact XML has been working very well for me, but I am looking into > alternatives simply because of the issue of using '>' and '<' in > attributes. I can convert them to '>' and '<', but that imposes > a cost in terms of readability. Precise syntax requires escaping. In fact, XML is not good enough at it. XML is far too much and too little at the same time. > Here is a simple example - > > > > > > > > > > > > > > Can anyone offer an alternative which is closer to my original intention? There's S-expressions: (case (compare #:src "_param.auto_party_id" #:op "is_not" #:tgt #f (case #:on-insert (auto-gen "_param.auto_party_id") #:not-exists (literal "")))) Marko From alister.ware at ntlworld.com Wed Aug 24 11:21:30 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 24 Aug 2016 15:21:30 GMT Subject: Alternatives to XML? References: Message-ID: <_Xivz.540281$SA.354411@fx44.am4> On Wed, 24 Aug 2016 16:58:54 +0200, Frank Millman wrote: > Hi all > > I have mentioned in the past that I use XML for storing certain > structures 'off-line', and I got a number of comments urging me to use > JSON or YAML instead. > > In fact XML has been working very well for me, but I am looking into > alternatives simply because of the issue of using '>' and '<' in > attributes. > I can convert them to '>' and '<', but that imposes a cost in > terms of readability. > are these files expected to be read/written by a human being or are they for your application to save & restore its settings? if the former then you probably need to choose a specification/format that was designed to be human readable form the start (such as the old .ini format) if it is primarily for your app then you need a format that efficiently & accurately saves the data you require, readability is a secondary (but still desirable) requirement for debugging & the rare case where a manual change is req. XLM is bulky with lots of redundant information & still not readily readable without extra tools. Json is quite terse but I find it quite readable & is well suited for saving most data structures pickle can save the data efficiently but is certainly not readable -- Even a blind pig stumbles upon a few acorns. From __peter__ at web.de Wed Aug 24 12:50:56 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Aug 2016 18:50:56 +0200 Subject: Alternatives to XML? References: Message-ID: Frank Millman wrote: > I have mentioned in the past that I use XML for storing certain structures > 'off-line', and I got a number of comments urging me to use JSON or YAML > instead. > > In fact XML has been working very well for me, but I am looking into > alternatives simply because of the issue of using '>' and '<' in > attributes. I can convert them to '>' and '<', but that imposes a > cost in terms of readability. > > Here is a simple example - > > > > > > > > > > > > > > > This is equivalent to the following python code - > > if _param.auto_party_id is not None: > if on_insert: > value = auto_gen(_param.auto_party_id) > elif not_exists: > value = '' I think we have a winner here ;) > The benefit of serialising it is partly to store it in a database and read > it in at runtime, and partly to allow non-trusted users to edit it without > raising security concerns. If you store what is basically a script as XML or JSON, why is that safer than Python or Javascript? From rgaddi at highlandtechnology.invalid Wed Aug 24 12:57:16 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 24 Aug 2016 16:57:16 -0000 (UTC) Subject: Python 3: Launch multiple commands(subprocesses) in parallel (but upto 4 any time at same time) AND store each of their outputs into a variable References: <42191edb-f272-490a-a846-867ec9fa6dfa@googlegroups.com> Message-ID: lax.clarke at gmail.com wrote: > Hi, > > I've been reading various forums and python documentation on subprocess, multithreading, PIPEs, etc. But I cannot seem to mash together several of my requirements into working code. > > I am trying to: > > 1) Use Python 3+ (specifically 3.4 if it matters) > 2) Launch N commands in background (e.g., like subprocess.call would for individual commands) > 3) But only limit P commands to run at same time > 4) Wait until all N commands are done > 5) Have an array of N strings with the stdout+stderr of each command in it. > > What is the best way to do this? > There are literally many variations of things in the Python documentation and Stackoverflow that I am unable to see the forest from trees (for my problem). > > Thank you very much! First off, I'm assuming that the stdout+stderr of these commands is of reasonable size rather than hundreds of megabytes. What you want is a finite pool of threads (or processes) that execute the tasks. multiprocessing.pool.Pool will do it. So will concurrent.futures, which is what I'd personally use just out of more familiarity with it. In either case your task should wrap a call to subprocess. subprocess.run is your easiest answer if you've got Python 3.5; the task would call it with stdout and stderr=subprocess.PIPE, get the CompletedProcess back, and then store the .stdout and .stderr string results. For older Python, create a subprocess.Popen (again with stdout and stderr=subprocess.PIPE) and call the communicate() method. There's probably a dozen other ways. That one there, that's your easiest. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rgaddi at highlandtechnology.invalid Wed Aug 24 13:03:49 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 24 Aug 2016 17:03:49 -0000 (UTC) Subject: degrees and radians. References: <3330ba.s35.ln@127.0.0.1> Message-ID: murdocksgranpa at gmail.com wrote: > On Saturday, May 4, 2002 at 3:37:07 AM UTC-4, Jim Richardson wrote: >> >> I am trying to get the math module to deal with degrees rather than >> radians. (that it deals with radians for the angular functions like >> sin() isn't mentioned in the docs, which was sort of an eyeopener :) I >> can't find any info on doing this. I can convert from-to degrees in the >> code calling the function, but that's a bit clunky. Any pointers to an >> FM to R? : >> >> -- >> Jim Richardson >> Anarchist, pagan and proud of it >> http://www.eskimo.com/~warlock >> Linux, from watches to supercomputers, for grandmas and geeks. > > For what is is worth.. Electrical Engineers for the most part work in degrees NOT Radians for example try doing polar to rectangular or vice versa in polar. > I have never seen it done. > While I fully admit to thinking in degrees, any time I'm actually doing any mathematical work my units are either radians or cycles. The representation of angle in fixed point cycles actually comes out really nicely; each bit takes you from hemispheres to quadrants to octants, etc. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rgaddi at highlandtechnology.invalid Wed Aug 24 13:12:34 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 24 Aug 2016 17:12:34 -0000 (UTC) Subject: Alternatives to XML? References: Message-ID: Frank Millman wrote: > Hi all > > I have mentioned in the past that I use XML for storing certain structures > 'off-line', and I got a number of comments urging me to use JSON or YAML > instead. > > In fact XML has been working very well for me, but I am looking into > alternatives simply because of the issue of using '>' and '<' in attributes. > I can convert them to '>' and '<', but that imposes a cost in terms of > readability. > > Here is a simple example - > > > > > > > > > > > > > > > This is equivalent to the following python code - > > if _param.auto_party_id is not None: > if on_insert: > value = auto_gen(_param.auto_party_id) > elif not_exists: > value = '' > > The benefit of serialising it is partly to store it in a database and read > it in at runtime, and partly to allow non-trusted users to edit it without > raising security concerns. > > I ran my XML through some online converters, but I am not happy with the > results. > > Here is a JSON version - > > { > "case": { > "compare": { > "-src": "_param.auto_party_id", > "-op": "is_not", > "-tgt": "$None", > "case": { > "on_insert": { > "auto_gen": { "-args": "_param.auto_party_id" } > }, > "not_exists": { > "literal": { "-value": "" } > } > } > } > } > } > > I can see how it works, but it does not seem as readable to me. It is not so > obvious that 'compare' has three arguments which are evaluated, and if true > the nested block is executed. > > Here is a YAML version - > > case: > compare: > case: > on_insert: > auto_gen: > _args: "_param.auto_party_id" > not_exists: > literal: > _value: "" > _src: "_param.auto_party_id" > _op: is_not > _tgt: "$None" > > This seems even worse from a readability point of view. The arguments to > 'compare' are a long way away from the block to be executed. > > Can anyone offer an alternative which is closer to my original intention? > > Thanks > > Frank Millman > You've been staring at that XML too long; you've become familiar with it. It's just as unreadable as the JSON or the YAML unless you already know what it says. That said, one core difference between XML and the other two is that XML allows for the idea that order matters. JSON and YAML both consider key/value pair objects in the same way Python does a dict -- there is AN order because there has to be one, but it's not expected to be important or preserved. And when you start talking about "close to" you're talking about preserving ordering. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From eryksun at gmail.com Wed Aug 24 17:00:50 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 24 Aug 2016 21:00:50 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Mon, Aug 22, 2016 at 5:24 PM, Chris Angelico wrote: > On Tue, Aug 23, 2016 at 3:13 AM, eryk sun wrote: >> On Mon, Aug 22, 2016 at 4:18 PM, Chris Angelico wrote: >>> >>>> The CON device should work if the process is attached to a console >>>> (i.e. a conhost.exe instance). >>> >>> No, I used Pike (to avoid any specifically-Python issues or >>> protections) running in a console. Attempting to write to "Logs/con" >>> wrote to the console, so I know the console device is active. >>> Attempting to write to "Logs/con.txt" failed as described. >> >> What version of Windows is this? If it's Windows 7 I'll have to check >> that later. If "Logs" is an existing directory, then both "Logs/con" >> and "Logs/con.txt" should refer to the console. If "Logs" doesn't >> exist, then both should fail. Virtual DOS devices only exist in >> existing directories. > > Yes, it was Windows 7 (running in a VM under Debian Jessie, though I > doubt that makes any difference). The Logs directory did exist (that's > why I used that otherwise-odd choice of name). I discovered why "Logs/con.txt" isn't working right in Windows 7, while "Logs/nul.txt" does get redirected correctly to r"\\.\nul". Prior to Windows 8 the console doesn't use an NT device, so the base API has a function named BaseIsThisAConsoleName that looks for names such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and returns either "CONIN$" or "CONOUT$" if there's a match. A match for just "CON" maps to one or the other of the latter depending on whether read or write access is desired. If there's a match, then a CreateFile call gets routed to OpenConsoleW, which uses the process ConsoleHandle to send the request to the attached instance of conhost.exe, which replies with a console pseudohandle. Note that the handle value is flagged by setting the lower 2 bits, i.e. 3, 7, 11, which allows routing to special console functions, such as WriteFile => WriteConsoleA. In Windows 8+ console handles are regular File handles (e.g. 24, 28, 32). When debugging this I observed that there's a performance hack in BaseIsThisAConsoleName. It only calls RtlIsDosDeviceName_U, which does a full check for "CON" in the path, if the name starts with '\\' , 'c', or 'C', or if it ends with 'n', 'N', ':', or '$'. This means r"C:\whatever\con.txt" works (the base path doesn't even have to exist), but not r"D:\whatever\con.txt". In your case the name starts with 'L', so BaseIsThisAConsoleName returns false and the code falls through to calling DosPathNameToRelativeNtPathName_U_WithStatus. This returns r"\??\con", which NtCreateFile fails to open. r"\??" is a virtual directory starting in Windows XP. For this directory the object manager first checks the logon session's DOS devices in r"\Sessions\0\DosDevices\[Logon Id]" and then the global DOS devices in r"\GLOBAL??". This is an improvement over the less flexible way that Windows 2000 managed DOS devices for terminal services and logon sessions. A DOS 'device' is an object symbolic link to the real NT device in the r"\Device" directory. In Windows 8+, r"\GLOBAL??\CON" is a link to r"\Device\ConDrv\Console", which is why opening "Logs/con.txt" worked 'correctly' for me in Windows 10. From roel at roelschroeven.net Wed Aug 24 17:40:54 2016 From: roel at roelschroeven.net (Roel Schroeven) Date: Wed, 24 Aug 2016 23:40:54 +0200 Subject: degrees and radians. In-Reply-To: References: <3330ba.s35.ln@127.0.0.1> Message-ID: murdocksgranpa at gmail.com schreef op 2016-08-24 06:08: > Also Borland C and C++ used Degrees and NOT Radians.. go look at the libraries You have a reference for that? Borland C++ Builder Help from C++ Builder 5 disagrees with you: "sin, sinl Header File math.h Category Math Routines Syntax #include double sin(double x); long double sinl(long double x); Description Calculates sine. sin computes the sine of the input value. Angles are specified in radians. sinl is the long double version; it takes a long double argument and returns a long double result. Error handling for these functions can be modified through the functions _matherr and _matherrl. Return Value sin and sinl return the sine of the input value." The VCL trigonometry functions also use radians. From the same help system: "SinCos Returns sine and cosine of an angle. Unit Math Category Trigonometry routines extern PACKAGE void __fastcall SinCos(Extended Theta, Extended &Sin, Extended &Cos); Description Call SinCos to obtain the sine and cosine of an angle expressed in radians. Theta specifies the angle. The sine and cosine are returned by the Sin and Cos parameters, respectively. SinCos is twice as fast as calling Sin and Cos separately for the same angle." I could also copy some entries from the C++Builder 2010 documentation; they all use radians too. Best regards, Roel -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From rosuav at gmail.com Wed Aug 24 17:48:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 07:48:13 +1000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <9035c731-3f57-6bfd-1c6f-d3d6695da3b6@gmail.com> <57ba51f2$0$1609$c3e8da3$5496439d@news.astraweb.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Thu, Aug 25, 2016 at 7:00 AM, eryk sun wrote: > I discovered why "Logs/con.txt" isn't working right in Windows 7, > while "Logs/nul.txt" does get redirected correctly to r"\\.\nul". > Prior to Windows 8 the console doesn't use an NT device, so the base > API has a function named BaseIsThisAConsoleName that looks for names > such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and > returns either "CONIN$" or "CONOUT$" if there's a match. A match for > just "CON" maps to one or the other of the latter depending on whether > read or write access is desired. See? This is why *even after I tested it* I wasn't sure I was right! The rules are... complicated. ChrisA From orbetellobabbello at gmx.com Wed Aug 24 19:52:04 2016 From: orbetellobabbello at gmx.com (SALVATORE ABBRASCIAGLIA MORGAN STANLEY SNR CNSLTNT) Date: Wed, 24 Aug 2016 16:52:04 -0700 (PDT) Subject: Redacted Message-ID: <4e443b2e-a465-45d9-a2df-ce41a64f2792@googlegroups.com> Redacted From orbetellobabbello at gmx.com Wed Aug 24 19:53:04 2016 From: orbetellobabbello at gmx.com (SALVATORE ABBRASCIAGLIA MORGAN STANLEY SNR CNSLTNT) Date: Wed, 24 Aug 2016 16:53:04 -0700 (PDT) Subject: Redacted Message-ID: <9d619163-aaaa-4ef5-a77e-bb86336c19d9@googlegroups.com> Redacted From rosuav at gmail.com Wed Aug 24 21:26:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 11:26:59 +1000 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2016 at 2:50 AM, Peter Otten <__peter__ at web.de> wrote: >> if _param.auto_party_id is not None: >> if on_insert: >> value = auto_gen(_param.auto_party_id) >> elif not_exists: >> value = '' > > I think we have a winner here ;) Agreed. http://thedailywtf.com/articles/The_Enterprise_Rules_Engine If you create a non-code way to do code, you either have to make it far FAR simpler than code, or... you should just use code. Use code. ChrisA From frank at chagford.com Thu Aug 25 01:33:41 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 25 Aug 2016 07:33:41 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:npkcnf$kq7$1 at blaine.gmane.org... > Hi all > I have mentioned in the past that I use XML for storing certain structures > 'off-line', and I got a number of comments urging me to use JSON or YAML > instead. > Can anyone offer an alternative which is closer to my original intention? Many thanks for the replies. I will respond to them all in one place. @alister "are these files expected to be read/written by a human being or are they for your application to save & restore its settings?" Good question. My project is a business/accounting system. It provides a number of tables and columns pre-defined, but it also allows users to create their own. I allow them to define business rules to be invoked at various points. Therefore it must be in a format which is readable/writable by humans, but executable at runtime by my program. .ini is an interesting idea - I will look into it @rob "You've been staring at that XML too long; you've become familiar with it. It's just as unreadable as the JSON or the YAML unless you already know what it says." Good comment! I am sure you are right. Whichever format I settle on, I will have to provide some sort of cheat-sheet explaining to non-technical users what the format is. Having said that, I do find the XML more readable in the sense that the attributes are closely linked with their elements. I think it is easier to explain what is doing than the equivalent in JSON or YAML. @Marko I have never heard of S-expressions before, but they look interesting. I will investigate further. @Peter/Chris I don't understand - please explain. If I store the business rule in Python code, how do I prevent untrusted users putting malicious code in there? I presume I would have to execute the code by calling eval(), which we all know is dangerous. Is there another way of executing it that I am unaware of? Frank From rosuav at gmail.com Thu Aug 25 01:46:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 15:46:07 +1000 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2016 at 3:33 PM, Frank Millman wrote: > @Peter/Chris > I don't understand - please explain. > > If I store the business rule in Python code, how do I prevent untrusted > users putting malicious code in there? I presume I would have to execute the > code by calling eval(), which we all know is dangerous. Is there another way > of executing it that I am unaware of? The real question is: How malicious can your users be? If the XML file is stored adjacent to the Python script that runs it, anyone who can edit one can edit the other. Ultimately, that means that (a) any malicious user can simply edit the Python script, and therefore (b) anyone who's editing the other file is not malicious. If that's not how you're doing things, give some more details of what you're trying to do. How are you preventing changes to the Python script? How frequent will changes be? Can you simply put all changes through a git repository and use a pull request workflow to ensure that a minimum of two people eyeball every change? ChrisA From marko at pacujo.net Thu Aug 25 01:59:46 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 25 Aug 2016 08:59:46 +0300 Subject: Alternatives to XML? References: Message-ID: <87poox5qy5.fsf@elektro.pacujo.net> "Frank Millman" : > If I store the business rule in Python code, how do I prevent > untrusted users putting malicious code in there? I presume I would > have to execute the code by calling eval(), which we all know is > dangerous. Is there another way of executing it that I am unaware of? This is a key question. A couple of days back I stated the principle that a programming language is better than a rule language. That principle is followed by PostScript printers, Java applets, web pages with JavaScript, emacs configuration files etc. The question is how do you get the desired benefits without opening the door to sabotage. You have to shield CPU usage, memory usage, disk access, network access etc. You can google for solutions with search terms such as "python sandbox", "linux sandbox" and "linux container sandbox". Marko From marko at pacujo.net Thu Aug 25 02:05:27 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 25 Aug 2016 09:05:27 +0300 Subject: Alternatives to XML? References: Message-ID: <87h9a95qoo.fsf@elektro.pacujo.net> Chris Angelico : > The real question is: How malicious can your users be? Oh, yes, the simple way to manage the situation is for the server to call seteuid() before executing the code after authenticating the user. Marko From frank at chagford.com Thu Aug 25 02:11:47 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 25 Aug 2016 08:11:47 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Chris Angelico" wrote in message news:CAPTjJmq2bcQPmQ9itVvZrBZJPcbYe5z6vDpKGYQj=8H+qKvXxQ at mail.gmail.com... On Thu, Aug 25, 2016 at 3:33 PM, Frank Millman wrote: > @Peter/Chris > > I don't understand - please explain. > > > > If I store the business rule in Python code, how do I prevent untrusted > > users putting malicious code in there? I presume I would have to execute > > the > > code by calling eval(), which we all know is dangerous. Is there another > > way > > of executing it that I am unaware of? > The real question is: How malicious can your users be? > If the XML file is stored adjacent to the Python script that runs it, > anyone who can edit one can edit the other. Ultimately, that means that > (a) any malicious user can simply edit the Python script, and therefore > (b) anyone who's editing the other file is not malicious. > If that's not how you're doing things, give some more details of what > you're trying to do. How are you preventing changes to the Python script? > How frequent will changes be? Can you simply put all changes through a git > repository and use a pull request workflow to ensure that a minimum of two > people eyeball every change? All interaction with users is via a gui. The database contains tables that define the database itself - tables, columns, form definitions, etc. These are not purely descriptive, they drive the entire system. So if a user modifies a definition, the changes are immediate. Does that answer your question? I can go into a lot more detail, but I am not sure where to draw the line. Frank From rosuav at gmail.com Thu Aug 25 02:29:47 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 16:29:47 +1000 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2016 at 4:11 PM, Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmq2bcQPmQ9itVvZrBZJPcbYe5z6vDpKGYQj=8H+qKvXxQ at mail.gmail.com... > > On Thu, Aug 25, 2016 at 3:33 PM, Frank Millman wrote: >> >> @Peter/Chris >> > I don't understand - please explain. >> > >> > If I store the business rule in Python code, how do I prevent untrusted >> > users putting malicious code in there? I presume I would have to execute >> > > the >> > code by calling eval(), which we all know is dangerous. Is there another >> > > way >> > of executing it that I am unaware of? > > >> The real question is: How malicious can your users be? > > >> If the XML file is stored adjacent to the Python script that runs it, >> anyone who can edit one can edit the other. Ultimately, that means that (a) >> any malicious user can simply edit the Python script, and therefore (b) >> anyone who's editing the other file is not malicious. > > >> If that's not how you're doing things, give some more details of what >> you're trying to do. How are you preventing changes to the Python script? >> How frequent will changes be? Can you simply put all changes through a git >> repository and use a pull request workflow to ensure that a minimum of two >> people eyeball every change? > > > All interaction with users is via a gui. The database contains tables that > define the database itself - tables, columns, form definitions, etc. These > are not purely descriptive, they drive the entire system. So if a user > modifies a definition, the changes are immediate. > > Does that answer your question? I can go into a lot more detail, but I am > not sure where to draw the line. Sounds to me like you have two very different concerns, then. My understanding of "GUI" is that it's a desktop app running on the user's computer, as opposed to some sort of client/server system - am I right? 1) Malicious users, as I describe above, can simply mess with your code directly, or bypass it and talk to the database, or anything. So you can ignore them. 2) Non-programmer users, without any sort of malice, want to be able to edit these scripts but not be caught out by a tiny syntactic problem. Concern #2 is an important guiding principle. You need your DSL to be easy to (a) read/write, and (b) verify/debug. That generally means restricting functionality some, but you don't have to dig into the nitty-gritty of security. If someone figures out a way to do something you didn't intend to be possible, no big deal; but if someone CANNOT figure out how to use the program normally, that's a critical failure. So I would recommend making your config files as simple and clean as possible. That might mean Python code; it does not mean XML, and probably not JSON either. It might mean YAML. It might mean creating your own DSL. It might be as simple as "Python code with a particular style guide". There are a lot of options. Here's your original XML and Python code: if _param.auto_party_id is not None: if on_insert: value = auto_gen(_param.auto_party_id) elif not_exists: value = '' Here's a very simple format, borrowing from RFC822 with a bit of Python added: if: _param.auto_party_id != None if: on_insert value: =auto_gen(_param.auto_party_id) elif: not_exists value: "if" and "elif" expect either a single boolean value, or two values separated by a known comparison operator (==, !=, <, >, etc). I'd exclude "is" and "is not" for parser simplicity. Any field name that isn't a keyword (in this case "value:") sets something to the given value, or evals the given string if it starts with an equals sign. You can mess around with the details all you like, but this is a fairly simple format (every line consists of "keyword: value" at some indentation level), and wouldn't be too hard to parse. The use of eval() means this is assuming non-malicious usage, which is (as mentioned above) essential anyway, as a full security check on this code would be virtually impossible. And, as mentioned, Python itself makes a fine DSL for this kind of thing. ChrisA From frank at chagford.com Thu Aug 25 03:13:14 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 25 Aug 2016 09:13:14 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Chris Angelico" wrote in message news:CAPTjJmof_sXqax0Ury5LsBEj7cdFv92WiWKbfvAC+bM=HwtHXA at mail.gmail.com... > Sounds to me like you have two very different concerns, then. My > understanding of "GUI" is that it's a desktop app running on the user's > computer, as opposed to some sort of client/server system - am I right? Not exactly, but the difference in not important, as you have got the essentials below spot on. For the record, the server runs an HTTP server, and anyone on the LAN/WAN can access the system using a browser. Because the tables that define the database are stored in the database itself, there is no difference between a form that allows a user to capture an invoice, and a form that allows a user to modify a column definition. It is all controlled through permissions, but technically they are identical. > 1) Malicious users, as I describe above, can simply mess with your code > directly, or bypass it and talk to the database, or anything. So you can > ignore them. Absolutely. If an organisation running my system wants to be secure, they should keep the server in a locked room only accessible by a trusted system administrator. > 2) Non-programmer users, without any sort of malice, want to be able to > edit these scripts but not be caught out by a tiny syntactic problem. Now we are getting to the nitty-gritty. [snip some good comments] > Here's a very simple format, borrowing from RFC822 with a bit of Python > added: if: _param.auto_party_id != None if: on_insert value: =auto_gen(_param.auto_party_id) elif: not_exists value: Getting close, but it is not *quite* that simple. For example, having isolated the LHS of the if clause, I process it something like this - if source.startswith("'"): source_value = source[1:-1] elif '.' in source: source_objname, source_colname = source.split('.', 1) source_record = caller.data_objects[source_objname] source_value = source_record.getval(source_colname) elif source == '$None': source_value = None elif source == '$True': source_value = True elif source == '$False': source_value = False elif source.startswith('int('): source_value = int(source[4:-1]) Anyway, you have isolated the essential issue. I need a DSL which is easy for a non-technical user to read/write, and easy to verify that it is achieving the desired result. I suspect that this is quite challenging whatever format I use. Up to now I have been using XML, and it works for me. As Rob pointed out, I have become too comfortable with it to be objective, but no-one has yet convinced me that the alternatives are any better. I may eventually end up with an additional layer that prompts the user through their requirement in 'wizard' style, and generates the underlying XML (or whatever) automatically. Frank From dieter at handshake.de Thu Aug 25 04:20:08 2016 From: dieter at handshake.de (dieter) Date: Thu, 25 Aug 2016 10:20:08 +0200 Subject: Knowing which thread had the GIL before References: <9cfa94e2-3f17-1cd1-9de8-e646ea9cd215@freelan.org> Message-ID: <87oa4huuo7.fsf@handshake.de> Julien Kauffmann writes: > ... > Is there a way to know which thread had the GIL right before my > profiling thread acquired it ? Perhaps through a C extension ? I've > seen such profilers for Linux that take advantage of an ITIMER signal > to do that. Sadly this is not an option on Windows. As you have already suspected, you will likely need a C extension: Python does (almost surely) not maintain a GIL history (not even a short one containing only the previous GIL holder). If you implement your monitoring thread (mostly) in C, there definitely is a global interpreter variable which contains a thread identifier for the thread currently holding the GIL. > Any feedback or remark concerning my technique is also welcome. It would be best if you could get informed when the GIL switches. Almost surely, this is unsupported on Python level. But perhaps on C level. I would check the (C) code for Python's GIL handling. From __peter__ at web.de Thu Aug 25 06:06:54 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 25 Aug 2016 12:06:54 +0200 Subject: Alternatives to XML? References: Message-ID: Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmof_sXqax0Ury5LsBEj7cdFv92WiWKbfvAC+bM=HwtHXA at mail.gmail.com... > >> Sounds to me like you have two very different concerns, then. My >> understanding of "GUI" is that it's a desktop app running on the user's >> computer, as opposed to some sort of client/server system - am I right? > > Not exactly, but the difference in not important, as you have got the > essentials below spot on. > > For the record, the server runs an HTTP server, and anyone on the LAN/WAN > can access the system using a browser. Because the tables that define the > database are stored in the database itself, there is no difference between > a form that allows a user to capture an invoice, and a form that allows a > user to modify a column definition. It is all controlled through > permissions, but technically they are identical. > >> 1) Malicious users, as I describe above, can simply mess with your code >> directly, or bypass it and talk to the database, or anything. So you can >> ignore them. > > Absolutely. If an organisation running my system wants to be secure, they > should keep the server in a locked room only accessible by a trusted > system administrator. > >> 2) Non-programmer users, without any sort of malice, want to be able to >> edit these scripts but not be caught out by a tiny syntactic problem. > > Now we are getting to the nitty-gritty. > > [snip some good comments] > >> Here's a very simple format, borrowing from RFC822 with a bit of Python >> added: > > if: _param.auto_party_id != None > if: on_insert > value: =auto_gen(_param.auto_party_id) > elif: not_exists > value: > > Getting close, but it is not *quite* that simple. > > For example, having isolated the LHS of the if clause, I process it > something like this - > > if source.startswith("'"): > source_value = source[1:-1] > elif '.' in source: > source_objname, source_colname = source.split('.', 1) > source_record = caller.data_objects[source_objname] > source_value = source_record.getval(source_colname) > elif source == '$None': > source_value = None > elif source == '$True': > source_value = True > elif source == '$False': > source_value = False > elif source.startswith('int('): > source_value = int(source[4:-1]) It may be a little more work, but whatever check you can think of for your XML can also be applied to a piece of Python code. For example the above might become $ cat eval_source.py import ast def source_value_dot(source_objname, source_colname): return "<{}.{}>".format(source_objname, source_colname) # may become: source_record = caller.data_objects[source_objname] return source_record.getval(source_colname) def get_source(node): # Modeled after ast.literal_eval() def _convert(node): if isinstance(node, ast.Expression): return _convert(node.body) elif isinstance(node, ast.Str): return node.s elif isinstance(node, ast.Attribute): if not isinstance(node.value, ast.Name): raise ValueError("only one dot, please", node) name = node.value.id column = node.attr return source_value_dot(name, column) elif isinstance(node, ast.NameConstant): return node.value elif isinstance(node, ast.Num): n = node.n if not isinstance(n, int): raise ValueError("only integers, please", node) return n else: raise ValueError("oops", node) return _convert(node) EXAMPLES = """\ None True 42 3.4 bar.baz foo.bar.baz 'whatever.you.want' """.splitlines() for source in EXAMPLES: e = ast.parse(source, mode="eval") try: result = get_source(e) except ValueError as e: result = "#error: {}".format(e.args[0]) print(source, "-->", result) $ python3 eval_source.py None --> None True --> True 42 --> 42 3.4 --> #error: only integers, please bar.baz --> foo.bar.baz --> #error: only one dot, please 'whatever.you.want' --> whatever.you.want Of course you could also use a slightly more general approach (use a whitelist of function names, feed dotted names to a function etc.), so that the resulting ast can safely be fed to exec(). Whatever you do, the most likely problem you may run into is annoyed users asking "This looks like Python, why can't I..." > Anyway, you have isolated the essential issue. I need a DSL which is easy > for a non-technical user to read/write, and easy to verify that it is > achieving the desired result. > > I suspect that this is quite challenging whatever format I use. Up to now > I have been using XML, and it works for me. As Rob pointed out, I have > become too comfortable with it to be objective, but no-one has yet > convinced me that the alternatives are any better. I may eventually end up > with an additional layer that prompts the user through their requirement > in 'wizard' style, and generates the underlying XML (or whatever) > automatically. > > Frank From rosuav at gmail.com Thu Aug 25 06:07:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 20:07:28 +1000 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2016 at 5:13 PM, Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmof_sXqax0Ury5LsBEj7cdFv92WiWKbfvAC+bM=HwtHXA at mail.gmail.com... > >> Sounds to me like you have two very different concerns, then. My >> understanding of "GUI" is that it's a desktop app running on the user's >> computer, as opposed to some sort of client/server system - am I right? > > > Not exactly, but the difference in not important, as you have got the > essentials below spot on. Cool. The essential difference, in this case, being where the edits are happening. > Absolutely. If an organisation running my system wants to be secure, they > should keep the server in a locked room only accessible by a trusted system > administrator. Right. That saves a ton of effort for you - it's no longer about malicious users. >> Here's a very simple format, borrowing from RFC822 with a bit of Python >> added: > > > if: _param.auto_party_id != None > if: on_insert > value: =auto_gen(_param.auto_party_id) > elif: not_exists > value: > > Getting close, but it is not *quite* that simple. > > For example, having isolated the LHS of the if clause, I process it > something like this - > > if source.startswith("'"): > source_value = source[1:-1] I don't like this; if you're going to surround strings with quotes, you either need some sort of escaping mechanism, or disallow the same quote character from coming up inside. (At very least, demand that the string ends with an apostrophe too, or you'll have a lot of VERY confusing failures if someone omits the closing apostrophe. Make it an instant error instead.) > elif '.' in source: > source_objname, source_colname = source.split('.', 1) > source_record = caller.data_objects[source_objname] > source_value = source_record.getval(source_colname) Fair enough; your dot notation is restricted to "this column from this thing", where a thing effectively identifies a table. Though I'm beginning to get the idea that the snippet you're showing here has elided all error checking - assuming that you actually do have that checking in the real version. > elif source == '$None': > source_value = None > elif source == '$True': > source_value = True > elif source == '$False': > source_value = False If you're looking for specific strings, you shouldn't need to adorn them. Point of note: "None" is a peculiarly Python concept, so anyone who's using it is going to need to understand its semantics from a Python perspective. If None is significant to your code, it's probably semantically important, and that might cause issues if anyone doesn't know code. And if everyone who edits this file knows Python, well, we're back to "why not just let them write in Python". > elif source.startswith('int('): > source_value = int(source[4:-1]) Hrm. This is where I'd move away from Python syntax. If you're only allowing a scant few typecasts, I'd go instead for an SQL-like syntax eg "some_value::int", or switch it around as "int::some_value", and make it clearly NOT a function call. Otherwise, I'd have a very simple "drop to Python" mode, which would be handled something like this: elif source.startswith("="): source_value = eval(source[1:]) > Anyway, you have isolated the essential issue. I need a DSL which is easy > for a non-technical user to read/write, and easy to verify that it is > achieving the desired result. Yes; or, if not easy to verify the result, at least make it easy to verify the syntax, even in the face of errors. > I suspect that this is quite challenging whatever format I use. Up to now I > have been using XML, and it works for me. As Rob pointed out, I have become > too comfortable with it to be objective, but no-one has yet convinced me > that the alternatives are any better. I may eventually end up with an > additional layer that prompts the user through their requirement in 'wizard' > style, and generates the underlying XML (or whatever) automatically. That's one option. At that point, the storage format becomes opaque to humans - it's just an interchange format between two programs. As such, I would advocate the use of JSON. Of course, you still need to build that UI, which is a lot of work. You'll also have to have GUI controls on your wizard for every possible option, which makes options very expensive (in contrast to a textual language, in which options are far cheaper) - every user has to see every option every time. So you have to strike a *very* careful balance between expressiveness/power and simplicity, and it's entirely possible to fall down on both fronts. ChrisA From frank at chagford.com Thu Aug 25 07:23:14 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 25 Aug 2016 13:23:14 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:nplvvl$ci2$1 at blaine.gmane.org... > Hi all > I have mentioned in the past that I use XML for storing certain structures > 'off-line', and I got a number of comments urging me to use JSON or YAML > instead. > Can anyone offer an alternative which is closer to my original intention? Thanks to Chris and Peter for their additional input - much appreciated. At the risk of disappointing some of you, this is how I am going to proceed. 1. I am writing tools to make it easier to develop business systems, and at the same time I am developing a business system. As the tools mature I am spending more time on the system, but as I work on the system I am finding shortcomings in the tools, so I am bouncing between the two at the moment. 2. There are many areas of the tools which other users will find confusing at first and will require explanations and documentation. I am more than ready to make changes based on the reactions I get. The subject of this thread is one small part of this. 3. My immediate priority is to finish the business system, get it out there, and get feedback. Hopefully other users will then start dabbling with the tools and provide feedback there as well. 4. As I have said already, for good or ill, I am comfortable with my current use of XML, so I do not have a pressing need to change to anything else. The problem that prompted this thread was the issue of storing '<' and '>' in attributes. I have come up with a simple workaround - pass the XML through a function between the database and the gui, converting from '>' to '>' in one direction, and back to '>' in the other. It works. 5. I have learned a lot from this thread, but for now it is staying in the back of my mind. If I ever get my project to the point where I need to move it to the front, I will know that I am getting somewhere! Thanks again to all. Frank From arsh840 at gmail.com Thu Aug 25 07:30:47 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Thu, 25 Aug 2016 04:30:47 -0700 (PDT) Subject: Holding until next value change In-Reply-To: References: Message-ID: <33acb6be-2708-4674-a654-e5c7a003e979@googlegroups.com> On Saturday, 20 August 2016 19:48:38 UTC+5:30, andrze... at gmail.com wrote: > prev = None > for value in main_call(): > if value==prev: > pass > else: > prev = value > if prev>0: > print('+v') > elif prev<0: > print('-v') > else: > print('0') No it did'nt worked. From rosuav at gmail.com Thu Aug 25 07:40:03 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Aug 2016 21:40:03 +1000 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On Thu, Aug 25, 2016 at 9:23 PM, Frank Millman wrote: > > At the risk of disappointing some of you, this is how I am going to proceed. > > 4. As I have said already, for good or ill, I am comfortable with my current > use of XML, so I do not have a pressing need to change to anything else. The > problem that prompted this thread was the issue of storing '<' and '>' in > attributes. I have come up with a simple workaround - pass the XML through a > function between the database and the gui, converting from '>' to '>' in > one direction, and back to '>' in the other. It works. Should be fine, as long as the actual XML file has < in it. It won't disappoint me at all, if the XML file is (primarily) a transport format between programs; just make sure it's always valid XML, rather than some "XML-like" file structure. In a greenfield project, I would advise strongly against this, but since you already have something to work with, it's not worth changing everything up. Know that every decision has consequences, and make your decisions with open eyes. I'm not disappointed by someone who, with greater knowledge of the situation and the likely consequences of various choices, chooses something different from what I, from thousands of miles away, recommended :) ChrisA From breamoreboy at gmail.com Thu Aug 25 07:56:21 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 25 Aug 2016 04:56:21 -0700 (PDT) Subject: Parlez-vous Python? Message-ID: <3472c70c-74a7-4702-8327-6ef9ca50983e@googlegroups.com> Well the language certainly is getting mentioned all over the place https://yougov.co.uk/news/2016/08/23/parlez-vous-python/ Kindest regards. Mark Lawrence. From __peter__ at web.de Thu Aug 25 09:58:24 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 25 Aug 2016 15:58:24 +0200 Subject: Alternatives to XML? References: Message-ID: Frank Millman wrote: > "Frank Millman" wrote in message news:nplvvl$ci2$1 at blaine.gmane.org... > >> Hi all > >> I have mentioned in the past that I use XML for storing certain >> structures 'off-line', and I got a number of comments urging me to use >> JSON or YAML instead. > >> Can anyone offer an alternative which is closer to my original intention? > > Thanks to Chris and Peter for their additional input - much appreciated. > > At the risk of disappointing some of you, this is how I am going to > proceed. 'Tis too late for me to stop ;) > 1. I am writing tools to make it easier to develop business systems, and > at the same time I am developing a business system. As the tools mature I > am spending more time on the system, but as I work on the system I am > finding shortcomings in the tools, so I am bouncing between the two at the > moment. > > 2. There are many areas of the tools which other users will find confusing > at first and will require explanations and documentation. I am more than > ready to make changes based on the reactions I get. The subject of this > thread is one small part of this. > > 3. My immediate priority is to finish the business system, get it out > there, and get feedback. Hopefully other users will then start dabbling > with the tools and provide feedback there as well. > > 4. As I have said already, for good or ill, I am comfortable with my > current use of XML, so I do not have a pressing need to change to anything > else. The problem that prompted this thread was the issue of storing '<' > and '>' in attributes. I have come up with a simple workaround - pass the > XML through a > function between the database and the gui, converting from '>' to '>' > in one direction, and back to '>' in the other. It works. As you have to keep the "<", why bother? > 5. I have learned a lot from this thread, but for now it is staying in the > back of my mind. If I ever get my project to the point where I need to > move it to the front, I will know that I am getting somewhere! At that point you may also look at my messy/buggy/incomplete attempt to convert between xml and python: $ cat convert.py import ast from xml.etree import ElementTree as etree XML = """\ """ tree = etree.fromstring(XML) TARGETS = { "$None": "None", } RTARGETS = { None: "$None", } assert len(RTARGETS) == len(TARGETS) OPS = { "is_not": "is not", } ROPS = {ast.IsNot: "is_not"} assert len(ROPS) == len(OPS) NAMES = {"on_insert", "not_exists"} FUNCS = {"auto_gen"} def getchildren(elem): yield from elem.getchildren() def to_python(elem, indent=""): # XXX build an AST rather than source code. if elem.tag == "compare": yield "{}if {} {} {}:".format( indent, elem.attrib["src"], OPS[elem.attrib["op"]], TARGETS[elem.attrib["tgt"]] ) [child] = getchildren(elem) yield from to_python(child, indent) elif elem.tag in NAMES: yield "{}if {}:".format(indent, elem.tag) for child in getchildren(elem): yield from to_python(child, indent + " ") elif elem.tag == "case": for child in getchildren(elem): yield from to_python(child, indent + " ") elif elem.tag == "literal": yield "{}value = {!r}".format(indent, elem.attrib["value"]) elif elem.tag in FUNCS: yield "{}auto_gen({})".format(indent, elem.attrib["args"]) else: raise ValueError("Unknown tag {!r}".format(elem.tag)) def dotted(node): if isinstance(node, ast.Attribute): return dotted(node.value) + "." + node.attr else: return node.id def to_xml(python): module = ast.parse(python) [body] = module.body root = etree.Element("case") def _convert(node, parent): if isinstance(node, ast.If): test = node.test if isinstance(test, ast.Compare): compare = etree.Element("compare") [op] = test.ops compare.attrib["src"] = dotted(test.left) compare.attrib["op"] = ROPS[type(op)] right = test.comparators[0].value compare.attrib["tgt"] = RTARGETS[right] case = etree.Element("case") compare.append(case) parent.append(compare) for child in node.body: _convert(child, case) elif isinstance(test, ast.Name): ename = etree.Element(test.id) parent.append(ename) for child in node.body: _convert(child, ename) elif isinstance(node, ast.Expr): evalue = node.value evalue.func.id invoke = etree.Element(evalue.func.id) invoke.attrib["args"] = dotted(evalue.args[0]) parent.append(invoke) elif isinstance(node, ast.Assign): assign = etree.Element("literal") [target] = node.targets assign.attrib["value"] = node.value.s parent.append(assign) else: global x x = node exit("unhandled") _convert(body, root) return root # http://stackoverflow.com/questions/17402323/ # use-xml-etree-elementtree-to-write-out-nicely-formatted-xml-files ElementTree = etree from xml.dom import minidom def prettify(elem): """Return a pretty-printed XML string for the Element. """ rough_string = ElementTree.tostring(elem, 'utf-8') reparsed = minidom.parseString(rough_string) return reparsed.toprettyxml(indent=" ") print("XML...") print(XML) print("\nbecomes Python...") python = "\n".join(to_python(next(getchildren(tree)))) print(python) print("\nbecomes XML:") root = to_xml(python) print(prettify(root)) $ python3 convert.py XML... becomes Python... if _param.auto_party_id is not None: if on_insert: auto_gen(_param.auto_party_id) if not_exists: value = '' becomes XML: From frank at chagford.com Thu Aug 25 10:45:47 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 25 Aug 2016 16:45:47 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Peter Otten" wrote in message news:npmti0$qvu$1 at blaine.gmane.org... > Frank Millman wrote: > > At the risk of disappointing some of you, this is how I am going to > > proceed. > 'Tis too late for me to stop ;) > > The problem that prompted this thread was the issue of storing '<' and > > '>' in attributes. I have come up with a simple workaround - pass the > > XML through a function between the database and the gui, converting from > > '>' to '>' in one direction, and back to '>' in the other. It > > works. > As you have to keep the "<", why bother? If you mean why don't I convert the '<' to '<', the answer is that I do - I just omitted to say so. However, explicit is better than implicit :-) > > If I ever get my project to the point where I need to move it to the > > front, I will know that I am getting somewhere! > At that point you may also look at my messy/buggy/incomplete attempt to > convert between xml and python: [snip some impressive code] Wow - that is impressive! I don't know if I will go that route, but I have kept a copy and will study it closely when I have time. Many thanks Frank From __peter__ at web.de Thu Aug 25 11:17:03 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 25 Aug 2016 17:17:03 +0200 Subject: Alternatives to XML? References: Message-ID: Frank Millman wrote: >> As you have to keep the "<", why bother? > > If you mean why don't I convert the '<' to '<', the answer is that I do > - I just omitted to say so. However, explicit is better than implicit :-) Doesn't that make the XML document invalid or changes it in an irreversible way? How would you know whether "" started out as "" or "<bar/>" ? From oscialanbaravan at gmx.com Thu Aug 25 15:15:40 2016 From: oscialanbaravan at gmx.com (CIRO SCIACQUALAQUAGLIA WALL STREET NEW YORK NAPLES) Date: Thu, 25 Aug 2016 12:15:40 -0700 (PDT) Subject: Redacted Message-ID: Redacted From piet-l at pietvanoostrum.com Thu Aug 25 15:38:02 2016 From: piet-l at pietvanoostrum.com (Piet van Oostrum) Date: Thu, 25 Aug 2016 21:38:02 +0200 Subject: Something wrong with the PIP, lots of friend have the same problem References: Message-ID: Wentao Liang writes: > even though I try to upgrade PIP[cid:beb9e7c5-7d98-4506-ba67-9e67890edec1] > > ________________________________ > From: Wentao Liang > Sent: Sunday, 21 August 2016 4:35:31 PM > To: python-list at python.org > Subject: Something wrong with the PIP, lots of friend have the same problem > > > [cid:1c20248f-a678-40c0-9563-da4f8a1ef3bd] > Is it the problem from PIP itself, or I used it in a wrong way? [?] > > Looking forwards to your reply! > > Thank you > > Wentao Liang Without more information it is difficult to say. What command did you use and what was the result? [Maybe you tried to include images in your post/mail but they did not come through. Just copy/paste the text.] -- Piet van Oostrum WWW: http://pietvanoostrum.com/ PGP key: [8DAE142BE17999C4] From sohcahtoa82 at gmail.com Thu Aug 25 18:15:27 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Thu, 25 Aug 2016 15:15:27 -0700 (PDT) Subject: Python non blocking multi-client service In-Reply-To: References: <9f9ba3da-64c0-40ef-9dea-91e1cf4cc695@googlegroups.com> Message-ID: On Tue, Aug 23, 2016 at 11:39 PM, wrote: > On Tuesday, August 23, 2016 at 6:42:53 AM UTC-7, Chris Angelico wrote: > > On Tuesday, August 23, 2016 at 4:09:07 PM UTC+3, dimao wrote: > > > except: > > > print ('Error') > > > > > > Don't do this. > > > > ChrisA > > I did that only for the debug reasons :-) That makes it even worse. If you're trying to debug something, you want as much information as you can get, and you're throwing it all away. If you want your program to keep going no matter what happened, at *least* print the stack trace, and not just "Error". From tjreedy at udel.edu Thu Aug 25 18:16:52 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 25 Aug 2016 18:16:52 -0400 Subject: Asynchronous programming In-Reply-To: References: <57abf6bd$0$1602$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 8/11/2016 11:55 PM, Lawrence D?Oliveiro wrote: > On Friday, August 12, 2016 at 2:25:05 AM UTC+12, Terry Reedy wrote: > >> When I read something like "Python finally acquired an event loop in >> 3.4" I wonder where people have been. The tk event loop has been in >> Python perhaps for 2 decades... > > As was pointed out to me just a few days ago, that?s Tcl, not Python. Tk is part of Tcl/Tk. Tk's mainloop() calls level tcl loop functions. The tkinter and _tkinter modules are Python stdlib modules. For timer-based callbacks, there is little difference between using tkinter and async modules. The fact that async uses a Python-coded event loop while tkinter uses a Tcl-coded loop is nearly invisible. Several other stdlib modules wrap 'foreign' code too. -- Terry Jan Reedy From frank at chagford.com Fri Aug 26 01:22:05 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 26 Aug 2016 07:22:05 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Peter Otten" wrote in message news:npn25e$s5n$1 at blaine.gmane.org... Frank Millman wrote: >> As you have to keep the "<", why bother? > > If you mean why don't I convert the '<' to '<', the answer is that I do > - I just omitted to say so. However, explicit is better than implicit :-) > Doesn't that make the XML document invalid or changes it in an > irreversible way? How would you know whether "" started out as "" or "<bar/>" ? I cheat ;-) It is *my* XML, and I know that I only use the offending characters inside attributes, and attributes are the only place where double-quote marks are allowed. So this is my conversion routine - lines = string.split('"') # split on attributes for pos, line in enumerate(lines): if pos%2: # every 2nd line is an attribute lines[pos] = line.replace('<', '<').replace('>', '>') return '"'.join(lines) Frank From mlzarathustra at gmail.com Fri Aug 26 02:20:17 2016 From: mlzarathustra at gmail.com (mlz) Date: Thu, 25 Aug 2016 23:20:17 -0700 (PDT) Subject: a *= b not equivalent to a = a*b Message-ID: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> I've been playing with the binomial function, and found that in the below code, rs *= x does not behave the same way as rs = rs * x. When I set FAIL to True, I get a different result. Both results are below. I had read that the two were equivalent. What am I missing? thanks, -= miles =- #!/usr/bin/python2 import sys FAIL= True if len(sys.argv)>1 else False def bin(n,k): rs=1 k=min(k,n-k) for i in range(1,k+1): if FAIL: rs *= (n-(i-1))/i # these should be the same, else: rs = rs * (n-(i-1))/i # but apparently are not return rs for n in range(10): for k in range(n+1): print bin(n,k), print'' ------------------- output ------------------------- $ pascal2 1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 1 7 21 35 35 21 7 1 1 8 28 56 70 56 28 8 1 1 9 36 84 126 126 84 36 9 1 $ pascal2 fail 1 1 1 1 2 1 1 3 3 1 1 4 4 4 1 1 5 10 10 5 1 1 6 12 12 12 6 1 1 7 21 21 21 21 7 1 1 8 24 48 48 48 24 8 1 1 9 36 72 72 72 72 36 9 1 From jussi.piitulainen at helsinki.fi Fri Aug 26 02:32:14 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 26 Aug 2016 09:32:14 +0300 Subject: a *= b not equivalent to a = a*b References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: mlz writes: > I've been playing with the binomial function, and found that in the > below code, rs *= x does not behave the same way as rs = rs * x. When > I set FAIL to True, I get a different result. Both results are below. > > I had read that the two were equivalent. What am I missing? You don't really have rs * x on the right-hand side. You have (rs * x)/y where rs * x is always divisible by y but x alone is not. For the record, your actual statement was rs = rs * (n-(i-1))/i, and you are using Python 2 where / denotes integer division. Try 3 * 4/6 and 3 * (4/6). From songofacandy at gmail.com Fri Aug 26 02:35:25 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Fri, 26 Aug 2016 15:35:25 +0900 Subject: a *= b not equivalent to a = a*b In-Reply-To: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: if FAIL: rs *= (n-(i-1))/i # these should be the same, This is equal to rs = rs * ((n-(i-1))/i) else: rs = rs * (n-(i-1))/i # but apparently are not This is equal to rs = (rs * (n-(i-1)))/i On Fri, Aug 26, 2016 at 3:20 PM, mlz wrote: > I've been playing with the binomial function, and found that in the below code, rs *= x does not behave the same way as rs = rs * x. When I set FAIL to True, I get a different result. Both results are below. > > I had read that the two were equivalent. What am I missing? > > thanks, > -= miles =- > > > #!/usr/bin/python2 > > import sys > FAIL= True if len(sys.argv)>1 else False > > def bin(n,k): > rs=1 > k=min(k,n-k) > > for i in range(1,k+1): > if FAIL: rs *= (n-(i-1))/i # these should be the same, > else: rs = rs * (n-(i-1))/i # but apparently are not > return rs > > > for n in range(10): > for k in range(n+1): > print bin(n,k), > print'' > > ------------------- output ------------------------- > > > $ pascal2 > 1 > 1 1 > 1 2 1 > 1 3 3 1 > 1 4 6 4 1 > 1 5 10 10 5 1 > 1 6 15 20 15 6 1 > 1 7 21 35 35 21 7 1 > 1 8 28 56 70 56 28 8 1 > 1 9 36 84 126 126 84 36 9 1 > > $ pascal2 fail > 1 > 1 1 > 1 2 1 > 1 3 3 1 > 1 4 4 4 1 > 1 5 10 10 5 1 > 1 6 12 12 12 6 1 > 1 7 21 21 21 21 7 1 > 1 8 24 48 48 48 24 8 1 > 1 9 36 72 72 72 72 36 9 1 > > > > > -- > https://mail.python.org/mailman/listinfo/python-list -- INADA Naoki From mlzarathustra at gmail.com Fri Aug 26 02:40:50 2016 From: mlzarathustra at gmail.com (mlzarathustra at gmail.com) Date: Thu, 25 Aug 2016 23:40:50 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Precedence, d'oh! rs *= (n-(i-1))/i is equivalent to: rs = rs * ((n-(i-1))/i) not rs = rs * (n-(i-1))/i which is the same as rs = ( rs * (n-(i-1)) ) /i Ken Iverson was right. Precedence is a bad idea. From __peter__ at web.de Fri Aug 26 02:51:06 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Aug 2016 08:51:06 +0200 Subject: a *= b not equivalent to a = a*b References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: mlz wrote: > I've been playing with the binomial function, and found that in the below > code, rs *= x does not behave the same way as rs = rs * x. When I set FAIL > to True, I get a different result. Both results are below. > > I had read that the two were equivalent. What am I missing? > > thanks, > -= miles =- > > > #!/usr/bin/python2 > > import sys > FAIL= True if len(sys.argv)>1 else False > > def bin(n,k): > rs=1 > k=min(k,n-k) > > for i in range(1,k+1): > if FAIL: rs *= (n-(i-1))/i # these should be the same, This is evaluated as rs = rs * ((n - (i - 1)) / i) > else: rs = rs * (n-(i-1))/i # but apparently are not while this is evaluated as rs = (rs * (n - (i - 1))/i so the two expressions do not really result in the same calculation. A simpler example: >>> (2*2)/3 1 >>> 2*(2/3) 0 Now "hide" the the order of evaluation: >>> a, b = 2, 3 >>> a *= a/b >>> a 0 >>> a, b = 2, 3 >>> a = a*a/b >>> a 1 > return rs > > > for n in range(10): > for k in range(n+1): > print bin(n,k), > print'' > > ------------------- output ------------------------- > > > $ pascal2 > 1 > 1 1 > 1 2 1 > 1 3 3 1 > 1 4 6 4 1 > 1 5 10 10 5 1 > 1 6 15 20 15 6 1 > 1 7 21 35 35 21 7 1 > 1 8 28 56 70 56 28 8 1 > 1 9 36 84 126 126 84 36 9 1 > > $ pascal2 fail > 1 > 1 1 > 1 2 1 > 1 3 3 1 > 1 4 4 4 1 > 1 5 10 10 5 1 > 1 6 12 12 12 6 1 > 1 7 21 21 21 21 7 1 > 1 8 24 48 48 48 24 8 1 > 1 9 36 72 72 72 72 36 9 1 From rosuav at gmail.com Fri Aug 26 02:51:40 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 16:51:40 +1000 Subject: a *= b not equivalent to a = a*b In-Reply-To: <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Message-ID: On Fri, Aug 26, 2016 at 4:40 PM, wrote: > Precedence, d'oh! > > rs *= (n-(i-1))/i > is equivalent to: > rs = rs * ((n-(i-1))/i) > > not > rs = rs * (n-(i-1))/i > > which is the same as > rs = ( rs * (n-(i-1)) ) /i > > > Ken Iverson was right. Precedence is a bad idea. No, precedence is not a bad idea. Making assumptions based on a lack of precedence, now, that's a bad idea. But the problem is the assumption :) ChrisA From mlzarathustra at gmail.com Fri Aug 26 03:10:20 2016 From: mlzarathustra at gmail.com (mlzarathustra at gmail.com) Date: Fri, 26 Aug 2016 00:10:20 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Message-ID: <76dfbd43-1bde-499c-8b35-f4064e5f395e@googlegroups.com> Yes, I just worked that out. It's the integer math that's the problem. I guess this has been fixed in python 3, but unfortunately it seems that most people are still using python 2. Thanks for all the help! From mlzarathustra at gmail.com Fri Aug 26 03:14:51 2016 From: mlzarathustra at gmail.com (mlzarathustra at gmail.com) Date: Fri, 26 Aug 2016 00:14:51 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Message-ID: I was being facetious, but behind it is a serious point. Neither the APL nor the J languages use precedence even though their inventor, Ken Iverson, was a mathematician. That was to support functional programming dating back to the 1970's. However, precedence wasn't the problem in this case, it was the type conversion. From jussi.piitulainen at helsinki.fi Fri Aug 26 03:43:49 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 26 Aug 2016 10:43:49 +0300 Subject: a *= b not equivalent to a = a*b References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> <76dfbd43-1bde-499c-8b35-f4064e5f395e@googlegroups.com> Message-ID: mlzarathustra at gmail.com writes: > Yes, I just worked that out. It's the integer math that's the problem. > > I guess this has been fixed in python 3 [- -] Note that division in Python 3 produces approximate results (floating point numbers). This may or may not be what you want in this exercise. I would blame this problem entirely on the precedence issue and just do the multiplication first. The algorithm is pretty neat that way. (Meta) Also, please leave some relevant context so it's easier to follow the discussion. It's a general principle, and particularly acute in comp.lang.python where many messages fail to identify their parent. From mlzarathustra at gmail.com Fri Aug 26 03:44:33 2016 From: mlzarathustra at gmail.com (mlzarathustra at gmail.com) Date: Fri, 26 Aug 2016 00:44:33 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: Here's the key: $ python2 Python 2.7.10 ... >>> 1/2 0 >>> $ python Python 3.5.1 ... >>> 1/2 0.5 >>> 1//2 0 >>> I read about this awhile ago, but it's not until it bites you that you remember fully. From python at lucidity.plus.com Fri Aug 26 03:50:43 2016 From: python at lucidity.plus.com (Erik) Date: Fri, 26 Aug 2016 08:50:43 +0100 Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Message-ID: <927eadbc-722a-b03e-666a-9ba14ed5217e@lucidity.plus.com> On 26/08/16 08:14, mlzarathustra at gmail.com wrote: > > I was being facetious, but behind it is a serious point. Neither the APL nor the J languages use precedence even though their inventor, Ken Iverson, was a mathematician. > > That was to support functional programming dating back to the 1970's. Precedence is not the issue here anyway. Both '*' and '/' have the same precedence. The two operations in your expressions have to be evaluated in SOME order - either left-to-right or right-to-left. The issue is twofold: Firstly, the compound assignments will always evaluate their RHS before performing the assignment - this is not the same as operator precedence - they have to, else what does Python pass to the function that knows what the compound assignment means for that type (e.g. sequences may be extended for '*=')? So for compound assignments there is always effectively an implicit set of parentheses around the RHS compared to the expression without the assignment operator (if you think of the compound assignment as being a _part_ of the overall expression). Secondly, the way you have chosen to layout your code has fooled your brain into thinking that the division is performed before the multiplication. Python doesn't care that you jammed the sub-expression together with no whitespace ;) E. From python at lucidity.plus.com Fri Aug 26 03:53:14 2016 From: python at lucidity.plus.com (Erik) Date: Fri, 26 Aug 2016 08:53:14 +0100 Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> Message-ID: <219db485-4229-5ee4-6c2e-f6c01b9a8f9b@lucidity.plus.com> On 26/08/16 08:44, mlzarathustra at gmail.com wrote: > Here's the key: > > $ python2 > Python 2.7.10 ... >>>> 1/2 > 0 >>>> > > $ python > Python 3.5.1 ... >>>> 1/2 > 0.5 >>>> 1//2 > 0 >>>> > > I read about this awhile ago, but it's not until it bites you that you remember fully. How is this related to your question? The example explicitly says Python 2 and doesn't use the '//' operator. E. From arsh840 at gmail.com Fri Aug 26 04:37:54 2016 From: arsh840 at gmail.com (Arshpreet Singh) Date: Fri, 26 Aug 2016 01:37:54 -0700 (PDT) Subject: Holding until next value change In-Reply-To: <57b7f3b6$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <57b7f3b6$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Saturday, 20 August 2016 11:38:03 UTC+5:30, Steve D'Aprano wrote: > state = ignore_negative # DON'T call the function yet > for value in main_call(): > print(value) # for testing > if state(value): > print("changing state") > state = TABLE[state] Above code works at some extent but after few minutes of running it returns and exists by printing 'None' on screen. Let me tell that main_call() generator spits out values like one more each second after going through some sort of heavy calculations(but even 50% of RAM is available) it exits by printing 'None' on screen. Is it some kind of recursion problem? From auriocus at gmx.de Fri Aug 26 04:42:19 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 26 Aug 2016 10:42:19 +0200 Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <219db485-4229-5ee4-6c2e-f6c01b9a8f9b@lucidity.plus.com> Message-ID: Am 26.08.16 um 09:53 schrieb Erik: > On 26/08/16 08:44, mlzarathustra at gmail.com wrote: >> Here's the key: >> >> $ python2 >> Python 2.7.10 ... >>>>> 1/2 >> 0 >>>>> >> >> $ python >> Python 3.5.1 ... >>>>> 1/2 >> 0.5 >>>>> 1//2 >> 0 >>>>> >> >> I read about this awhile ago, but it's not until it bites you that you >> remember fully. > > How is this related to your question? The example explicitly says Python > 2 and doesn't use the '//' operator. > It's related by the fact that a*b/c performs integer division (intended by the OP) which gives a different result than a*(b/c) (unintended by the OP). Floating point (as in Python 3) *also* may give a different result, but the deviation from the "true", i.e. mathematical value, is far less than with integer arithmetics. Christian From dfh at forestfield.co.uk Fri Aug 26 04:50:54 2016 From: dfh at forestfield.co.uk (dfh at forestfield.co.uk) Date: Fri, 26 Aug 2016 01:50:54 -0700 (PDT) Subject: Python 3 raising an error where Python 2 did not Message-ID: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> In a program I'm converting to Python 3 I'm examining a list of divisor values, some of which can be None, to find the first with a value greater than 1. Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> None > 1 False Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> None > 1 Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: NoneType() > int() I can live with that but I'm curious why it was decided that this should now raise an error. David Hughes Forestfield Software From dev at kou.li Fri Aug 26 04:55:54 2016 From: dev at kou.li (Kouli) Date: Fri, 26 Aug 2016 10:55:54 +0200 Subject: PEP 492: isn't the "await" redundant? Message-ID: Hello, recently, I have discovered Python's coroutines and enjoyed the whole asyncio system a lot. But I ask you to make me understand one thing in Python's coroutines design: why do we have to use "await" (or "yield from") in coroutines? Why can coroutines etc. not be used _from_coroutines_ (designated by 'async def') by a simple call-like syntax (i.e. without the 'await' keyword)? The same for "await with" and "await from". At most places a coroutine is referenced from another coroutine, it is referenced using "await". Couldn't it be avoided at theese places? This way, one would not have to differentiate between function and coroutine "call" from within a coroutine... Current syntax: async def work(x): await asyncio.sleep(x) def main(x): loop.run_until_complete(work(x)) Proposed syntax: async def work(x): asyncio.sleep(x) # compiler "adds" 'await' automatically when in 'async def' def main(x): loop.run_until_complete(work(x)) # compiler leaves as is when in 'def' Historically, generators were defined by using keyword 'yield' inside its definition. We now have explicit syntax with keyword 'async' so why should we use yet the additional keyword 'await'? I tried to show (a minor) example which would need the "leave as is" behavior inside 'async def', but I haven't found such a coroutine refence in examples. Should it be needed (please, tell me), then it requires a special syntax (at least for arguments - without arguments one can leave out the parentheses). Kouli From rosuav at gmail.com Fri Aug 26 05:07:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 19:07:55 +1000 Subject: Python 3 raising an error where Python 2 did not In-Reply-To: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> References: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> Message-ID: On Fri, Aug 26, 2016 at 6:50 PM, dfh at forestfield.co.uk wrote: > In a program I'm converting to Python 3 I'm examining a list of divisor values, some of which can be None, to find the first with a value greater than 1. > > Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> None > 1 > False > > Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> None > 1 > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: NoneType() > int() > > I can live with that but I'm curious why it was decided that this should now raise an error. Because it doesn't make sense to compare these things in this way. You can compare some things (eg integers and floats), but others don't usefully arrange themselves into any sort of order. It tends to lead to odd situations: rosuav at sikorsky:~$ python2 Python 2.7.12+ (default, Aug 4 2016, 20:04:34) [GCC 6.1.1 20160724] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 10 < "15" True >>> "15" < 20 False >>> "asdf" < (1,2,3) True >>> "asdf" > [1,2,3] True >>> rosuav at sikorsky:~$ python3 Python 3.6.0a4+ (default:4b64a049f451+, Aug 19 2016, 23:41:43) [GCC 6.1.1 20160802] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 10 < "15" Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'int' and 'str' >>> With Py3, it's simple and obvious: comparing integers and strings is an error. With Py2, there's the extremely odd rule that all integers are lower than all lists are lower than all strings are lower than all tuples - it's based on the name of the type. (Except for 'long', which appears to sort as though it were called 'int'. I think.) The Py2 behaviour allows you to get a consistent-but-meaningless sort order among disparate types, but it's a bug magnet, and it goes against Python's philosophy of telling you about problems right away. Py3 fixed quite a number of those kinds of issues (eg you can't combine byte strings and Unicode strings in Py3, yet you can in Py2), with the result that a number of data-dependent bugs in Py2 become instant exceptions in Py3. The simplest way to fix your code here is to explicitly provide a default. For instance: divisors = [None, 1, None, 2, 7, None, 1] greater_than_one = (div for div in divisors if (div or 0) > 1) The (div or 0) part means that any None will be treated as zero. (Also, zero will be treated as zero - be careful of this if you change the default, eg to 1.) At that point, all your comparisons will involve numbers, which have well-defined inequality comparisons. ChrisA From __peter__ at web.de Fri Aug 26 05:11:10 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Aug 2016 11:11:10 +0200 Subject: a *= b not equivalent to a = a*b References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> <76dfbd43-1bde-499c-8b35-f4064e5f395e@googlegroups.com> Message-ID: mlzarathustra at gmail.com wrote: > Yes, I just worked that out. It's the integer math that's the problem. > > I guess this has been fixed in python 3, but unfortunately it seems that > most people are still using python 2. Note that you can get Python 3's default behaviour in Python 2 with from __future__ import division at the beginning of the module. >>> 2/3 0 >>> from __future__ import division >>> 2/3 0.6666666666666666 From rosuav at gmail.com Fri Aug 26 05:18:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 19:18:24 +1000 Subject: PEP 492: isn't the "await" redundant? In-Reply-To: References: Message-ID: On Fri, Aug 26, 2016 at 6:55 PM, Kouli wrote: > recently, I have discovered Python's coroutines and enjoyed the whole > asyncio system a lot. But I ask you to make me understand one thing in > Python's coroutines design: why do we have to use "await" (or "yield > from") in coroutines? Why can coroutines etc. not be used > _from_coroutines_ (designated by 'async def') by a simple call-like > syntax (i.e. without the 'await' keyword)? Two reasons. One is that Python allows you to call any function and inspect its return value - async functions are no different, and they do return something. The other is that it makes yield points obvious. Consider this hypothetical function: async def get_user_data(id): await db.query("select name from users where id=?", (id,)) name = await db.fetchone()[0] # transaction handling elided return name Now, suppose we're trying to figure out what's going on. One good solid technique is what I call "IIDPIO debugging": If In Doubt, Print It Out. async def get_user_data(id): print("Starting g_u_d") q = db.query("select name from users where id=?", (id,)) print(q) await q f = db.fetchone()[0] print(f) name = await f # transaction handling still elided print("g_u_d: Returning %r" % name) return name It's completely obvious, here, that this function will call db.query, print stuff out, and then put itself on ice until the query's done, before attempting the fetch. If the call on the second line automatically put the function into waiting mode, this display would be impossible, and the wait points would be entirely implicit. (If you want implicit wait points, use threading, not async I/O.) It's also possible to wait for things that didn't come from function calls per se. For instance, the same database lookup could be implemented using an ORM, something like this: class Table: async def __getitem__(self, id): if id in self._cache: return self._cache[id] await db.query(...) data = await db.fetchone() self._cache[id] = self.make_object(...) return self._cache[id] Whether this is good code or not is up to you, but it's perfectly legal, and would be used as "users = Table(...); my_user = await users[123]". To allow that, Python absolutely has to allow arbitrary expressions to be waited on, not just function calls. Does that answer the question? ChrisA From __peter__ at web.de Fri Aug 26 05:37:44 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Aug 2016 11:37:44 +0200 Subject: Python 3 raising an error where Python 2 did not References: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> Message-ID: dfh at forestfield.co.uk wrote: > In a program I'm converting to Python 3 I'm examining a list of divisor > values, some of which can be None, to find the first with a value greater > than 1. > > Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] > on win32 Type "help", "copyright", "credits" or "license" for more > information. >>>> None > 1 > False > > Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64 > bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for > more information. >>>> None > 1 > Traceback (most recent call last): > File "", line 1, in > TypeError: unorderable types: NoneType() > int() > > I can live with that but I'm curious why it was decided that this should > now raise an error. Because allowing comparisons between object of arbitrary type does more harm than benefit? [Python 2] >>> sorted([1, "2", 3]) [1, 3, '2'] In the rare case where you actually want to compare different types you can make that explicit: [Python 3] >>> sorted([1, "2", 3], key=int) [1, '2', 3] >>> sorted([1, "2", 3], key=lambda x: (type(x).__name__, x)) [1, 3, '2'] >>> sorted([1, "2", 3], key=lambda x: (not isinstance(x, str), x)) ['2', 1, 3] >>> from itertools import product >>> for x, y in product([None, 1], repeat=2): ... print(x, ">", y, "-->", (x is not None, x) > (y is not None, y)) ... None > None --> False None > 1 --> False 1 > None --> True 1 > 1 --> False From marko at pacujo.net Fri Aug 26 05:41:38 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 26 Aug 2016 12:41:38 +0300 Subject: PEP 492: isn't the "await" redundant? References: Message-ID: <87h9a750kt.fsf@elektro.pacujo.net> Kouli : > We now have explicit syntax with keyword 'async' so why should we use > yet the additional keyword 'await'? This is an important question. > This way, one would not have to differentiate between function and > coroutine "call" from within a coroutine... You'd still need to remember to add the 'async' keyword all over the place. How about making *every* function *always* and async, unconditionally? That way *every* function would be an async and every function call would be an await. Marko From bc at freeuk.com Fri Aug 26 05:48:11 2016 From: bc at freeuk.com (BartC) Date: Fri, 26 Aug 2016 10:48:11 +0100 Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> Message-ID: On 26/08/2016 08:14, mlzarathustra at gmail.com wrote: > However, precedence wasn't the problem in this case, it was the type conversion. I think it was. I was puzzled as well. But apparently if you have: x * = expr That's like: x = x * (expr) # note the parentheses which may not always be the same as: x = x * expr as the latter may depends on how operators within expr relate to the '*'. In your example, "/" within expr has the same precedence as "*" is the "*" is done first not last. The type conversion may be another issue, but it doesn't explain why the results with the /same version/ of Python are different. -- Bartc From rosuav at gmail.com Fri Aug 26 05:49:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 19:49:57 +1000 Subject: PEP 492: isn't the "await" redundant? In-Reply-To: <87h9a750kt.fsf@elektro.pacujo.net> References: <87h9a750kt.fsf@elektro.pacujo.net> Message-ID: On Fri, Aug 26, 2016 at 7:41 PM, Marko Rauhamaa wrote: > How about making *every* function *always* and async, unconditionally? > That way *every* function would be an async and every function call > would be an await. If you want threading, you know where to find it. ChrisA From mlzarathustra at gmail.com Fri Aug 26 06:03:18 2016 From: mlzarathustra at gmail.com (mlz) Date: Fri, 26 Aug 2016 03:03:18 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> <76dfbd43-1bde-499c-8b35-f4064e5f395e@googlegroups.com> Message-ID: Aha. That's interesting. On Friday, August 26, 2016 at 2:11:32 AM UTC-7, Peter Otten wrote: > mlz wrote: > > > Yes, I just worked that out. It's the integer math that's the problem. > > > > I guess this has been fixed in python 3, but unfortunately it seems that > > most people are still using python 2. > > Note that you can get Python 3's default behaviour in Python 2 with > > from __future__ import division > > at the beginning of the module. > > >>> 2/3 > 0 > >>> from __future__ import division > >>> 2/3 > 0.6666666666666666 From marko at pacujo.net Fri Aug 26 06:08:11 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 26 Aug 2016 13:08:11 +0300 Subject: PEP 492: isn't the "await" redundant? References: <87h9a750kt.fsf@elektro.pacujo.net> Message-ID: <87d1kv4zck.fsf@elektro.pacujo.net> Chris Angelico : > On Fri, Aug 26, 2016 at 7:41 PM, Marko Rauhamaa wrote: >> How about making *every* function *always* and async, >> unconditionally? That way *every* function would be an async and >> every function call would be an await. > > If you want threading, you know where to find it. Ultimately, asyncio and multithreading might well merge. It will be difficult for a programmer to decide in the beginning of the design which way to go as the programming models are almost identical. Marko From mlzarathustra at gmail.com Fri Aug 26 06:10:01 2016 From: mlzarathustra at gmail.com (mlz) Date: Fri, 26 Aug 2016 03:10:01 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <39c80391-600d-4ca3-a80c-d06237f8c401@googlegroups.com> <927eadbc-722a-b03e-666a-9ba14ed5217e@lucidity.plus.com> Message-ID: Partly it's the layout, but mathematically speaking the two should be equal. (a*b)/c should equal a*(b/c) The fact that they're not is surprising, because python 2 so seamlessly supports big integers in a mathematically correct way. I consider such surprising behavior to be abstraction leak, which is probably why it has been fixed in python 3. -= m =- On Friday, August 26, 2016 at 12:54:02 AM UTC-7, Erik wrote: > On 26/08/16 08:14, mlz wrote: > > > > I was being facetious, but behind it is a serious point. Neither the APL nor the J languages use precedence even though their inventor, Ken Iverson, was a mathematician. > > > > That was to support functional programming dating back to the 1970's. > > Precedence is not the issue here anyway. Both '*' and '/' have the same > precedence. The two operations in your expressions have to be evaluated > in SOME order - either left-to-right or right-to-left. > > > The issue is twofold: > > Firstly, the compound assignments will always evaluate their RHS before > performing the assignment - this is not the same as operator precedence > - they have to, else what does Python pass to the function that knows > what the compound assignment means for that type (e.g. sequences may be > extended for '*=')? So for compound assignments there is always > effectively an implicit set of parentheses around the RHS compared to > the expression without the assignment operator (if you think of the > compound assignment as being a _part_ of the overall expression). > > Secondly, the way you have chosen to layout your code has fooled your > brain into thinking that the division is performed before the > multiplication. Python doesn't care that you jammed the sub-expression > together with no whitespace ;) > > E. From mlzarathustra at gmail.com Fri Aug 26 06:20:40 2016 From: mlzarathustra at gmail.com (mlz) Date: Fri, 26 Aug 2016 03:20:40 -0700 (PDT) Subject: a *= b not equivalent to a = a*b In-Reply-To: References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <219db485-4229-5ee4-6c2e-f6c01b9a8f9b@lucidity.plus.com> Message-ID: <5f70f1cd-2fd0-4671-ba0a-3657ff560a66@googlegroups.com> It's true that a*(b/c) yields fractions which would probably accrue accuracy errors depending on how those values are implemented. For example, it is possible to represent 1/3 internally as two numbers, numerator and denominator, thus avoiding the repeating decimal (or binimal, or whatever it's called). I believe there are languages that preserve exact accuracy in this way for rational fractions. I don't know if Python is one of them. On the other hand, (a*b)/c is safer since in this case (for the binomial coefficient) it always yields an integer. -= m =- On Friday, August 26, 2016 at 1:42:31 AM UTC-7, Christian Gollwitzer wrote: > Am 26.08.16 um 09:53 schrieb Erik: > > On 26/08/16 08:44, mlz wrote: > >> Here's the key: > >> > >> $ python2 > >> Python 2.7.10 ... > >>>>> 1/2 > >> 0 > >>>>> > >> > >> $ python > >> Python 3.5.1 ... > >>>>> 1/2 > >> 0.5 > >>>>> 1//2 > >> 0 > >>>>> > >> > >> I read about this awhile ago, but it's not until it bites you that you > >> remember fully. > > > > How is this related to your question? The example explicitly says Python > > 2 and doesn't use the '//' operator. > > > > It's related by the fact that a*b/c performs integer division (intended > by the OP) which gives a different result than a*(b/c) (unintended by > the OP). Floating point (as in Python 3) *also* may give a different > result, but the deviation from the "true", i.e. mathematical value, is > far less than with integer arithmetics. > > Christian From liik.joonas at gmail.com Fri Aug 26 06:37:30 2016 From: liik.joonas at gmail.com (Joonas Liik) Date: Fri, 26 Aug 2016 13:37:30 +0300 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On 26 August 2016 at 08:22, Frank Millman wrote: > "Peter Otten" wrote in message news:npn25e$s5n$1 at blaine.gmane.org... > > Frank Millman wrote: > >>> As you have to keep the "<", why bother? >> >> >> If you mean why don't I convert the '<' to '<', the answer is that I do >> - I just omitted to say so. However, explicit is better than implicit :-) > > >> Doesn't that make the XML document invalid or changes it in an >> irreversible way? How would you know whether > > > "" > > started out as > > "" > > or > > "<bar/>" > > ? > > I cheat ;-) > > It is *my* XML, and I know that I only use the offending characters inside > attributes, and attributes are the only place where double-quote marks are > allowed. > > So this is my conversion routine - > > lines = string.split('"') # split on attributes > for pos, line in enumerate(lines): > if pos%2: # every 2nd line is an attribute > lines[pos] = line.replace('<', '<').replace('>', '>') > return '"'.join(lines) > > Frank > > > -- > https://mail.python.org/mailman/listinfo/python-list or.. you could just escape all & as & before escaping the > and <, and do the reverse on decode From jussi.piitulainen at helsinki.fi Fri Aug 26 06:38:16 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Fri, 26 Aug 2016 13:38:16 +0300 Subject: a *= b not equivalent to a = a*b References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <219db485-4229-5ee4-6c2e-f6c01b9a8f9b@lucidity.plus.com> <5f70f1cd-2fd0-4671-ba0a-3657ff560a66@googlegroups.com> Message-ID: mlz writes: > It's true that a*(b/c) yields fractions which would probably accrue > accuracy errors depending on how those values are implemented. For > example, it is possible to represent 1/3 internally as two numbers, > numerator and denominator, thus avoiding the repeating decimal (or > binimal, or whatever it's called). I believe there are languages that > preserve exact accuracy in this way for rational fractions. I don't > know if Python is one of them. Python doesn't and isn't. Exact fractions are available in the standard library, but the notation 1/3 gives a floating point approximation. Floating point arithmetic is usually supported directly in the hardware in bounded space (64 bits per number). Exact rationals can acquire huge numerators and denominators surprisingly fast, slowing the computations down. It's a tradeoff. It's good to know about both. > On the other hand, (a*b)/c is safer since in this case (for the > binomial coefficient) it always yields an integer. When you compute them in that specific way, walking the numerator down and denominator up. From rosuav at gmail.com Fri Aug 26 06:48:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 20:48:59 +1000 Subject: PEP 492: isn't the "await" redundant? In-Reply-To: References: Message-ID: (Did you mean to send this to the list? I hope so; I'm replying to the list.) On Fri, Aug 26, 2016 at 8:30 PM, Milan Kr?m?? wrote: >> Two reasons. One is that Python allows you to call any function and >> inspect its return value - async functions are no different, and they >> do return something. The other is that it makes yield points obvious. >> Consider this hypothetical function: > > The return value could be (with proposed syntax) inspected as well. > The yield point is often visible just from the function you are using: > async.sleep() vs. sleep() etc. Not sure how it could be inspected - only the resulting value could. You couldn't see the Awaitable that gets returned in between. Depending on the situation, that could be extremely useful. >> Now, suppose we're trying to figure out what's going on. One good >> solid technique is what I call "IIDPIO debugging": If In Doubt, Print >> It Out. > > Yes, the 'q = ...; print(q); await q' is a use case to introduce await. > >> >> async def get_user_data(id): >> print("Starting g_u_d") >> q = db.query("select name from users where id=?", (id,)) >> print(q) >> await q > >> "users = Table(...); my_user = await users[123]" > > An interesting example. But the 'my_user = await users[123]' must have > appeared inside 'async def', > so would be written as 'my_user = users[123]' in "my" syntax... So what that really means is that, the instant something hits an awaitable, the entire thread gets paused. That's a perfectly reasonable way of thinking... if you have a concept of threads that get explicitly spun off. Otherwise, it's a bit tricky, because there's no easy way to implement the boundary - the point at which the awaitable gets added to a queue somewhere (ie the top of the event loop). > Chris, thank you for such a long reply. I feel being much more > reconciled with the "verbose" syntax ;-) No probs, happy to help out. ChrisA From rosuav at gmail.com Fri Aug 26 06:56:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Aug 2016 20:56:46 +1000 Subject: a *= b not equivalent to a = a*b In-Reply-To: <5f70f1cd-2fd0-4671-ba0a-3657ff560a66@googlegroups.com> References: <4e541044-ce56-4acd-8d84-97adff49b163@googlegroups.com> <219db485-4229-5ee4-6c2e-f6c01b9a8f9b@lucidity.plus.com> <5f70f1cd-2fd0-4671-ba0a-3657ff560a66@googlegroups.com> Message-ID: On Fri, Aug 26, 2016 at 8:20 PM, mlz wrote: > I believe there are languages that preserve exact accuracy in this way for rational fractions. I don't know if Python is one of them. It is, but only if you explicitly request it (due to the performance impact). Just import it: #!/usr/bin/python2 import fractions import sys FAIL= True if len(sys.argv)>1 else False def bin(n,k): rs=1 k=min(k,n-k) n = fractions.Fraction(n) for i in range(1,k+1): if FAIL: rs *= (n-(i-1))/i # these should be the same, else: rs = rs * (n-(i-1))/i # but apparently are not return rs for n in range(10): for k in range(n+1): print bin(n,k), print'' That's the only change you need. The arithmetic will then be done with ratios of integers, and it'll be exact. ChrisA From gandalf at shopzeus.com Fri Aug 26 06:59:03 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Fri, 26 Aug 2016 12:59:03 +0200 Subject: datetime, time zone and xs:datetime Message-ID: <2eba37e5-f041-2bbc-76c6-39a3fc2b572d@shopzeus.com> Hello, I'm having problems finding the "preferrably one" way to convert a datetime instance into an xs:datetime string. Here is an example datetime instance with is format: dt = datetime.now() print(dt.isoformat()) # prints "2016-08-26 12:41:13.426081+02:00" print(dt.strftime('%Y-%m-%dT%H:%M:%S%z')) # prints "2016-08-26T12:41:13+0200" The main flaw is that the xs:datetime format requires a colon. (See http://books.xmlschemata.org/relaxng/ch19-77049.html ) The %z format does not contain that. I'm not sure when and how the %z could be used out of the box, but probably there are datetime formats when they are needed. Of course, I can do a workaround: def datetime_to_xsd_timestamp(dt): s = dt.strftime('%Y-%m-%dT%H:%M:%S') tzs = dt.strftime("%z") if tzs: s += tzs[:3] + ":" + tzs[3:] return s But this does not seem to be the right way to do it. Could we have datetime.xsdformat()? Or maybe extra format characters in strftime that represent the hour and the minute part of the time zone offset? Or maybe I'm not aware of some method in the standard library that converts to this format back and forth? Thanks, Laszlo From jon+usenet at unequivocal.eu Fri Aug 26 07:28:54 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Fri, 26 Aug 2016 11:28:54 -0000 (UTC) Subject: Does This Scare You? References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On 2016-08-24, Chris Angelico wrote: > On Thu, Aug 25, 2016 at 7:00 AM, eryk sun wrote: >> I discovered why "Logs/con.txt" isn't working right in Windows 7, >> while "Logs/nul.txt" does get redirected correctly to r"\\.\nul". >> Prior to Windows 8 the console doesn't use an NT device, so the base >> API has a function named BaseIsThisAConsoleName that looks for names >> such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and >> returns either "CONIN$" or "CONOUT$" if there's a match. A match for >> just "CON" maps to one or the other of the latter depending on whether >> read or write access is desired. > > See? This is why *even after I tested it* I wasn't sure I was right! > The rules are... complicated. Hence my doubts about this function - it has almost no chance of not being broken, potentially leading to whatever code was using it having security holes. Perhaps it should be deprecated or at least attract a large caveat in the documentation. From frank at chagford.com Fri Aug 26 09:10:17 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 26 Aug 2016 15:10:17 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Joonas Liik" wrote in message news:CAB1GNpQnJDENaA-GZgt0TbcvWjaKNgD3YRoiXgyY+Mim7fw0zQ at mail.gmail.com... > On 26 August 2016 at 08:22, Frank Millman wrote: > > > > So this is my conversion routine - > > > > lines = string.split('"') # split on attributes > > for pos, line in enumerate(lines): > > if pos%2: # every 2nd line is an attribute > > lines[pos] = line.replace('<', '<').replace('>', '>') > > return '"'.join(lines) > > > > or.. you could just escape all & as & before escaping the > and <, > and do the reverse on decode > Thanks, Joonas, but I have not quite grasped that. Would you mind explaining how it would work? Just to confirm that we are talking about the same thing - This is not allowed - '' [A] >>> import xml.etree.ElementTree as etree >>> x = '' >>> y = etree.fromstring(x) Traceback (most recent call last): File "", line 1, in File "C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py", line 1320, in XML parser.feed(text) xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 17 You have to escape it like this - '' [B] >>> x = '' >>> y = etree.fromstring(x) >>> y.find('fld').get('name') '' >>> I want to convert the string from [B] to [A] for editing, and then back to [B] before saving. Thanks Frank From rk-list at simple-is-better.org Fri Aug 26 10:02:13 2016 From: rk-list at simple-is-better.org (Roland Koebler) Date: Fri, 26 Aug 2016 16:02:13 +0200 Subject: [Bulk] Re: Alternatives to XML? In-Reply-To: References: Message-ID: <20160826140213.GA17438@localhost> Hi, > It is *my* XML, and I know that I only use the offending characters inside > attributes, and attributes are the only place where double-quote marks are > allowed. > > So this is my conversion routine - > > lines = string.split('"') # split on attributes > for pos, line in enumerate(lines): > if pos%2: # every 2nd line is an attribute > lines[pos] = line.replace('<', '<').replace('>', '>') > return '"'.join(lines) OMG! So, you have a fileformat, which looks like XML, but actually isn't XML, and will break if used with some "real" XML. Although I don't like XML, if you want XML, you should follow Chris advice: On Thu, Aug 25, 2016 at 09:40:03PM +1000, Chris Angelico wrote: > just make sure it's always valid XML, rather > than some "XML-like" file structure. So, please: - Don't try to write your own (not-quite-)XML-parser. - Read how XML-files work. - Read https://docs.python.org/3/library/xml.html and https://pypi.python.org/pypi/defusedxml/ - Think what you have done. - Use a sensible XML-parser/dumper. This should escape most special- characters for you (at least: < > & " '). Roland From dfh at forestfield.co.uk Fri Aug 26 10:06:28 2016 From: dfh at forestfield.co.uk (dfh at forestfield.co.uk) Date: Fri, 26 Aug 2016 07:06:28 -0700 (PDT) Subject: Python 3 raising an error where Python 2 did not In-Reply-To: References: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> Message-ID: Thanks for the replies. My example seems to be from the fairly harmless end of a wedge of behaviours that are being handled much more sensibly in Python 3. From liik.joonas at gmail.com Fri Aug 26 10:58:46 2016 From: liik.joonas at gmail.com (Joonas Liik) Date: Fri, 26 Aug 2016 17:58:46 +0300 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On 26 August 2016 at 16:10, Frank Millman wrote: > "Joonas Liik" wrote in message > news:CAB1GNpQnJDENaA-GZgt0TbcvWjaKNgD3YRoiXgyY+Mim7fw0zQ at mail.gmail.com... > >> On 26 August 2016 at 08:22, Frank Millman wrote: >> > >> > So this is my conversion routine - >> > >> > lines = string.split('"') # split on attributes >> > for pos, line in enumerate(lines): >> > if pos%2: # every 2nd line is an attribute >> > lines[pos] = line.replace('<', '<').replace('>', '>') >> > return '"'.join(lines) >> > >> >> or.. you could just escape all & as & before escaping the > and <, >> and do the reverse on decode >> > > Thanks, Joonas, but I have not quite grasped that. > > Would you mind explaining how it would work? > > Just to confirm that we are talking about the same thing - > > This is not allowed - '' [A] > >>>> import xml.etree.ElementTree as etree >>>> x = '' >>>> y = etree.fromstring(x) > > Traceback (most recent call last): > File "", line 1, in > File > "C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py", > line 1320, in XML > parser.feed(text) > xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, > column 17 > > You have to escape it like this - '' > [B] > >>>> x = '' >>>> y = etree.fromstring(x) >>>> y.find('fld').get('name') > > '' >>>> >>>> > > I want to convert the string from [B] to [A] for editing, and then back to > [B] before saving. > > Thanks > > Frank > > > -- > https://mail.python.org/mailman/listinfo/python-list something like.. (untested) def escape(untrusted_string): ''' Use on the user provided strings to render them inert for storage escaping & ensures that the user cant type sth like '>' in source and have it magically decode as '>' ''' return untrusted_string.replace("&","&").replace("<", "<").replace(">", ">") def unescape(escaped_string): '''Once the user string is retreived from storage use this function to restore it to its original form''' return escaped_string.replace("<","<").replace(">", ">").replace("&", "&") i should note tho that this example is very ad-hoc, i'm no xml expert just know a bit about xml entities. if you decide to go this route there are probably some much better tested functions out there to escape text for storage in xml documents. From liik.joonas at gmail.com Fri Aug 26 11:00:30 2016 From: liik.joonas at gmail.com (Joonas Liik) Date: Fri, 26 Aug 2016 18:00:30 +0300 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: On 26 August 2016 at 17:58, Joonas Liik wrote: > On 26 August 2016 at 16:10, Frank Millman wrote: >> "Joonas Liik" wrote in message >> news:CAB1GNpQnJDENaA-GZgt0TbcvWjaKNgD3YRoiXgyY+Mim7fw0zQ at mail.gmail.com... >> >>> On 26 August 2016 at 08:22, Frank Millman wrote: >>> > >>> > So this is my conversion routine - >>> > >>> > lines = string.split('"') # split on attributes >>> > for pos, line in enumerate(lines): >>> > if pos%2: # every 2nd line is an attribute >>> > lines[pos] = line.replace('<', '<').replace('>', '>') >>> > return '"'.join(lines) >>> > >>> >>> or.. you could just escape all & as & before escaping the > and <, >>> and do the reverse on decode >>> >> >> Thanks, Joonas, but I have not quite grasped that. >> >> Would you mind explaining how it would work? >> >> Just to confirm that we are talking about the same thing - >> >> This is not allowed - '' [A] >> >>>>> import xml.etree.ElementTree as etree >>>>> x = '' >>>>> y = etree.fromstring(x) >> >> Traceback (most recent call last): >> File "", line 1, in >> File >> "C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py", >> line 1320, in XML >> parser.feed(text) >> xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, >> column 17 >> >> You have to escape it like this - '' >> [B] >> >>>>> x = '' >>>>> y = etree.fromstring(x) >>>>> y.find('fld').get('name') >> >> '' >>>>> >>>>> >> >> I want to convert the string from [B] to [A] for editing, and then back to >> [B] before saving. >> >> Thanks >> >> Frank >> >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > > something like.. (untested) > > def escape(untrusted_string): > ''' Use on the user provided strings to render them inert for storage > escaping & ensures that the user cant type sth like '>' in > source and have it magically decode as '>' > ''' > return untrusted_string.replace("&","&").replace("<", > "<").replace(">", ">") > > def unescape(escaped_string): > '''Once the user string is retreived from storage use this > function to restore it to its original form''' > return escaped_string.replace("<","<").replace(">", > ">").replace("&", "&") > > i should note tho that this example is very ad-hoc, i'm no xml expert > just know a bit about xml entities. > if you decide to go this route there are probably some much better > tested functions out there to escape text for storage in xml > documents. you might want to un-wrap that before testing tho.. no idea why my messages get mutilated like that :( (sent using gmail, maybe somebody can comment on that?) From Dean.Gonzales at amd.com Fri Aug 26 11:21:54 2016 From: Dean.Gonzales at amd.com (Gonzales, Dean) Date: Fri, 26 Aug 2016 15:21:54 +0000 Subject: wxPython Cookbook In-Reply-To: References: Message-ID: Very cool! Thanks for doing this. I can't wait to dig into your cookbook. Regards, Dean Gonzales -----Original Message----- From: Python-announce-list [mailto:python-announce-list-bounces+dean.gonzales=amd.com at python.org] On Behalf Of Mike Driscoll Sent: Wednesday, August 24, 2016 12:57 PM To: python-announce-list at python.org Subject: ANN: wxPython Cookbook Hi, Several years ago, the readers of my popular Python blog asked me to take some of my articles and turn them into a cookbook on wxPython. I have finally decided to do just that. I am including over 50 recipes that I am currently editing to make them more consistent and updating them to be compatible with the latest versions of wxPython. I currently have nearly 300 pages of content! If you'd like to check out the funding campaign for the book, you can find it here: https://www.kickstarter.com/projects/34257246/wxpython-cookbook/ Thanks, Mike -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/ From frank at chagford.com Fri Aug 26 11:23:10 2016 From: frank at chagford.com (Frank Millman) Date: Fri, 26 Aug 2016 17:23:10 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: "Joonas Liik" wrote in message news:CAB1GNpTP0GD4s4kx07r1ujRNuXtOij4vF5uNYE1cFr_Y0xvi1g at mail.gmail.com... > something like.. (untested) def escape(untrusted_string): ''' Use on the user provided strings to render them inert for storage escaping & ensures that the user cant type sth like '>' in source and have it magically decode as '>' ''' return untrusted_string.replace("&","&").replace("<", "<").replace(">", ">") def unescape(escaped_string): '''Once the user string is retreived from storage use this function to restore it to its original form''' return escaped_string.replace("<","<").replace(">", ">").replace("&", "&") > i should note tho that this example is very ad-hoc, i'm no xml expert just > know a bit about xml entities. if you decide to go this route there are > probably some much better tested functions out there to escape text for > storage in xml documents. Thanks very much, Joonas. I understand now, and it seems to work fine. As a bonus, I can now include '&' in my attributes in the future if the need arises. Much appreciated. Frank From marko at pacujo.net Fri Aug 26 13:48:00 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 26 Aug 2016 20:48:00 +0300 Subject: Alternatives to XML? References: Message-ID: <87y43j2zhr.fsf@elektro.pacujo.net> "Frank Millman" : > "Joonas Liik" wrote in message > news:CAB1GNpTP0GD4s4kx07r1ujRNuXtOij4vF5uNYE1cFr_Y0xvi1g at mail.gmail.com... >> i should note tho that this example is very ad-hoc, i'm no xml expert >> just know a bit about xml entities. if you decide to go this route >> there are probably some much better tested functions out there to >> escape text for storage in xml documents. > > Thanks very much, Joonas. > > I understand now, and it seems to work fine. > > As a bonus, I can now include '&' in my attributes in the future if the > need arises. > > Much appreciated. XML attributes are ridiculously complicated. From the standard: Before the value of an attribute is passed to the application or checked for validity, the XML processor MUST normalize the attribute value by applying the algorithm below, or by using some other method such that the value passed to the application is the same as that produced by the algorithm. 1. All line breaks MUST have been normalized on input to #xA as described in 2.11 End-of-Line Handling, so the rest of this algorithm operates on text normalized in this way. 2. Begin with a normalized value consisting of the empty string. 3. For each character, entity reference, or character reference in the unnormalized attribute value, beginning with the first and continuing to the last, do the following: * For a character reference, append the referenced character to the normalized value. * For an entity reference, recursively apply step 3 of this algorithm to the replacement text of the entity. * For a white space character (#x20, #xD, #xA, #x9), append a space character (#x20) to the normalized value. * For another character, append the character to the normalized value. If the attribute type is not CDATA, then the XML processor MUST further process the normalized attribute value by discarding any leading and trailing space (#x20) characters, and by replacing sequences of space (#x20) characters by a single space (#x20) character. Note that if the unnormalized attribute value contains a character reference to a white space character other than space (#x20), the normalized value contains the referenced character itself (#xD, #xA or #x9). This contrasts with the case where the unnormalized value contains a white space character (not a reference), which is replaced with a space character (#x20) in the normalized value and also contrasts with the case where the unnormalized value contains an entity reference whose replacement text contains a white space character; being recursively processed, the white space character is replaced with a space character (#x20) in the normalized value. All attributes for which no declaration has been read SHOULD be treated by a non-validating processor as if declared CDATA. It is an error if an attribute value contains a reference to an entity for which no declaration has been read. Following are examples of attribute normalization. Given the following declarations: the attribute specifications in the left column below would be normalized to the character sequences of the middle column if the attribute a is declared NMTOKENS and to those of the right columns if a is declared CDATA. ================================================================= Attribute specification: a=" xyz" a is NMTOKENS: x y z a is CDATA: #x20 #x20 x y z ================================================================= Attribute specification: a="&d;&d;A&a; &a;B&da;" a is NMTOKENS: A #x20 B a is CDATA: #x20 #x20 A #x20 #x20 #x20 B #x20 #x20 ================================================================= Attribute specification: a=" A B " a is NMTOKENS: #xD #xD A #xA #xA B #xD #xA a is CDATA: #xD #xD A #xA #xA B #xD #xA ================================================================= Note that the last example is invalid (but well-formed) if a is declared to be of type NMTOKENS. Marko From rk-list at simple-is-better.org Fri Aug 26 15:27:39 2016 From: rk-list at simple-is-better.org (Roland Koebler) Date: Fri, 26 Aug 2016 21:27:39 +0200 Subject: Alternatives to XML? In-Reply-To: References: Message-ID: <20160826192739.GA14849@localhost> Hi, after reading the mails of this thread, I would recommend one of the following ways: 1. Use a computer-readable format and some small editor for humans. The file-format could then be very simple -- I would recommend JSON. Or some kind of database (e.g. SQLite). For humans, you would have to write a (small/nice) graphical editor, where they can build the logic e.g. by clicking on buttons. This can also work for non-programmers, since the graphical editor can be adapted to the indended users, give help, run wizards etc. or: 2. Use a human-readable format and a parser for the computer. Then, the fileformat should be optimized for human readability. I would recommend a restricted subset of Python. This is much more readable/writeable for humans than any XML/JSON/YAML. And you could even add a graphical editor to further support non-programming-users. The computer would then need a special parser. But by using Python-expressions (only eval, no exec) and a parser for flow control (if/else/for/...) and assignments, this is not too much work and is good for many applications. I've written such a parser incl. some kind of (pseudo-)sandbox [2] for my template-engine "pyratemp" [1], and I've also used it for small user-created-procedures. [1] http://www.simple-is-better.org/template/pyratemp.html [2] It's not a real sandbox -- it's secured only by restricting the available commands. If you add unsafe commands to the pseudo-sandbox (e.g. Pythons "open"), the user can do bad things. But without manually adding unsafe commands, I don't know any way to get out of this pseudo-sandbox. And if you really need a sandbox which is more powerful than my pseudo-sandbox, you may want to have a look at the sandbox of PyPy. Trying to use a format which is both directly computer-readable (without a special parser) and well human readable never really works well in my experience. Then, you usually have to manually read/write/edit some kind of parse-tree, which is usually much harder to read/write than code. But if you want to do this, I recommend LISP ;). (By the way: If I did understand your mails correctly, your program would probably break if someone edits the XML-files manually, since you're using some kind of XML-like-fileformat with many non-intuitive assumptions.) Roland PS: On Wed, Aug 24, 2016 at 04:58:54PM +0200, Frank Millman wrote: > Here is a JSON version - > > { > "case": { > "compare": { > "-src": "_param.auto_party_id", > "-op": "is_not", > "-tgt": "$None", > "case": { > "on_insert": { > "auto_gen": { "-args": "_param.auto_party_id" } > }, > "not_exists": { > "literal": { "-value": "" } > } > } > } > } > } I think this is not really good. In JSON, you also have lists, and in this case, it would probably be better to use some lists instead of dicts, e.g.: [ ["if", ["_param.auto_party_id", "is not", "None"], ["if", ["on_insert"], ["set", "value", ["call", "auto_gen", "_param.auto_party_id"]]], ["elif", ["not_exists"], ["set", "value", "''"]] ] ] I think this is much more readable than your XML-code and the auto-converted JSON. And it's even less ambigious. (How do you distinguish between the variable _param.auto_party_id and the string "_param.auto_party_id" in your XML-example?) From tjreedy at udel.edu Fri Aug 26 15:52:20 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 26 Aug 2016 15:52:20 -0400 Subject: Python 3 raising an error where Python 2 did not In-Reply-To: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> References: <46d12e16-af23-4e14-9cbb-453e89f2b8d5@googlegroups.com> Message-ID: On 8/26/2016 4:50 AM, dfh at forestfield.co.uk wrote: > In a program I'm converting to Python 3 I'm examining a list of divisor values, some of which can be None, to find the first with a value greater than 1. > > Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> None > 1 > False The response 'False' is not a defined part of the language and no program should depend on it. An earlier version of CPython and another implementation of Python 1 or 2 might have said 1 or True. Then your 'first divisor > 1' would be None. Not good. -- Terry Jan Reedy From ycm.jason at gmail.com Fri Aug 26 16:19:47 2016 From: ycm.jason at gmail.com (ycm.jason at gmail.com) Date: Fri, 26 Aug 2016 13:19:47 -0700 (PDT) Subject: multiprocessing.pool.Pool.map should take more than one iterable Message-ID: Hello all, The official doc describes `Pool.map(func, iterable[, chunksize])` as "A parallel equivalent of the map() built-in function.". Since the function signature of `map` is `map(function, iterable, ...)`, I hereby suggest that `Pool.map` should change its function signature to `Pool.map(function, iterable, ... [, chunksize])`. This will bring true equivalent to these functions. Tell me what you think. Pool.map: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.map map: https://docs.python.org/3/library/functions.html#map Jason From ddream.merchantt at gmail.com Fri Aug 26 18:33:32 2016 From: ddream.merchantt at gmail.com (ddream.merchantt at gmail.com) Date: Fri, 26 Aug 2016 15:33:32 -0700 (PDT) Subject: Python script for searching variable strings between two constant strings Message-ID: import re infile = open('document.txt','r') outfile= open('output.txt','w') copy = False for line in infile: if line.strip() == "--operation():": bucket = [] copy = True elif line.strip() == "StartOperation": for strings in bucket: outfile.write( strings + ',') for strings in bucket: outfile.write('\n') copy = False elif copy: bucket.append(line.strip() -------------------------------------- CSV format is like this: id, name, poid, error 5896, AutoAuthOSUserSubmit, 900105270, 0x4002 My log file has several sections starting with ==== START ==== and ending with ==== END ====. I want to extract the string between --operation(): and StartOperation. For example, AutoAuthOSUserSubmit. I also want to extract the poid value from line poid: 900105270, poidLen: 9. Finally, I want to extract the return value, e.g 0x4002 if Roll back all updates is found after it. I am not even able to extract point the original text if Start and End are not on the same line. How do I go about doing that? This is a sample LOG extract with two paragraphs: -- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== START ====** open file /ecas/public/onsite-be/config/timer.conf failed INFO 08/24/16 02:07:56 salt1be-d1-ap(**5896**/0) main.c(780*****):--operation(): AutoAuthOSUserSubmit. StartOperation***** INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) main.c(784):--Client Information: Request from host 'malt-d1-wb' process id 12382. DEBUG 08/24/16 02:07:56 salt1be-d1-ap(5896/0) TOci.cc(571):FetchServiceObjects: ServiceCert.sql DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsserviceagent.cpp(517):Generate Certificate 2: c1cd00d5c3de082360a08730fef9cd1d DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) junk.c(1373):GenerateWebPin : poid: **900105270**, poidLen: 9 DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) junk.c(1408):GenerateWebPin : pinStr DEBUG 08/24/16 02:07:56 salt1be-d1-ap(5896/0) uaadapter_vasco_totp.c(275):UAVascoTOTPImpl.close() -- Releasing Adapter Context DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsenterprise.cpp(288):VSEnterprise::Engage returns 0x4002 - Unknown error code **(0x4002)** ERROR 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsautoauth.cpp(696):OSAAEndUserEnroll: error occurred. **Roll back** all updates! INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) uaotptokenstoreqmimpl.cpp(199):Close token store INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) main.c(990):-- EndOperation -- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== END ====** OPERATION = AutoAuthOSUserSubmit, rc = 0x0 (0) SYSINFO Elapse = 0.687, Heap = 1334K, Stack = 64K From dvl at psu.edu Fri Aug 26 19:58:43 2016 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Fri, 26 Aug 2016 19:58:43 -0400 Subject: Is duck-typing misnamed? Message-ID: <1472255923l.39846110l.0l@psu.edu> "If it walks like a duck, quacks like a duck,... " so there is indeed precedence for this so-called 'duck typing' but wouldn't it be more Pythonic to call this 'witch typing'? "How do you know she is a witch?" "She looks like one." etc. I do grant that ultimately, the duck does come into play, since the witch weighs the same as a duck. Roger Christman Electrical Engineering and Computer Science Pennsylvania State University From steve+python at pearwood.info Fri Aug 26 21:44:55 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 27 Aug 2016 11:44:55 +1000 Subject: Python script for searching variable strings between two constant strings References: Message-ID: <57c0f098$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sat, 27 Aug 2016 08:33 am, ddream.merchantt at gmail.com wrote: > My log file has several sections starting with ==== START ==== and ending > with ==== END ====. Um. Is this relevant? Are you saying that you only wish to search the file between those lines, and ignore anything outside of them? If the file looks like: xxxx xxxx xxxx --operation(): AutoAuthOSUserSubmit StartOperation xxxx xxxx ==== START ==== xxxx xxxx xxxx ==== END ==== xxxx xxxx xxxx --operation(): AutoAuthOSUserSubmit StartOperation xxxx do you expect to say that nothing is found? I'm going to assume that you wouldn't have mentioned this if it wasn't important, so let's start by filtering out everything outside of ===START=== and ===END=== sections. For that, we want a filter that swaps between "ignore these lines" and "search these lines" depending on whether you are inside or outside of a START...END section. We'll use regular expressions for matching. import re START = r''' (?x) (?# verbose mode) ={2,} (?# two or more equal signs) \s* (?# any amount of whitespace) START (?# the literal word START in uppercase) \s* (?# more optional whitespace) ={2,} (?# two or more equal signs) $ (?# end of the line) ''' END = r'={2,}\s*END\s*={2,}$' # Similar to above, without verbose mode. START = re.compile(START) END = re.compile(END) def filter_sections(lines): outside = True for line in lines: line = line.strip() # ignore leading and trailing whitespace if outside: # ignore all lines until we see START if re.match(START, line): outside = False else: pass # just ignore the line else: # pass on every line until we see END if re.match(END, line): outside = True else: yield line Now you need to test that this does what you expect: with("mylogfile.log") as f: for line in filter_sections(f): print(line) should print *only* the lines between the START and END lines. Once you are satisfied that this works correctly, move on to the next part: extracting the relevant information from each line. There are three things you wish to look for, so you want three regular expressions. I'm not being paid for this, so here's one, the other two are up to you: OPERATION = r''' (?x) (?# verbose mode) --operation\(\): (?# literal string) \s* (?# optional whitespace) (.*) (?# anything at all, in a group) \s* (?# more optional whitespace) StartOperation (?# another literal string) .*?$ (?# ignore everything to the end of the line) ''' OPERATION = re.compile(OPERATION) FOO = ... # match second thing, similar to above BAR = ... # match third thing Now let's extract the data we want: def extract(lines): for line in lines: line = line.strip() mo = (re.match(OPERATION, line) or re.match(FOO, line) or re.match(BAR, line) ) if mo: yield mo.groups(0) with open('mylogfile.log') as f: for match in extract(filter_sections(f)): print(match) By the way, the above code is untested. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Fri Aug 26 22:17:48 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 27 Aug 2016 14:17:48 +1200 Subject: PEP 492: isn't the "await" redundant? In-Reply-To: <87h9a750kt.fsf@elektro.pacujo.net> References: <87h9a750kt.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > How about making *every* function *always* and async, unconditionally? > That way *every* function would be an async and every function call > would be an await. 1. Many people regard it as a feature that you can see where potential suspension points are. 2. Doing this would require massive changes to the core interpreter and all C extensions. (The original version of Stackless Python did something similar, and it was judged far too big a change to incorporate into CPython.) -- Greg From eryksun at gmail.com Sat Aug 27 00:11:06 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 27 Aug 2016 04:11:06 +0000 Subject: Does This Scare You? In-Reply-To: References: <439fbfc4-7ea6-4605-837f-9c138d295e34@googlegroups.com> <1471870610.577230.702357777.12F5FE22@webmail.messagingengine.com> Message-ID: On Fri, Aug 26, 2016 at 11:28 AM, Jon Ribbens wrote: > On 2016-08-24, Chris Angelico wrote: >> On Thu, Aug 25, 2016 at 7:00 AM, eryk sun wrote: >>> I discovered why "Logs/con.txt" isn't working right in Windows 7, >>> while "Logs/nul.txt" does get redirected correctly to r"\\.\nul". >>> Prior to Windows 8 the console doesn't use an NT device, so the base >>> API has a function named BaseIsThisAConsoleName that looks for names >>> such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and >>> returns either "CONIN$" or "CONOUT$" if there's a match. A match for >>> just "CON" maps to one or the other of the latter depending on whether >>> read or write access is desired. >> >> See? This is why *even after I tested it* I wasn't sure I was right! >> The rules are... complicated. > > Hence my doubts about this function - it has almost no chance of not > being broken, potentially leading to whatever code was using it having > security holes. Perhaps it should be deprecated or at least attract a > large caveat in the documentation. I agree that the docs need a warning that the behavior of paths containing legacy DOS device names is inconsistent between versions of Windows and that the rules that Windows uses aren't explicitly documented. There's another inconsistency if Python is run under ReactOS (and probably Wine, too). The ReactOS equivalent to BaseIsThisAConsoleName is IntCheckForConsoleFileName, which doesn't have the (buggy) speed hack that Windows uses to gate the more expensive call to RtlIsDosDeviceName_U: http://preview.tinyurl.com/reactos-console-c-71210 Also note the comment on line 354 that "C:\some_path\CONIN$" should open the console. This highlights yet another problem with the current pathlib check: it doesn't reserve the names "CONIN$" and "CONOUT$". Actually, the comment is technically wrong for versions prior to Windows 8. The old implementation of RtlIsDosDeviceName_U only checks for "CON", not "CONIN$" or "CONOUT$". The latter two devices aren't inherited from DOS. They were added to the Windows API to allow opening console handles with both read and write access, e.g. to be able to call WriteConsoleInput and ReadConsoleOutput. "CON" doesn't allow this. However, the base API does allow opening plain "CONIN$" and "CONOUT$" without the "\\.\" device namespace prefix, so those names are at least reserved for the current directory. The problem is more encompassing for Windows 8+, which has a real console device. It no longer calls BaseIsThisAConsoleName to redirect CreateFile to OpenConsoleW. Instead it passes the regular NT paths r"\??\CON", r"\??\CONIN$", or r"\??\CONOUT$" to the NtCreateFile system call, which resolves the object manager symbolic links respectively to r"\Device\ConDrv\Console", r"\Device\ConDrv\CurrentIn", and r"\Device\ConDrv\CurrentOut". As part of the redesign, the base API moved the check for "CONIN$" and "CONOUT$" into the NT runtime library function RtlpIsDosDeviceName_Ustr. Now "CON", "CONIN$", and "CONOUT$" are reserved in every directory, just like how it's always worked for NUL, AUX, PRN, COM1-9, and LPT1-9. For example: Windows 10 >>> print(os.path.abspath('C:/Temp/conout$ : spam . eggs')) \\.\conout$ Windows 7 >>> print(os.path.abspath('C:/Temp/conout$ : spam . eggs')) C:\Temp\conout$ : spam . eggs From marko at pacujo.net Sat Aug 27 00:59:02 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 27 Aug 2016 07:59:02 +0300 Subject: PEP 492: isn't the "await" redundant? References: <87h9a750kt.fsf@elektro.pacujo.net> Message-ID: <87r39a3izt.fsf@elektro.pacujo.net> Gregory Ewing : > Marko Rauhamaa wrote: >> How about making *every* function *always* and async, >> unconditionally? That way *every* function would be an async and >> every function call would be an await. > > 1. Many people regard it as a feature that you can see where > potential suspension points are. Yeah, it's actually crucial since every suspension point will also require consideration for alternate stimuli like a possible cancellation or timeout. Marko From frank at chagford.com Sat Aug 27 01:29:13 2016 From: frank at chagford.com (Frank Millman) Date: Sat, 27 Aug 2016 07:29:13 +0200 Subject: [Bulk] Re: Alternatives to XML? In-Reply-To: <20160826140213.GA17438@localhost> References: <20160826140213.GA17438@localhost> Message-ID: "Roland Koebler" wrote in message news:20160826140213.GA17438 at localhost... > Hi, > OMG! > So, you have a fileformat, which looks like XML, but actually isn't XML, > and will break if used with some "real" XML. I don't want to pursue this too much further, but I would like to point out that my format is genuine XML. I can back that up with two pieces of supporting evidence - 1. After going through the unescape > gui > edit > back_to_application > escape routine, I validate the result before saving it - try: return etree.fromstring(value, parser=self.parser) except (etree.XMLSyntaxError, ValueError) as e: raise AibError(head=self.col_defn.short_descr, body='Xml error - {}'.format(e.args[0])) 2. This is how I process the result at run-time. I do not try to parse it myself. I convert it into an EtreeElement, and step through it. Each 'tag' in the XML maps to a function name in the processing module - for xml in elem: await globals()[xml.tag](caller, xml) The built-in ElementTree would work for this, but I actually use lxml, because I use a little bit of xpath in my processing, which ElementTree does not support. Frank From breamoreboy at gmail.com Sat Aug 27 06:08:52 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 27 Aug 2016 03:08:52 -0700 (PDT) Subject: Is duck-typing misnamed? In-Reply-To: References: <1472255923l.39846110l.0l@psu.edu> Message-ID: <37ba4d3a-2a32-40b8-a253-ea2b4b566ab3@googlegroups.com> This should go to Python ideas as it would involve a substantial change to the docs. Kindest regards. Mark Lawrence. From ned at nedbatchelder.com Sat Aug 27 06:35:15 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 27 Aug 2016 03:35:15 -0700 (PDT) Subject: Is duck-typing misnamed? In-Reply-To: References: <1472255923l.39846110l.0l@psu.edu> Message-ID: <758c11e0-85a1-43f2-8518-29681ddf76f4@googlegroups.com> On Saturday, August 27, 2016 at 5:50:30 AM UTC-4, ROGER GRAYDON CHRISTMAN wrote: > "If it walks like a duck, quacks like a duck,... " > > so there is indeed precedence for this so-called 'duck typing' > > > but wouldn't it be more Pythonic to call this 'witch typing'? > > "How do you know she is a witch?" > > "She looks like one." > > etc. > > > I do grant that ultimately, the duck does come into play, since the witch > weighs the same as a duck. +1 :) --Ned. From ddream.merchantt at gmail.com Sat Aug 27 11:04:33 2016 From: ddream.merchantt at gmail.com (ddream.merchantt at gmail.com) Date: Sat, 27 Aug 2016 08:04:33 -0700 (PDT) Subject: Python script for searching variable strings between two constant strings In-Reply-To: References: Message-ID: Thanks for the lead. I have big log file nearly 2 GB. Lets say I just want to extract the ;name' field only eg. AutoAuthOSUserSubmit.The code is failing with errors. Can you just give a tested code only for the name field. Other fields I will try to work out. ----------- On Saturday, August 27, 2016 at 4:03:59 AM UTC+5:30, ddream.m... at gmail.com wrote: > import re > > infile = open('document.txt','r') > outfile= open('output.txt','w') > copy = False > for line in infile: > > if line.strip() == "--operation():": > bucket = [] > copy = True > > elif line.strip() == "StartOperation": > for strings in bucket: > outfile.write( strings + ',') > for strings in bucket: > outfile.write('\n') > copy = False > > elif copy: > bucket.append(line.strip() > -------------------------------------- > > CSV format is like this: > id, name, poid, error > 5896, AutoAuthOSUserSubmit, 900105270, 0x4002 > > My log file has several sections starting with ==== START ==== and ending with ==== END ====. I want to extract the string between --operation(): and StartOperation. For example, AutoAuthOSUserSubmit. I also want to extract the poid value from line poid: 900105270, poidLen: 9. Finally, I want to extract the return value, e.g 0x4002 if Roll back all updates is found after it. > > I am not even able to extract point the original text if Start and End are not on the same line. How do I go about doing that? > > This is a sample LOG extract with two paragraphs: > -- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== START ====** > open file /ecas/public/onsite-be/config/timer.conf failed > INFO 08/24/16 02:07:56 salt1be-d1-ap(**5896**/0) main.c(780*****):--operation(): AutoAuthOSUserSubmit. StartOperation***** > INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) main.c(784):--Client Information: Request from host 'malt-d1-wb' process id 12382. > DEBUG 08/24/16 02:07:56 salt1be-d1-ap(5896/0) TOci.cc(571):FetchServiceObjects: ServiceCert.sql > DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsserviceagent.cpp(517):Generate Certificate 2: c1cd00d5c3de082360a08730fef9cd1d > DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) junk.c(1373):GenerateWebPin : poid: **900105270**, poidLen: 9 > DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) junk.c(1408):GenerateWebPin : pinStr > DEBUG 08/24/16 02:07:56 salt1be-d1-ap(5896/0) uaadapter_vasco_totp.c(275):UAVascoTOTPImpl.close() -- Releasing Adapter Context > DEBUG 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsenterprise.cpp(288):VSEnterprise::Engage returns 0x4002 - Unknown error code **(0x4002)** > ERROR 08/22/16 23:15:53 pepper1be-d1-ap(2680/0) vsautoauth.cpp(696):OSAAEndUserEnroll: error occurred. **Roll back** all updates! > INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) uaotptokenstoreqmimpl.cpp(199):Close token store > INFO 08/24/16 02:07:56 salt1be-d1-ap(5896/0) main.c(990):-- EndOperation > -- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== END ====** > OPERATION = AutoAuthOSUserSubmit, rc = 0x0 (0) > SYSINFO Elapse = 0.687, Heap = 1334K, Stack = 64K From larry.martell at gmail.com Sat Aug 27 11:54:18 2016 From: larry.martell at gmail.com (Larry Martell) Date: Sat, 27 Aug 2016 11:54:18 -0400 Subject: Is duck-typing misnamed? In-Reply-To: <1472255923l.39846110l.0l@psu.edu> References: <1472255923l.39846110l.0l@psu.edu> Message-ID: On Fri, Aug 26, 2016 at 7:58 PM, ROGER GRAYDON CHRISTMAN wrote: > "If it walks like a duck, quacks like a duck,... " > > so there is indeed precedence for this so-called 'duck typing' > > > but wouldn't it be more Pythonic to call this 'witch typing'? > > "How do you know she is a witch?" > > "She looks like one." > > etc. > > > I do grant that ultimately, the duck does come into play, since the witch > weighs the same as a duck. Great idea, I love it. Now go and change your armor. From gyan.am010 at gmail.com Sat Aug 27 12:45:28 2016 From: gyan.am010 at gmail.com (GP) Date: Sat, 27 Aug 2016 09:45:28 -0700 (PDT) Subject: Error numpy install Message-ID: <3860b67f-402c-40f5-89a0-138d30c7f5e5@googlegroups.com> I have installed numpy using the command pip install numpy from command prompt and I am getting the following error: Traceback (most recent call last): File "", line 1, in import numpy File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\__init__.py", line 180, in from . import add_newdocs File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\add_newdocs.py", line 13, in from numpy.lib import add_newdoc File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\lib\__init__.py", line 8, in from .type_check import * File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\lib\type_check.py", line 11, in import numpy.core.numeric as _nx File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\core\__init__.py", line 14, in from . import multiarray ImportError: cannot import name 'multiarray' I have also tried to install multiarray but it says: "Could not find a program that satisfies the requirement multiarray(from versions) Any suggestions on how to install From grant.b.edwards at gmail.com Sat Aug 27 13:24:48 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 27 Aug 2016 17:24:48 +0000 (UTC) Subject: integer's methods References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: On 2016-08-18, ast wrote: > Hello > > I wonder why calling a method on an integer > doesn't work ? > >>>> 123.bit_length() > SyntaxError: invalid syntax Becuase the parser thinks you've entered a floating point number with a fractional part of "bit_length". You need to enter the integer such that it's identified by the parser as an integer rather than as a broken floating point number: >>> (123).bit_length() 7 >>> 123 .bit_length() 7 -- Grant Edwards grant.b.edwards Yow! Look DEEP into the at OPENINGS!! Do you see any gmail.com ELVES or EDSELS ... or a HIGHBALL?? ... From random832 at fastmail.com Sat Aug 27 13:34:33 2016 From: random832 at fastmail.com (Random832) Date: Sat, 27 Aug 2016 13:34:33 -0400 Subject: integer's methods In-Reply-To: References: <57b5b111$0$3299$426a34cc@news.free.fr> Message-ID: <1472319273.1671661.707883121.44732922@webmail.messagingengine.com> On Sat, Aug 27, 2016, at 13:24, Grant Edwards wrote: > Becuase the parser thinks you've entered a floating point number with > a fractional part of "bit_length". 123.+456 doesn't think that the fractional part is "+456". (Of course, the real reason is "because it would be even more annoying to get random errors only with attributes that start with "e" or "j") From dev at kou.li Sat Aug 27 15:33:29 2016 From: dev at kou.li (Kouli) Date: Sat, 27 Aug 2016 21:33:29 +0200 Subject: PEP 492: isn't the "await" redundant? In-Reply-To: <87r39a3izt.fsf@elektro.pacujo.net> References: <87h9a750kt.fsf@elektro.pacujo.net> <87r39a3izt.fsf@elektro.pacujo.net> Message-ID: Thank you for all your answers. After all, I am more confident with the current syntax. The most important reason for 'await' to me now is the fact you quite _often_ need to prepare the 'awaitable' object to wait for it later (like the ChrisA's example with print()), i.e. split the expression into more lines: fut = coro(x) .... await fut I supposed it to be only a minor use case (compared to 'await coro(x)'), but I learned it isn't. Every time you need to "wait for more than one thing" (more than one 'future'), you also need the split. Not only for parallel branching, but also even for simple async operations combined with timeout - asyncio.wait_for() etc. And I prefer the explicit 'await' for simple waiting to special syntax for spliting (i.e. do simple waiting without 'await' as was the proposal at top of this thread - and - introduce more complicated syntax for split - something like functools.partial(coro, x)). Kouli From test at joepgen.com Sat Aug 27 15:35:15 2016 From: test at joepgen.com (Joe) Date: Sat, 27 Aug 2016 21:35:15 +0200 Subject: Multimeter USB output Message-ID: Hi, I'm using Python 3.5.1 with PyUSB 1.0 under Win 10 (64). We try to read the USB output of a DMM 'UT61B'. import usb.core import usb.util import usb.backend.libusb1 def Gosub(): dev = usb.core.find(idVendor=0x1a86, idProduct=0xe008) # Digital Multimeter UT61B if dev == None: print ('Multimeter not found') else: print ('Multimeter was found') dev.set_configuration() cfg = dev.get_active_configuration() intf = cfg[(0,0)] ep = usb.util.find_descriptor( intf, custom_match = \ lambda e: \ usb.util.endpoint_direction(e.bEndpointAddress) == \ usb.util.ENDPOINT_IN) if ep == None: print ('ep is None') else: s = ep.read(64, 500) print ('Len s: ' + len(s)) print ('Starting') Gosub() print ('Ready.-') Result: File "d:\work-d\PythonProgs\ut61b.py", line 27, in Gosub() File "d:\work-d\PythonProgs\ut61b.py", line 23, in Gosub s = ep.read(64, 500) File "D:\Python3\Lib\site-packages\usb\core.py", line 402, in read return self.device.read(self, size_or_buffer, timeout) File "D:\Python3\Lib\site-packages\usb\core.py", line 988, in read self.__get_timeout(timeout)) File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 851, in intr_read timeout) File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 936, in __read _check(retval) File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 595, in _check raise USBError(_strerror(ret), ret, _libusb_errno[ret]) usb.core.USBError: [Errno 10060] Operation timed out What's wrong? How to fix? Regards -- Joe From grant.b.edwards at gmail.com Sat Aug 27 15:46:48 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 27 Aug 2016 19:46:48 +0000 (UTC) Subject: integer's methods References: <57b5b111$0$3299$426a34cc@news.free.fr> <1472319273.1671661.707883121.44732922@webmail.messagingengine.com> Message-ID: On 2016-08-27, Random832 wrote: > On Sat, Aug 27, 2016, at 13:24, Grant Edwards wrote: >> Becuase the parser thinks you've entered a floating point number with >> a fractional part of "bit_length". > > 123.+456 doesn't think that the fractional part is "+456". That's because the parser (or more traditionally the lexical analyzer) treats '+' differently than it does the characters [a-zA-Z]. > (Of course, the real reason is "because it would be even more > annoying to get random errors only with attributes that start with > "e" or "j") -- Grant Edwards grant.b.edwards Yow! The entire CHINESE at WOMEN'S VOLLEYBALL TEAM all gmail.com share ONE personality -- and have since BIRTH!! From breamoreboy at gmail.com Sat Aug 27 16:06:54 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 27 Aug 2016 13:06:54 -0700 (PDT) Subject: Error numpy install In-Reply-To: <3860b67f-402c-40f5-89a0-138d30c7f5e5@googlegroups.com> References: <3860b67f-402c-40f5-89a0-138d30c7f5e5@googlegroups.com> Message-ID: <91c16cdc-76d6-492f-8a5d-3f9d4580b677@googlegroups.com> On Saturday, August 27, 2016 at 5:45:58 PM UTC+1, GP wrote: > I have installed numpy using the command pip install numpy from command prompt and I am getting the following error: > Traceback (most recent call last): > File "", line 1, in > import numpy > File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\__init__.py", line 180, in > from . import add_newdocs > File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\add_newdocs.py", line 13, in > from numpy.lib import add_newdoc > File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\lib\__init__.py", line 8, in > from .type_check import * > File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\lib\type_check.py", line 11, in > import numpy.core.numeric as _nx > File "C:\Users\GP\AppData\Local\Programs\Python\Python35\lib\site-packages\numpy\core\__init__.py", line 14, in > from . import multiarray > ImportError: cannot import name 'multiarray' > > I have also tried to install multiarray but it says: "Could not find a program that satisfies the requirement multiarray(from versions) > > Any suggestions on how to install This has been reported a lot over the last couple of years. I've always overcome any numpy problems by downloading the appropriate version from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy and then using pip install against the local file name. HTH. Kindest regards. Mark Lawrence. From cs at zip.com.au Sat Aug 27 18:27:50 2016 From: cs at zip.com.au (cs at zip.com.au) Date: Sun, 28 Aug 2016 08:27:50 +1000 Subject: Is duck-typing misnamed? In-Reply-To: <1472255923l.39846110l.0l@psu.edu> References: <1472255923l.39846110l.0l@psu.edu> Message-ID: <20160827222750.GA26497@cskk.homeip.net> On 26Aug2016 19:58, ROGER GRAYDON CHRISTMAN wrote: >"If it walks like a duck, quacks like a duck,... " >so there is indeed precedence for this so-called 'duck typing' > >but wouldn't it be more Pythonic to call this 'witch typing'? >"How do you know she is a witch?" >"She looks like one." >etc. > >I do grant that ultimately, the duck does come into play, since the witch >weighs the same as a duck. I disagree. They want to burn her because she's supposedly a witch, but the scientific test was that she weighed as much as a duck. So I think your second example is also duck typing: functioning like a duck. Cheers, Cameron Simpson From tjreedy at udel.edu Sat Aug 27 18:34:21 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Aug 2016 18:34:21 -0400 Subject: Is duck-typing misnamed? In-Reply-To: <1472255923l.39846110l.0l@psu.edu> References: <1472255923l.39846110l.0l@psu.edu> Message-ID: On 8/26/2016 7:58 PM, ROGER GRAYDON CHRISTMAN wrote: > "If it walks like a duck, quacks like a duck,... " > > so there is indeed precedence for this so-called 'duck typing' > > > but wouldn't it be more Pythonic to call this 'witch typing'? > > "How do you know she is a witch?" > > "She looks like one." Given that people were once burned to death for 'looking like a witch' (or sounding or acting), and can still suffer socially for such reasons, this it not funny to me. We should stick with ducks. -- Terry Jan Reedy From tjreedy at udel.edu Sat Aug 27 18:45:06 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Aug 2016 18:45:06 -0400 Subject: Multimeter USB output In-Reply-To: References: Message-ID: On 8/27/2016 3:35 PM, Joe wrote: > Hi, > > I'm using Python 3.5.1 with PyUSB 1.0 under Win 10 (64). We try to read > the USB output of a DMM 'UT61B'. > > import usb.core > import usb.util > import usb.backend.libusb1 > > def Gosub(): > dev = usb.core.find(idVendor=0x1a86, idProduct=0xe008) # Digital > Multimeter UT61B > if dev == None: > print ('Multimeter not found') > else: > print ('Multimeter was found') > dev.set_configuration() > cfg = dev.get_active_configuration() > intf = cfg[(0,0)] > ep = usb.util.find_descriptor( > intf, > custom_match = \ > lambda e: \ > usb.util.endpoint_direction(e.bEndpointAddress) == \ > usb.util.ENDPOINT_IN) > if ep == None: > print ('ep is None') > else: > s = ep.read(64, 500) > print ('Len s: ' + len(s)) > > print ('Starting') > Gosub() > print ('Ready.-') > > Result: I presume you saw Starting Multimeter was found > File "d:\work-d\PythonProgs\ut61b.py", line 27, in > Gosub() > File "d:\work-d\PythonProgs\ut61b.py", line 23, in Gosub > s = ep.read(64, 500) > File "D:\Python3\Lib\site-packages\usb\core.py", line 402, in read > return self.device.read(self, size_or_buffer, timeout) > File "D:\Python3\Lib\site-packages\usb\core.py", line 988, in read > self.__get_timeout(timeout)) > File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 851, in > intr_read > timeout) > File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 936, in > __read > _check(retval) > File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 595, in > _check > raise USBError(_strerror(ret), ret, _libusb_errno[ret]) > > usb.core.USBError: [Errno 10060] Operation timed out > > What's wrong? How to fix? Read (again?) the doc for the interface for the device. Because reading timed out, I suspect that it is waiting for a command for it to send something. -- Terry Jan Reedy From joel.goldstick at gmail.com Sat Aug 27 18:46:45 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Sat, 27 Aug 2016 18:46:45 -0400 Subject: Is duck-typing misnamed? In-Reply-To: References: <1472255923l.39846110l.0l@psu.edu> Message-ID: On Sat, Aug 27, 2016 at 6:34 PM, Terry Reedy wrote: > On 8/26/2016 7:58 PM, ROGER GRAYDON CHRISTMAN wrote: >> >> "If it walks like a duck, quacks like a duck,... " >> >> so there is indeed precedence for this so-called 'duck typing' >> >> >> but wouldn't it be more Pythonic to call this 'witch typing'? >> >> "How do you know she is a witch?" >> >> "She looks like one." > > > Given that people were once burned to death for 'looking like a witch' (or > sounding or acting), and can still suffer socially for such reasons, this it > not funny to me. We should stick with ducks. > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list which ducks? -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From dvl at psu.edu Sat Aug 27 19:28:09 2016 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Sat, 27 Aug 2016 19:28:09 -0400 Subject: Is duck-typing misnamed? In-Reply-To: 20160827222750.GA26497@cskk.homeip.net References: <1472255923l.39846110l.0l@psu.edu><20160827222750.GA26497@cskk.homeip.net> Message-ID: <1472340488l.28639294l.0l@psu.edu> Your response is appreciated. I just thought I'd comment a little more on the script: Woman: I'm not a witch! I'm not a witch! V: ehh... but you are dressed like one. W: They dressed me up like this! All: naah no we didn't... no. W: And this isn't my nose, it's a false one. (V lifts up carrot) V: Well? P1: Well we did do the nose V: The nose? P1: ...And the hat, but she is a witch! They took a woman who originally, I think we might agree, was not a witch, and they added features that were understood to be part of the protocol for witchiness. I think this is very much like me defining methods __iter__ and __next__ and voila, I've turned something into an iterator by witch -- er.. duck-typing! Perhaps she inherited her weight from her latent duckness. Thoughts? Roger Christman On Sat, Aug 27, 2016 06:27 PM, python-list at python.org wrote: > On 26Aug2016 19:58, ROGER GRAYDON CHRISTMAN wrote: >>"If it walks like a duck, quacks like a duck,... " >>so there is indeed precedence for this so-called 'duck typing' >> >>but wouldn't it be more Pythonic to call this 'witch typing'? >>"How do you know she is a witch?" >>"She looks like one." >>etc. >> >>I do grant that ultimately, the duck does come into play, since the witch >>weighs the same as a duck. > >I disagree. They want to burn her because she's supposedly a witch, but the >scientific test was that she weighed as much as a duck. So I think your second >example is also duck typing: functioning like a duck. > >Cheers, >Cameron Simpson > > > From ben+python at benfinney.id.au Sat Aug 27 20:26:32 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 28 Aug 2016 10:26:32 +1000 Subject: Is duck-typing misnamed? References: <1472255923l.39846110l.0l@psu.edu> <20160827222750.GA26497@cskk.homeip.net> Message-ID: <85shtpvivb.fsf@benfinney.id.au> cs at zip.com.au writes: > They want to burn her because she's supposedly a witch, but the > scientific test was that she weighed as much as a duck. So I think > your second example is also duck typing: functioning like a duck. Excellent reasoning! (Also, I agree that describing objects with ?looks like a witch? brings repressive social context, both historical and present-day, that should not be encouraged. I'd prefer that the Python community refrain from that.) Let's stick to the term ?duck typing?. -- \ ?Any sufficiently advanced bug is indistinguishable from a | `\ feature.? ?Rich Kulawiec | _o__) | Ben Finney From jpablo.romero at gmail.com Sat Aug 27 22:31:52 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Sat, 27 Aug 2016 19:31:52 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: 2016-08-14 7:29 GMT-07:00 Steven D'Aprano : > On Thu, 11 Aug 2016 06:33 am, Juan Pablo Romero M?ndez wrote: > > > I've been trying to find (without success so far) an example of a > > situation where the dynamic features of a language like Python provides a > > clear advantage over languages with more than one type. > > Python has more than one type. Don't confuse dynamic typing with weak > typing > or untyped (typeless) languages. More on this below. > Sorry I was not clear, I was thinking in something along these lines: https://existentialtype.wordpress.com/2011/03/19/dynamic-languages-are-static-languages/ (Warning: his pov is very different from yours) > > I don't believe that you will find "an example of a situation..." as you > say > above. There is a clear situation in my mind: both Python and JavaScript (the dynamic langs I'm most familiar with) provide an easy to use `eval` function. None of the static langs I know provide such functionality (that I'm aware of). Personally I don't have any use for eval, so I was thinking in situations other than that. > It sounds like you are hope to find a clear example of "If you do > This, then dynamic languages are the Clear Winner". But I don't think you > will. Dynamic languages tend to produce clear productivity improvements > over statically typed languages, but of course this is only "typically" > true, not a guarantee that applies to every single programmer or project. > The very few research done on the subject seems to indicate otherwise (here's a link if you are interested in such topics https://www.functionalgeekery.com/episode-55-andreas-stefik/#t=18:14.448). > > Typically: > > - dynamic languages are less verbose; > - dynamic languages are faster to develop in; many organisations > prototype applications in Python (say) before re-writing it in > C++/Java/whatever; > - far less time spent fighting the compiler; > - dynamic languages often have fewer bugs, because it is easier to > reason about the code (no "undefined behaviour" like in C!) and > fewer lines of code to reason about; - but statically typed languages allow you to prove the absence > of certain types of bugs. > > The exception is if you try to write statically typed code in a dynamic > language. Then you get the worst of both styles of coding: the verbose, > heavyweight style of many static languages, but without the automated > correctness proofs, plus the performance costs of dynamic typing, but > without the rapid development. > > > > Regarding types and type systems, if you haven't already read this, you > should: > > https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ Thanks for the link. I don't have one to give back, but what I can suggest is this book: http://haskellbook.com/. It seems to me that many of your opinions come from using C++ / Java. If that's the case they are completely understandable. Despite their popularity they are by no means good representatives of languages with modern and powerful type systems. F# (dot.net), Haskell, Scala (JVM / Browser), or Elm (Browser) provide much better examples. > > > "Static typing" (e.g. Pascal, C, Java, Haskell) and "dynamic typing" (e.g. > Python, Javascript, Ruby, Lua) differ on when and how values are checked > for type-compatibility. > > "Strong" and "weak" typing are ends of a continuum. Nearly all languages > are > a little bit weak (they allow automatic coercions between numeric types) > but mostly strong (they don't automatically coerce integers to arrays). > Javascript, Perl and PHP are weaker than Python because they'll coerce > strings to numbers automatically and Python won't. > > I don't know many untyped languages apart from machine code or maybe > assembly. Perhaps Forth? (Maybe not -- some Forths include a separate > floating point stack as well as the usual stack.) Hypertalk treated > everything as strings. Tcl treats nearly everything as strings, although it > also has arrays. > > So, Python has types, and it is a mostly strong typed language. It will do > relatively few automatic coercions. > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+python at pearwood.info Sun Aug 28 00:30:36 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 28 Aug 2016 14:30:36 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> On Sun, 28 Aug 2016 12:31 pm, Juan Pablo Romero M?ndez wrote: > 2016-08-14 7:29 GMT-07:00 Steven D'Aprano : > >> On Thu, 11 Aug 2016 06:33 am, Juan Pablo Romero M?ndez wrote: >> >> > I've been trying to find (without success so far) an example of a >> > situation where the dynamic features of a language like Python provides >> > a clear advantage over languages with more than one type. >> >> Python has more than one type. Don't confuse dynamic typing with weak >> typing or untyped (typeless) languages. More on this below. >> > > > Sorry I was not clear, I was thinking in something along these lines: > > https://existentialtype.wordpress.com/2011/03/19/dynamic-languages-are-static-languages/ > > (Warning: his pov is very different from yours) It is a great example of somebody suffering from the problem that when the only tool he has is a hammer, everything looks like an nail. He clearly is immersed in the world of formal type systems and understands their power. But now he sees everything from that single perspective. Now it is true that speaking in full generality, classes and types refer to different things. Or to be perhaps more accurate, *subclassing* and *subtyping* are different things: http://c2.com/cgi/wiki?SubTypingAndSubClassing Many languages treat them the same, but fundamentally they are different. (Note: for veteran Python programmers who remember the language before types and classes where unified in version 2.2, this is not the same thing! Prior to 2.2, both "types" and "classes" related to *subclassing*, albeit in a negative way for the built-in types: they couldn't be subclassed.) But the author of this piece ignores that standard distinction and invents his own non-standard one: to him, classes are merely different representations of the same data. E.g. his example of complex numbers, shown as Cartesian (x, y) values or polar (r, ?) values. These aren't two different "kinds of things", but merely two different ways of representing the same entity. That's not a good way to think about (say) Python lists and Python bools. Lists and bools are in no way the same kind of entity (except in the most general category of "they're both objects"). It's not even a very good way of thinking about complex numbers. Viewed from his perspective of type systems, the author makes what I call the food processor error. Food processors, blenders and other similar kitchen appliances are often advertised as having "five speeds", or ten speeds, or however many the machine is capable of, usually labelled as "chop", "dice", "whip", "puree", etc. And, far too often: "OFF". Since when is "off" a speed? If a blender that is turned off counts as a blending speed, then a simple bowl is a "one speed blender". You put food in the bowl, and it doesn't blend at all. That counts as "off" speed. The author is making the same mistake. He thinks that a language which lacks static typing counts as static typing. "Off" is a speed! "Bald" is a hair colour! "Raw" is a way of cooking food! In truth though, static and dynamic typing are very different. The author is fooled because you can emulate *one* part of dynamic typing in a statically typed system by adding one extra type, "Any", or "Duck Type", or whatever you want to call it. The static checker can then ignore anything declared as Any type. But deferring type checks to runtime is only part of dynamic typing. The other fundamental difference is: - statically typed languages associate types to variables; - dynamically typed languages associate types to values. This difference is more significant than the "run-time/compile-time" and its one which often confuses people. That's not surprising: if I say "x is an int", that's ambiguous whether I'm referring to the variable x or the value currently assigned to x. If you don't see the ambiguity, then you're seeing it purely from the perspective of either static or dynamic typing. In static typing, I somehow associate the name "x" with a tag that says "this may only be used with ints". Perhaps I have to declare it first, like in C or Pascal, or perhaps the compiler can infer the type, like in Haskell, but either way, "x" is now forever tagged as an int, so that the compiler can flag errors like: x = 1 # ... code can run here x.upper() The compiler knows that ints don't have a method "upper" and can flag this as an error. In such static languages, it is invariably an error to try to change the type of the variable (unless it has been tagged as "Anything" or "Duck Typed"). x = 1 x = "hello" # a type error, at compile time But in dynamic typing, the type information isn't associated with the name "x", but with the value 1 currently assigned to it. Change the assignment, and the type changes. As a consequence, it is necessary to move the type checks from compile time to runtime, but that's not the fundamental difference between the two. As further evidence that the author has missed the forest for all the trees, consider languages which actually do have only a single type: - in assembly language, everything is just bytes or words; - in Forth, similarly, everything is just a 16-bit or 32-bit word; - in Hypertalk, every value is stored internally as a string; to say nothing of more esoteric languages like Oook, Whitespace and BrainF*ck. TL;DR: The author of that blog post ignores actual untyped languages, misses the distinction between type information being associated with variables or values, and invents his own idiosyncratic distinction between types and classes while ignoring the standard one. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jpablo.romero at gmail.com Sun Aug 28 01:24:20 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Sat, 27 Aug 2016 22:24:20 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: 2016-08-27 21:30 GMT-07:00 Steve D'Aprano : > On Sun, 28 Aug 2016 12:31 pm, Juan Pablo Romero M?ndez wrote: > > > 2016-08-14 7:29 GMT-07:00 Steven D'Aprano : > > > >> On Thu, 11 Aug 2016 06:33 am, Juan Pablo Romero M?ndez wrote: > >> > >> > I've been trying to find (without success so far) an example of a > >> > situation where the dynamic features of a language like Python > provides > >> > a clear advantage over languages with more than one type. > >> > >> Python has more than one type. Don't confuse dynamic typing with weak > >> typing or untyped (typeless) languages. More on this below. > >> > > > > > > Sorry I was not clear, I was thinking in something along these lines: > > > > > https://existentialtype.wordpress.com/2011/03/19/ > dynamic-languages-are-static-languages/ > > > > (Warning: his pov is very different from yours) > > It is a great example of somebody suffering from the problem that when the > only tool he has is a hammer, everything looks like an nail. > > He clearly is immersed in the world of formal type systems and understands > their power. But now he sees everything from that single perspective. > > Now it is true that speaking in full generality, classes and types refer to > different things. Or to be perhaps more accurate, *subclassing* and > *subtyping* are different things: > > http://c2.com/cgi/wiki?SubTypingAndSubClassing > > Many languages treat them the same, but fundamentally they are different. > > (Note: for veteran Python programmers who remember the language before > types > and classes where unified in version 2.2, this is not the same thing! Prior > to 2.2, both "types" and "classes" related to *subclassing*, albeit in a > negative way for the built-in types: they couldn't be subclassed.) > > But the author of this piece ignores that standard distinction and invents > his own non-standard one: to him, classes are merely different > representations of the same data. E.g. his example of complex numbers, > shown as Cartesian (x, y) values or polar (r, ?) values. These aren't two > different "kinds of things", but merely two different ways of representing > the same entity. > > That's not a good way to think about (say) Python lists and Python bools. > Lists and bools are in no way the same kind of entity (except in the most > general category of "they're both objects"). > > It's not even a very good way of thinking about complex numbers. > > Viewed from his perspective of type systems, the author makes what I call > the food processor error. Food processors, blenders and other similar > kitchen appliances are often advertised as having "five speeds", or ten > speeds, or however many the machine is capable of, usually labelled > as "chop", "dice", "whip", "puree", etc. And, far too often: "OFF". > > Since when is "off" a speed? If a blender that is turned off counts as a > blending speed, then a simple bowl is a "one speed blender". You put food > in the bowl, and it doesn't blend at all. That counts as "off" speed. > > The author is making the same mistake. He thinks that a language which > lacks > static typing counts as static typing. "Off" is a speed! "Bald" is a hair > colour! "Raw" is a way of cooking food! > > In truth though, static and dynamic typing are very different. The author > is > fooled because you can emulate *one* part of dynamic typing in a statically > typed system by adding one extra type, "Any", or "Duck Type", or whatever > you want to call it. The static checker can then ignore anything declared > as Any type. > > But deferring type checks to runtime is only part of dynamic typing. The > other fundamental difference is: > > - statically typed languages associate types to variables; > - dynamically typed languages associate types to values. > > This difference is more significant than the "run-time/compile-time" and > its > one which often confuses people. That's not surprising: if I say "x is an > int", that's ambiguous whether I'm referring to the variable x or the value > currently assigned to x. If you don't see the ambiguity, then you're seeing > it purely from the perspective of either static or dynamic typing. > > In static typing, I somehow associate the name "x" with a tag that > says "this may only be used with ints". Perhaps I have to declare it first, > like in C or Pascal, or perhaps the compiler can infer the type, like in > Haskell, but either way, "x" is now forever tagged as an int, so that the > compiler can flag errors like: > > x = 1 > # ... code can run here > x.upper() > > The compiler knows that ints don't have a method "upper" and can flag this > as an error. In such static languages, it is invariably an error to try to > change the type of the variable (unless it has been tagged as "Anything" > or "Duck Typed"). > > x = 1 > x = "hello" # a type error, at compile time > > > But in dynamic typing, the type information isn't associated with the > name "x", but with the value 1 currently assigned to it. Change the > assignment, and the type changes. As a consequence, it is necessary to move > the type checks from compile time to runtime, but that's not the > fundamental difference between the two. > > As further evidence that the author has missed the forest for all the > trees, > consider languages which actually do have only a single type: > > - in assembly language, everything is just bytes or words; > > - in Forth, similarly, everything is just a 16-bit or 32-bit word; > > - in Hypertalk, every value is stored internally as a string; > > to say nothing of more esoteric languages like Oook, Whitespace and > BrainF*ck. > > > TL;DR: > > The author of that blog post ignores actual untyped languages, misses the > distinction between type information being associated with variables or > values, and invents his own idiosyncratic distinction between types and > classes while ignoring the standard one. > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Sun Aug 28 01:29:48 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Aug 2016 15:29:48 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 28, 2016 at 2:30 PM, Steve D'Aprano wrote: > But the author of this piece ignores that standard distinction and invents > his own non-standard one: to him, classes are merely different > representations of the same data. E.g. his example of complex numbers, > shown as Cartesian (x, y) values or polar (r, ?) values. These aren't two > different "kinds of things", but merely two different ways of representing > the same entity. > > That's not a good way to think about (say) Python lists and Python bools. > Lists and bools are in no way the same kind of entity (except in the most > general category of "they're both objects"). > > It's not even a very good way of thinking about complex numbers. It might be a good way of thinking about points on a Cartesian plane, though. Rectangular and polar coordinates truly are just different ways of expressing the same information. (How well 2D coordinates map to complex numbers is a separate question.) > In static typing, I somehow associate the name "x" with a tag that > says "this may only be used with ints". Perhaps I have to declare it first, > like in C or Pascal, or perhaps the compiler can infer the type, like in > Haskell, but either way, "x" is now forever tagged as an int, so that the > compiler can flag errors like: > > x = 1 > # ... code can run here > x.upper() > > The compiler knows that ints don't have a method "upper" and can flag this > as an error. In such static languages, it is invariably an error to try to > change the type of the variable (unless it has been tagged as "Anything" > or "Duck Typed"). So far, I completely agree with you; whether you declare "x takes integers only" or the compiler infers "x has been assigned to point to an integer" or any other form of it, attempting to call .upper() on the integer 1 is an error. > x = 1 > x = "hello" # a type error, at compile time > > > But in dynamic typing, the type information isn't associated with the > name "x", but with the value 1 currently assigned to it. Change the > assignment, and the type changes. As a consequence, it is necessary to move > the type checks from compile time to runtime, but that's not the > fundamental difference between the two. This is where I'm less sure. Sometimes a variable's type should be broader than just one concrete type - for instance, a variable might hold 1 over here, and 1.5 over there, and thus is storing either "int or float" or "any number". If you have a complex hierarchy of types, how do you know that this variable should be allowed to hold anything up to a certain level in the hierarchy, and no further? If what the compiler's doing is identifying what *is* assigned, then it's easy. You've given it an int over here and a float over there, and that's legal; from that point on, the compiler knows that this contains either an int or a float. (Let's assume it can't know for sure which, eg it has "if (cond) x=1; else x=1.5" where the condition can't be known till run-time.) But for your example of x="hello" to be a compilation error, it has to either assume that the first object given determines the type completely, or be told what types are permitted. So, for example, I could make a declaration in Pike that says: int|float x = 1; and then x can have either an integer (which, like in Python, is a bignum) or a float (IEEE 64-bit, again like Python), but not a string. I could equally say: string(8bit)|int x(0..) = 12345; which would allow x to store a byte-string (an eight-bit string, as opposed to a Unicode string which stores text) or a non-negative integer. A type inference system that can't handle variables like this is limited; but if it _can_ handle something like this, how can it flag an error at compile time? It'd just infer a more complicated type. How is this resolved in type-inferring languages? (Genuine question, not rhetorical. I haven't used type-inferring languages in this way.) > As further evidence that the author has missed the forest for all the trees, > consider languages which actually do have only a single type: > > - in assembly language, everything is just bytes or words; > > - in Forth, similarly, everything is just a 16-bit or 32-bit word; > > - in Hypertalk, every value is stored internally as a string; > > to say nothing of more esoteric languages like Oook, Whitespace and > BrainF*ck. Turing Tarpits (Ook, Brain*, etc) tend to be like assembly language, treating everything as cells (assembly language might call those cells either "bytes" or "words"). REXX is like Hypertalk - everything truly is a string. Shell scripting languages generally treat everything as strings, too (although bash has arrays too). I can't imagine any untyped language using anything other than bytes/words or strings, but I'm sure someone's done it somewhere. ChrisA From jpablo.romero at gmail.com Sun Aug 28 01:56:27 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Sat, 27 Aug 2016 22:56:27 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: 2016-08-27 21:30 GMT-07:00 Steve D'Aprano : > On Sun, 28 Aug 2016 12:31 pm, Juan Pablo Romero M?ndez wrote: > > > 2016-08-14 7:29 GMT-07:00 Steven D'Aprano : > > > >> On Thu, 11 Aug 2016 06:33 am, Juan Pablo Romero M?ndez wrote: > >> > >> > I've been trying to find (without success so far) an example of a > >> > situation where the dynamic features of a language like Python > provides > >> > a clear advantage over languages with more than one type. > >> > >> Python has more than one type. Don't confuse dynamic typing with weak > >> typing or untyped (typeless) languages. More on this below. > >> > > > > > > Sorry I was not clear, I was thinking in something along these lines: > > > > > https://existentialtype.wordpress.com/2011/03/19/ > dynamic-languages-are-static-languages/ > > > > (Warning: his pov is very different from yours) > > It is a great example of somebody suffering from the problem that when the > only tool he has is a hammer, everything looks like an nail. > > He clearly is immersed in the world of formal type systems and understands > their power. But now he sees everything from that single perspective. > > Now it is true that speaking in full generality, classes and types refer to > different things. Or to be perhaps more accurate, *subclassing* and > *subtyping* are different things: > > http://c2.com/cgi/wiki?SubTypingAndSubClassing > > Many languages treat them the same, but fundamentally they are different. > Oh, I don't think he is thinking in terms of OO "classes", I think he meant two different "kinds" or "varieties" of values (although kind has a technical meaning) In TypeScript terms what he is saying can be described like this: type Complex = { real: number, i: number } | { r: number, ?: number} const c1: Complex = { real: 1, i: 1 } const c2: Complex = { r: 1, ?: 0.5 } You have two values of the same type but different representation. > > (Note: for veteran Python programmers who remember the language before > types > and classes where unified in version 2.2, this is not the same thing! Prior > to 2.2, both "types" and "classes" related to *subclassing*, albeit in a > negative way for the built-in types: they couldn't be subclassed.) > > But the author of this piece ignores that standard distinction and invents > his own non-standard one: to him, classes are merely different > representations of the same data. E.g. his example of complex numbers, > shown as Cartesian (x, y) values or polar (r, ?) values. These aren't two > different "kinds of things", but merely two different ways of representing > the same entity. > > That's not a good way to think about (say) Python lists and Python bools. > Lists and bools are in no way the same kind of entity (except in the most > general category of "they're both objects"). > > It's not even a very good way of thinking about complex numbers. > > Viewed from his perspective of type systems, the author makes what I call > the food processor error. Food processors, blenders and other similar > kitchen appliances are often advertised as having "five speeds", or ten > speeds, or however many the machine is capable of, usually labelled > as "chop", "dice", "whip", "puree", etc. And, far too often: "OFF". > > Since when is "off" a speed? If a blender that is turned off counts as a > blending speed, then a simple bowl is a "one speed blender". You put food > in the bowl, and it doesn't blend at all. That counts as "off" speed. > > The author is making the same mistake. He thinks that a language which > lacks > static typing counts as static typing. "Off" is a speed! "Bald" is a hair > colour! "Raw" is a way of cooking food! > > In truth though, static and dynamic typing are very different. The author > is > fooled because you can emulate *one* part of dynamic typing in a statically > typed system by adding one extra type, "Any", or "Duck Type", or whatever > you want to call it. The static checker can then ignore anything declared > as Any type. > > But deferring type checks to runtime is only part of dynamic typing. The > other fundamental difference is: > > - statically typed languages associate types to variables; > - dynamically typed languages associate types to values. > > This difference is more significant than the "run-time/compile-time" and > its > one which often confuses people. That's not surprising: if I say "x is an > int", that's ambiguous whether I'm referring to the variable x or the value > currently assigned to x. If you don't see the ambiguity, then you're seeing > it purely from the perspective of either static or dynamic typing. > > In static typing, I somehow associate the name "x" with a tag that > says "this may only be used with ints". Perhaps I have to declare it first, > like in C or Pascal, or perhaps the compiler can infer the type, like in > Haskell, but either way, "x" is now forever tagged as an int, so that the > compiler can flag errors like: > > x = 1 > # ... code can run here > x.upper() > > The compiler knows that ints don't have a method "upper" and can flag this > as an error. In such static languages, it is invariably an error to try to > change the type of the variable (unless it has been tagged as "Anything" > or "Duck Typed"). > > x = 1 > x = "hello" # a type error, at compile time > > > But in dynamic typing, the type information isn't associated with the > name "x", but with the value 1 currently assigned to it. Change the > assignment, and the type changes. As a consequence, it is necessary to move > the type checks from compile time to runtime, but that's not the > fundamental difference between the two. > > As further evidence that the author has missed the forest for all the > trees, > consider languages which actually do have only a single type: > > - in assembly language, everything is just bytes or words; > > - in Forth, similarly, everything is just a 16-bit or 32-bit word; > > - in Hypertalk, every value is stored internally as a string; > > to say nothing of more esoteric languages like Oook, Whitespace and > BrainF*ck. > > > TL;DR: > > The author of that blog post ignores actual untyped languages, misses the > distinction between type information being associated with variables or > values, and invents his own idiosyncratic distinction between types and > classes while ignoring the standard one. > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From jussi.piitulainen at helsinki.fi Sun Aug 28 02:13:43 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sun, 28 Aug 2016 09:13:43 +0300 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: Chris Angelico writes: > On Sun, Aug 28, 2016 at 2:30 PM, Steve D'Aprano wrote: >> But in dynamic typing, the type information isn't associated with the >> name "x", but with the value 1 currently assigned to it. Change the >> assignment, and the type changes. As a consequence, it is necessary >> to move the type checks from compile time to runtime, but that's not >> the fundamental difference between the two. > > This is where I'm less sure. Sometimes a variable's type should be > broader than just one concrete type - for instance, a variable might > hold 1 over here, and 1.5 over there, and thus is storing either "int > or float" or "any number". If you have a complex hierarchy of types, > how do you know that this variable should be allowed to hold anything > up to a certain level in the hierarchy, and no further? It's not just literal values that give potential type information in a dynamically typed language. Another source is functions that the compiler knows, and this information propagates back and forth in the analysis of the control flow. For example, below the compiler might infer that x must be a number but not a complex number, then generate one type check (which it might be able to prove redundant) and calls to specialized versions of ceiling and floor. d = ceiling(x) - floor(x) Also known is that the results of the calls are numbers and the difference of numbers is a number, so d gets assigned a number. Perhaps ceiling and floor in the language always return an int. Then d is known to be an int. And so on. I think "soft typing" refers to this kind of work. Scheme implementations have specialized arithmetic operators that require their arguments to be floats. The generic operators do the same thing when their arguments are floats, but the specialized operators provide this type information to an optimizing compiler. This is related to specialized container types that can only contain floats. This way it may be possible to arrange the code so that the compiler knows the type of everything, or almost everything, which helps the compiler in its art. From rosuav at gmail.com Sun Aug 28 02:49:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Aug 2016 16:49:13 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Aug 28, 2016 at 4:13 PM, Jussi Piitulainen wrote: >> This is where I'm less sure. Sometimes a variable's type should be >> broader than just one concrete type - for instance, a variable might >> hold 1 over here, and 1.5 over there, and thus is storing either "int >> or float" or "any number". If you have a complex hierarchy of types, >> how do you know that this variable should be allowed to hold anything >> up to a certain level in the hierarchy, and no further? > > It's not just literal values that give potential type information in a > dynamically typed language. Another source is functions that the > compiler knows, and this information propagates back and forth in the > analysis of the control flow. > > For example, below the compiler might infer that x must be a number but > not a complex number, then generate one type check (which it might be > able to prove redundant) and calls to specialized versions of ceiling > and floor. > > d = ceiling(x) - floor(x) > > Also known is that the results of the calls are numbers and the > difference of numbers is a number, so d gets assigned a number. Perhaps > ceiling and floor in the language always return an int. Then d is known > to be an int. And so on. Right, and I understand this concept. Consider this code: x = 5; ... if (some_condition) x = "five"; else x = [0, 0, 0, 0, 0]; (adjust syntax to whatever language you like) Does this mean that the type of x is int|string|list, or will this be an error? Assuming the condition can't be known until run time (eg it involves user input), there's no way for a static analyzer to differentiate between this code and the form that Steven put forward: > x = 1 > x = "hello" # a type error, at compile time Simple type inference would either see this as meaning that x is int|string, or possibly it'd say "x is an int up to that second line, and a string thereafter" (which is basically like dynamic typing but statically checked - it's the value, not the variable, that has a type, and checks like x.upper() would take note of that). But if it flags it as an error, that would basically mean that the type system is (probably deliberately) simplistic and restrictive, requiring that x be EITHER an integer variable OR a string variable, and not both. Which is a perfectly viable stance, but I'm just not sure if it's (a) what is done, or (b) ideal. Particularly since it'd end up requiring some annoying rules, like "integers and floats are compatible, but nothing else, including user-defined types" or "integers and floats are fundamentally different things, and if you want your variable ever to contain a float, you have to always use 1.0 instead of just 1", neither of which I like. ChrisA From steve+comp.lang.python at pearwood.info Sun Aug 28 03:04:30 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Aug 2016 17:04:30 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57c28d00$0$11098$c3e8da3@news.astraweb.com> On Sunday 28 August 2016 15:56, Juan Pablo Romero M?ndez wrote: > 2016-08-27 21:30 GMT-07:00 Steve D'Aprano : [...] >> Now it is true that speaking in full generality, classes and types refer to >> different things. Or to be perhaps more accurate, *subclassing* and >> *subtyping* are different things: >> >> http://c2.com/cgi/wiki?SubTypingAndSubClassing >> >> Many languages treat them the same, but fundamentally they are different. > > Oh, I don't think he is thinking in terms of OO "classes", I think he meant > two different "kinds" or "varieties" of values (although kind has a > technical meaning) That's not quite right either. "Two different kinds or varieties" is what both subclassing and subtyping aim to describe, in different ways: e.g. dogs and cats are two different kinds of mammal. What you are describing here: > In TypeScript terms what he is saying can be described like this: > > type Complex = > { real: number, i: number } > | { r: number, ?: number} > > const c1: Complex = { real: 1, i: 1 } > const c2: Complex = { r: 1, ?: 0.5 } > > You have two values of the same type but different representation. seems to be more like what is called "variant record" in Pascal. http://www.freepascal.org/docs-html/ref/refsu16.html I'm not familiar with TypeScript. How does this work? If I say: const c1: Complex = {real: 1, imag: 1 } print c1.r print c1.? what do I get? If r and ? are merely alternative names for "real" and "imag", that's not good enough. Given real=1, imag=1, then we need to get r=1.414213 and ? = pi/4 without any extra effort on the part of the user. The point is, the complex number (1, 1) in Cartesian coordinates and (sqrt(2), pi/4) in polar coordinates aren't two different kinds of things, they are two different ways of writing the same value. Like writing 1A in hex and 26 in decimal. Somebody may choose to implement this as two different classes ("CartesianComplex" and "PolarComplex") but that's a limitation of their code, or of their language, it doesn't reflect a real difference between two different kinds of things. Another way to put it: If you were programming a first person shooter game, you might choose each of the enemies as an object. Let's say you have an enemy called the Grue. You can view the Grue from the front or from the back, depending on which way it is standing when you see it. Would you implement this as two different classes? GrueSeenFromFront GrueSeenFromBack I should hope not. It's the same object, the same Grue, it just looks different depending on which way you approach it. -- Steve From jussi.piitulainen at helsinki.fi Sun Aug 28 03:31:28 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sun, 28 Aug 2016 10:31:28 +0300 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: Chris Angelico writes: > On Sun, Aug 28, 2016 at 4:13 PM, Jussi Piitulainen wrote: >>> This is where I'm less sure. Sometimes a variable's type should be >>> broader than just one concrete type - for instance, a variable might >>> hold 1 over here, and 1.5 over there, and thus is storing either >>> "int or float" or "any number". If you have a complex hierarchy of >>> types, how do you know that this variable should be allowed to hold >>> anything up to a certain level in the hierarchy, and no further? >> >> It's not just literal values that give potential type information in >> a dynamically typed language. Another source is functions that the >> compiler knows, and this information propagates back and forth in the >> analysis of the control flow. >> >> For example, below the compiler might infer that x must be a number >> but not a complex number, then generate one type check (which it >> might be able to prove redundant) and calls to specialized versions >> of ceiling and floor. >> >> d = ceiling(x) - floor(x) >> >> Also known is that the results of the calls are numbers and the >> difference of numbers is a number, so d gets assigned a number. >> Perhaps ceiling and floor in the language always return an int. Then >> d is known to be an int. And so on. > > Right, and I understand this concept. Consider this code: > > x = 5; > ... > if (some_condition) > x = "five"; > else > x = [0, 0, 0, 0, 0]; > > (adjust syntax to whatever language you like) > > Does this mean that the type of x is int|string|list, or will this be > an error? Assuming the condition can't be known until run time (eg it > involves user input), there's no way for a static analyzer to > differentiate between this code and the form that Steven put forward: I'm thinking of a dynamically typed language, so the type of x is the type of a value, so: Before the conditional, x is known to be 5 (an int). After the conditional, x is known to be "five" or [0,0,0,0,0] (a string or a list of int; not an int). If the next statement is to return -x, *that* is an error, because -x does not make sense after either branch of the conditional. It the next statement is to return x.swapcase(), the compiler can replace the conditional with if (some_condition) return "FIVE" # assuming local x else raise Objection("list don't have no .swapcase() method") In no case would I say that a mere assignment to a variable is a type error in a dynamically typed language. >> x = 1 >> x = "hello" # a type error, at compile time > > Simple type inference would either see this as meaning that x is > int|string, or possibly it'd say "x is an int up to that second line, > and a string thereafter" (which is basically like dynamic typing but > statically checked - it's the value, not the variable, that has a > type, and checks like x.upper() would take note of that). But if it > flags it as an error, that would basically mean that the type system > is (probably deliberately) simplistic and restrictive, requiring that > x be EITHER an integer variable OR a string variable, and not both. I'd say that the compiler of a dynamically typed language has different information about the type of (the value of) x after the first statement and after the second statement. If that is considered an error instead, as the comment says, then the language is statically typed (the type pertains to the variable). > Which is a perfectly viable stance, but I'm just not sure if it's (a) > what is done, or (b) ideal. Particularly since it'd end up requiring > some annoying rules, like "integers and floats are compatible, but > nothing else, including user-defined types" or "integers and floats > are fundamentally different things, and if you want your variable ever > to contain a float, you have to always use 1.0 instead of just 1", > neither of which I like. I suppose statically typed type-inferencing languages do that, but I don't have much experience with them. They may not be happy until they can infer a concrete implementation type for every variable, and there may be some awkward corners then. From steve+comp.lang.python at pearwood.info Sun Aug 28 04:33:40 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Aug 2016 18:33:40 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> On Sunday 28 August 2016 15:29, Chris Angelico wrote: > On Sun, Aug 28, 2016 at 2:30 PM, Steve D'Aprano > wrote: >> But the author of this piece ignores that standard distinction and invents >> his own non-standard one: to him, classes are merely different >> representations of the same data. E.g. his example of complex numbers, >> shown as Cartesian (x, y) values or polar (r, ?) values. These aren't two >> different "kinds of things", but merely two different ways of representing >> the same entity. >> >> That's not a good way to think about (say) Python lists and Python bools. >> Lists and bools are in no way the same kind of entity (except in the most >> general category of "they're both objects"). >> >> It's not even a very good way of thinking about complex numbers. > > It might be a good way of thinking about points on a Cartesian plane, > though. Rectangular and polar coordinates truly are just different > ways of expressing the same information. That's exactly my point, and that's why you shouldn't implement them as different classes. If you do, that's a limitation of your code and/or the language. (There may be *implementation specific* reasons why you are forced to, for example to avoid rounding errors due to finite precision: say, my polar number (1, 45?) may not evaluate as *exactly* (sqrt(2)/2, sqrt(2)/2) in Cartesian coordinates due to rounding. But that's a case of a leaky abstraction.) > (How well 2D coordinates map > to complex numbers is a separate question.) Mathematically speaking, they map together perfectly well. [...] > This is where I'm less sure. Sometimes a variable's type should be > broader than just one concrete type - for instance, a variable might > hold 1 over here, and 1.5 over there, and thus is storing either "int > or float" or "any number". If you have a complex hierarchy of types, > how do you know that this variable should be allowed to hold anything > up to a certain level in the hierarchy, and no further? This depends on the sophistication of the type system and support (or lack of support) for polymorphism: https://en.wikipedia.org/wiki/Polymorphism_%28computer_science%29 Type punning is normally considered a way to subvert or bypass the type system, and is normally considered a bad but necessary thing: https://en.wikipedia.org/wiki/Type_punning In general, primitive type systems like that used by Pascal (and C?) don't deal well, or at all, with the scenario you describe. Often the built-in functions can hard-code support for multiple numeric types, automatically promoting one type to another as necessary, but the same effect is almost impossible to achieve in (say) standard Pascal. Other type-checkers can deal better with polymorphism. But there's a trade-off: the more kinds of things a value or variable might be, the less certain you are of what is allowed ahead of time. That's why dynamic typed languages traditionally skipped *all* ahead-of-time type checking and instead relied entirely on runtime type errors, while traditional compilers restrict what you can do as the trade-off for catching more errors ahead of time. (That's where the reputation for flexibility of dynamic typing comes from: you never need to fight the compiler to do something you know will be okay, like passing an int to a function that expects a float.) I might be able to tell the compiler that x is Union[int, str] (a number, or a string) but that limits the ability of the compiler to tell what is and what isn't safe. If I declare that x is either an int or a str, what can we say about x.upper()? Is it safe? If x happens to be an int at the time we call x.upper(), will the language raise a runtime exception or will it blindly try to execute some arbitrary chunk of memory as the upper() method? This is why static and dynamic typing are slowly converging: statically typed languages are slowly gaining dynamic features, like C++ and vtables: https://en.wikipedia.org/wiki/Virtual_method_table while dynamically typed languages are slowly gaining smarter compilers capable of doing some "best effort" compile-time type-checking. Or at least allowing external type-checkers/linters to do so. The bottom line is that if a human reader can read the source code and deduce that x.upper() is safe because in this branch of the code, x must be a string rather than an int, then *in principle* a type-checker could do the same. Possibly better than a human, or possibly worse. Depends on the intelligence of the type-checker and the code being checked. A good enough type-checker can find infinite loops: http://perl.plover.com/yak/typing/notes.html > If what the compiler's doing is identifying what *is* assigned, then > it's easy. You've given it an int over here and a float over there, > and that's legal; from that point on, the compiler knows that this > contains either an int or a float. (Let's assume it can't know for > sure which, eg it has "if (cond) x=1; else x=1.5" where the condition > can't be known till run-time.) But for your example of x="hello" to be > a compilation error, it has to either assume that the first object > given determines the type completely, or be told what types are > permitted. I'm not aware of any [about Pike] > string(8bit)|int x(0..) = 12345; > > which would allow x to store a byte-string (an eight-bit string, as > opposed to a Unicode string which stores text) or a non-negative > integer. Okay. What happens when you say: if random() < 0.5: x = 1234 else: x = "surprise!" y = 3*(x + 1) z = x.find("p") # or however Pike does string functions/methods What is y? What is z? There are solutions to this conundrum. One is weak typing: 3*("surprise!" + 1) evaluates as 3*(0 + 1) or just 3, while (1234).find("p") coerces 1234 to the string "1234". Another is runtime exceptions. A third is "don't do that, if you do, you can deal with the segmentation fault". A fourth would be that the type-checker is smart enough to recognise that only one of those two assignments is valid, the second must be illegal, and flag the whole thing. That's what a human would do -- I don't know if any type systems are that sophisticated. -- Steve From test at joepgen.com Sun Aug 28 05:13:29 2016 From: test at joepgen.com (Joe) Date: Sun, 28 Aug 2016 11:13:29 +0200 Subject: Multimeter USB output Message-ID: Am 28.08.2016 um 00:45 schrieb Terry Reedy: > On 8/27/2016 3:35 PM, Joe wrote: >> Hi, >> >> I'm using Python 3.5.1 with PyUSB 1.0 under Win 10 (64). We try to read >> the USB output of a DMM 'UT61B'. >> >> import usb.core >> import usb.util >> import usb.backend.libusb1 >> >> def Gosub(): >> dev = usb.core.find(idVendor=0x1a86, idProduct=0xe008) # Digital >> Multimeter UT61B >> if dev == None: >> print ('Multimeter not found') >> else: >> print ('Multimeter was found') >> dev.set_configuration() >> cfg = dev.get_active_configuration() >> intf = cfg[(0,0)] >> ep = usb.util.find_descriptor( >> intf, >> custom_match = \ >> lambda e: \ >> usb.util.endpoint_direction(e.bEndpointAddress) == \ >> usb.util.ENDPOINT_IN) >> if ep == None: >> print ('ep is None') >> else: >> s = ep.read(64, 500) >> print ('Len s: ' + len(s)) >> >> print ('Starting') >> Gosub() >> print ('Ready.-') >> >> Result: > > I presume you saw > Starting > Multimeter was found > >> File "d:\work-d\PythonProgs\ut61b.py", line 27, in >> Gosub() >> File "d:\work-d\PythonProgs\ut61b.py", line 23, in Gosub >> s = ep.read(64, 500) >> File "D:\Python3\Lib\site-packages\usb\core.py", line 402, in read >> return self.device.read(self, size_or_buffer, timeout) >> File "D:\Python3\Lib\site-packages\usb\core.py", line 988, in read >> self.__get_timeout(timeout)) >> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 851, in >> intr_read >> timeout) >> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 936, in >> __read >> _check(retval) >> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 595, in >> _check >> raise USBError(_strerror(ret), ret, _libusb_errno[ret]) >> >> usb.core.USBError: [Errno 10060] Operation timed out >> >> What's wrong? How to fix? > > Read (again?) the doc for the interface for the device. Because reading > timed out, I suspect that it is waiting for a command for it to send > something. > Yes, I saw this: Starting Multimeter was found The UT61B has two interfaces, a RS232C interface and this usb interface. The RS232 interface works well with PySerial. It continously transmits 2 .. 3 Pakets per second with 14 Bytes each. This happens unsolicited without any command as long as the RS232C/USB button on the DMM is active. So I assumed the USB interface also doesn't need any command and also transmit this stream of 2 to 3 Pakets per second. But unfortunately I don't have any doc for the USB interface for this device. To the accompanying software of the UT61B there is a ready windos app which shows and logs the output of the UT61B. This app can be switched between RS232C and USB; both work. I asked the manufacturer (Uni-T in Shenzen) for additional info and are now waiting for an answer. Assumed the USB interface sends this 2 to 3 pakets per second unsolicited - should the code shown work? Is this ok: >> lambda e: \ >> usb.util.endpoint_direction(e.bEndpointAddress) == \ >> usb.util.ENDPOINT_IN) I am in doubt: Is usb.util.ENDPOINT_IN really correct? Thanx - Joe From test at joepgen.com Sun Aug 28 05:25:25 2016 From: test at joepgen.com (Joe) Date: Sun, 28 Aug 2016 11:25:25 +0200 Subject: Is duck-typing misnamed? In-Reply-To: References: <1472255923l.39846110l.0l@psu.edu> Message-ID: Am 28.08.2016 um 00:34 schrieb Terry Reedy: > On 8/26/2016 7:58 PM, ROGER GRAYDON CHRISTMAN wrote: >> "If it walks like a duck, quacks like a duck,... " >> >> so there is indeed precedence for this so-called 'duck typing' >> >> >> but wouldn't it be more Pythonic to call this 'witch typing'? >> >> "How do you know she is a witch?" >> >> "She looks like one." > > Given that people were once burned to death for 'looking like a witch' > (or sounding or acting), and can still suffer socially for such reasons, > this it not funny to me. We should stick with ducks. > Agree. From rosuav at gmail.com Sun Aug 28 05:28:02 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Aug 2016 19:28:02 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> Message-ID: On Sun, Aug 28, 2016 at 6:33 PM, Steven D'Aprano wrote: > On Sunday 28 August 2016 15:29, Chris Angelico wrote: >> It might be a good way of thinking about points on a Cartesian plane, >> though. Rectangular and polar coordinates truly are just different >> ways of expressing the same information. > > That's exactly my point, and that's why you shouldn't implement them as > different classes. If you do, that's a limitation of your code and/or the > language. > > (There may be *implementation specific* reasons why you are forced to, for > example to avoid rounding errors due to finite precision: say, my polar number > (1, 45?) may not evaluate as *exactly* (sqrt(2)/2, sqrt(2)/2) in Cartesian > coordinates due to rounding. But that's a case of a leaky abstraction.) class Complex(complex): @property def r(self): return abs(self) @property def phi(self): return cmath.phase(self) One value, two ways of looking at it. One type. One class. > I might be able to tell the compiler that x is Union[int, str] (a number, or a > string) but that limits the ability of the compiler to tell what is and what > isn't safe. If I declare that x is either an int or a str, what can we say > about x.upper()? Is it safe? If x happens to be an int at the time we call > x.upper(), will the language raise a runtime exception or will it blindly try > to execute some arbitrary chunk of memory as the upper() method? That's fine if you *tell* the compiler this. My question came from your statement that a type *inference* system can detect errors of assignment - not method/operator usage. A dynamic type inference system could easily cope with this: x = 5 y = x**2 x = "five" z = x.upper() and correctly deduce that, at the time of y's assignment, exponentiation of x was legal, and at the time of z's, uppercasing was. But you said that the type system could flag the third line as an error, saying "hey, I'm expecting this to be integers only". Here's what you said: > x = 1 > x = "hello" # a type error, at compile time If I were doing type inference, with my limited knowledge of the field, I would do one of two things: 1) Infer that x holds only integers (or maybe broaden it to "numbers"), and then raise an error on the second line; this basically restricts the type system to be union-free 2) Infer that x holds Union[int, str] in PEP 484 notation, or int|string in Pike notation, and permit it to carry either type. Which way is it? Do you get errors, as per your example, and thus are never allowed to have union types? And if so, what happens with compatible types (notably, int and float)? Can user-defined types be deemed "compatible"? Are the same types always compatible? Or are no types ever compatible, and you just have a single Number type, like in ECMAScript? > Okay. What happens when you say: > > if random() < 0.5: > x = 1234 > else: > x = "surprise!" > > y = 3*(x + 1) > z = x.find("p") # or however Pike does string functions/methods > > > What is y? What is z? In Pike, variables get declared. So we have a few possibilities: 1) Declaration was "int x;" and the else clause is a compile-time error 2) Declaration was "string x;" and the converse 3) Declaration was "int|string x;" or "mixed x;" or some other broad form, and they are both accepted. > There are solutions to this conundrum. One is weak typing: 3*("surprise!" + 1) > evaluates as 3*(0 + 1) or just 3, while (1234).find("p") coerces 1234 to the > string "1234". > > Another is runtime exceptions. In this example, y would be "surprise!1surprise!1surprise!1", because Pike allows strings and integers to be added (representing the int with ASCII decimal digits), but if the expression were 3*(x-1) instead, then these would be run-time exceptions. Pike, like Python, strongly types its values, so if the variable declaration doesn't prevent something illogical from being compiled, it'll throw a nice tidy exception at you. The find method, being applied to an integer, would definitely be an exception, more-or-less "integers don't have such a method, moron". > A third is "don't do that, if you do, you can deal with the segmentation > fault". Only in C, where segfaults are considered a normal part of life. In high level languages, no thank you. > A fourth would be that the type-checker is smart enough to recognise that only > one of those two assignments is valid, the second must be illegal, and flag the > whole thing. That's what a human would do -- I don't know if any type systems > are that sophisticated. Ooh that would be VERY sophisticated. I don't know of anything that does that, but it could be done on the same basis as the C "undefined behaviour" thing you so revile against - basically, the compiler says "well, if the programmer knows what he's doing, x MUST be an integer at this point, ergo I can assume that it really will be an integer". I've seen levels of sophistication like that in tools like Coverity and how it detects buffer size problems or null pointers, so it wouldn't surprise me (though it would impress me!) if something could deduce this kind of incompatibility. ChrisA From tjreedy at udel.edu Sun Aug 28 06:31:51 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 28 Aug 2016 06:31:51 -0400 Subject: Multimeter USB output In-Reply-To: References: Message-ID: On 8/28/2016 5:13 AM, Joe wrote: > > Am 28.08.2016 um 00:45 schrieb Terry Reedy: >> On 8/27/2016 3:35 PM, Joe wrote: >>> Hi, >>> >>> I'm using Python 3.5.1 with PyUSB 1.0 under Win 10 (64). We try to read >>> the USB output of a DMM 'UT61B'. >>> >>> import usb.core >>> import usb.util >>> import usb.backend.libusb1 >>> >>> def Gosub(): >>> dev = usb.core.find(idVendor=0x1a86, idProduct=0xe008) # Digital >>> Multimeter UT61B >>> if dev == None: >>> print ('Multimeter not found') >>> else: >>> print ('Multimeter was found') >>> dev.set_configuration() >>> cfg = dev.get_active_configuration() >>> intf = cfg[(0,0)] >>> ep = usb.util.find_descriptor( >>> intf, >>> custom_match = \ >>> lambda e: \ >>> usb.util.endpoint_direction(e.bEndpointAddress) == \ >>> usb.util.ENDPOINT_IN) >>> if ep == None: >>> print ('ep is None') >>> else: >>> s = ep.read(64, 500) >>> print ('Len s: ' + len(s)) >>> >>> print ('Starting') >>> Gosub() >>> print ('Ready.-') >>> >>> Result: >> >> I presume you saw >> Starting >> Multimeter was found >> >>> File "d:\work-d\PythonProgs\ut61b.py", line 27, in >>> Gosub() >>> File "d:\work-d\PythonProgs\ut61b.py", line 23, in Gosub >>> s = ep.read(64, 500) >>> File "D:\Python3\Lib\site-packages\usb\core.py", line 402, in read >>> return self.device.read(self, size_or_buffer, timeout) >>> File "D:\Python3\Lib\site-packages\usb\core.py", line 988, in read >>> self.__get_timeout(timeout)) >>> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 851, in >>> intr_read >>> timeout) >>> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 936, in >>> __read >>> _check(retval) >>> File "D:\Python3\Lib\site-packages\usb\backend\libusb1.py", line 595, in >>> _check >>> raise USBError(_strerror(ret), ret, _libusb_errno[ret]) >>> >>> usb.core.USBError: [Errno 10060] Operation timed out >>> >>> What's wrong? How to fix? >> >> Read (again?) the doc for the interface for the device. Because reading >> timed out, I suspect that it is waiting for a command for it to send >> something. >> > > Yes, I saw this: > > Starting > Multimeter was found > > The UT61B has two interfaces, a RS232C interface and this usb interface. > The RS232 interface works well with PySerial. It continously transmits 2 > .. 3 Pakets per second with 14 Bytes each. This happens unsolicited > without any command as long as the RS232C/USB button on the DMM is active. > > So I assumed the USB interface also doesn't need any command and also > transmit this stream of 2 to 3 Pakets per second. But unfortunately I > don't have any doc for the USB interface for this device. > > To the accompanying software of the UT61B there is a ready windos app > which shows and logs the output of the UT61B. This app can be switched > between RS232C and USB; both work. > > I asked the manufacturer (Uni-T in Shenzen) for additional info and are > now waiting for an answer. > > Assumed the USB interface sends this 2 to 3 pakets per second > unsolicited - should the code shown work? Is this ok: > > >>> lambda e: \ >>> usb.util.endpoint_direction(e.bEndpointAddress) == \ >>> usb.util.ENDPOINT_IN) > > I am in doubt: Is usb.util.ENDPOINT_IN really correct? I do not know as I have not used the usb package. -- Terry Jan Reedy From esj at harvee.org Sun Aug 28 10:09:47 2016 From: esj at harvee.org (Eric S. Johansson) Date: Sun, 28 Aug 2016 10:09:47 -0400 Subject: Is duck-typing misnamed? In-Reply-To: <1472340488l.28639294l.0l@psu.edu> References: <1472255923l.39846110l.0l@psu.edu> <20160827222750.GA26497@cskk.homeip.net> <1472340488l.28639294l.0l@psu.edu> Message-ID: <97e6c7fd-0c01-87b8-a516-9fad85dff1ef@harvee.org> On 8/27/2016 7:28 PM, ROGER GRAYDON CHRISTMAN wrote: > Your response is appreciated. I just thought I'd comment a little more on the > script: > > Woman: I'm not a witch! I'm not a witch! > > V: ehh... but you are dressed like one. > > W: They dressed me up like this! > > All: naah no we didn't... no. > > W: And this isn't my nose, it's a false one. > > (V lifts up carrot) > > V: Well? > > P1: Well we did do the nose > > V: The nose? > > P1: ...And the hat, but she is a witch! > > > They took a woman who originally, I think we might agree, was not a witch, > and they added features that were understood to be part of the protocol > for witchiness. not a witch?? https://www.youtube.com/watch?v=zrzMhU_4m-g start at 3:30 listen to what she says after they discover she weights as much as a duck. From jpablo.romero at gmail.com Sun Aug 28 14:25:06 2016 From: jpablo.romero at gmail.com (=?UTF-8?Q?Juan_Pablo_Romero_M=C3=A9ndez?=) Date: Sun, 28 Aug 2016 11:25:06 -0700 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c28d00$0$11098$c3e8da3@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c28d00$0$11098$c3e8da3@news.astraweb.com> Message-ID: 2016-08-28 0:04 GMT-07:00 Steven D'Aprano < steve+comp.lang.python at pearwood.info>: > On Sunday 28 August 2016 15:56, Juan Pablo Romero M?ndez wrote: > > > 2016-08-27 21:30 GMT-07:00 Steve D'Aprano : > [...] > >> Now it is true that speaking in full generality, classes and types > refer to > >> different things. Or to be perhaps more accurate, *subclassing* and > >> *subtyping* are different things: > >> > >> http://c2.com/cgi/wiki?SubTypingAndSubClassing > >> > >> Many languages treat them the same, but fundamentally they are > different. > > > > Oh, I don't think he is thinking in terms of OO "classes", I think he > meant > > two different "kinds" or "varieties" of values (although kind has a > > technical meaning) > > That's not quite right either. "Two different kinds or varieties" is what > both > subclassing and subtyping aim to describe, in different ways: e.g. dogs and > cats are two different kinds of mammal. > > What you are describing here: > > > In TypeScript terms what he is saying can be described like this: > > > > type Complex = > > { real: number, i: number } > > | { r: number, ?: number} > > > > const c1: Complex = { real: 1, i: 1 } > > const c2: Complex = { r: 1, ?: 0.5 } > > > > You have two values of the same type but different representation. > > seems to be more like what is called "variant record" in Pascal. > > http://www.freepascal.org/docs-html/ref/refsu16.html > > > I'm not familiar with TypeScript. How does this work? If I say: > > const c1: Complex = {real: 1, imag: 1 } > > print c1.r > print c1.? > > what do I get? If r and ? are merely alternative names for "real" and > "imag", > that's not good enough. Given real=1, imag=1, then we need to get > r=1.414213 > and ? = pi/4 without any extra effort on the part of the user. > > Here's a more complete example: type CartesianC = { real: number, i: number } type PolarC = { r: number, ?: number} type Complex = PolarC | CartesianC const c1: Complex = { real: 1, i: 1 }; const c2: Complex = { r: 1, ?: 0.5 }; // This is called a Type Guard function isCartesian(c: Complex): c is CartesianC { return ( c).real !== undefined; } if(isCartesian(c1)) { // c1 is a CartesianC here c1.real } else { // and a PolarC here. Using c1.real is a compile error c1.r } TypeScript doesn't support pattern matching so there's some boilerplate involved: you need to define a TypeGuard so that inside the if branch TS allows you to treat c1 as a CartesianC; within the else branch it is treated as a PolarC. > > The point is, the complex number (1, 1) in Cartesian coordinates and > (sqrt(2), > pi/4) in polar coordinates aren't two different kinds of things, they are > two > different ways of writing the same value. Like writing 1A in hex and 26 in > decimal. Somebody may choose to implement this as two different classes > ("CartesianComplex" and "PolarComplex") but that's a limitation of their > code, > or of their language, it doesn't reflect a real difference between two > different kinds of things. > Well conceptually you might think of c1 as the abstract representation of a complex number but at runtime they have very different constitutions. > > Another way to put it: > > If you were programming a first person shooter game, you might choose each > of > the enemies as an object. Let's say you have an enemy called the Grue. You > can > view the Grue from the front or from the back, depending on which way it is > standing when you see it. Would you implement this as two different > classes? > > GrueSeenFromFront > > GrueSeenFromBack > > I should hope not. It's the same object, the same Grue, it just looks > different > depending on which way you approach it. > > > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From torriem at gmail.com Sun Aug 28 18:26:01 2016 From: torriem at gmail.com (Michael Torrie) Date: Sun, 28 Aug 2016 16:26:01 -0600 Subject: Is duck-typing misnamed? In-Reply-To: <1472340488l.28639294l.0l@psu.edu> References: <1472255923l.39846110l.0l@psu.edu> <20160827222750.GA26497@cskk.homeip.net> <1472340488l.28639294l.0l@psu.edu> Message-ID: On 08/27/2016 05:28 PM, ROGER GRAYDON CHRISTMAN wrote: > They took a woman who originally, I think we might agree, was not a witch, Umm no, she was actually a witch. Which makes the scene even funnier. "Fair caught," she says at the end. From ben+python at benfinney.id.au Sun Aug 28 20:34:50 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 29 Aug 2016 10:34:50 +1000 Subject: Is duck-typing misnamed? References: <1472255923l.39846110l.0l@psu.edu> <20160827222750.GA26497@cskk.homeip.net> <1472340488l.28639294l.0l@psu.edu> Message-ID: <85oa4cv2dx.fsf@benfinney.id.au> Michael Torrie writes: > Umm no, she was actually a witch. Which makes the scene even funnier. > "Fair caught," she says at the end. She says [0] ?It's a fair cop?, which is using the term ?cop? to mean the arrest or sentence, asserting that it's justified. Hence, the British term ?copper?, meaning a police officer: the one who does the cop (the capture or arrest) for a crime. has quotations showing usage, as does . [0] -- \ ?Most people are other people. Their thoughts are someone | `\ else?s opinions, their lives a mimicry, their passions a | _o__) quotation.? ?Oscar Wilde, _De Profundis_, 1897 | Ben Finney From steve+python at pearwood.info Sun Aug 28 22:43:32 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 29 Aug 2016 12:43:32 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> Message-ID: <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> On Sun, 28 Aug 2016 07:28 pm, Chris Angelico wrote: > On Sun, Aug 28, 2016 at 6:33 PM, Steven D'Aprano > wrote: >> On Sunday 28 August 2016 15:29, Chris Angelico wrote: >>> It might be a good way of thinking about points on a Cartesian plane, >>> though. Rectangular and polar coordinates truly are just different >>> ways of expressing the same information. >> >> That's exactly my point, and that's why you shouldn't implement them as >> different classes. If you do, that's a limitation of your code and/or the >> language. [... snip digression over possible leaky abstractions due to floating point rounding ...] > class Complex(complex): > @property > def r(self): > return abs(self) > @property > def phi(self): > return cmath.phase(self) > > One value, two ways of looking at it. One type. One class. I can't tell if you're saying this to agree with me or to disagree with me. >> I might be able to tell the compiler that x is Union[int, str] (a number, >> or a string) but that limits the ability of the compiler to tell what is >> and what isn't safe. If I declare that x is either an int or a str, what >> can we say about x.upper()? Is it safe? If x happens to be an int at the >> time we call x.upper(), will the language raise a runtime exception or >> will it blindly try to execute some arbitrary chunk of memory as the >> upper() method? > > That's fine if you *tell* the compiler this. I trust you don't actually mean that it is fine for a high-level (non-assembly) language to blindly execute some arbitrary memory address. > My question came from > your statement that a type *inference* system can detect errors of > assignment - not method/operator usage. Ignore the question of inference versus declaration. At least for simple cases, there's no real difference between the C-like declaration and assignment: int x = 5; and the assignment: x = 5; It doesn't require super-human, or even human, intelligence to infer that if x is assigned the value 5, x must be an int. So a *simple* inference engine isn't very sophisticated. Your question seems to be, what happens if you follow that with an assignment to a different type? x = 5 some_code(x) x = "hello world" Will the type-checker consider that an error ("you're assigning a str to an int") or will it infer that x is the Union[int, str]? Surely that depends on the type-checker! I don't think there's any hard and fast rule about that, but as far as I know, all statically typed languages consider than an error. Remember, that's the definition of *static typing*: the variable carries the type, not the value. So x is an int, and "hello world" isn't an int, so this MUST be an error. In dynamically typed languages... I don't know. I think that (again) I would expect that *by default* the checker should treat this as an invalid assignment. The whole point of a static type-checker is to bring some simulacrum of static typing to a dynamic language, so if you're going to enthusiastically infer union types every time you see an unexpected type assignment, it sort of defeats the purpose... "x was declared int, and you then call function foo() which is declared to return an int, but sometimes returns a str... oh, they must have meant a union, so that's okay..." > A dynamic type inference > system could easily cope with this: > > x = 5 > y = x**2 > x = "five" > z = x.upper() > > and correctly deduce that, at the time of y's assignment, > exponentiation of x was legal, and at the time of z's, uppercasing > was. Could it? Do you have an example of a language or type-checker that can do that? This isn't a rhetorical question. My understanding is that all the standard algorithms for checking types are based on the principle that a variable only has a single type in any one scope. Global x and local x may be different, but once you explicitly or implicitly set x to an int, then you can't set it to a str. So I'm not sure that existing compile-time type-checkers could deal with that, even in a dynamic language. At best they might say "oh well, x is obviously duck-typed, so don't bother trying to check it". At least that's my understanding -- perhaps I'm wrong. I daresay that what you want is *possible*. If the human reader can do it, then an automated checker should be able to. The question is not "is this possible?" but "has anyone done it yet?". > But you said that the type system could flag the third line as an > error, saying "hey, I'm expecting this to be integers only". Here's > what you said: > >> x = 1 >> x = "hello" # a type error, at compile time > > If I were doing type inference, with my limited knowledge of the > field, I would do one of two things: > > 1) Infer that x holds only integers (or maybe broaden it to > "numbers"), and then raise an error on the second line; this basically > restricts the type system to be union-free No, it only means that the system won't *infer* type unions just from assignment. And that's probably what we want: the whole point is to catch type errors. One type error is to x = 1 # type:int # later... x = spam() do_something_with(x) # that expects an int the type-checker should check that spam() will always return an int. If there are any circumstances where spam() will actually return (say) a str, this should be an error, not just infer that it could be either an int or a str. That would mean you could never catch type errors in assignments. There are other ways that you can infer a union type. The most obvious is to declare it, but MyPy can infer that x is Union[int, str] following: if isinstance(x, int) or isinstance(x, str): ... http://mypy-lang.blogspot.com.au/2016/07/mypy-043-released.html But it really depends on how smart the type inference is. A simple-minded one might be no smarter than like Pascal type-checking, except you don't need the initial var declaration. The MyPy one is pretty smart. > 2) Infer that x holds Union[int, str] in PEP 484 notation, or > int|string in Pike notation, and permit it to carry either type. Unless the type-checker was *really* smart, that would probably defeat the purpose. > Which way is it? Do you get errors, as per your example, and thus are > never allowed to have union types? And if so, what happens with > compatible types (notably, int and float)? Who says they're compatible? py> (23).bit_length() 5 py> (23.0).bit_length() Traceback (most recent call last): File "", line 1, in AttributeError: 'float' object has no attribute 'bit_length' Some languages (Erlang?) are so strict that they forbid mixed arithmetic and require you to explicitly coerce one type to another. That's not a bad idea... consider adding an int n to a float x, can that ever raise? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Aug 28 22:51:46 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 29 Aug 2016 12:51:46 +1000 Subject: Is duck-typing misnamed? References: <1472255923l.39846110l.0l@psu.edu> Message-ID: <57c3a343$0$1583$c3e8da3$5496439d@news.astraweb.com> On Sun, 28 Aug 2016 08:34 am, Terry Reedy wrote: > On 8/26/2016 7:58 PM, ROGER GRAYDON CHRISTMAN wrote: >> "If it walks like a duck, quacks like a duck,... " >> >> so there is indeed precedence for this so-called 'duck typing' >> >> >> but wouldn't it be more Pythonic to call this 'witch typing'? >> >> "How do you know she is a witch?" >> >> "She looks like one." > > Given that people were once burned to death for 'looking like a witch' > (or sounding or acting), and can still suffer socially for such reasons, > this it not funny to me. We should stick with ducks. Black humour is still humour. And it is an important way of dealing with distress, and of instituting social change. Belief in the supernatural and superstition is on the rise again, including witchcraft. If it were limited to just a few benighted and ignorant migrants from Africa, that would be bad enough, but I see disturbing signs that the Satanic Ritual panic from the 80s and 90s is on its way back. (Or perhaps it never quite went away, just faded from the headlines.) Likewise the anti-paedophile frenzy, where the mere (false) accusation of paedophilia is enough to start a frenzy of abuse and even killing: http://www.telegraph.co.uk/news/uknews/crime/10422771/How-wild-rumour-led-a-mob-to-murder-an-innocent-man.html Black humour is a weapon against the ignorance and intolerance that feeds hate crimes and witch hunts (whether legally sanctioned or not, whether about literal witches or any other demonised subgroup). We can and should take every opportunity to remind people of the absurdity of relying on torture to gain confessions, and the abuses of this sort of single-minded, hysterical moral panic. And humour is a most effective way to do so. Nobody likes to be hectored and lectured as I'm lecturing you now *wink* but turning it into a joke can get the point across. We should not lose sight of the economic, political, racial reasons for witch-hunts, but equally we should not forget that when a moral panic is in full force, people behave absurdly, and the best antidote to absurd behaviour is to take the mickey out of it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Aug 28 23:09:28 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Aug 2016 13:09:28 +1000 Subject: What's the best way to minimize the need of run time checks? Message-ID: On Mon, Aug 29, 2016 at 12:43 PM, Steve D'Aprano wrote: > On Sun, 28 Aug 2016 07:28 pm, Chris Angelico wrote: > >> On Sun, Aug 28, 2016 at 6:33 PM, Steven D'Aprano >> wrote: >>> On Sunday 28 August 2016 15:29, Chris Angelico wrote: >>>> It might be a good way of thinking about points on a Cartesian plane, >>>> though. Rectangular and polar coordinates truly are just different >>>> ways of expressing the same information. >>> >>> That's exactly my point, and that's why you shouldn't implement them as >>> different classes. If you do, that's a limitation of your code and/or the >>> language. > [... snip digression over possible leaky abstractions due to floating point > rounding ...] > >> class Complex(complex): >> @property >> def r(self): >> return abs(self) >> @property >> def phi(self): >> return cmath.phase(self) >> >> One value, two ways of looking at it. One type. One class. > > I can't tell if you're saying this to agree with me or to disagree with me. Agreeing, and positing that you don't even need two ways of storing it (modulo FP rounding) - just two ways of *looking* at it. >>> I might be able to tell the compiler that x is Union[int, str] (a number, >>> or a string) but that limits the ability of the compiler to tell what is >>> and what isn't safe. >> >> That's fine if you *tell* the compiler this. > > I trust you don't actually mean that it is fine for a high-level > (non-assembly) language to blindly execute some arbitrary memory address. Quote trimmed to clarify my point. Of course I don't want a high level language to blindly execute random memory. > Your question seems to be, what happens if you follow that with an > assignment to a different type? > > x = 5 > some_code(x) > x = "hello world" > > > Will the type-checker consider that an error ("you're assigning a str to an > int") or will it infer that x is the Union[int, str]? > > Surely that depends on the type-checker! I don't think there's any hard and > fast rule about that, but as far as I know, all statically typed languages > consider than an error. Remember, that's the definition of *static typing*: > the variable carries the type, not the value. So x is an int, and "hello > world" isn't an int, so this MUST be an error. So in statically-typed inferred-type languages, unions are impossible. Got it. That's perfectly acceptable with strings and integers, but ints and floats are more problematic. More on that below. > In dynamically typed languages... I don't know. I think that (again) I would > expect that *by default* the checker should treat this as an invalid > assignment. The whole point of a static type-checker is to bring some > simulacrum of static typing to a dynamic language, so if you're going to > enthusiastically infer union types every time you see an unexpected type > assignment, it sort of defeats the purpose... > > "x was declared int, and you then call function foo() which is declared to > return an int, but sometimes returns a str... oh, they must have meant a > union, so that's okay..." Not really; you forfeit any kind of assignment checking (since assignment will simply expand the type union), but you still get static type checking of operators, methods, attributes, etc, and of function calls (passing a string|int to something that expects a list? Error!). You still get a lot of the benefit. >> A dynamic type inference >> system could easily cope with this: >> >> x = 5 >> y = x**2 >> x = "five" >> z = x.upper() >> >> and correctly deduce that, at the time of y's assignment, >> exponentiation of x was legal, and at the time of z's, uppercasing >> was. > > Could it? Do you have an example of a language or type-checker that can do > that? This isn't a rhetorical question. No, I don't, because the type-checking languages I use have declarations. But since C compilers are capable of detecting that "this variable isn't used after this point, so I can reuse its register", the equivalent in type checking should be possible. > My understanding is that all the standard algorithms for checking types are > based on the principle that a variable only has a single type in any one > scope. Global x and local x may be different, but once you explicitly or > implicitly set x to an int, then you can't set it to a str. So I'm not sure > that existing compile-time type-checkers could deal with that, even in a > dynamic language. At best they might say "oh well, x is obviously > duck-typed, so don't bother trying to check it". > > At least that's my understanding -- perhaps I'm wrong. > > I daresay that what you want is *possible*. If the human reader can do it, > then an automated checker should be able to. The question is not "is this > possible?" but "has anyone done it yet?". Understood. That's what I wanted to clear up - that type inference systems aim for a single type for any given variable. > No, it only means that the system won't *infer* type unions just from > assignment. And that's probably what we want: the whole point is to catch > type errors. > > There are other ways that you can infer a union type. The most obvious is to > declare it, but MyPy can infer that x is Union[int, str] following: > > if isinstance(x, int) or isinstance(x, str): > ... > Yep, that answers that question. Basically, an inferred type MUST come from a single logical unit, eg a single statement. That makes fine sense, as long as there is a way to declare union types. >> Which way is it? Do you get errors, as per your example, and thus are >> never allowed to have union types? And if so, what happens with >> compatible types (notably, int and float)? > > Who says they're compatible? > > > py> (23).bit_length() > 5 > py> (23.0).bit_length() > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'float' object has no attribute 'bit_length' > > Some languages (Erlang?) are so strict that they forbid mixed arithmetic and > require you to explicitly coerce one type to another. That's not a bad > idea... consider adding an int n to a float x, can that ever raise? It can, and that's the thing. You could simply declare that floats and ints are completely and utterly different beasts: 1.0 != 1 1.0 + 1 # TypeError x = 0; x = 1.5; # TypeError But Python has deemed, and I think a lot of people here would agree, that there's an abstract concept of a "number", and that float and int (and complex) are different representations of them - and that a float is equal to an int if they represent the same number, they can be summed, etc, etc, etc. I'm pretty sure most people on this list (not all, but most) agree with the Python 3.0 change that means that the quotient of the integers 7 and 2 is the float 3.5, not the integer 3. In terms of a type checker, we end up with something like this: value = 0 for flag in collection: if flag.condition1: value += 0.9 if flag.condition2: value *= 2 If ints and floats are type-incompatible, the third line would throw an error, because you'd clearly be assigning a float to an integer variable. So you have to say "value = 0.0" at the top? Okay. What about the multiplication? "value *= 2.0"? Do we have to mark every single value with a decimal to make them compatible with floats? Seems a lot of unnecessary bother. The one thing I really REALLY do not want to see is some kind of internal magic that makes ints and floats compatible, without extending that offer to user-defined types. Complete incompatibility would work, but probably not in Python. ChrisA From test at joepgen.com Mon Aug 29 04:54:19 2016 From: test at joepgen.com (Joe) Date: Mon, 29 Aug 2016 10:54:19 +0200 Subject: Multimeter USB output In-Reply-To: References: Message-ID: Am 28.08.2016 um 17:22 schrieb Dennis Lee Bieber: > If you can read spaghetti coded C, you might want to study > https://sourceforge.net/projects/ut61/ Interesting, but... The last time I did something with c, it was with BDS-C under CM/M. Somebody remenbering this no-fp compiler from the dark age before PC und Linux? From test at joepgen.com Mon Aug 29 04:56:30 2016 From: test at joepgen.com (Joe) Date: Mon, 29 Aug 2016 10:56:30 +0200 Subject: Multimeter USB output In-Reply-To: References: Message-ID: Am 29.08.2016 um 10:54 schrieb Joe: > it was with BDS-C under CM/M. under CP/M, of course. From mviljamaa at kapsi.fi Mon Aug 29 07:34:57 2016 From: mviljamaa at kapsi.fi (Matti Viljamaa) Date: Mon, 29 Aug 2016 14:34:57 +0300 Subject: freqz -style plot of "window design" FIR? Message-ID: <2617B74F-F69F-49AF-A876-FFE016151DFA@kapsi.fi> I?m trying to design an arbitrary frequency response filter as described here: http://www.dspguide.com/ch17/1.htm The technique is said to result in an impulse response in time domain and later in to a filter kernel. I?ve been using scipy.signal.freqz to make magnitude response plots: e.g. fs = 44100 # Design a low-pass filter using remez. cutoff = 2000.0 transition_width = 200 bands = np.array([0, cutoff - 0.5*transition_width, cutoff + 0.5*transition_width, fs/2.0]) / fs desired = [1, 0] lpf = remez(513, bands, desired) # Plot the frequency response of the filter. w, h = freqz(lpf) plt.figure(1) plt.plot(fs*w/(2*np.pi), 20*np.log10(abs(h))) plt.xlim(0, fs/2) plt.xlabel('Frequency (Hz)') plt.ylabel('Gain (dB)') plt.grid(True) But my question is, if using the above arbitrary frequency response design technique, would I be able to use freqz? freqz takes as a parameter ?numerator of a linear filter? and remez is returning an array of coefficients, which I read to be the same thing. But in the case of the arbitrary frequency response filter, what can I put into freqz? Is filter kernel perhaps the same as coefficients? From breamoreboy at gmail.com Mon Aug 29 07:46:51 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 29 Aug 2016 04:46:51 -0700 (PDT) Subject: Is duck-typing misnamed? In-Reply-To: References: <1472255923l.39846110l.0l@psu.edu> <20160827222750.GA26497@cskk.homeip.net> <1472340488l.28639294l.0l@psu.edu> <85oa4cv2dx.fsf@benfinney.id.au> Message-ID: <62035fcc-f74d-41c2-a4d4-a32943f27249@googlegroups.com> On Monday, August 29, 2016 at 12:08:26 PM UTC+1, Ben Finney wrote: > Michael Torrie writes: > > > Umm no, she was actually a witch. Which makes the scene even funnier. > > "Fair caught," she says at the end. > > She says [0] ?It's a fair cop?, which is using the term ?cop? to mean > the arrest or sentence, asserting that it's justified. > > Hence, the British term ?copper?, meaning a police officer: the one who > does the cop (the capture or arrest) for a crime. > Also used in the Dead Bishop sketch. Klaus: It's a fair cop, but society's to blame. Detective: Agreed. We'll be charging them too. Kindest regards. Mark Lawrence. From bc at freeuk.com Mon Aug 29 08:13:47 2016 From: bc at freeuk.com (BartC) Date: Mon, 29 Aug 2016 13:13:47 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/08/2016 03:43, Steve D'Aprano wrote: > Your question seems to be, what happens if you follow that with an > assignment to a different type? > > x = 5 > some_code(x) > x = "hello world" > > > Will the type-checker consider that an error ("you're assigning a str to an > int") or will it infer that x is the Union[int, str]? > > Surely that depends on the type-checker! I don't think there's any hard and > fast rule about that, but as far as I know, all statically typed languages > consider than an error. In C, you can write this: int x; x = 5; x = "hello"; With certain compilers (eg. gcc) you only get a warning. (And since I don't show warnings to avoid inundation, that seems to compile fine for me!) But you write: x = 5.1; with no errors or warnings. >> A dynamic type inference >> system could easily cope with this: >> >> x = 5 >> y = x**2 >> x = "five" >> z = x.upper() >> >> and correctly deduce that, at the time of y's assignment, >> exponentiation of x was legal, and at the time of z's, uppercasing >> was. > > Could it? Do you have an example of a language or type-checker that can do > that? This isn't a rhetorical question. Actually the above example is one of the easiest to perform type-inference on, where the variable has been assigned a known type on the previous line! -- Bartc From rosuav at gmail.com Mon Aug 29 08:18:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Aug 2016 22:18:14 +1000 Subject: PSA: Debian Stretch, apt-listchanges, and Python 3 Message-ID: If, like me, you build Python 3.6 from source and make it your default 'python3' binary, you may run into issues with the latest apt-listchanges, which has switched to using Python 3. The symptom is an ImportError trying to "import debconf", which is available in dist-packages in the system Python, but (naturally) isn't part of Python 3.6's install tree. My solution: sudo ln -s /usr/lib/python3/dist-packages/debconf.py /usr/local/lib/python3.6/site-packages/ And then apt-listchanges happily sets itself up. ChrisA From rosuav at gmail.com Mon Aug 29 08:31:39 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Aug 2016 22:31:39 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: > In C, you can write this: > > int x; > > x = 5; > x = "hello"; > > With certain compilers (eg. gcc) you only get a warning. (And since I don't > show warnings to avoid inundation, that seems to compile fine for me!) That's because strings, in C, are really pointers-to-char, and for hysterical raisins, pointers can be assigned to integers with just a warning. (Good code should have an explicit cast here.) You probably should have warnings enabled. There are a lot of faults that newer languages flag as errors, but due to backward compat restraints, C has to accept - but smart compilers will flag them as warnings. Getting inundated with warnings would be a major code smell. ChrisA From steve+python at pearwood.info Mon Aug 29 08:46:55 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 29 Aug 2016 22:46:55 +1000 Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> On Mon, 29 Aug 2016 10:31 pm, Chris Angelico wrote: > On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >> In C, you can write this: >> >> int x; >> >> x = 5; >> x = "hello"; >> >> With certain compilers (eg. gcc) you only get a warning. (And since I >> don't show warnings to avoid inundation, that seems to compile fine for >> me!) > > That's because strings, in C, are really pointers-to-char, and for > hysterical raisins, pointers can be assigned to integers with just a > warning. (Good code should have an explicit cast here.) Let me see if I've got this straight... Bart's second assignment will allocate a block of memory at least five bytes in size, stuff the ASCII codes for 'h', 'e', 'l', 'l' and 'o' in that block (possibly with a null byte at the end?) and then assign x to the address of that block. Am I right? That's better than my first thought, which was that it would write either 0x6865 ('he', if int is 16 bits) or 0x68656c6c ('hell', if int is 32 bits) to x, and either discard the rest of the characters or just blindly write them over the top of whatever variable (if any) happens to follow x in memory. > Getting inundated with warnings would be a major code smell. "The low oil warning light in my car was always on, so I put some masking tape over it so it wasn't bothering me any more." It freaks me out something wicked when I run a major GUI application like Firefox from the command line. Have you seen how many warnings and failed assertions it generates? It is scary. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From auriocus at gmx.de Mon Aug 29 09:07:19 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 29 Aug 2016 15:07:19 +0200 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 29.08.16 um 14:46 schrieb Steve D'Aprano: >> On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >>> In C, you can write this: >>> >>> int x; >>> >>> x = 5; >>> x = "hello"; > Let me see if I've got this straight... Bart's second assignment will > allocate a block of memory at least five bytes in size, stuff the ASCII > codes for 'h', 'e', 'l', 'l' and 'o' in that block (possibly with a null > byte at the end?) and then assign x to the address of that block. > > Am I right? Almost - the block of memory is statically allocated, i.e. there is a block of memory reserved for "constants" that are loaded on start-up of the program. x is assigned a pointer to that string. The difference is that you are not allowed to write to this memory area, for some compilers it causes a segfault if you try to write there. YMMV - I consider this a gross abuse of "dark corners" in C. Christian From rosuav at gmail.com Mon Aug 29 09:13:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Aug 2016 23:13:32 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Mon, Aug 29, 2016 at 10:46 PM, Steve D'Aprano wrote: > On Mon, 29 Aug 2016 10:31 pm, Chris Angelico wrote: > >> On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >>> In C, you can write this: >>> >>> int x; >>> >>> x = 5; >>> x = "hello"; >>> >>> With certain compilers (eg. gcc) you only get a warning. (And since I >>> don't show warnings to avoid inundation, that seems to compile fine for >>> me!) >> >> That's because strings, in C, are really pointers-to-char, and for >> hysterical raisins, pointers can be assigned to integers with just a >> warning. (Good code should have an explicit cast here.) > > Let me see if I've got this straight... Bart's second assignment will > allocate a block of memory at least five bytes in size, stuff the ASCII > codes for 'h', 'e', 'l', 'l' and 'o' in that block (possibly with a null > byte at the end?) and then assign x to the address of that block. > > Am I right? Mostly. Six for the \0 at the end, and the actual allocation happens at compile time; this is a literal, so it means "have a thing somewhere in memory that has h, e, l, l, o, \0, and I am its address". So the actual run-time effect of this second assignment is simply to set x to that address - which is effectively equivalent to "x = 5;" with some other value. That's why it's permitted (with a warning). In situations where you really WANT that, it's simple enough to demand it: x = (int)"hello"; but, as mentioned, backward compatibility means that this is permitted without the explicit cast. Personally, I'd call it an error, and all my C/C++ programs are designed to be compiled with -Wall, although I don't necessarily demand utter silence from the compiler. Every warning should be understood. > That's better than my first thought, which was that it would write either > > 0x6865 ('he', if int is 16 bits) > > or > > 0x68656c6c ('hell', if int is 32 bits) > > > to x, and either discard the rest of the characters or just blindly write > them over the top of whatever variable (if any) happens to follow x in > memory. Right, that's what would happen if you use single quotes (although that's an extension). Interestingly, even though I'm on a 64-bit system, gcc seems to work with 32-bit character constants at most; it's happy to set x = 'helo', but x = 'hello' produces a "character constant too long" warning and just uses 'ello'. (Actually, ANY multi-character constant produces a warning, but that's because I have it set to warn about all use of non-standard features. Not everyone will.) >> Getting inundated with warnings would be a major code smell. > > "The low oil warning light in my car was always on, so I put some masking > tape over it so it wasn't bothering me any more." > > > It freaks me out something wicked when I run a major GUI application like > Firefox from the command line. Have you seen how many warnings and failed > assertions it generates? It is scary. > Agreed, although not all of them are the fault of Firefox itself. I've seen some GUI libraries that themselves produce warnings. Not good, IMO. ChrisA From ben.usenet at bsb.me.uk Mon Aug 29 09:31:04 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 29 Aug 2016 14:31:04 +0100 Subject: What's the best way to minimize the need of run time checks? References: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> <57c42ec1$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87lgzfya5j.fsf@bsb.me.uk> Steve D'Aprano writes: > On Mon, 29 Aug 2016 10:31 pm, Chris Angelico wrote: > >> On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >>> In C, you can write this: >>> >>> int x; >>> >>> x = 5; >>> x = "hello"; >>> >>> With certain compilers (eg. gcc) you only get a warning. (And since I >>> don't show warnings to avoid inundation, that seems to compile fine for >>> me!) >> >> That's because strings, in C, are really pointers-to-char, They are really arrays of char, but array values convert to pointers to the first element in so many situations that people end up thinking that they are pointers. The trouble is, once you start thinking they really are pointers-to-char you have a harder time understanding things like &"hello" and sizeof "hello". >> and for >> hysterical raisins, pointers can be assigned to integers with just a >> warning. (Good code should have an explicit cast here.) The code is wrong without a cast. Many compilers, in their default setting, are very generous about this situation, but the code is as wrong a C code can be. The assignment violates a language constraint and the compiler would be permitted to refuse to generate any code at all. Given all that, it's something of an understatement to say that good code should use a cast (which is always explicit). (I'd go further and say that good code should try to avoid doing this at all, and when it absolutely must, it should use intptr_t rather than int.) > Let me see if I've got this straight... Bart's second assignment will > allocate a block of memory at least five bytes in size, stuff the ASCII > codes for 'h', 'e', 'l', 'l' and 'o' in that block (possibly with a null > byte at the end?) and then assign x to the address of that block. Close, but the array of char is not generated by the assignment. String literals give rise to arrays with static storage duration (in effect they exist before the start of the execution). It will be 6 bytes in size. The assignment might do nothing at all because it violates a language constraint about permitted types in an assignment. But if it does do something, a good bet will be to do the usual array to pointer conversion and then try to convert that pointer to an int for assignment. > Am I right? > > That's better than my first thought, which was that it would write either > > 0x6865 ('he', if int is 16 bits) > > or > > 0x68656c6c ('hell', if int is 32 bits) > > to x, and either discard the rest of the characters or just blindly write > them over the top of whatever variable (if any) happens to follow x in > memory. Well, your first thought is not actually wrong. In fact, a compiler would be allowed to do that since the behaviour of a program that violates a constraint is undefined by the language standard. It would not sell or get used much because people expect the traditional lax treatment of pointers as integers, but it could do that. >> Getting inundated with warnings would be a major code smell. > > "The low oil warning light in my car was always on, so I put some masking > tape over it so it wasn't bothering me any more." Yup. Gcc lets you put masking tape on (-Wno-int-conversion) but, in fairness, it also lets you treat all or selected warnings as hard errors (-Werror=int-conversion). Unfortunately there is no single option that treats all serious situations as hard errors, presumably because no two people agree on what should be considered serious for a particular architecture. -- Ben. From mviljamaa at kapsi.fi Mon Aug 29 09:59:47 2016 From: mviljamaa at kapsi.fi (Matti Viljamaa) Date: Mon, 29 Aug 2016 16:59:47 +0300 Subject: freqz -style plot of "window design" FIR? In-Reply-To: <2617B74F-F69F-49AF-A876-FFE016151DFA@kapsi.fi> References: <2617B74F-F69F-49AF-A876-FFE016151DFA@kapsi.fi> Message-ID: <80085821-C7DA-47DE-9AC3-D57C26564CD6@kapsi.fi> Since I noticed this is better suited to the SciPy Users List, I moved it there: https://mail.scipy.org/pipermail/scipy-user/2016-August/037023.html > On 29 Aug 2016, at 14:34, Matti Viljamaa wrote: > > I?m trying to design an arbitrary frequency response filter as described here: > http://www.dspguide.com/ch17/1.htm > > The technique is said to result in an impulse response in time domain and later in to a filter kernel. > > I?ve been using scipy.signal.freqz to make magnitude response plots: > > e.g. > > fs = 44100 > > # Design a low-pass filter using remez. > cutoff = 2000.0 > transition_width = 200 > bands = np.array([0, cutoff - 0.5*transition_width, > cutoff + 0.5*transition_width, fs/2.0]) / fs > desired = [1, 0] > lpf = remez(513, bands, desired) > > # Plot the frequency response of the filter. > w, h = freqz(lpf) > plt.figure(1) > plt.plot(fs*w/(2*np.pi), 20*np.log10(abs(h))) > plt.xlim(0, fs/2) > plt.xlabel('Frequency (Hz)') > plt.ylabel('Gain (dB)') > plt.grid(True) > > But my question is, if using the above arbitrary frequency response design technique, would I be able to use freqz? > > freqz takes as a parameter ?numerator of a linear filter? and remez is returning an array of coefficients, which I read to be the same thing. > > But in the case of the arbitrary frequency response filter, what can I put into freqz? Is filter kernel perhaps the same as coefficients? > -- > https://mail.python.org/mailman/listinfo/python-list From grant.b.edwards at gmail.com Mon Aug 29 10:00:30 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 29 Aug 2016 14:00:30 +0000 (UTC) Subject: What's the best way to minimize the need of run time checks? References: <57aa834a$0$1599$c3e8da3$5496439d@news.astraweb.com> <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-08-29, Chris Angelico wrote: > On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >> In C, you can write this: >> >> int x; >> >> x = 5; >> x = "hello"; >> >> With certain compilers (eg. gcc) you only get a warning. (And since >> I don't show warnings to avoid inundation, that seems to compile >> fine for me!) If you're ignoring compiler warnings, you're asking for problems and deserve whatever trouble you get. > That's because strings, in C, are really pointers-to-char, and for > hysterical raisins, pointers can be assigned to integers with just a > warning. (Good code should have an explicit cast here.) > > You probably should have warnings enabled. IMO, when doing doing development with GCC, -Wall -Werror is the absolute minimum standard. -- Grant Edwards grant.b.edwards Yow! Will it improve my at CASH FLOW? gmail.com From bc at freeuk.com Mon Aug 29 10:13:02 2016 From: bc at freeuk.com (BartC) Date: Mon, 29 Aug 2016 15:13:02 +0100 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: <57b0804b$0$22140$c3e8da3$5496439d@news.astraweb.com> <57c268ee$0$1591$c3e8da3$5496439d@news.astraweb.com> <57c2a1e6$0$2921$c3e8da3$76491128@news.astraweb.com> <57c3a156$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/08/2016 15:00, Grant Edwards wrote: > On 2016-08-29, Chris Angelico wrote: >> On Mon, Aug 29, 2016 at 10:13 PM, BartC wrote: >>> In C, you can write this: >>> >>> int x; >>> >>> x = 5; >>> x = "hello"; >>> >>> With certain compilers (eg. gcc) you only get a warning. (And since >>> I don't show warnings to avoid inundation, that seems to compile >>> fine for me!) > > If you're ignoring compiler warnings, you're asking for problems and > deserve whatever trouble you get. (It's my IDE that's not displaying the warnings because it can use any C compiler and some of them really will swamp the output with harmless warnings that often cannot be turned off (about unused labels for example). The situation is a little unusual as normally the C being compiled has been machine-generated so the actual source has already been mostly verified. (And in that source language, assigning "hello" directly to x is an error as a cast is mandatory.) >> That's because strings, in C, are really pointers-to-char, and for >> hysterical raisins, pointers can be assigned to integers with just a >> warning. (Good code should have an explicit cast here.) >> >> You probably should have warnings enabled. (Possibly they should be displayed when compiling 'straight' C. But then I wouldn't be able to 'assign a string to an integer' without any apparent problem!) -- Bartc From random832 at fastmail.com Mon Aug 29 10:38:26 2016 From: random832 at fastmail.com (Random832) Date: Mon, 29 Aug 2016 10:38:26 -0400 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> Message-ID: <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> Directing this to python-list because it's really not on the topic of the idea being discussed. On Mon, Aug 29, 2016, at 05:37, Chris Angelico wrote: > Suppose I come to python-ideas and say "Hey, the MUD community would > really benefit from a magic decoder that would use UTF-8 where > possible, ISO-8859-1 as fall-back, and Windows-1252 for characters not > in 8859-1". Apart from responding that 8859-1 is a complete subset of > 1252, ISO-8859-1, with a dash in between "ISO" and "8859" is not a complete subset of 1252. In fact, ISO-8859-1-with-a-dash incorporates ISO 6429 for 0x80-0x9F, and thereby has no bytes that do not map to characters. The magic encoding that people often ask for or use is to use UTF-8 first, Windows-1252 as a fallback, and ISO 6429 as the final fallback (and may or may not involve a "side trip" through Windows-1252 for UTF-8 encodings purportedly of code points between U+0080 and U+009F). Incidentally, many Windows encodings, including 1252, as they are actually used do use ISO 6429 for bytes that do not map to characters, even when best fit mappings are not accepted. It is unclear why they published tables that define these bytes as undefined, which have been picked up by independent implementations of these encodings such as the ones in Python. The only reason I can think of is to reserve the ability to add new mappings later, as they did for 0x80 to U+20AC. > there's not really a lot that you could discuss about that > proposal, unless I were to show you some of my code. I can tell you > about the number of MUDs that I play, the number of MUD clients that > I've written, and some stats from my MUD server, and say "The MUD > community needs this support", but it's of little value compared to > actual code. > > (For the record, a two-step decode of "UTF-8, fall back on 1252" is > exactly what I do... in half a dozen lines of code. So this does NOT > need to be implemented.) And what level is the fallback done at? Per line? Per character? Per read result? Does encountering an invalid-for-UTF-8 byte put it permanently in Windows-1252 mode? Does it "retroactively" affect earlier bytes? Can it be used as a stream encoding, or does it require you to use bytes-based I/O and a separate .decode step? I assume a MUD server isn't blocking on each client socket waiting for a newline character, so how does such a decoding step mesh with whatever such a server does to handle I/O asynchronously? Are there any frameworks that you could be using that you can't if it's not an encoding? What happens if it's being used as an incremental decoder, encounters a valid UTF-8 lead byte on a buffer boundary, and then must "reject" (i.e. decode as the fallback encoding) it afterwards because an invalid trail byte follows it in the next buffer? What happens if a buffer consists only of a valid partial UTF-8 character? I can probably implement the fallback as an error handler in half a dozen lines, but it's not obvious and I suspect it's not what a lot of people do. It would probably take a bit more than half a dozen lines to implement it as an encoding. From rosuav at gmail.com Mon Aug 29 11:14:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 01:14:00 +1000 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> Message-ID: On Tue, Aug 30, 2016 at 12:38 AM, Random832 wrote: > Directing this to python-list because it's really not on the topic of > the idea being discussed. > > On Mon, Aug 29, 2016, at 05:37, Chris Angelico wrote: >> Suppose I come to python-ideas and say "Hey, the MUD community would >> really benefit from a magic decoder that would use UTF-8 where >> possible, ISO-8859-1 as fall-back, and Windows-1252 for characters not >> in 8859-1". Apart from responding that 8859-1 is a complete subset of >> 1252, > > ISO-8859-1, with a dash in between "ISO" and "8859" is not a complete > subset of 1252. In fact, ISO-8859-1-with-a-dash incorporates ISO 6429 > for 0x80-0x9F, and thereby has no bytes that do not map to characters. > Incidentally, many Windows encodings, including 1252, as they are > actually used do use ISO 6429 for bytes that do not map to characters, > even when best fit mappings are not accepted. It is unclear why they > published tables that define these bytes as undefined, which have been > picked up by independent implementations of these encodings such as the > ones in Python. The only reason I can think of is to reserve the ability > to add new mappings later, as they did for 0x80 to U+20AC. Huh, okay. Anyway, point is that it's a magical decoder that tries UTF-8, and if that fails, uses an eight-bit encoding. >> there's not really a lot that you could discuss about that >> proposal, unless I were to show you some of my code. I can tell you >> about the number of MUDs that I play, the number of MUD clients that >> I've written, and some stats from my MUD server, and say "The MUD >> community needs this support", but it's of little value compared to >> actual code. >> >> (For the record, a two-step decode of "UTF-8, fall back on 1252" is >> exactly what I do... in half a dozen lines of code. So this does NOT >> need to be implemented.) > > And what level is the fallback done at? Per line? Per character? Per > read result? Does encountering an invalid-for-UTF-8 byte put it > permanently in Windows-1252 mode? Does it "retroactively" affect earlier > bytes? Can it be used as a stream encoding, or does it require you to > use bytes-based I/O and a separate .decode step? Currently? UTF-8 is attempted on an entire read result, and if it fails, the data is cracked into individual lines and retried, using the fallback as per the above. So in effect, it's per line. I basically assume that a naive byte-oriented server is usually going to be spitting out data from one client at a time, and each client is either emitting UTF-8 or its native encoding. (Since I have no way of knowing what native encoding a given client was using, I just pick Western Europe as the most likely codepage and run with it. The algorithm would work just the same if I picked, say, Windows-1250 as the eight-bit encoding.) > I assume a MUD server isn't blocking on each client socket waiting for a > newline character, so how does such a decoding step mesh with whatever > such a server does to handle I/O asynchronously? Are there any > frameworks that you could be using that you can't if it's not an > encoding? This magic started out in my MUD client, where it's connecting to a naive server that echoes whatever it's given. The same logic is now in my MUD server, too. It's pretty simple in both cases; the client is built around asynchronous I/O, the server is threaded, but both of them have a single point in the code where new bytes come in. There's one function that converts bytes to text, and it operates on the above algorithm. > What happens if it's being used as an incremental decoder, encounters a > valid UTF-8 lead byte on a buffer boundary, and then must "reject" (i.e. > decode as the fallback encoding) it afterwards because an invalid trail > byte follows it in the next buffer? What happens if a buffer consists > only of a valid partial UTF-8 character? Hmm, I don't remember if there's any actual handling of this. If there's a problem, my solution is simple: split on 0x0A first, and then decode, which means I'm decoding one line at a time. Both server and client already are fundamentally line-based anyway, and depending on byte value 0x0A always and only representing U+000A is valid in all of the encodings that I'm willing to accept. > I can probably implement the fallback as an error handler in half a > dozen lines, but it's not obvious and I suspect it's not what a lot of > people do. It would probably take a bit more than half a dozen lines to > implement it as an encoding. Please don't. :) This is something that belongs in the application; it's somewhat hacky, and I don't see any benefit to it going into the language. For one thing, I could well imagine making the fallback encoding configurable (it isn't currently, but it could easily be), and that doesn't really fit into the Python notion of error handler. For another, this is a fairly rare concept - I don't see dozens of programs out there using the exact same strange logic, and even if there were, there'd be small differences (eg whether or not the fallback is applied line-by-line). This was intended as an example of something that does NOT belong in the core language, and while I appreciate the offer of help, it's not something I'd support polluting the language with :) (Plus, my server's not written in Python. Nor is the client that this started in, although I have considered writing a version of it in Python, which would in theory benefit from this.) ChrisA From random832 at fastmail.com Mon Aug 29 11:28:18 2016 From: random832 at fastmail.com (Random832) Date: Mon, 29 Aug 2016 11:28:18 -0400 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> Message-ID: <1472484498.3606734.709275897.7029FB27@webmail.messagingengine.com> On Mon, Aug 29, 2016, at 11:14, Chris Angelico wrote: > Please don't. :) This is something that belongs in the application; > it's somewhat hacky, and I don't see any benefit to it going into the > language. For one thing, I could well imagine making the fallback > encoding configurable (it isn't currently, but it could easily be), > and that doesn't really fit into the Python notion of error handler. Well, yeah, if anything implementing it as an error handler is a hack, I just meant it's just the least hacky way I can think that fits in the size "half a dozen lines". > For another, this is a fairly rare concept - I don't see dozens of > programs out there using the exact same strange logic, and even if > there were, there'd be small differences That is actually an argument in favor of putting it in the stdlib, assuming few of those small differences are truly considered and intentional. The main thrust of my post was that this is one of the things that's harder than it sounds to get right due to edge cases, just like the clip/clamp function being discussed last month. > (eg whether or not the > fallback is applied line-by-line). This was intended as an example of > something that does NOT belong in the core language, and while I > appreciate the offer of help, it's not something I'd support polluting > the language with :) > > (Plus, my server's not written in Python. Nor is the client that this > started in, although I have considered writing a version of it in > Python, which would in theory benefit from this.) From rosuav at gmail.com Mon Aug 29 11:59:09 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 01:59:09 +1000 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: <1472484498.3606734.709275897.7029FB27@webmail.messagingengine.com> References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> <1472484498.3606734.709275897.7029FB27@webmail.messagingengine.com> Message-ID: On Tue, Aug 30, 2016 at 1:28 AM, Random832 wrote: > On Mon, Aug 29, 2016, at 11:14, Chris Angelico wrote: >> Please don't. :) This is something that belongs in the application; >> it's somewhat hacky, and I don't see any benefit to it going into the >> language. For one thing, I could well imagine making the fallback >> encoding configurable (it isn't currently, but it could easily be), >> and that doesn't really fit into the Python notion of error handler. > > Well, yeah, if anything implementing it as an error handler is a hack, I > just meant it's just the least hacky way I can think that fits in the > size "half a dozen lines". > >> For another, this is a fairly rare concept - I don't see dozens of >> programs out there using the exact same strange logic, and even if >> there were, there'd be small differences > > That is actually an argument in favor of putting it in the stdlib, > assuming few of those small differences are truly considered and > intentional. The main thrust of my post was that this is one of the > things that's harder than it sounds to get right due to edge cases, just > like the clip/clamp function being discussed last month. Fair enough. If this were something that a lot of programs wanted, then yeah, there'd be good value in stdlibbing it. Character encodings ARE hard to get right, and this kind of thing does warrant some help. But I think it's best not done in core - at least, not until we see a lot more people doing the same :) ChrisA From chivalry at mac.com Mon Aug 29 12:17:12 2016 From: chivalry at mac.com (Charles Ross) Date: Mon, 29 Aug 2016 09:17:12 -0700 Subject: How to get Read the Docs to generate py-modindex.html? Message-ID: <34018AE3-8C4E-4497-BB51-96E7D4991EAD@mac.com> Since I appear to be experiencing one of the few times that Stack Overflow hasn?t provided any quick answers to a problem (http://stackoverflow.com/questions/39197037/how-to-get-read-the-docs-to-generate-py-modindex-html ), I?ll try posting my question here and see if anyone has a suggestion. I'm trying to get Read the Docs to generate the `py-modindex.html` file. Research into a related question lead me to the following setup: - `setup.py` in the project directory has the following contents, which were the minimum needed to get `pytest` to work and haven't been changed since I got that part of my project working: import setuptools setuptools.setup( name='polygons', packages=setuptools.find_packages(), ) - `docs/requirements.txt` contains a single line: sphinx-autodoc-annotation - The Read the Docs repository URL points to my repository (https://github.com/chivalry/polygons). - The RtD setting for "Install your project inside a virtualenv using setup.py install" is checked. - The RtD setting for "Requirements file" points to `docs/requirements.txt`. The "Module Index" link gets included in `index.html` (http://polygons.readthedocs.io/en/latest/), but the `py-modindex.html` file is missing. My understanding is that with the virtualenv setting above, RtD will use the `setup.py` file to install the project so that Sphinx can read the documentation found in the Python docstrings. I'm using function annotations and would like `sphyinx-autodoc-annotation` to make use of those when creating the built docs files. All of this works splendidly on my local machine when I run `make html` while in the `docs` folder. Now I'm trying to get it to work on Read the Docs. Thanks, Chuck From rgaddi at highlandtechnology.invalid Mon Aug 29 13:19:54 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 29 Aug 2016 17:19:54 -0000 (UTC) Subject: Multimeter USB output References: Message-ID: Joe wrote: > The UT61B has two interfaces, a RS232C interface and this usb interface. > The RS232 interface works well with PySerial. It continously transmits 2 > .. 3 Pakets per second with 14 Bytes each. This happens unsolicited > without any command as long as the RS232C/USB button on the DMM is active. > >From someone who does a LOT of code to talk to DMMs, scopes, counters, etc? Word of advice. You're done. Declare victory and move on. You've got it working on one interface. The other is even less well documented and requires doing deep-down stuff against libusb because there's no actual kernel driver, and you've got a Chinese manufacturer of a low-end device being of little to no help so now you're trying to reverse engineer the protocol from bad code without even the help of a packet sniffer. This is the sort of rabbit hole that weeks manage to fall into with nothing to show for them. If this is an exercise in learning to work with libusb then have at. But if what you actually need from this is to talk to the meter? You're talking to it already. Take the win and walk. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ganesh1pal at gmail.com Mon Aug 29 13:30:35 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Mon, 29 Aug 2016 23:00:35 +0530 Subject: Helloworld with Python C extension Message-ID: Hello Team , I need you input on the below hello world program. I a m trying to add a python binding which will return the character for the given index . I am on Python 2.7 and linux Example : >> string ='helloworld' >>dda_hello(5) >> 'w' /* + * Hello world example for python bindings + */ + +static char* string = "helloworld"; +char dda_hello(int i) + { + return string[i]; + } + +static PyObject * +py_dda_hello(PyObject *self, PyObject *args ) +{ + int index; + char char1; + if (!PyArg_ParseTuple(args, "i", &index)) + return NULL; + char1 = dda_hello(index); + return Py_BuildValue("s",char1); +} + +/* @@ -1674,6 +1705,10 @@ PyMethodDef xyz_methods[] = { + {"dda_hello", py_dda_hello, METH_VARARGS, + "Returns the character entered for a given index"}, >>> import as.ds.dss as daa >>> print dda.dda_hello(1) zsh: segmentation fault (core dumped) python Apologies for posting the diff , I didn't find a better way Regards, Ganesh From rosuav at gmail.com Mon Aug 29 13:36:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 03:36:10 +1000 Subject: Helloworld with Python C extension In-Reply-To: References: Message-ID: On Tue, Aug 30, 2016 at 3:30 AM, Ganesh Pal wrote: > +char dda_hello(int i) > + { > + return string[i]; > + } > + > + return Py_BuildValue("s",char1); Py_BuildValue with an "s" expects a C string - that is, a pointer to char, not just a single character. You'd need to do something like this: char buf[2] = {char1, 0}; return Py_BuildValue("s", buf); ChrisA From toby at tobiah.org Mon Aug 29 13:57:22 2016 From: toby at tobiah.org (Tobiah) Date: Mon, 29 Aug 2016 10:57:22 -0700 Subject: importing down in code rather than at top of file. Message-ID: Is it worth while to defer the import of a large module that seldom gets used in the script? import sys import os if hardly_ever_happens(): import large_module large_module.do_task() I imagine it takes a certain amount of processing power and memory to import a module, so it seems like I'd save those resources with the above pattern. The down side would be that it's nice to see all of the imports at the top which would follow convention. Should I care? Tobiah From python at mrabarnett.plus.com Mon Aug 29 13:58:27 2016 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Aug 2016 18:58:27 +0100 Subject: Helloworld with Python C extension In-Reply-To: References: Message-ID: <0592c293-40d1-71fc-17b1-7eb04a9017b5@mrabarnett.plus.com> On 2016-08-29 18:30, Ganesh Pal wrote: > Hello Team , > > I need you input on the below hello world program. I a m trying to add a > python binding which will return the character for the given index . I am > on Python 2.7 and linux > > Example : >>> string ='helloworld' >>>dda_hello(5) >>> 'w' > > > /* > + * Hello world example for python bindings > + */ > + > +static char* string = "helloworld"; > +char dda_hello(int i) > + { > + return string[i]; > + } > + > +static PyObject * > +py_dda_hello(PyObject *self, PyObject *args ) > +{ > + int index; > + char char1; > + if (!PyArg_ParseTuple(args, "i", &index)) > + return NULL; > + char1 = dda_hello(index); This bit is wrong: > + return Py_BuildValue("s",char1); The format "s" is for a null-terminated C string (char *), but you're giving it a C int. The format you should be using is "c" (C int representing a character). > +} > + > +/* > > @@ -1674,6 +1705,10 @@ PyMethodDef xyz_methods[] = { > + {"dda_hello", py_dda_hello, METH_VARARGS, > + "Returns the character entered for a given index"}, > > >>>> import as.ds.dss as daa >>>> print dda.dda_hello(1) > zsh: segmentation fault (core dumped) python > > Apologies for posting the diff , I didn't find a better way > It would be a good idea to check the index and raise an exception if it's out the bounds. From rosuav at gmail.com Mon Aug 29 14:15:05 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 04:15:05 +1000 Subject: importing down in code rather than at top of file. In-Reply-To: References: Message-ID: On Tue, Aug 30, 2016 at 3:57 AM, Tobiah wrote: > Is it worth while to defer the import of a large module that seldom > gets used in the script? > > > import sys > import os > > if hardly_ever_happens(): > > import large_module > large_module.do_task() > > > > I imagine it takes a certain amount of processing > power and memory to import a module, so it seems > like I'd save those resources with the above pattern. > > The down side would be that it's nice to see all of the > imports at the top which would follow convention. Should > I care? Don't imagine; test. Time your program with and without a top-of-file import. If the module really is large (and by "large" I mean on the scale of numpy - huge stuff), then yes, it's worth it. If you can't measure the difference, though, don't bother - just import it and have done with it. ChrisA From alister.ware at ntlworld.com Mon Aug 29 14:18:30 2016 From: alister.ware at ntlworld.com (alister) Date: Mon, 29 Aug 2016 18:18:30 GMT Subject: importing down in code rather than at top of file. References: Message-ID: On Mon, 29 Aug 2016 10:57:22 -0700, Tobiah wrote: > Is it worth while to defer the import of a large module that seldom > gets used in the script? > > > import sys import os > > if hardly_ever_happens(): > > import large_module large_module.do_task() > > > > I imagine it takes a certain amount of processing power and memory to > import a module, so it seems like I'd save those resources with the > above pattern. > > The down side would be that it's nice to see all of the imports at the > top which would follow convention. Should I care? > > > Tobiah That depends does it actually make a significant delay to the loading of your application (have you timed it yet) if the module does take a noticeable time to load do you want that additional delay in your rarely access conditions? IMO start-up times for an application have to be significantly long before they become an issue unless they are a shout quick utility example Libra office an take a few seconds to start but then will be running for a long time - the start-up time is not really significant. a utility to perform a bulk rename you would not want to take 30 seconds to start if it would then complete in less than 1 remember the golden riules for optimisation 1) don't 2) (for advanced programmers only) Don't yet. -- In a world without fences who needs Gates? From stefan_ml at behnel.de Mon Aug 29 17:34:22 2016 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 29 Aug 2016 23:34:22 +0200 Subject: Helloworld with Python C extension In-Reply-To: References: Message-ID: Ganesh Pal schrieb am 29.08.2016 um 19:30: > I need you input on the below hello world program. I a m trying to add a > python binding which will return the character for the given index . I am > on Python 2.7 and linux > > Example : > >>> string ='helloworld' > >>> dda_hello(5) > >>> 'w' > > /* > + * Hello world example for python bindings > + */ > + > +static char* string = "helloworld"; > +char dda_hello(int i) > + { > + return string[i]; > + } > + > +static PyObject * > +py_dda_hello(PyObject *self, PyObject *args ) > +{ > + int index; > + char char1; > + if (!PyArg_ParseTuple(args, "i", &index)) > + return NULL; > + char1 = dda_hello(index); > + return Py_BuildValue("s",char1); > +} > + > +/* > > @@ -1674,6 +1705,10 @@ PyMethodDef xyz_methods[] = { > + {"dda_hello", py_dda_hello, METH_VARARGS, > + "Returns the character entered for a given index"}, Here's a Cython implementation (http://cython.org) of your example: cdef str string = "helloworld" def dda_hello(int i): return string[i] It uses a lot less code than the C-implemented version, but is compatible with Python 2 and Python 3 and avoids pitfalls like the crash you are seeing, as well as raising a proper IndexError for invalid index arguments (and it supports negative indexing). I also wouldn't be surprised if it's visibly faster than your C implementation. Unless your intention is to explicitly learn how to use the CPython C-API, you should give Cython a try instead. Stefan From nobody at nowhere.invalid Mon Aug 29 18:05:51 2016 From: nobody at nowhere.invalid (Nobody) Date: Mon, 29 Aug 2016 23:05:51 +0100 Subject: importing down in code rather than at top of file. References: Message-ID: On Tue, 30 Aug 2016 04:15:05 +1000, Chris Angelico wrote: > Don't imagine; test. Testing alone isn't really good enough. There may be perfectly valid reasons to avoid the import which won't show up in anything less than the most thorough testing imaginable. Personally I wouldn't defer an import just because it might be slow, particularly if the criterion affecting whether the import actually occurs is "probabilistic". [If it might be slow because the module is doing stuff it really shouldn't, like executing external commands during import, I'd probably just avoid the package altogether.] If the import /might/ happen as a result of normal usage, I'd want any failure to occur when the parent module is imported, not at some rather arbitrary later point. If I defer imports, it's because they're only used for features which are in some sense optional. And the documentation will state the conditions under which the import is performed (if it's for a specific method, it will be in the method's docstring). From tjreedy at udel.edu Mon Aug 29 20:40:21 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 29 Aug 2016 20:40:21 -0400 Subject: importing down in code rather than at top of file. In-Reply-To: References: Message-ID: On 8/29/2016 1:57 PM, Tobiah wrote: > Is it worth while to defer the import of a large module that seldom > gets used in the script? > > > import sys > import os > > if hardly_ever_happens(): > > import large_module > large_module.do_task() > > > > I imagine it takes a certain amount of processing > power and memory to import a module, so it seems > like I'd save those resources with the above pattern. > > The down side would be that it's nice to see all of the > imports at the top which would follow convention. Should > I care? If you delay the import, you can put it a function, such as 'almost never', and then document the delayed import with comments either mixin in with or below the real ones, such as # import large_module # in almost_never -- Terry Jan Reedy From m at funkyhat.org Mon Aug 29 21:58:24 2016 From: m at funkyhat.org (Matt Wheeler) Date: Tue, 30 Aug 2016 01:58:24 +0000 Subject: PSA: Debian Stretch, apt-listchanges, and Python 3 In-Reply-To: References: Message-ID: I think the real PSA is "don't mess with the system python(3) version". On Mon, 29 Aug 2016 at 13:18 Chris Angelico wrote: > If, like me, you build Python 3.6 from source and make it your default > 'python3' binary, you may run into issues with the latest > apt-listchanges, which has switched to using Python 3. The symptom is > This is a good example of the reason why it's a bad idea to mess with the system python on any OS (and every so often people get berated on this list for doing just that, although usually on OSX, where it's actually less of an issue :). > an ImportError trying to "import debconf", which is available in > dist-packages in the system Python, but (naturally) isn't part of > Python 3.6's install tree. > > My solution: > > sudo ln -s /usr/lib/python3/dist-packages/debconf.py > /usr/local/lib/python3.6/site-packages/ > > And then apt-listchanges happily sets itself up. > So you've managed to fix this issue, but what if there are others? I think packages on Debian have a reasonable right to expect that the version of python3 in the release is the version they are going to run on if they look like `#!/usr/bin/env python3 ...` Why not simply alias python3 to python3.6 for your user, or make use of virtualenvs? From rosuav at gmail.com Mon Aug 29 23:56:17 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 13:56:17 +1000 Subject: PSA: Debian Stretch, apt-listchanges, and Python 3 In-Reply-To: References: Message-ID: On Tue, Aug 30, 2016 at 11:58 AM, Matt Wheeler wrote: > I think the real PSA is "don't mess with the system python(3) version". > > So you've managed to fix this issue, but what if there are others? I think > packages on Debian have a reasonable right to expect that the version of > python3 in the release is the version they are going to run on if they look > like `#!/usr/bin/env python3 ...` > > Why not simply alias python3 to python3.6 for your user, or make use of > virtualenvs? Up until very recently, the system Python has only been the Python 2. (For a while, it wasn't even 2.7, but I could easily use altinstall for that.) I have deliberately wanted to use 3.6 for ALL testing, not just by messing with venvs or shell aliases, in order to enforce dogfooding on everything (and for consistency); older Pythons are available as python3.4 and python3.5, but the live python3 has always been the one built from tip. And even as Debian starts to embrace Py3 internally, this hasn't been a problem. If your goal is stability, sure. Don't mess with what 'env python3' runs. But if you want to thoroughly test stuff? That's different. ChrisA From orgnut at yahoo.com Tue Aug 30 00:21:05 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Mon, 29 Aug 2016 21:21:05 -0700 Subject: Multimeter USB output In-Reply-To: References: Message-ID: On 08/29/2016 01:54 AM, Joe wrote: [snip...] > Interesting, but... The last time I did something with c, it was with BDS-C under CM/M. Somebody > remenbering this no-fp compiler from the dark age before PC und Linux? I remember it well. It's what I used to initially learn C. I'm a completely self-taught, hobby programmer. Been around since the MITS Altair. How many remember that beast?? (And yes -- as you already corrected yourself -- that's CP/M not CM/M.) -- -=- Larry -=- From no.email at nospam.invalid Tue Aug 30 00:24:35 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 29 Aug 2016 21:24:35 -0700 Subject: Multimeter USB output References: Message-ID: <877fayq3y4.fsf@jester.gateway.pace.com> Larry Hudson writes: >> with BDS-C under CP/M. Somebody remenbering this no-fp compiler from >> the dark age before PC und Linux? > I remember it well. It's what I used to initially learn C. Source code is online here: http://www.bdsoft.com/resources/bdsc.html I've looked at it a little. I don't know if I ever might have had it in me to write big chunks of asm code like that. Wow! From matt.ruffalo at gmail.com Tue Aug 30 01:48:41 2016 From: matt.ruffalo at gmail.com (Matt Ruffalo) Date: Tue, 30 Aug 2016 01:48:41 -0400 Subject: PSA: Debian Stretch, apt-listchanges, and Python 3 In-Reply-To: References: Message-ID: <45537340-6284-58ac-c432-ddfd04d2d071@gmail.com> On 2016-08-29 23:56, Chris Angelico wrote: > Up until very recently, the system Python has only been the Python 2. > (For a while, it wasn't even 2.7, but I could easily use altinstall > for that.) I have deliberately wanted to use 3.6 for ALL testing, not > just by messing with venvs or shell aliases, in order to enforce > dogfooding on everything (and for consistency); older Pythons are > available as python3.4 and python3.5, but the live python3 has always > been the one built from tip. And even as Debian starts to embrace Py3 > internally, this hasn't been a problem. > > If your goal is stability, sure. Don't mess with what 'env python3' > runs. But if you want to thoroughly test stuff? That's different. > > ChrisA This seems like a reasonably large problem on Debian's part. I was under the impression that system-critical Python scripts used absolute paths in their hashbang lines, to avoid exactly this situation. If something really depends on the system Python version (whichever 2.x or 3.x that might be), it should explicitly use that Python installation. User scripts, of course, should use "/usr/bin/env python3" to be usable in a virtualenv or under whatever Python installation the user chooses. I believe the Gentoo package manager has logic to adjust the hashbang lines of Python scripts if a package requests it, and I'm not as familiar with Debian packaging but I'd be surprised if the Debian tools were missing this functionality. MMR... From ganesh1pal at gmail.com Tue Aug 30 02:06:59 2016 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Tue, 30 Aug 2016 11:36:59 +0530 Subject: Helloworld with Python C extension In-Reply-To: References: Message-ID: > > > > Py_BuildValue with an "s" expects a C string - that is, a pointer to > char, not just a single character. You'd need to do something like > this: > > char buf[2] = {char1, 0}; > return Py_BuildValue("s", buf); > > ChrisA Thanks Chris for the clue's it worked, I was just wondering how could the C extension be debugged ? We have pdb at python side and gdb for C , can we run gdb on python side ? if there is a crash like the one we saw in the above diff we are clueless of what's happening? any idea or recommendation on how we handle such cases Regards, Ganesh From test at joepgen.com Tue Aug 30 03:01:15 2016 From: test at joepgen.com (Joe) Date: Tue, 30 Aug 2016 09:01:15 +0200 Subject: Multimeter USB output In-Reply-To: <877fayq3y4.fsf@jester.gateway.pace.com> References: <877fayq3y4.fsf@jester.gateway.pace.com> Message-ID: Am 30.08.2016 um 06:24 schrieb Paul Rubin: > Larry Hudson writes: >>> with BDS-C under CP/M. Somebody remenbering this no-fp compiler from >>> the dark age before PC und Linux? >> I remember it well. It's what I used to initially learn C. > > Source code is online here: > > http://www.bdsoft.com/resources/bdsc.html > > I've looked at it a little. I don't know if I ever might have had it in > me to write big chunks of asm code like that. Wow! > Great surprise. Very interesting this link. Thank you, Paul. From auriocus at gmx.de Tue Aug 30 03:12:26 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 30 Aug 2016 09:12:26 +0200 Subject: Helloworld with Python C extension In-Reply-To: References: Message-ID: Am 30.08.16 um 08:06 schrieb Ganesh Pal: >> >> >> >> Py_BuildValue with an "s" expects a C string - that is, a pointer to >> char, not just a single character. You'd need to do something like >> this: >> >> char buf[2] = {char1, 0}; >> return Py_BuildValue("s", buf); >> >> ChrisA > > > Thanks Chris for the clue's it worked, I was just wondering how > could the > C extension be debugged ? > > We have pdb at python side and gdb for C , can we run gdb on python side > ? if there is a crash like the one we saw in the above diff we are > clueless of what's happening? any idea or recommendation on how we handle > such cases 1. Write your Python code into a file (e.g. test.py) 2. Run gdb --args python test.py Then press "r" to start your program. At the crash, gdb should stop your program, maybe inside of Py_BuildValue. You must compile your extension with debug symbols (-g switch to the compiler) enabled to see line numbers. In case of memory errors like the above, they can sometimes go unnoticed for a while, which makes them hard to debug. If you are on Linux, valgrind is the most powerful tool to find these. Run valgrind python test.py It'll show out-of-bounds accesses for arrays immediately. Christian From dieter at handshake.de Tue Aug 30 03:13:55 2016 From: dieter at handshake.de (dieter) Date: Tue, 30 Aug 2016 09:13:55 +0200 Subject: importing down in code rather than at top of file. References: Message-ID: <87vayi685o.fsf@handshake.de> Tobiah writes: > Is it worth while to defer the import of a large module that seldom > gets used in the script? > > > import sys > import os > > if hardly_ever_happens(): > > import large_module > large_module.do_task() I have used delayed import for different reasons: * to avoid cyclical imports * to avoid import deadlocks in multi-tasking programs (Python 2 (at least) used to protect the import machinery with a lock; which under some conditions could lead to deadlocks in a multi-tasking program). Typically, the delayed import was then in a function - relying on the fact that importing an already imported module is fast (thus, we do not lose much even if the function is called multiple times). From dfnsonfsduifb at gmx.de Tue Aug 30 05:36:02 2016 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Tue, 30 Aug 2016 11:36:02 +0200 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> <1472484498.3606734.709275897.7029FB27@webmail.messagingengine.com> Message-ID: On 29.08.2016 17:59, Chris Angelico wrote: > Fair enough. If this were something that a lot of programs wanted, > then yeah, there'd be good value in stdlibbing it. Character encodings > ARE hard to get right, and this kind of thing does warrant some help. > But I think it's best not done in core - at least, not until we see a > lot more people doing the same :) I hope this kind of botchery never makes it in the stdlib. It directly contradicts "In the face of ambiguity, refuse the temptation to guess." If you don't know what the charset is, don't guess. It'll introduce subtle ambiguities and ugly corner cases and will make the life for the rest of us -- who are trying to get their charsets straight and correct -- a living hell. Having such silly "magic" guessing stuff is actually detrimental to the whole concept of properly identifying and using character sets. Everything about the thought makes me shiver. Cheers, Johannes -- >> Wo hattest Du das Beben nochmal GENAU vorhergesagt? > Zumindest nicht ?ffentlich! Ah, der neueste und bis heute genialste Streich unsere gro?en Kosmologen: Die Geheim-Vorhersage. - Karl Kaos ?ber R?diger Thomas in dsa From rosuav at gmail.com Tue Aug 30 06:03:13 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 20:03:13 +1000 Subject: Magic UTF-8/Windows-1252 encodings In-Reply-To: References: <20160829014404.GB29601@kundert.designers-guide.com> <20160829032945.GD29601@kundert.designers-guide.com> <20160829090817.GF19357@kundert.designers-guide.com> <1472481506.3596247.709174081.0CDAEDAD@webmail.messagingengine.com> <1472484498.3606734.709275897.7029FB27@webmail.messagingengine.com> Message-ID: On Tue, Aug 30, 2016 at 7:36 PM, Johannes Bauer wrote: > On 29.08.2016 17:59, Chris Angelico wrote: > >> Fair enough. If this were something that a lot of programs wanted, >> then yeah, there'd be good value in stdlibbing it. Character encodings >> ARE hard to get right, and this kind of thing does warrant some help. >> But I think it's best not done in core - at least, not until we see a >> lot more people doing the same :) > > I hope this kind of botchery never makes it in the stdlib. It directly > contradicts "In the face of ambiguity, refuse the temptation to guess." > > If you don't know what the charset is, don't guess. It'll introduce > subtle ambiguities and ugly corner cases and will make the life for the > rest of us -- who are trying to get their charsets straight and correct > -- a living hell. > > Having such silly "magic" guessing stuff is actually detrimental to the > whole concept of properly identifying and using character sets. > Everything about the thought makes me shiver. In the clinical purity of theoretical work, I absolutely agree with you, and for that reason, this definitely doesn't belong in the stdlib. But designers need to leave their wonderlands - the real world is not so wonderful. (Nan Sharpe, to Alice Liddell.) If every program in the world understood character encodings and correctly decoded bytes using a known encoding and encoded text using the same encoding (preferably UTF-8), then sure, it'd be easy. But when your program has to cope with other people's bytes-that-ought-to-represent-text, sometimes guessing IS better than choking. This example is a perfect one; a naive byte-oriented server accepts ASCII-compatible text from a variety of clients, and sends it out to all clients. (Since all the parts that the server actually parses are ASCII, this works.) Very commonly, naive Windows clients send text in the native encoding, eg CP-1252, but smarter clients generally send UTF-8. I want my client to interoperate perfectly with other UTF-8 clients, which is generally easy (the only breakage is if the server attempts to letter-wrap a massively long word, and ends up breaking a UTF-8 sequence across lines), but I also want to have a decent fallback for the eight-bit clients. Obviously I can't *know* the encoding used - if they were smart enough to send encoding info, they'd most likely use UTF-8 - so it's either guess, or choke on any non-ASCII bytes. Another place where guessing is VERY useful is when I'm leafing through 300 subtitles files for "Tangled" and want to know whether they're accurate transcriptions or not. (Not hypothetical. Been doing exactly that for a lot of this weekend. It seemed logical, since I've done the same for "Frozen", and both movies are excellent.) All I have is a file - a sequence of bytes. I know it's an ASCII-compatible encoding because the numeric positioning info looks correct. If my program "avoided the temptation to guess", I would have to manually test a dozen encodings until one of them looked right to me, the human; but instead, I use chardet plus some other heuristics, and generally the program's right on either the first or second guess. That means just two encodings for me to look at, often just one, and only going to the full dozen or so if it gets it completely wrong. The principle "refuse the temptation to guess" applies to core data types and such (and not even universally there), but NOT to applications, where you need domain knowledge to make that kind of call. ChrisA From smith at smith.com Tue Aug 30 06:24:50 2016 From: smith at smith.com (Smith) Date: Tue, 30 Aug 2016 12:24:50 +0200 Subject: Itertools Python3 Message-ID: Hi, I can not write to the file. Can someone help me? Thanks from itertools import product valore = input('Inserisci un valore: ') risultato = product(valore, repeat = 3) with open("file.txt", "w") as result: for i in risultato: print (result,"".join(i)) From winkidney at gmail.com Tue Aug 30 06:25:32 2016 From: winkidney at gmail.com (Kidney Win) Date: Tue, 30 Aug 2016 03:25:32 -0700 (PDT) Subject: [CmdTree] sub-command friendly cli library for python : ) Message-ID: <321eda35-f745-4d98-b159-a5ffad47c750@googlegroups.com> Hi there, I'm winkidney:), Recently when I work on a cli auto-generating task, I tryed "click" and "argparse" to handle it. But I have to write a library myself to do this job finally. I wish this library helps you ?? Project Github Repo: https://github.com/winkidney/cmdtree Why CmdTree? Alternatives: click library argparse library But when you should choose cmdtree? If you need: + fully sub-command support(not group in click) + Higher-level api support(compare to argparse) + More arg-type support(compare to argparse) + decorators has no side-effect on function call You may be a proper user of CmdTree Best wishes From rosuav at gmail.com Tue Aug 30 06:28:25 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 20:28:25 +1000 Subject: Itertools Python3 In-Reply-To: References: Message-ID: On Tue, Aug 30, 2016 at 8:24 PM, Smith wrote: > I can not write to the file. > Can someone help me? > Thanks > > from itertools import product > valore = input('Inserisci un valore: ') > risultato = product(valore, repeat = 3) > with open("file.txt", "w") as result: > for i in risultato: > print (result,"".join(i)) Do you get an exception, possibly from the product() call? When you ask for help, copy and paste the entire traceback and error message; it's extremely useful information. I'm pretty sure I know what the problem is here, but I want you to post the traceback, so that you learn how debugging of Python code works :) ChrisA From smith at smith.com Tue Aug 30 06:45:54 2016 From: smith at smith.com (Smith) Date: Tue, 30 Aug 2016 12:45:54 +0200 Subject: Itertools Python3 In-Reply-To: References: Message-ID: Il 30/08/2016 12:28, Chris Angelico ha scritto: > Do you get an exception, possibly from the product() call? When you > ask for help, copy and paste the entire traceback and error message; > it's extremely useful information. > > I'm pretty sure I know what the problem is here, but I want you to > post the traceback, so that you learn how debugging of Python code > works python3 toolgen.py Inserisci un valore: dog <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddg <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> dod <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> doo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> dog <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> dgd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> dgo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> dgg <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> odd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> odo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> odg <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ood <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ooo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> oog <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ogd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ogo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ogg <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> gdd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> gdo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> gdg <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> god <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> goo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> gog <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ggd <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ggo <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ggg From rosuav at gmail.com Tue Aug 30 07:01:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Aug 2016 21:01:22 +1000 Subject: Itertools Python3 In-Reply-To: References: Message-ID: On Tue, Aug 30, 2016 at 8:45 PM, Smith wrote: > Il 30/08/2016 12:28, Chris Angelico ha scritto: >> >> Do you get an exception, possibly from the product() call? When you >> ask for help, copy and paste the entire traceback and error message; >> it's extremely useful information. >> >> I'm pretty sure I know what the problem is here, but I want you to >> post the traceback, so that you learn how debugging of Python code >> works > > python3 toolgen.py > Inserisci un valore: dog > <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddd > <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddo > <_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'> ddg > ... So it's not giving you an exception, just outputting to the screen instead of the file. Okay. Next thing to look into: How does the print function actually get told about a file to output to? You can type help(print) at the interactive prompt, or look in the docs online: https://docs.python.org/3/library/functions.html#print Check how you're using that function, because it doesn't seem to be doing what you expect. ChrisA From darcy at Vex.Net Tue Aug 30 07:01:54 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Tue, 30 Aug 2016 07:01:54 -0400 Subject: Multimeter USB output In-Reply-To: References: Message-ID: <20160830070154.5cb89421@imp> On Mon, 29 Aug 2016 21:21:05 -0700 Larry Hudson via Python-list wrote: > I remember it well. It's what I used to initially learn C. I'm a > completely self-taught, hobby programmer. Been around since the MITS > Altair. How many remember that beast?? Remember it and still have it in the basement. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From smith at smith.com Tue Aug 30 08:38:59 2016 From: smith at smith.com (Smith) Date: Tue, 30 Aug 2016 14:38:59 +0200 Subject: Itertools Python3 In-Reply-To: References: Message-ID: Il 30/08/2016 12:28, Chris Angelico ha scritto: > On Tue, Aug 30, 2016 at 8:24 PM, Smith wrote: >> I can not write to the file. >> Can someone help me? >> Thanks >> >> from itertools import product >> valore = input('Inserisci un valore: ') >> risultato = product(valore, repeat = 3) >> with open("file.txt", "w") as result: >> for i in risultato: >> print (result,"".join(i)) > > Do you get an exception, possibly from the product() call? When you > ask for help, copy and paste the entire traceback and error message; > it's extremely useful information. > > I'm pretty sure I know what the problem is here, but I want you to > post the traceback, so that you learn how debugging of Python code > works :) > > ChrisA > The problem is solved I thank you for your time from itertools import product valore = input('Inserisci un valore: ') risultato = product(valore, repeat = 3) with open("file.txt", "w") as result: for i in risultato: print ("".join(i),file=result) From test at joepgen.com Tue Aug 30 09:56:07 2016 From: test at joepgen.com (Joe) Date: Tue, 30 Aug 2016 15:56:07 +0200 Subject: Multimeter USB output In-Reply-To: References: <20160830070154.5cb89421@imp> Message-ID: Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: > On Mon, 29 Aug 2016 21:21:05 -0700 > Larry Hudson via Python-list wrote: >> I remember it well. It's what I used to initially learn C. I'm a >> completely self-taught, hobby programmer. Been around since the MITS >> Altair. How many remember that beast?? > > Remember it and still have it in the basement. > I read a lot about the Altair in Byte in those days, but never had a chance to touch it. Wasn't it horrible expensive? From darcy at Vex.Net Tue Aug 30 11:52:15 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Tue, 30 Aug 2016 11:52:15 -0400 Subject: [OT] Altair In-Reply-To: References: <20160830070154.5cb89421@imp> Message-ID: <20160830115215.78a9ec7f@imp> On Tue, 30 Aug 2016 15:56:07 +0200 Joe wrote: > Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: > > On Mon, 29 Aug 2016 21:21:05 -0700 > > Larry Hudson via Python-list wrote: > >> I remember it well. It's what I used to initially learn C. I'm a > >> completely self-taught, hobby programmer. Been around since the > >> MITS Altair. How many remember that beast?? > > > > Remember it and still have it in the basement. > > > I read a lot about the Altair in Byte in those days, but never had a > chance to touch it. Wasn't it horrible expensive? I can't remember what is was going for but I bought mine used for $1,000. It had a number of add-ons including a keyboard and floppy drives. The power supply was also beefed up. It also had a replacement bezel. It seems that the original Altair's silk screened front panel was crappy and rubbed off easily. Some company sold one but it says "Cycloid" instead of "Altair" on it. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From test at joepgen.com Tue Aug 30 14:51:54 2016 From: test at joepgen.com (Joe) Date: Tue, 30 Aug 2016 20:51:54 +0200 Subject: [OT] Altair In-Reply-To: References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: Am 30.08.2016 um 17:52 schrieb D'Arcy J.M. Cain: > On Tue, 30 Aug 2016 15:56:07 +0200 > Joe wrote: >> Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: >>> On Mon, 29 Aug 2016 21:21:05 -0700 >>> Larry Hudson via Python-list wrote: >>>> I remember it well. It's what I used to initially learn C. I'm a >>>> completely self-taught, hobby programmer. Been around since the >>>> MITS Altair. How many remember that beast?? >>> >>> Remember it and still have it in the basement. >>> >> I read a lot about the Altair in Byte in those days, but never had a >> chance to touch it. Wasn't it horrible expensive? > > I can't remember what is was going for but I bought mine used for > $1,000. It had a number of add-ons including a keyboard and floppy > drives. The power supply was also beefed up. > > It also had a replacement bezel. It seems that the original Altair's > silk screened front panel was crappy and rubbed off easily. Some > company sold one but it says "Cycloid" instead of "Altair" on it. > I think the first BASIC Interpreter ever sold by Gates & Friend was for this machine? How did you use your floppy drives on this machine (Open-Write-Close)? From darcy at Vex.Net Tue Aug 30 15:48:24 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Tue, 30 Aug 2016 15:48:24 -0400 Subject: [OT] Altair In-Reply-To: References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: <20160830154824.6605e910@imp> On Tue, 30 Aug 2016 20:51:54 +0200 Joe wrote: > I think the first BASIC Interpreter ever sold by Gates & Friend was > for this machine? How did you use your floppy drives on this machine > (Open-Write-Close)? The floppy drive came with an operating system called MDOS - Micropolis Disk Operating System. Once you loaded it by selecting the ROM address on the front panel, setting the PC and hitting RUN it took over. There was a bit of configuration, handled through the switches, that had to be done and then any further changes were done to the image in memory and saved to disk. When I finally upgraded to a whopping 64K of RAM (with a Morrow S-100 memory board) I had to blank out the section where the disk controller sat in memory. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From orgnut at yahoo.com Tue Aug 30 15:52:05 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Tue, 30 Aug 2016 12:52:05 -0700 Subject: Multimeter USB output In-Reply-To: References: <20160830070154.5cb89421@imp> Message-ID: On 08/30/2016 04:01 AM, D'Arcy J.M. Cain wrote: > On Mon, 29 Aug 2016 21:21:05 -0700 > Larry Hudson via Python-list wrote: >> I remember it well. It's what I used to initially learn C. I'm a >> completely self-taught, hobby programmer. Been around since the MITS >> Altair. How many remember that beast?? > > Remember it and still have it in the basement. > Mine is stuffed into the back of a closet. :-) It was still working when I stored it, but I don't think I could remember how to bring it up again. As I recall, you had to set the starting memory address via the front-panel switches ? but the details have long since evaporated from my memory. :-( -- -=- Larry -=- From none at invalid.com Tue Aug 30 16:01:47 2016 From: none at invalid.com (mm0fmf) Date: Tue, 30 Aug 2016 21:01:47 +0100 Subject: Multimeter USB output In-Reply-To: References: Message-ID: On 29/08/2016 09:54, Joe wrote: > Am 28.08.2016 um 17:22 schrieb Dennis Lee Bieber: >> If you can read spaghetti coded C, you might want to study >> https://sourceforge.net/projects/ut61/ > > Interesting, but... The last time I did something with c, it was with > BDS-C under CM/M. Somebody remenbering this no-fp compiler from the dark > age before PC und Linux? Yes. It was a long time ago when I used BDS-C on initially Apple II + Microsoft CP/M card. Then on other Z80 CP/M systems. I also was introduced to the editor MINCE (Mince is not complete Emacs) which was compiled with BDS-C. 33 years ago and it seems like yesterday! From orgnut at yahoo.com Tue Aug 30 16:26:59 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Tue, 30 Aug 2016 13:26:59 -0700 Subject: [OT] Altair In-Reply-To: References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: On 08/30/2016 11:51 AM, Joe wrote: > Am 30.08.2016 um 17:52 schrieb D'Arcy J.M. Cain: >> On Tue, 30 Aug 2016 15:56:07 +0200 >> Joe wrote: >>> Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: >>>> On Mon, 29 Aug 2016 21:21:05 -0700 >>>> Larry Hudson via Python-list wrote: >>>>> I remember it well. It's what I used to initially learn C. I'm a >>>>> completely self-taught, hobby programmer. Been around since the >>>>> MITS Altair. How many remember that beast?? >>>> >>>> Remember it and still have it in the basement. >>>> >>> I read a lot about the Altair in Byte in those days, but never had a >>> chance to touch it. Wasn't it horrible expensive? >> I don't think so. The figure I have in my (very fallible) mind is $300 (or maybe $600) for my original Altair with 1K RAM and no peripherals. >> I can't remember what is was going for but I bought mine used for >> $1,000. It had a number of add-ons including a keyboard and floppy >> drives. The power supply was also beefed up. >> >> It also had a replacement bezel. It seems that the original Altair's >> silk screened front panel was crappy and rubbed off easily. Some >> company sold one but it says "Cycloid" instead of "Altair" on it. >> > > I think the first BASIC Interpreter ever sold by Gates & Friend was for this machine? How did > you use your floppy drives on this machine (Open-Write-Close)? > ('Friend' is Paul Allan.) My first floppy was from Northstar Computers. It used the first 5? drive made (by Shugart). It came with it's own DOS and BASIC -- somewhat incompatible with Gates' Altair BASIC, but very usable. (Anyway, Altair BASIC did not handle disks ? at least the original version did not.) The Northstar DOS was somewhat weird due to where it was located in memory. The bottom 8K of RAM was free, the next 2K was the DOS (that's right a complete Disk-Operating-System in 2K of memory!). The rest of available memory above that was also free. The BASIC was also loaded in this memory above the DOS. Trivia: (and perhaps wrong) ? The original name of Northstar Computers was Kentucky Fried Computers. -- -=- Larry -=- From christopher_reimer at icloud.com Tue Aug 30 16:54:37 2016 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Tue, 30 Aug 2016 13:54:37 -0700 Subject: [OT] Altair In-Reply-To: References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: > On Aug 30, 2016, at 11:51 AM, Joe wrote: > >> Am 30.08.2016 um 17:52 schrieb D'Arcy J.M. Cain: >> On Tue, 30 Aug 2016 15:56:07 +0200 >> Joe wrote: >>> Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: >>>> On Mon, 29 Aug 2016 21:21:05 -0700 >>>> Larry Hudson via Python-list wrote: >>>>> I remember it well. It's what I used to initially learn C. I'm a >>>>> completely self-taught, hobby programmer. Been around since the >>>>> MITS Altair. How many remember that beast?? >>>> >>>> Remember it and still have it in the basement. >>> I read a lot about the Altair in Byte in those days, but never had a >>> chance to touch it. Wasn't it horrible expensive? >> >> I can't remember what is was going for but I bought mine used for >> $1,000. It had a number of add-ons including a keyboard and floppy >> drives. The power supply was also beefed up. >> >> It also had a replacement bezel. It seems that the original Altair's >> silk screened front panel was crappy and rubbed off easily. Some >> company sold one but it says "Cycloid" instead of "Altair" on it. > > I think the first BASIC Interpreter ever sold by Gates & Friend was for this machine? How did you use your floppy drives on this machine (Open-Write-Close)? Paper tape reader. The first time I came across a paper tape reader when I visited the university as a teenager in 1984. A CNC machine read the paper tape to drill six holes in a piece of metal. Chris R. From rosuav at gmail.com Tue Aug 30 17:16:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Aug 2016 07:16:51 +1000 Subject: What's the best way to minimize the need of run time checks? In-Reply-To: References: Message-ID: On Wed, Aug 31, 2016 at 4:30 AM, Quivis wrote: > On Mon, 29 Aug 2016 13:09:28 +1000, Chris Angelico wrote: > >> What's the best way to minimize the need of run time checks? > > Don't do programming! > > Did I win something? You might have won the 100m dash, except that we couldn't verify your velocity of locomotion without a... run time check. Badumtish. ChrisA From sivan at vitakka.co Tue Aug 30 17:56:44 2016 From: sivan at vitakka.co (Sivan Greenberg) Date: Wed, 31 Aug 2016 00:56:44 +0300 Subject: [OT] Altair In-Reply-To: <20160830115215.78a9ec7f@imp> References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: I had a Spectra Video, and I do remember fondly for allowing to experiment with fractal and all sorts of programming in BASIC. I recently watched the Steve Jobs movie and it was nice seeing how Apple was promoting that stage of early personal computers. On Tue, Aug 30, 2016 at 6:52 PM, D'Arcy J.M. Cain wrote: > On Tue, 30 Aug 2016 15:56:07 +0200 > Joe wrote: > > Am 30.08.2016 um 13:01 schrieb D'Arcy J.M. Cain: > > > On Mon, 29 Aug 2016 21:21:05 -0700 > > > Larry Hudson via Python-list wrote: > > >> I remember it well. It's what I used to initially learn C. I'm a > > >> completely self-taught, hobby programmer. Been around since the > > >> MITS Altair. How many remember that beast?? > > > > > > Remember it and still have it in the basement. > > > > > I read a lot about the Altair in Byte in those days, but never had a > > chance to touch it. Wasn't it horrible expensive? > > I can't remember what is was going for but I bought mine used for > $1,000. It had a number of add-ons including a keyboard and floppy > drives. The power supply was also beefed up. > > It also had a replacement bezel. It seems that the original Altair's > silk screened front panel was crappy and rubbed off easily. Some > company sold one but it says "Cycloid" instead of "Altair" on it. > > -- > D'Arcy J.M. Cain > System Administrator, Vex.Net > http://www.Vex.Net/ IM:darcy at Vex.Net > VoIP: sip:darcy at Vex.Net > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sivan Greenberg Co founder & CTO Vitakka Consulting From grant.b.edwards at gmail.com Tue Aug 30 18:02:46 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 30 Aug 2016 22:02:46 +0000 (UTC) Subject: [OT] Altair References: <20160830070154.5cb89421@imp> <20160830115215.78a9ec7f@imp> Message-ID: On 2016-08-30, Christopher Reimer wrote: > Paper tape reader. The first time I came across a paper tape reader > when I visited the university as a teenager in 1984. A CNC machine > read the paper tape to drill six holes in a piece of metal. The file containing the list of coordinates generated by a CAD program for holes in a circuit board is still often referred to as a "drill tape". -- Grant Edwards grant.b.edwards Yow! Now that I have my at "APPLE", I comprehend COST gmail.com ACCOUNTING!! From orgnut at yahoo.com Tue Aug 30 22:36:00 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Tue, 30 Aug 2016 19:36:00 -0700 Subject: Multimeter USB output In-Reply-To: <877fayq3y4.fsf@jester.gateway.pace.com> References: <877fayq3y4.fsf@jester.gateway.pace.com> Message-ID: On 08/29/2016 09:24 PM, Paul Rubin wrote: > Larry Hudson writes: >>> with BDS-C under CP/M. Somebody remenbering this no-fp compiler from >>> the dark age before PC und Linux? >> I remember it well. It's what I used to initially learn C. > > Source code is online here: > > http://www.bdsoft.com/resources/bdsc.html > [...] I remember reading a magazine interview with Leor Zolman (the author of BDS-C) where he mentioned what the BDS stood for... He said is was his nickname in college: Brain Dead. Actually "Brain Dead Software" it was not! It was really quite good, and rather widely used at the time. -- -=- Larry -=- From no.email at nospam.invalid Wed Aug 31 00:28:48 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 30 Aug 2016 21:28:48 -0700 Subject: Multimeter USB output References: <877fayq3y4.fsf@jester.gateway.pace.com> Message-ID: <87twe1h88v.fsf@nightsong.com> Larry Hudson writes: > Actually "Brain Dead Software" it was not! Brain Damage Software, apparently: https://en.wikipedia.org/wiki/BDS_C From nomail at com.invalid Wed Aug 31 03:08:52 2016 From: nomail at com.invalid (ast) Date: Wed, 31 Aug 2016 09:08:52 +0200 Subject: Variables visibility for methods Message-ID: <57c6828c$0$19736$426a74cc@news.free.fr> Hello I made few experiments about variables visibility for methods. class MyClass: a = 1 def test(self): print(a) obj = MyClass() obj.test() Traceback (most recent call last): File "", line 1, in obj.test() File "", line 4, in test print(a) NameError: name 'a' is not defined =========== RESTART: Shell ============== a = 1 class MyClass: def test(self): print(a) obj = MyClass() obj.test() 1 So it seems that when an object's m?thod is executed, variables in the scope outside the object's class can be read (2nd example), but not variables inside the class (1st example). For 1st example, I know that print(MyClass.a) or print(self.a) would have work. Any comments are welcome. From dieter at handshake.de Wed Aug 31 04:03:02 2016 From: dieter at handshake.de (dieter) Date: Wed, 31 Aug 2016 10:03:02 +0200 Subject: Variables visibility for methods References: <57c6828c$0$19736$426a74cc@news.free.fr> Message-ID: <87poopjrgp.fsf@handshake.de> "ast" writes: > ... > So it seems that when an object's m?thod is executed, variables > in the scope outside the object's class can be read (2nd example), > but not variables inside the class (1st example). > > For 1st example, I know that print(MyClass.a) or print(self.a) > would have work. > > Any comments are welcome. You are right. And it is documented this way. From nomail at com.invalid Wed Aug 31 04:04:29 2016 From: nomail at com.invalid (ast) Date: Wed, 31 Aug 2016 10:04:29 +0200 Subject: Variables visibility for methods In-Reply-To: References: <57c6828c$0$19736$426a74cc@news.free.fr> <87poopjrgp.fsf@handshake.de> Message-ID: <57c68f95$0$7966$426a74cc@news.free.fr> "dieter" a ?crit dans le message de news:mailman.63.1472630594.24387.python-list at python.org... > "ast" writes: > > You are right. And it is documented this way. > Thank you From 864483459 at qq.com Wed Aug 31 05:04:38 2016 From: 864483459 at qq.com (=?gb18030?B?V2FuZGVyZXI=?=) Date: Wed, 31 Aug 2016 17:04:38 +0800 Subject: Fatal Python error Message-ID: Dear friendI'm very sorry to bother you in a busy schedule. But I have a questions about Sigil's environment that really need your help. I'm trying to use Sigil at a windows system,the follow are the relation of Sigil build and run environment. OS:WIN10 Python:python-3.6.0a4-amd64 QT:qt5.5.1 ActivePerl:ActivePerl-5.24.0.2400-MSWin32-x64-300558 Ruby:rubyinstaller-2.3.1-x64 Sigil:Sigil-0.9.6-Code VS:VS2015 The system environment I alse configured.The Sigil project also build successed. But When I run the Sigil.exe that follow errors occured. ------------------ Fatal Python error: Py_Initialize: unable to load the file system codec ------------------ The Sigil.exe can not run, and the main sigil form also can not appearses. Can you help me which I should do to solove this problems? Thanks very much. From p.f.moore at gmail.com Wed Aug 31 08:21:46 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 31 Aug 2016 05:21:46 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <87fupy8aqf.fsf@elektro.pacujo.net> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> Message-ID: <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> On Sunday, 21 August 2016 15:20:39 UTC+1, Marko Rauhamaa wrote: > > Aren?t makefiles data-driven? > > Yes, "make" should be added to my sin list. > > > [Personally Ive always believed that jam is better than make and is > > less used for entirely historical reasons; something like half the > > world eoling with crlf and half with lf. But maybe make is really a > > better design because more imperative?] > > Don't know jam, but can heartily recommend SCons. The data driven side of make is the target: sources part. But (particularly as a Python programmer, where build dependencies are less of an issue) a huge part of make usage is in my experience, simply name: actions pairs (which is the less data driven aspect), maybe with an element of "always do X before Y". I've generally found "make successors" like SCons and waf to be less useful, precisely because they focus on the dependency graph (the data driven side) and less on the trigger-action aspect. Has anyone else found this to be the case? Is there any "make replacement" out there that focuses more on named sets of actions (maybe with prerequisite/successor type interdependencies), and less on building file dependency graphs? Paul From cfkaran2 at gmail.com Wed Aug 31 08:49:27 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Wed, 31 Aug 2016 08:49:27 -0400 Subject: The Joys Of Data-Driven Programming In-Reply-To: <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> Message-ID: <0BFF624B-D294-4012-8F41-19B246101C95@gmail.com> On Aug 31, 2016, at 8:21 AM, Paul Moore wrote: > On Sunday, 21 August 2016 15:20:39 UTC+1, Marko Rauhamaa wrote: >>> Aren?t makefiles data-driven? >> >> Yes, "make" should be added to my sin list. >> >>> [Personally Ive always believed that jam is better than make and is >>> less used for entirely historical reasons; something like half the >>> world eoling with crlf and half with lf. But maybe make is really a >>> better design because more imperative?] >> >> Don't know jam, but can heartily recommend SCons. > > The data driven side of make is the target: sources part. But (particularly as a Python programmer, where build dependencies are less of an issue) a huge part of make usage is in my experience, simply name: actions pairs (which is the less data driven aspect), maybe with an element of "always do X before Y". > > I've generally found "make successors" like SCons and waf to be less useful, precisely because they focus on the dependency graph (the data driven side) and less on the trigger-action aspect. > > Has anyone else found this to be the case? Is there any "make replacement" out there that focuses more on named sets of actions (maybe with prerequisite/successor type interdependencies), and less on building file dependency graphs? Maybe Ninja (https://ninja-build.org/)? I personally like it because of how simple it is, and the fact that it doesn't use leading tabs the way that make does. It is intended to be the assembler for higher-level build systems which are more like compilers. I personally use it as a make replacement because it does what I tell it to do, and nothing else. It may fit what you're after. Thanks, Cem Karan From p.f.moore at gmail.com Wed Aug 31 09:02:37 2016 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 31 Aug 2016 14:02:37 +0100 Subject: The Joys Of Data-Driven Programming In-Reply-To: <0BFF624B-D294-4012-8F41-19B246101C95@gmail.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> <0BFF624B-D294-4012-8F41-19B246101C95@gmail.com> Message-ID: On 31 August 2016 at 13:49, Cem Karan wrote: >> Has anyone else found this to be the case? Is there any "make replacement" out there that focuses more on named sets of actions (maybe with prerequisite/successor type interdependencies), and less on building file dependency graphs? > > Maybe Ninja (https://ninja-build.org/)? I personally like it because of how simple it is, and the fact that it doesn't use leading tabs the way that make does. It is intended to be the assembler for higher-level build systems which are more like compilers. I personally use it as a make replacement because it does what I tell it to do, and nothing else. It may fit what you're after. It still seems focused on the file dependency graph (at least, from a quick look). I'm thinking more of the makefile pattern myproj.whl: pip wheel . ve: build virtualenv ve ve/bin/python -m pip install ./*.whl test: ve push ve bin/python -m py.test popd clean: rm -rf ve Basically, a couple of "subcommands", one of which has 2 prerequisites that are run if needed. Little more in practice than 2 shell scripts with a bit of "if this is already done, skip" logic. Most makefiles I encounter or write are of this form, and make essentially no use of dependency rules or anything more complex than "does the target already exist" checks. Make would be fine for this except for the annoying "must use tabs" rule, and the need to rely on shell (= non-portable, generally unavailable on Windows) constructs for any non-trivial logic. In the days when make was invented, not compiling a source file whose object file was up to date was a worthwhile time saving. Now I'm more likely to just do "cc -c *.c" and not worry about it. Paul From cfkaran2 at gmail.com Wed Aug 31 09:44:27 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Wed, 31 Aug 2016 09:44:27 -0400 Subject: The Joys Of Data-Driven Programming In-Reply-To: References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> <0BFF624B-D294-4012-8F41-19B246101C95@gmail.com> Message-ID: On Aug 31, 2016, at 9:02 AM, Paul Moore wrote: > On 31 August 2016 at 13:49, Cem Karan wrote: >>> Has anyone else found this to be the case? Is there any "make replacement" out there that focuses more on named sets of actions (maybe with prerequisite/successor type interdependencies), and less on building file dependency graphs? >> >> Maybe Ninja (https://ninja-build.org/)? I personally like it because of how simple it is, and the fact that it doesn't use leading tabs the way that make does. It is intended to be the assembler for higher-level build systems which are more like compilers. I personally use it as a make replacement because it does what I tell it to do, and nothing else. It may fit what you're after. > > It still seems focused on the file dependency graph (at least, from a > quick look). > > I'm thinking more of the makefile pattern > > myproj.whl: > pip wheel . > ve: build > virtualenv ve > ve/bin/python -m pip install ./*.whl > test: ve > push ve > bin/python -m py.test > popd > clean: > rm -rf ve > > Basically, a couple of "subcommands", one of which has 2 prerequisites > that are run if needed. Little more in practice than 2 shell scripts > with a bit of "if this is already done, skip" logic. > > Most makefiles I encounter or write are of this form, and make > essentially no use of dependency rules or anything more complex than > "does the target already exist" checks. Make would be fine for this > except for the annoying "must use tabs" rule, and the need to rely on > shell (= non-portable, generally unavailable on Windows) constructs > for any non-trivial logic. > > In the days when make was invented, not compiling a source file whose > object file was up to date was a worthwhile time saving. Now I'm more > likely to just do "cc -c *.c" and not worry about it. OK, I see what you're doing, and you're right, Ninja could be forced to do what you want, but it isn't the tool that you need. Thanks, Cem Karan From marko at pacujo.net Wed Aug 31 09:55:10 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 31 Aug 2016 16:55:10 +0300 Subject: The Joys Of Data-Driven Programming References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> <0BFF624B-D294-4012-8F41-19B246101C95@gmail.com> Message-ID: <87shtlavr5.fsf@elektro.pacujo.net> Cem Karan : > On Aug 31, 2016, at 9:02 AM, Paul Moore wrote: >> In the days when make was invented, not compiling a source file whose >> object file was up to date was a worthwhile time saving. Now I'm more >> likely to just do "cc -c *.c" and not worry about it. > > OK, I see what you're doing, and you're right, Ninja could be forced to > do what you want, but it isn't the tool that you need. So if you always need to perform a quick batch of commands, a simple build.sh script would do, right? Marko From rustompmody at gmail.com Wed Aug 31 09:55:14 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 31 Aug 2016 06:55:14 -0700 (PDT) Subject: The Joys Of Data-Driven Programming In-Reply-To: <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> References: <239aa219-4cd6-4c67-af11-21b2f655c644@googlegroups.com> <87y43uadke.fsf@elektro.pacujo.net> <79bd604d-f42a-44de-81fc-cd549d81fbb3@googlegroups.com> <8737lzammr.fsf@elektro.pacujo.net> <16f15ed9-65d2-4e4f-b2cd-1f59d4a15921@googlegroups.com> <19e40e2c-4e9e-4349-b704-35ff2aee338f@googlegroups.com> <87fupy8aqf.fsf@elektro.pacujo.net> <9d167ff0-455c-4c2e-96a4-2152cf25172d@googlegroups.com> Message-ID: On Wednesday, August 31, 2016 at 5:52:02 PM UTC+5:30, Paul Moore wrote: > On Sunday, 21 August 2016 15:20:39 UTC+1, Marko Rauhamaa wrote: > > > Aren?t makefiles data-driven? > > > > Yes, "make" should be added to my sin list. > > > > > [Personally Ive always believed that jam is better than make and is > > > less used for entirely historical reasons; something like half the > > > world eoling with crlf and half with lf. But maybe make is really a > > > better design because more imperative?] > > > > Don't know jam, but can heartily recommend SCons. > > The data driven side of make is the target: sources part. But (particularly as a Python programmer, where build dependencies are less of an issue) a huge part of make usage is in my experience, simply name: actions pairs (which is the less data driven aspect), maybe with an element of "always do X before Y". > > I've generally found "make successors" like SCons and waf to be less useful, precisely because they focus on the dependency graph (the data driven side) and less on the trigger-action aspect. > > Has anyone else found this to be the case? Is there any "make replacement" out there that focuses more on named sets of actions (maybe with prerequisite/successor type interdependencies), and less on building file dependency graphs? > > Paul Dont know much of jam. Still here goes Jam users write two files Jamfile (like Makefile) and Jambase (the make rules reified which in practice is rarely needed to be modified by a vanilla user) You probably want to use jam and write your own jambase https://swarm.workshop.perforce.com/view/guest/perforce_software/jam/src/Jambase.html From vern.muhr at gmail.com Wed Aug 31 14:29:27 2016 From: vern.muhr at gmail.com (vern.muhr at gmail.com) Date: Wed, 31 Aug 2016 11:29:27 -0700 (PDT) Subject: Variables visibility for methods In-Reply-To: <57c6828c$0$19736$426a74cc@news.free.fr> References: <57c6828c$0$19736$426a74cc@news.free.fr> Message-ID: <750367f7-3d63-4e0c-aaa3-e45d9cced7bf@googlegroups.com> On Wednesday, August 31, 2016 at 12:09:16 AM UTC-7, ast wrote: > Hello > > I made few experiments about variables visibility > for methods. > > class MyClass: > a = 1 > def test(self): > print(a) > > obj = MyClass() > obj.test() > > Traceback (most recent call last): > File "", line 1, in > obj.test() > File "", line 4, in test > print(a) > NameError: name 'a' is not defined > > =========== RESTART: Shell ============== > > a = 1 > > class MyClass: > def test(self): > print(a) > > obj = MyClass() > obj.test() > 1 > > So it seems that when an object's m?thod is executed, variables > in the scope outside the object's class can be read (2nd example), > but not variables inside the class (1st example). > > For 1st example, I know that print(MyClass.a) or print(self.a) > would have work. > > Any comments are welcome. In your example a is an attribute of MyClass, not instances of MyClass like obj. Refrencing a as MyClass.a returns 1. See below: >>> class MyClass: ... a = 1 ... def test(self): ... print(MyClass.a) ... >>> obj = MyClass() >>> obj.test() 1 From eryksun at gmail.com Wed Aug 31 16:20:07 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 31 Aug 2016 20:20:07 +0000 Subject: Fatal Python error In-Reply-To: References: Message-ID: On Wed, Aug 31, 2016 at 9:04 AM, Wanderer <864483459 at qq.com> wrote: > The system environment I alse configured.The Sigil project also build successed. > But When I run the Sigil.exe that follow errors occured. > ------------------ > Fatal Python error: Py_Initialize: unable to load the file system codec > ------------------ This is an expected error if Python can't find the standard library. For example: C:\>cmd /c "set PYTHONHOME=C:\ & python" Fatal Python error: Py_Initialize: unable to load the file system codec ImportError: No module named 'encodings' Current thread 0x00000730 (most recent call first): Maybe you have a stale PYTHONHOME setting (generally this variable should not be set permanently), or maybe you've put the standard library (possibly python36.zip) somewhere unexpected. From yuri.abzyanov at gmail.com Wed Aug 31 23:07:02 2016 From: yuri.abzyanov at gmail.com (juraseg) Date: Wed, 31 Aug 2016 20:07:02 -0700 (PDT) Subject: Variables visibility for methods In-Reply-To: <57c6828c$0$19736$426a74cc@news.free.fr> References: <57c6828c$0$19736$426a74cc@news.free.fr> Message-ID: <6f769d9e-1eb3-469d-bfb2-3ff5122ab2f0@googlegroups.com> ?????, 31 ??????? 2016 ?., 14:09:16 UTC+7 ???????????? ast ???????: > Hello > > I made few experiments about variables visibility > for methods. > > class MyClass: > a = 1 > def test(self): > print(a) > > obj = MyClass() > obj.test() > > Traceback (most recent call last): > File "", line 1, in > obj.test() > File "", line 4, in test > print(a) > NameError: name 'a' is not defined > > =========== RESTART: Shell ============== > > a = 1 > > class MyClass: > def test(self): > print(a) > > obj = MyClass() > obj.test() > 1 > > So it seems that when an object's m?thod is executed, variables > in the scope outside the object's class can be read (2nd example), > but not variables inside the class (1st example). > > For 1st example, I know that print(MyClass.a) or print(self.a) > would have work. > > Any comments are welcome. Class construction in Python does not have scope in usual way. In fact "class" construct is essentially a syntactic sugar to call of "type" built-in function (https://docs.python.org/3.5/library/functions.html#type) and then adding all the parts manually. When you put a variable inside class definition but outside of any method body the variable becomes class attribute (so can be accessed by using MyClass.var or self.var if inside method body, keep in mind though that assigning to self.var creates instance attribute instead of changing class attribute) From jobmattcon at gmail.com Wed Aug 31 23:26:13 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 31 Aug 2016 20:26:13 -0700 (PDT) Subject: how to set paramiko to use ssh version 1? Message-ID: <420297f3-f534-4be2-b42e-760f6f6c25d2@googlegroups.com> how to set paramiko to use ssh version 1?