From steve+python at pearwood.info Sun Oct 1 01:25:28 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 01 Oct 2017 16:25:28 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> On Sun, 1 Oct 2017 11:07 am, Bill wrote: > You and Ned are both right. Up until a few minutes ago, I wasn't > thinking about a class having more than 1 attribute that I wanted to > change. And now I realize that __get__ doesn't really make sense in > that context (in the back of my mind was the notion that @property > defined __get__, __set__ and __del__) and I allowed that to obscure my > vision. I was on the verge of giving up anything to do with computers, > forever. : ) property *does* define __get__ and __set__, but not __del__, rather it is called __delete__. Confusing, I know, but neither __del__ nor __delete__ should be used very often. __del__ is the instance destructor. When you have an object: obj = Some_Class() and the last reference to that object goes out of scope, the garbage collector collects the object and reclaims its memory. Before doing so, it calls obj.__del__ if it exists. (That's a simplified version.) __delete__ on the other hand is part of the descriptor protocol, which is how Python manages: - methods - classmethod and staticmethod - property and more. Generally speaking, the descriptor protocol is considered "advanced", not as advanced or scary as metaclasses, but not for beginners either. However, using property is much less complex. Properties are *computed attributes*. Here is a grossly simplified explanation of how Python does attribute look-ups, closer to what Python did in version 1.5 than what it does now, but its a good place to start. There are three things you can so with an attribute: get it, delete it, or set it. (A) When you try getting an attribute: result = obj.spam the interpreter starts by looking in the object's instance namespace for a key 'spam': obj.__dict__['spam'] If it finds it, it returns the associated value and we're done. If not, it next looks at the object's class: obj.__class__.__dict__['spam'] and if not, then it looks in any superclasses, then it looks for a __getattr__ method, and finally if all else fails it raises AttributeError. (B) Deleting an attribute: del obj.spam is pretty much the same. (C) When you try setting an attribute: obj.spam = eggs the interpreter assigns to the instance namespace: obj.__class__.__dict__['spam'] = eggs So that's (roughly) how Python worked back in 1998 or so, and its still conceptually close to what happens now. Now let's introduce an extra layer of complexity: properties[1]. (A) if the result of looking up obj.spam is a property object, then instead of returning the property object itself, the interpreter calls the property's getter method, and returns what it returns. (B) Likewise, deleting the property calls the property's deleter method. (Its rare to bother with one of them, so in practice you can ignore it.) (C) And setting obj.spam to a new value calls the property's setter method, which is intended to handle setting the value somewhere. (Again, I'm ignoring much of the complexity needed to by the actual implementation, in order to focus on just the basic conceptual steps.) > BTW, your example (below) is very nice! I may have seen something > similar before, but I am starting to appreciate it better now. I think > all of this would have made a bit more sense (to me), if instead of just > "@property", the syntax was "@property.getter". One of the most common use for properties is read-only attributes, so the decision was made long ago to have property alone to be equivalent to property.getter. But if you feel the need, you can write: @property.getter def spam(self): ... but if you do, expect people to look at you funny and say "what's that do?". > Now I am forced to ask > the question, why did they use the underscore (on temperature) in the > example on the bottom of this page? Is one forced to introduce new > identifiers in order to define a setter? Forced to? Not exactly. A computed attribute need not have any storage at all, or it could only use public attributes, like my earlier Circle example. Circle didn't use any setters, but I could have let you set the diameter, which in turn would set the radius: circle.radius = 2 assert circle.diameter == 4 circle.diameter == 2 # requires a setter assert circle.radius == 1 Getting that to work is left as an exercise :-) But most commonly, computed attributes need to store some data aside, somewhere. You could use a global variable, or write it to a file, or stick it in a list. All of these things have obvious problems, so the most sensible approach it to stick the data in a private attribute. The interpreter doesn't enforce notions of private/public when it comes to Python classes, but there's a very strong convention that anything starting with a single underscore is private. [1] Technically, the interpreter knows nothing about properties. What it cares about is *descriptors*. Properties are just one kind of descriptor, as are methods. But I'm intentionally not talking about the gory details of descriptors. Feel free to ask if you care, but honestly, you don't need to care unless you are writing your own descriptor class. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From stephanh42 at gmail.com.invalid Sun Oct 1 01:42:20 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 01 Oct 2017 05:42:20 GMT Subject: merits of Lisp vs Python References: <87mv5bewys.fsf@elektro.pacujo.net> Message-ID: Op 2017-09-30, Marko Rauhamaa schreef : > Robert L. is only trolling. He uses fake technical comments to spread > white supremacy in his signatures. My apologies. Stephan From BILL_NOSPAM at whoknows.net Sun Oct 1 02:46:53 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Sun, 1 Oct 2017 02:46:53 -0400 Subject: newb question about @property In-Reply-To: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > Circle > didn't use any setters, but I could have let you set the diameter, which in > turn would set the radius: > > circle.radius = 2 > assert circle.diameter == 4 > circle.diameter == 2 # requires a setter > assert circle.radius == 1 > > Getting that to work is left as an exercise :-) I may start that exercise in a few minutes! > > But most commonly, computed attributes need to store some data aside, somewhere. > You could use a global variable, or write it to a file, or stick it in a list. > All of these things have obvious problems, so the most sensible approach it to > stick the data in a private attribute. > > The interpreter doesn't enforce notions of private/public when it comes to > Python classes, but there's a very strong convention that anything starting > with a single underscore is private. > > > > [1] Technically, the interpreter knows nothing about properties. What it cares > about is *descriptors*. Properties are just one kind of descriptor, as are > methods. But I'm intentionally not talking about the gory details of > descriptors. Feel free to ask if you care, but honestly, you don't need to care > unless you are writing your own descriptor class. > > Thank you, and everyone else who has contributed to this thread, for helping me. Each contribution I read helped me to get further ahead! I watched an example on YouTube where someone wrote a simple descriptor ("@Time_it) to output the amount of time that it took ordinary functions to complete. To be honest, I AM interested in descriptors. I may reexamine whether David Beazley has more to say about them in his book "Python: Essential Reference", which I have been reading. Obviously, I still have some gaps in my understanding after my first reading. If you were going to show non-Python users, say science undergraduates and faculty, that Python is an interesting tool (in 45 minutes), would one delve into descriptors? I am thinking maybe. Here is what I am thinking at this moment: trivial applications (probably), list comprehensions (definitely), generators (maybe briefly). Whatever I would discuss, I think ending with descriptors could be a strong finish. But I'm certainly not merely interested for the sake of my talk, I obtain some satisfaction in learning how things work. If you can suggest any references for descriptors which you think are good, I would be interested. Thanks, Bill From BILL_NOSPAM at whoknows.net Sun Oct 1 04:10:32 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Sun, 1 Oct 2017 04:10:32 -0400 Subject: newb question about @property In-Reply-To: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > > [1] Technically, the interpreter knows nothing about properties. What it cares > about is *descriptors*. Properties are just one kind of descriptor, as are > methods. But I'm intentionally not talking about the gory details of > descriptors. Feel free to ask if you care, but honestly, you don't need to care > unless you are writing your own descriptor class. > I found the following page to be a very easily accessible discussion about descriptors (it represents the state of my knowledge about descriptors). You got more? : ) https://www.programiz.com/python-programming/decorator From steve+python at pearwood.info Sun Oct 1 04:54:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 01 Oct 2017 19:54:38 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> On Sun, 1 Oct 2017 05:46 pm, Bill wrote: > If you were going to show non-Python users, say science undergraduates > and faculty, that Python is an interesting tool (in 45 minutes), would > one delve into descriptors? Hell no :-) I think there's a hierarchy of difficulty/complexity/mind-bogglingness in Python. From least complex to most: - Using Python in an imperative fashion, as in simple scripts. - Writing your own functions. - Writing your own classes. - Writing generators. - Using decorators, including property. - Writing your own decorators. - Writing your own descriptors. - Writing your own metaclasses (a.k.a. "the killer joke"). I wouldn't touch the last three in a beginner's class, not unless they already had a significant amount of programming experience. > I am thinking maybe. Here is what I am > thinking at this moment: trivial applications (probably), list > comprehensions (definitely), generators (maybe briefly). Whatever I > would discuss, I think ending with descriptors could be a strong finish. That depends on whether your aim is to confuse them or not :-) I don't think the descriptor protocol is something you'll be able to explain in five or ten minutes. *Using* descriptors like property, sure, that's fine. > But I'm certainly not merely interested for the sake of my talk, I > obtain some satisfaction in learning how things work. ?If you can > suggest any references for descriptors which you think are good, I would > be interested. The definitive explanation of descriptors is here: https://docs.python.org/3/howto/descriptor.html -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjol at tjol.eu Sun Oct 1 05:23:42 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 1 Oct 2017 11:23:42 +0200 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <951412ed-abf8-c927-d5cc-cb0ec3cb2eb4@mrabarnett.plus.com> Message-ID: On 01/10/17 03:52, Stefan Ram wrote: > MRAB writes: >> raise ValueError("Temperature below -273 is not possible") > -273.15 > Either way, that depends. https://en.wikipedia.org/wiki/Negative_temperature#Examples From ml at fam-goebel.de Sun Oct 1 13:20:51 2017 From: ml at fam-goebel.de (Ulrich Goebel) Date: Sun, 1 Oct 2017 19:20:51 +0200 Subject: Escape-Sequenzen in einem String identifizieren Message-ID: <6868515c-4fe2-16ba-7701-6b2edc980602@fam-goebel.de> Hallo, ich lese mit Python aus einer PostgreSQL-Datenbank strings (SQL-Typ: text), die ?ber LibreOffice dort hinein geschrieben werden. Diese Strings k?nnen Zeilenschaltungen enthalten, von denen ich aber nicht wei?, wie sie kodiert sind: vielleicht \n, \r, \n\r, \r\n oder sonstwie. Um das herauszufinden, suche ich eine Funktion zeige_escape(string), die mir liefert: s = 'Hallo\nNeue Zeile' >>> zeige_escape(s) Hallo\nNeue Zeile print hilft nicht, denn >>> print(s) Hallo Neue Zeile Hat jemand eine Idee? Dank und Gru? Ulrich -- Ulrich Goebel Am B?chel 57, 53173 Bonn From python at mrabarnett.plus.com Sun Oct 1 13:44:11 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 1 Oct 2017 18:44:11 +0100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <951412ed-abf8-c927-d5cc-cb0ec3cb2eb4@mrabarnett.plus.com> Message-ID: On 2017-10-01 02:52, Stefan Ram wrote: > MRAB writes: >>raise ValueError("Temperature below -273 is not possible") > > -273.15 > I think you've trimmed a little too much. In my reply I was only copying what someone else had written. From tjol at tjol.eu Sun Oct 1 14:02:27 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 1 Oct 2017 20:02:27 +0200 Subject: Escape-Sequenzen in einem String identifizieren In-Reply-To: <6868515c-4fe2-16ba-7701-6b2edc980602@fam-goebel.de> References: <6868515c-4fe2-16ba-7701-6b2edc980602@fam-goebel.de> Message-ID: <96a8f44e-4549-3c84-2c37-6a07ef03feed@tjol.eu> On 01/10/17 19:20, Ulrich Goebel wrote: > Hallo, > > ich lese mit Python aus einer PostgreSQL-Datenbank strings (SQL-Typ: > text), die ?ber LibreOffice dort hinein geschrieben werden. Diese > Strings k?nnen Zeilenschaltungen enthalten, von denen ich aber nicht > wei?, wie sie kodiert sind: vielleicht \n, \r, \n\r, \r\n oder sonstwie. Hallo Ulrich, diese Mailingliste ist englischsprachig. F?r Fragen und Diskussionen auf deutsch gibt es eine eigene Liste, python-de at python.org mit angeschlossener Newsgroup. Anscheinend gibt es auch ein Forum. Siehe: https://wiki.python.org/moin/GermanLanguage#Community_.28Wikis.2C_Mailinglisten.2C_Newsgroups_etc..29 https://mail.python.org/mailman/listinfo/python-de > > Um das herauszufinden, suche ich eine Funktion zeige_escape(string), > die mir liefert: > > s = 'Hallo\nNeue Zeile' > >>>> zeige_escape(s) > Hallo\nNeue Zeile Um die Frage zu beantworten: an der interaktiven Konsole brauchst du sowas gar nicht. >>> s = 'two\nlines' >>> s 'two\nlines' Hier benutzt Python intern die Function repr(): >>> repr(s) "'two\\nlines'" >>> print(s) two lines >>> print(repr(s)) 'two\nlines' >>> > Hat jemand eine Idee? Im Prinzip w?rde ich in so einem Fall eifach empfehlen, alle Zeilenumbr?che, egal wie sie aussehen, pauschal zu ersetzen: s.replace('\r\n', '\n') -- Thomas Jollans From python at mrabarnett.plus.com Sun Oct 1 14:26:44 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 1 Oct 2017 19:26:44 +0100 Subject: Escape-Sequenzen in einem String identifizieren In-Reply-To: <6868515c-4fe2-16ba-7701-6b2edc980602@fam-goebel.de> References: <6868515c-4fe2-16ba-7701-6b2edc980602@fam-goebel.de> Message-ID: On 2017-10-01 18:20, Ulrich Goebel wrote: > Hallo, > > ich lese mit Python aus einer PostgreSQL-Datenbank strings (SQL-Typ: > text), die ?ber LibreOffice dort hinein geschrieben werden. Diese > Strings k?nnen Zeilenschaltungen enthalten, von denen ich aber nicht > wei?, wie sie kodiert sind: vielleicht \n, \r, \n\r, \r\n oder sonstwie. > > Um das herauszufinden, suche ich eine Funktion zeige_escape(string), die > mir liefert: > > s = 'Hallo\nNeue Zeile' > >>>> zeige_escape(s) > Hallo\nNeue Zeile > > print hilft nicht, denn > >>>> print(s) > Hallo > Neue Zeile > > Hat jemand eine Idee? > >>> s = 'Hallo\nNeue Zeile' >>> print(ascii(s)) 'Hallo\nNeue Zeile' From stephanh42 at gmail.com.invalid Sun Oct 1 15:36:14 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 01 Oct 2017 19:36:14 GMT Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Op 2017-10-01, Bill schreef : > I watched an example on YouTube where someone wrote a simple descriptor > ("@Time_it) to output the amount of time that it took ordinary functions > to complete. To be honest, I AM interested in descriptors. Are you sure you are not confusing deSCRIPTtors and deCORAtors here? @Time_it is decorator syntax. Despite the similarity in the words, there are totally different things. Descriptors are objects with __get__, and optionally __set__ and __delete__ methods (i.e. they implement the descriptor protocols). Decorators aren't really an official type, but loosely speaking these are any functions which can be applied meaningfully with a single function or class as argument. Some very mundane functions can be (ab)used as decorators. In [1]: @repr ...: def hello(): ...: pass ...: In [2]: hello Out[2]: '' Stephan From stephanh42 at gmail.com.invalid Sun Oct 1 15:46:08 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 01 Oct 2017 19:46:08 GMT Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Op 2017-10-01, Bill schreef : > Steve D'Aprano wrote: >> >> [1] Technically, the interpreter knows nothing about properties. What >> it cares about is *descriptors*. Properties are just one kind of >> descriptor, as are methods. But I'm intentionally not talking about >> the gory details of descriptors. Feel free to ask if you care, but >> honestly, you don't need to care unless you are writing your own >> descriptor class. >> > I found the following page to be a very easily accessible discussion > about descriptors (it represents the state of my knowledge about > descriptors). You got more? : ) > > https://www.programiz.com/python-programming/decorator I found the page to be a discussion about decorators (unsurprisingly given the URL) and not containing the word "descriptor" at all... Note that the remark from Steve is on the topic of descriptors. I suppose the first advice to anybody wanting to learn about either descriptors or decorators is to not confuse them with the other thing. Stephan From dbastos at toledo.com Sun Oct 1 17:27:58 2017 From: dbastos at toledo.com (Daniel Bastos) Date: Sun, 01 Oct 2017 18:27:58 -0300 Subject: on a very slow function Message-ID: <86h8vi36e9.fsf@toledo.com> def make_sequence_non_recursive(N, x0 = 2, c = -1): "What's wrong with this function? It's very slow." last = x0 def sequence(): nonlocal last next = last last = last**2 + c return next % N return sequence It crawls pretty soon. Please advise? Thank you. >>> f = make_sequence_non_recursive(1032) >>> timeit("f()", setup="from __main__ import f", number=1) 2.851021349670191e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 2.851021349670191e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 2.851021349670191e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 3.1361234960058937e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 3.1361234960058937e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 3.9914298213261645e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 4.276531967661867e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 4.276531967661867e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 4.846736260333273e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 5.131838292982138e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 7.412655463667761e-06 >>> timeit("f()", setup="from __main__ import f", number=1) 1.3684902228305873e-05 >>> timeit("f()", setup="from __main__ import f", number=1) 3.250164286328072e-05 >>> timeit("f()", setup="from __main__ import f", number=1) 9.009227323986124e-05 >>> timeit("f()", setup="from __main__ import f", number=1) 0.0002594429389546349 >>> timeit("f()", setup="from __main__ import f", number=1) 0.0007706310592539012 >>> timeit("f()", setup="from __main__ import f", number=1) 0.002319305833339058 >>> timeit("f()", setup="from __main__ import f", number=1) 0.007012372007011436 >>> timeit("f()", setup="from __main__ import f", number=1) 0.021163131162552418 >>> timeit("f()", setup="from __main__ import f", number=1) 0.06563849334929728 >>> timeit("f()", setup="from __main__ import f", number=1) 0.1944633166216363 >>> timeit("f()", setup="from __main__ import f", number=1) 0.5879943492758457 >>> timeit("f()", setup="from __main__ import f", number=1) 1.7068785165565714 >>> timeit("f()", setup="from __main__ import f", number=1) 5.1329479702866365 >>> timeit("f()", setup="from __main__ import f", number=1) 15.436177179570109 >>> From rosuav at gmail.com Sun Oct 1 17:35:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 08:35:24 +1100 Subject: on a very slow function In-Reply-To: <86h8vi36e9.fsf@toledo.com> References: <86h8vi36e9.fsf@toledo.com> Message-ID: On Mon, Oct 2, 2017 at 8:27 AM, Daniel Bastos wrote: > def make_sequence_non_recursive(N, x0 = 2, c = -1): > "What's wrong with this function? It's very slow." > last = x0 > def sequence(): > nonlocal last > next = last > last = last**2 + c > return next % N > return sequence > > It crawls pretty soon. Please advise? Thank you. For a start, it should probably be implemented as a generator. I'm not sure what this is even trying to do. For seconds, the variable names could do with some improvement. Ditto. ChrisA From BILL_NOSPAM at whoknows.net Sun Oct 1 18:17:06 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Sun, 1 Oct 2017 18:17:06 -0400 Subject: newb question about @property In-Reply-To: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Sun, 1 Oct 2017 05:46 pm, Bill wrote: > >> If you were going to show non-Python users, say science undergraduates >> and faculty, that Python is an interesting tool (in 45 minutes), would >> one delve into descriptors? > Hell no :-) Oops, I see I used the word "descriptor", where I meant "decorator" (at least newb is still in the subject line). I don't even know what a descriptor is yet--I know a bit more about meta-classes! %-) So, on your list, I'm basically up to the leading edge of (5), writing decorators. But my previous programming experience helped me to blast through (1) to (4). Going forward, it appears the walkway will be a little steeper. I do appreciate your list as it improves my perspective. From the point of view of getting others to be interested, I'm not sure *classes* and object oriented design/thinking do it. I think they are more of an acquired taste... Functions, on the other hand, are easy to like, I think--especially in Python. > > I think there's a hierarchy of difficulty/complexity/mind-bogglingness in > Python. From least complex to most: > > - Using Python in an imperative fashion, as in simple scripts. > > - Writing your own functions. > > - Writing your own classes. > > - Writing generators. > > - Using decorators, including property. > > - Writing your own decorators. > > - Writing your own descriptors. > > - Writing your own metaclasses (a.k.a. "the killer joke"). > > I wouldn't touch the last three in a beginner's class, not unless they already > had a significant amount of programming experience. > > > >> I am thinking maybe. Here is what I am >> thinking at this moment: trivial applications (probably), list >> comprehensions (definitely), generators (maybe briefly). Whatever I >> would discuss, I think ending with descriptors could be a strong finish. > That depends on whether your aim is to confuse them or not :-) > > I don't think the descriptor protocol is something you'll be able to explain in > five or ten minutes. *Using* descriptors like property, sure, that's fine. > > >> But I'm certainly not merely interested for the sake of my talk, I >> obtain some satisfaction in learning how things work. If you can >> suggest any references for descriptors which you think are good, I would >> be interested. > The definitive explanation of descriptors is here: > > https://docs.python.org/3/howto/descriptor.html Thanks! Bill > > > From dbastos at toledo.com Sun Oct 1 18:22:37 2017 From: dbastos at toledo.com (Daniel Bastos) Date: Sun, 01 Oct 2017 19:22:37 -0300 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> Message-ID: <8637721paq.fsf@toledo.com> Chris Angelico writes: > On Mon, Oct 2, 2017 at 8:27 AM, Daniel Bastos wrote: >> def make_sequence_non_recursive(N, x0 = 2, c = -1): >> "What's wrong with this function? It's very slow." >> last = x0 >> def sequence(): >> nonlocal last >> next = last >> last = last**2 + c >> return next % N >> return sequence >> >> It crawls pretty soon. Please advise? Thank you. > > For a start, it should probably be implemented as a generator. Maybe something like this? def make_generator(N, last = 2, c = -1): while True: yield last last = (last**2 + c) % N I wish I had an argument that would give me the ith element of this sequence. Like this: def make_sequence(N, x0 = 2, c = -1): def sequence(i): if i == 0: return x0 return (sequence(i - 1)**2 + c) % N return sequence With this function, I can do: f = make_sequence(1032) [f(0), f(1), f(32), f(59)] But I can't ask for high values because eventually I hit the recursion limit. So I'd like a function that's (1) non recursive and hopefully (2) would somehow save what it has computed (during its life time) to avoid recomputing. So if called in series such as in f(0), f(1), ..., f(i), f(i + 1), ... It's then quick because to compute f(i), it needs f(0), f(1), ..., f(i - 1), which it has already computed and remembers it. I think I can look up efficient implementations of the Fibonacci sequence. IIRC, Paul Graham's ANSI Common Lisp shows how to do it for Fibonacci. Maybe it'll help me. I'll look it up. > I'm not sure what this is even trying to do. That function produces a function which yields the values of the sequence x^2 - 1 mod N One value per call. So f = make_sequence_non_recursive(55) [f() for i in range(3) ] == [2, 3, 8] >From this point on, f() produces 8 forever. These sequences are studied in pseudo-random number generation. See for example ``The Art of Computer Programming'', D.E. Knuth, volume 2, seminumerical algorithms, chapter 3, section 3.2.1, ``The Linear Congruential Method''. In such contexts, the choice of variable names is /usually/ clear. I apologize if not introducing the context made it hard to understand. It didn't occur to me until you mentioned. Thank you! From BILL_NOSPAM at whoknows.net Sun Oct 1 18:47:32 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Sun, 1 Oct 2017 18:47:32 -0400 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Stephan Houben wrote: > Op 2017-10-01, Bill schreef : >> I watched an example on YouTube where someone wrote a simple descriptor >> ("@Time_it) to output the amount of time that it took ordinary functions >> to complete. To be honest, I AM interested in descriptors. > Are you sure you are not confusing deSCRIPTtors and deCORAtors here? Yet, you are absolutely correct! Thank you for clarifying! From your description, I can see that it was *decorators*, which drew my interest. It appears that *property* is perhaps both a decorator and a descriptor, at least when used as in the examples we have been discussing. According to the language grammar, which all newbys like me should have handy (remember my ADT acronym from another thread?), decorators can be applied to a classdef, a funcdef, or a async_funcdef (the latter I assume is a "callback" function definition). Surely the difference in syntax between funcdef and async_funcdef will be revealed to me by looking closer at the grammar! : ) Bill > > @Time_it > > is decorator syntax. > > Despite the similarity in the words, there are totally different things. > > Descriptors are objects with __get__, and optionally __set__ and > __delete__ methods (i.e. they implement the descriptor protocols). > > Decorators aren't really an official type, but loosely speaking these > are any functions which can be applied meaningfully with a single > function or class as argument. Some very mundane functions can be > (ab)used as decorators. > > In [1]: @repr > ...: def hello(): > ...: pass > ...: > > In [2]: hello > Out[2]: '' > > Stephan From ben.usenet at bsb.me.uk Sun Oct 1 18:49:32 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 01 Oct 2017 23:49:32 +0100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> Message-ID: <874lrieb5v.fsf@bsb.me.uk> Daniel Bastos writes: > def make_sequence_non_recursive(N, x0 = 2, c = -1): > "What's wrong with this function? It's very slow." > last = x0 > def sequence(): > nonlocal last > next = last > last = last**2 + c > return next % N > return sequence > > It crawls pretty soon. Please advise? A mathematical rather than Python answer... change it to last = (last**2 + c) % N return next -- Ben. From BILL_NOSPAM at whoknows.net Sun Oct 1 18:52:00 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Sun, 1 Oct 2017 18:52:00 -0400 Subject: newb question about @property In-Reply-To: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > The definitive explanation of descriptors is here: > https://docs.python.org/3/howto/descriptor.html Thank you! It is next on my list. Then I'll try that Circle problem you mentioned as an exercise last night! I don't expect run into any difficulties. : ) From rosuav at gmail.com Sun Oct 1 18:55:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 09:55:37 +1100 Subject: on a very slow function In-Reply-To: <8637721paq.fsf@toledo.com> References: <86h8vi36e9.fsf@toledo.com> <8637721paq.fsf@toledo.com> Message-ID: On Mon, Oct 2, 2017 at 9:22 AM, Daniel Bastos wrote: > Chris Angelico writes: > >> For a start, it should probably be implemented as a generator. > > Maybe something like this? > > def make_generator(N, last = 2, c = -1): > while True: > yield last > last = (last**2 + c) % N Yeah, that's a lot less guff, and it clearly shows that you're generating a series of values. Will you ever provide 'last' or 'c' as actual arguments? If not, they shouldn't be there in the function signature. There are all sorts of crazy shenanigans done by people who think recursion is the only way to do things, and we who use generator functions don't have to do them. One last thing: can you name the function something more useful? def flurble(n): value = 2 c = -1 while True: yield value value = (value**2 + c) % n But if you would be providing the initial value and the constant, then they do belong in the parameters, so ignore this. > I wish I had an argument that would give me the ith element of this > sequence. Like this: > > def make_sequence(N, x0 = 2, c = -1): > def sequence(i): > if i == 0: > return x0 > return (sequence(i - 1)**2 + c) % N > return sequence > > With this function, I can do: > > f = make_sequence(1032) > [f(0), f(1), f(32), f(59)] > > But I can't ask for high values because eventually I hit the recursion > limit. So I'd like a function that's (1) non recursive and hopefully > (2) would somehow save what it has computed (during its life time) to > avoid recomputing. Sounds like what you want is "memoization". Here's the easiest way to do it: def flurble(n, start=2, c=-1): # still wants a better name than flurble cache = [start] def sequence(i): while i >= len(cache): cache.append((cache[-1]**2 + c) % n) return cache[i] return sequence > So if called in series such as in > > f(0), f(1), ..., f(i), f(i + 1), ... > > It's then quick because to compute f(i), it needs > > f(0), f(1), ..., f(i - 1), > > which it has already computed and remembers it. Yep, exactly. In this case, since the sole argument is a counting number, a list can be used to great efficiency; in the more general case, a dictionary is more appropriate. Check out functools.lru_cache for a general memoization decorator. >> I'm not sure what this is even trying to do. > > That function produces a function which yields the values of theof > sequence > > x^2 - 1 mod N > > One value per call. So > > f = make_sequence_non_recursive(55) > [f() for i in range(3) ] == [2, 3, 8] > > From this point on, f() produces 8 forever. These sequences are studied > in pseudo-random number generation. See for example ``The Art of > Computer Programming'', D.E. Knuth, volume 2, seminumerical algorithms, > chapter 3, section 3.2.1, ``The Linear Congruential Method''. > > In such contexts, the choice of variable names is /usually/ clear. I > apologize if not introducing the context made it hard to understand. It > didn't occur to me until you mentioned. Yeah, and I do know plenty of situations where you're directly transforming a mathematical equation into code, so all those one-letter names make good sense. But it would be helpful to include a link or reference in your original post :) Maybe "linear_congruential" would be a good name for the function? I don't know. Anyhow, the basic memoization technique should help you some. ChrisA From rosuav at gmail.com Sun Oct 1 18:59:18 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 09:59:18 +1100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: On Mon, Oct 2, 2017 at 9:47 AM, Bill wrote: > Stephan Houben wrote: >> >> Op 2017-10-01, Bill schreef : >>> >>> I watched an example on YouTube where someone wrote a simple descriptor >>> ("@Time_it) to output the amount of time that it took ordinary functions >>> to complete. To be honest, I AM interested in descriptors. >> >> Are you sure you are not confusing deSCRIPTtors and deCORAtors here? > > > Yet, you are absolutely correct! Thank you for clarifying! From your > description, I can see that it was *decorators*, which drew my interest. It > appears that *property* is perhaps both a decorator and a descriptor, at > least when used as in the examples we have been discussing. Yes, that's correct. The *descriptor* protocol is what allows "foo.bar" to cause a function to be executed; the *decorator* protocol is what lets you "tag" a function: class Foo: @property # this is being used as a decorator def bar(self): return 42 foo = Foo(); print(foo.bar) # this uses descriptor protocol to find the function Decorators are fairly straight-forward if you understand higher-order functions. If you DON'T understand higher-order functions, look those up first and get a solid understanding of what it means to pass a function as a parameter to another function. ChrisA From breamoreboy at gmail.com Sun Oct 1 20:24:50 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 1 Oct 2017 17:24:50 -0700 (PDT) Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <951412ed-abf8-c927-d5cc-cb0ec3cb2eb4@mrabarnett.plus.com> Message-ID: <1ac331ff-107d-48f3-8874-dd8e2336c574@googlegroups.com> On Sunday, October 1, 2017 at 6:47:34 PM UTC+1, MRAB wrote: > On 2017-10-01 02:52, Stefan Ram wrote: > > MRAB writes: > >>raise ValueError("Temperature below -273 is not possible") > > > > -273.15 > > > I think you've trimmed a little too much. In my reply I was only copying > what someone else had written. At least it doesn't contain the bloody irritating ? and ? which drive my autistic head right up the wall. -- Kindest regards. Mark Lawrence. From steve+python at pearwood.info Sun Oct 1 20:34:24 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 02 Oct 2017 11:34:24 +1100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> Message-ID: <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> On Mon, 2 Oct 2017 09:49 am, Ben Bacarisse wrote: > Daniel Bastos writes: > >> def make_sequence_non_recursive(N, x0 = 2, c = -1): >> "What's wrong with this function? It's very slow." >> last = x0 >> def sequence(): >> nonlocal last >> next = last >> last = last**2 + c >> return next % N >> return sequence >> >> It crawls pretty soon. Please advise? > > A mathematical rather than Python answer... Which is the best sort of answer. When possible, simplifying your algorithm is better than speeding up your code. > change it to > > last = (last**2 + c) % N > return next Better: last = (pow(last, 2, N) + (2 % N)) % N will almost certainly be faster for large values of last. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sun Oct 1 20:38:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 11:38:13 +1100 Subject: on a very slow function In-Reply-To: <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> Message-ID: On Mon, Oct 2, 2017 at 11:34 AM, Steve D'Aprano wrote: >> change it to >> >> last = (last**2 + c) % N >> return next > > Better: > > last = (pow(last, 2, N) + (2 % N)) % N > > will almost certainly be faster for large values of last. I think possibly you mean (c % N) in the middle there? Not sure but there ought to be a reference to c somewhere. ChrisA From ben.usenet at bsb.me.uk Sun Oct 1 21:00:49 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 02 Oct 2017 02:00:49 +0100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> Message-ID: <87y3oucqim.fsf@bsb.me.uk> Steve D'Aprano writes: > On Mon, 2 Oct 2017 09:49 am, Ben Bacarisse wrote: > >> Daniel Bastos writes: >> >>> def make_sequence_non_recursive(N, x0 = 2, c = -1): >>> "What's wrong with this function? It's very slow." >>> last = x0 >>> def sequence(): >>> nonlocal last >>> next = last >>> last = last**2 + c >>> return next % N >>> return sequence >>> >>> It crawls pretty soon. Please advise? >> >> A mathematical rather than Python answer... > > Which is the best sort of answer. When possible, simplifying your algorithm is > better than speeding up your code. > >> change it to >> >> last = (last**2 + c) % N >> return next > > Better: > > last = (pow(last, 2, N) + (2 % N)) % N You meant c rather than 2, I think. And I'm not convinced all the %Ns are worth while. Will typical implementations spot that c does not change and calculate c % N only once? Also, a very naive test (I don't know much about how to profile Python) suggests that my line is faster for the specific N being used in the OP's example. > will almost certainly be faster for large values of last. Do you mean for large values of N? If the calculations are mod N, it seems like N will the number that matters. -- Ben. From steve+python at pearwood.info Sun Oct 1 22:14:46 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 02 Oct 2017 13:14:46 +1100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> <87y3oucqim.fsf@bsb.me.uk> Message-ID: <59d1a117$0$14948$b1db1813$d948b532@news.astraweb.com> On Mon, 2 Oct 2017 12:00 pm, Ben Bacarisse wrote: >> Better: >> >> last = (pow(last, 2, N) + (2 % N)) % N > > You meant c rather than 2, I think. Oops, yes, that was a typo. > And I'm not convinced all the %Ns > are worth while. They are all necessary. py> (2**75 + 7) % 12 # Expected value. 3 py> ((2**75) % 12 + (7 % 12)) % 12 # Correct. 3 py> (2**75) % 12 + (7 % 12) # Incorrect. 15 This follows from the rules of modulo arithmetic: if a ? x (mod N) and b ? y (mod N) then a + b ? x + y (mod N) > Will typical implementations spot that c does not > change and calculate c % N only once? No. > Also, a very naive test (I don't > know much about how to profile Python) suggests that my line is faster > for the specific N being used in the OP's example. > >> will almost certainly be faster for large values of last. > > Do you mean for large values of N? If the calculations are mod N, it > seems like N will the number that matters. No, I meant "last". Although on testing, I think you might need so really big values before you'll see a difference. Like hundreds of digits or more. I just tried it with: last = 123456789012345**85 which has over 1000 digits, comparing: (last**2 + 17) % 95 versus: (pow(last, 2, 95) + (17 % 95)) % 95 and the second form is about ten times faster. But for smaller values of last, I agree, the first form will be faster. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From auriocus at gmx.de Mon Oct 2 01:19:14 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 2 Oct 2017 07:19:14 +0200 Subject: on a very slow function In-Reply-To: <86h8vi36e9.fsf@toledo.com> References: <86h8vi36e9.fsf@toledo.com> Message-ID: Am 01.10.17 um 23:27 schrieb Daniel Bastos: > def make_sequence_non_recursive(N, x0 = 2, c = -1): > "What's wrong with this function? It's very slow." > last = x0 > def sequence(): > nonlocal last > next = last > last = last**2 + c > return next % N > return sequence > > It crawls pretty soon. Please advise? Thank you. You seem to do modular arithmetics with modulus N. You can take the modulus after each single operation. This line: last = last**2 + c will quadratically grow the internal state of the generator. Instead, you should write last = (last**2 + c) % N If I have understood what it does, then the result should be the same. Furthermore, as Chris said, Python has built-in generators for this thing. Christian From marko at pacujo.net Mon Oct 2 02:34:42 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Oct 2017 09:34:42 +0300 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: <87d1669hx9.fsf@elektro.pacujo.net> Chris Angelico : > Yes, that's correct. The *descriptor* protocol is what allows > "foo.bar" to cause a function to be executed That mechanism allows you to expose data fields in the API. If the implementation later changes, you can emulate the data fields. I must say, though, I have yet to run into a need for descriptors. > the *decorator* protocol is what lets you "tag" a function: i have yet to need that, either. I have *seen* a semi-useful decorator in code once (@contextlib.contextmanager) but still would prefer explicit dunder methods. Marko From rosuav at gmail.com Mon Oct 2 03:02:23 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 18:02:23 +1100 Subject: newb question about @property In-Reply-To: <87d1669hx9.fsf@elektro.pacujo.net> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <87d1669hx9.fsf@elektro.pacujo.net> Message-ID: On Mon, Oct 2, 2017 at 5:34 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> Yes, that's correct. The *descriptor* protocol is what allows >> "foo.bar" to cause a function to be executed > > That mechanism allows you to expose data fields in the API. If the > implementation later changes, you can emulate the data fields. > > I must say, though, I have yet to run into a need for descriptors. The beauty of Python (over, say, C++) is that you can transparently convert something from being a simple data attribute to being a descriptor. The mere fact that they *exist* benefits your code; they're like a safety net that lets you do what makes sense without worrying that someday, maybe, one of these things might have to become a function. In C++, if something might ever need to be a function, it has to be a function *now*, so Best Practice is to write getters and setters for everything, just in case. In Python, you can convert it to use @property if you ever actually need to, which means you do nothing now. >> the *decorator* protocol is what lets you "tag" a function: > > i have yet to need that, either. I have *seen* a semi-useful decorator > in code once (@contextlib.contextmanager) but still would prefer > explicit dunder methods. There are plenty of programs that don't need decorators, but in some contexts, they are just beautiful. Building a web app in Flask or Django involves functions that get decorated to say what endpoints they handle, for instance. I've periodically used a simple decorator plus some info in the function's docstring to do more magic. I'm not sure where dunder methods come into this, though, as they're completely unrelated. ChrisA From __peter__ at web.de Mon Oct 2 03:41:08 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 02 Oct 2017 09:41:08 +0200 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> Message-ID: Daniel Bastos wrote: > def make_sequence_non_recursive(N, x0 = 2, c = -1): > "What's wrong with this function? It's very slow." > last = x0 > def sequence(): > nonlocal last > next = last > last = last**2 + c > return next % N > return sequence > > It crawls pretty soon. Please advise? Thank you. Let's change your code a bit to get a feel for the size of the numbers you are dealing with: >>> def make_sequence_non_recursive(N, x0 = 2, c = -1): ... "What's wrong with this function? It's very slow." ... last = x0 ... def sequence(): ... nonlocal last ... next = last ... last = last**2 + c ... return next % N ... def get_last(): return last ... return sequence, get_last ... >>> f, get_last = make_sequence_non_recursive(1032) >>> for i in range(100): print(i, f()) ... 0 2 1 3 2 8 3 63 4 872 5 831 6 152 7 399 8 272 9 711 10 872 11 831 12 152 13 399 14 272 15 711 16 872 17 831 18 152 19 399 20 272 21 711 22 872 23 831 ^CTraceback (most recent call last): File "", line 1, in File "", line 7, in sequence KeyboardInterrupt >>> x = get_last() I'd rather not show the actual number, but >>> x.bit_length() 12534884 So at this point it alreay would take 1.5 MB to store the number in binary. The actual format requires even a bit more memory: >>> import sys >>> sys.getsizeof(x) 1671344 So for every operation you have to touch a lot of memory -- and that takes time. Now let's apply Ben's modification: >>> f, get_last = make_sequence_non_recursive(1032) >>> for i in range(24): f() ... 2 8 872 152 272 872 152 272 872 152 272 872 152 272 872 152 272 872 152 272 872 152 272 872 >>> get_last().bit_length() 8 OK. I dare to have that one printed: >>> get_last() 152 I did not time it, but in general less memory touched will translate into faster execution. Expect a huge speed-up... From steve+python at pearwood.info Mon Oct 2 04:37:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 02 Oct 2017 19:37:04 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <87d1669hx9.fsf@elektro.pacujo.net> Message-ID: <59d1fab2$0$14947$b1db1813$d948b532@news.astraweb.com> On Mon, 2 Oct 2017 05:34 pm, Marko Rauhamaa wrote: > I must say, though, I have yet to run into a need for descriptors. You've never called a method? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From marko at pacujo.net Mon Oct 2 04:51:43 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Oct 2017 11:51:43 +0300 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <87d1669hx9.fsf@elektro.pacujo.net> Message-ID: <87d1660w68.fsf@elektro.pacujo.net> Chris Angelico : > On Mon, Oct 2, 2017 at 5:34 PM, Marko Rauhamaa wrote: >> I have *seen* a semi-useful decorator in code once >> (@contextlib.contextmanager) but still would prefer explicit dunder >> methods. > > [...] I'm not sure where dunder methods come into this, though, as > they're completely unrelated. A context manager must implement __enter__() and __exit__(). @contextlib.contextmanager implements them for you. Marko From tjol at tjol.eu Mon Oct 2 05:46:23 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 2 Oct 2017 11:46:23 +0200 Subject: newb question about @property In-Reply-To: <87d1660w68.fsf@elektro.pacujo.net> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <87d1669hx9.fsf@elektro.pacujo.net> <87d1660w68.fsf@elektro.pacujo.net> Message-ID: On 2017-10-02 10:51, Marko Rauhamaa wrote: > Chris Angelico : > >> On Mon, Oct 2, 2017 at 5:34 PM, Marko Rauhamaa wrote: >>> I have *seen* a semi-useful decorator in code once >>> (@contextlib.contextmanager) but still would prefer explicit dunder >>> methods. >> >> [...] I'm not sure where dunder methods come into this, though, as >> they're completely unrelated. > > A context manager must implement __enter__() and __exit__(). > @contextlib.contextmanager implements them for you. Let's revisit the bit of Chris' mail you didn't quote: On 2017-10-02 09:02, Chris Angelico wrote: > > There are plenty of programs that don't need decorators, but in some > contexts, they are just beautiful. Building a web app in Flask or > Django involves functions that get decorated to say what endpoints > they handle, for instance. The point is that there are plenty of useful decorators that have nothing to do with dunder methods. -- Thomas Jollans From steve+python at pearwood.info Mon Oct 2 06:01:00 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 02 Oct 2017 21:01:00 +1100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> Message-ID: <59d20e5e$0$14938$b1db1813$d948b532@news.astraweb.com> On Mon, 2 Oct 2017 06:41 pm, Peter Otten wrote: >>>> x = get_last() > > I'd rather not show the actual number, but > >>>> x.bit_length() > 12534884 Which is approximately 3773408 decimal digits. Using the American system of large numbers, that's approximately a duotrigintillion-duotrigintillion-duotrigintillion-duotrigintillion- duotrigintillion-duotrigintillion-duotrigintillion-duotrigintillion- duotrigintillion-duotrigintillion-duotrigintillion-duotrigintillion- duotrigintillion-duotrigintillion-...-duotrigintillion or so, where the dots represent thirty-eight thousand one hundred more duotrigintillions. A trifling number, much smaller than a googolplex. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Oct 2 06:04:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 02 Oct 2017 21:04:04 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <87d1669hx9.fsf@elektro.pacujo.net> <87d1660w68.fsf@elektro.pacujo.net> Message-ID: <59d20f15$0$14938$b1db1813$d948b532@news.astraweb.com> On Mon, 2 Oct 2017 07:51 pm, Marko Rauhamaa wrote: > Chris Angelico : > >> On Mon, Oct 2, 2017 at 5:34 PM, Marko Rauhamaa wrote: >>> I have *seen* a semi-useful decorator in code once >>> (@contextlib.contextmanager) but still would prefer explicit dunder >>> methods. >> >> [...] I'm not sure where dunder methods come into this, though, as >> they're completely unrelated. > > A context manager must implement __enter__() and __exit__(). > @contextlib.contextmanager implements them for you. Nobody is holding a gun to your head and forcing you to use @contextmanager. Its a convenience, nothing more. You can still write your own __enter__ and __exit__ methods if you prefer. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Mon Oct 2 06:40:15 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 02 Oct 2017 11:40:15 +0100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> <87y3oucqim.fsf@bsb.me.uk> <59d1a117$0$14948$b1db1813$d948b532@news.astraweb.com> Message-ID: <87shf1de9c.fsf@bsb.me.uk> Steve D'Aprano writes: > On Mon, 2 Oct 2017 12:00 pm, Ben Bacarisse wrote: > > >>> Better: >>> >>> last = (pow(last, 2, N) + (2 % N)) % N >> >> You meant c rather than 2, I think. > > Oops, yes, that was a typo. > > >> And I'm not convinced all the %Ns >> are worth while. > > They are all necessary. I meant for the program. It makes no difference to the result if last is left entirely un-reduced (that was the original problem) so there is no need (as far as the program is concerned) to ensure that last is fully reduced. Another naive timing tests suggests that last = (pow(last, 2, N) + (c % N)) % N is measurably slower than last = (pow(last, 2, N) + c) % N and last = pow(last, 2, N) + c is a little faster still. None of these allow last to grow uncontrollably. Obviously we need to know more about the potential range of N and c to do any realistic measurements. >>> will almost certainly be faster for large values of last. >> >> Do you mean for large values of N? If the calculations are mod N, it >> seems like N will the number that matters. > > No, I meant "last". Although on testing, I think you might need so really big > values before you'll see a difference. Like hundreds of digits or > more. Ah, again I meant for the program. Any large value of last will exist for one call only. Obviously there will be /some/ point at which a single call to the the three arg version of pow is better than ** and %, but for that to sustained (in the program), N must be huge too. Another simple test suggests that last * last is faster than last**2 so the best so far (for the N and c originally posted) is the simpler: last = (last * last + c) % N Again I mean in the program as posted, not as a line on its own with arbitrary values of 'last'. -- Ben. From bc at freeuk.com Mon Oct 2 07:24:03 2017 From: bc at freeuk.com (bartc) Date: Mon, 2 Oct 2017 12:24:03 +0100 Subject: on a very slow function In-Reply-To: References: <86h8vi36e9.fsf@toledo.com> Message-ID: On 02/10/2017 08:41, Peter Otten wrote: > Daniel Bastos wrote: > >> def make_sequence_non_recursive(N, x0 = 2, c = -1): >> "What's wrong with this function? It's very slow." >> last = x0 >> def sequence(): >> nonlocal last >> next = last >> last = last**2 + c >> return next % N >> return sequence >>>> x.bit_length() > 12534884 > > So at this point it alreay would take 1.5 MB to store the number in binary. > The actual format requires even a bit more memory: > >>>> import sys >>>> sys.getsizeof(x) > 1671344 > > So for every operation you have to touch a lot of memory -- and that takes > time. If it recalculates 'last' once for each of those couple of dozen printed lines, that I doubt accessing a few MB of memory is the issue. More doing such a big calculation. -- bartc From rosuav at gmail.com Mon Oct 2 07:54:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 2 Oct 2017 22:54:16 +1100 Subject: on a very slow function In-Reply-To: References: <86h8vi36e9.fsf@toledo.com> Message-ID: On Mon, Oct 2, 2017 at 10:24 PM, bartc wrote: > On 02/10/2017 08:41, Peter Otten wrote: >> >> Daniel Bastos wrote: >> >>> def make_sequence_non_recursive(N, x0 = 2, c = -1): >>> "What's wrong with this function? It's very slow." >>> last = x0 >>> def sequence(): >>> nonlocal last >>> next = last >>> last = last**2 + c >>> return next % N >>> return sequence > > >>>>> x.bit_length() >> >> 12534884 >> >> So at this point it alreay would take 1.5 MB to store the number in >> binary. >> The actual format requires even a bit more memory: >> >>>>> import sys >>>>> sys.getsizeof(x) >> >> 1671344 >> >> So for every operation you have to touch a lot of memory -- and that takes >> time. > > > If it recalculates 'last' once for each of those couple of dozen printed > lines, that I doubt accessing a few MB of memory is the issue. More doing > such a big calculation. Yes, but when you're working with a number with that many limbs, it's bound to take some time. Squaring arbitrary numbers is a big job - it's more efficient than O(n?) but still a lot worse than linear time, so on huge numbers it's going to take hugely huge time. *handwave furiously* ChrisA From jimmy.thrasibule at gmail.com Mon Oct 2 09:26:24 2017 From: jimmy.thrasibule at gmail.com (Jimmy Thrasibule) Date: Mon, 2 Oct 2017 15:26:24 +0200 Subject: Distributing multiple packages with on setup.py In-Reply-To: <20170930220913.GA45820@cskk.homeip.net> References: <20170930220913.GA45820@cskk.homeip.net> Message-ID: > I do this with my stuff, but instead of keeping a common setup.py I have an > elaborate and clumsy system that rolls a package or module distro on the > fly, writing a setup.py file in the process. > > So each package/module I publish has a dict names "DISTINFO" in the top > level file, looking like this: > > DISTINFO = { > 'keywords': ["python2", "python3"], > 'classifiers': [ > "Programming Language :: Python", > "Programming Language :: Python :: 2", > "Programming Language :: Python :: 3", > ], > 'install_requires': [ > 'cs.app.flag', > 'cs.env', > 'cs.logutils', > 'cs.pfx', > 'cs.psutils', > ], > 'entry_points': { > 'console_scripts': [ > 'svcd = cs.app.svcd:main' > ], > }, > } I think I will head this direction. And with `setup.cfg `_, it is quite easy to keep the package's metadata in a standard way and feed this to setup(). Regards, Jimmy From thrasibule.jimmy at gmail.com Mon Oct 2 09:27:14 2017 From: thrasibule.jimmy at gmail.com (Jimmy Thrasibule) Date: Mon, 2 Oct 2017 15:27:14 +0200 Subject: Distributing multiple packages with on setup.py In-Reply-To: <20170930220913.GA45820@cskk.homeip.net> References: <20170930220913.GA45820@cskk.homeip.net> Message-ID: I think I will head this direction. And with `setup.cfg `_, it is quite easy to keep the package's metadata in a standard way and feed this to setup(). Regards, Jimmy From __peter__ at web.de Mon Oct 2 09:54:30 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 02 Oct 2017 15:54:30 +0200 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> Message-ID: bartc wrote: > On 02/10/2017 08:41, Peter Otten wrote: >> Daniel Bastos wrote: >> >>> def make_sequence_non_recursive(N, x0 = 2, c = -1): >>> "What's wrong with this function? It's very slow." >>> last = x0 >>> def sequence(): >>> nonlocal last >>> next = last >>> last = last**2 + c >>> return next % N >>> return sequence > >>>>> x.bit_length() >> 12534884 >> >> So at this point it alreay would take 1.5 MB to store the number in >> binary. The actual format requires even a bit more memory: >> >>>>> import sys >>>>> sys.getsizeof(x) >> 1671344 >> >> So for every operation you have to touch a lot of memory -- and that >> takes time. > > If it recalculates 'last' once for each of those couple of dozen printed > lines, that I doubt accessing a few MB of memory is the issue. More > doing such a big calculation. You are probably right that the calculation requires a significant amount of the total time here, but it's not just "a few MB". If you look at >>> last = last**2 + c the required memory doubles on every iteration. You will soon run into the problem even under the (overly optimistic) assumption that the calculation requires time proportional to memory. From bc at freeuk.com Mon Oct 2 12:02:53 2017 From: bc at freeuk.com (bartc) Date: Mon, 2 Oct 2017 17:02:53 +0100 Subject: on a very slow function In-Reply-To: References: <86h8vi36e9.fsf@toledo.com> Message-ID: On 02/10/2017 14:54, Peter Otten wrote: > bartc wrote: > >> On 02/10/2017 08:41, Peter Otten wrote: >>> Daniel Bastos wrote: >>> >>>> def make_sequence_non_recursive(N, x0 = 2, c = -1): >>>> "What's wrong with this function? It's very slow." >>>> last = x0 >>>> def sequence(): >>>> nonlocal last >>>> next = last >>>> last = last**2 + c >>>> return next % N >>>> return sequence >> >>>>>> x.bit_length() >>> 12534884 >>> >>> So at this point it alreay would take 1.5 MB to store the number in >>> binary. The actual format requires even a bit more memory: >>> >>>>>> import sys >>>>>> sys.getsizeof(x) >>> 1671344 >>> >>> So for every operation you have to touch a lot of memory -- and that >>> takes time. >> >> If it recalculates 'last' once for each of those couple of dozen printed >> lines, that I doubt accessing a few MB of memory is the issue. More >> doing such a big calculation. > > You are probably right that the calculation requires a significant amount of > the total time here, but it's not just "a few MB". If you look at > >>>> last = last**2 + c > > the required memory doubles on every iteration. You will soon run into the > problem even under the (overly optimistic) assumption that the calculation > requires time proportional to memory. On my machine, the iterations whizzed up the screen until I noticed it pausing on the 21st iteration, when the memory size of last was 0.5MB. When I started last as "A", and multiplied by 2 at each step, it started to pause at the 27th iteration when the size of last was 268MB. I would estimate then another 9 steps of the original (30th iteration) before memory usage has the same effect. Although it would probably already have ground to a halt long before then. -- bartc From marko at pacujo.net Mon Oct 2 12:19:21 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 02 Oct 2017 19:19:21 +0300 Subject: style: single and multiple lines References: Message-ID: <87fub1zfna.fsf@elektro.pacujo.net> ram at zedat.fu-berlin.de (Stefan Ram): > def f(x): return 2*x > > . So this single-line style should not be that bad. I very rarely allow myself to write single-line complex statements. It is usually when defining exceptions: class SyntaxError(Exception): pass if even then. > def f(x): > return 2*x Despite PEP 8, I always separate binary operators from operands with spaces: def f(x): return 2 * x I have made myself abide by PEP 8 wrt named arguments (no space around '='): def f(x=7): return 2 * x y = f(x=3) > ? And is > > def f(x): > y = x*2 > return y > > better than > > def f(x): > y = x*2; return y Yes. Semicolons can be defended only in single-line shell command trickery. > PS: The most difficult part for me is to write > > f(x) > > instead of > > f( x ) For whatever reason, I find it clearest to put separate braces and brackets with spaces: vip = [ employee in faculty for employee.pay_grade >= BOSS.pay_grade ] return { "name" : "length", "value" : 17 } Marko From steve+python at pearwood.info Mon Oct 2 12:56:50 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 03 Oct 2017 03:56:50 +1100 Subject: style: single and multiple lines References: Message-ID: <59d26fd4$0$14953$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 03:00 am, Stefan Ram wrote: > My copy of pep 8 (from 2016) says: Why don't you look at the current version, which is conveniently available to anyone on the internet, for free? https://www.python.org/dev/peps/pep-0008/? (Last commit to the PEP was Jul 12, 2017.) > Yes: > > def f(x): return 2*x > > . So this single-line style should not be that bad. That is specifically contrasted with using a lambda: f = lambda x: 2*x It shouldn't be read as a general approval to try to fit functions all in one line. > So, is this better: > > def f(x): > return 2*x Yes. > ? And is > > def f(x): > y = x*2 > return y > > better than > > def f(x): > y = x*2; return y Hell yes! A thousand times YES. One statement per line, except under the most unusual circumstances. Semi-colon separated statements are a convenience of use at the command line and REPL. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dbastos at toledo.com Mon Oct 2 13:20:02 2017 From: dbastos at toledo.com (Daniel Bastos) Date: Mon, 02 Oct 2017 14:20:02 -0300 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <8637721paq.fsf@toledo.com> Message-ID: <86poa5xy9p.fsf@toledo.com> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Daniel Bastos writes: >> That function produces a function which yields the values of the >> sequence x^2 - 1 mod N > > Thats a term with two free variables. > I am not sure what the sequence is. > > And if that's > > ( x^2 - 1 )mod N That's correct. > there might be a way to calculate it without > calculating the intermediate value of ?x^2 - 1?. I'd be amazed to see how. Thanks! From dbastos at toledo.com Mon Oct 2 13:21:38 2017 From: dbastos at toledo.com (Daniel Bastos) Date: Mon, 02 Oct 2017 14:21:38 -0300 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> Message-ID: <86efqlxy71.fsf@toledo.com> Ben Bacarisse writes: > Daniel Bastos writes: > >> def make_sequence_non_recursive(N, x0 = 2, c = -1): >> "What's wrong with this function? It's very slow." >> last = x0 >> def sequence(): >> nonlocal last >> next = last >> last = last**2 + c >> return next % N >> return sequence >> >> It crawls pretty soon. Please advise? > > A mathematical rather than Python answer... change it to > > last = (last**2 + c) % N > return next Amazing! That was an accident. Thanks for pointing that out! From ian.g.kelly at gmail.com Mon Oct 2 13:23:18 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 2 Oct 2017 11:23:18 -0600 Subject: on a very slow function In-Reply-To: <59d1a117$0$14948$b1db1813$d948b532@news.astraweb.com> References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> <87y3oucqim.fsf@bsb.me.uk> <59d1a117$0$14948$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 1, 2017 at 8:14 PM, Steve D'Aprano wrote: > > On Mon, 2 Oct 2017 12:00 pm, Ben Bacarisse wrote: > > > >> Better: > >> > >> last = (pow(last, 2, N) + (2 % N)) % N > > > > You meant c rather than 2, I think. > > Oops, yes, that was a typo. > > > > And I'm not convinced all the %Ns > > are worth while. > > They are all necessary. > > > py> (2**75 + 7) % 12 # Expected value. > 3 > py> ((2**75) % 12 + (7 % 12)) % 12 # Correct. > 3 > py> (2**75) % 12 + (7 % 12) # Incorrect. > 15 No, only the final one is necessary. Modding the result of the exponentiation might be useful for performance, but if c is expected to be small then it may be pointless to mod that as well. py> ((2**75) % 12 + 7) % 12 # Still correct. 3 From dbastos at toledo.com Mon Oct 2 13:26:24 2017 From: dbastos at toledo.com (Daniel Bastos) Date: Mon, 02 Oct 2017 14:26:24 -0300 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <8637721paq.fsf@toledo.com> Message-ID: <864lrhxxz3.fsf@toledo.com> Chris Angelico writes: [...] > Maybe "linear_congruential" would be a good name for the function? I > don't know. Sounds good to me --- in absence of a smaller name. > Anyhow, the basic memoization technique should help you some. It did! Thanks so much to you and to everyone who contributed to this thread. I really appreciated! From rhodri at kynesim.co.uk Mon Oct 2 14:42:43 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 2 Oct 2017 19:42:43 +0100 Subject: style: single and multiple lines In-Reply-To: References: Message-ID: On 02/10/17 17:00, Stefan Ram wrote: > My copy of pep 8 (from 2016) says: > > Yes: > > def f(x): return 2*x > > . So this single-line style should not be that bad. > > However, I remember someone saying that the multiline > style is more phytonic? > > So, is this better: > > def f(x): > return 2*x Most of the time, yes. The whitespace on the left-hand side is a good visual cue that something content-like is happening, in this case the body of a function. The fact that it has shape makes it easier to comprehend at a glance. > ? And is > > def f(x): > y = x*2 > return y > > better than > > def f(x): > y = x*2; return y Hell yes. One thought per line, please. Something I keep repeating to clients is that whitespace is not the enemy. Not even in C. Judicious use of spacing can make code *much* easier to comprehend. Densely-written code makes you work hard to break it down into manageable chunks; something as simple as the odd blank line to "paragraph" your code can make that a lot easier. Experience also suggests a correlation between code that's hard to read and code that's rather crap. -- Rhodri James *-* Kynesim Ltd From orgnut at yahoo.com Mon Oct 2 15:23:18 2017 From: orgnut at yahoo.com (Larry Hudson) Date: Mon, 2 Oct 2017 12:23:18 -0700 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/01/2017 03:52 PM, Bill wrote: > Steve D'Aprano wrote: >> The definitive explanation of descriptors is here: >> https://docs.python.org/3/howto/descriptor.html > > Thank you!? It is next on my list.?? Then I'll try that Circle problem you mentioned as an > exercise last night!? I don't expect run into any difficulties.? : ) > Except perhaps for your sense of time... "I'll try" implies the future, "last night" is the past. :-) :-) (Couldn't resist...) -- -=- Larry -=- From rosuav at gmail.com Mon Oct 2 15:28:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Oct 2017 06:28:24 +1100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 3, 2017 at 6:23 AM, Larry Hudson via Python-list wrote: > On 10/01/2017 03:52 PM, Bill wrote: >> >> Steve D'Aprano wrote: >>> >>> The definitive explanation of descriptors is here: >>> https://docs.python.org/3/howto/descriptor.html >> >> >> Thank you! It is next on my list. Then I'll try that Circle problem you >> mentioned as an exercise last night! I don't expect run into any >> difficulties. : ) >> > > Except perhaps for your sense of time... "I'll try" implies the future, > "last night" is the past. :-) :-) > > (Couldn't resist...) Yes, but "you mentioned" implies the past. I think you have an operator precedence issue. Kappa ChrisA From BILL_NOSPAM at whoknows.net Mon Oct 2 15:32:09 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Mon, 2 Oct 2017 15:32:09 -0400 Subject: newb question about @property In-Reply-To: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > Circle didn't use any setters, but I could have let you set the > diameter, which in > turn would set the radius: > > circle.radius = 2 > assert circle.diameter == 4 > circle.diameter == 2 # requires a setter > assert circle.radius == 1 > > Getting that to work is left as an exercise :-) > It WAS a good exercise!! I was concerned about "infinite recursion" between my two property setters.. Thanks! Next? :) Bill import math class Circle(object): """ Define a circle class with radius and diameter""" def __init__(self, radius): self.radius = radius self.diameter =2*radius @property def radius(self): return self._radius @radius.setter def radius(self, value): self._radius = value self._diameter=2*value @property def diameter(self): return self._diameter @diameter.setter def diameter(self, value): self._diameter = value self._radius = value /2 @property def area(self): return math.pi *self.radius **2 @property def circumference(self): return 2 * math.pi *self.radius ## Start here ## circle = Circle(1 / math.pi) print("Area = {:.2f}".format(circle.area)) print("Circumference = {:.2f}".format(circle.circumference)) circle.radius =2 assert circle.diameter ==4 print("Area = {:.2f}".format(circle.area)) print("Circumference = {:.2f}".format(circle.circumference)) circle.diameter =2 # requires a setter assert circle.radius ==1 print("Area = {:.2f}".format(circle.area)) print("Circumference = {:.2f}".format(circle.circumference)) From BILL_NOSPAM at whoknows.net Mon Oct 2 15:51:20 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Mon, 2 Oct 2017 15:51:20 -0400 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: Chris Angelico wrote: > On Tue, Oct 3, 2017 at 6:23 AM, Larry Hudson via Python-list > wrote: >> On 10/01/2017 03:52 PM, Bill wrote: >>> Steve D'Aprano wrote: >>>> The definitive explanation of descriptors is here: >>>> https://docs.python.org/3/howto/descriptor.html >>> >>> Thank you! It is next on my list. Then I'll try that Circle problem you >>> mentioned as an exercise last night! I don't expect run into any >>> difficulties. : ) >>> >> Except perhaps for your sense of time... "I'll try" implies the future, >> "last night" is the past. :-) :-) >> >> (Couldn't resist...) > Yes, but "you mentioned" implies the past. I think you have an > operator precedence issue. Kappa > > ChrisA Reading the "definitive explanation of descriptors" took me longer than expected.. I am as overly optimistic as any programming person.. ;) Can you inspire me with a good decorator problem (standard homework exercise-level will be fine)? Otherwise I will go create one which will prints a row of asterisks before and after the call to the original function (which I think I should do). But maybe you have something more interesting? Or maybe you can refer me to a good source of Python problems, so I can bug you less? Bill From rosuav at gmail.com Mon Oct 2 15:59:00 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Oct 2017 06:59:00 +1100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 3, 2017 at 6:51 AM, Bill wrote: > Can you inspire me with a good decorator problem (standard homework > exercise-level will be fine)? Otherwise I will go create one which will > prints a row of asterisks before and after the call to the original function > (which I think I should do). But maybe you have something more interesting? > Or maybe you can refer me to a good source of Python problems, so I can bug > you less? > Start with the row of asterisks. Then change your function to make the "ending" line also say how long the function took to complete. That's a handy tool (albeit a simplistic implementation of it). You may find codewars.com useful, though I don't know how many Python-specific puzzles they have. Or just search the web for "programming puzzles" and see what you find. ChrisA From jjenne025 at go.tahomasd.us Mon Oct 2 16:31:30 2017 From: jjenne025 at go.tahomasd.us (3.1415926535897932384626433832795) Date: Mon, 2 Oct 2017 13:31:30 -0700 Subject: Research paper "Energy Efficiency across Programming Languages: How does energy, time, and memory relate?" Message-ID: The page is fixed BTW >>>import this On Sun, Sep 17, 2017 at 6:13 AM, wrote: > Send Python-list mailing list submissions to > python-list at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-list > or, via email, send a message with subject or body 'help' to > python-list-request at python.org > > You can reach the person managing the list at > python-list-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-list digest..." > > Today's Topics: > > 1. Re: Old Man Yells At Cloud (Dennis Lee Bieber) > 2. Re: Old Man Yells At Cloud (Stefan Ram) > 3. Re: Which database system? (Stefan Ram) > 4. CPython has hit 100,000 commits (breamoreboy at gmail.com) > 5. [RELEASE] Python 2.7.14 (Benjamin Peterson) > 6. Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > (breamoreboy at gmail.com) > 7. Re: Old Man Yells At Cloud (Steve D'Aprano) > 8. Re: Old Man Yells At Cloud (John Pote) > 9. Re: Old Man Yells At Cloud (Steve D'Aprano) > 10. Re: Old Man Yells At Cloud (MRAB) > 11. Re: Old Man Yells At Cloud (Chris Angelico) > 12. Re: Old Man Yells At Cloud (Chris Angelico) > 13. Re: Old Man Yells At Cloud (Dennis Lee Bieber) > 14. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Steve > D'Aprano) > 15. Re: Old Man Yells At Cloud (Stefan Ram) > 16. Re: Old Man Yells At Cloud (Paul Rubin) > 17. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Ian Kelly) > 18. Re: Old Man Yells At Cloud (Stefan Ram) > 19. Re: ttk.Notebook Tabs Question (Abdur-Rahmaan Janhangeer) > 20. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Terry Reedy) > 21. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Chris > Angelico) > 22. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Terry Reedy) > 23. Re: Old Man Yells At Cloud (Gregory Ewing) > 24. Re: Old Man Yells At Cloud (Gregory Ewing) > 25. Re: Old Man Yells At Cloud (mm0fmf) > 26. Re: Old Man Yells At Cloud (Steve D'Aprano) > 27. Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" (Steve > D'Aprano) > 28. Re: Which database system? (Amirouche Boubekki) > 29. Typo-squatting PyPi (Alain Ketterlin) > 30. Re: Old Man Yells At Cloud (Leam Hall) > 31. Re: Old Man Yells At Cloud (Abdur-Rahmaan Janhangeer) > 32. speech_to_text python command not working (pizza python) > 33. Re: Old Man Yells At Cloud (Leam Hall) > 34. Re: Old Man Yells At Cloud (Chris Angelico) > 35. Python built-ins versus Javascript [was Re: Old Man Yells At > Cloud] (Steve D'Aprano) > 36. Re: Old Man Yells At Cloud (Steve D'Aprano) > 37. Unicode (was: Old Man Yells At Cloud) (Leam Hall) > 38. Re: Unicode (was: Old Man Yells At Cloud) (Paul Moore) > 39. Re: Unicode (was: Old Man Yells At Cloud) (Chris Angelico) > 40. Re: Unicode (Leam Hall) > 41. Re: Old Man Yells At Cloud (Steve D'Aprano) > 42. Re: Unicode (Peter Otten) > > > ---------- Forwarded message ---------- > From: Dennis Lee Bieber > To: python-list at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 12:52:59 -0400 > Subject: Re: Old Man Yells At Cloud > On Sat, 16 Sep 2017 09:59:43 -0500, Tim Daneliuk > declaimed the following: > > > > >Well, the whole integer floor division thing years ago was the beginning > >of the end - Python was doomed ... > > Yes -- that would give me fits if I were using Python3 currently... > Since so many of the languages I've learned over the past years use the > concept integer / integer => integer_result > > Come to think of it -- wasn't there a post from someone (seems to > expired or been filtered from my client) asking for help with some sorting > program... I'm pretty sure the partitioning logic would croak on Python3 > due to floating point results in the slice indices: > > pA = part[:n/2] > pB = part[n/2:] > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > > > > ---------- Forwarded message ---------- > From: Stefan Ram > To: python-list at python.org > Cc: > Bcc: > Date: 16 Sep 2017 18:00:00 GMT > Subject: Re: Old Man Yells At Cloud > Steve D'Aprano writes: > >"Hi, I've been programming in Python for what seems like days now, and > here's > >all the things that you guys are doing wrong. > > I never ever have written a line of Python 2. I started with > Python 3.6.0. Yet a very frequent mistake of mine is the > imission of parentheses after ?print?. > > WRT my other common errors, It thought of writing > a program that autocorrects my source code as follows: > > Whenever the next line is indented more, add a colon: > > abd def ghi > jkl mno pqr > > ---------------------> > > abd def ghi: > jkl mno pqr > > Whenever the next line is indented more, add "def" to > what looks like a call (and does not start with ?class? > or ?def?; > > f( x ): > return x * x > > ---------------------> > > def f( x ): > return x * x > > Whenever a line starts with "print" and no parentheses > are following, add them: > > print x, 2 > > ---------------------> > > print( x, 2 ) > > Implementing algorithms from "The Art of Computer > Programming" might sometimes be difficult in Python, since > those algorithms IIRC often use ?goto?. > > > > > ---------- Forwarded message ---------- > From: Stefan Ram > To: python-list at python.org > Cc: > Bcc: > Date: 16 Sep 2017 19:12:58 GMT > Subject: Re: Which database system? > Peter Otten <__peter__ at web.de> writes: > >Based on the example I wonder if you really want a table, or whether > >something tree-like would be more appropriate: > > Thanks for the suggestions! > > I will think about the tree approach. > > > > > ---------- Forwarded message ---------- > From: breamoreboy at gmail.com > To: python-list at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 12:30:07 -0700 (PDT) > Subject: CPython has hit 100,000 commits > Looks as if people have been busy over the years. Read all about it > https://github.com/python/cpython > > -- > Kindest regards. > > Mark Lawrence. > > > > ---------- Forwarded message ---------- > From: Benjamin Peterson > To: "python-announce-list at python.org" , " > python-list at python.org" , python-dev at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 15:05:32 -0700 > Subject: [RELEASE] Python 2.7.14 > I'm happy to announce to the immediate availability of Python 2.7.14, > yet another bug fix release in the Python 2.7 series. 2.7.14 includes 9 > months of conservative bug fixes from the 3.x branch. > > Downloads of source code and binaries are at: > https://www.python.org/downloads/release/python-2714/ > > Bugs may be reported at > https://bugs.python.org/ > > Warmly, > Benjamin > 2.7 release manager > (on behalf of all of 2.7's contributors) > > > > ---------- Forwarded message ---------- > From: breamoreboy at gmail.com > To: python-list at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 16:04:03 -0700 (PDT) > Subject: Research paper "Energy Efficiency across Programming Languages: > How does energy, time, and memory relate?" > I thought some might find this https://sites.google.com/view/ > energy-efficiency-languages/ interesting. > > -- > Kindest regards. > > Mark Lawrence. > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 11:00:42 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 02:52 am, Dennis Lee Bieber wrote: > > > > Yes -- that would give me fits if I were using Python3 currently... > > Since so many of the languages I've learned over the past years use the > > concept integer / integer => integer_result > > That would be C, and C derived languages, perhaps? > > Pascal uses integer / integer => float (except Pascal calls it "Real"). If > you > want integer division, use "div". > > I think so did Algol. > > > > Come to think of it -- wasn't there a post from someone (seems to > > expired or been filtered from my client) asking for help with some > sorting > > program... I'm pretty sure the partitioning logic would croak on Python3 > > due to floating point results in the slice indices: > > > > pA = part[:n/2] > > pB = part[n/2:] > > > If you want integer division, you have to use integer division :-) > > pA = part[:n//2] > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: John Pote > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 01:15:41 +0100 > Subject: Re: Old Man Yells At Cloud > > > On 16/09/2017 19:00, Stefan Ram wrote: > >> Steve D'Aprano writes: >> >>> "Hi, I've been programming in Python for what seems like days now, and >>> here's >>> all the things that you guys are doing wrong. >>> >> I never ever have written a line of Python 2. I started with >> Python 3.6.0. Yet a very frequent mistake of mine is the >> imission of parentheses after ?print?. >> >> WRT my other common errors, It thought of writing >> a program that autocorrects my source code as follows: >> >> Whenever the next line is indented more, add a colon: >> >> abd def ghi >> jkl mno pqr >> > I've used Komodo in the past and that editor works other way round, type a > ':' at the end of a line and the next line is indented. I would presume > other language aware editors take the same approach. > print """ > Your idea would require considerable inteligence in the editor, > 1. Otherwise the previous line would print with a ':' > 2. That would frustrate me..... > """ > >> >> ---------------------> >> >> abd def ghi: >> jkl mno pqr >> >> Whenever the next line is indented more, add "def" to >> what looks like a call (and does not start with ?class? >> or ?def?; >> >> f( x ): >> return x * x >> >> ---------------------> >> >> def f( x ): >> return x * x >> >> Whenever a line starts with "print" and no parentheses >> are following, add them: >> >> print x, 2 >> > Strangely I find it irksome to have to write print as a function call in > Python 3. Must reflect those long ago days when I started to program (oops, > I mean code - programming came later). Come to think of it there's > something irksome about every language I've ever used. Python probably the > least..... > >> >> ---------------------> >> print( x, 2 ) >> >> Implementing algorithms from "The Art of Computer >> Programming" might sometimes be difficult in Python, since >> those algorithms IIRC often use ?goto?. >> > Good point. I also have a copy of that work of art (and the fascicles - I > must get out more). Sadly little work these days requires diving into them. > In fairness to Knuth his pseudo code is a high level assembler fully > described in Vol:1 and done as such so folk from any high level language > background could follow the algorithms. Back then many like myself, or > indeed most coders, had a hardware background and well used to assembler. > The nearest thing most silicon has to structured programming is call and > return, gotos are everywhere. > I always think one of the reasons Basic became so popular was its /goto > /statement. The other was the run command, no need to compile. > See /http://entrian.com/goto// for a Python implementation from many > years ago. No idea if it still works. To be honist never needed it - Python > has exceptions. > Also there was a interesting thread '/"Goto" statement in Python/' in > April this year (not the 1st). Although mainly interesting for its > references to gotos in 'C' which I use daily now. err 'C' that is. . . . . . > Used to wind up colleagues by saying loudly "Great, I can use a /*goto > */here.....". This attracted considerable verbal abuse from purists in > earshot and instruction from the boss NOT TO which I pretended to ignore. > After all Python programming is supposed to be fun! > > >> > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 11:09:18 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 04:00 am, Stefan Ram wrote: > > > Steve D'Aprano writes: > >>"Hi, I've been programming in Python for what seems like days now, and > here's > >>all the things that you guys are doing wrong. > > > > I never ever have written a line of Python 2. I started with > > Python 3.6.0. Yet a very frequent mistake of mine is the > > imission of parentheses after ?print?. > > That's remarkable. > > Do you also forget the parentheses around `len`, and `iter`, and > `math.sin()`? > If not, what's so special about print? > > Javascript (at least the Rhino interpreter, if not others) includes a print > function. It too requires parentheses: > > js> print 21 > js: "", line 5: missing ; before statement > js: print 21 > js: .......^ > js> print(21) > 21 > > > > WRT my other common errors, It thought of writing > > a program that autocorrects my source code as follows: > [...] > > Whenever the next line is indented more, add "def" to > > what looks like a call (and does not start with ?class? > > or ?def?; > > > > f( x ): > > return x * x > > Seriously Stefan, forgetting colons is one thing, but if you regularly > forget to > start function definitions with "def", I fear that you're just not paying > attention to what you are doing. > > Do you occasionally find yourself halfway down the street after leaving > home > when you realise you're not wearing any pants? :-) > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: MRAB > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 02:28:37 +0100 > Subject: Re: Old Man Yells At Cloud > On 2017-09-17 02:00, Steve D'Aprano wrote: > >> On Sun, 17 Sep 2017 02:52 am, Dennis Lee Bieber wrote: >> >> >> Yes -- that would give me fits if I were using Python3 currently... >>> Since so many of the languages I've learned over the past years use the >>> concept integer / integer => integer_result >>> >> >> That would be C, and C derived languages, perhaps? >> >> Or Fortran. > > [snip] > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 11:42:05 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, Sep 17, 2017 at 11:28 AM, MRAB wrote: > > On 2017-09-17 02:00, Steve D'Aprano wrote: > >> > >> On Sun, 17 Sep 2017 02:52 am, Dennis Lee Bieber wrote: > >> > >> > >>> Yes -- that would give me fits if I were using Python3 currently... > >>> Since so many of the languages I've learned over the past years use the > >>> concept integer / integer => integer_result > >> > >> > >> That would be C, and C derived languages, perhaps? > >> > > Or Fortran. > > Or machine code on every CPU I've ever worked with. Dividing integers > yields an integer. > > ChrisA > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 11:42:22 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, Sep 17, 2017 at 11:42 AM, Chris Angelico wrote: > > On Sun, Sep 17, 2017 at 11:28 AM, MRAB > wrote: > >> On 2017-09-17 02:00, Steve D'Aprano wrote: > >>> > >>> On Sun, 17 Sep 2017 02:52 am, Dennis Lee Bieber wrote: > >>> > >>> > >>>> Yes -- that would give me fits if I were using Python3 currently... > >>>> Since so many of the languages I've learned over the past years use > the > >>>> concept integer / integer => integer_result > >>> > >>> > >>> That would be C, and C derived languages, perhaps? > >>> > >> Or Fortran. > > > > Or machine code on every CPU I've ever worked with. Dividing integers > > yields an integer. > > (Or more technically, yields two integers - div/mod) > > ChrisA > > > > ---------- Forwarded message ---------- > From: Dennis Lee Bieber > To: python-list at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 21:56:25 -0400 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 11:00:42 +1000, Steve D'Aprano > declaimed the following: > > >On Sun, 17 Sep 2017 02:52 am, Dennis Lee Bieber wrote: > > > > > >> Yes -- that would give me fits if I were using Python3 currently... > >> Since so many of the languages I've learned over the past years use the > >> concept integer / integer => integer_result > > > >That would be C, and C derived languages, perhaps? > > > > FORTRAN, C/C++, Ada (and, of course, Python 1.4 or thereabouts > through > 2.x) > > > > > > >If you want integer division, you have to use integer division :-) > > > >pA = part[:n//2] > > > My thought was that the OP of the sort question might have been > using a > 2.x source, but running a 3.x Python... Since they didn't provide any > explanation of what the wrong behavior was I couldn't comment on if it were > a traceback for having a float index, or some other matter. > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 12:01:25 +1000 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On Sun, 17 Sep 2017 09:04 am, breamoreboy at gmail.com wrote: > > > I thought some might find this > > https://sites.google.com/view/energy-efficiency-languages/ interesting. > > "Made with the new Google Sites, an effortless way to create beautiful > sites." > > More like an effortless way to create a complete dog's breakfast. Once > upon a > time, web sites would degrade gracefully. If something interrupted the page > loading, or something wasn't quite right, or you'd still get something > usable. > Now, if the tiniest thing goes wrong, you get a junk. > > I've tried to see the results, but I just get a bunch of broken images :-( > > > On the linked page, starting from the top and scrolling down, I see: > > - about two screens worth of black white space; > > - followed by three giant black horizontal bars, each one about an inch > high; > > - more white space; > > - what looks like something that was intended to be a side-bar, containing: > > SLE'17 > Home > Results > Setup > More > > - a giant down-pointing arrowhead, about three inches tall, which turns > grey when you mouse-over it but doesn't do anything when clicked; > > - three more links: > > Home > Results > Setup > > which disappear when you mouse-over them; > > - finally some content! > > The tools and graphical data pointed by this page are included in the > research paper "Energy Efficiency across Programming Languages: How > does > Energy, Time and Memory Relate?", accepted at the International > Conference > on Software Language Engineering (SLE) > > [1] Measuring Framework & Benchmarks > [2] Complete Set of Results > [3] Setup > [4] Paper > > > where the last four bits are links; > > - the smug, self-congratulatory comment quoted above about "beautiful > sites"; > > - a button "Create a site" > > - What was presumably intended to be a link, but is actually just a piece > of > plain text: "Report abuse"; > > - more whitespace; > > - and finally a giant blue "i", pointed at the bottom, and slanted at 45 > degrees. Presumably a logo for somebody or something. > > > And yes, I am allowing scripts from Google and Gstatic to run, and the > page is > still broken. > > > Including the hyperlinks, that's about 700 bytes of actual content. Let's > double > it for the overhead of HTML over plain text, so somewhat less than 1.5 KiB > of > content. > > The actual page is 27285 bytes or over 26 KiB. That gives us something > with a > useful content to bloat factor of 1:17, and *the page still doesn't work.* > > And that's not even counting any additional files the page requires, like > CSS, > external javascript files, images, ads, web-bugs, etc. You want to know why > browsing the web today on full ADSL or faster speeds is *slower* than > using a > dial up modem in the 1990s? This is why. > > www.antipope.org/charlie/blog-static/2008/05/why_your_ > internet_experience_i.html > > Nine years later, and the problem is worse, not better. > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: Stefan Ram > To: python-list at python.org > Cc: > Bcc: > Date: 17 Sep 2017 03:37:43 GMT > Subject: Re: Old Man Yells At Cloud > Steve D'Aprano writes: > >Do you also forget the parentheses around `len`, and `iter`, and > `math.sin()`? > > (Parentheses around `len`, and `iter`, and `math.sin()`? > Like in ?(len)?, ?(iter)?, and ?(math.sin())?? No, whenever > I write in LISP, I do not forget them.) > > No. I also not forget them after > ?java.lang.System.out.println?. > > >If not, what's so special about print? > > Maybe it's a habit of an old BASIC programmer. It might > be triggered by a line starting with the word ?print?. > > >Do you occasionally find yourself halfway down the street after leaving > home > >when you realise you're not wearing any pants? :-) > > So far, this only happened in dreams IIRC, it surely was > embarrassing. > > > > > ---------- Forwarded message ---------- > From: Paul Rubin > To: python-list at python.org > Cc: > Bcc: > Date: Sat, 16 Sep 2017 21:07:57 -0700 > Subject: Re: Old Man Yells At Cloud > Steve D'Aprano writes: > >> concept integer / integer => integer_result > > That would be C, and C derived languages, perhaps? > > Certainly not. Fortran, machine languages, etc. all do that too. > > Haskell does the right thing and makes int/int a compile time type > error. Its integer division functions are div and quot. > > Python 2 does something reasonable and Python 3 does something that is > also debatably reasonable. Switching from one reasonable thing to > another without a very good reason is called fixing things that weren't > broken. Python 2 wasn't broken in that area and didn't need to be > fixed. > > > > ---------- Forwarded message ---------- > From: Ian Kelly > To: Python > Cc: > Bcc: > Date: Sat, 16 Sep 2017 22:22:32 -0600 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On Sat, Sep 16, 2017 at 8:01 PM, Steve D'Aprano > wrote: > > On Sun, 17 Sep 2017 09:04 am, breamoreboy at gmail.com wrote: > > > >> I thought some might find this > >> https://sites.google.com/view/energy-efficiency-languages/ interesting. > > > > "Made with the new Google Sites, an effortless way to create beautiful > sites." > > > > More like an effortless way to create a complete dog's breakfast. Once > upon a > > time, web sites would degrade gracefully. If something interrupted the > page > > loading, or something wasn't quite right, or you'd still get something > usable. > > Now, if the tiniest thing goes wrong, you get a junk. > > > > I've tried to see the results, but I just get a bunch of broken images > :-( > > > > > > On the linked page, starting from the top and scrolling down, I see: > > > > - about two screens worth of black white space; > > > > - followed by three giant black horizontal bars, each one about an inch > high; > > > > - more white space; > > > > - what looks like something that was intended to be a side-bar, > containing: > > > > SLE'17 > > Home > > Results > > Setup > > More > > > > - a giant down-pointing arrowhead, about three inches tall, which turns > > grey when you mouse-over it but doesn't do anything when clicked; > > > > - three more links: > > > > Home > > Results > > Setup > > > > which disappear when you mouse-over them; > > > > - finally some content! > > > > The tools and graphical data pointed by this page are included in the > > research paper "Energy Efficiency across Programming Languages: How > does > > Energy, Time and Memory Relate?", accepted at the International > Conference > > on Software Language Engineering (SLE) > > > > [1] Measuring Framework & Benchmarks > > [2] Complete Set of Results > > [3] Setup > > [4] Paper > > > > > > where the last four bits are links; > > > > - the smug, self-congratulatory comment quoted above about "beautiful > sites"; > > > > - a button "Create a site" > > > > - What was presumably intended to be a link, but is actually just a > piece of > > plain text: "Report abuse"; > > > > - more whitespace; > > > > - and finally a giant blue "i", pointed at the bottom, and slanted at 45 > > degrees. Presumably a logo for somebody or something. > > > > > > And yes, I am allowing scripts from Google and Gstatic to run, and the > page is > > still broken. > > It looks fine to me. > > > > Including the hyperlinks, that's about 700 bytes of actual content. > Let's double > > it for the overhead of HTML over plain text, so somewhat less than 1.5 > KiB of > > content. > > > > The actual page is 27285 bytes or over 26 KiB. That gives us something > with a > > useful content to bloat factor of 1:17, and *the page still doesn't > work.* > > > > And that's not even counting any additional files the page requires, > like CSS, > > external javascript files, images, ads, web-bugs, etc. You want to know > why > > browsing the web today on full ADSL or faster speeds is *slower* than > using a > > dial up modem in the 1990s? This is why. > > > > www.antipope.org/charlie/blog-static/2008/05/why_your_ > internet_experience_i.html > > > > Nine years later, and the problem is worse, not better. > > If you're using a cell phone over 2G, then I tentatively agree. But on > my laptop over WiFi, this page that you're complaining about loaded in > 783 ms when I tried it. > > > > ---------- Forwarded message ---------- > From: Stefan Ram > To: python-list at python.org > Cc: > Bcc: > Date: 17 Sep 2017 04:31:00 GMT > Subject: Re: Old Man Yells At Cloud > Paul Rubin writes: > >Haskell does the right thing and makes int/int a compile time type > >error. Its integer division functions are div and quot. > > In VBA, one uses ?\? for integer division, which just so > happens to be the symbol that also is used for integer > division in traditional mathematics. (And also for set > difference IIRC.) > > >Python 2 does something reasonable and Python 3 does something that is > >also debatably reasonable. Switching from one reasonable thing to > >another without a very good reason is called fixing things that weren't > >broken. Python 2 wasn't broken in that area and didn't need to be > >fixed. > > I agree. > > However, not having octal literals, like ?0123?, and ?3/2? > having decimal places makes teaching to beginners slightly > more easy. > > > > > ---------- Forwarded message ---------- > From: Abdur-Rahmaan Janhangeer > To: Wildman > Cc: python-list at python.org > Bcc: > Date: Sun, 17 Sep 2017 08:45:27 +0400 > Subject: Re: ttk.Notebook Tabs Question > by widget["width"] i meant replace widget with your widget > > Abdur-Rahmaan Janhangeer, > Mauritius > abdurrahmaanjanhangeer.wordpress.com > > On 12 Sep 2017 06:45, "Wildman via Python-list" > wrote: > > > I am working on a program that has a ttk.Notebook with > > 12 tabs. Is there a way to determine the total width > > of the tabs in pixels. Just to be clear I am not talking > > about width of the nb container. I am talking about > > tabs themselves that contain the text. > > > > I want the program to be resizable but I don't want to > > allow the window width to fall below that of the tabs > > which would hide them. Since I can find no way to have > > more than one row of tabs, this appears to be my only > > option. Any suggestions would be appreciated. > > > > -- > > GNU/Linux user #557453 > > May the Source be with you. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > ---------- Forwarded message ---------- > From: Terry Reedy > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 02:00:25 -0400 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On 9/16/2017 7:04 PM, breamoreboy at gmail.com wrote: > >> I thought some might find this https://sites.google.com/view/ >> energy-efficiency-languages/ interesting. >> > > By 'energy', they only mean electricity, not food calories. This is the > email I sent to the authors. > ----------- > > As a two-decade user of Python, I was interested to read your paper. > Unfortunately, it is deeply flawed with respect to Python in the sense that > your conclusions are inapplicable to real-world usage of Python. > > The problem is your use of the Computer Language Benchmark Game. As the > name says, it is a *game*. As a game, it has artificial rules dictated by > the game masters. It uses toy problems, and for Python, the rules dictate > unrealistic toy solutions. In particular, it appears to disallow use of > 'import' with 3rd-party modules, whereas real-world Python is expected to > use them, and nearly always does. > > The particular crippler for CLBG problems is the non-use of numpy in > numerical calculations, such as the n-body problem. Numerical python > extensions are over two decades old and give Python code access to > optimized, compiled BLAS, LinPack, FFTPack, and so on. The current one, > numpy, is the third of the series. It is both a historical accident and a > continuing administrative convenience that numpy is not part of the Python > stdlib. However, it is easily installed from the PSF-maintained repository > (python -m pip install numpy), and is included with most third-party > distributions of Python. > > The numerical extensions have been quasi-official in the sense that at > least 3 language enhancements have been make for their use. Nearly all > real-world scientific, financial, and neural-network Python programs are > build on top of numpy. When a Python program spend 95% of the time in the > optimized compiled C routines, it is nearly as fast as a 100% C solution. > The reason people use Python instead of C for the other 5% is to save human > time. > > Even when it come to executing the pure Python solutions, the CLBG rules > apparently require the slowest execution possible. Execution would be at > least 2 to 5 times faster if compilation to machine code were allowed, > either before or during execution. But the point of the game is to provide > a 'level' playing field for competition between Python programmers, even if > the cost is to cripple comparison with other language solution. > > Terry Jan Reedy > > > > > -- > Terry Jan Reedy > > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 16:04:32 +1000 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On Sun, Sep 17, 2017 at 4:00 PM, Terry Reedy wrote: > > The numerical extensions have been quasi-official in the sense that at > least > > 3 language enhancements have been make for their use. > > I know about the matrix multiplication operator. What are the other > two (or more)? > > ChrisA > > > > ---------- Forwarded message ---------- > From: Terry Reedy > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 02:16:42 -0400 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On 9/17/2017 2:04 AM, Chris Angelico wrote: > >> On Sun, Sep 17, 2017 at 4:00 PM, Terry Reedy wrote: >> >>> The numerical extensions have been quasi-official in the sense that at >>> least >>> 3 language enhancements have been make for their use. >>> >> >> I know about the matrix multiplication operator. What are the other >> two (or more)? >> > > Stride slicing, which later became valid in regular code, and Ellipsis. (I > could be wrong on the latter.) > > -- > Terry Jan Reedy > > > > > ---------- Forwarded message ---------- > From: Gregory Ewing > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 18:19:59 +1200 > Subject: Re: Old Man Yells At Cloud > Chris Angelico wrote: > >> Or machine code on every CPU I've ever worked with. Dividing integers >> yields an integer. >> > > Every machine language I've used has different sets > of instructions for int and float arithmetic, so > there's no chance of confusion. > > -- > Greg > > > > ---------- Forwarded message ---------- > From: Gregory Ewing > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 18:27:37 +1200 > Subject: Re: Old Man Yells At Cloud > Paul Rubin wrote: > >> Python 2 does something reasonable >> > > I don't agree. It might be reasonable in a statically-typed > language, but in a dynamically-typed language where you're > encouraged to use ints as stand-ins for integer-valued floats, > it's an invitation for trouble. There are good reasons it was > changed in Python 3. > > -- > Greg > > > > ---------- Forwarded message ---------- > From: mm0fmf > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 07:56:39 +0100 > Subject: Re: Old Man Yells At Cloud > On 16/09/2017 17:52, Dennis Lee Bieber wrote: > >> On Sat, 16 Sep 2017 09:59:43 -0500, Tim Daneliuk >> declaimed the following: >> >> >> >> Well, the whole integer floor division thing years ago was the beginning >>> of the end - Python was doomed ... >>> >> >> Yes -- that would give me fits if I were using Python3 >> currently... >> Since so many of the languages I've learned over the past years use the >> concept integer / integer => integer_result >> >> Come to think of it -- wasn't there a post from someone (seems to >> expired or been filtered from my client) asking for help with some sorting >> program... I'm pretty sure the partitioning logic would croak on Python3 >> due to floating point results in the slice indices: >> >> pA = part[:n/2] >> pB = part[n/2:] >> >> > What does 2to3 do when fed code involving division? > > I've only used it once and did good job on the code I fed it. But it would > not have been too hard to manually convert the Python2 code in that case. > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 17:54:24 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 02:07 pm, Paul Rubin wrote: > > > Steve D'Aprano writes: > >>> concept integer / integer => integer_result > >> That would be C, and C derived languages, perhaps? > > > > Certainly not. Fortran, machine languages, etc. all do that too. > > > > Haskell does the right thing and makes int/int a compile time type > > error. > > Its not the right thing. Pascal did the right thing: int/int returns a > floating > point value. Python does the right thing. Pocket calculators do the right > thing. > > Haskell is great in many ways, but it is also obnoxiously pedantic about > type > safety. At the point that the compiler won't even automatically coerce > ints to > floats, but requires you to sign a release form that you take all > responsibility for it, er I mean manually do the coercion yourself, that's > excessive. > > There's no reason why they couldn't have allowed int/int to return a float > of > some sort, or a fraction if they want to be exact. In my arrogant opinion, > its > just obnoxious, pretentious purity-beyond-all-sense to make "int / int" an > error. Its not like division on integers is undefined, its just not > closed, but > lots of operations in Haskell aren't closed. The Haskell equivalent of > len(some_string) does not return a string. > > To even *know* that there are branches of maths where int/int isn't > defined, you > need to have learned aspects of mathematics that aren't even taught in most > undergrad maths degrees. (I have a degree in maths, and if we ever covered > areas where int/int was undefined, it was only briefly, and I've long since > forgotten it.) > > There's a perfectly good, almost intuitive, understanding of int/int > giving some > sort of rational value. Why don't they use it? They go so far as to > provide two > pairs of division/remainder functions, differing only in their treatment of > negative values: > > (x ?quot? y)?y + (x ?rem? y) == x > (x ?div? y)?y + (x ?mod? y) == x > > but can't provide an operator to do ordinary rational-valued division. > > > > Its integer division functions are div and quot. > > quot is "integer division truncated toward zero", div is "integer division > truncated toward negative infinity". > > > > > Python 2 does something reasonable > > Except that it isn't. It's surprising, and error prone, and was changed > because > it was a bug magnet. > > > > and Python 3 does something that is > > also debatably reasonable. Switching from one reasonable thing to > > another without a very good reason is called fixing things that weren't > > broken. Python 2 wasn't broken in that area and didn't need to be > > fixed. > > Python 2 was badly broken in this area. The / operator doing integer > division on > integers and true division on everything else was a constant source of > pain and > silent errors. You would write a perfectly reasonable maths expression: > > y = math.sin((2*x - 1)/(3*x + 1)) > > and test it with floats and it would work perfectly, then some day someone > slips > an integer into x and you silently get garbage output, which you would > probably > never even realise. > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 18:14:37 +1000 > Subject: Re: Research paper "Energy Efficiency across Programming > Languages: How does energy, time, and memory relate?" > On Sun, 17 Sep 2017 04:16 pm, Terry Reedy wrote: > > > On 9/17/2017 2:04 AM, Chris Angelico wrote: > >> On Sun, Sep 17, 2017 at 4:00 PM, Terry Reedy wrote: > >>> The numerical extensions have been quasi-official in the sense that at > least > >>> 3 language enhancements have been make for their use. > >> > >> I know about the matrix multiplication operator. What are the other > >> two (or more)? > > > > Stride slicing, which later became valid in regular code, and Ellipsis. > > (I could be wrong on the latter.) > > > Nope, both are correct. > > Slice strides were first supported in Python 1.4, but used exclusively by > Numerical Python (numpy's ancient predecessor), and didn't finally get > supported by Python builtins until as late as version 2.3! > > https://docs.python.org/2.3/whatsnew/section-slices.html > > Ellipsis was used for multi-dimensional slicing, and was added for > Numerical > Python. It wasn't until recently (Python 3.4 perhaps?) that it finally > became > legal to write '...' instead of 'Ellipsis' outside of slice notation. > > Here's Peter Otten talking about it way back in 2004: > > http://grokbase.com/t/python/python-list/042jd5y60n/ellipsis-usage > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: Amirouche Boubekki > To: Stefan Ram > Cc: python-list > Bcc: > Date: Sun, 17 Sep 2017 01:21:21 -0700 > Subject: Re: Which database system? > Le 15 sept. 2017 20:05, "Stefan Ram" a ?crit : > > When one is building an in-memory database that has a single > table that is built at the start of the program and then one > writes some complex queries to the table, what can be expected > to be faster: > > - implementing the table as a builtins.list of builtins.tuples > with builtins.dicts as indexes for faster lookups and > additional sorted builtins.lists for sorted "views" on the > table > > - implementing the table as a database table in sqlite3 > (":memory:") and using SQL commands for insertion > > > There is other solutions like shelve mentioned previously or plyvel (easy > api) or my preferred wiredtiger. But the issue with maintenance costs is > still valid. Choose the later if nothing else works. > > > > ---------- Forwarded message ---------- > From: Alain Ketterlin > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 10:37:24 +0200 > Subject: Typo-squatting PyPi > > In case you haven't heard about this: > > https://developers.slashdot.org/story/17/09/16/2030229/ > pythons-official-repository-included-10-malicious-typo-squatting-modules > > Here is the Slashdot summary: > > | The Slovak National Security Office (NBU) has identified ten malicious > | Python libraries uploaded on PyPI -- Python Package Index -- the > | official third-party software repository for the Python programming > | language. NBU experts say attackers used a technique known as > | typosquatting to upload Python libraries with names similar to > | legitimate packages -- e.g.: "urlib" instead of "urllib." The PyPI > | repository does not perform any types of security checks or audits > | when developers upload new libraries to its index, so attackers had no > | difficulty in uploading the modules online. > | > | Developers who mistyped the package name loaded the malicious > | libraries in their software's setup scripts. "These packages contain > | the exact same code as their upstream package thus their functionality > | is the same, but the installation script, setup.py, is modified to > | include a malicious (but relatively benign) code," NBU explained. > | Experts say the malicious code only collected information on infected > | hosts, such as name and version of the fake package, the username of > | the user who installed the package, and the user's computer hostname. > | Collected data, which looked like "Y:urllib-1.21.1 admin testmachine", > | was uploaded to a Chinese IP address. NBU officials contacted PyPI > | administrators last week who removed the packages before officials > | published a security advisory on Saturday." > > -- Alain. > > > > ---------- Forwarded message ---------- > From: Leam Hall > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 06:03:16 -0400 > Subject: Re: Old Man Yells At Cloud > Different view, I guess. I'm glad all the young Javascripters have that > issue. As an old guy trying to re-learn more python it gives me an > advantage. I'm usually interested in the best thislanguage-native way to do > something. Doing so makes me learn the language faster and tends to > generate better code. > > That said, I'll often steal what I've learned before to understand the > new. Some helpful folks on IRC asked why I was using getopt instead of > argparse. Mostly because I come from a bash background. Looking at Python's > argparse would have stumped me if I hadn't already done the same thing with > Ruby's argparse. > > I'm still trying to figure out how to convert a string to unicode in > Python 2. I've done it in Ruby 1.8.7 so I assume Python 2 can do it and > that I'm just a bit slow. > > Leam > > > > ---------- Forwarded message ---------- > From: Abdur-Rahmaan Janhangeer > To: "Steve D'Aprano" > Cc: python-list at python.org > Bcc: > Date: Sun, 17 Sep 2017 14:02:47 +0400 > Subject: Re: Old Man Yells At Cloud > as someone who really dislike js, i have to admit : python's globals are > really really bad ! > > js is a charm at that a real charm ! > > Abdur-Rahmaan Janhangeer, > Mauritius > abdurrahmaanjanhangeer.wordpress.com > > On 16 Sep 2017 09:40, "Steve D'Aprano" wrote: > > > /rant on > > > > So apparently everyone who disagrees that Python should be more like > > Javascript > > is an old greybeard fuddy-duddy yelling "Get off my lawn!" to the cool > > kids -- > > and is also too stupid to know how dumb they are. > > > > "Hi, I've been programming in Python for what seems like days now, and > > here's > > all the things that you guys are doing wrong. I insist that you fix them > > immediately, it doesn't matter how much code it will break, that's not > > important. What is important is that Javascript programmers like me > > shouldn't > > be expected to learn anything new or different when they program with > > Python." > > > > /rant off > > > > And no, for once it wasn't Ranting Rick. > > > > > > > > > > -- > > Steve > > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > > enough, things got worse. > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > ---------- Forwarded message ---------- > From: pizza python > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 12:11:44 +0200 > Subject: speech_to_text python command not working > Hello all. > > > I'm on Linux Mint 18.2 Cinnamon 64-bit. > > I am trying to get IBM Watson BlueMix Speech-To-Text to transcribe my > spoken-word audio files. Because I'm not a coder, I tried to find the > simplest way to use BlueMix Speech-to-Text. And what I found > is [1]https://github.com/rmotr/speech-to-text > > It's in PyPi Libary: https://pypi.python.org/pypi/speech-to-text/0.0.1 > > Step 1: "pip install speech-to-text". I ran it with sudo to make it work. > > > > > Step 2: I then ran: speech_to_text -u myUserNameGoesHere -p > myPasswordGoesHere > -f html -i voice2.ogg transcript.html Starting Upload. > [=========================================================== > ==============] > 100% Upload finished. Waiting for Transcript Traceback (most recent call > last): > File "/usr/local/bin/speech_to_text", line 11, in > sys.exit(speech_to_text()) File > "/usr/local/lib/python2.7/dist-packages/click/core.py", line 722, in > __call__ > return self.main(*args, **kwargs) File > "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in > main rv = > self.invoke(ctx) File "/usr/local/lib/python2.7/ > dist-packages/click/core.py", > line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File > "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in > invoke > return callback(*args, **kwargs) File > "/usr/local/lib/python2.7/dist-packages/speech_to_text/command.py", line > 60, in > speech_to_text formatted_output = FormatterClass().format(result) File > "/usr/local/lib/python2.7/dist-packages/speech_to_text/formatters.py", > line 36, > in format for obj in self._parse(data)) File > "/usr/local/lib/python2.7/dist-packages/speech_to_text/formatters.py", > line 10, > in _parse for obj in data[`results']) KeyError: `results' > > > > __________ > > I was expecting an html with the transcript. So why did I get the errors > above? > > > > > > > > python > Python 2.7.12 (default, Nov 19 2016, 06:48:10) > [GCC 5.4.0 20160609] on linux2 > > __________________________________ > > dpkg -l python > Desired=Unknown/Install/Remove/Purge/Hold > | > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/ > trig-aWait/Trig-pend > |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > ||/ Name Version Architecture Description > +++-==============-============-============-=============== > ================== > ii python 2.7.11-1 amd64 interactive high-level > object-ori > ____________________________________ > > python3 > Python 3.5.2 (default, Nov 17 2016, 17:05:23) > [GCC 5.4.0 20160609] on linux > > ____________________________________ > > dpkg -l python3 > Desired=Unknown/Install/Remove/Purge/Hold > | > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/ > trig-aWait/Trig-pend > |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > ||/ Name Version Architecture Description > +++-==============-============-============-=============== > ================== > ii python3 3.5.1-3 amd64 interactive high-level > object-ori > > References > > Visible links > 1. https://github.com/rmotr/speech-to-text > > > > ---------- Forwarded message ---------- > From: Leam Hall > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 06:22:34 -0400 > Subject: Re: Old Man Yells At Cloud > Hmm... scratch the "young" and "Javascripters". Why lump them together > since I bet it's just a vocal few? Better to have said "people who don't > want to really learn the new language". > > On 09/17/2017 06:03 AM, Leam Hall wrote: > >> Different view, I guess. I'm glad all the young Javascripters have that >> issue. As an old guy trying to re-learn more python it gives me an >> advantage. I'm usually interested in the best thislanguage-native way to do >> something. Doing so makes me learn the language faster and tends to >> generate better code. >> >> That said, I'll often steal what I've learned before to understand the >> new. Some helpful folks on IRC asked why I was using getopt instead of >> argparse. Mostly because I come from a bash background. Looking at Python's >> argparse would have stumped me if I hadn't already done the same thing with >> Ruby's argparse. >> >> I'm still trying to figure out how to convert a string to unicode in >> Python 2. I've done it in Ruby 1.8.7 so I assume Python 2 can do it and >> that I'm just a bit slow. >> >> Leam >> > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 20:43:02 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, Sep 17, 2017 at 5:54 PM, Steve D'Aprano > wrote: > > To even *know* that there are branches of maths where int/int isn't > defined, you > > need to have learned aspects of mathematics that aren't even taught in > most > > undergrad maths degrees. (I have a degree in maths, and if we ever > covered > > areas where int/int was undefined, it was only briefly, and I've long > since > > forgotten it.) > > How about this: > > >>> (1<<10000)/2 > Traceback (most recent call last): > File "", line 1, in > OverflowError: integer division result too large for a float > > int/int is now undefined. In Py2, it perfectly correctly returns > another integer (technically a long), but in Py3, it can't return a > float, so it errors out. This is nothing to do with the mathematical > notion of a "real", which is a superset of the mathematical notion of > an "integer"; it's all to do with the Python notion of a "float", > which is NOT a superset of the Python notion of an "integer". > > In Python 2, an ASCII string could be implicitly promoted to a Unicode > string: > > >>> user_input = u"Real Soon Now?" > >>> print("> " + user_input + " <") > > Real Soon Now? < > > In Python 2 and 3, a small integer can be implicitly promoted to float: > > >>> user_input = 3.14159 > >>> print(user_input + 1) > 4.14159 > > Both conversions can cause data-dependent failures when used with > arbitrary input, but are unlikely to cause problems when you're > promoting literals. Both conversions require proximity to the other > type. As long as you're explicit about the data type used for user > input, you can short-hand your literals and get away with it: > > >>> # more likely, input came as text > >>> user_input = float("1.234") > >>> print(user_input + 1) > 2.234 > >>> # and hey, it works with other types too! > >>> user_input = decimal.Decimal("1.234") > >>> print(user_input + 1) > 2.234 > >>> user_input = fractions.Fraction("1.234") > >>> print(user_input + 1) > 1117/500 > > The trouble only comes when you take two pieces of user input in > different types, and try to combine them: > > >>> user_1 = float("1.234") > >>> user_2 = int("9"*999) # imagine someone typed it > >>> user_1 + user_2 > Traceback (most recent call last): > File "", line 1, in > OverflowError: int too large to convert to float > > Solution? Always use the right data types for user input. Easy enough. > > Python 3 introduces a completely different way to get failure, though. > You can be 100% consistent with your data types, but then get > data-dependent failures if, and only if, you divide. (Technically, not > completely new in Py3; you can get this in Py2 with exponentiation - > "2**-1" will yield a float. Far less likely to be hit, but could > potentially cause the same problems.) I don't know of any exploits > that involve this, but I can imagine that you could attack a Python > script by forcing it to go floating-point, then either crashing it > with a huge integer, or exploiting round-off, depending on whether the > program is assuming floats or assuming ints. > > Python 3 *removed* one of these data-dependent distinctions, by making > bytes+text into an error: > > >>> b"asdf" + u"qwer" > u'asdfqwer' > > >>> b"asdf" + u"qwer" > Traceback (most recent call last): > File "", line 1, in > TypeError: can't concat str to bytes > > But it added a different one, by allowing a common and normal > operation to change a data type. Is it better to make things > convenient for the case of small integers (the ones that are perfectly > representable as floats), while potentially able to have problems on > larger ones? Considering how large a "small integer" can be, most > programmers won't think to test for overflow - just as many > programmers won't test non-ASCII data. Thanks to Python 3, the > "non-ASCII data" one isn't a problem, because you'll get the same > exception with ASCII data as with any other; but the "small integer" > one now is. > > Data-dependent type errors don't seem like a smart thing to me. > > ChrisA > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 20:54:51 +1000 > Subject: Python built-ins versus Javascript [was Re: Old Man Yells At > Cloud] > On Sun, 17 Sep 2017 08:02 pm, Abdur-Rahmaan Janhangeer wrote: > > > as someone who really dislike js, i have to admit : python's globals are > > really really bad ! > > > > js is a charm at that a real charm ! > > Can you explain what you think is so bad about them, and why Javascript's > are > better? > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 21:25:21 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 08:03 pm, Leam Hall wrote: > > > I'm still trying to figure out how to convert a string to unicode in > > Python 2. > > > A Python 2 string is a string of bytes, so you need to know what encoding > they > are in. Let's assume you got them from a source using UTF-8. Then you > would do: > > mystring.decode('utf-8') > > and it will return a Unicode string of "code points" (think: more or less > characters). > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: Leam Hall > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 07:38:35 -0400 > Subject: Unicode (was: Old Man Yells At Cloud) > On 09/17/2017 07:25 AM, Steve D'Aprano wrote: > >> On Sun, 17 Sep 2017 08:03 pm, Leam Hall wrote: >> >> I'm still trying to figure out how to convert a string to unicode in >>> Python 2. >>> >> >> >> A Python 2 string is a string of bytes, so you need to know what encoding >> they >> are in. Let's assume you got them from a source using UTF-8. Then you >> would do: >> >> mystring.decode('utf-8') >> >> and it will return a Unicode string of "code points" (think: more or less >> characters). >> > > > Still trying to keep this Py2 and Py3 compatible. > > The Py2 error is: > UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' > in position 8: ordinal not in range(128) > > even when the string is manually converted: > name = unicode(self.name) > > Same sort of issue with: > name = self.name.decode('utf-8') > > > Py3 doesn't like either version. > > > > > > ---------- Forwarded message ---------- > From: Paul Moore > To: Leam Hall > Cc: "python-list at python.org" > Bcc: > Date: Sun, 17 Sep 2017 12:51:21 +0100 > Subject: Re: Unicode (was: Old Man Yells At Cloud) > On 17 September 2017 at 12:38, Leam Hall wrote: > > On 09/17/2017 07:25 AM, Steve D'Aprano wrote: > >> > >> On Sun, 17 Sep 2017 08:03 pm, Leam Hall wrote: > >> > >>> I'm still trying to figure out how to convert a string to unicode in > >>> Python 2. > >> > >> > >> > >> A Python 2 string is a string of bytes, so you need to know what > encoding > >> they > >> are in. Let's assume you got them from a source using UTF-8. Then you > >> would do: > >> > >> mystring.decode('utf-8') > >> > >> and it will return a Unicode string of "code points" (think: more or > less > >> characters). > > > > > > > > Still trying to keep this Py2 and Py3 compatible. > > > > The Py2 error is: > > UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' > > in position 8: ordinal not in range(128) > > > > even when the string is manually converted: > > name = unicode(self.name) > > > > Same sort of issue with: > > name = self.name.decode('utf-8') > > > > > > Py3 doesn't like either version. > > Your string is likely not UTF-8 with a character \xf6 in it. Maybe > it's latin-1? The key here is there's no way for Python (or any > program) to know the encoding of the byte string, so you have to tell > it. > > Paul > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 22:30:26 +1000 > Subject: Re: Unicode (was: Old Man Yells At Cloud) > On Sun, Sep 17, 2017 at 9:38 PM, Leam Hall wrote: > > Still trying to keep this Py2 and Py3 compatible. > > > > The Py2 error is: > > UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' > > in position 8: ordinal not in range(128) > > > > even when the string is manually converted: > > name = unicode(self.name) > > > > Same sort of issue with: > > name = self.name.decode('utf-8') > > > > > > Py3 doesn't like either version. > > You got a Unicode *EN*code error when you tried to *DE* code. That's a > quirk of Py2's coercion behaviours, so the error's a bit obscure, but > it means that you (most likely) actually have a Unicode string > already. Check what type(self.name) is, and see if the problem is > actually somewhere else. > > (It's hard to give more specific advice based on this tiny snippet, sorry.) > > ChrisA > > > > ---------- Forwarded message ---------- > From: Leam Hall > To: "python-list at python.org" > Cc: > Bcc: > Date: Sun, 17 Sep 2017 08:44:24 -0400 > Subject: Re: Unicode > On 09/17/2017 08:30 AM, Chris Angelico wrote: > >> On Sun, Sep 17, 2017 at 9:38 PM, Leam Hall wrote: >> >>> Still trying to keep this Py2 and Py3 compatible. >>> >>> The Py2 error is: >>> UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' >>> in position 8: ordinal not in range(128) >>> >>> even when the string is manually converted: >>> name = unicode(self.name) >>> >>> Same sort of issue with: >>> name = self.name.decode('utf-8') >>> >>> >>> Py3 doesn't like either version. >>> >> >> You got a Unicode *EN*code error when you tried to *DE* code. That's a >> quirk of Py2's coercion behaviours, so the error's a bit obscure, but >> it means that you (most likely) actually have a Unicode string >> already. Check what type(self.name) is, and see if the problem is >> actually somewhere else. >> >> (It's hard to give more specific advice based on this tiny snippet, >> sorry.) >> >> ChrisA >> >> > Chris, thanks! I see what you mean. > > The string source is a SQLite3 database with a bunch of names. Some have > non-ASCII characters. The database is using varchar which seems to be > utf-8, utf-16be or utf-16le. I probably need to purge the data. > > What I find interesting is that utf-8 works in the Ruby script that pulls > from the same database. That's what makes me think it's utf-8. > > I've tried different things in lines 45 and 61. > > https://gist.github.com/LeamHall/054f9915af17dc1b1a33597b9b45d2da > > Leam > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 23:03:36 +1000 > Subject: Re: Old Man Yells At Cloud > On Sun, 17 Sep 2017 08:43 pm, Chris Angelico wrote: > > > On Sun, Sep 17, 2017 at 5:54 PM, Steve D'Aprano > > wrote: > >> To even *know* that there are branches of maths where int/int isn't > defined, > >> you need to have learned aspects of mathematics that aren't even taught > in > >> most undergrad maths degrees. (I have a degree in maths, and if we ever > >> covered areas where int/int was undefined, it was only briefly, and > I've long > >> since forgotten it.) > > > > How about this: > > > >>>> (1<<10000)/2 > > Traceback (most recent call last): > > File "", line 1, in > > OverflowError: integer division result too large for a float > > > > int/int is now undefined. > > No, it's perfectly defined: you get an overflow error if the arguments are > too > big to convert, or an underflow error if the denominator is too small, or a > divide by zero error if you divide by zero... > > > What do you make of this? > > py> float(1<<10000)/2.0 > Traceback (most recent call last): > File "", line 1, in > OverflowError: int too large to convert to float > > > Would you like to argue that this shows that coercing ints to floats > is "undefined"? > > > Overflow and underflow errors are limitations of the float data type. We > could > fix that in a couple of ways: > > - silently underflow to zero (as Python already does!) or infinity, as > needed; > > - use a bigger ~~boat~~ float; > > - or even an arbitrary precision float; > > - or return a rational number (fraction or similar); > > - or introduce a float context that allows you to specify the behaviour > that you want, as the decimal module does. > > There may be other solutions I haven't thought of. But these will do. > > The distinction between Python floats and real numbers ? is a red-herring. > It > isn't relevant. > > > > In Py2, it perfectly correctly returns > > another integer (technically a long), but in Py3, it can't return a > > float, so it errors out. > > Apart from your "correctly", which I disagree with, that's a reasonable > description. The problem is that your example returns the correct result by > accident. Forget such ludicrously large values, and try something more > common: > > 1/2 > > Most people aren't expecting integer division, but true division, and > silently > returning the wrong result (0 instead of 0.5) is a silent source of bugs. > > This isn't some theoretical problem that might, maybe, perhaps, be an > issue for > some people sometimes. It was a regular source of actual bugs leading to > code > silently returning garbage. > > > > This is nothing to do with the mathematical > > notion of a "real", > > I don't believe I ever mentioned Reals. I was pretty careful not to. > > > > which is a superset of the mathematical notion of > > an "integer"; it's all to do with the Python notion of a "float", > > which is NOT a superset of the Python notion of an "integer". > > So? Operations don't *have* to return values from their operands' type. > > len('abc') doesn't return a string. > > alist.find(1) doesn't have to return either a list or an int. > > And 1/2 doesn't have to return an int. Why is this such a big deal? > > > > In Python 2, an ASCII string could be implicitly promoted to a Unicode > string: > > > >>>> user_input = u"Real Soon Now?" > >>>> print("> " + user_input + " <") > >> Real Soon Now? < > > And that was a bug magnet, like using / for integer division sometimes and > true > division other times was a big magnet. So Python 3 got rid of both bad > design > decisions. > > > > In Python 2 and 3, a small integer can be implicitly promoted to float: > > > >>>> user_input = 3.14159 > >>>> print(user_input + 1) > > 4.14159 > > Yes, as it should. Why force the user to call float() on one argument when > the > interpreter can do it? What advantage is there? > > Can you demonstrate any failure of dividing two ints n/m which wouldn't > equally > fail if you called float(n)/float(m)? I don't believe that there is any > such > failure mode. Forcing the user to manually coerce to floats doesn't add any > protection. > > > > Both conversions can cause data-dependent failures when used with > > arbitrary input, > > There's a difference: > > - with automatic promotion of bytes to Unicode, you get errors that > pass silently and garbage results; > > - with automatic promotion of bytes to Unicode, you get errors that > pass silently and garbage results; > > - but with true division, if int/int cannot be performed using > floats, you get an explicit error. > > > Silently returning the wrong result was a very common consequence of the > int/int > behaviour in Python 2. Is there any evidence of common, real-world bugs > caused > by true division? > > Beginners who make assumptions that Python is C (or any other language) and > use / when they should use // don't count: that's no different from > somebody > using ^ for exponentiation. > > > [...] > > The trouble only comes when you take two pieces of user input in > > different types, and try to combine them: > > > >>>> user_1 = float("1.234") > >>>> user_2 = int("9"*999) # imagine someone typed it > >>>> user_1 + user_2 > > Traceback (most recent call last): > > File "", line 1, in > > OverflowError: int too large to convert to float > > I'm sorry, I fail to see why you think this is "trouble". It's just normal > Python behaviour in the face of errors: raise an exception. If you pass a > bad > value, you get an exception of some kind. > > Are these "trouble" too? > > py> ''[5] > Traceback (most recent call last): > File "", line 1, in > IndexError: string index out of range > > py> int('xyz') > Traceback (most recent call last): > File "", line 1, in > ValueError: invalid literal for int() with base 10: 'xyz' > > > Getting an explicit exception on error is the right thing to do. Silently > returning garbage is not. > > If you want to argue that int/int should return infinity, or a NAN, on > overflow, > that's possibly defensible. But arguing that somehow the division operator > is > uniquely or specifically "trouble" because it raises an exception when > given > bad data, well, that's just weird. > > > > Python 3 introduces a completely different way to get failure, though. > > You can be 100% consistent with your data types, but then get > > data-dependent failures if, and only if, you divide. > > Its true that most operations on integers will succeed. But not all. > > Try (1<<10000)**(1<<10000) if you really think that integer ops are > guaranteed > to succeed. (I'm scared to try it myself, because I've had bad experiences > in > the past with unreasonably large ints.) > > But then, what of it? All that means is that division can fail. But even > integer > division can fail: > > py> 1//0 > Traceback (most recent call last): > File "", line 1, in > ZeroDivisionError: integer division or modulo by zero > > > [...] > > I don't know of any exploits > > that involve this, but I can imagine that you could attack a Python > > script by forcing it to go floating-point, then either crashing it > > with a huge integer, or exploiting round-off, depending on whether the > > program is assuming floats or assuming ints. > > You're not seriously arguing that true division is a security > vulnerability? > > In any case, the error here is an exception, not silent failures. > > "I find it amusing when novice programmers believe their > main job is preventing programs from crashing. ... More > experienced programmers realize that correct code is > great, code that crashes could use improvement, but > incorrect code that doesn?t crash is a horrible nightmare." > -- Chris Smith > > > Using / for integer division, if and only if both arguments are integers, > was > exactly that horrible nightmare. > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: Peter Otten <__peter__ at web.de> > To: python-list at python.org > Cc: > Bcc: > Date: Sun, 17 Sep 2017 15:13:09 +0200 > Subject: Re: Unicode > Leam Hall wrote: > > > On 09/17/2017 08:30 AM, Chris Angelico wrote: > >> On Sun, Sep 17, 2017 at 9:38 PM, Leam Hall wrote: > >>> Still trying to keep this Py2 and Py3 compatible. > >>> > >>> The Py2 error is: > >>> UnicodeEncodeError: 'ascii' codec can't encode character > >>> u'\xf6' in position 8: ordinal not in range(128) > >>> > >>> even when the string is manually converted: > >>> name = unicode(self.name) > >>> > >>> Same sort of issue with: > >>> name = self.name.decode('utf-8') > >>> > >>> > >>> Py3 doesn't like either version. > >> > >> You got a Unicode *EN*code error when you tried to *DE* code. That's a > >> quirk of Py2's coercion behaviours, so the error's a bit obscure, but > >> it means that you (most likely) actually have a Unicode string > >> already. Check what type(self.name) is, and see if the problem is > >> actually somewhere else. > >> > >> (It's hard to give more specific advice based on this tiny snippet, > >> sorry.) > >> > >> ChrisA > >> > > > > Chris, thanks! I see what you mean. > > I don't think so. You get a unicode from the database, > > $ python > Python 2.7.6 (default, Oct 26 2016, 20:30:19) > [GCC 4.8.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sqlite3 > >>> db = sqlite3.connect(":memory:") > >>> cs = db.cursor() > >>> cs.execute("select 'foo';").fetchone() > (u'foo',) > >>> > > and when you try to decode it (which is superfluous as you already have > unicode!) Python does what you ask for. But to be able to decode it has to > encode first and by default it uses the ascii codec for that attempt. For > an > all-ascii string > > u"foo".encode("ascii") --> "foo" > > and thus > > u"foo".decode("utf-8) > > implemented as > > u"foo".encode("ascii").decode("utf-8") --> u"foo" > > is basically a noop. However > > u"???".encode("ascii") --> raises UnicodeENCODEError > > and thus > > u"???".decode("utf-8") > > fails with that. Unfortunately nobody realizes that the encoding failed and > thus will unsuccessfully try and specify other encodings for the decoding > step > > u"???".decode("latin1") # also fails > > Solution: if you already have unicode, leave it alone. > > > The string source is a SQLite3 database with a bunch of names. Some have > > non-ASCII characters. The database is using varchar which seems to be > > utf-8, utf-16be or utf-16le. I probably need to purge the data. > > > > What I find interesting is that utf-8 works in the Ruby script that > > pulls from the same database. That's what makes me think it's utf-8. > > > > I've tried different things in lines 45 and 61. > > > > https://gist.github.com/LeamHall/054f9915af17dc1b1a33597b9b45d2da > > > > Leam > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > > ... > > [Message clipped] -- *Joseph * From ian.g.kelly at gmail.com Mon Oct 2 18:48:16 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 2 Oct 2017 16:48:16 -0600 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: On Mon, Oct 2, 2017 at 1:32 PM, Bill wrote: > @property def circumference(self): > return 2 * math.pi *self.radius Of course the *proper* formula here is just math.tau * self.radius. From steve+python at pearwood.info Mon Oct 2 20:00:24 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 03 Oct 2017 11:00:24 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d2d31b$0$14962$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 06:32 am, Bill wrote: > Steve D'Aprano wrote: >> Circle didn't use any setters, but I could have let you set the >> diameter, which in >> turn would set the radius: >> >> circle.radius = 2 >> assert circle.diameter == 4 >> circle.diameter == 2 # requires a setter >> assert circle.radius == 1 >> >> Getting that to work is left as an exercise :-) >> > It WAS a good exercise!! I was concerned about "infinite recursion" > between my two property setters.. Thanks! Next? :) > > Bill > > > import math > > > class Circle(object): > """ Define a circle class with radius and diameter""" > def __init__(self, radius): > self.radius = radius > self.diameter =2*radius There's no need to set the radius and the diameter, as one is completely derived from the other and the transformation is cheap enough to perform on the fly as needed. Consider what happens if, due to a bug or an accident, you end up with a Circle instance that says the radius is 5 and the diameter is 20. They can't *both* be right, and you will get inconsistent results depending on whether your other methods happen to use the diameter or the radius. Instead, we should steal the Single Source Of Truth principle from informations systems theory: https://en.wikipedia.org/wiki/Single_source_of_truth https://en.wikipedia.org/wiki/Single_version_of_the_truth There are valid cases for violating this principle in object design, e.g. caches are technically a violation of SSOT but an acceptable one. With that in mind, your class simplifies to: class Circle(object): """ Define a circle class with radius and diameter""" def __init__(self, radius): self.radius = radius @property def radius(self): return self._radius @radius.setter def radius(self, value): self._radius = value @property def diameter(self): return 2*self._radius @diameter.setter def diameter(self, value): self._radius = value/2 @property def area(self): return math.pi*self.radius**2 @property def circumference(self): return math.pi*self.diameter -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From BILL_NOSPAM at whoknows.net Mon Oct 2 21:07:22 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Mon, 2 Oct 2017 21:07:22 -0400 Subject: newb question about @property In-Reply-To: <59d2d31b$0$14962$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d2d31b$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > > There's no need to set the radius and the diameter, as one is completely derived > from the other Good point; I'm glad I submitted my code for grading. Sort of a "trick question" to ask me to add diameter and then take off points because I used it! ; ) Bill > and the transformation is cheap enough to perform on the fly as > needed. > > Consider what happens if, due to a bug or an accident, you end up with a Circle > instance that says the radius is 5 and the diameter is 20. They can't *both* be > right, and From mosama221122 at gmail.com Mon Oct 2 21:54:45 2017 From: mosama221122 at gmail.com (mosama221122 at gmail.com) Date: Mon, 2 Oct 2017 18:54:45 -0700 (PDT) Subject: INSTRUCTOR SOLUTIONS MANUAL Linear Systems And Signals 2nd ED by B P Lathi In-Reply-To: References: Message-ID: <226f60a4-4cce-4437-b4f8-839055cccf3a@googlegroups.com> i need instructor of second edition of linear system and signals by lathi From BILL_NOSPAM at whoknows.net Mon Oct 2 23:39:43 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Mon, 2 Oct 2017 23:39:43 -0400 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Chris Angelico wrote: > Decorators are fairly straight-forward if you understand higher-order > functions. > > ChrisA I was just minding my own business, and thought to write my first decorator for a simple *recursive* function f. The decorator WORKS if f does not make a call to itself. Otherwise, f seems to have "difficulty" calling itself (I get a typerror, f(n) has value "NoneType"). What is the explanation for this? Does f have a new name because it has a decorator on it now? Note: I am not using functools.wraps since I don't yet understand the reason I might do that yet (first things first... ). Thanks! Bill From rosuav at gmail.com Mon Oct 2 23:48:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 3 Oct 2017 14:48:39 +1100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 3, 2017 at 2:39 PM, Bill wrote: > Chris Angelico wrote: >> >> Decorators are fairly straight-forward if you understand higher-order >> functions. >> >> ChrisA > > > > I was just minding my own business, and thought to write my first decorator > for a simple *recursive* function f. The decorator WORKS if f does not make > a call to itself. Otherwise, f seems to have "difficulty" calling itself > (I get a typerror, f(n) has value "NoneType"). What is the explanation for > this? Does f have a new name because it has a decorator on it now? > > Note: I am not using functools.wraps since I don't yet understand the reason > I might do that yet (first things first... ). I would recommend posting your code, possibly in a new thread. ChrisA From steve+python at pearwood.info Tue Oct 3 01:41:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 03 Oct 2017 16:41:38 +1100 Subject: INSTRUCTOR SOLUTIONS MANUAL Linear Systems And Signals 2nd ED by B P Lathi References: <226f60a4-4cce-4437-b4f8-839055cccf3a@googlegroups.com> Message-ID: <59d32314$0$14938$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 12:54 pm, mosama221122 at gmail.com wrote: > i need instructor of second edition of linear system and signals by lathi Don't reply to spammers and don't reply to them. They are the scum of the earth. They are literally criminals, they use computer viruses and malware to hijack people's computers to send their spam, and you want to trust them and buy from them? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From BILL_NOSPAM at whoknows.net Tue Oct 3 02:01:28 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Tue, 3 Oct 2017 02:01:28 -0400 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> Message-ID: Bill wrote: > Chris Angelico wrote: >> Decorators are fairly straight-forward if you understand higher-order >> functions. >> >> ChrisA > > > I was just minding my own business, and thought to write my first > decorator for a simple *recursive* function f. The decorator WORKS if > f does not make a call to itself > (it really wasn't). Using the (PyCharm) debugger, I determined that my inner function that was calling my wrapped (Fibonacci sequence) function but wasn't returning anything to the invoking environment. I fixed it for the sake of a good, if costly, lesson. Not much of a wrapper, but it "works". A version which merely prints a tab before the function call, instead of a row of asterisks produces output which is more interesting to look at. def wrap(func): def inner(*args, **kwargs): print('*'*20) a= func(*args, **kwargs) print(a) print('*'*20) return a return inner Bill From ranasaad935 at gmail.com Tue Oct 3 02:13:55 2017 From: ranasaad935 at gmail.com (ranasaad935 at gmail.com) Date: Mon, 2 Oct 2017 23:13:55 -0700 (PDT) Subject: Solution Manuals and Testbanks 2017-2018 In-Reply-To: References: Message-ID: <1374ea87-329a-477a-9da5-53a22c53e427@googlegroups.com> Do u have testbanks for ??? THE LAW OF WORK: INDUSTRIAL RELATIONS AND COLLECTIVE BARGAINING Author(s): David J. Doorey ISBN: 978-1-77255-166-2 Publisher:Emond Publishing From alister.ware at ntlworld.com Tue Oct 3 05:17:50 2017 From: alister.ware at ntlworld.com (alister) Date: Tue, 03 Oct 2017 09:17:50 GMT Subject: Spam References: <226f60a4-4cce-4437-b4f8-839055cccf3a@googlegroups.com> <59d32314$0$14938$b1db1813$d948b532@news.astraweb.com> Message-ID: <2BIAB.1155567$HS1.909036@fx12.am4> > They are literally criminals, they use computer viruses and malware to > hijack people's computers to send their spam, and you want to trust them > and buy from them? this was probably a "Drive By" posy to get the original spam more attention & possibly bypass spam filters -- Come live with me, and be my love, And we will some new pleasures prove Of golden sands, and crystal brooks, With silken lines, and silver hooks. -- John Donne From steve+python at pearwood.info Tue Oct 3 06:41:55 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 03 Oct 2017 21:41:55 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 06:51 am, Bill wrote: > Can you inspire me with a good decorator problem (standard homework > exercise-level will be fine)? Here is a nice even dozen problems for you. Please ask for clarification if any are unclear. (1) Write a decorator which simply prints a descriptive message and the name of the decorated function once, when the function is first decorated. E.g. if you write: @decorate def spam(x): return x + 1 # for example print(spam(1)) print(spam(2)) Python should print: Decorating function spam. 2 3 Note: "spam" must not be hard-coded, it must be taken from the function being decorated. (Hint: all functions have their name available as func.__name__.) (2) Modify the decorator from (1) so that calling the wrapped function also print a descriptive message such as "Calling function spam". The expected output will be: Decorating function spam. Calling function spam. 2 Calling function spam. 3 (3) Write a decorator that checks that the decorated function's first argument is a non-empty string, raising an appropriate exception if it is not, and lets through any other arguments unchanged. (4) Same as above, except the first argument is automatically stripped of leading and trailing whitespace and forced to uppercase. (5) Write a decorator which injects the argument 10 into the list of arguments received by the wrapped function. E.g. if you write: @inject def add(a, b): return a + b @inject def sub(a, b): return a - b print(add(5), sub(5)) Python should print "15 5". (And *not* "15 -5".) (6) [ADVANCED] Modify the decorator in (5) so that it takes an argument telling it what value to inject into the list of arguments: @inject(99) def sub(a, b): return a - b print(sub(5)) will now print "94". (7) Write a decorator which checks the decorated function's two arguments are given smallest first, swapping them around if needed. (8) Write a decorator which prints the name of the wrapped function, its arguments, and the time, each time the wrapped function is called. (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying the path to a file, and use the logging module to log the details to that file instead of printing them. (10) Write a decorator which adds an "cache" attribute initialised to an empty dictionary to the decorated function. (11) Write a decorator which wraps a class (not function!), and adds a "help" method to the class which prints a message as shown below. For example: @addhelp class Spam: pass @addhelp class Eggs: pass x = Spam() x.help() y = Eggs() y.help() will print: See http://example.com/Spam See http://example.com/Eggs (Hint: classes also have a __name__ attribute.) (12) [ADVANCED] Write a decorator which wraps a class, and applies the decorator from (10) above to each non-dunder? method in the class. That is, after: @addcaches class MyClass: def foo(self): pass def bar(self): pass print(MyClass.foo.cache, MyClass.bar.cache) should print "{} {}". ? Remember that dunder methods are those that start with two leading and trailing underscores: "Double UNDERscore" methods. * * * Bruce Eckel has an excellent introduction to Python decorators, from way back when they were first introduced in 2008. His introduction is notable because: - he points out explicitly that Python decorators are not the same as the Decorator design pattern (I thought they were!); - he recommends using a class as the decorator, and building the extra functionality in object oriented fashion, rather than functional programming fashion (this may give an easier introduction to those who aren't familiar with functional idioms); - and he correctly predicted that the introduction of the @ syntactic sugar would have a big impact on the way people think about Python code. http://www.artima.com/weblogs/viewpost.jsp?thread=240808 Feel free to read his post before trying the problems I set. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From 88onlinepoker88 at gmail.com Tue Oct 3 06:56:47 2017 From: 88onlinepoker88 at gmail.com (fuad) Date: Tue, 3 Oct 2017 03:56:47 -0700 (PDT) Subject: INSTRUCTOR SOLUTIONS MANUAL Elementary Linear Algebra by Kuttler In-Reply-To: References: Message-ID: <20bdafc3-cc72-44a4-b622-6f8ce25abb22@googlegroups.com> Engineering Fluid Mechanics, 10th Edition by Crowe, Elger solution fuadnassar99 at gmail.com From lele at metapensiero.it Tue Oct 3 07:01:58 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Tue, 03 Oct 2017 13:01:58 +0200 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: <878tgs8pg9.fsf@metapensiero.it> Steve D'Aprano writes: > (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying the > path to a file, and use the logging module to log the details to that file > instead of printing them. This may suffer of excessive creativity, as usually the details of *where* a logger writes the messages are better left to a configuration done at another level :) ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From steve+python at pearwood.info Tue Oct 3 07:50:11 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 03 Oct 2017 22:50:11 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <878tgs8pg9.fsf@metapensiero.it> Message-ID: <59d37976$0$14935$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 10:01 pm, Lele Gaifax wrote: > Steve D'Aprano writes: > >> (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying >> the path to a file, and use the logging module to log the details to that >> file instead of printing them. > > This may suffer of excessive creativity, as usually the details of *where* a > logger writes the messages are better left to a configuration done at another > level :) Right. I didn't say the path has to be hard-coded in the source. @addlogging(config.logfile or default_logfile) def function(x, y, z): ... In production code, I'd probably pass a logger instance rather than a file name. But in any case, its only a programming exercise, not meant to be production-ready code. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ian.g.kelly at gmail.com Tue Oct 3 10:39:01 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 3 Oct 2017 08:39:01 -0600 Subject: newb question about @property In-Reply-To: <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 3, 2017 at 4:41 AM, Steve D'Aprano wrote: > On Tue, 3 Oct 2017 06:51 am, Bill wrote: > >> Can you inspire me with a good decorator problem (standard homework >> exercise-level will be fine)? > > > Here is a nice even dozen problems for you. Please ask for clarification if any > are unclear. > > > > (1) Write a decorator which simply prints a descriptive message and the name of > the decorated function once, when the function is first decorated. > > E.g. if you write: > > @decorate > def spam(x): > return x + 1 # for example > > print(spam(1)) > print(spam(2)) > > > Python should print: > > Decorating function spam. > 2 > 3 > > > Note: "spam" must not be hard-coded, it must be taken from the function being > decorated. (Hint: all functions have their name available as func.__name__.) > > > (2) Modify the decorator from (1) so that calling the wrapped function also > print a descriptive message such as "Calling function spam". The expected > output will be: > > Decorating function spam. > Calling function spam. > 2 > Calling function spam. > 3 > > > (3) Write a decorator that checks that the decorated function's first argument > is a non-empty string, raising an appropriate exception if it is not, and lets > through any other arguments unchanged. > > > (4) Same as above, except the first argument is automatically stripped of > leading and trailing whitespace and forced to uppercase. > > > (5) Write a decorator which injects the argument 10 into the list of arguments > received by the wrapped function. E.g. if you write: > > @inject > def add(a, b): > return a + b > > @inject > def sub(a, b): > return a - b > > print(add(5), sub(5)) > > Python should print "15 5". (And *not* "15 -5".) > > > (6) [ADVANCED] Modify the decorator in (5) so that it takes an argument telling > it what value to inject into the list of arguments: > > @inject(99) > def sub(a, b): > return a - b > > print(sub(5)) > > will now print "94". > > > (7) Write a decorator which checks the decorated function's two arguments are > given smallest first, swapping them around if needed. > > > (8) Write a decorator which prints the name of the wrapped function, its > arguments, and the time, each time the wrapped function is called. > > > (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying the > path to a file, and use the logging module to log the details to that file > instead of printing them. > > > (10) Write a decorator which adds an "cache" attribute initialised to an empty > dictionary to the decorated function. > > > (11) Write a decorator which wraps a class (not function!), and adds a "help" > method to the class which prints a message as shown below. For example: > > @addhelp > class Spam: > pass > > @addhelp > class Eggs: > pass > > x = Spam() > x.help() > y = Eggs() > y.help() > > will print: > > See http://example.com/Spam > See http://example.com/Eggs > > (Hint: classes also have a __name__ attribute.) > > > (12) [ADVANCED] Write a decorator which wraps a class, and applies the decorator > from (10) above to each non-dunder? method in the class. That is, after: > > @addcaches > class MyClass: > def foo(self): > pass > def bar(self): > pass > > print(MyClass.foo.cache, MyClass.bar.cache) > > should print "{} {}". > > > > ? Remember that dunder methods are those that start with two leading and > trailing underscores: "Double UNDERscore" methods. I also suggest: (13) Modify the decorator from (8) so that the wrapper has the same name and doc string as the wrapped function. (Hint: use functools.wraps) From bc at freeuk.com Tue Oct 3 11:00:16 2017 From: bc at freeuk.com (bartc) Date: Tue, 3 Oct 2017 16:00:16 +0100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: On 03/10/2017 15:39, Ian Kelly wrote: > On Tue, Oct 3, 2017 at 4:41 AM, Steve D'Aprano > wrote: >> On Tue, 3 Oct 2017 06:51 am, Bill wrote: >> >>> Can you inspire me with a good decorator problem (standard homework >>> exercise-level will be fine)? >> >> >> Here is a nice even dozen problems for you. Please ask for clarification if any >> are unclear. >> >> >> >> (1) Write a decorator which simply prints a descriptive message and the name of >> the decorated function once, when the function is first decorated. >> >> E.g. if you write: >> >> @decorate >> def spam(x): >> return x + 1 # for example >> >> print(spam(1)) >> print(spam(2)) >> >> >> Python should print: >> >> Decorating function spam. >> 2 >> 3 >> >> >> Note: "spam" must not be hard-coded, it must be taken from the function being >> decorated. (Hint: all functions have their name available as func.__name__.) >> >> >> (2) Modify the decorator from (1) so that calling the wrapped function also >> print a descriptive message such as "Calling function spam". The expected >> output will be: >> >> Decorating function spam. >> Calling function spam. >> 2 >> Calling function spam. >> 3 >> >> >> (3) Write a decorator that checks that the decorated function's first argument >> is a non-empty string, raising an appropriate exception if it is not, and lets >> through any other arguments unchanged. >> >> >> (4) Same as above, except the first argument is automatically stripped of >> leading and trailing whitespace and forced to uppercase. >> >> >> (5) Write a decorator which injects the argument 10 into the list of arguments >> received by the wrapped function. E.g. if you write: >> >> @inject >> def add(a, b): >> return a + b >> >> @inject >> def sub(a, b): >> return a - b >> >> print(add(5), sub(5)) >> >> Python should print "15 5". (And *not* "15 -5".) >> >> >> (6) [ADVANCED] Modify the decorator in (5) so that it takes an argument telling >> it what value to inject into the list of arguments: >> >> @inject(99) >> def sub(a, b): >> return a - b >> >> print(sub(5)) >> >> will now print "94". >> >> >> (7) Write a decorator which checks the decorated function's two arguments are >> given smallest first, swapping them around if needed. >> >> >> (8) Write a decorator which prints the name of the wrapped function, its >> arguments, and the time, each time the wrapped function is called. >> >> >> (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying the >> path to a file, and use the logging module to log the details to that file >> instead of printing them. >> >> >> (10) Write a decorator which adds an "cache" attribute initialised to an empty >> dictionary to the decorated function. >> >> >> (11) Write a decorator which wraps a class (not function!), and adds a "help" >> method to the class which prints a message as shown below. For example: >> >> @addhelp >> class Spam: >> pass >> >> @addhelp >> class Eggs: >> pass >> >> x = Spam() >> x.help() >> y = Eggs() >> y.help() >> >> will print: >> >> See http://example.com/Spam >> See http://example.com/Eggs >> >> (Hint: classes also have a __name__ attribute.) >> >> >> (12) [ADVANCED] Write a decorator which wraps a class, and applies the decorator >> from (10) above to each non-dunder? method in the class. That is, after: >> >> @addcaches >> class MyClass: >> def foo(self): >> pass >> def bar(self): >> pass >> >> print(MyClass.foo.cache, MyClass.bar.cache) >> >> should print "{} {}". >> >> >> >> ? Remember that dunder methods are those that start with two leading and >> trailing underscores: "Double UNDERscore" methods. [Sorry can't see Steve's original post.] Does all this advanced stuff (which I don't understand and which doesn't look very appealing either; hopefully I will never come across such code) still count as programming? It seems to me the equivalent of an advanced driving course teaching you how to customise your car rather than involving any actual driving. -- bartc From ndbecker2 at gmail.com Tue Oct 3 11:08:39 2017 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 03 Oct 2017 11:08:39 -0400 Subject: when is filter test applied? Message-ID: In the following code (python3): for rb in filter (lambda b : b in some_seq, seq): ... some code that might modify some_seq I'm assuming that the test 'b in some_seq' is applied late, at the start of each iteration (but it doesn't seem to be working that way in my real code), so that if 'some_seq' is modified during a previous iteration the test is correctly performed on the latest version of 'some_seq' at the start of each iteration. Is this correct, and is this guaranteed? From jlgimeno71 at gmail.com Tue Oct 3 11:16:53 2017 From: jlgimeno71 at gmail.com (Jorge Gimeno) Date: Tue, 3 Oct 2017 08:16:53 -0700 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: No, I see this as teaching the skills involved to drive a car. Practicing a turn, scanning gauges, and checking blind spots are all a part of driving. When one is learning, it's easier to learn these in isolation so when the problem must be solved in real time, you know what to do. This is no different. You may never need to use a decorator ever in your development career, but the tool is there in case the problem you have can be elegantly solved using one. -Jorge On Tue, Oct 3, 2017 at 8:00 AM, bartc wrote: > On 03/10/2017 15:39, Ian Kelly wrote: > >> On Tue, Oct 3, 2017 at 4:41 AM, Steve D'Aprano >> wrote: >> >>> On Tue, 3 Oct 2017 06:51 am, Bill wrote: >>> >>> Can you inspire me with a good decorator problem (standard homework >>>> exercise-level will be fine)? >>>> >>> >>> >>> Here is a nice even dozen problems for you. Please ask for clarification >>> if any >>> are unclear. >>> >>> >>> >>> (1) Write a decorator which simply prints a descriptive message and the >>> name of >>> the decorated function once, when the function is first decorated. >>> >>> E.g. if you write: >>> >>> @decorate >>> def spam(x): >>> return x + 1 # for example >>> >>> print(spam(1)) >>> print(spam(2)) >>> >>> >>> Python should print: >>> >>> Decorating function spam. >>> 2 >>> 3 >>> >>> >>> Note: "spam" must not be hard-coded, it must be taken from the function >>> being >>> decorated. (Hint: all functions have their name available as >>> func.__name__.) >>> >>> >>> (2) Modify the decorator from (1) so that calling the wrapped function >>> also >>> print a descriptive message such as "Calling function spam". The expected >>> output will be: >>> >>> Decorating function spam. >>> Calling function spam. >>> 2 >>> Calling function spam. >>> 3 >>> >>> >>> (3) Write a decorator that checks that the decorated function's first >>> argument >>> is a non-empty string, raising an appropriate exception if it is not, >>> and lets >>> through any other arguments unchanged. >>> >>> >>> (4) Same as above, except the first argument is automatically stripped of >>> leading and trailing whitespace and forced to uppercase. >>> >>> >>> (5) Write a decorator which injects the argument 10 into the list of >>> arguments >>> received by the wrapped function. E.g. if you write: >>> >>> @inject >>> def add(a, b): >>> return a + b >>> >>> @inject >>> def sub(a, b): >>> return a - b >>> >>> print(add(5), sub(5)) >>> >>> Python should print "15 5". (And *not* "15 -5".) >>> >>> >>> (6) [ADVANCED] Modify the decorator in (5) so that it takes an argument >>> telling >>> it what value to inject into the list of arguments: >>> >>> @inject(99) >>> def sub(a, b): >>> return a - b >>> >>> print(sub(5)) >>> >>> will now print "94". >>> >>> >>> (7) Write a decorator which checks the decorated function's two >>> arguments are >>> given smallest first, swapping them around if needed. >>> >>> >>> (8) Write a decorator which prints the name of the wrapped function, its >>> arguments, and the time, each time the wrapped function is called. >>> >>> >>> (9) [ADVANCED] Modify the decorator from (8) to take an argument >>> specifying the >>> path to a file, and use the logging module to log the details to that >>> file >>> instead of printing them. >>> >>> >>> (10) Write a decorator which adds an "cache" attribute initialised to an >>> empty >>> dictionary to the decorated function. >>> >>> >>> (11) Write a decorator which wraps a class (not function!), and adds a >>> "help" >>> method to the class which prints a message as shown below. For example: >>> >>> @addhelp >>> class Spam: >>> pass >>> >>> @addhelp >>> class Eggs: >>> pass >>> >>> x = Spam() >>> x.help() >>> y = Eggs() >>> y.help() >>> >>> will print: >>> >>> See http://example.com/Spam >>> See http://example.com/Eggs >>> >>> (Hint: classes also have a __name__ attribute.) >>> >>> >>> (12) [ADVANCED] Write a decorator which wraps a class, and applies the >>> decorator >>> from (10) above to each non-dunder? method in the class. That is, after: >>> >>> @addcaches >>> class MyClass: >>> def foo(self): >>> pass >>> def bar(self): >>> pass >>> >>> print(MyClass.foo.cache, MyClass.bar.cache) >>> >>> should print "{} {}". >>> >>> >>> >>> ? Remember that dunder methods are those that start with two leading and >>> trailing underscores: "Double UNDERscore" methods. >>> >> > [Sorry can't see Steve's original post.] > > Does all this advanced stuff (which I don't understand and which doesn't > look very appealing either; hopefully I will never come across such code) > still count as programming? > > It seems to me the equivalent of an advanced driving course teaching you > how to customise your car rather than involving any actual driving. > > -- > bartc > -- > https://mail.python.org/mailman/listinfo/python-list > From p.f.moore at gmail.com Tue Oct 3 11:35:47 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Oct 2017 16:35:47 +0100 Subject: when is filter test applied? In-Reply-To: References: Message-ID: My intuition is that the lambda creates a closure that captures the value of some_seq. If that value is mutable, and "modify some_seq" means "mutate the value", then I'd expect each element of seq to be tested against the value of some_seq that is current at the time the test occurs, i.e. when the entry is generated from the filter. You say that doesn't happen, so my intuition (and yours) seems to be wrong. Can you provide a reproducible test case? I'd be inclined to run that through dis.dis to see what bytecode was produced. Paul On 3 October 2017 at 16:08, Neal Becker wrote: > In the following code (python3): > > for rb in filter (lambda b : b in some_seq, seq): > ... some code that might modify some_seq > > I'm assuming that the test 'b in some_seq' is applied late, at the start of > each iteration (but it doesn't seem to be working that way in my real code), > so that if 'some_seq' is modified during a previous iteration the test is > correctly performed on the latest version of 'some_seq' at the start of each > iteration. Is this correct, and is this guaranteed? > > > -- > https://mail.python.org/mailman/listinfo/python-list From ndbecker2 at gmail.com Tue Oct 3 11:38:42 2017 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 03 Oct 2017 15:38:42 +0000 Subject: when is filter test applied? In-Reply-To: References: Message-ID: I'm not certain that it isn't behaving as expected - my code is quite complicated. On Tue, Oct 3, 2017 at 11:35 AM Paul Moore wrote: > My intuition is that the lambda creates a closure that captures the > value of some_seq. If that value is mutable, and "modify some_seq" > means "mutate the value", then I'd expect each element of seq to be > tested against the value of some_seq that is current at the time the > test occurs, i.e. when the entry is generated from the filter. > > You say that doesn't happen, so my intuition (and yours) seems to be > wrong. Can you provide a reproducible test case? I'd be inclined to > run that through dis.dis to see what bytecode was produced. > > Paul > > On 3 October 2017 at 16:08, Neal Becker wrote: > > In the following code (python3): > > > > for rb in filter (lambda b : b in some_seq, seq): > > ... some code that might modify some_seq > > > > I'm assuming that the test 'b in some_seq' is applied late, at the start > of > > each iteration (but it doesn't seem to be working that way in my real > code), > > so that if 'some_seq' is modified during a previous iteration the test is > > correctly performed on the latest version of 'some_seq' at the start of > each > > iteration. Is this correct, and is this guaranteed? > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Tue Oct 3 11:45:14 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 03 Oct 2017 17:45:14 +0200 Subject: when is filter test applied? References: Message-ID: Neal Becker wrote: > In the following code (python3): > > for rb in filter (lambda b : b in some_seq, seq): > ... some code that might modify some_seq > > I'm assuming that the test 'b in some_seq' is applied late, at the start > of each iteration (but it doesn't seem to be working that way in my real > code), so that if 'some_seq' is modified during a previous iteration the > test is correctly performed on the latest version of 'some_seq' at the > start of each > iteration. Is this correct, and is this guaranteed? Yes. filter() is lazy in Python 3, and otherwise the usual rules for binding, mutating, and lookup of some_seq apply. Example: >>> some_seq = set("abc") >>> for c in filter(lambda x: x in some_seq, "abbbcccCBBBBAAA"): ... some_seq.remove(c) ... >>> some_seq = set("abc") >>> for c in filter(lambda x: x in some_seq, "abbbcccCBBBBAAA"): ... print(c) ... some_seq.remove(c) ... if not some_seq: some_seq = set("ABC") ... a b c # at that point some_seq will be bound to a new set("ABC") C B A # as above A From p.f.moore at gmail.com Tue Oct 3 12:35:57 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Oct 2017 17:35:57 +0100 Subject: when is filter test applied? In-Reply-To: References: Message-ID: On 3 October 2017 at 16:38, Neal Becker wrote: > I'm not certain that it isn't behaving as expected - my code is quite > complicated. OK, so I'd say the filtering would follow any changes made to some_seq - not because it's a documented guarantee as such, but simply as a consequence of the behaviour of generators and closures. I'd tend to steer clear of code that relied on that behaviour in practice, as I think it's likely to be hard to understand/maintain, but I'm aware that reality's never quite that simple :-) Paul From ian.g.kelly at gmail.com Tue Oct 3 12:40:47 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 3 Oct 2017 10:40:47 -0600 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 3, 2017 at 9:16 AM, Jorge Gimeno wrote: > No, I see this as teaching the skills involved to drive a car. Practicing a > turn, scanning gauges, and checking blind spots are all a part of driving. > When one is learning, it's easier to learn these in isolation so when the > problem must be solved in real time, you know what to do. This is no > different. You may never need to use a decorator ever in your development > career, but the tool is there in case the problem you have can be elegantly > solved using one. I have to agree. Learning the language without taking the time to learn all of its features is like learning to drive without bothering to learn how to use the tachometer or the hand brake. Sure you can drive without those things, but that doesn't make them not useful. From rosuav at gmail.com Tue Oct 3 13:35:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Oct 2017 04:35:33 +1100 Subject: optional int- and float arguments In-Reply-To: References: Message-ID: On Wed, Oct 4, 2017 at 3:18 AM, Stefan Ram wrote: > ?int? and ?float? seem to behave quite similar: > > |>>> int( x = 8 ) > |8 > |>>> float( x = 8.0 ) > |8.0 > |>>> int() > |0 > |>>> float() > |0.0 > > . Yet the ways their parameters are being documented in > "The Python Library Reference, Release 3.6.0" seem to differ: > > |class float([x]) > |class int(x=0) > Huh. I just checked help() in Python 3.7, and it's the other way around. | float(x=0, /) | int([x]) -> integer > Would there be any error in describing ?float? using > > |class float(x=0.0) > > ? I don't think so. This is a strange inconsistency in the docs, given that they behave the same way. Would be worth opening a tracker issue, mentioning all four things you found. ChrisA From rhodri at kynesim.co.uk Tue Oct 3 13:45:02 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 3 Oct 2017 18:45:02 +0100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On 03/10/17 18:29, Stefan Ram wrote: > Is this the best way to write a "loop and a half" in Python? Define "best". > x = 1 > while x: > x = int( input( "Number (enter 0 to terminate)? " )) > if x: > print( f'Square = { x**2 }' ) I'd usually write it as while True: x = tedious_function_call_with_sentinel() if x == 0: break do_something_with(x) ...or alternatively x = tedious_function_call_with_sentinel() while x != 0: do_something_with(x) x = tedious_function_call_with_sentinel() ...or some other refactoring. > In a C-like language, one could write: > > while x = int( input( "Number (enter 0 to terminate)? " )) > print( f'Square = { x**2 }' ) One could. One would richly deserve the compiler warnings one got as a result, but one could. -- Rhodri James *-* Kynesim Ltd From bcskinner1 at gmail.com Tue Oct 3 14:02:40 2017 From: bcskinner1 at gmail.com (Breta Skinner) Date: Tue, 3 Oct 2017 12:02:40 -0600 Subject: F5 not working? Message-ID: Hello, I tried looking around the website, but didn't see a question about function keys. Somehow, I "turned off" the F5 function in the IDLE. It no longer "run module", but just does nothing. I have downloaded the newest version, 3.7, but that did not solve the problem. I admit to being clumsy, so I'm guessing I hit an odd key combination which disabled F5 for Python IDLE on this machine (windows 64 bit HP desk top). I tried to check the configuration, but it looks like the key is set up: [image: Inline image 1] Any suggestions? It was nice to have the key working. Thanks, Breta. From __peter__ at web.de Tue Oct 3 14:10:29 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 03 Oct 2017 20:10:29 +0200 Subject: The "loop and a half" References: Message-ID: Stefan Ram wrote: > Is this the best way to write a "loop and a half" in Python? > > x = 1 > while x: > x = int( input( "Number (enter 0 to terminate)? " )) > if x: > print( f'Square = { x**2 }' ) > > In a C-like language, one could write: > > while x = int( input( "Number (enter 0 to terminate)? " )) > print( f'Square = { x**2 }' ) Use iter() and a for loop: >>> def input_int(): ... return int(input("Enter number (0 to terminate): ")) ... >>> for x in iter(input_int, 0): ... print(f"Square = { x**2 }") ... Enter number (0 to terminate): 1 Square = 1 Enter number (0 to terminate): 2 Square = 4 Enter number (0 to terminate): 0 >>> From tjol at tjol.eu Tue Oct 3 14:20:44 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 3 Oct 2017 20:20:44 +0200 Subject: The "loop and a half" In-Reply-To: References: Message-ID: <0aa7417d-2d05-6b6f-2e09-65b13744b326@tjol.eu> On 03/10/17 20:05, Dennis Lee Bieber wrote: > > while True: > x = input("Enter a number (blank to exit) => ") > if x: > try: > ix = int(x) > print("Square = %s" % (ix * ix) ) #why invoke exponential > except checkTheDocsForException: The nice thing about how the stdlib does exceptions is that you can be pretty certain that this has simply got to be a ValueError. It's still best to check (or, in this case, try it out) - but you just know that there's nothing else it can be. > print(" '%s' is not a valid integer" % x ) > else: > break PS: I was weak. I tried it out before sending this mail. From BILL_NOSPAM at whoknows.net Tue Oct 3 14:55:11 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Tue, 3 Oct 2017 14:55:11 -0400 Subject: newb question about @property In-Reply-To: <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Tue, 3 Oct 2017 06:51 am, Bill wrote: > >> Can you inspire me with a good decorator problem (standard homework >> exercise-level will be fine)? > > Here is a nice even dozen problems for you. Please ask for clarification if any > are unclear. Thank you for sharing the problems on decorators! I just finished reading Bruce Eckels' note on decorators (as well as some of the comments left by readers), which you shared a link to, and found the whole discussion very informative. If I was to point to two details, the analogy of decorator's with macros is helpful to bear in mind, as is the remark that "@Wrapper(def f..)" reassigns f to the composition. With the latter unstated, the matter can cause confusion! %-) Bill > > > > (1) Write a decorator which simply prints a descriptive message and the name of > the decorated function once, when the function is first decorated. > > E.g. if you write: > > @decorate > def spam(x): > return x + 1 # for example > > print(spam(1)) > print(spam(2)) > > > Python should print: > > Decorating function spam. > 2 > 3 > > > Note: "spam" must not be hard-coded, it must be taken from the function being > decorated. (Hint: all functions have their name available as func.__name__.) > > > (2) Modify the decorator from (1) so that calling the wrapped function also > print a descriptive message such as "Calling function spam". The expected > output will be: > > Decorating function spam. > Calling function spam. > 2 > Calling function spam. > 3 > > > (3) Write a decorator that checks that the decorated function's first argument > is a non-empty string, raising an appropriate exception if it is not, and lets > through any other arguments unchanged. > > > (4) Same as above, except the first argument is automatically stripped of > leading and trailing whitespace and forced to uppercase. > > > (5) Write a decorator which injects the argument 10 into the list of arguments > received by the wrapped function. E.g. if you write: > > @inject > def add(a, b): > return a + b > > @inject > def sub(a, b): > return a - b > > print(add(5), sub(5)) > > Python should print "15 5". (And *not* "15 -5".) > > > (6) [ADVANCED] Modify the decorator in (5) so that it takes an argument telling > it what value to inject into the list of arguments: > > @inject(99) > def sub(a, b): > return a - b > > print(sub(5)) > > will now print "94". > > > (7) Write a decorator which checks the decorated function's two arguments are > given smallest first, swapping them around if needed. > > > (8) Write a decorator which prints the name of the wrapped function, its > arguments, and the time, each time the wrapped function is called. > > > (9) [ADVANCED] Modify the decorator from (8) to take an argument specifying the > path to a file, and use the logging module to log the details to that file > instead of printing them. > > > (10) Write a decorator which adds an "cache" attribute initialised to an empty > dictionary to the decorated function. > > > (11) Write a decorator which wraps a class (not function!), and adds a "help" > method to the class which prints a message as shown below. For example: > > @addhelp > class Spam: > pass > > @addhelp > class Eggs: > pass > > x = Spam() > x.help() > y = Eggs() > y.help() > > will print: > > See http://example.com/Spam > See http://example.com/Eggs > > (Hint: classes also have a __name__ attribute.) > > > (12) [ADVANCED] Write a decorator which wraps a class, and applies the decorator > from (10) above to each non-dunder? method in the class. That is, after: > > @addcaches > class MyClass: > def foo(self): > pass > def bar(self): > pass > > print(MyClass.foo.cache, MyClass.bar.cache) > > should print "{} {}". > > > > ? Remember that dunder methods are those that start with two leading and > trailing underscores: "Double UNDERscore" methods. > > > * * * > > > Bruce Eckel has an excellent introduction to Python decorators, from way back > when they were first introduced in 2008. His introduction is notable because: > > - he points out explicitly that Python decorators are not the same as > the Decorator design pattern (I thought they were!); > > - he recommends using a class as the decorator, and building the extra > functionality in object oriented fashion, rather than functional > programming fashion (this may give an easier introduction to those > who aren't familiar with functional idioms); > > - and he correctly predicted that the introduction of the @ syntactic > sugar would have a big impact on the way people think about Python > code. > > > http://www.artima.com/weblogs/viewpost.jsp?thread=240808 > > Feel free to read his post before trying the problems I set. > > > From tjreedy at udel.edu Tue Oct 3 15:09:55 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 3 Oct 2017 15:09:55 -0400 Subject: F5 not working? In-Reply-To: References: Message-ID: On 10/3/2017 2:02 PM, Breta Skinner wrote: > Hello, > I tried looking around the website, but didn't see a question about > function keys. > Somehow, I "turned off" the F5 function in the IDLE. It no longer "run > module", but just does nothing. I have downloaded the newest version, 3.7, > but that did not solve the problem. I admit to being clumsy, so I'm > guessing I hit an odd key combination which disabled F5 for Python IDLE on > this machine (windows 64 bit HP desk top). > I tried to check the configuration, but it looks like the key is set up: > [image: Inline image 1] This is text only list; images are discarded. If you run IDLE, select Options => Preferences => Keys and scroll down to 'run-module', it will will say for a built-in keyset until you have changed idlelib/config-keys.def and should say for custom key sets until you changed it, in which case, you should be able to change it back. If it says F5, which is perhaps what you meant above, and F5 does not work, then you might have changed your keyboard, outside of IDLE, so that it does not generate the expected keycode. Have you rebooted? > Any suggestions? It was nice to have the key working. -- Terry Jan Reedy From BILL_NOSPAM at whoknows.net Tue Oct 3 15:11:42 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Tue, 3 Oct 2017 15:11:42 -0400 Subject: The "loop and a half" In-Reply-To: References: Message-ID: Stefan Ram wrote: > Is this the best way to write a "loop and a half" in Python? Is your goal brevity or clarity, or something else (for instance, what does the code written by the other members of your "team" look like--woudn't it be nice if it matched)? Bill > > x = 1 > while x: > x = int( input( "Number (enter 0 to terminate)? " )) > if x: > print( f'Square = { x**2 }' ) > > In a C-like language, one could write: > > while x = int( input( "Number (enter 0 to terminate)? " )) > print( f'Square = { x**2 }' ) > > . > From kryptxy at protonmail.com Tue Oct 3 15:23:24 2017 From: kryptxy at protonmail.com (Kryptxy) Date: Tue, 03 Oct 2017 15:23:24 -0400 Subject: Need some help with argparse Message-ID: Hi, I am trying to figure out something with argparse. Here is a scenario: I have a group of arguments, say: (-a, -b, -c, -d, -e) [lets call it group1] I have another group, say: (-z, -y, -x, -w) [lets call it group2] Code: import argparse parser = argparse.ArgumentParser(description="Test this shiz") group1= parser.add_argument_group("Group 1") group2= parser.add_argument_group("Group 2") group1.add_argument('-a', '--extendA', action="store_true", help="HELP") group1.add_argument('-b', '--extendB', action="store_true", help="HELP") group1.add_argument('-c', '--extendC', action="store_true", help="HELP") group1.add_argument('-d', '--extendD', action="store_true", help="HELP") group1.add_argument('-e', '--extendE', action="store_true", help="HELP") # Similarly for group 2: group2.add_argument('-z', '--extendZ', action="store_true", help="HELP") group2.add_argument('-y', '--extendY', action="store_true", help="HELP") group2.add_argument('-x', '--extendX', action="store_true", help="HELP") group2.add_argument('-w', '--extendW', action="store_true", help="HELP") args = parser.parse_args() Now I want to get arguments of group1 and group2 separately. If I print(args) - I get arguments from both the groups. Also group1.parse_args() does not work (since ArgumentGroup does not have parse_args() method) How can I get all arguments of group 1 ONLY? Same goes for group 2? I tried subparsers too - but they require a mandatory `positional argument` - which is not application's requirement. Any kind of help is appreciated. Thank you! From ben+python at benfinney.id.au Tue Oct 3 16:01:22 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 04 Oct 2017 07:01:22 +1100 Subject: Need some help with argparse References: Message-ID: <854lrg3srx.fsf@benfinney.id.au> Kryptxy via Python-list writes: > I have a group of arguments, say: (-a, -b, -c, -d, -e) [lets call it group1] > I have another group, say: (-z, -y, -x, -w) [lets call it group2] Argument groups are a feature to control the help output from the parser: When an argument is added to the group, the parser treats it just like a normal argument, but displays the argument in a separate group for help messages. > Now I want to get arguments of group1 and group2 separately. You don't want to use the ?argument groups? feature for that, because that's not what it does. > How can I get all arguments of group 1 ONLY? Same goes for group 2? > I tried subparsers too - but they require a mandatory `positional > argument` - which is not application's requirement. I don't know of a way to have the argument parser know the separation you're talking about, without using a subparser. You will need to maintain that separation yourself, outside the parser. One simple way (I have not tried this):: argument_names_by_group = { 'foo': {'lorem', 'donec', 'fusce'}, 'bar': {'aliquam', 'nunc'}, } args = parser.parse_args() args_by_group = { group_name: { getattr(arg_name) for arg_name in argument_names_by_group[group_name] } for group_name in argument_names_by_group } -- \ ?I believe our future depends powerfully on how well we | `\ understand this cosmos, in which we float like a mote of dust | _o__) in the morning sky.? ?Carl Sagan, _Cosmos_, 1980 | Ben Finney From nad at python.org Tue Oct 3 16:06:44 2017 From: nad at python.org (Ned Deily) Date: Tue, 3 Oct 2017 16:06:44 -0400 Subject: [RELEASE] Python 3.6.3 is now available Message-ID: <3D8F5841-F970-4C00-9A49-B4F8FD24CF79@python.org> On behalf of the Python development community and the Python 3.6 release team, I am happy to announce the availability of Python 3.6.3, the third maintenance release of Python 3.6. Detailed information about the changes made in 3.6.3 can be found in the change log here: https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-3-final Please see "What?s New In Python 3.6" for more information about the new features in Python 3.6: https://docs.python.org/3.6/whatsnew/3.6.html You can download Python 3.6.3 here: https://www.python.org/downloads/release/python-363/ The next maintenance release of Python 3.6 is expected to follow in about 3 months, around the end of 2017-12. More information about the 3.6 release schedule can be found here: https://www.python.org/dev/peps/pep-0494/ Enjoy! -- Ned Deily nad at python.org -- [] From dvl at psu.edu Tue Oct 3 16:31:15 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Tue, 3 Oct 2017 16:31:15 -0400 Subject: when is the filter test applied? In-Reply-To: mailman.40.1507046404.28091.python-list@python.org References: Message-ID: <1507062675l.16646208l.0l@psu.edu> On Tue, Oct 3, 2017 15:38:42, Neal Becker wrote: > I'm not certain that it isn't behaving as expected - my code is quite >complicated. > >On Tue, Oct 3, 2017 at 11:35 AM Paul Moore wrote: > >> My intuition is that the lambda creates a closure that captures the >> value of some_seq. If that value is mutable, and "modify some_seq" >> means "mutate the value", then I'd expect each element of seq to be >> tested against the value of some_seq that is current at the time the >> test occurs, i.e. when the entry is generated from the filter. >> >> You say that doesn't happen, so my intuition (and yours) seems to be >> wrong. Can you provide a reproducible test case? I'd be inclined to >> run that through dis.dis to see what bytecode was produced. >> >> Paul >> >> On 3 October 2017 at 16:08, Neal Becker wrote: >> > In the following code (python3): >> > >> > for rb in filter (lambda b : b in some_seq, seq): >> > ... some code that might modify some_seq >> > >> > I'm assuming that the test 'b in some_seq' is applied late, at the start of >> > each iteration (but it doesn't seem to be working that way in my real code), >> > so that if 'some_seq' is modified during a previous iteration the test is >> > correctly performed on the latest version of 'some_seq' at the start of each >> > iteration. Is this correct, and is this guaranteed? > > I think one must always be very careful about modifying anything that you are iterating across -- if not avoiding that sort of thing entirely. In particular, avoid anything that might add or remove elements in the middle of the sequence. YMMV, but you can imagine the iterator behaving like some of these (all naturally implementation-dependent) Traversing an indexable sequence (such as a list or string) might maintain an internal counter that increments for each iteration. Traversing a linked sequence (such as a linked list) might maintain an internal reference that advances down the chain. If you were to remove the element that you were currently visiting (such was done in "for c in x: x.remove(c)")) you may end up missing items. For example, if you remove element 10 from a list or string, element 11 moves forward to position 10, but the internal counter mentioned above would still advance to 11 -- skipping over the item that followed what you removed. Similarly, if you to remove the linked list node that contained the data you tried to remove, you might have an unspecified behavior of where the linked list advancement would take you. It would be safer to make a deep copy of the data iterable data and iterate down the copy, so you can be sure that all your intended changes to the original data do not interfere with your iteration. Here is an illustration of something going awry: ('.'s added in case my mailer destroys indents) x = list('aaabaaaacaaaaadaaaaaae') for c in x: ... if c == 'a': ....... x.remove('a') ''.join(x) 'bcaadaaaaaae' Of the original 18 a's, 10 were removed. The test for (c == 'a') only applied to every other a, because the deletions removed some a's forward, while the iterator kept advancing. (And of course, it always deleted the first a in the list, even if that is not where my iterator was looking) And yes, I would expect the filter to be a lazy application to the iterable you are filtering, so anything that removes from your iterable is going to cause you filter so skip something. That can also be fixed by copying the iterable first. Or alternatively, you can go more functional and instead create a new iterable set of data from the old, instead of mutating what you have. Roger Christman Pennsylvania State University From rosuav at gmail.com Tue Oct 3 16:57:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Oct 2017 07:57:45 +1100 Subject: when is the filter test applied? In-Reply-To: <1507062675l.16646208l.0l@psu.edu> References: <1507062675l.16646208l.0l@psu.edu> Message-ID: On Wed, Oct 4, 2017 at 7:31 AM, ROGER GRAYDON CHRISTMAN wrote: >>> > for rb in filter (lambda b : b in some_seq, seq): >>> > ... some code that might modify some_seq > I think one must always be very careful about modifying anything > that you are iterating across -- if not avoiding that sort of thing entirely. > In particular, avoid anything that might add or remove elements > in the middle of the sequence. It isn't, though. That's iterating over seq, and possibly modifying some_seq. So it's safe. ChrisA From pcornwell77 at gmail.com Tue Oct 3 17:06:02 2017 From: pcornwell77 at gmail.com (Patty Kramer) Date: Tue, 3 Oct 2017 14:06:02 -0700 (PDT) Subject: testbank Message-ID: <34052f0e-8c3d-45d8-9664-18b5224f0300@googlegroups.com> do you have testbank for Small Business Management: Launching & Growing Entrepreneurial Ventures, 18th Edition by Justin G. LongeneckerJ. William Petty From steve+python at pearwood.info Tue Oct 3 17:40:00 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 08:40:00 +1100 Subject: The "loop and a half" References: Message-ID: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 04:45 am, Rhodri James wrote: > On 03/10/17 18:29, Stefan Ram wrote: >> Is this the best way to write a "loop and a half" in Python? > > Define "best". I'd start with "define loop and a half". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Tue Oct 3 18:07:17 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 3 Oct 2017 18:07:17 -0400 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On 10/3/2017 2:10 PM, Peter Otten wrote: > Stefan Ram wrote: > >> Is this the best way to write a "loop and a half" in Python? [snip while loop] > Use iter() and a for loop: > >>>> def input_int(): > ... return int(input("Enter number (0 to terminate): ")) > ... >>>> for x in iter(input_int, 0): > ... print(f"Square = { x**2 }") > ... > Enter number (0 to terminate): 1 > Square = 1 > Enter number (0 to terminate): 2 > Square = 4 > Enter number (0 to terminate): 0 I think this is the best answer. Using a for loop cleanly separates the source of a sequence of objects from processing the objects. While loop raret in Python as they are only needed for other patterns, such as 'while mutable object is not in the state we want: modify some more', and the number of modifications needed is not necessarity known ahead of time. -- Terry Jan Reedy From steve+python at pearwood.info Tue Oct 3 18:18:30 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 09:18:30 +1100 Subject: optional int- and float arguments References: Message-ID: <59d40cb8$0$14962$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 03:18 am, Stefan Ram wrote: > ?int? and ?float? seem to behave quite similar: > > |>>> int( x = 8 ) > |8 > |>>> float( x = 8.0 ) > |8.0 I expect that these functions taking a *named* parameter "x" is an accident that shouldn't be relied on. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From orgnut at yahoo.com Tue Oct 3 22:17:49 2017 From: orgnut at yahoo.com (Larry Hudson) Date: Tue, 3 Oct 2017 19:17:49 -0700 Subject: The "loop and a half" In-Reply-To: References: Message-ID: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> On 10/03/2017 10:29 AM, Stefan Ram wrote: > Is this the best way to write a "loop and a half" in Python? > > x = 1 > while x: > x = int( input( "Number (enter 0 to terminate)? " )) > if x: > print( f'Square = { x**2 }' ) > > In a C-like language, one could write: > > while x = int( input( "Number (enter 0 to terminate)? " )) > print( f'Square = { x**2 }' ) > This does not answer your question and is only somewhat related. Here is a generic number input function that I wrote for my own (hobby-programmer) use. The docstring explains it. -------------------- def get_num(prmt, lo=None, hi=None, dflt=None, flt=False, abrt=False): """Get a number from the console Parameters: prmt: The prompt to be used with the input function, required. lo: The minimum permissible value, or None if no minimum (the default) hi: The maximum permissible value, or None if no maximum (the default) dflt: Default value to return with empty input, None is no default value flt: If True, accepts and returns floats If False, accepts and returns integers (the default) abrt: If True empty input aborts and returns None If False (the default) empty input is handled as invalid data Invalid input or out-of-range values are not accepted and a warning message is displayed. It will then repeat the input prompt for a new value. Note: If the dflt parameter is given, its data type is not checked and could be anything, and could also be used as an abort signal. """ while True: val = input(prmt).strip() if not val: if abrt: return None if dflt is not None: return dflt try: num = float(val) if flt else int(val) except ValueError: print('Invalid input, try again') continue # We have a valid number here, check for in-range if (lo is None or num >= lo) and (hi is None or num <= hi): return num print('Number is out of range') ------------------ FWIW, as to your question, my preference is for the pseudo-do loop using 'while True' with a break as described in other answers here. Using this function, your loop could be: while True: x = get_num('Number (enter ) to terminate: ') if x == 0: # Or if not x: break print(f'Square = { x**2 }') OR my preference: Use empty input to terminate by making the input and test: x = get_num('Number ( to terminate: ', abrt=True) if x is None: break -- -=- Larry -=- From rosuav at gmail.com Tue Oct 3 22:40:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Oct 2017 13:40:45 +1100 Subject: The "loop and a half" In-Reply-To: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> Message-ID: On Wed, Oct 4, 2017 at 1:17 PM, Larry Hudson via Python-list wrote: > def get_num(prmt, lo=None, hi=None, dflt=None, flt=False, abrt=False): > """Get a number from the console > > Parameters: > prmt: The prompt to be used with the input function, required. > lo: The minimum permissible value, or None if no minimum (the > default) > hi: The maximum permissible value, or None if no maximum (the > default) > dflt: Default value to return with empty input, None is no default > value > flt: If True, accepts and returns floats > If False, accepts and returns integers (the default) > abrt: If True empty input aborts and returns None > If False (the default) empty input is handled as invalid > data > > Invalid input or out-of-range values are not accepted and a warning > message > is displayed. It will then repeat the input prompt for a new value. > Note: If the dflt parameter is given, its data type is not checked and > could be anything, and could also be used as an abort signal. > """ > > while True: > val = input(prmt).strip() > if not val: > if abrt: > return None > if dflt is not None: > return dflt > try: > num = float(val) if flt else int(val) > except ValueError: > print('Invalid input, try again') > continue > # We have a valid number here, check for in-range > if (lo is None or num >= lo) and (hi is None or num <= hi): > return num > print('Number is out of range') You know, you don't HAVE to economize on letters. It's okay to call your parameters "prompt" instead of "prmt". Remember, that's part of your API. I'd whitewash the bikeshed a slightly different colour, personally. _SENTINEL = object() def get_number(prompt, *, min=None, max=None, default=_SENTINEL, type=int): Broadly the same as you have (modulo variable names), but there's no need for the boolean 'abrt', since setting default to None will have the same effect - hence the dedicated sentinel object to represent "no default". (I also mandated keyword args, since that's how you're using them anyway.) The one real change I'd make is to replace the boolean flag "flt" with a type, so you now say: x = get_number("Number (enter to terminate): ", type=float) It reads more cleanly than "flt=True", and you don't have to look up any docs to grok it; plus, if you ever need to, you can use "type=Decimal" or something. It's simpler inside the code too - "num = type(val)". If you don't like shadowing the built-in name 'type', feel free to call this something else, but IMO it's okay to shadow here. ChrisA From steve+python at pearwood.info Tue Oct 3 23:44:09 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 14:44:09 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 09:08 am, Stefan Ram wrote: > Steve D'Aprano writes: >>On Wed, 4 Oct 2017 04:45 am, Rhodri James wrote: >>>On 03/10/17 18:29, Stefan Ram wrote: >>>>Is this the best way to write a "loop and a half" in Python? >>>Define "best". >>I'd start with "define loop and a half". [...] > I might got Sahami wrong, but it took it to mean this kind > of loop (in pseudo code, which is not necessarily Python): > > while condition > ... > ... > ... > if condition > ... > end if > end while > > The two ?condition?s above are the same condition. Hmmm. To me, that appears to be just one case of an unlimited number of different possibilities, none more special or general than any of the others: while condition ... if condition ... else ... end if ... if condition ... else ... end if ... if not condition ... end if ... ... # etc end while I don't see why your version is any more special, or deserving of a name, than any other variety: while condition ... while condition ... end while ... if not condition ... end if while not condition ... if not condition ... else ... end if ... end while ... while condition ... end while ... end while There is literally an infinite number of possibilities for what can go inside a while loop. More while loops, if...else, for loops, any structure you like. What is so special about "if condition..." that it deserves a special name or syntax? > Initially, one loops through this loop an integral > number of times. > > But eventually the conditions is made false by the > upper half of the loop, so then the lower half is > skipped. Or the middle third is skipped. Of the second and third parts of five. Or the fifth, eighth and eleventh parts of twelve. > But at this point in the class, I only have > shown ?while? and ?if? so far, iterators and for-loops > will only be introduced later. Why? Teaching while loops before for-each loops is backwards. That's one of the bigger problems with "Learn Python the Hard Way", and I see people struggle, really really struggle, so solve problems using a while loop which are *trivial* using a for loop. For-each loops are MUCH easier to understand, and should be taught first. "Do something to each of these items" is a very natural, almost instinctive operation. While loops are harder to write correctly, harder to reason about, and harder to learn. Just because for-each loops are, in some sense, derived from while loops, doesn't mean you should teach them first. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 3 23:44:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 14:44:56 +1100 Subject: optional int- and float arguments References: <59d40cb8$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d45939$0$14941$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 12:17 pm, Stefan Ram wrote: > |>>> str(object='abc') > |'abc' That's probably also a bug. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 3 23:48:26 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 14:48:26 +1100 Subject: The "loop and a half" References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> Message-ID: <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: > You know, you don't HAVE to economize on letters. It's okay to call > your parameters "prompt" instead of "prmt". Remember, that's part of > your API. Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of vwel wth nt nuff 4 vrybdy. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 3 23:51:36 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 14:51:36 +1100 Subject: on a very slow function References: <86h8vi36e9.fsf@toledo.com> <874lrieb5v.fsf@bsb.me.uk> <59d18993$0$14978$b1db1813$d948b532@news.astraweb.com> <87y3oucqim.fsf@bsb.me.uk> <59d1a117$0$14948$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d45aca$0$14949$b1db1813$d948b532@news.astraweb.com> On Tue, 3 Oct 2017 04:23 am, Ian Kelly wrote: >> py> (2**75 + 7) % 12 # Expected value. >> 3 >> py> ((2**75) % 12 + (7 % 12)) % 12 # Correct. >> 3 >> py> (2**75) % 12 + (7 % 12) # Incorrect. >> 15 > > No, only the final one is necessary. Modding the result of the > exponentiation might be useful for performance, but if c is expected > to be small then it may be pointless to mod that as well. > > py> ((2**75) % 12 + 7) % 12 # Still correct. > 3 Its been years since I've had to tackle modulo arithmetic, but the more I think about it, the more I think you are right. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Oct 3 23:54:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 4 Oct 2017 14:54:37 +1100 Subject: The "loop and a half" In-Reply-To: <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, Oct 4, 2017 at 2:48 PM, Steve D'Aprano wrote: > On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: > >> You know, you don't HAVE to economize on letters. It's okay to call >> your parameters "prompt" instead of "prmt". Remember, that's part of >> your API. > > > Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of vwel wth > nt nuff 4 vrybdy. > And then English will turn into an abjad like Arabic. Yes, I see how it is. Python is a sneaky plan by Islamic extremists to make us use up all our vowels. (I'd end the post there, except that I've learned from past experience that there is no statement so ridiculous that people will know for sure that you're joking. So for the humour-impaired: THIS POST WAS A JOKE. Okay? Good.) ChrisA From steve+python at pearwood.info Wed Oct 4 01:32:59 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 04 Oct 2017 16:32:59 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 02:00 am, bartc wrote: > Does all this advanced stuff (which I don't understand and which doesn't > look very appealing either; hopefully I will never come across such > code) still count as programming? I could not have hoped to see a more perfect example of the Blub effect in action if I had invented it myself. As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub. http://www.paulgraham.com/avg.html > It seems to me the equivalent of an advanced driving course teaching you > how to customise your car rather than involving any actual driving. You don't think that adding a cache for an expensive function is programming? If you had ten expensive functions, and you wanted to add a cache to each of them, would you write out ten separate caches (probably copying and pasting the code each time)? Or would you write a function that builds a cache and adds it to the expensive function *once*, then call it ten times, once for each function being wrapped? My examples didn't include a full-blown cache, just the beginnings of one, but the principle still stands. Program smart, not hard: don't do things by hand if you can write a function to do them. That you think that this is not programming is an indictment off your programming skills. These sorts of functional programming techniques go back to the 1950s, Lisp is literally the second oldest high-level language ever (only Fortran is older). You may or may not have been a hotshot in your little corner of the programming world, but there's an entire world out there. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Wed Oct 4 03:16:29 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 04 Oct 2017 20:16:29 +1300 Subject: The "loop and a half" In-Reply-To: <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: > >>You know, you don't HAVE to economize on letters. It's okay to call >>your parameters "prompt" instead of "prmt". Remember, that's part of >>your API. > > Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of vwel wth > nt nuff 4 vrybdy. I blame the Dutch. They're clearly using more than their fair share of the world's vowels. -- Greg From alister.ware at ntlworld.com Wed Oct 4 04:33:55 2017 From: alister.ware at ntlworld.com (alister) Date: Wed, 04 Oct 2017 08:33:55 GMT Subject: The "loop and a half" References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, 04 Oct 2017 20:16:29 +1300, Gregory Ewing wrote: > Steve D'Aprano wrote: >> On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: >> >>>You know, you don't HAVE to economize on letters. It's okay to call >>>your parameters "prompt" instead of "prmt". Remember, that's part of >>>your API. >> >> Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of >> vwel wth nt nuff 4 vrybdy. > > I blame the Dutch. They're clearly using more than their fair share of > the world's vowels. but that is compensated for by the welsh which don't seem to use any -- Never trust an operating system. From robin at reportlab.com Wed Oct 4 05:01:07 2017 From: robin at reportlab.com (Robin Becker) Date: Wed, 4 Oct 2017 10:01:07 +0100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Given the prevalence of the loop and a half idea in python I wonder why we don't have a "do" or "loop" statement to start loops without a test. > C:\Python27\Lib>grep "while True" *.py | wc -l > 99 > > C:\Python27\Lib>grep "while 1" *.py | wc -l > 117 > C:\Python36\Lib>grep "while True" *.py | wc -l > 131 > > C:\Python36\Lib>grep "while 1" *.py | wc -l > 44 How much does the while True actually cost compared to nothing? -- Robin Becker From ben.usenet at bsb.me.uk Wed Oct 4 05:14:18 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 04 Oct 2017 10:14:18 +0100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: <87fuaz6zrp.fsf@bsb.me.uk> Steve D'Aprano writes: > On Wed, 4 Oct 2017 04:45 am, Rhodri James wrote: > >> On 03/10/17 18:29, Stefan Ram wrote: >>> Is this the best way to write a "loop and a half" in Python? >> >> Define "best". > > I'd start with "define loop and a half". What it means to me is this pattern: while True: ... statements that don't exit the loop ... if condition: break ... more statements that don't exit the loop ... When the first set of statements is null, it's just a while loop. When the second is null, it's just a do ... while (in languages that have it). Where both sets are present some languages encourage putting the first set into the condition of the loop. For example, in Algol 68 an expression can be replaced by a statement sequence that ends in an expression. C encourages the same, provided the first set of statements is very simple. The classic example being while ((some_var = some_input_function()) != some_error_value) { ... process some_var ... } In Pascal, you sometimes had to duplicate the first set of statements: ... statements that don't exit the loop ... while condition begin ... more statements that don't exit the loop ... ... statements that don't exit the loop ... end -- Ben. From tomuxiong at gmx.com Wed Oct 4 05:19:21 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Wed, 4 Oct 2017 11:19:21 +0200 Subject: Multithreaded compression/decompression library with python bindings? Message-ID: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> Hello, I was wondering if anyone here knew of any python libraries with interfaces similar to the bzip2 module which is also multithreaded in (de)compression? Something along the lines of (say) the pbip2 program but with bindings for python? Obviously the multi-threaded part will need to be handled by the underlying library code and not the python code itself. If that's too much to ask (I haven't found anything myself after all), if anyone here happens to have any experience using similar C++ (or possibly C) libraries I am certainly open for any recommendations. :) Thanks in advance! Thomas From leamhall at gmail.com Wed Oct 4 06:49:42 2017 From: leamhall at gmail.com (Leam Hall) Date: Wed, 4 Oct 2017 06:49:42 -0400 Subject: Good virtualenv and packaging tutorials for beginner? Message-ID: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> Folks on IRC have suggested using virtualenv to test code under different python versions. Sadly, I've not found a virtualenv tutorial I understand. Anyone have a link to a good one? The next step will be to figure out how to package a project; a good tutorial URL would be appreciated on that, too. Thanks! Leam From leamhall at gmail.com Wed Oct 4 06:53:51 2017 From: leamhall at gmail.com (Leam Hall) Date: Wed, 4 Oct 2017 06:53:51 -0400 Subject: Python community "welcoming" feedback Message-ID: A while back I pointed out some challenges for the Python community's intake of new coders. Mostly focusing on IRC and the Python e-mail list. Several people have stepped up their "welcome" game and I've been very impressed with the way things are going. Great job! Leam From rhodri at kynesim.co.uk Wed Oct 4 06:57:21 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 4 Oct 2017 11:57:21 +0100 Subject: The "loop and a half" In-Reply-To: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> On 04/10/17 10:01, Robin Becker wrote: > Given the prevalence of the loop and a half idea in python I wonder why > we don't have a "do" or "loop" statement to start loops without a test. See PEP 315. Guido's rejection note is here: https://mail.python.org/pipermail/python-ideas/2013-June/021610.html -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Wed Oct 4 07:07:35 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 12:07:35 +0100 Subject: newb question about @property In-Reply-To: <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: On 04/10/2017 06:32, Steve D'Aprano wrote: > On Wed, 4 Oct 2017 02:00 am, bartc wrote: > >> Does all this advanced stuff (which I don't understand and which doesn't >> look very appealing either; hopefully I will never come across such >> code) still count as programming? > > I could not have hoped to see a more perfect example of the Blub effect in > action if I had invented it myself. > > > As long as our hypothetical Blub programmer is looking > down the power continuum, he knows he's looking down. > Languages less powerful than Blub are obviously less > powerful, because they're missing some feature he's used > to. But when our hypothetical Blub programmer looks in > the other direction, up the power continuum, he doesn't > realize he's looking up. What he sees are merely weird > languages. He probably considers them about equivalent > in power to Blub, but with all this other hairy stuff > thrown in as well. Blub is good enough for him, because > he thinks in Blub. > > > http://www.paulgraham.com/avg.html I've seen that example brought up before. It still doesn't cut any ice. You might as well be condescending of someone who finds Joyce or Proust unreadable, and prefers McBain, Simenon or Chandler. (Sorry, can't think of any modern pulp novelists). It is just being elitist. I have a preference for keeping things simple and avoiding unnecessary complexity. But with programming languages many do have a penchant for the latter. As an example, a recent discussion on comp.lang.c was about implementing named constants. I proposed one very simple way of doing it, other people were talking about using #define, enum, const, static const, or importing constexpr and special rules for 'const' from C++. All unsatisfactory and each having their own problems. For that matter, I don't think Python has such a feature either. So that you write for example: const C = 123345 and then whenever C appears within the code, it's implemented as: LOAD_CONST (123345) I'm pretty sure that there are very complicated ways of achieving something similar, maybe with all your decorators, or using PyMacro or whatever. But not doing it straightforwardly. [Python's design makes a simple implementation harder.] Anyway, what I'm saying is that languages sometimes neglect the basics in favour of all this other stuff. > You don't think that adding a cache for an expensive function is programming? > > If you had ten expensive functions, and you wanted to add a cache to each of > them, would you write out ten separate caches (probably copying and pasting > the code each time)? > > Or would you write a function that builds a cache and adds it to the expensive > function *once*, then call it ten times, once for each function being > wrapped? I'm trying to think of a real example where I've had to add a cache to to a function, whatever that even means (memoisation?). Is it looking to see if a certain combination of parameters has been used before and retrieving the return value that resulted on that occasion without having to redo the calculation? In my sort of coding that would need be done on a per-function basis because there would be other considerations. It does still sound like adding a turbo-charger to your engine, rather than actually going anywhere. That at least might be useful, but it's still not driving. But if the ultimate purpose is more speed, then piling on yet more code and more layers may be counter-productive! > That you think that this is not programming is an indictment off your > programming skills. These sorts of functional programming techniques go back > to the 1950s, Lisp is literally the second oldest high-level language ever > (only Fortran is older). You may or may not have been a hotshot in your > little corner of the programming world, but there's an entire world out > there. What other languages apart from Python have equivalent features to decorators and, what's the other one, descriptors? Apart from Lisp. -- bartc From ben+python at benfinney.id.au Wed Oct 4 07:15:47 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 04 Oct 2017 22:15:47 +1100 Subject: Good virtualenv and packaging tutorials for beginner? References: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> Message-ID: <85zi972mfw.fsf@benfinney.id.au> Leam Hall writes: > Folks on IRC have suggested using virtualenv to test code under > different python versions. Sadly, I've not found a virtualenv tutorial > I understand. Anyone have a link to a good one? The Python Packaging Authority has a guide which seems good to me. The standard library documentation for the ?venv? library is essential. > The next step will be to figure out how to package a project; a good > tutorial URL would be appreciated on that, too. Follow the documentation maintained by the Python Packaging Authority . -- \ ?Courage is not the absence of fear, but the decision that | `\ something else is more important than fear.? ?Ambrose Redmoon | _o__) | Ben Finney From frank at chagford.com Wed Oct 4 07:22:22 2017 From: frank at chagford.com (Frank Millman) Date: Wed, 4 Oct 2017 13:22:22 +0200 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: "Stefan Ram" wrote in message news:VBA-loops-20171004114215 at ram.dialup.fu-berlin.de... > > Robin Becker writes: > >Given the prevalence of the loop and a half idea in python I wonder why > >we don't > >have a "do" or "loop" statement to start loops without a test. > > VBA has quite an orthogonal way to build loop control: > > pre-checked positive: > > Do While ... > ... > Loop > > post-checked positive: > > Do > ... > Loop While ... > > pre-checked negative: > > Do Until ... > ... > Loop > > post-checked negative > > Do > ... > Loop Until ... > > "endless": > > Do > ... > Loop There is a language called Pick Basic (which most of you have never heard of) which has a single syntax covering all possibilities - LOOP {optional statements} WHILE {condition} or UNTIL {condition} DO {optional statements} REPEAT The DO and the REPEAT are necessary because it does not use indentation or braces to terminate blocks, it uses keywords. Frank Millman From bc at freeuk.com Wed Oct 4 07:29:41 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 12:29:41 +0100 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: On 04/10/2017 11:42, Stefan Ram wrote: > Robin Becker writes: >> Given the prevalence of the loop and a half idea in python I wonder why we don't >> have a "do" or "loop" statement to start loops without a test. > > VBA has quite an orthogonal way to build loop control: > > pre-checked positive: > > Do While ... > ... > Loop > > post-checked positive: > > Do > ... > Loop While ... > > pre-checked negative: > > Do Until ... > ... > Loop > > post-checked negative > > Do > ... > Loop Until ... > > "endless": > > Do > ... > Loop None of them appear to be loop-and-a-halves. (Note that your reverse-indentation style is confusing! Imagine that Python was written like this: for i in R: print (i) ) Anyway I think what is needed in Python for a proper loop-and-a-half is this: while: s1 s2 do cond: s3 s4 s1, s2, s3, s4 are executed in sequence while cond is true. When cond is false, only s1, s2 are executed in that iteration, and the loop terminates. (A few syntactical details to be worked out such as whether while, do and else line up or not.) This is equivalent I think to the current: while 1: s1 s2 if cond: s3 s4 else: break But this would be a less satisfactory, ad-hoc approach. For example, there can be several such if-blocks within the same loop. They could be nested. There might not be any, so can't at a glance distinguish an endless loop from a loop-and-a-half. My proposed while-do has a less chaotic structure. It wouldn't preclude 'break' from still being used, but there would be less need. -- bartc From leamhall at gmail.com Wed Oct 4 08:30:40 2017 From: leamhall at gmail.com (leam hall) Date: Wed, 4 Oct 2017 08:30:40 -0400 Subject: Good virtualenv and packaging tutorials for beginner? In-Reply-To: <85zi972mfw.fsf@benfinney.id.au> References: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> <85zi972mfw.fsf@benfinney.id.au> Message-ID: On Wed, Oct 4, 2017 at 7:15 AM, Ben Finney wrote: > Leam Hall writes: > > > Folks on IRC have suggested using virtualenv to test code under > > different python versions. Sadly, I've not found a virtualenv tutorial > > I understand. Anyone have a link to a good one? > > The Python Packaging Authority has a guide > packages/#creating-virtual-environments> > which seems good to me. > > The standard library documentation for the ?venv? library > is essential. > > > The next step will be to figure out how to package a project; a good > > tutorial URL would be appreciated on that, too. > > Follow the documentation maintained by the Python Packaging Authority > . > > Ben, thanks! I'm off to print and study... Leam From neilc at norwich.edu Wed Oct 4 08:33:32 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Wed, 4 Oct 2017 12:33:32 +0000 (UTC) Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-04, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >>Maybe this way: Use a while-loop and try-catch to get values >>from an iterator until exhausted, and then introduce the >>for-loop as an abbreviation for that. > > # while-loop > > iterable = range( 3 ) > > iterator = iter( iterable ) > try: > while True: > i = next( iterator ) > print( i ) > except StopIteration: > pass > > iterator = iter( iterable ) > try: > while True: > i = next( iterator ) > print( i ) > except StopIteration: > pass > > # for-loop as-an-abbreviation > > iterable = range( 3 ) > > for i in iterable: > print( i ) > > for i in iterable: > print( i ) That looks the opposite of an excellent way to teach iterators. If you insist they understand the iterator protocol and exception handling first they're bound to think iteration is a hovercraft full of eels. -- Neil Cerutti From ben.usenet at bsb.me.uk Wed Oct 4 08:34:27 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 04 Oct 2017 13:34:27 +0100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: <877ewb6qi4.fsf@bsb.me.uk> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Steve D'Aprano writes: >>For-each loops are MUCH easier to understand, and should be taught first. > > I prefer a bottom-up approach. > > For loops are based on iterators. > > So, "bottom-up" in this case means: iterators should be > taught before for-loops. > > But iterators are too abstract to be taught very early. I think this may be a problem with your style. From your other postings, I think you value precision and exactness over broad understanding, and maybe you teach like that. I my view, it's possible to explain enough about iterators to understand a huge range of for loops without having to go into the gory details. I find the image of a conjurer pulling knotted hankies out of a hat a good one -- they may go on forever and you don't know if there is rabbit in there knotting them and deciding which colour comes next. > But I will teach iterators and for loops not much later than > while-loop. > > Maybe this way: Use a while-loop and try-catch to get values > from an iterator until exhausted, and then introduce the > for-loop as an abbreviation for that. That sounds very complicated, but I think I favour the other extreme to you. -- Ben. From rhodri at kynesim.co.uk Wed Oct 4 08:46:53 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 4 Oct 2017 13:46:53 +0100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <00d4788d-0bbc-7113-f500-4c0c5d64e189@kynesim.co.uk> On 04/10/17 12:07, bartc wrote: > I've seen that example brought up before. It still doesn't cut any ice. > > You might as well be condescending of someone who finds Joyce or Proust > unreadable, and prefers McBain, Simenon or Chandler. (Sorry, can't think > of any modern pulp novelists). I don't think your comparison is appropriate. Joyce and Proust strike me as the literary equivalent of Perl or APL; very clever but nearly unreadable even for experts. No, think rather of Terry Pratchett. (Almost) anyone can read a Pratchett novel and enjoy it. Most people will not even notice maybe half the jokes. This is most obvious with The Colour of Magic and The Light Fantastic, where long-time Fantasy readers will spot Fafhrd and the Grey Mouser, recognise the tropes that are cheerfully being exploded, and so on. You can write decent Python without using decorators, properties, metaclasses or comprehensions in much the same way that you can enjoy Pratchett without ever having heard of Fritz Lieber. For a straightforward enough problem, writing your Python as if it was C won't even cause you any trouble. But if you think using these extra tools isn't programming, you are as flat out wrong as if you think Small Gods is just about a deity having to work on being believed in. -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Wed Oct 4 08:50:50 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 13:50:50 +0100 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> Message-ID: On 04/10/2017 11:57, Rhodri James wrote: > On 04/10/17 10:01, Robin Becker wrote: >> Given the prevalence of the loop and a half idea in python I wonder >> why we don't have a "do" or "loop" statement to start loops without a >> test. > > See PEP 315.? Guido's rejection note is here: > https://mail.python.org/pipermail/python-ideas/2013-June/021610.html Oh, OK, similar to what I proposed earlier. That it was rejected is not a surprise. Programming languages seem to hate special forms for basic constructs such as looping, even though they can express more directly exactly what the coder intended. But this is what it says about it: "Please reject the PEP. More variations along these lines won't make the language more elegant or easier to learn. They'd just save a few hasty folks some typing while making others who have to read/maintain their code wonder what it means." Exactly the same could be said of pretty much any of the advanced features that /have/ been added. -- bartc From ben.usenet at bsb.me.uk Wed Oct 4 08:59:31 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 04 Oct 2017 13:59:31 +0100 Subject: The "loop and a half" References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: <87zi975arw.fsf@bsb.me.uk> ram at zedat.fu-berlin.de (Stefan Ram) writes: > bartc writes: >>Note that your reverse-indentation style is confusing! > > In Python, indentation can be significant. > > Sometimes, some lines in Python must be indented by 0. > > This dictates that Python code cannot be indented > in posts to differentiate it from the natural-language > body of the post. Therefore, it's rather the > natural-language body that has to be indented. It does not /dictate/ it. It just makes it convenient for people with certain news readers. If you posted def f(x): return x*x I can select the block starting in column 2 and paste it into a python REPL quite easily. I prefer software that copes with existing conventions rather than have the conventions change to make it easy for the software. But if I felt I should not indent my python code, I would certainly not indent my text commentary. -- Ben. From robin at reportlab.com Wed Oct 4 09:02:22 2017 From: robin at reportlab.com (Robin Becker) Date: Wed, 4 Oct 2017 14:02:22 +0100 Subject: The "loop and a half" In-Reply-To: <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> Message-ID: <4bfba6c7-bbd1-3a12-7be5-8715542ae9dc@chamonix.reportlab.co.uk> On 04/10/2017 11:57, Rhodri James wrote: > On 04/10/17 10:01, Robin Becker wrote: >> Given the prevalence of the loop and a half idea in python I wonder why we >> don't have a "do" or "loop" statement to start loops without a test. > > See PEP 315.? Guido's rejection note is here: > https://mail.python.org/pipermail/python-ideas/2013-June/021610.html > seems fair enough; I suppose the cost is negligible or perhaps there's peephole optimization for this common case. -- Robin Becker From skip.montanaro at gmail.com Wed Oct 4 09:07:44 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 4 Oct 2017 08:07:44 -0500 Subject: How best to initialize in unit tests? Message-ID: Suppose you want to test a package (in the general sense of the word, not necessarily a Python package). You probably have specific unit tests, maybe some doctests scattered around in doc strings. Further, suppose that package requires you call an initialize function of some sort. Where does that go? I know about setUp and setUpClass methods in unittest.TestCase. Is order of execution of doctests deterministic across modules (say, when tests are run through nosetests)? In my case, I didn't know what order things would be called, so I added a call to initialize() at the start of every doctest and added a setUpClass class method to all my TestCase subclasses. Just in case. It worked okay because my initialize function can be called multiple times. What about situations where it can only be called once? Do you have to define some sort of super_initialize() function for your tests which guarantees that your initialize function is called precisely once? This all seems rather messy. I'm open to better ways to do this, but as I've only had one cup of coffee this morning, no spark of insight has zapped my frontal cortex as yet. Thx, Skip From bc at freeuk.com Wed Oct 4 09:14:38 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 14:14:38 +0100 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> <4bfba6c7-bbd1-3a12-7be5-8715542ae9dc@chamonix.reportlab.co.uk> Message-ID: On 04/10/2017 14:02, Robin Becker wrote: > On 04/10/2017 11:57, Rhodri James wrote: >> On 04/10/17 10:01, Robin Becker wrote: >>> Given the prevalence of the loop and a half idea in python I wonder >>> why we don't have a "do" or "loop" statement to start loops without a >>> test. >> >> See PEP 315.? Guido's rejection note is here: >> https://mail.python.org/pipermail/python-ideas/2013-June/021610.html >> > seems fair enough; I suppose the cost is negligible or perhaps there's > peephole optimization for this common case. Read the thread. Not having a dedicated feature means I counted at least half a dozen different ideas for implementing what the OP really wanted to do, which was expressed using C-like syntax. None of which were as obvious. And that C-like one worked because it could use an assignment within a while-condition. -- bartc From christopher_reimer at icloud.com Wed Oct 4 09:27:19 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Wed, 04 Oct 2017 06:27:19 -0700 Subject: How best to initialize in unit tests? In-Reply-To: References: Message-ID: On Oct 4, 2017, at 6:07 AM, Skip Montanaro wrote: > > Suppose you want to test a package (in the general sense of the word, > not necessarily a Python package). You probably have specific unit > tests, maybe some doctests scattered around in doc strings. Further, > suppose that package requires you call an initialize function of some > sort. Where does that go? I know about setUp and setUpClass methods in > unittest.TestCase. Is order of execution of doctests deterministic > across modules (say, when tests are run through nosetests)? > > In my case, I didn't know what order things would be called, so I > added a call to initialize() at the start of every doctest and added a > setUpClass class method to all my TestCase subclasses. Just in case. > It worked okay because my initialize function can be called multiple > times. What about situations where it can only be called once? Do you > have to define some sort of super_initialize() function for your tests > which guarantees that your initialize function is called precisely > once? > > This all seems rather messy. I'm open to better ways to do this, but > as I've only had one cup of coffee this morning, no spark of insight > has zapped my frontal cortex as yet. > > Thx, > > Skip > -- > https://mail.python.org/mailman/listinfo/python-list Seems like you?re overthinking this. You should be able to unit test any part of your code in isolation. If not, you need to refactor your code. I generally don?t use test classes in pytest. Chris R. From ian.g.kelly at gmail.com Wed Oct 4 09:41:44 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Oct 2017 07:41:44 -0600 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, Oct 4, 2017 at 5:07 AM, bartc wrote: > It is just being elitist. I have a preference for keeping things simple and > avoiding unnecessary complexity. But with programming languages many do have > a penchant for the latter. > > As an example, a recent discussion on comp.lang.c was about implementing > named constants. I proposed one very simple way of doing it, other people > were talking about using #define, enum, const, static const, or importing > constexpr and special rules for 'const' from C++. All unsatisfactory and > each having their own problems. > > For that matter, I don't think Python has such a feature either. So that you > write for example: > > const C = 123345 > > and then whenever C appears within the code, it's implemented as: > > LOAD_CONST (123345) > > I'm pretty sure that there are very complicated ways of achieving something > similar, maybe with all your decorators, or using PyMacro or whatever. But > not doing it straightforwardly. [Python's design makes a simple > implementation harder.] Python has the simplest named constants of all: C = 12345 As long as you don't subsequently change it, it's a constant. And it's very simple because it works just like any other variable. Python also has a particularly flexible Enum implementation, but if you don't want it then don't use it. >> You don't think that adding a cache for an expensive function is >> programming? >> >> If you had ten expensive functions, and you wanted to add a cache to each >> of >> them, would you write out ten separate caches (probably copying and >> pasting >> the code each time)? >> >> Or would you write a function that builds a cache and adds it to the >> expensive >> function *once*, then call it ten times, once for each function being >> wrapped? > > > I'm trying to think of a real example where I've had to add a cache to to a > function, whatever that even means (memoisation?). You've never used a dynamic programming algorithm? >> That you think that this is not programming is an indictment off your >> programming skills. These sorts of functional programming techniques go >> back >> to the 1950s, Lisp is literally the second oldest high-level language ever >> (only Fortran is older). You may or may not have been a hotshot in your >> little corner of the programming world, but there's an entire world out >> there. > > > What other languages apart from Python have equivalent features to > decorators and, what's the other one, descriptors? Apart from Lisp. Literally any language with first-class function types, whether they have the @decorator-style syntactic sugar for it or not. Descriptors are a bit unique to Python, I'll grant, but mostly they're just used in the form of properties. Here's a list of languages that support properties: https://en.wikipedia.org/wiki/Property_(programming)#Example_syntax From jpolo at mail.usf.edu Wed Oct 4 09:44:47 2017 From: jpolo at mail.usf.edu (john polo) Date: Wed, 4 Oct 2017 08:44:47 -0500 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/3/2017 5:08 PM, Stefan Ram wrote: > Steve D'Aprano writes: >> On Wed, 4 Oct 2017 04:45 am, Rhodri James wrote: >>> On 03/10/17 18:29, Stefan Ram wrote: >>>> Is this the best way to write a "loop and a half" in Python? >>> Define "best". >> I'd start with "define loop and a half". > I encountered this phenomenon decades ago. But only > some years ago, I heard someone using a special word > for it! Let me try to find it ... > > Yes, it was Mehran Sahami who used it, possibly in one of > his lectures of the course "Programming Methodology" > (CS 106A) at "Stanford Engineering". > > > ... > > The history of this problem goes way back into the past. > In the 1950s (and possibly still inthe 1960s), such mundane > loops were research topics, and Knuth and others wrote > research articles about how to write such a loop in such a > way that no test or statement has to be repeated. > (I sometimes liked to read old magazines from those years.) > > Thanks for all the answers so far! I'm adding to what Stefan wrote, simply to point out how a newb like me came across this issue formally, besides actually dealing with this in my practice. In Python Programming Fundamentals 2nd ed., the author, Kent D. Lee, brings up loop and a half in ch. 3, Repetitive Tasks (p. 82). He wrote: "Whether you are writing code in Python or some other language, this Reading Records From a File pattern comes up over and over again. It is sometimes called the loop and a half problem. The idea is that you must attempt to read a line from the ?le before you know whether you are at the end of ?le or not. This can also be done if a boolean variable is introduced to help with the while loop. This boolean variable is the condition that gets you out of the while loop and the ?rst time through it must be set to get your code to execute the while loop at least one." best, John From p.f.moore at gmail.com Wed Oct 4 09:46:45 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 14:46:45 +0100 Subject: Good virtualenv and packaging tutorials for beginner? In-Reply-To: References: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> <85zi972mfw.fsf@benfinney.id.au> Message-ID: On 4 October 2017 at 13:30, leam hall wrote: > On Wed, Oct 4, 2017 at 7:15 AM, Ben Finney > wrote: > >> Leam Hall writes: >> >> > Folks on IRC have suggested using virtualenv to test code under >> > different python versions. Sadly, I've not found a virtualenv tutorial >> > I understand. Anyone have a link to a good one? >> >> The Python Packaging Authority has a guide >> > packages/#creating-virtual-environments> >> which seems good to me. >> >> The standard library documentation for the ?venv? library >> is essential. >> >> > The next step will be to figure out how to package a project; a good >> > tutorial URL would be appreciated on that, too. >> >> Follow the documentation maintained by the Python Packaging Authority >> . >> >> > Ben, thanks! I'm off to print and study... Leam - if the guide is difficult to follow, feel free to ask here, or raise issues on the project tracker at https://github.com/pypa/python-packaging-user-guide/. Most of us working on packaging are way too involved in the details to know how it really feels to a newcomer to be faced with all this stuff, so if you have any comments (or even better, suggestions as to how we could improve things) that would be immensely valuable. Paul From p.f.moore at gmail.com Wed Oct 4 09:54:53 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 14:54:53 +0100 Subject: How best to initialize in unit tests? In-Reply-To: References: Message-ID: I've not had this problem myself, but py.test has the idea of "autouse fixtures" which would work for this situation. Define your setup call in a function, declare it with the pytest.fixture decorator with autouse=True, and it'll be run before every test. The declaration goes in a conftest.py file alongside your test files. I don't know if unittest or doctest has anything similar, but I suspect not (or at least not something as simple). Whether you're comfortable with the level of magic behaviour pytest has is probably something you'll have to decide for yourself (although I believe it does discover unittest and doctest tests, so you don't need to change all your tests over). There's an example of an autouse fixture in pip's test suite, although it's massively more complex than what you're describing, so don't be put off by the size of it :-) Paul On 4 October 2017 at 14:07, Skip Montanaro wrote: > Suppose you want to test a package (in the general sense of the word, > not necessarily a Python package). You probably have specific unit > tests, maybe some doctests scattered around in doc strings. Further, > suppose that package requires you call an initialize function of some > sort. Where does that go? I know about setUp and setUpClass methods in > unittest.TestCase. Is order of execution of doctests deterministic > across modules (say, when tests are run through nosetests)? > > In my case, I didn't know what order things would be called, so I > added a call to initialize() at the start of every doctest and added a > setUpClass class method to all my TestCase subclasses. Just in case. > It worked okay because my initialize function can be called multiple > times. What about situations where it can only be called once? Do you > have to define some sort of super_initialize() function for your tests > which guarantees that your initialize function is called precisely > once? > > This all seems rather messy. I'm open to better ways to do this, but > as I've only had one cup of coffee this morning, no spark of insight > has zapped my frontal cortex as yet. > > Thx, > > Skip > -- > https://mail.python.org/mailman/listinfo/python-list From p.f.moore at gmail.com Wed Oct 4 10:01:55 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 15:01:55 +0100 Subject: The "loop and a half" In-Reply-To: <4bfba6c7-bbd1-3a12-7be5-8715542ae9dc@chamonix.reportlab.co.uk> References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> <4bfba6c7-bbd1-3a12-7be5-8715542ae9dc@chamonix.reportlab.co.uk> Message-ID: On 4 October 2017 at 14:02, Robin Becker wrote: > On 04/10/2017 11:57, Rhodri James wrote: >> >> On 04/10/17 10:01, Robin Becker wrote: >>> >>> Given the prevalence of the loop and a half idea in python I wonder why >>> we don't have a "do" or "loop" statement to start loops without a test. >> >> >> See PEP 315. Guido's rejection note is here: >> https://mail.python.org/pipermail/python-ideas/2013-June/021610.html >> > seems fair enough; I suppose the cost is negligible or perhaps there's > peephole optimization for this common case. There is: >>> def f(): ... while True: ... pass ... >>> import dis >>> dis.dis(f) 2 0 SETUP_LOOP 4 (to 6) 3 >> 2 JUMP_ABSOLUTE 2 4 POP_BLOCK >> 6 LOAD_CONST 0 (None) 8 RETURN_VALUE Paul From dvl at psu.edu Wed Oct 4 10:19:41 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Wed, 4 Oct 2017 10:19:41 -0400 Subject: Python-list Digest, Vol 169, Issue 5 In-Reply-To: mailman.893.1507117203.2818.python-list@python.org References: Message-ID: <1507126781l.12451984l.0l@psu.edu> On Wed, Oct 4, 2017 14:54, Chris Angelico wrote: > On Wed, Oct 4, 2017 at 2:48 PM, Steve D'Aprano > wrote: >> On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: >> >> You know, you don't HAVE to economize on letters. It's okay to call >> your parameters "prompt" instead of "prmt". Remember, that's part of >> your API. >> >> >> Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of vwel wth >> nt nuff 4 vrybdy. >> > >And then English will turn into an abjad like Arabic. Yes, I see how >it is. Python is a sneaky plan by Islamic extremists to make us use up >all our vowels. > >(I'd end the post there, except that I've learned from past experience >that there is no statement so ridiculous that people will know for >sure that you're joking. So for the humour-impaired: THIS POST WAS A >JOKE. Okay? Good.) > > Gotta watch out for all those Islamic extremists. Somewhere along the line when I wasn't paying attention, they got us all using Arabic numerals. Roger Christman Pennsylvania State University From skip.montanaro at gmail.com Wed Oct 4 10:20:26 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 4 Oct 2017 09:20:26 -0500 Subject: How best to initialize in unit tests? In-Reply-To: References: Message-ID: > I've not had this problem myself, but py.test has the idea of "autouse > fixtures" which would work for this situation. Define your setup call > in a function, declare it with the pytest.fixture decorator with > autouse=True, and it'll be run before every test. The declaration goes > in a conftest.py file alongside your test files. Thanks. I'm not a py.test user, but it turns out that nose (which I do use) appears to have something similar: https://nose.readthedocs.io/en/latest/doc_tests/test_init_plugin/init_plugin.html Thanks for the nudge in the right direction. Skip From steve+python at pearwood.info Wed Oct 4 10:53:59 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 01:53:59 +1100 Subject: How best to initialize in unit tests? References: Message-ID: <59d4f609$0$14973$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 12:07 am, Skip Montanaro wrote: > Suppose you want to test a package (in the general sense of the word, > not necessarily a Python package). I'm... not sure I understand. Given that we're discussing Python, in a Python group, what are senses are relevant? > You probably have specific unit > tests, maybe some doctests scattered around in doc strings. Further, > suppose that package requires you call an initialize function of some > sort. Where does that go? Ew. Sounds like a badly designed package. Importing the package should initialise it. We write: import math x = math.sin(1) not: import math math.initialise() # I actually do want to use the module I just imported x = math.sin(1) # fails mysteriously if we forget to initialise first If you see a code unit (a package, a module, a class, or even a function) that requires a manual call to something called "initialise" or equivalent, you probably have a poorly designed code unit. (I suppose there may be -- hopefully rare -- cases where that's unavoidable, or less bad than the alternative.) But given that you have to test the code you have, not the code you wish you had... Where are the doc tests? (1) If they are in the module you are testing itself, you will probably need to call initialise() at the start of each doc string before running any tests. That's because doctest makes a shallow copy of the module's globals before running the tests, so any global state initialised in one doc string will be lost when the next is called. https://docs.python.org/2/library/doctest.html#what-s-the-execution-context (2) If the doc tests are in a separate file, the entire file will be run in a single context. You probably already start the file by importing the module you are testing: >>> import Module so just add the initalise call after that. In unit tests, you import the module, so just call initialise immediately: import Module Module.initialise() and then you don't have to worry about it in the test cases themselves. > I know about setUp and setUpClass methods in > unittest.TestCase. Is order of execution of doctests deterministic > across modules (say, when tests are run through nosetests)? I can't speak for nose, but doctest runs the tests in undocumented, and therefore arbitrary, order. > In my case, I didn't know what order things would be called, so I > added a call to initialize() at the start of every doctest and added a > setUpClass class method to all my TestCase subclasses. Just in case. > It worked okay because my initialize function can be called multiple > times. What about situations where it can only be called once? If the initialise function isn't idempotent, there may be subtle problems with doctest if making a shallow copy of the module globals isn't enough to reset all the initialised state. > Do you > have to define some sort of super_initialize() function for your tests > which guarantees that your initialize function is called precisely > once? No. > This all seems rather messy. Indeed. And now you know why importing a module/package should be sufficient to initialise it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Wed Oct 4 11:03:39 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 16:03:39 +0100 Subject: newb question about @property In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <5L6BB.1362664$ji4.1213053@fx10.am4> On 04/10/2017 14:41, Ian Kelly wrote: > On Wed, Oct 4, 2017 at 5:07 AM, bartc wrote: >> For that matter, I don't think Python has such a feature either. So that you >> write for example: >> >> const C = 123345 >> >> and then whenever C appears within the code, it's implemented as: >> >> LOAD_CONST (123345) > Python has the simplest named constants of all: > > C = 12345 > > As long as you don't subsequently change it, it's a constant. And it's > very simple because it works just like any other variable. Yes, but you don't want it to work like other variables! In other languages, a known compile-time constant is essential in certain contexts, but even in Python there are advantages to such a feature, for example in more opportunities for optimisation. Even if it's just so it can use LOAD_CONST instead of LOAD_GLOBAL (as such constants often are global). The problems with doing this in current CPython are first, that all such identifiers are dynamic: you can assign anything to them at any time. Also that some may be inside imported files, which the byte-code compiler will not know about until its too late. (There are ways around those but it would be an impossible sell when most people are not convinced that lightweight named constants are useful.) >> I'm trying to think of a real example where I've had to add a cache to to a >> function, whatever that even means (memoisation?). > > You've never used a dynamic programming algorithm? I had to look up what it means, but I still didn't see any practical examples. Probably I've done such coding, but I didn't give it a fancy name, and more likely just considered it data caching. > Descriptors are a bit unique to Python, I'll grant, but mostly they're > just used in the form of properties. Here's a list of languages that > support properties: > https://en.wikipedia.org/wiki/Property_(programming)#Example_syntax "A property, in some object-oriented programming languages, is a special sort of class member, intermediate in functionality between a field (or data member) and a method." But Python has some problems just in using fields. If you wanted say a data type with two fields called x and y, then AFAIK you just create any suitable class, but you don't need to specifically declare those two fields: class any(): pass p=any() p.x=10 p.y=20 But you also do this: p.z=30 and it magically acquires a third field! Or you want to modify p.x, but accidentally type: p.c=40 No error. Some would perceive all this as an advantage, but it means you can't just declare a lightweight struct or record 'Point' with exactly two fields x and y. You have to use other solutions ('namedtuples' or whatever, which probably are immutable so that don't work the same way). This is another example of neglecting the basics, but going for more advanced, perhaps more sexy features instead. Result? You can't just look at my 'any' class and see what fields it uses. You can't even just look at the static source code. You have to run the program to find out. And it might be different each time. -- bartc From steve+python at pearwood.info Wed Oct 4 11:08:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:08:06 +1100 Subject: Constants [was Re: newb question about @property] References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d4f958$0$14942$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 12:41 am, Ian Kelly wrote: > Python has the simplest named constants of all: > > C = 12345 > > As long as you don't subsequently change it, it's a constant. And it's > very simple because it works just like any other variable. I do feel that the lack of constants[1] is a (mild) weakness of Python. In small projects, if you want a constant: pi = 3.1415 and you don't want to change it, it is easy enough to remember to just not change it. Especially if you follow a naming convention like using ALLCAPS for constants. But in large projects, especially those where you cannot trust every module in the project to obey the naming convention, I can see that this lack might contribute to the perception, if not the fact, of Python being a bit too unsafe for big projects. We have read-only attributes in classes, but not read-only names in modules. That makes me a little bit sad. Back in the day when I used Pascal regularly, I recall having the compiler complain when I accidentally tried to assign a new value to a constant. But I was a much less experienced coder back then. In all honesty, I can't remember the last time I accidentally reassigned to something intended as a constant. Nevertheless, constants are a safety net I would appreciate. [1] By which I mean names which can only be bound once, but not rebound. This is not about mutability, it is about whether or not the name can be rebound. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 4 11:08:17 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:08:17 +1100 Subject: Multithreaded compression/decompression library with python bindings? References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> Message-ID: <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 08:19 pm, Thomas Nyberg wrote: > Hello, > > I was wondering if anyone here knew of any python libraries with > interfaces similar to the bzip2 module which is also multithreaded in > (de)compression? Something along the lines of (say) the pbip2 program > but with bindings for python? pbip2? Never heard of it, and googling comes up with nothing relevant. Got a link? > Obviously the multi-threaded part will > need to be handled by the underlying library code and not the python > code itself. Why obviously? > > If that's too much to ask (I haven't found anything myself after all), > if anyone here happens to have any experience using similar C++ (or > possibly C) libraries I am certainly open for any recommendations. :) > > Thanks in advance! > Thomas -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 4 11:13:54 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:13:54 +1100 Subject: The "loop and a half" References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> <22cdc062-4aa2-aea8-e019-234519859583@kynesim.co.uk> Message-ID: <59d4fab4$0$14941$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 11:50 pm, bartc wrote: > But this is what it says about it: > > "Please reject the PEP. More variations along these lines won't make the > language more elegant or easier to learn. They'd just save a few hasty > folks some typing while making others who have to read/maintain their > code wonder what it means." > > Exactly the same could be said of pretty much any of the advanced > features that have been added. People can say any old rubbish they like, and frequently do. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From skip.montanaro at gmail.com Wed Oct 4 11:23:48 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 4 Oct 2017 10:23:48 -0500 Subject: How best to initialize in unit tests? In-Reply-To: <59d4f609$0$14973$b1db1813$d948b532@news.astraweb.com> References: <59d4f609$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, Oct 4, 2017 at 9:53 AM, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 12:07 am, Skip Montanaro wrote: > >> Suppose you want to test a package (in the general sense of the word, >> not necessarily a Python package). > > I'm... not sure I understand. Given that we're discussing Python, in a Python > group, what are senses are relevant? I have to test an entire application, which involves at least one package, unit tests and doctests. I suppose I should have used the word "application." My apology. > >> You probably have specific unit >> tests, maybe some doctests scattered around in doc strings. Further, >> suppose that package requires you call an initialize function of some >> sort. Where does that go? > > Ew. Sounds like a badly designed package. I will refer interested readers to the nose documentation: https://nose.readthedocs.io/en/latest/doc_tests/test_init_plugin/init_plugin.html where the author writes, "Many applications, especially those using web frameworks like Pylons or Django, can?t be tested without first being configured or otherwise initialized." My system, while not a web framework, does still require the caller of the client libraries (both Python and R, as it happens) to tell the system what server to contact. In the case of my test scenario, that is a Flask server on the local host with a SQLite database behind it, as opposed to a Flask server somewhere out on the network. (I actually attempt to test both client and server at the same time.) Skip From steve+python at pearwood.info Wed Oct 4 11:26:10 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:26:10 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 09:35 pm, Stefan Ram wrote: > Steve D'Aprano writes: >>For-each loops are MUCH easier to understand, and should be taught first. > > I prefer a bottom-up approach. I prefer an effective approach that starts with the simplest concepts first, not the most complex. > For loops are based on iterators. That's an implementation detail that is not necessary to understand in order to use for-loops. Python had for-loops for many years before it gained iterators, and even today in Python 3 you can iterate over non-iterators. py> class X: # not an iterator! ... def __getitem__(self, i): ... if i < 3: ... return str(i)*2 ... raise IndexError ... py> for obj in X(): ... print(obj) ... 00 11 22 X instances have no __iter__ or __next__ methods and are not iterators. > So, "bottom-up" in this case means: iterators should be > taught before for-loops. Why? As an educator, you should care about teaching topics in whatever order makes learning easier, not according to some theoretically pure order. Do you really think that it is hard to teach a simple "do this three times" example: for i in range(3): print("do this") example until you've covered iterators, exceptions, try...except StopIteration, and while loops? > But iterators are too abstract to be taught very early. I agree with that. So don't teach them. You don't need to care about the implementation of iteration in order to teach iteration. Most beginners probably won't write their own iterators for many years. > But I will teach iterators and for loops not much later than > while-loop. > > Maybe this way: Use a while-loop and try-catch to get values > from an iterator until exhausted, and then introduce the > for-loop as an abbreviation for that. Ah, I get it now... this is a cunning plan to make Python seem like a confusing, difficult language, so your students will learn Javascript instead. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.f.moore at gmail.com Wed Oct 4 11:33:15 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 16:33:15 +0100 Subject: newb question about @property In-Reply-To: <5L6BB.1362664$ji4.1213053@fx10.am4> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> Message-ID: On 4 October 2017 at 16:03, bartc wrote: > No error. Some would perceive all this as an advantage, but it means you > can't just declare a lightweight struct or record 'Point' with exactly two > fields x and y. You have to use other solutions ('namedtuples' or whatever, > which probably are immutable so that don't work the same way). > > This is another example of neglecting the basics, but going for more > advanced, perhaps more sexy features instead. It's another example of a consistent design philosophy (highly dynamic classes) that you might not like - possibly even enough that Python isn't the best language for you. It's not an advantage or a disadvantage, just an approach. Many people like it, you may not. Specifically, yes you can't "just declare a lightweight struct or record with exactly two fields". Python doesn't enforce things like that, but leaves it to the programmer(s) to agree on (and follow) conventions in the code. This means that certain classes of error (e.g. mistyping an attribute name) can go unnoticed until later than in other languages, but conversely it means that things like monkeypatching of 3rd party code are possible. The popularity of Python is evidence that the flexibility this allows is useful to many people. Expecting Python to have the same design as other languages does a disservice to both Python and those other languages. There are trade-offs in these things, and Python's choices won't be the best in all circumstances. All we can really say is that they have turned out to be pretty useful and popular in many situations... Paul From p.f.moore at gmail.com Wed Oct 4 11:34:25 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 16:34:25 +0100 Subject: Multithreaded compression/decompression library with python bindings? In-Reply-To: <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 4 October 2017 at 16:08, Steve D'Aprano wrote: > On Wed, 4 Oct 2017 08:19 pm, Thomas Nyberg wrote: > >> Hello, >> >> I was wondering if anyone here knew of any python libraries with >> interfaces similar to the bzip2 module which is also multithreaded in >> (de)compression? Something along the lines of (say) the pbip2 program >> but with bindings for python? > > pbip2? Never heard of it, and googling comes up with nothing relevant. > > Got a link? It's a typo. pbzip2 - a search found http://compression.ca/pbzip2/ but I don't know anything about it. Paul From steve+python at pearwood.info Wed Oct 4 11:35:16 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:35:16 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 12:44 am, john polo wrote: > I'm adding to what Stefan wrote, simply to point out how a newb like me > came across this issue formally, besides actually dealing with this in > my practice. > > In Python Programming Fundamentals 2nd ed., the author, Kent D. Lee, > brings up loop and a half in ch. 3, Repetitive Tasks (p. 82). He wrote: > > "Whether you are writing code in Python or some other language, this Reading > Records From a File pattern comes up over and over again. No it doesn't. Who is this Kent Lee, and is he actually fluent in Python? Because the following problem doesn't seem like it: > It is sometimes called > the loop and a half problem. The idea is that you must attempt to read a > line from the > ?le before you know whether you are at the end of ?le or not. Utter nonsense. There's no "must" here. I'll accept the remote possibility that maybe one time in a million you have to do what he says, but the other 999999 times you just read from the file in a for-loop: for line in file: process(line) > This can also be done > if a boolean variable is introduced to help with the while loop. This > boolean variable > is the condition that gets you out of the while loop and the ?rst time > through it must > be set to get your code to execute the while loop at least one." I've been programming in Python for twenty years, and I don't think I have ever once read from a file using a while loop. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From breamoreboy at gmail.com Wed Oct 4 11:36:51 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 4 Oct 2017 08:36:51 -0700 (PDT) Subject: The "loop and a half" In-Reply-To: References: <34adnYcGP-RS2UnEnZ2dnUU7-dnNnZ2d@giganews.com> <59d45a0b$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: <9214f99a-b292-4cdc-a2d6-cd5a69d966b6@googlegroups.com> On Wednesday, October 4, 2017 at 9:34:09 AM UTC+1, alister wrote: > On Wed, 04 Oct 2017 20:16:29 +1300, Gregory Ewing wrote: > > > Steve D'Aprano wrote: > >> On Wed, 4 Oct 2017 01:40 pm, Chris Angelico wrote: > >> > >>>You know, you don't HAVE to economize on letters. It's okay to call > >>>your parameters "prompt" instead of "prmt". Remember, that's part of > >>>your API. > >> > >> Whn u wste vwels lik that, dn't b srprsd whn u run ot n hav shrtg of > >> vwel wth nt nuff 4 vrybdy. > > > > I blame the Dutch. They're clearly using more than their fair share of > > the world's vowels. > > but that is compensated for by the welsh which don't seem to use any > Careful lad, that's my late mum's nation you're talking about. Of course I take it that you know the definition of a Welshman? It's an Irishman who couldn't swim. That is also a joke. > -- > Never trust an operating system. -- Kindest regards. Mark Lawrence. From steve+python at pearwood.info Wed Oct 4 11:52:40 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 02:52:40 +1100 Subject: How best to initialize in unit tests? References: <59d4f609$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d503c9$0$14958$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 02:23 am, Skip Montanaro wrote: [...] >> Ew. Sounds like a badly designed package. > > I will refer interested readers to the nose documentation: > > https://nose.readthedocs.io/en/latest/doc_tests/test_init_plugin/init_plugin.html > > where the author writes, "Many applications, especially those using > web frameworks like Pylons or Django, can?t be tested without first > being configured or otherwise initialized." I haven't used Pylons or Django, but frameworks are a different story to libraries, which is what I understood you to be discussing. In any case, I would expect any decent framework to automatically read from a configuration file, so that importing the framework automatically initialises it as well (provided it can find a configuration file, and if not, it should noisily complain unless explicitly silenced). > My system, while not a web framework, does still require the caller of > the client libraries (both Python and R, as it happens) to tell the > system what server to contact. And you do that globally? What if you want to talk to two servers? Anyway, whatever your reasons (good, bad or indifferent) for designing the system the way it is, you have to deal with it as it exists, not the way I would prefer it. Did you read my suggestion to just put the initialise call immediately after the import? import system system.initialise(server='foo') You can certainly do that with unittest; doing so in doctest requires a bit more nuance and care, but is still possible. If this doesn't solve your problem, I'm afraid I don't understand your problem. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.f.moore at gmail.com Wed Oct 4 11:56:16 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 16:56:16 +0100 Subject: The "loop and a half" In-Reply-To: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: On 4 October 2017 at 16:35, Steve D'Aprano wrote: > I've been programming in Python for twenty years, and I don't think I have > ever once read from a file using a while loop. Twenty years isn't long enough :-) The pattern the OP is talking about was common in "Jackson Structured Programming" from back in the 1980s. At the time (from what I recall, it *was* 35+ years ago after all!), it was a reasonably elegant way of handling reading of data that's structured as header subheader body body subheader body body while maintaining per-section style subtotals (I believe this is referred to as a "break report" in certain business contexts). Nowadays, I wouldn't code such a report like that. Like you say, I'd use a custom iterator or something similar. Although getting the details and edge cases right of such a pattern is still a surprisingly tricky problem. So I guess it's a good "learning to program" exercise, but the "while loop with break part way through" pattern for solving it is not really what I'd want to see taught to beginning Python programmers... Paul From rhodri at kynesim.co.uk Wed Oct 4 12:02:40 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 4 Oct 2017 17:02:40 +0100 Subject: newb question about @property In-Reply-To: References: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> Message-ID: <05870148-c97d-57a8-c4b6-6112b371dbd4@kynesim.co.uk> On 04/10/17 16:33, Paul Moore wrote: > On 4 October 2017 at 16:03, bartc wrote: >> No error. Some would perceive all this as an advantage, but it means you >> can't just declare a lightweight struct or record 'Point' with exactly two >> fields x and y. You have to use other solutions ('namedtuples' or whatever, >> which probably are immutable so that don't work the same way). >> >> This is another example of neglecting the basics, but going for more >> advanced, perhaps more sexy features instead. > It's another example of a consistent design philosophy (highly dynamic > classes) that you might not like - possibly even enough that Python > isn't the best language for you. > > It's not an advantage or a disadvantage, just an approach. Many people > like it, you may not. Specifically, yes you can't "just declare a > lightweight struct or record with exactly two fields". Actually you can: >>> class Point: ... __slots__ = ("x", "y") ... def __init__(self, x, y): ... self.x = x ... self.y = y ... def __str__(self): ... return "({0},{1})".format(self.x, self.y) ... >>> p = Point(3,4) >>> print(p) (3,4) >>> print(p.x) 3 >>> p.x = 7 >>> print(p) (7,4) >>> p.z = 2 Traceback (most recent call last): File "", line 1, in AttributeError: 'Point' object has no attribute 'z' I pretty much never bother to do this because (bart to the contrary) it isn't useful if you're thinking in Pythonic terms, but it can be done pretty easily. -- Rhodri James *-* Kynesim Ltd From ian.g.kelly at gmail.com Wed Oct 4 12:15:26 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 4 Oct 2017 10:15:26 -0600 Subject: Constants [was Re: newb question about @property] In-Reply-To: <59d4f958$0$14942$b1db1813$d948b532@news.astraweb.com> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <59d4f958$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, Oct 4, 2017 at 9:08 AM, Steve D'Aprano wrote: > But in large projects, especially those where you cannot trust every module in > the project to obey the naming convention, I can see that this lack might > contribute to the perception, if not the fact, of Python being a bit too > unsafe for big projects. We have read-only attributes in classes, but not > read-only names in modules. That makes me a little bit sad. Which brings up the point that you can hack it in if you want it. $ cat demo.py import sys class DemoModule: @property def foo(self): return 42 sys.modules['demo'] = DemoModule() $ python3 -c 'import demo; print(demo.foo); demo.foo = 14' 42 Traceback (most recent call last): File "", line 1, in AttributeError: can't set attribute From lab at 2020fresno.com Wed Oct 4 12:42:18 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Wed, 4 Oct 2017 09:42:18 -0700 Subject: Easier way to do this? Message-ID: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Looking for advice for what looks to me like clumsy code. I have a large csv (effectively garbage) dump.? I have to pull out sales information per employee and count them by price range. I've got my code working, but I'm thinking there must be a more refined way of doing this. ---snippet of what I have--- EMP1 = [0,0] EMP2 = [0,0] EMP3 = [0,0] for line in (inputfile): ??? content = line.split(",") ??? if content[18] == "EMP1": ??????? if float(content[24]) < 99.75: ??????????? EMP1[0] += 1 ??????? elif float(content[24]) > 99.74: ??????????? EMP1[1] += 1 ??? if content[18] == "EMP2": ??????? if float(content[24]) < 99.75: ??????????? EMP2[0] += 1 ??????? elif float(content[24]) > 99.74: ??????????? EMP2[1] += 1 ??? if content[18] == "EMP3": ??????? if float(content[24]) < 99.75: ??????????? EMP3[0] += 1 ??????? elif float(content[24]) > 99.74: ??????????? EMP3[1] += 1 and repeat if statements for the rest of 25+ employees.? I can make a list of the employees, but I'd prefer to pull them from the csv, as our turnover is rather high (however this is not important).? I'm thinking another "for employee in content[18]" should be there, but when I tried, my numbers were incorrect. Any help / advice is appreciated, Matt From p.f.moore at gmail.com Wed Oct 4 12:52:29 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 17:52:29 +0100 Subject: newb question about @property In-Reply-To: <05870148-c97d-57a8-c4b6-6112b371dbd4@kynesim.co.uk> References: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> <05870148-c97d-57a8-c4b6-6112b371dbd4@kynesim.co.uk> Message-ID: On 4 October 2017 at 17:02, Rhodri James wrote: > Actually you can: > >>>> class Point: > ... __slots__ = ("x", "y") > ... def __init__(self, x, y): > ... self.x = x > ... self.y = y > ... def __str__(self): > ... return "({0},{1})".format(self.x, self.y) > ... >>>> p = Point(3,4) >>>> print(p) > (3,4) >>>> print(p.x) > 3 >>>> p.x = 7 >>>> print(p) > (7,4) >>>> p.z = 2 > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'Point' object has no attribute 'z' > > I pretty much never bother to do this because (bart to the contrary) it > isn't useful if you're thinking in Pythonic terms, but it can be done pretty > easily. Good point. I'd forgotten that - like you say, it's not common to want to constrain things to this level in idiomatic Python code. Paul From p.f.moore at gmail.com Wed Oct 4 13:00:40 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Oct 2017 18:00:40 +0100 Subject: Constants [was Re: newb question about @property] In-Reply-To: References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <59d4f958$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 4 October 2017 at 17:15, Ian Kelly wrote: > On Wed, Oct 4, 2017 at 9:08 AM, Steve D'Aprano > wrote: >> But in large projects, especially those where you cannot trust every module in >> the project to obey the naming convention, I can see that this lack might >> contribute to the perception, if not the fact, of Python being a bit too >> unsafe for big projects. We have read-only attributes in classes, but not >> read-only names in modules. That makes me a little bit sad. > > Which brings up the point that you can hack it in if you want it. > > $ cat demo.py > import sys > > class DemoModule: > @property > def foo(self): > return 42 > > sys.modules['demo'] = DemoModule() > > $ python3 -c 'import demo; print(demo.foo); demo.foo = 14' > 42 > Traceback (most recent call last): > File "", line 1, in > AttributeError: can't set attribute I wonder - would the people who want "real constants" find the following confusing: >>> from demo import foo >>> foo = 14 >>> foo 14 It's fundamental to the way the import function works, and how names in Python behave, but I can see someone with a background in other languages with "real" constants thinking "but foo is a constant, and importing it stops it being a constant!" Paul From grant.b.edwards at gmail.com Wed Oct 4 13:24:32 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 4 Oct 2017 17:24:32 +0000 (UTC) Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-04, Steve D'Aprano wrote: >> It is sometimes called the loop and a half problem. The idea is >> that you must attempt to read a line from the ?le before you know >> whether you are at the end of ?le or not. > > Utter nonsense. There's no "must" here. I'll accept the remote > possibility that maybe one time in a million you have to do what he > says, but the other 999999 times you just read from the file in a > for-loop: > > for line in file: > process(line) > >> This can also be done if a boolean variable is introduced to help >> with the while loop. This boolean variable is the condition that >> gets you out of the while loop and the ?rst time through it must be >> set to get your code to execute the while loop at least one." > > I've been programming in Python for twenty years, and I don't think I have > ever once read from a file using a while loop. You're right, that construct isn't used for reading from files in Python. It _is_ commonly used for reading from things like sockets: mysock.connect(...) while True: data = mysock.recv(9999) if not data: break do_something_with(data) mysock.close() -- Grant Edwards grant.b.edwards Yow! Now I am depressed ... at gmail.com From breamoreboy at gmail.com Wed Oct 4 15:47:14 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 4 Oct 2017 12:47:14 -0700 (PDT) Subject: Easier way to do this? In-Reply-To: References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: <0b2f9abb-8e90-41f5-b580-a6ed6ae6db5d@googlegroups.com> On Wednesday, October 4, 2017 at 8:29:26 PM UTC+1, 20/20 Lab wrote: > Looking for advice for what looks to me like clumsy code. > > I have a large csv (effectively garbage) dump.? I have to pull out sales > information per employee and count them by price range. I've got my code > working, but I'm thinking there must be a more refined way of doing this. > > ---snippet of what I have--- > > EMP1 = [0,0] > EMP2 = [0,0] > EMP3 = [0,0] > > for line in (inputfile): > ??? content = line.split(",") > ??? if content[18] == "EMP1": > ??????? if float(content[24]) < 99.75: > ??????????? EMP1[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP1[1] += 1 > ??? if content[18] == "EMP2": > ??????? if float(content[24]) < 99.75: > ??????????? EMP2[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP2[1] += 1 > ??? if content[18] == "EMP3": > ??????? if float(content[24]) < 99.75: > ??????????? EMP3[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP3[1] += 1 > > and repeat if statements for the rest of 25+ employees.? I can make a > list of the employees, but I'd prefer to pull them from the csv, as our > turnover is rather high (however this is not important).? I'm thinking > another "for employee in content[18]" should be there, but when I tried, > my numbers were incorrect. > > Any help / advice is appreciated, > > Matt Use the csv module https://docs.python.org/3/library/csv.html to read the file with a Counter https://docs.python.org/3/library/collections.html#collections.Counter. I'm sorry but I'm too knackered to try writing the code for you :-( -- Kindest regards. Mark Lawrence. From bc at freeuk.com Wed Oct 4 15:55:06 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 20:55:06 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: On 04/10/2017 14:44, john polo wrote: > In Python Programming Fundamentals 2nd ed., the author, Kent D. Lee, > brings up loop and a half in ch. 3, Repetitive Tasks (p. 82). He wrote: > > "Whether you are writing code in Python or some other language, this > Reading > Records From a File pattern comes up over and over again. It is > sometimes called > the loop and a half problem. The idea is that you must attempt to read a > line from the > ?le before you know whether you are at the end of ?le or not. This can > also be done > if a boolean variable is introduced to help with the while loop. This > boolean variable > is the condition that gets you out of the while loop and the ?rst time > through it must > be set to get your code to execute the while loop at least one." while not eof(f): # read next bit of the file But you need an eof(f) function that tells you if you are at the end of the file. Some people might be concerned that the status could change between checking for eof, and doing a subsequent read (but I've never had such problems). This is suitable for reading perhaps binary files in uneven chunks, which are dependent on the last bit read, so iterating over the file by line or by byte won't work. -- bartc From dvl at psu.edu Wed Oct 4 16:06:52 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Wed, 4 Oct 2017 16:06:52 -0400 Subject: newb question about @property In-Reply-To: mailman.39.1507132804.6204.python-list@python.org References: Message-ID: <1507147612l.12255482l.0l@psu.edu> On Wed, Oct 4, 2017 14:03 PM, bartc wrote > "A property, in some object-oriented programming languages, is a special >sort of class member, intermediate in functionality between a field (or >data member) and a method." > >But Python has some problems just in using fields. If you wanted say a >data type with two fields called x and y, then AFAIK you just create any >suitable class, but you don't need to specifically declare those two fields: > > class any(): > pass > > p=any() > p.x=10 > p.y=20 > >But you also do this: > > p.z=30 > >and it magically acquires a third field! Or you want to modify p.x, but >accidentally type: > > p.c=40 > >No error. Some would perceive all this as an advantage, but it means you >can't just declare a lightweight struct or record 'Point' with exactly >two fields x and y. You have to use other solutions ('namedtuples' or >whatever, which probably are immutable so that don't work the same way). > >This is another example of neglecting the basics, but going for more >advanced, perhaps more sexy features instead. > >Result? You can't just look at my 'any' class and see what fields it >uses. You can't even just look at the static source code. You have to >run the program to find out. And it might be different each time. > > I think you might be missing a small feature buried in the language: class any():.. __slots__ = ["x","y"] >>> p = any() >>> p.x = 10 >>> p.y = 20 >>> p.z = 30 Traceback (most recent call last): File "", line 1, in p.z = 30 AttributeError: 'any' object has no attribute 'z' >>> p.c = 40 Traceback (most recent call last): File "", line 1, in p.c = 40 AttributeError: 'any' object has no attribute 'c' >>> p.__slots__ ['x', 'y'] So, you can prevent new fields from being added without forcing yourself into an immutable named tuple. And if you really need to know what is in there, it tells you! Oh, and as regards to it being different each time: >>> p.__slots__ = ['x','y','z'] Traceback (most recent call last): File "", line 1, in p.__slots__ = ['x','y','z'] AttributeError: 'any' object attribute '__slots__' is read-only Oh, and here's a little thing from the Python 3.6.2 Glossary: __slots__ A declaration inside a class that saves memory by pre-declaring space for instance attributes and eliminating instance dictionaries. Though popular, the technique is somewhat tricky to get right and is best reserved for rare cases where there are large numbers of instances in a memory-critical application. Roger Christman Pennsylvania State University From tjol at tjol.eu Wed Oct 4 16:11:46 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 4 Oct 2017 22:11:46 +0200 Subject: Easier way to do this? In-Reply-To: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: On 04/10/17 18:42, 20/20 Lab wrote: > Looking for advice for what looks to me like clumsy code. > > I have a large csv (effectively garbage) dump. I have to pull out > sales information per employee and count them by price range. I've got > my code working, but I'm thinking there must be a more refined way of > doing this. > > ---snippet of what I have--- > > EMP1 = [0,0] > EMP2 = [0,0] > EMP3 = [0,0] > > for line in (inputfile): > content = line.split(",") First of all, you don't want to make a habit of parsing common file formats by hand. CSV is simple enough, but there are some edge cases that might come up when you're least expecting them. The stdlib has a csv module: https://docs.python.org/3/library/csv.html > if content[18] == "EMP1": > if float(content[24]) < 99.75: > EMP1[0] += 1 > elif float(content[24]) > 99.74: > EMP1[1] += 1 > if content[18] == "EMP2": > if float(content[24]) < 99.75: > EMP2[0] += 1 > elif float(content[24]) > 99.74: > EMP2[1] += 1 > if content[18] == "EMP3": > if float(content[24]) < 99.75: > EMP3[0] += 1 > elif float(content[24]) > 99.74: > EMP3[1] += 1 This just cries out for something like pandas (everyone's favourite data analysis library) Be warned, pandas is part of the scientific python stack, which is immensely powerful and popular, but it does have a distinctive style that may appear cryptic if you're used to the way the rest of the world writes Python. Your code could end up looking something like this: import pandas as pd employee_data = pd.read_csv(csv_filename) employee_data['is_great'] = employee_data['cryptic_number'] > 99.74 employee_data['is_fantastic'] = employee_data['cryptic_number'] < 99.75 by_employee = employee_data.groupby('employee').sum() greatness_dict = dict(by_employee['is_great']) fantasy_dict = dict(by_employee['is_fantastic']) > > and repeat if statements for the rest of 25+ employees. I can make a > list of the employees, but I'd prefer to pull them from the csv, as > our turnover is rather high (however this is not important). I'm > thinking another "for employee in content[18]" should be there, but > when I tried, my numbers were incorrect. > > Any help / advice is appreciated, > > Matt > From dvl at psu.edu Wed Oct 4 16:17:31 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Wed, 4 Oct 2017 16:17:31 -0400 Subject: The "loop and a half" In-Reply-To: mailman.39.1507132804.6204.python-list@python.org References: Message-ID: <1507148251l.21889118l.0l@psu.edu> I teach a course in programming to students who have no plans to be programmers, scientists, or engineers. And I deliberately lied today about the for loop. In my lecture slide, I said that the for loop could only be used if you had a collection of values (e.g. list, tuple, dict, string, or range) where all the data was in hand, and should only be used when truly planning to visit the entire collection. The slide essentially claimed, you could not use the for loop to input a series of values from the keyboard, until seeing a blank line or zero. or to search a list until you found the first even value, and stopping when you get there. If you want to stop repeating for an arbitrary reason, you must use a while loop. Deliberate lie. Verbally I said it was actually a fib, and that there was a way to do these cleanly with a for loop, but it's a method that I postpone until halfway through the second course for students in the major, not the sort of thing I would teach to this particular audience this early. But yes, we can use yield, and iter(), and itertools to do both of these examples very clearly with a for loop (and without an explicit break statement), but I'd rather keep my course content small and simple. Roger Christman Pennsylvania State University > > From auriocus at gmx.de Wed Oct 4 16:29:20 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 4 Oct 2017 22:29:20 +0200 Subject: The "loop and a half" In-Reply-To: <877ewb6qi4.fsf@bsb.me.uk> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <877ewb6qi4.fsf@bsb.me.uk> Message-ID: Am 04.10.17 um 14:34 schrieb Ben Bacarisse: > ram at zedat.fu-berlin.de (Stefan Ram) writes: > >> Steve D'Aprano writes: >>> For-each loops are MUCH easier to understand, and should be taught first. >> >> I prefer a bottom-up approach. >> >> For loops are based on iterators. >> >> So, "bottom-up" in this case means: iterators should be >> taught before for-loops. >> >> But iterators are too abstract to be taught very early. > > I think this may be a problem with your style. From your other > postings, I think you value precision and exactness over broad > understanding, and maybe you teach like that. To understand Stefan's way of teaching, take a look at his other courses, for example the C++ course: http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/c++-kurs He defends the opinion that you learn a programming language best by studying syntax diagrams. He even writes in his FAQ about the language courses: http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/faq_programmieren "Neuere Forschungen haben gezeigt, da? es erfolgreicher ist, formale Systeme (wie Programmiersprachen oder Mathematik) zu unterrichten, wenn dabei kein k?nstlicher ?Praxisbezug? durch ?real-world examples? (Beispiele aus dem ?wahren Leben?) hergestellt wird!" which means "Recent research results show that it is more successful, to teach formal systems (like programming language or maths), if no artificial practical use-case is demonstrated by "real-world examples" with a reference: "They said students who were taught abstract math concepts fared better in experiments than those taught with real-world examples, such as story problems. Adding extraneous details makes it hard for students to extract the basic mathematical concepts and apply them to new problems, they said. "We're really making it difficult for students because we are distracting them from the underlying math," said Jennifer Kaminski, a research scientist at Ohio State University, whose study appears in the journal Science.Reuters, 25. April 2008" Christian From fabien.maussion at gmail.com Wed Oct 4 16:47:02 2017 From: fabien.maussion at gmail.com (Fabien) Date: Wed, 4 Oct 2017 22:47:02 +0200 Subject: Easier way to do this? References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: On 10/04/2017 10:11 PM, Thomas Jollans wrote: > Be warned, pandas is part of the scientific python stack, which is > immensely powerful and popular, but it does have a distinctive style > that may appear cryptic if you're used to the way the rest of the world > writes Python. Can you elaborate on this one? As a scientist, I am curious ;-) From ben.usenet at bsb.me.uk Wed Oct 4 16:55:38 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 04 Oct 2017 21:55:38 +0100 Subject: Easier way to do this? References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: <87r2ui63at.fsf@bsb.me.uk> 20/20 Lab writes: > Looking for advice for what looks to me like clumsy code. > > I have a large csv (effectively garbage) dump.? I have to pull out > sales information per employee and count them by price range. I've got > my code working, but I'm thinking there must be a more refined way of > doing this. I second the suggestion to use the CSV module. It's very simple to use. > ---snippet of what I have--- > > EMP1 = [0,0] > EMP2 = [0,0] > EMP3 = [0,0] > > for line in (inputfile): > ??? content = line.split(",") > ??? if content[18] == "EMP1": > ??????? if float(content[24]) < 99.75: > ??????????? EMP1[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP1[1] += 1 > ??? if content[18] == "EMP2": > ??????? if float(content[24]) < 99.75: > ??????????? EMP2[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP2[1] += 1 > ??? if content[18] == "EMP3": > ??????? if float(content[24]) < 99.75: > ??????????? EMP3[0] += 1 > ??????? elif float(content[24]) > 99.74: > ??????????? EMP3[1] += 1 > > and repeat if statements for the rest of 25+ employees. Eek! When you have named objects selected using a string that is the object's name you know you want a dict. You'd have a single dict for all employees, keyed by the tag in field 18 of the file. Does that help? I'm deliberately not saying more because this looks like a learning exercise and you probably want to do most of it yourself. -- Ben. From lab at 2020fresno.com Wed Oct 4 17:29:27 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Wed, 4 Oct 2017 14:29:27 -0700 Subject: Easier way to do this? In-Reply-To: <0b2f9abb-8e90-41f5-b580-a6ed6ae6db5d@googlegroups.com> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <0b2f9abb-8e90-41f5-b580-a6ed6ae6db5d@googlegroups.com> Message-ID: <3ec5c8be-00d9-a111-e210-8bbff314b330@2020fresno.com> On 10/04/2017 12:47 PM, breamoreboy at gmail.com wrote: > On Wednesday, October 4, 2017 at 8:29:26 PM UTC+1, 20/20 Lab wrote: >> >> >> Any help / advice is appreciated, >> >> Matt > Use the csv module https://docs.python.org/3/library/csv.html to read the file with a Counter https://docs.python.org/3/library/collections.html#collections.Counter. I'm sorry but I'm too knackered to try writing the code for you :-( > > -- > Kindest regards. > > Mark Lawrence. This looks to be exactly what I want.? I'll get to reading.? Thank you very much. Matt From lab at 2020fresno.com Wed Oct 4 17:31:41 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Wed, 4 Oct 2017 14:31:41 -0700 Subject: Easier way to do this? In-Reply-To: <87r2ui63at.fsf@bsb.me.uk> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <87r2ui63at.fsf@bsb.me.uk> Message-ID: <4798c71e-f1e1-06bd-c138-1663c090043a@2020fresno.com> On 10/04/2017 01:55 PM, Ben Bacarisse wrote: > 20/20 Lab writes: > >> Looking for advice for what looks to me like clumsy code. >> >> I have a large csv (effectively garbage) dump.? I have to pull out >> sales information per employee and count them by price range. I've got >> my code working, but I'm thinking there must be a more refined way of >> doing this. > I second the suggestion to use the CSV module. It's very simple to use. > >> ---snippet of what I have--- >> >> EMP1 = [0,0] >> EMP2 = [0,0] >> EMP3 = [0,0] >> >> for line in (inputfile): >> ??? content = line.split(",") >> ??? if content[18] == "EMP1": >> ??????? if float(content[24]) < 99.75: >> ??????????? EMP1[0] += 1 >> ??????? elif float(content[24]) > 99.74: >> ??????????? EMP1[1] += 1 >> ??? if content[18] == "EMP2": >> ??????? if float(content[24]) < 99.75: >> ??????????? EMP2[0] += 1 >> ??????? elif float(content[24]) > 99.74: >> ??????????? EMP2[1] += 1 >> ??? if content[18] == "EMP3": >> ??????? if float(content[24]) < 99.75: >> ??????????? EMP3[0] += 1 >> ??????? elif float(content[24]) > 99.74: >> ??????????? EMP3[1] += 1 >> >> and repeat if statements for the rest of 25+ employees. > Eek! When you have named objects selected using a string that is the > object's name you know you want a dict. You'd have a single dict for > all employees, keyed by the tag in field 18 of the file. Does that > help? > > I'm deliberately not saying more because this looks like a learning > exercise and you probably want to do most of it yourself. > > Looks like I'll be going with the CSV module.? Should trim it up nicely. It's not quite a 'learning exercise', but I learn on my own if I treat it as such.? This is just to cut down a few hours of time for me every week filtering the file by hand for the office manager. Thanks for the pointers, Matt From BILL_NOSPAM at whoknows.net Wed Oct 4 17:39:21 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 4 Oct 2017 17:39:21 -0400 Subject: Python community "welcoming" feedback In-Reply-To: References: Message-ID: Leam Hall wrote: > A while back I pointed out some challenges for the Python community's > intake of new coders. Mostly focusing on IRC and the Python e-mail list. What is the Python e-mail list? Thanks, Bill > > Several people have stepped up their "welcome" game and I've been very > impressed with the way things are going. > > Great job! > > Leam From tjol at tjol.eu Wed Oct 4 18:00:32 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 00:00:32 +0200 Subject: Easier way to do this? In-Reply-To: References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: <8a6af43a-b323-aa0d-609f-d0ae3da354f3@tjol.eu> On 04/10/17 22:47, Fabien wrote: > On 10/04/2017 10:11 PM, Thomas Jollans wrote: >> Be warned, pandas is part of the scientific python stack, which is >> immensely powerful and popular, but it does have a distinctive style >> that may appear cryptic if you're used to the way the rest of the world >> writes Python. > > Can you elaborate on this one? As a scientist, I am curious ;-) Sure. Python is GREAT at iterating. Generators are everywhere. Everyone loves for loops. List comprehensions and generator expressions are star features. filter and map are builtins. reduce used be a builtin, even though almost nobody really understood what it did. In [1]: import numpy as np In the world of numpy (and the greater scientific stack), you don't iterate. You don't write for loops. You have a million floats in memory that you want to do math on - you don't want to wait for ten million calls to __class__.__dict__['__getattr__']('__add__').__call__() or whatever to run. In numpy land, numpy writes your loops for you. In FORTRAN. (well ... probably C) As I see it the main cultural difference between "traditional" Python and numpy-Python is that numpy implicitly iterates over arrays all the time. Python never implicitly iterates. Python is not MATLAB. In [2]: np.array([1, 2, 3]) + np.array([-3, -2, -1]) Out[2]: array([-2, 0, 2]) In [3]: [1, 2, 3] + [-3, -2, -1] Out[3]: [1, 2, 3, -3, -2, -1] In numpy, operators don't mean what you think they mean. In [4]: a = (np.random.rand(30) * 10).astype(np.int64) In [5]: a Out[5]: array([6, 1, 6, 9, 1, 0, 3, 5, 8, 5, 2, 6, 1, 1, 2, 2, 4, 2, 4, 2, 5, 3, 7, 8, 2, 5, 8, 1, 0, 8]) In [6]: a > 5 Out[6]: array([ True, False, True, True, False, False, False, False, True, False, False, True, False, False, False, False, False, False, False, False, False, False, True, True, False, False, True, False, False, True], dtype=bool) In [7]: list(a) > 5 --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () ----> 1 list(a) > 5 TypeError: unorderable types: list() > int() Suddenly, you can even compare sequences and scalars! And > no longer gives you a bool! Madness! Now, none of this, so far, has been ALL THAT cryptic as far as I can tell. It's when you do more complicated things, and start combining different parts of the numpy toolbox, that it becomes clear that numpy-Python is kind of a different language. In [8]: a[(np.sqrt(a).astype(int)**2 == a) & (a < 5)] Out[8]: array([1, 1, 0, 1, 1, 4, 4, 1, 0]) In [9]: import math In [10]: [i for i in a if int(math.sqrt(i))**2 == i and i < 5] Out[10]: [1, 1, 0, 1, 1, 4, 4, 1, 0] Look at my pandas example from my previous post. If you're a Python-using scientist, even if you're not very familiar with pandas, you'll probably be able to see more or less how it works. I imagine that there are plenty of experienced Pythonistas on this list who never need to deal with large amounts of numeric data that are completely nonplussed by it, and I wouldn't blame them. The style and the idiosyncrasies of array-heavy scientific Python and stream or iterator-heavy scripting and networking Python are just sometimes rather different. Cheers Thomas From tjol at tjol.eu Wed Oct 4 18:07:12 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 00:07:12 +0200 Subject: Python community "welcoming" feedback In-Reply-To: References: Message-ID: On 04/10/17 23:39, Bill wrote: > Leam Hall wrote: >> A while back I pointed out some challenges for the Python community's >> intake of new coders. Mostly focusing on IRC and the Python e-mail list. > > What is the Python e-mail list? You're on it right now! There is a two-way mirror between comp.lang.python and python-list at python.org https://mail.python.org/mailman/listinfo/python-list There are a bunch of other Python mailing lists as well (https://mail.python.org/mailman/listinfo) - the most important ones, apart from regional and language-specific ones, are probably tutor, python-dev and python-ideas. -- Thomas > > Thanks, > Bill > > >> >> Several people have stepped up their "welcome" game and I've been >> very impressed with the way things are going. >> >> Great job! >> >> Leam > From bc at freeuk.com Wed Oct 4 18:08:27 2017 From: bc at freeuk.com (bartc) Date: Wed, 4 Oct 2017 23:08:27 +0100 Subject: newb question about @property In-Reply-To: References: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> <05870148-c97d-57a8-c4b6-6112b371dbd4@kynesim.co.uk> Message-ID: On 04/10/2017 17:02, Rhodri James wrote: > On 04/10/17 16:33, Paul Moore wrote: >> It's not an advantage or a disadvantage, just an approach. Many people >> like it, you may not. Specifically, yes you can't "just declare a >> lightweight struct or record with exactly two fields". > > Actually you can: > > >>> class Point: > ...?? __slots__ = ("x", "y") > ...?? def __init__(self, x, y): > ...???? self.x = x > ...???? self.y = y > ...?? def __str__(self): > ...???? return "({0},{1})".format(self.x, self.y) > ... > >>> p = Point(3,4) > >>> print(p) > (3,4) > >>> print(p.x) > 3 > >>> p.x = 7 > >>> print(p) > (7,4) > >>> p.z = 2 > Traceback (most recent call last): > ? File "", line 1, in > AttributeError: 'Point' object has no attribute 'z' > > I pretty much never bother to do this because (bart to the contrary) But it's something you'd have to explicitly do (unless perhaps you make use of those decorators, which AFAICS can do magic). And when I tried, it didn't really work in Python 2 (extra attributes could still be created, and .__slots__ wasn't readonly); only Py3. > it > isn't useful if you're thinking in Pythonic terms, I clearly don't. When I do this in my non-Python language it would be just: record any = (var x,y) p := any(10,20) println p # (10,20) println p.y, p.x # 20 10 println p.len # 2 println p[2] # 20 println p = any(10,20) # 1 (compares fields) No __slots__, __init__ or __str__ need to be defined; it just works. If I needed a version suitable for foreign function interfacing, a little different: type point = struct (real64 x,y) p := point(10,20) println p # (10.0000,20.0000) println p.bytes # 16 It's not Pythonic, but it is pretty handy.... (I know Python can also do structs like this but it appeared to be quite a slog last time I looked.) I guess my kind of coding is somewhat less esoteric than the kind of thing I typically see in Python here. -- bartc From tjol at tjol.eu Wed Oct 4 19:15:19 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 01:15:19 +0200 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> Message-ID: <59de33b4-96b9-a3f2-e6fa-7e01e5c0b15e@tjol.eu> On 05/10/17 00:42, Stefan Ram wrote: > Steve D'Aprano writes: >>> So, "bottom-up" in this case means: iterators should be >>> taught before for-loops. >> Why? > The syntax for is (from memory): > > for in : > > . As an example, I might show: > > for i in range( 3 ): ... > > . This raises the question: > > "What is the value of ?range( 3 )??". > > "Bottom up" means that the simple and constituent parts > are taught before the complex and compound parts. I.e., > ?range( 3 )? is explained before ?for i in range( 3 ):?. If you apply that kind of thinking consistently with Python you'll get caught in an infinite loop trying to explain attribute access before even teaching print(). To call print, you need to access print.__call_. To access that attribute, you need to check print.__dict__ (and some other things), which is itself an attribute of print. To access that, (...) >>> print.__class__.__dict__['__call__'].__call__(print, "hello world") hello world >>> Of course that's not actually what happens when you call the builtin print(). You can delve into the constituent parts of your favourite Python implementation or the language reference to prove it, but that's no longer simple. "simple and constituent parts" is, quite often, not well-defined. Iterators are pretty simple, and, while they have constituent parts, pretty fundamental to Python, at least in spirit. -- Thomas > From tjreedy at udel.edu Wed Oct 4 19:56:31 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 4 Oct 2017 19:56:31 -0400 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/4/2017 1:24 PM, Grant Edwards wrote: > On 2017-10-04, Steve D'Aprano wrote: > >>> It is sometimes called the loop and a half problem. The idea is >>> that you must attempt to read a line from the ?le before you know >>> whether you are at the end of ?le or not. >> >> Utter nonsense. There's no "must" here. I'll accept the remote >> possibility that maybe one time in a million you have to do what he >> says, but the other 999999 times you just read from the file in a >> for-loop: >> >> for line in file: >> process(line) >> >>> This can also be done if a boolean variable is introduced to help >>> with the while loop. This boolean variable is the condition that >>> gets you out of the while loop and the ?rst time through it must be >>> set to get your code to execute the while loop at least one." >> >> I've been programming in Python for twenty years, and I don't think I have >> ever once read from a file using a while loop. > > You're right, that construct isn't used for reading from files in > Python. It _is_ commonly used for reading from things like socket > mysock.connect(...) > while True: > data = mysock.recv(9999) > if not data: > break > do_something_with(data) > mysock.close() I contend that it would be better in this case also to separate data access from data processing. def sockreader(socket, size): while True: data = socket.recv(size) if data: yield data else: break for data in socketreader(mysock, 9999): do_something_with(data) Perhaps the socket module needs an update to make the boilerplate generator function a method, such as 'iterread'. The separation lets one write source-agnostic processing functions. def do_something(source): for data in source: One can now 'do_something' with data from pipe, file, socket, or list. -- Terry Jan Reedy From Irv at furrypants.com Wed Oct 4 20:11:49 2017 From: Irv at furrypants.com (Irv Kalb) Date: Wed, 4 Oct 2017 17:11:49 -0700 Subject: Easier way to do this? In-Reply-To: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: <302A89DC-2C11-4EDC-8986-4314BE7ADBF2@furrypants.com> I'm assuming from your posts that you are not a student. If that is the case, look at my solution below. > On Oct 4, 2017, at 9:42 AM, 20/20 Lab wrote: > > Looking for advice for what looks to me like clumsy code. > > I have a large csv (effectively garbage) dump. I have to pull out sales information per employee and count them by price range. I've got my code working, but I'm thinking there must be a more refined way of doing this. > > ---snippet of what I have--- > > EMP1 = [0,0] > EMP2 = [0,0] > EMP3 = [0,0] > > for line in (inputfile): > content = line.split(",") > if content[18] == "EMP1": > if float(content[24]) < 99.75: > EMP1[0] += 1 > elif float(content[24]) > 99.74: > EMP1[1] += 1 > if content[18] == "EMP2": > if float(content[24]) < 99.75: > EMP2[0] += 1 > elif float(content[24]) > 99.74: > EMP2[1] += 1 > if content[18] == "EMP3": > if float(content[24]) < 99.75: > EMP3[0] += 1 > elif float(content[24]) > 99.74: > EMP3[1] += 1 > > and repeat if statements for the rest of 25+ employees. I can make a list of the employees, but I'd prefer to pull them from the csv, as our turnover is rather high (however this is not important). I'm thinking another "for employee in content[18]" should be there, but when I tried, my numbers were incorrect. > > Any help / advice is appreciated, > > Matt > You could certainly use the csv module if you want, but this builds on your start of dealing with the data line by line. Completely untested, but this approach works by building a dictionary on the fly from your data. Each key is an employee name. The data associated with each key is a two item list of counts. # Constants NAME_INDEX = 18 SALES_INDEX = 24 THRESHHOLD = 99.75 salesCountDict = {} # start with an empty dict for line in (inputfile): content = line.split(",") # split the line into a list name = content[NAME_INDEX] # extract the name from the content list # If we have not seen this employee name before, add it to the dictionary # like key value pair: '': [0, 0] if not(name in employeeDataDict): salesCountDict[name] = [0, 0] price = float(content[SALES_INDEX]) # extract the price # If the price is under some threshhold, increment one value in the associated sales list # otherwise increment the other if price < THRESHHOLD: salesCountDict[name][0] += 1 else: salesCountDict[name][1] += 1 # Now you should have a dictionary. Do what you want with it. For example: for name in salesCountDict: salesList = salesCountDict[name] print(name, salesList) # Assuming Python 3 From steve+python at pearwood.info Wed Oct 4 20:16:45 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 11:16:45 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 02:56 am, Paul Moore wrote: > On 4 October 2017 at 16:35, Steve D'Aprano > wrote: >> I've been programming in Python for twenty years, and I don't think I have >> ever once read from a file using a while loop. > > Twenty years isn't long enough :-) The pattern the OP is talking about > was common in "Jackson Structured Programming" from back in the 1980s. In context, I was replying to a quoted author who claims this pattern is universal in Python code. I don't doubt that pattern exists *at all*, only that it is common in Python. That pattern wasn't even universal in the 80s. In Pascal, for example, you read from a file using something very close to this: reset(infile); while not eof(indeck) do begin read(indeck, buffer); {now process the buffer} end; I recall that the Pascal compiler had to do some clever behind the scenes jiggery-pokery to get eof() to work, but that's what compilers are supposed to do: make common tasks easy for the programmer. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From toby at tobiah.org Wed Oct 4 20:18:21 2017 From: toby at tobiah.org (Tobiah) Date: Wed, 4 Oct 2017 17:18:21 -0700 Subject: Creating a MIDI file Message-ID: What would be the best library to use for creating MIDI files that I could import into a DAW like Reaper? Thanks, Tobiah From steve+python at pearwood.info Wed Oct 4 21:41:54 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 12:41:54 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> Message-ID: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 07:17 am, ROGER GRAYDON CHRISTMAN wrote: > I teach a course in programming to students who have no plans to be > programmers, scientists, or engineers. And I deliberately lied today > about the for loop. Well done! (I'm not being sarcastic.) This is entirely appropriate: all education is best understood as a series of "lies to children" asymptotically approaching reality, where "children" should be read figuratively not literally. https://en.wikipedia.org/wiki/Lie-to-children (If you prefer a more academic, philosophical term, Wittgenstein's Ladder is often used for more or less the same concept.) The term is provocative and evocative, but it is also slightly unfortunate in that some people (who should know better!) take issue with it due to an entirely avoidable misunderstanding of what is meant by "lie". E.g. the quote from Mishra in the above Wikipedia article *completely* misses the point; Kharem and Collura's lament that it is "stupidification". What are these people thinking? Is it stupidification to teach music students scales before atonal music? Do they honestly think that we should start teaching science with quantum mechanics and general relativity? (And even if we did, it would still be a lie: at *best*, QM and GR are themselves incomplete descriptions of the universe and therefore any description of reality in terms of QM or GR must be strictly false.) Strictly speaking, everything we human beings are capable of understanding about the universe falls short of reality and is therefore, in some sense, a lie. But lies-to-children are simplifications and half-truths which lead to better understanding, not outright porkies: "The pedagogical point is to avoid unnecessary burdens on the student?s first encounter with the concept." (Jeffrey and Corless, 2011, quoted in Wikipedia above.) Despite Mishra's opinion, "babies are delivered by the stork" is not a lie-to- children in this sense. Its just a dishonest untruth told by parents too embarrassed (or not smart enough) to explain where babies come from in a simplified, honest way to their children. Lies-to-children are not dishonest untruths. At various stages of education, we teach many lies-to-children, including: - supply and demand curves; - human beings make rational economic choices; - well, on average, human beings collectively make rational economic choices; - "AEIOU" are the English vowels; - the pyramids were built by slaves; - you cannot subtract a larger number from a smaller; - or take the square root of a negative number; - if the discriminant is negative, the quadratic equation cannot be factorized; - mass is conserved; - water is incompressible; - the gas laws; - Germany was the aggressor in World War 2; - well, Germany and Japan; - *surely* it must be Germany, Italy and Japan; - Hookes' Law for springs; - any table of values, or graph, used in engineering; - Newton's laws of motion; - Ampere's Law and Ohm's Law; - Special and General Relativity; - matter is made of indivisible atoms; - transformation of elements (such as lead to gold) is impossible; - there are three forms of radioactive decay: alpha, beta and gamma; - and two forms of carbon, graphite and diamond; - people and animals come in exactly two distinct sexes; - tides are caused by the moon; - well, the moon and the sun; Every one of these have to eventually be followed with: "Well, yes, that's kind of true *but* ..." > In my lecture slide, I said that the for loop could only be used if you had > a collection of values (e.g. list, tuple, dict, string, or range) > where all the data was in hand, and should only be used when > truly planning to visit the entire collection. Kastens & Chayes (2011) give some recommendations for effective use of the lies-to-children as a pedagogical technique. Since it is unavoidable to "lie" to your students, how can you make sure the lies help their education rather than hinder it? https://web.archive.org/web/20160205090900/http://serc.carleton.edu/earthandmind/posts/lies_children.html > The slide essentially claimed, you could not use the for loop to > input a series of values from the keyboard, until seeing a blank line or > zero. or to search a list until you found the first even value, and stopping > when you get there. > If you want to stop repeating for an arbitrary reason, you must use a while > loop. > > Deliberate lie. > > Verbally I said it was actually a fib, and that there was a way to > do these cleanly with a for loop, but it's a method that I postpone until > halfway through the second course for students in the major, > not the sort of thing I would teach to this particular audience this early. > > But yes, we can use yield, and iter(), and itertools to do both of these > examples very clearly with a for loop (and without an explicit break > statement), but I'd rather keep my course content small and simple. > > > Roger Christman > Pennsylvania State University >> > >> -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From BILL_NOSPAM at whoknows.net Wed Oct 4 22:02:50 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 4 Oct 2017 22:02:50 -0400 Subject: Creating a Dictionary In-Reply-To: References: Message-ID: Stefan Ram wrote: > One might wish to implement a small language with these commands: Explain why. What is the advantage? > > F - move forward > B - move backward > L - larger stepsize > S - smaller stepsize > > . One could start with the following pseudocode for a dictionary: > > { 'F': lambda: myturtle.forward( s ), > 'B': lambda: myturtle.backward( s ), > 'L': lambda: global s; s *= 2, > 'S': lambda: global s; s /= 2 } > > . But lambda-expressions cannot contain statements. > > In real Python one could write something like (untested): > > def f(): > myturtle.forward( s ) > > def b(): > myturtle.backward( s ) > > def l(): > global siz > size *= 2 > > def s(): > global siz > size /= 2 > > { 'F': f, > 'B': b, > 'L': l, > 'S': s } > > . Is this more readable or less readable? > > Any other suggestions? > From ned at nedbatchelder.com Wed Oct 4 22:06:17 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 4 Oct 2017 22:06:17 -0400 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: On 10/4/17 7:37 AM, Stefan Ram wrote: > bartc writes: >> Note that your reverse-indentation style is confusing! > In Python, indentation can be significant. > > Sometimes, some lines in Python must be indented by 0. > > This dictates that Python code cannot be indented > in posts to differentiate it from the natural-language > body of the post. Therefore, it's rather the > natural-language body that has to be indented. > FWIW, I also find your posts hard to read because of the inverted indentation.? We don't have difficulty understanding the meaning of block-indented chunks of Python code. --Ned. From steve+python at pearwood.info Wed Oct 4 22:12:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 13:12:08 +1100 Subject: newb question about @property References: <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> <05870148-c97d-57a8-c4b6-6112b371dbd4@kynesim.co.uk> Message-ID: <59d594f9$0$14932$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 09:08 am, bartc wrote: [...] > And when I tried, it didn't really work in Python 2 (extra attributes > could still be created, and .__slots__ wasn't readonly); only Py3. Not quite, but I don't blame you for the mistake. Its an easy one to make. __slots__ only works in "new style classes", not "classic classes" in Python 2. And yes, it is a wart in Python 2 that there are two subtly different kinds of class. This is due to historical reasons, and its fixed in Python 3. In Python 2, "classic classes" are declared like this: class MyClass: ... (or by inheriting from another classic class). New-style classes, or "types", inherit from object: class MyClass(object): ... (or some other new-style type, like int, dict, list, etc.) Yes yes yes, I completely agree that this is a suboptimal situation. It is a language wart and a trap for the beginner, or even the experienced coder. Use Python 3, where it is fixed. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 4 22:20:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 13:20:08 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d596da$0$14973$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 09:42 am, Stefan Ram wrote: > Steve D'Aprano writes: >>>So, "bottom-up" in this case means: iterators should be >>>taught before for-loops. >>Why? > > The syntax for is (from memory): > > for in : > > . As an example, I might show: > > for i in range( 3 ): ... > > . This raises the question: > > "What is the value of ?range( 3 )??". That's easy: in Python 2, it is a list [0, 1, 2]. In Python 3, it is a range object, a lazily calculated list-like object. > "Bottom up" means that the simple and constituent parts > are taught before the complex and compound parts. I.e., > ?range( 3 )? is explained before ?for i in range( 3 ):?. This is reductionist thinking. Which is sometimes useful, but not here. If you were teaching people to drive a car, would you insist on teaching them how to pull out and rebuild the engine before sitting them in the drivers seat? If you were teaching people to swim, would you teach them the laws of hydrodynamics and the chemistry of water before going near a swimming pool? If you were teaching people to ride a horse, would you start by having them dissect a dead horse and learn the biomechanics of the horse skeletal and muscular systems? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From christopher_reimer at icloud.com Wed Oct 4 22:23:26 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Wed, 04 Oct 2017 19:23:26 -0700 Subject: How to determine lowest version of Python 3 to run? Message-ID: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Greetings, I've always installed the latest and greatest version of Python 3 to develop my own programs. I'm planning to release a program to the public. I could toss in a note that the program runs on the latest version of Python 3.6 but I haven't tested on earlier versions (i.e., 3.4 and 3.5). AFAIK, I haven't written any version-specific code. How do I determine the lowest version of Python to run? I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. Thank you, Chris R. From rosuav at gmail.com Wed Oct 4 22:24:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 13:24:57 +1100 Subject: Creating a Dictionary In-Reply-To: References: Message-ID: On Thu, Oct 5, 2017 at 12:24 PM, Stefan Ram wrote: > One might wish to implement a small language with these commands: > > F - move forward > B - move backward > L - larger stepsize > S - smaller stepsize > > . One could start with the following pseudocode for a dictionary: > > { 'F': lambda: myturtle.forward( s ), > 'B': lambda: myturtle.backward( s ), > 'L': lambda: global s; s *= 2, > 'S': lambda: global s; s /= 2 } > > . But lambda-expressions cannot contain statements. > > Any other suggestions? There are a few options here. One is to make use of a simple "collector decorator": commands = {} def cmd(f): commands[f.__name__.upper()] = f return f @cmd def f(): myturtle.forward( s ) @cmd def b(): myturtle.backward( s ) @cmd def l(): global siz size *= 2 @cmd def s(): global siz size //= 2 (Also untested, but the pattern is one I've used many times.) Another is to deploy the whole thing as a class, with no globals: class Turtle: def __init__(self): self.size = ... self.turtle = ... def cmd_F(self): self.turtle.forward(self.size) def cmd_B(self): self.turtle.backward(self.size) def cmd_L(self): self.size *= 2 def cmd_S(self): self.size //= 2 Note that I've added "cmd_" in front of the names, which means you can safely getattr(turtle, "cmd_" + letter) and be confident you won't accidentally grab something that isn't a command. (Note also that I used // for division. If your size is actually a float, then change those back to single slashes.) Either of these patterns would work fairly well. There are others, too, but I'd be inclined to use one of these. ChrisA From steve+python at pearwood.info Wed Oct 4 22:28:52 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 13:28:52 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> <59de33b4-96b9-a3f2-e6fa-7e01e5c0b15e@tjol.eu> Message-ID: <59d598e5$0$14946$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 10:15 am, Thomas Jollans wrote: > If you apply that kind of thinking consistently with Python you'll get > caught in an infinite loop trying to explain attribute access before > even teaching print(). > > To call print, you need to access print.__call_. To access that > attribute, you need to check print.__dict__ (and some other things), > which is itself an attribute of print. To access that, (...) > >>>> print.__class__.__dict__['__call__'].__call__(print, "hello world") > hello world Its worse than that. Conceptually, you are calling type(print).__call__, not print.__call__. But how do you call type? With type.__call__(print), except that it is actually type(type).__call__(type, print) which is recursive... Obviously the real interpreter doesn't do that. Doing attribute access on the type is magic, and built into the interpreter: there's no infinite regress. > "simple and constituent parts" is, quite often, not well-defined. Indeed. > Iterators are pretty simple, and, while they have constituent parts, > pretty fundamental to Python, at least in spirit. But you don't need to know the slightest thing about iterators in order to learn how to iterate over lists, strings, tuples, dicts etc. In fact, iterators are merely a subset of a more fundamental "kind" of object, iterables: anything which can be iterated over. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 4 22:34:09 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 13:34:09 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d59a23$0$14934$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 10:56 am, Terry Reedy wrote: >> You're right, that construct isn't used for reading from files in >> Python. It _is_ commonly used for reading from things like socket > >> mysock.connect(...) >> while True: >> data = mysock.recv(9999) >> if not data: >> break >> do_something_with(data) >> mysock.close() > > I contend that it would be better in this case also to separate data > access from data processing. [...] > Perhaps the socket module needs an update to make the boilerplate > generator function a method, such as 'iterread'. I was thinking the same thing as Terry. Obviously we need the low-level socket module for those (hopefully rare) times we need to interface with a socket at a low level. But I was thinking, why isn't there an equivalent high-level socket interface, like for files, so we can simply write something like: with Socket(spam) as mysock: for block in mysock(blocksize): ... I haven't done enough [read: any] socket programming to tell how useful and practical this is, but it seems like an obvious enhancement. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Wed Oct 4 22:34:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 13:34:24 +1100 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Message-ID: On Thu, Oct 5, 2017 at 1:23 PM, Christopher Reimer wrote: > Greetings, > > I've always installed the latest and greatest version of Python 3 to develop my own programs. I'm planning to release a program to the public. I could toss in a note that the program runs on the latest version of Python 3.6 but I haven't tested on earlier versions (i.e., 3.4 and 3.5). AFAIK, I haven't written any version-specific code. > > How do I determine the lowest version of Python to run? > > I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. > By "major" and "minor", do you mean that you're testing on the latest "3.5" (currently 3.5.4)? Usually a three-part version number is described as "Major.Minor.Revision" [1], so you're working with the single major version "3". My recommendation is to quote only two-part version numbers. Test on the last few, then say "Requires Python 3.4 or greater" if that's how far back you tested. Generally, you won't have to test on multiple revisions of the same minor version; if it runs on 3.5.2, it should run on 3.5.3 and 3.5.1, unless you depend on some specific bug that got fixed. And how many versions back should you test? If you want to be thorough, look at which versions are still in support - either with your OS repositories or at Python.org - and test on all of those. If you're like me, though, just test on whichever ones you happen to have around, and then quote that :) You can shortcut this whole process, though, if you know for certain that you've used a newly-added feature. For example, if your code depends on the order of arguments in **kwargs, don't bother testing on 3.5 and getting flaky results - that changed in 3.6, so just quote that you need 3.6+. If you use the matrix multiplication operator "@", test on 3.5 and 3.6 and that's it. Obviously this only helps if you KNOW about a new feature, but I've had a number of projects where I get all excited about a flashy new feature and immediately start using it... :) Hope that helps! ChrisA [1] The music nerd in me wants "minor" to be lower-cased, but I told me to shut up. From rosuav at gmail.com Wed Oct 4 22:37:01 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 13:37:01 +1100 Subject: The "loop and a half" In-Reply-To: <59d596da$0$14973$b1db1813$d948b532@news.astraweb.com> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> <59d596da$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 1:20 PM, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 09:42 am, Stefan Ram wrote: > >> Steve D'Aprano writes: >>>>So, "bottom-up" in this case means: iterators should be >>>>taught before for-loops. >>>Why? >> >> The syntax for is (from memory): >> >> for in : >> >> . As an example, I might show: >> >> for i in range( 3 ): ... >> >> . This raises the question: >> >> "What is the value of ?range( 3 )??". > > That's easy: in Python 2, it is a list [0, 1, 2]. In Python 3, it is a range > object, a lazily calculated list-like object. Even easier: in any version of Python, it is the series of numbers 0, 1, 2. You can go a rather long way without worrying about what "series" actually means at the concrete level. It's a thing you can iterate over, and that's all that matters to 99%+ of range() usage. ChrisA From rosuav at gmail.com Wed Oct 4 22:43:35 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 13:43:35 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 1:21 PM, Stefan Ram wrote: > Steve D'Aprano writes: >>At various stages of education, we teach many lies-to-children, including: > > Many of those lies can be perfectly true in some sense. > I pick some examples: > >>- "AEIOU" are the English vowels; > > One is free to define this term this way. This is just > a definition. Definitions cannot be false. The definition > use by phoneticians is not "more correct" than the > definition used in everyday life. One is free to be factually wrong, like claiming that Australia is not a continent. (I've seen that, in a quiz game.) Definitions can most certainly be incorrect. >>- you cannot subtract a larger number from a smaller; > > In a certain theory one cannot, indeed. In another, one can. Until negative numbers have been taught, you cannot. Once they have, you can. >>- mass is conserved; > > This also depends on the theory. > > (Ok, the theory where it is /not/ conserved better > describes our world. [But the theory where it /is/ > conserved might be more useful in everyday life.]) You can get through a lot of life believing that mass is conserved, but technically it is not, as can be proven. Ergo it is a lie to claim that it is, which fits into Steven's post. >>- Germany was the aggressor in World War 2; >>- well, Germany and Japan; >>- *surely* it must be Germany, Italy and Japan; > > This listing style reminds me of of a listing style used in > ?Falsehoods Programmers Believe About Names?: > > |32. People's names are assigned at birth. > |33. OK, maybe not at birth, but at least pretty close to birth. > |34. Alright, alright, within a year or so of birth. > |35. Five years? > |36. You're kidding me, right? There are a lot of those kinds of posts now, and I like the style. (That's why I made my own take on it, about PEP 8.) Yes, okay, that last one "You're kidding me, right?" doesn't technically fit into the concept, but seriously, if nobody had told you that some people are nameless for 5+ years, would you have guessed that that's the case? ChrisA From rosuav at gmail.com Wed Oct 4 22:45:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 13:45:58 +1100 Subject: Easier way to do this? In-Reply-To: <302A89DC-2C11-4EDC-8986-4314BE7ADBF2@furrypants.com> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <302A89DC-2C11-4EDC-8986-4314BE7ADBF2@furrypants.com> Message-ID: On Thu, Oct 5, 2017 at 11:11 AM, Irv Kalb wrote: > # If we have not seen this employee name before, add it to the dictionary > # like key value pair: '': [0, 0] > if not(name in employeeDataDict): > salesCountDict[name] = [0, 0] Python provides a "not in" operator, which is far superior to parenthesizing the expression. But, even better, you can use a defaultdict to create this automatically - and you could even, with a slight tweak, make use of collections.Counter to do all the work for you. ChrisA From yennguyen.agar at gmail.com Wed Oct 4 22:48:07 2017 From: yennguyen.agar at gmail.com (yennguyen.agar at gmail.com) Date: Wed, 4 Oct 2017 19:48:07 -0700 (PDT) Subject: Asking help about Python Message-ID: Hi Everyone, I now do need to re-compile the Python 2.7 with VisualStudio 2012. Can anyone here kindly give me any help? I appreciate any kind of help: hints, learning sources, or ideally show me some instructions :(. Thank you. From python at mrabarnett.plus.com Wed Oct 4 22:57:00 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 5 Oct 2017 03:57:00 +0100 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Message-ID: On 2017-10-05 03:23, Christopher Reimer wrote: > Greetings, > > I've always installed the latest and greatest version of Python 3 to develop my own programs. I'm planning to release a program to the public. I could toss in a note that the program runs on the latest version of Python 3.6 but I haven't tested on earlier versions (i.e., 3.4 and 3.5). AFAIK, I haven't written any version-specific code. > > How do I determine the lowest version of Python to run? > > I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. > > Thank you, > If you want to know whether it'll run on 3.4, just run it on 3.4. There may not be much point in running it on 3.5 because if it's OK on 3.4, then it should also be OK on 3.5. Of course, if you're talking about stuff that needs to be compiled, such as an extension written in C, then you'll need to test it on every version that you're willing to support. From rosuav at gmail.com Wed Oct 4 23:07:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 14:07:12 +1100 Subject: Asking help about Python In-Reply-To: References: Message-ID: On Thu, Oct 5, 2017 at 1:48 PM, wrote: > Hi Everyone, > > I now do need to re-compile the Python 2.7 with VisualStudio 2012. > Can anyone here kindly give me any help? I appreciate any kind of help: hints, learning sources, or ideally show me some instructions :(. > > Thank you. This is not going to be easy. Why do you want to? Are you trying to make a patched version of Python, or are you simply looking to get the latest? Python 2.7 is not intended to be compiled with VS 2012. The official Windows builds use VS 2008. Trying to change compilers - even versions of the same compiler - is going to be a hairy job. Unless you are EXTREMELY comfortable with building large applications from source, I do not recommend this. ChrisA From rosuav at gmail.com Wed Oct 4 23:09:35 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 14:09:35 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> <59d596da$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 1:36 PM, Stefan Ram wrote: > Steve D'Aprano writes: >>If you were teaching people to drive a car, would you insist on teaching them >>how to pull out and rebuild the engine before sitting them in the drivers >>seat? > > If I would have to teach people to drive a car, I would > exercise with them how to move the steering wheel to control > the direction, how to change gears, how to look over the > shoulder to check the blind spot, and I would first exercise > each of these actions /in isolation/ at a quiet training > area, before the people then possible have to do several of > these actions at the same time in real road traffic. But you wouldn't dig down into the more concrete level of what actually happens when you change gears (explaining both automatic AND manual transmission, for completeness). You'd just say "change gears by moving this thingy over to there". (Caveat: I don't drive a car. Experienced drivers probably don't call the thingy a thingy, they probably call it a gear lever or something boring like that.) ChrisA From steve+python at pearwood.info Wed Oct 4 23:22:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 14:22:14 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 01:21 pm, Stefan Ram wrote: > Steve D'Aprano writes: >>At various stages of education, we teach many lies-to-children, including: > > Many of those lies can be perfectly true in some sense. Well of course, that's the whole point of them being "lies-to-children". In fact, only one of them (to the best of my knowledge) is an outright untruth that should be wiped from the education syllabus: the myth that the Egyptian pyramids were built by slave labour. And even that has a tiny kernel of fact. Of course there were some slaves who would have worked on the pyramids, probably on medial jobs, but the bulk of the work was done by free labourers and craftsmen, who were not only well-paid but were organised enough to go on strike when their pay was late. > I pick some examples: > >>- "AEIOU" are the English vowels; > > One is free to define this term this way. This is just > a definition. Definitions cannot be false. Yes they can. I can define the letter M as a vowel, but the meaning of the word "vowel" and the sound of the letter "M" are in contradiction. One or the other can be correct, but not both. > The definition > use by phoneticians is not "more correct" than the > definition used in everyday life. Yes it is. Even in everyday life, there are clear, simple and obvious counter-examples to the claimed "AEIOU" vowel rule. In fact, mY mind boogles at the mYsterY of whY people insist that AEIOU are the onlY vowels. It is, I think, an example of a stupid English language folklore that people repeat unthinkingly, even though the counter-examples are obvious and common. Like "I before E except after C", which is utter rubbish. It gets worse: sometimes W is a vowel too (although mostly only in archaic spelling, or in loan words like "cwm", from Welsh, pronounced "coom"), or at least *part* of a vowel. But vowels and constonants represent types of *sounds*, not letters, so we shouldn't even be talking about letters being vowels at all. (Nevertheless, the practice is probably too ingrained to ever completely stamp out.) The A and E in the word "are" are not vowels, since they are silent. The U in "unicorn" and "university" are not vowels either, and if you write "an unicorn" you are making a mistake. >>- you cannot subtract a larger number from a smaller; > > In a certain theory one cannot, indeed. In another, one can. > >>- mass is conserved; > > This also depends on the theory. Forget theory, in *reality* mass is not conserved. > (Ok, the theory where it is /not/ conserved better > describes our world. [But the theory where it /is/ > conserved might be more useful in everyday life.]) Of course it is useful. Its just not true. That's the whole point of calling it "lies to children". >>- Germany was the aggressor in World War 2; >>- well, Germany and Japan; >>- *surely* it must be Germany, Italy and Japan; > > This listing style reminds me of of a listing style used in > ?Falsehoods Programmers Believe About Names?: Yes. For the record: The USSR and Germany invaded Poland simultaneously, and the two countries divided Poland between them. Churchill himself wrote about having difficulty convincing people to care, since Poland had earlier invaded Czechoslovakia (opportunistically while the Germans were seizing the Sudetenland) and many people in England thought that Poland deserved their fate. Likewise the USSR had invaded Finland. The UK invaded and occupied neutral Iceland in 1940, handing over control to the USA in 1941. Despite signing an agreement in 1948 to leave within 6 months, US military forces actually did not finally leave Iceland, and return full sovereignty to the Icelander government, until 2006. In the East, while Japan did take the first overtly military action against the US, the US had (in some sense) first engaged in hostile behaviour against Japan by unilaterally imposing, and enforcing, sanctions on Japan. (I make no comment on whether such sanctions were justified or not, only that they can be seen as a form of aggressive economic warfare.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 4 23:26:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 14:26:01 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> <59d596da$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5a64a$0$14956$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 01:36 pm, Stefan Ram wrote: > Steve D'Aprano writes: >>If you were teaching people to drive a car, would you insist on teaching >>them how to pull out and rebuild the engine before sitting them in the >>drivers seat? > > If I would have to teach people to drive a car, I would > exercise with them how to move the steering wheel to control > the direction, how to change gears, how to look over the > shoulder to check the blind spot, and I would first exercise > each of these actions /in isolation/ at a quiet training > area, before the people then possible have to do several of > these actions at the same time in real road traffic. My very first driving lesson had the instructor sit me in his car in my drive way, show me how to start the car, where the gear, accelerator and brake pedals were, and have me drive straight out onto a moderately busy road. That certainly made sure I was paying attention. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Wed Oct 4 23:54:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 14:54:13 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 2:22 PM, Steve D'Aprano wrote: > It is, I think, an example of a stupid English language folklore that people > repeat unthinkingly, even though the counter-examples are obvious and common. > Like "I before E except after C", which is utter rubbish. That's because it's only half the rule. I before E except after C, when the sound is "aye" or the sound is "ee". If the sound isn't one of those, the rule doesn't apply. So yeah, in that form, it's utter rubbish. (There are exceptions even to the longer form of the rule, but only a handful. English isn't a tidy language.) > But vowels and constonants represent types of *sounds*, not letters, so we > shouldn't even be talking about letters being vowels at all. (Nevertheless, > the practice is probably too ingrained to ever completely stamp out.) > > The A and E in the word "are" are not vowels, since they are silent. The U > in "unicorn" and "university" are not vowels either, and if you write "an > unicorn" you are making a mistake. In explaining how spelling works, it's better to talk in terms of *phonograms*, not letters. In Turkish, they correspond almost perfectly - each letter makes a sound on its own - but in English, there are a good number of two-letter phonograms, and a few longer ones. I'm sure nobody here will be surprised to think of "th" as a phonogram, but of course it isn't always so; in the word "courthouse", the two letters happen to abut, but are in separate syllables. In your example, "are" has two phonograms: "ar", and a silent "e". That said, though, none of this matters in contexts like game shows. If you'd like to buy a vowel, people are going to look VERY strangely at you when you ask for "w". And you can't choose the letter ? ("thorn") and expect to get both the "t" and the "h" given to you. No, you work with the basic 26 ASCII Latin letters in most contexts, and the only uncertain one is "Y". > The UK invaded and occupied neutral Iceland in 1940, handing over control to > the USA in 1941. Despite signing an agreement in 1948 to leave within 6 > months, US military forces actually did not finally leave Iceland, and return > full sovereignty to the Icelander government, until 2006. Huh. I didn't know that part. > In the East, while Japan did take the first overtly military action against > the US, the US had (in some sense) first engaged in hostile behaviour against > Japan by unilaterally imposing, and enforcing, sanctions on Japan. > > (I make no comment on whether such sanctions were justified or not, only that > they can be seen as a form of aggressive economic warfare.) Economic warfare is different from military action, though. Economic sanctions may be a precursor to war but they are not the first blow in warfare. However, that's off topic for this list. ChrisA From christopher_reimer at icloud.com Thu Oct 5 00:01:34 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Wed, 04 Oct 2017 21:01:34 -0700 Subject: Good virtualenv and packaging tutorials for beginner? In-Reply-To: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> References: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> Message-ID: <5C2B73F3-D1E9-4036-A3CC-D3ADE2239F3D@icloud.com> On Oct 4, 2017, at 3:49 AM, Leam Hall wrote: > > Folks on IRC have suggested using virtualenv to test code under different python versions. Sadly, I've not found a virtualenv tutorial I understand. Anyone have a link to a good one? > > The next step will be to figure out how to package a project; a good tutorial URL would be appreciated on that, too. > > Thanks! > > Leam > -- > https://mail.python.org/mailman/listinfo/python-list I'm looking at tox that will automatically set up virtualenv for running unit tests under different versions of Python. https://tox.readthedocs.io/en/latest/ Chris R. From flebber.crue at gmail.com Thu Oct 5 00:13:33 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Wed, 4 Oct 2017 21:13:33 -0700 (PDT) Subject: Suggestions on storing, caching, querying json Message-ID: <45d21aad-3db7-4d56-ae07-85f9797a3469@googlegroups.com> HI Looking for suggestions around json libraries. with Python. I am looking for suggestions around a long term solution to store and query json documents across many files. I will be accessing an api and downloading approx 20 json files from an api a week. Having downloaded this year I have over 200 files already. So it will grow at a reasonable rate. What I have initially done is store them into a mongo db. Now I am wondering if this is useful or prudent since other than querying the json I wont have much use of other mongo features. When querying the json files though queries will utilise multiple json files at once, not just retrieving a single record. The usage is for data analysis. Is there a good json storage option, with caching and optimal querying etc. Regarding querying I did find a library for json searching called ObjectPath written in Python http://objectpath.org/reference.html Looking to leverage your experience. Cheers Sayth From steve+python at pearwood.info Thu Oct 5 00:23:41 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 15:23:41 +1100 Subject: OT aggression [was Re: Lies in education [was Re: The "loop and a half"]] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5b3cf$0$14936$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 02:54 pm, Chris Angelico wrote: > On Thu, Oct 5, 2017 at 2:22 PM, Steve D'Aprano > wrote: [...] >> In the East, while Japan did take the first overtly military action against >> the US, the US had (in some sense) first engaged in hostile behaviour >> against Japan by unilaterally imposing, and enforcing, sanctions on Japan. >> >> (I make no comment on whether such sanctions were justified or not, only >> that they can be seen as a form of aggressive economic warfare.) > > Economic warfare is different from military action, though. Economic > sanctions may be a precursor to war but they are not the first blow in > warfare. I carefully choose my words: I spoke of *the aggressor*, not the first country to fire shots. Nor is the aggressor necessarily in the wrong. There are many ways to be aggressive. On a personal level, you don't have to throw the first punch to be the aggressor -- if you crowd somebody, getting right in their face, blocking their exit and forcing them to back up into a corner, you are the aggressor even if they are the first to throw a punch. The US of course is a military superpower, with enormous sea and land borders and control over its airspace. Nobody could successfully blockade them. But if somebody choose to enforce a blockade of (lets say) Iceland, forcing back all planes and ships, that would rightly be seen as an act of aggression. Likewise, if (say) North Korea were to flood the US with billions of counterfeit $100 notes, I don't think anyone would fail to consider that an act of war. Its a sign of just how dysfunctional US politics has become that the revelations of just how much Russia manipulated the last federal election, and how deeply the current administration is under Russia's influence, has failed to make any practical difference to US federal politics. Had the USSR done in 1976 what Russia did in 2016, at the very least the government would have fallen and the replacement government threaten reprisals. > However, that's off topic for this list. Since when does that stop us? :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Oct 5 00:24:07 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 15:24:07 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 02:14 pm, Stefan Ram wrote: > Chris Angelico writes: >>You can get through a lot of life believing that mass is conserved, >>but technically it is not, as can be proven. > > Well, in fact, it is conserved. It certainly is not. The whole point of Einstein's equation E = mc? is that energy and mass are freely convertible. Neither energy nor mass alone are conserved. > When an electron and a positron annihilate to give a gas of > two photons, this gas as a whole still has the same mass as > the electron before. And its center of gravity moves with > less than c. That doesn't sound like any description of matter/anti-matter annihilation I've every seen before. I think you may be conflating the concepts of clouds of virtual electron/positron particles with actual annihilation events between real electron/positron particles. In actual annihilation events, there is (as far as I know) generally a single real photon produced, with momentum equal to the sum of the momentum vectors of the original electron and positron. That moves away from the point of production at the speed of light. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Oct 5 00:32:50 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 15:32:50 +1100 Subject: Constants [was Re: newb question about @property] References: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <59d4f958$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5b5f4$0$14931$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 04:00 am, Paul Moore wrote: > I wonder - would the people who want "real constants" find the > following confusing: > >>>> from demo import foo >>>> foo = 14 >>>> foo > 14 > > It's fundamental to the way the import function works, and how names > in Python behave, but I can see someone with a background in other > languages with "real" constants thinking "but foo is a constant, and > importing it stops it being a constant!" Of course it would be confusing. Just as "from module import foo" can be confusing today, without constants. People are surprised by at least two things: - if foo is mutable, they may be surprised that importing it doesn't make a copy; mutating the "imported copy" will show up everywhere; - they may be surprised that the local name "foo" isn't an alias to the qualified name "demo.foo": if demo.foo changes, foo does not. So is early binding of function defaults. And in other contexts, so is late binding of function defaults. Scoping and name binding rules are something one has to learn. When I first learned Python, they caused me trouble, and I'm sure they will cause any beginner to programming trouble. Adding constants to the language won't change that. Besides, if we had constants: const foo = 1234 then we could have: from demo import const foo -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From breamoreboy at gmail.com Thu Oct 5 00:43:29 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 4 Oct 2017 21:43:29 -0700 (PDT) Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thursday, October 5, 2017 at 4:22:26 AM UTC+1, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 01:21 pm, Stefan Ram wrote: > > >>- Germany was the aggressor in World War 2; > >>- well, Germany and Japan; > >>- *surely* it must be Germany, Italy and Japan; > > > > This listing style reminds me of of a listing style used in > > ?Falsehoods Programmers Believe About Names?: > > Yes. > > For the record: > > The USSR and Germany invaded Poland simultaneously, and the two countries > divided Poland between them. Churchill himself wrote about having difficulty > convincing people to care, since Poland had earlier invaded Czechoslovakia > (opportunistically while the Germans were seizing the Sudetenland) and many > people in England thought that Poland deserved their fate. Likewise the USSR > had invaded Finland. Germany had all ready grabbed Austria and pretty much all of Czechoslovakia. The USSR invaded Poland 16 days after Germany did. You've missed out Rumania and Hungary grabbing bits of central Europe while they had a chance. > > The UK invaded and occupied neutral Iceland in 1940, handing over control to > the USA in 1941. Despite signing an agreement in 1948 to leave within 6 > months, US military forces actually did not finally leave Iceland, and return > full sovereignty to the Icelander government, until 2006. The Icelandic government quite happily worked with the UK government as their ruler, the King of Denmark, was preoccupied with the state of affairs at home. The USA has a habit of hanging around but they disliked European colonialists. Pot, kettle, black? > > In the East, while Japan did take the first overtly military action against > the US, the US had (in some sense) first engaged in hostile behaviour against > Japan by unilaterally imposing, and enforcing, sanctions on Japan. > > (I make no comment on whether such sanctions were justified or not, only that > they can be seen as a form of aggressive economic warfare.) Japan had been in control of Korea since 1905, had been involved in China from 1931 and been in all out war with the Chinese from 1937. As the USA had a strong relationship with China it does not surprise me that they took such action. > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. -- Kindest regards. Mark Lawrence. From steve+python at pearwood.info Thu Oct 5 00:47:29 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 15:47:29 +1100 Subject: OT I before E [was Re: Lies in education [was Re: The "loop and a half"]] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5b964$0$14964$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 02:54 pm, Chris Angelico wrote: > On Thu, Oct 5, 2017 at 2:22 PM, Steve D'Aprano > wrote: >> It is, I think, an example of a stupid English language folklore that >> people repeat unthinkingly, even though the counter-examples are obvious >> and common. Like "I before E except after C", which is utter rubbish. > > That's because it's only half the rule. I before E except after C, > when the sound is "aye" or the sound is "ee". If the sound isn't one > of those, the rule doesn't apply. So yeah, in that form, it's utter > rubbish. Even when the sound is "A" or "EE", there are still exceptions. (As you say.) http://www.worldwidewords.org/articles/ar-ibe1.htm The best version of the rule I ever heard was: "I before E, EXCEPT after C, AND when sounding like A, as in neighbor, and weigh, and on weekends, and holidays, and all throughout May, and you'll always be wrong NO MATTER WHAT YOU SAY!" https://en.wikiquote.org/wiki/Brian_Regan > (There are exceptions even to the longer form of the rule, but only a > handful. English isn't a tidy language.) Even with the longer version of the rule, there are so few applicable cases, and enough difficulty in applying the rule correctly, that the rule is not worth the breath it takes to say it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Thu Oct 5 01:21:07 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Oct 2017 16:21:07 +1100 Subject: How to determine lowest version of Python 3 to run? References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Message-ID: <85vaju2mrg.fsf@benfinney.id.au> Christopher Reimer writes: > How do I determine the lowest version of Python to [declare that my > code supports]? You can determine that by installing all the Python versions you want to try, and running your code's unit test suite on each of them. Those versions where the unit test suite fails, you can know that your code does not run on that version of Python. do this each time you make a significant change to the code; certainly before you release any changes to the world. Automate this procedure with an automated testing tool such as Tox . > I'm leaning towards installing the latest minor version of each > available major version, running tox to run the unit tests against > each one, and seeing what blows up. That sounds to me like a good approach. -- \ ?Working out the social politics of who you can trust and why | `\ is, quite literally, what a very large part of our brain has | _o__) evolved to do.? ?Douglas Adams | Ben Finney From stephanh42 at gmail.com.invalid Thu Oct 5 01:44:02 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 05 Oct 2017 05:44:02 GMT Subject: How to determine lowest version of Python 3 to run? References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> <85vaju2mrg.fsf@benfinney.id.au> Message-ID: Op 2017-10-05, Ben Finney schreef : > Christopher Reimer writes: > >> How do I determine the lowest version of Python to [declare that my >> code supports]? > > You can determine that by installing all the Python versions you want to > try, and running your code's unit test suite on each of them. An easy way to get a bunch of different Python versions is to just pull the various Docker containers for them. https://hub.docker.com/r/_/python/ Then run your unit tests in each of them. Note that Python 3.3 support officially ended on 2017-09-29, so at the moment I wouldn't bother to support anything lower than 3.4. That means testing 3.4, 3.5 and 3.6 (and 3.7 prerelease if you want to be thorough). Stephan From timothy.c.delaney at gmail.com Thu Oct 5 02:05:26 2017 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Thu, 5 Oct 2017 17:05:26 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On 5 October 2017 at 14:22, Steve D'Aprano wrote: > The A and E in the word "are" are not vowels, since they are silent. The U > in "unicorn" and "university" are not vowels either, and if you write "an > unicorn" you are making a mistake. > There are dialects of English where the "u" in unicorn or university would be pronounced "oo" (e.g. where heavily influenced by Spanish or Portuguese), in which case writing "an unicorn" would not necessarily be a mistake. For a similar example, I and most Australians would say that writing "an herb" is a mistake since we pronounce the "h", but millions of people elsewhere would disagree with us. Tim Delaney From steve+python at pearwood.info Thu Oct 5 02:11:16 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 17:11:16 +1100 Subject: newb question about @property References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <00d4788d-0bbc-7113-f500-4c0c5d64e189@kynesim.co.uk> Message-ID: <59d5cd06$0$14936$b1db1813$d948b532@news.astraweb.com> On Wed, 4 Oct 2017 11:46 pm, Rhodri James wrote: > On 04/10/17 12:07, bartc wrote: >> I've seen that example brought up before. It still doesn't cut any ice. >> >> You might as well be condescending of someone who finds Joyce or Proust >> unreadable, and prefers McBain, Simenon or Chandler. (Sorry, can't think >> of any modern pulp novelists). > > I don't think your comparison is appropriate. Joyce and Proust strike > me as the literary equivalent of Perl or APL; very clever but nearly > unreadable even for experts. No, think rather of Terry Pratchett. +1 for mentioning Sir PTerry! And an extra bonus for it actually being relevant :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From stephanh42 at gmail.com.invalid Thu Oct 5 02:24:56 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 05 Oct 2017 06:24:56 GMT Subject: Multithreaded compression/decompression library with python bindings? References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: Op 2017-10-04, Paul Moore schreef : > On 4 October 2017 at 16:08, Steve D'Aprano wrote: >> On Wed, 4 Oct 2017 08:19 pm, Thomas Nyberg wrote: >> >>> Hello, >>> >>> I was wondering if anyone here knew of any python libraries with >>> interfaces similar to the bzip2 module which is also multithreaded in >>> (de)compression? Something along the lines of (say) the pbip2 program >>> but with bindings for python? In a pinch: with open("myoutfile.gz", "wb") as f: sp = subprocess.Popen(("gzip",), stdin=subprocess.PIPE, stdout=f) sp.stdin.write(b"Hello, world\n") sp.stdin.close() Does compression in a separate process ;-) Stephan From steve+python at pearwood.info Thu Oct 5 02:45:09 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 05 Oct 2017 17:45:09 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d5d4f8$0$14951$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 03:54 pm, Stefan Ram wrote: > Steve D'Aprano writes: >>In actual annihilation events, there is (as far as I know) >>generally a single real photon produced > > ?There are only a very limited set of possibilities for > the final state. The most probable is the creation of > two or more gamma ray photons? > > en.wikipedia.org/wiki/Electron%E2%80%93positron_annihilation. > > "two or more" Ah, that makes sense: as you say: > (Conservation of momentum requires that two particles whose > center of gravity is at rest cannot annihilate to a single > photon with a non-zero momentum.) Of course it can't. In hindsight it is obvious. But that demonstrates exactly the sort of thing I'm talking about. I'm not an expert at quantum mechanics. I've seen (or at least, I remember seeing) diagrams of matter/antimatter annihilation with the two particles coming together and a single photon coming out: a simplified and strictly wrong view of the physics. e+ \ \ /-\ /-\ / \-/ \-/ / e- It turns out that's half the Feynman diagram for Bhabha scattering, not annihilation. I cannot say for sure whether I have ever been explicitly taught (in a book, say) that there is one photon, but I'm sure I've seen equations like: e+ + e- -> ? Writing the equation as: e+ + e- -> 2? would be equally wrong, since there may not be two photons -- that's just the most likely case. So thank you, today I learned something. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Thu Oct 5 02:57:22 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 05 Oct 2017 19:57:22 +1300 Subject: The "loop and a half" In-Reply-To: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > I recall that the Pascal compiler had to do some clever behind the scenes > jiggery-pokery to get eof() to work, but that's what compilers are supposed > to do: make common tasks easy for the programmer. Sometimes the jiggery-pokery worked, sometimes it didn't. For example, the following wouldn't work as expected: while not eof(input) do begin write(output, 'Enter something:'); readln(input, buffer); process(buffer); end because the eof() would block waiting for you to enter something, so the prompt wouldn't get printed at the right time. Basically, Pascal's eof-flag model was designed for batch processing, and didn't work very well for interactive use. Unix and Windows sidestep the issue by not trying to pretend you can detect EOF independently of reading data, but using that model gracefully requires a loop-and-a-half. -- Greg From __peter__ at web.de Thu Oct 5 02:59:55 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 05 Oct 2017 08:59:55 +0200 Subject: The "loop and a half" References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: Stefan Ram wrote: > bartc writes: >>Note that your reverse-indentation style is confusing! > > In Python, indentation can be significant. > > Sometimes, some lines in Python must be indented by 0. Are there any editors that do not support a dedent operation? In the interactive interpreter you can work around print("Oh my god, this won't run!") with >>> if "heureka": ... print("Oh my god, this won't run!") ... Oh my god, this won't run! > This dictates that Python code cannot be indented > in posts to differentiate it from the natural-language > body of the post. Therefore, it's rather the > natural-language body that has to be indented. That appears compelling. Can you find a flaw in the logic? From rosuav at gmail.com Thu Oct 5 03:11:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 18:11:51 +1100 Subject: The "loop and a half" In-Reply-To: References: <1ac5c25a-8a78-7683-ef5f-66176ed4bcbd@chamonix.reportlab.co.uk> Message-ID: On Thu, Oct 5, 2017 at 5:59 PM, Peter Otten <__peter__ at web.de> wrote: >> This dictates that Python code cannot be indented >> in posts to differentiate it from the natural-language >> body of the post. Therefore, it's rather the >> natural-language body that has to be indented. > > That appears compelling. Can you find a flaw in the logic? > It assumes that the Python code and the English code are completely independent. If they are not, and if (as is very common) the Python is logically nested inside the English, then according to the rules of both languages, the Python code should be indented. ChrisA From tjol at tjol.eu Thu Oct 5 04:23:18 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 10:23:18 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-05 06:24, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 02:14 pm, Stefan Ram wrote: > >> Chris Angelico writes: >>> You can get through a lot of life believing that mass is conserved, >>> but technically it is not, as can be proven. >> >> Well, in fact, it is conserved. > > It certainly is not. The whole point of Einstein's equation E = mc? is that > energy and mass are freely convertible. Neither energy nor mass alone are > conserved. You're both right, you're just defining "mass" in different ways. > > >> When an electron and a positron annihilate to give a gas of >> two photons, this gas as a whole still has the same mass as >> the electron before. And its center of gravity moves with >> less than c. > > That doesn't sound like any description of matter/anti-matter annihilation > I've every seen before. > > I think you may be conflating the concepts of clouds of virtual > electron/positron particles with actual annihilation events between real > electron/positron particles. > > In actual annihilation events, there is (as far as I know) generally a single > real photon produced, with momentum equal to the sum of the momentum vectors > of the original electron and positron. That moves away from the point of > production at the speed of light. > > > From tjol at tjol.eu Thu Oct 5 04:30:16 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 10:30:16 +0200 Subject: OT I before E [was Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: <59d5b964$0$14964$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b964$0$14964$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-05 06:47, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 02:54 pm, Chris Angelico wrote: >> (There are exceptions even to the longer form of the rule, but only a >> handful. English isn't a tidy language.) > > Even with the longer version of the rule, there are so few applicable cases, > and enough difficulty in applying the rule correctly, that the rule is not > worth the breath it takes to say it. Maybe we should just all switch to Dutch on this list. Might be easier. Certainly more consistent. groetjes Thomas From tomuxiong at gmx.com Thu Oct 5 04:38:33 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 5 Oct 2017 10:38:33 +0200 Subject: Multithreaded compression/decompression library with python bindings? In-Reply-To: <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/04/2017 05:08 PM, Steve D'Aprano wrote: > pbip2? Never heard of it, and googling comes up with nothing relevant. > > Got a link? Sorry it was a typo as Paul Moore said. pbzip2 is a parellelized implementation of bzip2: http://compression.ca/pbzip2/ > Why obviously? Sorry again. I certainly shouldn't have said obviously. In fact, I may be wrong. However, due to the fact that the compression algorithms are almost entirely CPU-bound, I would be impressed if someone managed to parallelize it within python itself due to the gil etc. But I don't want to discount the possibility. In my experience many people throw their hands up too early and decide something is impossible due to the gil even when it's not. (As apparently I did...) Basically here's the gist of the problem (at least as I understand it). The way that bzip2 works is that it breaks data into blocks of a certain size and then compresses those blocks individually. This effectively makes the operation embarrassingly parallel on these blocks. The pbzip2 program takes advantage of this and does the compression and decompression in parallel on these blocks leading to an almost linear speedup in the number of cpus available. Both the inputs and outputs of bzip2 and pbzip2 are compatible, though to gain the speedup the files must be compressed with pbzip2. (I.e. for some reason pbzip2 cannot decompress a file in parallel if it's been compressed with bzip2. There is apparently some freedom in the object format. I'm not totally sure why this is.) I did some work on this yesterday so I'll put in the code I've gotten so far which mostly works (I have an issue with waiting on a subprocess as explained below). Also I'm running debian and using that heavily so I presume nothing here works on Windows (sorry :( ). In my case what I essentially wanted was just a very simple "compressed archiver". I.e. a bzip2-compressed tar archive that I could fill up in one pass and read from in one pass. I already had one using python's bz2 module, but I wanted a parallel version. The version I came up with is the following (which is essentially what Stephen Houben was proposing): pbz2.py -------------------------------------------- import io import tarfile from subprocess import Popen, PIPE class Archive: def __init__(self, filepath, mode="r"): assert mode in {"r", "w"} if mode == "r": self._p = Popen(["pbzip2", "-dc", filepath], stdout=PIPE) self._tar = tarfile.open(filepath, mode="r|", fileobj=self._p.stdout) # Seems like an odd way to do this, but works for now. def tar_yielder(): for tarinfo in self._tar: file_name = tarinfo.name file_bytes = self._tar.extractfile(tarinfo).read() file_contents = file_bytes.decode('utf-8') yield file_name, file_contents self._tar_yielder = tar_yielder() else: self._p = Popen(["pbzip2", "-zc"], stdin=PIPE, stdout=open(filepath, "w")) self._tar = tarfile.open(filepath, mode="w|", fileobj=self._p.stdin) def __enter__(self): return self def __exit__(self, *args): self.close() def __iter__(self): return self def __next__(self): return next(self._tar_yielder) def write(self, file_name, file_contents): file_contents = file_contents.encode('utf-8') # The tar archiver requires file objects. bytesstring = io.BytesIO(file_contents) info = tarfile.TarInfo(name=file_name) info.size=len(bytesstring.getbuffer()) self._tar.addfile(tarinfo=info, fileobj=bytesstring) def close(self): self._tar.close() -------------------------------------------- You could use it as follows: -------------------------------------------- with Archive("archive.tar.bz2", mode="w") as outfile: outfile.write("file1", "file 1 contents") outfile.write("file2", "file 2 contents") outfile.write("file3", "file 3 contents") -------------------------------------------- And then can iterate this way: -------------------------------------------- with Archive("archive.tar.bz2") as infile: for file_name, file_contents in infile: print(file_name) print(file_contents) -------------------------------------------- One problem though is that I have to run those two parts as two _separate_ scripts. Otherwise I get errors from pbzip2. I'm pretty sure it's a race condition. I.e. when the first archive finishes, it is not waiting on the subprocess and so the second archive may try to iterate before that part is finished. Does anyone here know the best way to wait on the subprocess? Naively the right things seems to be to add a wait in to the close method as follows: -------------------------------------------- def close(self): self._tar.close() self._p.wait() -------------------------------------------- However if I do that, the (parent) process will simply hang. If I don't have it everything seems to work fine as long as I let the parent process itself finish. Does anyone here know the best way to wait on the subprocess? Thanks for any help! Cheers, Thomas From tomuxiong at gmx.com Thu Oct 5 04:46:00 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 5 Oct 2017 10:46:00 +0200 Subject: Multithreaded compression/decompression library with python bindings? In-Reply-To: References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: <8c9801db-eece-2c87-e322-314b1f9d3e20@gmx.com> Btw if anyone knows a better way to handle this sort of thing, I'm all ears. Given my current implementation I could use any compression that works with stdin/stdout as long as I could sort out the waiting on the subprocess. In fact, bzip2 is probably more than I need...I've half used it out of habit rather than anything else. I do personally really like the convenience of this compressed tar archive format since I can just grab the files and run `tar xf filename` in a terminal and get them all back again, but if there's an entirely different and better way which allows me to stream in my fashion, I'm certainly curious to hear about it. Cheers, Thomas From fabien.maussion at gmail.com Thu Oct 5 05:46:46 2017 From: fabien.maussion at gmail.com (Fabien) Date: Thu, 5 Oct 2017 11:46:46 +0200 Subject: Easier way to do this? References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <8a6af43a-b323-aa0d-609f-d0ae3da354f3@tjol.eu> Message-ID: On 10/05/2017 12:00 AM, Thomas Jollans wrote: > On 04/10/17 22:47, Fabien wrote: >> On 10/04/2017 10:11 PM, Thomas Jollans wrote: >>> Be warned, pandas is part of the scientific python stack, which is >>> immensely powerful and popular, but it does have a distinctive style >>> that may appear cryptic if you're used to the way the rest of the world >>> writes Python. >> Can you elaborate on this one? As a scientist, I am curious;-) > Sure. > > Python is GREAT at iterating. Generators are everywhere. Everyone loves > for loops. List comprehensions and generator expressions are star > features. filter and map are builtins. reduce used be a builtin, even > though almost nobody really understood what it did. > > In [1]: import numpy as np > > In the world of numpy (and the greater scientific stack), you don't > iterate. You don't write for loops. Right, now I see what you meant. The first thing I teach to my students is that loops are evil, and that they should always avoid them when possible ;-) Cheers, Fabien From bc at freeuk.com Thu Oct 5 06:56:59 2017 From: bc at freeuk.com (bartc) Date: Thu, 5 Oct 2017 11:56:59 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 05/10/2017 07:57, Gregory Ewing wrote: > Steve D'Aprano wrote: >> I recall that the Pascal compiler had to do some clever behind the scenes >> jiggery-pokery to get eof() to work, but that's what compilers are >> supposed >> to do: make common tasks easy for the programmer. > > Sometimes the jiggery-pokery worked, sometimes it didn't. > For example, the following wouldn't work as expected: > > ?? while not eof(input) do begin > ????? write(output, 'Enter something:'); > ????? readln(input, buffer); > ????? process(buffer); > ?? end > > because the eof() would block waiting for you to enter > something, so the prompt wouldn't get printed at the > right time. > > Basically, Pascal's eof-flag model was designed for > batch processing, and didn't work very well for > interactive use. This doesn't make sense. For interactive use, you wouldn't bother testing for eof, as you'd be testing the eof status of the keyboard. You might want a way of the user indicating end-of-data, but that's different; you don't want to abruptly send an EOF (via Ctrl-C, D, Z, Break or whatever). That would be a crass way of doing it. Besides you might want to continue interacting with the next part of the program. So the loop would be like this (Python 3; don't know why it doesn't work in Python 2): while 1: buffer = input("Enter something (Type quit to finish): ") if buffer == "quit": break print ("You typed:", buffer) # process(buffer) print ("Bye") This is a loop-and-a-half. -- bartc From rosuav at gmail.com Thu Oct 5 07:09:02 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 5 Oct 2017 22:09:02 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 9:56 PM, bartc wrote: > This doesn't make sense. For interactive use, you wouldn't bother testing > for eof, as you'd be testing the eof status of the keyboard. You mean the way heaps and heaps of Unix programs work, processing until EOF of stdin? Yeah, totally makes no sense, man, no sense at all. ChrisA From marko at pacujo.net Thu Oct 5 07:15:58 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 05 Oct 2017 14:15:58 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: <871smhu9ox.fsf@elektro.pacujo.net> ram at zedat.fu-berlin.de (Stefan Ram): > Some of you are experts in Python, but are only > half-educated when it comes to physics, C++ or other topics. > I wish those persons would not broadcast their > half-knowledge in the form of confident statements, I don't wish that. That would only establish priesthoods and be an obstacle to learning. > and I wish they would talk about those subjects in their > appropriate newsgroups. (For example, one can ask questions > about physics in "sci.physics.research".) That's good advice, but it's not all that dangerous to express off-topic statements in this newsgroup. Marko From tomuxiong at gmx.com Thu Oct 5 07:18:39 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 5 Oct 2017 13:18:39 +0200 Subject: How do native namespaces work? Message-ID: Hello, I'm trying to understand native namespaces. I'm currently using python 3.5 as packaged in debian 9. I've been following the instructions here: https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages Those instructions link to the following example: https://github.com/pypa/sample-namespace-packages/tree/master/native I presume I'm doing something silly, but I can't get that to work. I've tried creating a virtual environment as: -------------------------------------------------- $ python3 -m venv venv $ source venv/bin/activate -------------------------------------------------- And then I've gone into those repo folders and run their setup.py files (e.g. for namespace a): -------------------------------------------------- $ cd sample-namespace-packages/native/pkg_a/ $ python3 setup.py install -------------------------------------------------- Then if I try to run their sample verification file, things fail: -------------------------------------------------- $ cat sample-namespace-packages/verify_packages.py from example_pkg import a from example_pkg import b print(a.name) print(a.__path__) print(b.name) print(b.__path__) $ python3 sample-namespace-packages/verify_packages.py Traceback (most recent call last): File "sample-namespace-packages/verify_packages.py", line 1, in from example_pkg import a ImportError: No module named 'example_pkg' -------------------------------------------------- Things seem to be installing: -------------------------------------------------- $ ls venv/lib/python3.5/site-packages/ easy-install.pth pkg_resources easy_install.py pkg_resources-0.0.0.dist-info example_pkg_a-1-py3.5.egg __pycache__ example_pkg_b-1-py3.5.egg setuptools pip setuptools-32.3.1.dist-info pip-9.0.1.dist-info -------------------------------------------------- Am I missing something totally obvious here? Does anyone here see what I'm doing wrong? Thanks for any help! Cheers, Thomas From bc at freeuk.com Thu Oct 5 07:26:26 2017 From: bc at freeuk.com (bartc) Date: Thu, 5 Oct 2017 12:26:26 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 05/10/2017 12:09, Chris Angelico wrote: > On Thu, Oct 5, 2017 at 9:56 PM, bartc wrote: >> This doesn't make sense. For interactive use, you wouldn't bother testing >> for eof, as you'd be testing the eof status of the keyboard. > > You mean the way heaps and heaps of Unix programs work, processing > until EOF of stdin? Yeah, totally makes no sense, man, no sense at > all. Out of the hundreds, perhaps thousands of such input loops I must have written, how many needed to test EOF? Hmm, somewhere around zero I think. Oh hang on, I wasn't using Unix; does that make a difference? If you're referring to the ability to redirect stdin so that input can come from a file as well as from a live keyboard, then you're doing file handling; it's NOT interactive. (And I've used such programs where you get no prompt as to what to type, or how to finish, and you just blindly press Ctrl C, Ctrl D, Ctrl Z or Ctrl Break in turn until it stops. Really friendly. Because the program reads from stdin but assumes it comes a from a file or pipe anyway.) Note that Unix way isn't the ONLY way of doing things. -- bartc From greg.ewing at canterbury.ac.nz Thu Oct 5 07:29:16 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Oct 2017 00:29:16 +1300 Subject: newb question about @property In-Reply-To: <5L6BB.1362664$ji4.1213053@fx10.am4> References: <59d022b7$0$14927$b1db1813$d948b532@news.astraweb.com> <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> Message-ID: bartc wrote: > Result? You can't just look at my 'any' class and see what fields it > uses. You can't even just look at the static source code. You have to > run the program to find out. And it might be different each time. You can usually get a pretty good idea of what attributes a class has by looking at its definition. The vast majority of classes will either initialise their attributes in the __init__ method or provide defaults as class variables. While in theory it's possible for code to add attributes later after initialisation, in practice this is hardly ever done. -- Greg From bc at freeuk.com Thu Oct 5 07:51:03 2017 From: bc at freeuk.com (bartc) Date: Thu, 5 Oct 2017 12:51:03 +0100 Subject: newb question about @property In-Reply-To: References: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> Message-ID: On 05/10/2017 12:29, Gregory Ewing wrote: > bartc wrote: >> Result? You can't just look at my 'any' class and see what fields it >> uses. You can't even just look at the static source code. You have to >> run the program to find out. And it might be different each time. > > You can usually get a pretty good idea of what attributes a > class has by looking at its definition. The vast majority of > classes will either initialise their attributes in the __init__ > method or provide defaults as class variables. > > While in theory it's possible for code to add attributes > later after initialisation, in practice this is hardly ever > done. Yeah, but, like many other things in this language, there are million ways of doing it. Just had a quick look, the first hit was talking about using a module 'pyrecord'. Another mentioned 'namedtuples'. Another used the '__slots__' method already mentioned (remembering the difference between old and new classes in Python 2...) One more uses a module 'recordtype'. And then there is just using a normal class, where you have to look closely at the code to see what it's implementing. It allow ad-hoc fields to be created, or it may define __init__ etc. And all with different capabilities regarding adding extra fields, having mutable records, initialising a record, comparing them, printing them, .... Am I allowed to say that it all seems a bit of a mess? -- bartc From dvl at psu.edu Thu Oct 5 08:37:26 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Thu, 5 Oct 2017 08:37:26 -0400 Subject: Python-list Digest, Vol 169, Issue 7 In-Reply-To: mailman.1097.1507171029.2818.python-list@python.org References: Message-ID: <1507207046l.3997912l.0l@psu.edu> On Wed, Oct 4, 2017 22:42 Stefan Ram (ram at zedat.fu-berlin.de) wrote: > Steve D'Aprano writes: >>So, "bottom-up" in this case means: iterators should be >>taught before for-loops. >>Why? > > The syntax for is (from memory): > >for in : > > . As an example, I might show: > >for i in range( 3 ): ... > > . This raises the question: > > "What is the value of ?range( 3 )??". > > "Bottom up" means that the simple and constituent parts > are taught before the complex and compound parts. I.e., > ?range( 3 )? is explained before ?for i in range( 3 ):?. > > The easy answer here is to not use the range in the first for loop. I introduce for with actual collections (e.g. lists, strings, dictionaries) It is very easy to figure out what "for i in [2, 6,32, 7, 5]" means. This is very easy to understand, and I teach it early, to avoid this common error: "if i == 2 or 6 or 32 or 7 or 5" which plagues Python and C programmers alike (only Java complains about that one) You can even access the proper subscripts for a list without using range(), if you are willing to use enumerate(). So, by teaching the for loop without the range, not only do you get to postpone the details of what range() does (and can even point out -- hey, that's exactly the same numbers as enumerate listed!) but you also get your students to believe that there is a better way to access the elements of a list than counting down the subscripts. I still see a lot of instructors and programmers who newly migrate to Python that use counting a range() as their first choice in traversing a list. Teaching the for loop without using range, I believe, is the best way to forestall that habit. Oh, and it also steers people from this error I often see: "for i in len(list)" Roger Christman Pennsylvania State University From steve+python at pearwood.info Thu Oct 5 09:13:55 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 00:13:55 +1100 Subject: newb question about @property References: <59d07c4b$0$14954$b1db1813$d948b532@news.astraweb.com> <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> Message-ID: <59d63014$0$14973$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 10:51 pm, bartc wrote: > Am I allowed to say that it all seems a bit of a mess? You may or may not be pleased to learn that there's a push to create a "record like" or "struct like" datatype for Python 3.7 or 3.8, tentatively called a "Data Class" for now. The proposed syntax will use class syntax, to make it easy to add methods. For example: @dataclass class InventoryItem: name: str unit_price: float quantity_on_hand: int = 0 def total_cost(self) -> float: return self.unit_price * self.quantity_on_hand which will fill in the boilerplate for you: - an appropriate initialiser __init__ - a good-looking __repr__ - equality, inequality, and rich comparison operators; - an optional __hash__ method. https://www.python.org/dev/peps/pep-0557/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From __peter__ at web.de Thu Oct 5 09:57:48 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 05 Oct 2017 15:57:48 +0200 Subject: Introducing the "for" loop References: <1507207046l.3997912l.0l@psu.edu> Message-ID: Stefan Ram wrote: > "ROGER GRAYDON CHRISTMAN" writes: >>On Wed, Oct 4, 2017 22:42 Stefan Ram (ram at zedat.fu-berlin.de) wrote: >>Steve D'Aprano writes: >>>>So, "bottom-up" in this case means: iterators should be >>>>taught before for-loops. >>>>Why? >>The easy answer here is to not use the range in the first for loop. > > I never intended to use ?range?. But I also will not use lists. > > Very early in the course, I teach numeric and string literals: > > 1, 2.3, 'abc' > > then come operators, functions and ?if?, ?while? and ?try?. > But neither ?range? nor lists have been shown so far. > > The basic course may already and there after about 12 - 18 hours. > (This time includes many exercises in the classroom.) > > But if I have time to introduce ?for?, I'll do it as follows > at this point in the course: > > To me it looks like Roger has some practical experience in that area. You should seriously consider his advice lest your students end up missing the forest for the trees. From __peter__ at web.de Thu Oct 5 10:07:48 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 05 Oct 2017 16:07:48 +0200 Subject: How do native namespaces work? References: Message-ID: Thomas Nyberg wrote: > Hello, > > I'm trying to understand native namespaces. I'm currently using python > 3.5 as packaged in debian 9. I've been following the instructions here: > > https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages > > Those instructions link to the following example: > > https://github.com/pypa/sample-namespace-packages/tree/master/native > > I presume I'm doing something silly, but I can't get that to work. I've > tried creating a virtual environment as: > > -------------------------------------------------- > $ python3 -m venv venv > $ source venv/bin/activate > -------------------------------------------------- > > And then I've gone into those repo folders and run their setup.py files > (e.g. for namespace a): > > -------------------------------------------------- > $ cd sample-namespace-packages/native/pkg_a/ > $ python3 setup.py install Are you sure you are using the correct interpreter? When I activate a virtual environment it changes the prompt like so: $ python3 -m venv venv $ . venv/bin/activate (venv) $ (venv) $ which python3 /home/peter/venv/bin/python3 (venv) $ deactivate $ $ which python3 /usr/bin/python3 > -------------------------------------------------- > > Then if I try to run their sample verification file, things fail: > > -------------------------------------------------- > $ cat sample-namespace-packages/verify_packages.py > from example_pkg import a > from example_pkg import b > > print(a.name) > print(a.__path__) > print(b.name) > print(b.__path__) > $ python3 sample-namespace-packages/verify_packages.py > Traceback (most recent call last): > File "sample-namespace-packages/verify_packages.py", line 1, in > from example_pkg import a > ImportError: No module named 'example_pkg' > -------------------------------------------------- > > Things seem to be installing: > > -------------------------------------------------- > $ ls venv/lib/python3.5/site-packages/ > easy-install.pth pkg_resources > easy_install.py pkg_resources-0.0.0.dist-info > example_pkg_a-1-py3.5.egg __pycache__ > example_pkg_b-1-py3.5.egg setuptools > pip setuptools-32.3.1.dist-info > pip-9.0.1.dist-info > -------------------------------------------------- > > Am I missing something totally obvious here? Does anyone here see what > I'm doing wrong? Thanks for any help! > > Cheers, > Thomas From tomuxiong at gmx.com Thu Oct 5 10:18:55 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Thu, 5 Oct 2017 16:18:55 +0200 Subject: How do native namespaces work? In-Reply-To: References: Message-ID: On 10/05/2017 04:07 PM, Peter Otten wrote: > Are you sure you are using the correct interpreter? When I activate a > virtual environment it changes the prompt like so: Sorry I just cut out the extra cruft from my prompt for clarity. (In hindsight, I should probably have left the `(venv)` prompt in my cleaned up prompt...) The venv was definitely activated (and in fact I tried again). Cheers, Thomas From neilc at norwich.edu Thu Oct 5 10:28:30 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Thu, 5 Oct 2017 14:28:30 +0000 (UTC) Subject: Easier way to do this? References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <87r2ui63at.fsf@bsb.me.uk> <4798c71e-f1e1-06bd-c138-1663c090043a@2020fresno.com> Message-ID: On 2017-10-04, 20/20 Lab wrote: > It's not quite a 'learning exercise', but I learn on my own if > I treat it as such.? This is just to cut down a few hours of > time for me every week filtering the file by hand for the > office manager. That looks like a 30-second job using a pivot table in Excel. Office manager, learn thy Excel! On the other hand, I think Python's csv module is a killer app, so I do recommend taking the opportunity to learn csv.DictReader and csv.DictWriter for your own enjoyment. -- Neil Cerutti From steve+python at pearwood.info Thu Oct 5 10:45:05 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 01:45:05 +1100 Subject: How to determine lowest version of Python 3 to run? References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Message-ID: <59d64573$0$14951$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 01:23 pm, Christopher Reimer wrote: > Greetings, > > I've always installed the latest and greatest version of Python 3 to develop > my own programs. I'm planning to release a program to the public. I could > toss in a note that the program runs on the latest version of Python 3.6 but > I haven't tested on earlier versions (i.e., 3.4 and 3.5). AFAIK, I haven't > written any version-specific code. > > How do I determine the lowest version of Python to run? If you know your code uses a feature introduced in version X, then X is the lowest version it will run in. E.g. if you use the u'' syntax for Unicode strings, which was re-introduced in 3.3 to make it easier to write Python 2/3 compatible code, then it won't run it Python 3.1 or 3.2. > I'm leaning towards installing the latest minor version of each available > major version, running tox to run the unit tests against each one, and > seeing what blows up. That seems reasonable to me, except I think you mean latest release of each minor version e.g.: 3.5.1 = major.minor.release I'd also consider testing against the *earliest* release, rather than the latest. If it works with (say) 3.5.0, then you should be safe to claim it works for all 3.5. But if it works for 3.5.4, say, you aren't *quite* so safe to make the same claim. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lab at 2020fresno.com Thu Oct 5 11:22:23 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Thu, 5 Oct 2017 08:22:23 -0700 Subject: Easier way to do this? In-Reply-To: References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <87r2ui63at.fsf@bsb.me.uk> <4798c71e-f1e1-06bd-c138-1663c090043a@2020fresno.com> Message-ID: <35bd1942-3285-b877-2a73-b5426c9ac62e@2020fresno.com> On 10/05/2017 07:28 AM, Neil Cerutti wrote: > On 2017-10-04, 20/20 Lab wrote: >> It's not quite a 'learning exercise', but I learn on my own if >> I treat it as such.? This is just to cut down a few hours of >> time for me every week filtering the file by hand for the >> office manager. > That looks like a 30-second job using a pivot table in Excel. > Office manager, learn thy Excel! > > On the other hand, I think Python's csv module is a killer app, > so I do recommend taking the opportunity to learn csv.DictReader > and csv.DictWriter for your own enjoyment. > It would be if our practice management software would export to excel, or even a realistic csv.? Problem is that it only exports to csv and it's 80-90% garbage and redundant information.? I've taken to bringing the csv into excel and refining it so I can do that, but again.? I'd rather take half a second and have a program do it for me.? ;) From lab at 2020fresno.com Thu Oct 5 11:34:24 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Thu, 5 Oct 2017 08:34:24 -0700 Subject: Easier way to do this? In-Reply-To: References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> Message-ID: <90393524-e646-e0e5-7237-1d330991c32a@2020fresno.com> On 10/04/2017 04:48 PM, Dennis Lee Bieber wrote: > On Wed, 4 Oct 2017 09:42:18 -0700, 20/20 Lab declaimed > the following: > > Well -- since your later post implies this is not some "homework > assignment"... > >> Looking for advice for what looks to me like clumsy code. >> >> EMP1 = [0,0] >> EMP2 = [0,0] >> EMP3 = [0,0] >> > EEEK! Don't do that! Especially as... > >> for line in (inputfile): >> ??? content = line.split(",") > You've already been told to use the CSV module, since it should handle > tricky cases (quoted strings with embedded commas, say). > >> ??? if content[18] == "EMP1": > ... the name is part of the input data. Use a data structure in which the > name is part of the data -- like a dictionary. > >> ??????? if float(content[24]) < 99.75: >> ??????????? EMP1[0] += 1 >> ??????? elif float(content[24]) > 99.74: >> ??????????? EMP1[1] += 1 > Pardon? Floating point numbers are not exact... It is possible that > some entry, in floating binary, is really between 99.75 and 99.74, so which > should it be counted as? At the least, just use an else: for the other > case. > > > > employees = {} > for row in csvfile: #pseudo-code for however you read each row of data > > emp = employees.get(content[18], [0, 0]) > if float(content[24]) < 99.75 > emp[0] += 1 > else: > emp[1] += 1 > employees[content[18]] = emp > > >> and repeat if statements for the rest of 25+ employees.? I can make a >> list of the employees, but I'd prefer to pull them from the csv, as our >> turnover is rather high (however this is not important).? I'm thinking >> another "for employee in content[18]" should be there, but when I tried, >> my numbers were incorrect. > Actually -- I'd be more likely to load the data into an SQLite3 > database, and use SQL queries to produce the above summary report. I'd have > to experiment with subselects to get the > and < sets, and then use a > count() and groupby to put them in order. Thanks!? I knew there was an more refined way to do this.? I'm still learning, so this is a huge help. The data actually already comes from a database, but doesnt allow me to produce a summary report, just a list of of each item.? Which I can then export to pdf, or the csv that I'm working with.? The developers have a ticket for a feature request, but they've had it there for over five years and I dont see them implementing it anytime soon. From lab at 2020fresno.com Thu Oct 5 11:43:30 2017 From: lab at 2020fresno.com (20/20 Lab) Date: Thu, 5 Oct 2017 08:43:30 -0700 Subject: Easier way to do this? In-Reply-To: <302A89DC-2C11-4EDC-8986-4314BE7ADBF2@furrypants.com> References: <96fd5bca-77ef-3a97-786e-c50d5e6ffbae@2020fresno.com> <302A89DC-2C11-4EDC-8986-4314BE7ADBF2@furrypants.com> Message-ID: <21876594-5990-0d07-1e2b-6efb1800df6e@2020fresno.com> On 10/04/2017 05:11 PM, Irv Kalb wrote: > I'm assuming from your posts that you are not a student. If that is the case, look at my solution below. > >> On Oct 4, 2017, at 9:42 AM, 20/20 Lab wrote: >> >> Looking for advice for what looks to me like clumsy code. >> >> I have a large csv (effectively garbage) dump. I have to pull out sales information per employee and count them by price range. I've got my code working, but I'm thinking there must be a more refined way of doing this. >> >> ---snippet of what I have--- >> >> EMP1 = [0,0] >> EMP2 = [0,0] >> EMP3 = [0,0] >> >> for line in (inputfile): >> content = line.split(",") >> if content[18] == "EMP1": >> if float(content[24]) < 99.75: >> EMP1[0] += 1 >> elif float(content[24]) > 99.74: >> EMP1[1] += 1 >> if content[18] == "EMP2": >> if float(content[24]) < 99.75: >> EMP2[0] += 1 >> elif float(content[24]) > 99.74: >> EMP2[1] += 1 >> if content[18] == "EMP3": >> if float(content[24]) < 99.75: >> EMP3[0] += 1 >> elif float(content[24]) > 99.74: >> EMP3[1] += 1 >> >> and repeat if statements for the rest of 25+ employees. I can make a list of the employees, but I'd prefer to pull them from the csv, as our turnover is rather high (however this is not important). I'm thinking another "for employee in content[18]" should be there, but when I tried, my numbers were incorrect. >> >> Any help / advice is appreciated, >> >> Matt >> > > You could certainly use the csv module if you want, but this builds on your start of dealing with the data line by line. > > Completely untested, but this approach works by building a dictionary on the fly from your data. Each key is an employee name. The data associated with each key is a two item list of counts. > > > # Constants > NAME_INDEX = 18 > SALES_INDEX = 24 > THRESHHOLD = 99.75 > > salesCountDict = {} # start with an empty dict > > for line in (inputfile): > content = line.split(",") # split the line into a list > name = content[NAME_INDEX] # extract the name from the content list > > # If we have not seen this employee name before, add it to the dictionary > # like key value pair: '': [0, 0] > if not(name in employeeDataDict): > salesCountDict[name] = [0, 0] > > price = float(content[SALES_INDEX]) # extract the price > > # If the price is under some threshhold, increment one value in the associated sales list > # otherwise increment the other > if price < THRESHHOLD: > salesCountDict[name][0] += 1 > else: > salesCountDict[name][1] += 1 > > > # Now you should have a dictionary. Do what you want with it. For example: > > for name in salesCountDict: > salesList = salesCountDict[name] > print(name, salesList) # Assuming Python 3 > > > > > Thanks for this.? I've recently had a hard time discerning when to use / the differences of the dict, list, set, tuple.? So this is a huge help. From grant.b.edwards at gmail.com Thu Oct 5 11:54:05 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 5 Oct 2017 15:54:05 +0000 (UTC) Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-05, bartc wrote: > Note that Unix way isn't the ONLY way of doing things. Of course not, but it is the CORRECT way of doing things. :) -- Grant Edwards grant.b.edwards Yow! Am I in GRADUATE at SCHOOL yet? gmail.com From navyad.pro at gmail.com Thu Oct 5 12:17:23 2017 From: navyad.pro at gmail.com (Naveen Yadav) Date: Thu, 5 Oct 2017 09:17:23 -0700 (PDT) Subject: type and object Message-ID: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> Hi folks, >> isinstance(type, object) is True # 1 and >> isinstance(object, type) is True # 2 its means type is object is type, But i am not sure how is this. For what i can understand is for #1: since every thing is object in python, type is also an object. and for #2: object is a base type. therefore object is type >> object.__doc__ >> 'The most base type' Am I understanding this correct ? Please shed light on this. From rosuav at gmail.com Thu Oct 5 13:01:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 04:01:45 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, Oct 5, 2017 at 10:26 PM, bartc wrote: > On 05/10/2017 12:09, Chris Angelico wrote: >> >> On Thu, Oct 5, 2017 at 9:56 PM, bartc wrote: >>> >>> This doesn't make sense. For interactive use, you wouldn't bother testing >>> for eof, as you'd be testing the eof status of the keyboard. >> >> >> You mean the way heaps and heaps of Unix programs work, processing >> until EOF of stdin? Yeah, totally makes no sense, man, no sense at >> all. > > > Out of the hundreds, perhaps thousands of such input loops I must have > written, how many needed to test EOF? Hmm, somewhere around zero I think. > > Oh hang on, I wasn't using Unix; does that make a difference? > > If you're referring to the ability to redirect stdin so that input can come > from a file as well as from a live keyboard, then you're doing file > handling; it's NOT interactive. How would you write a sort program? How would you tell it that you're done entering data? And yes, I have used sort(1) interactively, with no redirection whatsoever. ChrisA From rosuav at gmail.com Thu Oct 5 13:04:28 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 04:04:28 +1100 Subject: type and object In-Reply-To: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> References: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> Message-ID: On Fri, Oct 6, 2017 at 3:17 AM, Naveen Yadav wrote: > Hi folks, > > >>> isinstance(type, object) is True # 1 > and >>> isinstance(object, type) is True # 2 > > > its means type is object is type, But i am not sure how is this. > > For what i can understand is > for #1: since every thing is object in python, type is also an object. > and > for #2: object is a base type. therefore object is type >>> object.__doc__ >>> 'The most base type' > > > Am I understanding this correct ? Please shed light on this. Yep :) Everything's an object, everything has a type, every type has an object. The hierarchy has a little loop in it at the top. ChrisA From steve+python at pearwood.info Thu Oct 5 13:04:53 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 04:04:53 +1100 Subject: Pedagogical style [was Re: The "loop and a half"] References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <877ewb6qi4.fsf@bsb.me.uk> Message-ID: <59d66637$0$14949$b1db1813$d948b532@news.astraweb.com> On Thu, 5 Oct 2017 07:29 am, Christian Gollwitzer wrote: > To understand Stefan's way of teaching, take a look at his other > courses, for example the C++ course: Thanks for this Christian. It has been an eye-opener. More comments below. > http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/c++-kurs > > He defends the opinion that you learn a programming language best by > studying syntax diagrams. He even writes in his FAQ about the language > courses: > http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/faq_programmieren > > "Neuere Forschungen haben gezeigt, da? es erfolgreicher ist, formale > Systeme (wie Programmiersprachen oder Mathematik) zu unterrichten, wenn > dabei kein k?nstlicher ?Praxisbezug? durch ?real-world examples? > (Beispiele aus dem ?wahren Leben?) hergestellt wird!" > > which means > > "Recent research results show that it is more successful, to teach > formal systems (like programming language or maths), if no artificial > practical use-case is demonstrated by "real-world examples" with a > reference: "They said students who were taught abstract math concepts > fared better in experiments than those taught with real-world examples, > such as story problems. > > Adding extraneous details makes it hard for students to extract the > basic mathematical concepts and apply them to new problems, they said. > > "We're really making it difficult for students because we are > distracting them from the underlying math," said Jennifer Kaminski, a > research scientist at Ohio State University, whose study appears in the > journal Science.Reuters, 25. April 2008" There's no link to the original paper, only to secondary sources that discuss it, e.g.: http://phys.org/pdf128266927.pdf so I cannot really judge the quality of the paper. It might be good, it might be full of holes, there's no way to tell. But even if it appears really good, until it is replicated, a single paper -- especially one which claims to overturn decades of teaching experience and convention -- should not be treated as anything more than *possibly* suggestive. It certainly should not be treated as a proven fact. Unfortunately, most scientific papers are wrong: https://phys.org/news/2016-10-scientific-wrong.html http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 The above is written from the perspective of a medical researcher, but the general conclusion holds across all the sciences, even more so for soft sciences like educational studies. Until the Kaminski study is replicated, our default assumption should be that it is "wrong", or at least that it doesn't generalise beyond the narrow examples tested in the study. The most obvious potential/hypothetical objection is, this study seems to go against the phenomenon that most people do poorly at abstract reasoning, the more abstract, the less well they do, but pose the same problem in concrete terms and they do much better.[1] This study implies the opposite should be the case, and I find that implausible. In my own experience in teaching maths, I can see some of the issues the Kaminski study discusses: e.g. the majority of my students find it difficult to generalise from one or three examples to a general principle, or to apply a principle from one area to another. (E.g. students who have successfully mastered Pythagoras' Theorem in the context of geometry, will nevertheless often fail to apply it in the context of trigonometry problems.) I would never expect the majority of my students to generalise from a handful of concrete examples to a general principle, but my experience strongly suggests that concrete examples can and do aid in understanding general principles. The fact that this study found the opposite, that concrete examples *suppressed* understanding, is so counter to my experience (including my own learning style) that I feel very skeptical about its results. Nevertheless, the study is interesting and necessary. Even if the study turns out to be correct, or mostly correct, or even partly correct, there is still a big gulf between what the study shows and Stefan's teaching style: (1) Programming in Python is NOT a formal system. Most programming isn't, and Python *in particular* is further away from the mathematical purity of (say) Haskell. There is a reason why we talk about "software engineering" and why "computer science" is so little like a science. (2) Kaminski's conclusion is: "We really need to strip these concepts down to very symbolic representations such as variables and numbers," she said. "Then students are better prepared to apply those concepts in a variety of situations." But she is talking specifically about symbolic maths here, not programming. There's no reason to think that even if she is right about maths, that the same will apply to teaching programming skills. Different skill sets require different approaches to teaching. (3) Stefan's so-called "bottom up" approach is only bottom up from a particular perspective. Another perspective is that he is unnecessarily giving *implementation details* which are mostly irrelevant to the *interface* presented by the language. E.g. Python could introduce a completely new iteration protocol, and iterating over a string or list would remain 100% backwards compatible, while Stefan's preferred while-loop example would no longer be an accurate description of the "bottom up" approach. This is not a far-fetched example. Python has already changed its iteration protocol once before, and it could do so again, without changing the language interface. (4) Stefan's own example of how to walk a string character-by-character is complicated, and even worse, it teaches poor programming style for Python: # Stefan's approach walker = iter( 'abc' ) try: while True: print( next( walker )) except StopIteration: pass That's six lines of code, involving five built-ins, four keywords, and at least six extraneous concepts unconnected to the immediate problem: * assignment/name binding; * iterator protocol; * try...except (exception handling); * while loops; * booleans; * the syntactic necessity for pass ("do nothing"). And the result is poor-quality Python code. Such code would never pass code review among even moderately experienced Python coders, let alone experts. By teaching this first, Stefan's students have to literally unlearn what he teaches them, in order to learn the right way of walking a string: for char in 'abc': print(char) That's two lines, one built-in, and two keywords, and it's practically self-documenting. More importantly, it is the much better code. Even after a lifetime of experience with Python, you won't find a better way to walk a string printing it character by character than those two lines. In fairness to Stefan, we should also include iteration and assignment as concepts being learned, but in this case they are directly connected to the immediate problem rather than at best only indirectly, and distantly, related. So the bottom line is: - Stefan's approach requires the student to learn fifteen concepts to get poor-quality Python code; - the idiomatic approach requires the student to learn five concepts to get high-quality Python code (for this toy problem). I think Stefan would probably love the approach taken by Zed Shaw's "Learn Python The Hard Way" book. I think it is horribly overrated: my experience is that beginners trying to learn from that book struggle terribly and eventually give up. I think that praise for the Hard Way is a classic example of Survivorship Bias: we only see the praise from those who succeeded in learning Python despite the book's pedagogical style, not because of it. Those who were discouraged and dropped out and gave up on Python or programming altogether remain invisible and unseen. https://en.wikipedia.org/wiki/Survivorship_bias I expect that Stefan's approach suffers from the same problem. [1] Anecdotes are not data, but for what it is worth, just in the last two days I came across two examples of this. Teaching a boy in Year 10 maths about logarithms, he struggled with purely algebraic questions involving solving exponential equations by using logs, but when given a concrete problem involving an investment he was able to solve it immediately. The second example involved a girl in Year 8 maths, who again struggled with abstract questions about adding and multiplying fractions. In particular, she overgeneralised from fraction multiplication to addition, thinking that 1/4 + 1/4 must add to 2/8. But when put into concrete geometric terms, showing physical shapes divided into quarters, she could instantly tell that 1/4 plus 1/4 must be 1/2. As I said, anecdotes are not data, but when research claims to show that apples fall upwards in contradiction to anecdotal evidence that they fall downwards, we would be wise to be cautious before accepting the research as fact. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Oct 5 13:21:30 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 04:21:30 +1100 Subject: type and object References: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> Message-ID: <59d66a1c$0$14974$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 03:17 am, Naveen Yadav wrote: > Hi folks, > > >>> isinstance(type, object) is True # 1 > and >>> isinstance(object, type) is True # 2 > > > its means type is object is type, But i am not sure how is this. No. type and object are not the same thing: py> type is object False py> id(type), id(object) (2, 3) (You may get different numbers if you call id() on your system. Remember that id() returns an opaque ID number. What matters is that they are different, not the specific values.) > For what i can understand is > for #1: since every thing is object in python, type is also an object. Correct. type is an instance of object: py> isinstance(type, object) True But type is also a subclass of object: py> issubclass(type, object) True That means if we ask type what its superclasses are, object will be included in the list of parent classes (superclasses): py> type.__mro__ (, ) > and > for #2: object is a base type. therefore object is type No. That doesn't mean that object is type. It just means all objects inherit from object, including type. type and object are special, because object is also an instance of type! py> isinstance(type, object) True py> isinstance(object, type) True So which came first? If type is an instance of object, then object must have came first; but if object is an instance of type, then type must have come first. Paradox! The answer to this paradox is that type is special, and the Python interpreter creates type before anything else. Once type is boot-strapped, then object can be created, and then type can be turned into an object instance. All the magic happens inside the interpreter. This is one of the more brain-melting parts of Python, but don't worry about it. In practice, you almost never need to care about it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From __peter__ at web.de Thu Oct 5 13:41:45 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 05 Oct 2017 19:41:45 +0200 Subject: How do native namespaces work? References: Message-ID: Thomas Nyberg wrote: > Hello, > > I'm trying to understand native namespaces. I'm currently using python > 3.5 as packaged in debian 9. I've been following the instructions here: > > https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages > > Those instructions link to the following example: > > https://github.com/pypa/sample-namespace-packages/tree/master/native > > I presume I'm doing something silly, but I can't get that to work. I've > tried creating a virtual environment as: > > -------------------------------------------------- > $ python3 -m venv venv > $ source venv/bin/activate > -------------------------------------------------- > > And then I've gone into those repo folders and run their setup.py files > (e.g. for namespace a): > > -------------------------------------------------- > $ cd sample-namespace-packages/native/pkg_a/ > $ python3 setup.py install > -------------------------------------------------- > > Then if I try to run their sample verification file, things fail: > > -------------------------------------------------- > $ cat sample-namespace-packages/verify_packages.py > from example_pkg import a > from example_pkg import b > > print(a.name) > print(a.__path__) > print(b.name) > print(b.__path__) > $ python3 sample-namespace-packages/verify_packages.py > Traceback (most recent call last): > File "sample-namespace-packages/verify_packages.py", line 1, in > from example_pkg import a > ImportError: No module named 'example_pkg' > -------------------------------------------------- > > Things seem to be installing: > > -------------------------------------------------- > $ ls venv/lib/python3.5/site-packages/ > easy-install.pth pkg_resources > easy_install.py pkg_resources-0.0.0.dist-info > example_pkg_a-1-py3.5.egg __pycache__ > example_pkg_b-1-py3.5.egg setuptools > pip setuptools-32.3.1.dist-info > pip-9.0.1.dist-info > -------------------------------------------------- > > Am I missing something totally obvious here? Does anyone here see what > I'm doing wrong? Thanks for any help! For me it's failing in the same way. I then tried pip install path/to/pkg_a which works. Comparing the site-packages directories gave added example_pkg/a/__init__.py added example_pkg/a/__pycache__/__init__.cpython-34.pyc added example_pkg_a-1.egg-info/PKG-INFO added example_pkg_a-1.egg-info/SOURCES.txt added example_pkg_a-1.egg-info/dependency_links.txt added example_pkg_a-1.egg-info/installed-files.txt added example_pkg_a-1.egg-info/top_level.txt removed easy-install.pth removed example_pkg_a-1-py3.4.egg i. e. it looks like the modules have to be extracted from the egg and cannot be imported directly even though that is what I thought the pth file was supposed to achieve. This is all a bit of black magic to me -- I've no idea if this is some kind of misconfiguration or a version problem or whatever. From roel at roelschroeven.net Thu Oct 5 13:58:54 2017 From: roel at roelschroeven.net (Roel Schroeven) Date: Thu, 05 Oct 2017 19:58:54 +0200 Subject: OT I before E [was Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b964$0$14964$b1db1813$d948b532@news.astraweb.com> Message-ID: Thomas Jollans schreef op 5/10/2017 10:30: > On 2017-10-05 06:47, Steve D'Aprano wrote: >> On Thu, 5 Oct 2017 02:54 pm, Chris Angelico wrote: >>> (There are exceptions even to the longer form of the rule, but only a >>> handful. English isn't a tidy language.) >> Even with the longer version of the rule, there are so few applicable cases, >> and enough difficulty in applying the rule correctly, that the rule is not >> worth the breath it takes to say it. > > Maybe we should just all switch to Dutch on this list. Might be easier. > Certainly more consistent. Although that way may not be obvious at first unless you're Dutch. (or Flemish) There are a lot of exceptions, weird rules and other difficulties in Dutch as well; we native speakers just don't notice them that much. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven From stephanh42 at gmail.com.invalid Thu Oct 5 14:38:18 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 05 Oct 2017 18:38:18 GMT Subject: Multithreaded compression/decompression library with python bindings? References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> <8c9801db-eece-2c87-e322-314b1f9d3e20@gmx.com> Message-ID: Op 2017-10-05, Thomas Nyberg schreef : > Btw if anyone knows a better way to handle this sort of thing, I'm all > ears. Given my current implementation I could use any compression that > works with stdin/stdout as long as I could sort out the waiting on the > subprocess. In fact, bzip2 is probably more than I need...I've half used > it out of habit rather than anything else. lzma ("xv" format) compression is generally both better and faster than bzip2. So that gives you already some advantage. Moreover, the Python lzma docs say: "When opening a file for reading, the input file may be the concatenation of multiple separate compressed streams. These are transparently decoded as a single logical stream." This seems to open the possibility to simply divide your input into, say, 100 MB blocks, compress each of them in a separate thread/process and then concatenate them. Stephan From bc at freeuk.com Thu Oct 5 14:45:13 2017 From: bc at freeuk.com (bartc) Date: Thu, 5 Oct 2017 19:45:13 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 05/10/2017 18:01, Chris Angelico wrote: > On Thu, Oct 5, 2017 at 10:26 PM, bartc wrote: >> On 05/10/2017 12:09, Chris Angelico wrote: >>> >>> On Thu, Oct 5, 2017 at 9:56 PM, bartc wrote: >>>> >>>> This doesn't make sense. For interactive use, you wouldn't bother testing >>>> for eof, as you'd be testing the eof status of the keyboard. >>> >>> >>> You mean the way heaps and heaps of Unix programs work, processing >>> until EOF of stdin? Yeah, totally makes no sense, man, no sense at >>> all. >> >> >> Out of the hundreds, perhaps thousands of such input loops I must have >> written, how many needed to test EOF? Hmm, somewhere around zero I think. >> >> Oh hang on, I wasn't using Unix; does that make a difference? >> >> If you're referring to the ability to redirect stdin so that input can come >> from a file as well as from a live keyboard, then you're doing file >> handling; it's NOT interactive. > > How would you write a sort program? How would you tell it that you're > done entering data? How do you do it with any kind of iterative program? It can be done with an in-band end-of-data code, although it might take some effort especially if you also want that to be invoked by pressing Ctrl-D. Then you can continue using a simple loop, and ignore dealing with 'EOF' on a keyboard. > And yes, I have used sort(1) interactively, with no redirection whatsoever. Yes, I tried typing 'sort' in Linux, where it apparently hangs (same on Windows actually). The reason: because it might have killed someone to have added a message saying what you are expected to type and how to end it. (Namely, press Ctrl-D start at the start of a line in Linux, and Ctrl-Z followed by Enter, I think also at the start, in Windows.) -- bartc From tjol at tjol.eu Thu Oct 5 15:11:29 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 5 Oct 2017 21:11:29 +0200 Subject: Multithreaded compression/decompression library with python bindings? In-Reply-To: References: <9b3b60d3-0ed2-6377-9e5f-b1b587479fb3@gmx.com> <59d4f962$0$14942$b1db1813$d948b532@news.astraweb.com> <8c9801db-eece-2c87-e322-314b1f9d3e20@gmx.com> Message-ID: <470aa4eb-31cb-144e-2e8c-42f4525268d5@tjol.eu> On 05/10/17 20:38, Stephan Houben wrote: > Op 2017-10-05, Thomas Nyberg schreef : >> Btw if anyone knows a better way to handle this sort of thing, I'm all >> ears. Given my current implementation I could use any compression that >> works with stdin/stdout as long as I could sort out the waiting on the >> subprocess. In fact, bzip2 is probably more than I need...I've half used >> it out of habit rather than anything else. > lzma ("xv" format) compression is generally both better and faster than > bzip2. So that gives you already some advantage. > > Moreover, the Python lzma docs say: > > "When opening a file for reading, the input file may be the concatenation > of multiple separate compressed streams. These are transparently decoded > as a single logical stream." > > This seems to open the possibility to simply divide your input into, > say, 100 MB blocks, compress each of them in a separate thread/process > and then concatenate them. Perhaps, but - this will probably be less space-efficient (by a small enough fraction, if your blocks are large enough) - this *might* be a Python implementation detail (I doubt that, but who knows) - this obviously won't work for decompression unless you know a priori that there are division points, and where they are. Thomas From ben.usenet at bsb.me.uk Thu Oct 5 15:18:47 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 05 Oct 2017 20:18:47 +0100 Subject: Pedagogical style [was Re: The "loop and a half"] References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <877ewb6qi4.fsf@bsb.me.uk> <59d66637$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: <87efqh5roo.fsf@bsb.me.uk> Steve D'Aprano writes: > There's no link to the original paper, only to secondary sources that discuss > it, e.g.: > > http://phys.org/pdf128266927.pdf > [1] Anecdotes are not data, but for what it is worth, just in the last two > days I came across two examples of this. Teaching a boy in Year 10 maths > about logarithms, he struggled with purely algebraic questions involving > solving exponential equations by using logs, but when given a concrete > problem involving an investment he was able to solve it immediately. > > The second example involved a girl in Year 8 maths, who again struggled with > abstract questions about adding and multiplying fractions. In particular, she > overgeneralised from fraction multiplication to addition, thinking that 1/4 + > 1/4 must add to 2/8. But when put into concrete geometric terms, showing > physical shapes divided into quarters, she could instantly tell that 1/4 plus > 1/4 must be 1/2. > > As I said, anecdotes are not data, but when research claims to show that > apples fall upwards in contradiction to anecdotal evidence that they fall > downwards, we would be wise to be cautious before accepting the research as > fact. I think the paper is this one: http://faculty.psy.ohio-state.edu/sloutsky/pdf/KSH-published.pdf (You can find more recent papers by searching the Ohio State University site.) >From what I've read, your anecdotes are not in contradiction to the paper's claims. -- Ben. From irmen.NOSPAM at xs4all.nl Thu Oct 5 16:11:27 2017 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Thu, 5 Oct 2017 22:11:27 +0200 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> Message-ID: <59d691ee$0$786$e4fe514c@news.xs4all.nl> On 10/05/2017 04:23 AM, Christopher Reimer wrote: > I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. Perhaps you can use the service of Travis (travis-ci.org) to avoid installing the Python versions yourself. They have lots of older versions available to run tests on. Irmen From BILL_NOSPAM at whoknows.net Thu Oct 5 16:27:22 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Thu, 5 Oct 2017 16:27:22 -0400 Subject: Introducing the "for" loop In-Reply-To: References: <1507207046l.3997912l.0l@psu.edu> Message-ID: Stefan Ram wrote: > "ROGER GRAYDON CHRISTMAN" writes: >> On Wed, Oct 4, 2017 22:42 Stefan Ram (ram at zedat.fu-berlin.de) wrote: >> Steve D'Aprano writes: >>>> So, "bottom-up" in this case means: iterators should be >>>> taught before for-loops. >>>> Why? >> The easy answer here is to not use the range in the first for loop. > I never intended to use ?range?. But I also will not use lists. > > Very early in the course, I teach numeric and string literals: > > 1, 2.3, 'abc' > > then come operators, functions and ?if?, ?while? and ?try?. > But neither ?range? nor lists have been shown so far. As long as I have two teachers here, which textbooks are you using? I am hoping to teach a college course in Python next fall. Thanks, Bill > > The basic course may already and there after about 12 - 18 hours. > (This time includes many exercises in the classroom.) > > But if I have time to introduce ?for?, I'll do it as follows > at this point in the course: > > > > We want to walk through (traverse) a string > character-by-character: > > To do this we need a walker. A walker can be > obtained using ?iter?: > > |>>> walker = iter( 'abc' ) > > Now, we can get character after character from the walker > using ?next? (transcript simplified): > > |>>> next( walker ) > |'a' > |>>> next( walker ) > |'b' > |>>> next( walker ) > |'c' > |>>> next( walker ) > |StopIteration > > We can use ?while? to automate this: > > def example(): > walker = iter( 'abc' ) > try: > while True: > print( next( walker )) > except StopIteration: > pass > > A walker also is known as an /iterator/. > > An object, one can get an iterator from > is called an /iterable object/. > > Strings are iterable objects. > > This for-loop does just what our previous > while-loop did: > > def example(): > for ch in 'abc': > print( ch ) > > It gets an iterator from ?'abc'? and then does the suite > repeatedly with ?ch? being the result of a each ?next? > call until StopIteration. > > > > No ?range?, no list. > > (Yes, ?print('\n'.join('abc'))? can do the same and will > be shown later in the course if there is still time.) > From auriocus at gmx.de Thu Oct 5 16:31:49 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 5 Oct 2017 22:31:49 +0200 Subject: Pedagogical style [was Re: The "loop and a half"] In-Reply-To: <59d66637$0$14949$b1db1813$d948b532@news.astraweb.com> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <877ewb6qi4.fsf@bsb.me.uk> <59d66637$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: Am 05.10.17 um 19:04 schrieb Steve D'Aprano: > On Thu, 5 Oct 2017 07:29 am, Christian Gollwitzer wrote: > >> To understand Stefan's way of teaching, take a look at his other >> courses, for example the C++ course: > > Thanks for this Christian. It has been an eye-opener. More comments below. You're welcome. Stefan is notorious for posting similar questions in c.l.c++ for years - asking about minute details of the C++ syntax (whether or not to call a "number literal" a "numeral" or similar) accompanied with code examples in a distinct unusual style. He's actually quite good in finding very concise C++ code for some (mostly mathematical) problems. >> http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/c++-kurs > There's no link to the original paper, only to secondary sources that discuss > it, e.g.: > > http://phys.org/pdf128266927.pdf > > so I cannot really judge the quality of the paper. It might be good, it might > be full of holes, there's no way to tell. Well you could read others who tried to replicate the study and found partially different results, and especially interpretations: http://www.staff.science.uu.nl/~savel101/fi-msecint/literature/Bock2011.pdf > (1) Programming in Python is NOT a formal system. Most programming isn't, and > Python *in particular* is further away from the mathematical purity of (say) > Haskell. I think this is the main argument against this way of teaching. As shown in the subsequent study above, the "formal way" may have an edge over the concrete examples when the task at hand involves working with abstract sets - however programming is not about knowing all details of the syntax. Instead, programming is the skill to break a given task down into steps and either attacking the subtasks in isolation or finding a library which does it. Syntax plays a minor role in that skill. Similarly, in Germany many people learn both Latin and English in high school. Latin is taught mostly by studying grammar books and learning vocabulary. English is taught mostly by stories and conversation. Most students can reasonably communicate in English, but they are usually not able to form Latin sentences on their own. However they can enumerate the inflection cases of nouns and verbs. Christian From BILL_NOSPAM at whoknows.net Thu Oct 5 16:37:39 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Thu, 5 Oct 2017 16:37:39 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <871smhu9ox.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> <871smhu9ox.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > That's good advice, but it's not all that dangerous to express off-topic > statements in this newsgroup. > It may not be "dangerous", but I find it a little annoying. I wasn't going to say anything, but now you are bringing it up explicitly. From fetchinson at googlemail.com Thu Oct 5 17:06:49 2017 From: fetchinson at googlemail.com (Fetchinson .) Date: Thu, 5 Oct 2017 23:06:49 +0200 Subject: why does memory consumption keep growing? Message-ID: Hi folks, I have a rather simple program which cycles through a bunch of files, does some operation on them, and then quits. There are 500 files involved and each operation takes about 5-10 MB of memory. As you'll see I tried to make every attempt at removing everything at the end of each cycle so that memory consumption doesn't grow as the for loop progresses, but it still does. import os for f in os.listdir( '.' ): x = [ ] for ( i, line ) in enumerate( open( f ) ): import mystuff x.append( mystuff.expensive_stuff( line ) ) del mystuff import mystuff mystuff.some_more_expensive_stuff( x ) del mystuff del x What can be the reason? I understand that mystuff might be leaky, but if I delete it, doesn't that mean that whatever memory was allocated is freed? Similary x is deleted so that can't possibly make the memory consumption go up. Any hint would be much appreciated, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From BILL_NOSPAM at whoknows.net Thu Oct 5 17:24:03 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Thu, 5 Oct 2017 17:24:03 -0400 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: Fetchinson . wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up. > > Any hint would be much appreciated, > Daniel > > Try calling the garbage collector explicitly. From rosuav at gmail.com Thu Oct 5 17:30:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 08:30:15 +1100 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: On Fri, Oct 6, 2017 at 8:06 AM, Fetchinson . via Python-list wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up. You're not actually deleting anything. When you say "del x", all you're doing is removing the *name* x. Especially, deleting an imported module basically does nothing; it's a complete waste of time. Modules are kept in their own special cache. ChrisA From pankajahire at gmail.com Thu Oct 5 17:35:06 2017 From: pankajahire at gmail.com (Pankaj L Ahire) Date: Thu, 05 Oct 2017 21:35:06 +0000 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: On Thu, Oct 5, 2017 at 17:07 Fetchinson . via Python-list < python-list at python.org> wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up. > > Any hint would be much appreciated, > Daniel You are not closing f anywhere. Better to use a context manager, so it does the clean up on exit. for f in os.listdir( '.' ): with open(f) as fh: for (i, line) in enumerate(fh): #your code. #for loop done #context manager flushes, closes fh nicely. P > > > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown > -- > https://mail.python.org/mailman/listinfo/python-list > From fetchinson at googlemail.com Thu Oct 5 17:42:32 2017 From: fetchinson at googlemail.com (Fetchinson .) Date: Thu, 5 Oct 2017 23:42:32 +0200 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: On 10/5/17, Chris Angelico wrote: > On Fri, Oct 6, 2017 at 8:06 AM, Fetchinson . via Python-list > wrote: >> Hi folks, >> >> I have a rather simple program which cycles through a bunch of files, >> does some operation on them, and then quits. There are 500 files >> involved and each operation takes about 5-10 MB of memory. As you'll >> see I tried to make every attempt at removing everything at the end of >> each cycle so that memory consumption doesn't grow as the for loop >> progresses, but it still does. >> >> import os >> >> for f in os.listdir( '.' ): >> >> x = [ ] >> >> for ( i, line ) in enumerate( open( f ) ): >> >> import mystuff >> x.append( mystuff.expensive_stuff( line ) ) >> del mystuff >> >> import mystuff >> mystuff.some_more_expensive_stuff( x ) >> del mystuff >> del x >> >> >> What can be the reason? I understand that mystuff might be leaky, but >> if I delete it, doesn't that mean that whatever memory was allocated >> is freed? Similary x is deleted so that can't possibly make the memory >> consumption go up. > > You're not actually deleting anything. When you say "del x", all > you're doing is removing the *name* x. Especially, deleting an > imported module basically does nothing; it's a complete waste of time. > Modules are kept in their own special cache. Meaning that if mystuff has some leaky stuff in it, there is no way for me to recover? Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From bc at freeuk.com Thu Oct 5 17:46:57 2017 From: bc at freeuk.com (bartc) Date: Thu, 5 Oct 2017 22:46:57 +0100 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: <8LxBB.1363204$zs1.1224847@fx15.am4> On 05/10/2017 22:06, Fetchinson . wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) What if you change this line to: mystuff.expensive_stuff( line ) If it's still growing (perhaps a bit less as it's not added to x), then something in mystuff is remembering data about each line. You might try removing the line completely too (temporarily of course). Other replies suggest that deleting the import [name] doesn't affect any data it contains. So might as well move it to the top where it belongs. > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) You'll have to comment these lines out first I think. -- bartc From prabu.ts at gmail.com Thu Oct 5 18:16:32 2017 From: prabu.ts at gmail.com (prabu.ts at gmail.com) Date: Thu, 5 Oct 2017 15:16:32 -0700 (PDT) Subject: stop/start windows services -python command Message-ID: hello all,what is the command to stop and start windows services ? i can't install win32serviceutil bec am using latest python version. From greg.ewing at canterbury.ac.nz Thu Oct 5 18:26:55 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Oct 2017 11:26:55 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5d4f8$0$14951$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b3e8$0$14936$b1db1813$d948b532@news.astraweb.com> <59d5d4f8$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > I've seen (or at least, I remember seeing) > diagrams of matter/antimatter annihilation with the two particles coming > together and a single photon coming out: a simplified and strictly wrong view > of the physics. It may or may not be wrong, depending on what the diagram was supposed to represent. Each vertex in a QED Feynman diagram involves one electron, one positron and one photon. That much is perfectly true. But if you work out the probability of that process happening on its own, it turns out to be zero. You need a diagram with at least two vertices to get a non-zero probability. Another way to say it is that a single-vertex diagram correctly represents a virtual process (involving virtual particles, which don't have to conserve energy or momentum), but it doesn't correspond to any real process (since real particles have to conserve both). -- Greg From prabu.ts at gmail.com Thu Oct 5 18:32:33 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Thu, 5 Oct 2017 15:32:33 -0700 (PDT) Subject: stop/start windows services -python command In-Reply-To: References: Message-ID: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: > hello all,what is the command to stop and start windows services ? > i can't install win32serviceutil bec am using latest python version. Please advice on this From christopher_reimer at icloud.com Thu Oct 5 18:34:14 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Thu, 05 Oct 2017 15:34:14 -0700 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: <59d691ee$0$786$e4fe514c@news.xs4all.nl> References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> <59d691ee$0$786$e4fe514c@news.xs4all.nl> Message-ID: On Oct 5, 2017, at 1:11 PM, Irmen de Jong wrote: > >> On 10/05/2017 04:23 AM, Christopher Reimer wrote: >> >> I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. > > Perhaps you can use the service of Travis (travis-ci.org) to avoid installing the Python > versions yourself. They have lots of older versions available to run tests on. > > Irmen > -- > https://mail.python.org/mailman/listinfo/python-list Travis might be a bit of an overkill for my project. Someone suggested docker containers and there is a docker plugin for tox that looks promising. Chris R. From greg.ewing at canterbury.ac.nz Thu Oct 5 18:42:20 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Oct 2017 11:42:20 +1300 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: bartc wrote: > It can be done with an in-band end-of-data code, Then you need some way of escaping that in-band code, in case it happens to equal some of the data you want to sort. And you need to ensure that your input is escaped properly. All of which adds a lot of complexity. > Yes, I tried typing 'sort' in Linux, where it apparently hangs (same on > Windows actually). The reason: because it might have killed someone to > have added a message saying what you are expected to type and how to end > it. (Namely, press Ctrl-D start at the start of a line in Linux, and > Ctrl-Z followed by Enter, I think also at the start, in Windows.) How to signal EOF from the keyboard is a fundamental piece of knowledge about the OS. It's not the responsibility of individual programs to teach it to you, any more than it's your car's responsibility to explain what the steering wheel does each time you start the engine. -- Greg From marko at pacujo.net Thu Oct 5 18:57:51 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 01:57:51 +0300 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <87y3opw6c0.fsf@elektro.pacujo.net> Gregory Ewing : > bartc wrote: >> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >> on Windows actually). The reason: because it might have killed >> someone to have added a message saying what you are expected to type >> and how to end it. (Namely, press Ctrl-D start at the start of a line >> in Linux, and Ctrl-Z followed by Enter, I think also at the start, in >> Windows.) > > How to signal EOF from the keyboard is a fundamental piece of > knowledge about the OS. It's not the responsibility of individual > programs to teach it to you, any more than it's your car's > responsibility to explain what the steering wheel does each time you > start the engine. Gladly, there is an individual program that *does* teach it to you on Linux. Just give the command: stty -a and you will see (among other things): [...]; eof = ^D; [...] It is actually the terminal driver/emulator that gives Ctrl-D its special meaning in accordance with how it has been configured. As for informational messages, it is part of deep-seated Unix culture to have quiet commands. The purpose of the silence is so you can easily compose new commands out of existing commands via pipelines and scripts. It would be inconvenient if you typed the command: grep ython message.txt | sort and the sort command instructed you to press Ctrl-D. Marko From jclopezsmu at gmail.com Thu Oct 5 19:39:20 2017 From: jclopezsmu at gmail.com (jclopezsmu at gmail.com) Date: Thu, 5 Oct 2017 16:39:20 -0700 (PDT) Subject: Case Solution: Carolinas HealthCare System Consumer Analytics by John A. Quelch, Margaret Rodriguez In-Reply-To: References: Message-ID: <0f192906-303b-419b-95c1-68c1ac079c8c@googlegroups.com> Carolinas HealthCare System: Consumer Analytics John A. Quelch; Margaret Rodriguez Source Harvard Business School Product # 515060-PDF-ENG General Instructions Please download this case from the Harvard website that you used to purchase the course pack at the beginning of the semester. The exam case was included in your coursepack; you have already paid for it. Attempt all case discussion questions listed below. Please DO NOT write a nicely flowing essay, but answer each question separately, as directly and succinctly as possible. To save space, you may use bulleted lists where appropriate. Also, simply reference the question number; you do not have to re-type the question. The write-up should consist of no more than 5 pages (double spaced) of text. In addition, it may have an unlimited number of exhibits in an appendix, e.g., tables for comparing and contrasting, 2x2 frameworks that summarize facts from the case situation, or diagrams that demonstrate your analysis of the case. Please note, copying exhibits from the case as appendices is not appropriate. Theories, frameworks and concepts covered in the course should be applied in the analysis of the empirical situation. Rely on these to justify your answers rather than on additional information from outside sources (e.g., Internet search or the firm?s economic success or failure since the case was written). Also, appropriately reference (through footnotes or endnotes) the articles you are drawing on or citing in your write-up. This final is regarded as individual work, implying that you are expected to complete it independently, without any assistance from others either in or outside of class. Please direct any questions you may have to your professor. You should not be discussing the case with your peers. Deadline for submission of this take-home final: Monday, October 9, at midnight (US Central time). Please submit your take-home final on Canvas. Please make sure the submission has been successful by checking to see whether it has been saved after you have submitted it. Points will be subtracted for late submissions. Discussion Questions (Total: 50 points) Why has Carolinas HealthCare System (CHS) invested in Dickson Advanced Analytics (DA?)? In your answer, be sure to also address the reasons for investing in DA? as a centralized, in-house provider of analytics. (10 points) Has DA? been successful thus far? Evaluate DA??s performance and identify the factors critical to its success or lack thereof. (10 points) Which of the projects DA? has piloted to date do you consider the most challenging? Why? (4 points) In an effort to develop a strategy roadmap, CHS summarized their strategic priorities and related measures in Exhibits 3 and 4. Help CHS complete its roadmap by building a Balanced Score Card (BSC) Strategy Map that makes the strategy elements captured in these exhibits more actionable by streamlining them and relating them to each other. To the extent that you have to make assumptions about their strategy as you formulate your recommended BSC strategy map, provide short arguments for them. (16 points) What are Dulin?s most important challenges going forward? Which of these challenges would you tackle first, why, and how would you go about addressing it? (10 points) On Saturday, September 2, 2017 at 5:58:15 AM UTC-5, Case Solution & Analysis wrote: > Carolinas HealthCare System: Consumer Analytics Case Solution, Case Analysis, Case Study Solution by John A. Quelch, Margaret Rodriguez is available at a lowest price, send email to casesolutionscentre(at)gmail(dot)com if you want to order the Case Solution. > > Case Study ID: 9-515-060 > > Get Case Study Solution and Analysis of Carolinas HealthCare System: Consumer Analytics in a FAIR PRICE!! > > Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . > > YOU MUST WRITE THE FOLLOWING WHILE PLACING YOUR ORDER: > Complete Case Study Name > Authors > Case Study ID > Publisher of Case Study > Your Requirements / Case Questions > > Note: Do not REPLY to this post because we do not reply to posts here. If you need any Case Solution please send us an email. We can help you to get it. From bc at freeuk.com Thu Oct 5 20:08:11 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 01:08:11 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 05/10/2017 23:42, Gregory Ewing wrote: > bartc wrote: >> It can be done with an in-band end-of-data code, > > Then you need some way of escaping that in-band code, in case > it happens to equal some of the data you want to sort. It needn't be a big deal. You can do this (you have to in Python as the normal 255 character signifying eof on a character stream is trapped): EOD=chr(26) # can be anything that is an unlikely input line. def myinput(prompt): try: return input(prompt) except EOFError: return EOD Then you can keep using a simple type of loop without worrying about EOF status: print ("Type Ctrl-Z on its own line to finish.") # Windows while 1: buffer=myinput("Enter something: ") if buffer==EOD: break print ("You typed:",buffer) # process(buffer) print ("Bye") >> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >> on Windows actually). The reason: because it might have killed someone >> to have added a message saying what you are expected to type and how >> to end it. (Namely, press Ctrl-D start at the start of a line in >> Linux, and Ctrl-Z followed by Enter, I think also at the start, in >> Windows.) > > How to signal EOF from the keyboard is a fundamental piece > of knowledge about the OS. It's not the responsibility of > individual programs to teach it to you, any more than it's > your car's responsibility to explain what the steering wheel > does each time you start the engine. If you have a utility that is designed to also be used interactively, but it shows absolutely nothing when you start it, like this: >sort then it stinks. You wouldn't think much of a shell prompt that literally showed nothing at all instead of something like: c:\python39> How would you even know what program you were in the middle of after the initial command has scrolled off the screen? How would you now it was still running? As typing a line into it give no response. I don't run these things often enough to remember exactly how to specify an EOF with the keyboard. It might be: - One of Ctrl C, D, Z or Break - It may or not may not only work at the start of a line - It may or not require to be followed by Enter I anyway would never inflict such an interface on my users, who would also associate Ctrl C etc with aborting a program that's gone wrong. -- bartc From bc at freeuk.com Thu Oct 5 20:20:11 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 01:20:11 +0100 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: On 06/10/2017 00:38, Stefan Ram wrote: > "Fetchinson ." writes: >> I have a rather simple program which cycles through a bunch of files, >> does some operation on them, and then quits. There are 500 files >> involved and each operation takes about 5-10 MB of memory. As you'll >> see I tried to make every attempt at removing everything at the end of >> each cycle so that memory consumption doesn't grow as the for loop >> progresses, but it still does. > > "2x 8GB DIMM DDR3-1600" cost $95.99 according to a web page. > This might be in the order of magnitude of the hourly rate > of a freelancing programmer. Brilliant. We can forget about freeing memory or garbage collection, just keep adding a 16GB bank of memory every few hours. It's cheaper that paying a programmer to write memory-efficient code or fixing the memory leak! (That's assuming such a program will only run on the programmer's machine and not a million consumer machines who would also need to fork out for extra memory. Their salaries may not stretch as far.) -- bartc From python at mrabarnett.plus.com Thu Oct 5 20:32:41 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 6 Oct 2017 01:32:41 +0100 Subject: stop/start windows services -python command In-Reply-To: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> Message-ID: On 2017-10-05 23:32, Prabu T.S. wrote: > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: >> hello all,what is the command to stop and start windows services ? >> i can't install win32serviceutil bec am using latest python version. > > Please advice on this > Ask Google: windows services start stop command line From wordpyai at gmail.com Thu Oct 5 20:35:45 2017 From: wordpyai at gmail.com (wordpyai at gmail.com) Date: Thu, 5 Oct 2017 17:35:45 -0700 (PDT) Subject: php to python code converter In-Reply-To: References: Message-ID: <3eaad906-a93e-427e-957a-c03e67118342@googlegroups.com> On Friday, May 8, 2009 at 3:38:25 AM UTC-7, bvidinli wrote: > if anybody needs: > http://code.google.com/p/phppython/ The link is down. I'd like to give a plug to mynewly released product `pyx.php` Python module: https://wordpy.com/pyx/php/ https://github.com/wordpy/pyx Run Converted PHP Codes in Python with the Speed of Compiled-C pyx.php is a Cython compiled module that you can use to convert or translate 99% of most common PHP source codes to pure Python. In another words, it is a PHP-to-Python syntax emulation library in Cython. The converted Python codes only require a Python 3.x interpreter, the modules in the pyx repository, and some standard Python libraries. PHP interpreter is not needed at all. If we have already translated most of the WordPress core and other scripts from PHP to Python using pyx.php, you can convert almost any PHP code into Python. With the speed of compiled Cython, running Python code translated from PHP using pyx.php might be even faster than running the original PHP code in the same computer. Installation $ git clone https://github.com/wordpy/pyx/ Currently, pyx.php is only available for Python 3.x running 64-bit Linux. Python 2.x, Mac, or other platforms can be compiled when there are many requests. Quick Start $ python # or ipython >>> import pyx.php as Php; array = Php.array >>> arr1 = array( (0,'1-0'),('a','1-a'),('b','1-b'),) >>> arr2 = array( (0,'2-0'),( 1,'2-1'),('b','2-b'),('c','2-c'),) >>> arr1 + arr2 # same as: Php.array_plus(arr1, arr2), see below >>> Php.array_merge(arr1, arr2) .... From prabu.ts at gmail.com Thu Oct 5 20:37:11 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Thu, 5 Oct 2017 17:37:11 -0700 (PDT) Subject: stop/start windows services -python command In-Reply-To: References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> Message-ID: <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: > On 2017-10-05 23:32, Prabu T.S. wrote: > > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: > >> hello all,what is the command to stop and start windows services ? > >> i can't install win32serviceutil bec am using latest python version. > > > > Please advice on this > > > Ask Google: windows services start stop command line asking for python. From steve+python at pearwood.info Thu Oct 5 20:46:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 11:46:06 +1100 Subject: why does memory consumption keep growing? References: Message-ID: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 10:38 am, Stefan Ram wrote: > "Fetchinson ." writes: >>I have a rather simple program which cycles through a bunch of files, >>does some operation on them, and then quits. There are 500 files >>involved and each operation takes about 5-10 MB of memory. As you'll >>see I tried to make every attempt at removing everything at the end of >>each cycle so that memory consumption doesn't grow as the for loop >>progresses, but it still does. > > "2x 8GB DIMM DDR3-1600" cost $95.99 according to a web page. > This might be in the order of magnitude of the hourly rate > of a freelancing programmer. Plus the downtime and labour needed to install the memory, if the computer will even take it. And then if Fetchinson is right that there is a memory leak, those 16 GB will soon be not enough, and they'll need to buy another 16GB plus more downtime and more installation costs. Memory leaks expand to fill the available memory. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Thu Oct 5 20:46:32 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 11:46:32 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 11:08 AM, bartc wrote: > I don't run these things often enough to remember exactly how to specify an > EOF with the keyboard. It might be: > > - One of Ctrl C, D, Z or Break This is your problem, not the program's. ChrisA From ethan at stoneleaf.us Thu Oct 5 20:59:27 2017 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 05 Oct 2017 17:59:27 -0700 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <59d4fd94$0$14943$b1db1813$d948b532@news.astraweb.com> <59de33b4-96b9-a3f2-e6fa-7e01e5c0b15e@tjol.eu> Message-ID: <59D6D56F.9080805@stoneleaf.us> On 10/04/2017 04:50 PM, Stefan Ram wrote: > names: are sequences of letters and stand for values > function names: names that stand for callable values > function call operator: calls a callable value (first operand) > argument: if present, it's value is passed to the function > > And, lo, I have explained everything one needs to know > to understand the expression No, you haven't. What is a value? What does callable mean? Why is a function a value? I taught myself assembly from a book when I was 16, and that was far easier to understand than your explanation. I understand that teaching is hard, but please don't make learning harder than it already is. One does not need to understand fluid dynamics nor specific gravity in order to swim. -- ~Ethan~ From python at mrabarnett.plus.com Thu Oct 5 21:00:04 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 6 Oct 2017 02:00:04 +0100 Subject: stop/start windows services -python command In-Reply-To: <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> Message-ID: <8a1d168e-8f46-7f3a-a556-ea94d9697c64@mrabarnett.plus.com> On 2017-10-06 01:37, Prabu T.S. wrote: > On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: >> On 2017-10-05 23:32, Prabu T.S. wrote: >> > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: >> >> hello all,what is the command to stop and start windows services ? >> >> i can't install win32serviceutil bec am using latest python version. >> > >> > Please advice on this >> > >> Ask Google: windows services start stop command line > > asking for python. > Again, ask Google: windows services start stop python Those results talk about "win32serviceutil", which is not part of the standard library, but part of pywin32, which you can download. From Irv at furrypants.com Thu Oct 5 21:03:22 2017 From: Irv at furrypants.com (Irv Kalb) Date: Thu, 5 Oct 2017 18:03:22 -0700 Subject: Introducing the "for" loop In-Reply-To: References: <1507207046l.3997912l.0l@psu.edu> Message-ID: <5CDA51C6-CC65-400E-898A-7B5DC47F7991@furrypants.com> > > As long as I have two teachers here, which textbooks are you using? I am hoping to teach a college course in Python next fall. > > Thanks, > Bill > > At least one more. I teach Intro to Python courses at two colleges in Silicon Valley. These courses are aimed at students who have zero background in programming. In fact, one of the colleges is a game design college and most students are art students. I was contacted by a publisher about turning my course into a book. After about a year, the book was published in Sept. 2016 by aPress, a division of Springer. The book is a superset of the material I teach in my classes. The book's title is "Learn to Program with Python" and is available on the aPress web site, on Amazon.com , and at better bookstore everywhere :) aPress: http://www.apress.com/us/book/9781484218686 They also have an eBook version available Amazon: https://www.amazon.com/Learn-Program-Python-Irv-Kalb/dp/148421868X/ref=sr_1_1?ie=UTF8&qid=1507250994&sr=8-1&keywords=kalb+python In retrospect, the only problem with the book is that all the example code is built in Python 2.7 (which is what we were using at both colleges at the time). However, in the small number of cases where there are differences, I explain the Python 3 way of doing the same thing (e.g., the print statement vs the print function). Irv PS: I teach the while loop way before I teach the for loop. That's because I use while loops to show how to allow the user to do things like play a small game over and over until the user says that they want to quit. I introduce the for loop only after teaching about lists, as a means of iterating through a list. The range function is discussed after that. From steve+python at pearwood.info Thu Oct 5 21:07:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 12:07:56 +1100 Subject: The "loop and a half" References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> Message-ID: <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 09:57 am, Marko Rauhamaa wrote: [quoting Bart] >>> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >>> on Windows actually). The reason: because it might have killed >>> someone to have added a message saying what you are expected to type >>> and how to end it. (Namely, press Ctrl-D start at the start of a line >>> in Linux, and Ctrl-Z followed by Enter, I think also at the start, in >>> Windows.) Waiting for input isn't "hangs". That's an ignorant and foolish thing to say, more suited for a wet-behind-the-ears newbie than somebody who claims to be a long-time old-school programmer. [Greg] >> How to signal EOF from the keyboard is a fundamental piece of >> knowledge about the OS. Indeed. The Unix commandline interface is not designed to be user-friendly for newbies (or experts, for that matter). It is terse, often *painfully* so (would it have killed the author to have spelled `umount` correctly?), the UI is inconsistent, and it has a very steep learning curve -- a lot of effort is required to make a little bit of progress. But even a dilettante Unix user like me knows how to signal EOF. It is one of a handful of indispensable skills you need to use Unix effectively, just as you can't use Windows effectively without knowing how to right-click. Some things you just have to learn. [Marko] > As for informational messages, it is part of deep-seated Unix culture to > have quiet commands. The purpose of the silence is so you can easily > compose new commands out of existing commands via pipelines and scripts. > It would be inconvenient if you typed the command: > > grep ython message.txt | sort > > and the sort command instructed you to press Ctrl-D. Indeed it would. But in fairness, if the author of the `sort` command had a commitment to friendliness in their programs, they could have `sort` only print a message when it is reading from stdin and writing to stdout, much as `ls` defaults to outputting control characters but automatically swaps to replacing them with ? when writing to a terminal. I believe that even Unix experts would be more effective with a judicious amount of not so much hand-holding as gentle guidance. Even experts aren't expert on every single command line tool. But the OS is what it is, and the culture has a certain level of commandline machismo, so that's unlikely to change. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From torriem at gmail.com Thu Oct 5 21:14:33 2017 From: torriem at gmail.com (Michael Torrie) Date: Thu, 5 Oct 2017 19:14:33 -0600 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/05/2017 06:08 PM, bartc wrote: > then it stinks. You wouldn't think much of a shell prompt that literally > showed nothing at all instead of something like: Indeed many programs do go to extra effort to detect if the connecting stream is an interactive device (a tty), and if so they do emit a prompt and enable a REPL loop. For example the python interpreter itself does this. However there are some fundamental commands that don't such as cat. It all depends on what the command's purpose is for, and how it's designed to be chained with other commands (cat for example). From steve+python at pearwood.info Thu Oct 5 21:22:26 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 12:22:26 +1100 Subject: why does memory consumption keep growing? References: Message-ID: <59d6dad3$0$14942$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 08:06 am, Fetchinson . wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. How do you know memory consumption is still growing? I'm not saying it isn't, but knowing what the symptoms are and how you determined the memory leak may be important in solving it. > import os > for f in os.listdir( '.' ): > x = [ ] > for ( i, line ) in enumerate( open( f ) ): > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff As Chris has already said, deleting and re-importing mystuff is a waste of time. > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, You know it is, you have good reason to think it is, or you're just guessing? > but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up. You are not deleting the list. You are deleting a single reference to the list, which may or may not allow the list to be garbage collected. I hesitate to suggest this, because I don't want to encourage superstitious cargo-cult programming, but: (1) assuming you have good evidence that memory consumption is increasing; (2) rather than `del x`, try `x[:] = []`; (3) then if memory consumption stops increasing that may be evidence of a leak, and provide a reasonable work around until the leak can be fixed. But in a well-behaved program, you shouldn't need to manually delete x at all. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Thu Oct 5 21:37:12 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 06 Oct 2017 02:37:12 +0100 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: <878tgp3vlj.fsf@bsb.me.uk> Steve D'Aprano writes: > On Fri, 6 Oct 2017 09:57 am, Marko Rauhamaa wrote: > > [quoting Bart] >>>> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >>>> on Windows actually). The reason: because it might have killed >>>> someone to have added a message saying what you are expected to type >>>> and how to end it. (Namely, press Ctrl-D start at the start of a line >>>> in Linux, and Ctrl-Z followed by Enter, I think also at the start, in >>>> Windows.) > > Waiting for input isn't "hangs". That's an ignorant and foolish thing to say, > more suited for a wet-behind-the-ears newbie than somebody who claims to be a > long-time old-school programmer. I suspect it's a wind-up. > [Marko] >> As for informational messages, it is part of deep-seated Unix culture to >> have quiet commands. The purpose of the silence is so you can easily >> compose new commands out of existing commands via pipelines and scripts. >> It would be inconvenient if you typed the command: >> >> grep ython message.txt | sort >> >> and the sort command instructed you to press Ctrl-D. > > Indeed it would. > > But in fairness, if the author of the `sort` command had a commitment to > friendliness in their programs, they could have `sort` only print a message > when it is reading from stdin and writing to stdout, I think you mean "when reading from a terminal". In the example given sort /is/ reading from stdin and writing to stdout. > much as `ls` defaults to > outputting control characters but automatically swaps to replacing them > with ? when writing to a terminal. ls often behaves completely differently when writing to a terminal. The main one is that it tabulates the file names into columns! That's very old behaviour. A more modern innovation is coloured output. > I believe that even Unix experts would be more effective with a judicious > amount of not so much hand-holding as gentle guidance. Even experts aren't > expert on every single command line tool. That's true. Some of it is here already. I am addicted to tab completion, especially when it is command-aware. And there's a flip side. I've come across a few too many programs lately clearly written by people who want to be helpful, but the wordy output is hard to parse when using the program in a script. Some programs offer a flag to simplify the output so it can be processed more easily, but not all... > But the OS is what it is, and the culture has a certain level of commandline > machismo, so that's unlikely to change. -- Ben. From rosuav at gmail.com Thu Oct 5 21:45:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 12:45:37 +1100 Subject: The "loop and a half" In-Reply-To: <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 12:07 PM, Steve D'Aprano wrote: > But in fairness, if the author of the `sort` command had a commitment to > friendliness in their programs, they could have `sort` only print a message > when it is reading from stdin and writing to stdout, much as `ls` defaults to > outputting control characters but automatically swaps to replacing them > with ? when writing to a terminal. > > I believe that even Unix experts would be more effective with a judicious > amount of not so much hand-holding as gentle guidance. Even experts aren't > expert on every single command line tool. It's not as simple as that, because you can't always know whether you're working with a console or a pipe. And it gets annoying when you have to force some program to behave interactively when it thought you were piping it into something, or vice versa. The sort program is *usually* reading from stdin and writing to stdout. Did you mean "reading from a TTY and writing to a TTY"? That's the nearest, but it's still not perfect. ChrisA From breamoreboy at gmail.com Thu Oct 5 22:31:43 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 5 Oct 2017 19:31:43 -0700 (PDT) Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: <05904e08-b82c-4a55-8c3f-ca81126bf025@googlegroups.com> On Thursday, October 5, 2017 at 10:07:05 PM UTC+1, Fetchinson . wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up. > > Any hint would be much appreciated, > Daniel > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown Nothing stands out so I'd start by closing all the file handles. As you don't need the call to `enumerate` as you don't use the `i` something like:- with open(f) as g: for line in g: ... -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Thu Oct 5 22:45:26 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 5 Oct 2017 19:45:26 -0700 (PDT) Subject: Introducing the "for" loop In-Reply-To: References: <1507207046l.3997912l.0l@psu.edu> <5CDA51C6-CC65-400E-898A-7B5DC47F7991@furrypants.com> Message-ID: On Friday, October 6, 2017 at 2:05:58 AM UTC+1, Irv Kalb wrote: > > The range function is discussed after that. > FWIW range isn't a function in Python 3. From https://docs.python.org/3/library/functions.html#func-range "Rather than being a function, range is actually an immutable sequence type, as documented in Ranges and Sequence Types ? list, tuple, range.". -- Kindest regards. Mark Lawrence. From prabu.ts at gmail.com Thu Oct 5 23:52:43 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Thu, 5 Oct 2017 20:52:43 -0700 (PDT) Subject: stop/start windows services -python command In-Reply-To: References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> <8a1d168e-8f46-7f3a-a556-ea94d9697c64@mrabarnett.plus.com> Message-ID: <5d589da4-b6a6-4c34-a745-62b241fde0ef@googlegroups.com> On Thursday, October 5, 2017 at 9:00:19 PM UTC-4, MRAB wrote: > On 2017-10-06 01:37, Prabu T.S. wrote: > > On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: > >> On 2017-10-05 23:32, Prabu T.S. wrote: > >> > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: > >> >> hello all,what is the command to stop and start windows services ? > >> >> i can't install win32serviceutil bec am using latest python version. > >> > > >> > Please advice on this > >> > > >> Ask Google: windows services start stop command line > > > > asking for python. > > > Again, ask Google: windows services start stop python > > Those results talk about "win32serviceutil", which is not part of the > standard library, but part of pywin32, which you can download. i tried pywin32, but its not compatible with python 3.6. Is there anyway i can implement start and stop services in python 3.6 version. From reembeshay555 at gmail.com Fri Oct 6 00:23:15 2017 From: reembeshay555 at gmail.com (reembeshay555 at gmail.com) Date: Thu, 5 Oct 2017 21:23:15 -0700 (PDT) Subject: Solutions Manual Test Bank for Human Anatomy 9th Edition by Martini In-Reply-To: References: Message-ID: <97ca9c91-63e8-4da6-8e20-31c0c9aeb501@googlegroups.com> On Monday, July 3, 2017 at 11:25:21 AM UTC-4, Test Banks wrote: > Greetings, > > You can get Solution Manuals and Test Bank for " Human Anatomy, 9th Edition by Frederic H. Martini, Robert B. Tallitsch, Judi L. Nath " at very reasonable price. Our team is available 24/7 and 365 days / year to respond your requests. Send us an email at pro.fast(@)hotmail(dot)com > > Place your order at PRO.FAST(@)HOTMAIL(DOT)COM > > ISBN Numbers for this book (ISBN-10: 013432076X and ISBN-13: 9780134320762) > > HUMAN ANATOMY 9TH EDITION BY MARTINI SOLUTIONS MANUAL TEST BANK > > You can also email for other Anatomy and Physiology books Solutions and Test Bank at low prices and our team will try to get all resources you need. > > Do not post your reply here. Simply send us an email at PRO.FAST (AT) HOTMAIL (DOT) COM > > Cheers, Hello, I want to order this book but I want to see the sample one before I order it. Thank you, From dan at tombstonezero.net Fri Oct 6 00:38:38 2017 From: dan at tombstonezero.net (Dan Sommers) Date: Fri, 6 Oct 2017 04:38:38 +0000 (UTC) Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: On Thu, 05 Oct 2017 19:14:33 -0600, Michael Torrie wrote: > It all depends on what the command's purpose is for, and how it's > designed to be chained with other commands (cat for example). They're almost all designed to be chained with other commands. Even diff and ed are designed to work together in a pipe; the output from "diff -e" can be piped into ed to make ed apply changes to a file. Dan From greg.ewing at canterbury.ac.nz Fri Oct 6 01:14:04 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 06 Oct 2017 18:14:04 +1300 Subject: why does memory consumption keep growing? In-Reply-To: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> References: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > Plus the downtime and labour needed to install the memory, if the computer > will even take it. Obviously we need an architecture that supports hot-swappable robot-installable RAM. -- Greg From navyad.pro at gmail.com Fri Oct 6 01:31:15 2017 From: navyad.pro at gmail.com (Naveen Yadav) Date: Thu, 5 Oct 2017 22:31:15 -0700 (PDT) Subject: type and object In-Reply-To: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> References: <230d5275-afb0-4df5-8373-86cf20fe264a@googlegroups.com> Message-ID: <620ad5c7-4145-4276-bb4f-973b184a6df8@googlegroups.com> On Thursday, 5 October 2017 21:47:34 UTC+5:30, Naveen Yadav wrote: > Hi folks, > > > >> isinstance(type, object) is True # 1 > and > >> isinstance(object, type) is True # 2 > > > its means type is object is type, But i am not sure how is this. > > For what i can understand is > for #1: since every thing is object in python, type is also an object. > and > for #2: object is a base type. therefore object is type > >> object.__doc__ > >> 'The most base type' > > > Am I understanding this correct ? Please shed light on this. thanks guys for clearing this out. From rosuav at gmail.com Fri Oct 6 01:51:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 16:51:37 +1100 Subject: why does memory consumption keep growing? In-Reply-To: References: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 4:14 PM, Gregory Ewing wrote: > Steve D'Aprano wrote: >> >> Plus the downtime and labour needed to install the memory, if the computer >> will even take it. > > > Obviously we need an architecture that supports hot-swappable > robot-installable RAM. > Cloud computing is the answer. If you don't believe me, just watch the sky for a while - new clouds get added without the sky turning off and on again. ChrisA From torriem at gmail.com Fri Oct 6 02:53:44 2017 From: torriem at gmail.com (Michael Torrie) Date: Fri, 6 Oct 2017 00:53:44 -0600 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <00146308-9e9a-c1f6-ec15-c7e0ba1c4b1f@gmail.com> On 10/05/2017 10:38 PM, Dan Sommers wrote: > On Thu, 05 Oct 2017 19:14:33 -0600, Michael Torrie wrote: > >> It all depends on what the command's purpose is for, and how it's >> designed to be chained with other commands (cat for example). > > They're almost all designed to be chained with other commands. Even > diff and ed are designed to work together in a pipe; the output from > "diff -e" can be piped into ed to make ed apply changes to a file. Right. And my point was several of them are smart about detecting whether they are actually connected to a pipe or to a real tty. And they change their mode to be more interactive when not on a pipe. For example: /usr/bin/python3 /usr/bin/bc /bin/less (reverts to acting like cat when outputting to pipe) I'm sure there are others. So Bart's complaint does have merit. If a developer wanted to make a command operate both interactively and not so in a more verbose manner, he could. From steve+python at pearwood.info Fri Oct 6 03:09:05 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 18:09:05 +1100 Subject: why does memory consumption keep growing? References: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d72c12$0$14952$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 04:51 pm, Chris Angelico wrote: > On Fri, Oct 6, 2017 at 4:14 PM, Gregory Ewing > wrote: >> Steve D'Aprano wrote: >>> >>> Plus the downtime and labour needed to install the memory, if the computer >>> will even take it. >> >> >> Obviously we need an architecture that supports hot-swappable >> robot-installable RAM. Isn't there a firmware update to the RAM that will double the available memory? > Cloud computing is the answer. > > If you don't believe me, just watch the sky for a while - new clouds > get added without the sky turning off and on again. Yeah, but they also disappear without notice, just when you need them the most. And there's no privacy, they leak private information constantly: everyone in the area knows exactly how many clouds you have and what they're doing. There's even a website that tells you what clouds have been ordered and where they're going: http://www.bom.gov.au/ -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From skysign at gmail.com Fri Oct 6 03:16:47 2017 From: skysign at gmail.com (skysign at gmail.com) Date: Fri, 6 Oct 2017 00:16:47 -0700 (PDT) Subject: problem with Python 3.6 + PyX In-Reply-To: References: <87inoirsvl.fsf@metapensiero.it> Message-ID: <588fd4dc-8a8b-40bc-8bc5-7943dc7705ad@googlegroups.com> Please install below. in my case, it is resolved. sudo apt install texinfo sudo apt install texlive-binaries 2017? 2? 11? ??? ?? 8? 49? 36? UTC+9, stalker5 ?? ?: > Yes, I have a Tex engine on my system. > How can i fix the problem with the PATH ? > Even if i execute the .py script directly by python (I mean :not inside > a LateX file) pyx need to reach the TeX system ? > Is there a config file to fix the PATH ? Where ? > > Thanks a lot !! Ciao > -- > Olivier > > > PyX requires TeX to render textual glyphs into the image. Do you have it > > installed? If so, is it "reachable" by your $PATH variable? > > > > A quick test to re-render my PyX-based images with Python 3.6 didn't reveal > > any problem... > > > > ciao, lele. > > From __peter__ at web.de Fri Oct 6 03:35:08 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 06 Oct 2017 09:35:08 +0200 Subject: why does memory consumption keep growing? References: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: Chris Angelico wrote: > On Fri, Oct 6, 2017 at 4:14 PM, Gregory Ewing > wrote: >> Steve D'Aprano wrote: >>> >>> Plus the downtime and labour needed to install the memory, if the >>> computer will even take it. >> >> >> Obviously we need an architecture that supports hot-swappable >> robot-installable RAM. >> > > Cloud computing is the answer. It's raining RAM! Hallelujah! It's raining RAM! Aye mem! > If you don't believe me, just watch the sky for a while - new clouds > get added without the sky turning off and on again. > > ChrisA From steve+python at pearwood.info Fri Oct 6 04:09:16 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 19:09:16 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> Message-ID: <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 12:37 pm, Ben Bacarisse wrote: >> But in fairness, if the author of the `sort` command had a commitment to >> friendliness in their programs, they could have `sort` only print a message >> when it is reading from stdin and writing to stdout, > > I think you mean "when reading from a terminal". In the example given > sort /is/ reading from stdin and writing to stdout. Indeed I did, it was a slip of the brain. What are the right ways for a Python script to detect these sorts of situations? (1) Standard input is coming from a pipe; (2) Stdin is being read from a file; (3) Stdin is coming from a human at a terminal; I get these. How did I do? # 1 detect input coming from a pipe. import sys import os from stat import S_ISFIFO if S_ISFIFO(os.fstat(0).st_mode): print("Reading from a pipe") # 2 detect input coming from a regular file. from stat import S_ISREG if S_ISREG(os.fstat(0).st_mode): print("Reading from a file.") # 3 detect a terminal, hopefully with a human typing at it if os.isatty(0): print("Oy noddy, wake up and type something, I'm waiting for you!") I feel a bit weird about using the magic constant 0 here. Is that guaranteed to be stdin on all platforms? Or should I be using sys.stdin.fileno()? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.f.moore at gmail.com Fri Oct 6 04:29:27 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 09:29:27 +0100 Subject: stop/start windows services -python command In-Reply-To: <5d589da4-b6a6-4c34-a745-62b241fde0ef@googlegroups.com> References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> <8a1d168e-8f46-7f3a-a556-ea94d9697c64@mrabarnett.plus.com> <5d589da4-b6a6-4c34-a745-62b241fde0ef@googlegroups.com> Message-ID: On 6 October 2017 at 04:52, Prabu T.S. wrote: > On Thursday, October 5, 2017 at 9:00:19 PM UTC-4, MRAB wrote: >> On 2017-10-06 01:37, Prabu T.S. wrote: >> > On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: >> >> On 2017-10-05 23:32, Prabu T.S. wrote: >> >> > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: >> >> >> hello all,what is the command to stop and start windows services ? >> >> >> i can't install win32serviceutil bec am using latest python version. >> >> > >> >> > Please advice on this >> >> > >> >> Ask Google: windows services start stop command line >> > >> > asking for python. >> > >> Again, ask Google: windows services start stop python >> >> Those results talk about "win32serviceutil", which is not part of the >> standard library, but part of pywin32, which you can download. > > i tried pywin32, but its not compatible with python 3.6. Is there anyway i can implement start and stop services in python 3.6 version. pywin32 *is* available for Python 3.6. Either from https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/ (a wininst installer, which is not compatible with pip but which nevertheless can be installed in your system Python) or from http://www.lfd.uci.edu/~gohlke/pythonlibs/ which hosts a lot of wheels for Windows,or as pypiwin32 from PyPI (https://pypi.python.org/pypi/pypiwin32/220). It's possible to find at least some of these via Google searches, too. Paul From hjp-usenet3 at hjp.at Fri Oct 6 04:36:39 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 10:36:39 +0200 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-06 08:09, Steve D'Aprano wrote: > What are the right ways for a Python script to detect these sorts of > situations? > > (1) Standard input is coming from a pipe; > > (2) Stdin is being read from a file; > > (3) Stdin is coming from a human at a terminal; > > I get these. How did I do? > > > # 1 detect input coming from a pipe. > import sys > import os > from stat import S_ISFIFO > if S_ISFIFO(os.fstat(0).st_mode): > print("Reading from a pipe") > > > # 2 detect input coming from a regular file. > from stat import S_ISREG > if S_ISREG(os.fstat(0).st_mode): > print("Reading from a file.") > > # 3 detect a terminal, hopefully with a human typing at it > if os.isatty(0): > print("Oy noddy, wake up and type something, I'm waiting for you!") I'd do it the same way. > I feel a bit weird about using the magic constant 0 here. Is that guaranteed > to be stdin on all platforms? It is guaranteed on POSIX compatible OSs. I think it is also guaranteed on Windows, Don't know about VMS or the IBM OSs. > Or should I be using sys.stdin.fileno()? I'm not conviced that this is much more portable: A platform where stdin doesn't use file descriptor 0 under the hood might not even use simple integers as file descriptors at all. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From rosuav at gmail.com Fri Oct 6 04:39:03 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 19:39:03 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 7:09 PM, Steve D'Aprano wrote: > What are the right ways for a Python script to detect these sorts of > situations? > > (1) Standard input is coming from a pipe; > > (2) Stdin is being read from a file; > > (3) Stdin is coming from a human at a terminal; > > I get these. How did I do? > > # 3 detect a terminal, hopefully with a human typing at it > if os.isatty(0): > print("Oy noddy, wake up and type something, I'm waiting for you!") This ought to be the only one that matters. It's the closest thing you have to "we're working in interactive mode". Generally speaking, you shouldn't care about the difference between a pipe and a file; and remember, you can have stdin be anything else, too (eg a Unix or TCP socket). > I feel a bit weird about using the magic constant 0 here. Is that guaranteed > to be stdin on all platforms? Or should I be using sys.stdin.fileno()? I believe it is. At any rate, any system that doesn't have 0/1/2 = in/out/err is likely to have far more differences than this sort of thing will cope with, so you won't be able to support it that easily. So go ahead and assume. ChrisA From zak.fenton at outlook.com Fri Oct 6 04:40:39 2017 From: zak.fenton at outlook.com (Zak Fenton) Date: Fri, 6 Oct 2017 08:40:39 +0000 Subject: Platform preferences survey (your chance to gripe about app stores!) Message-ID: Hi everyone! I'm launching a business offering new tools and services that might be of interest to users of scripting languages, so I'm interested in hearing your input (again). Thanks to those of you who participated in the earlier survey I've been busy improving the product suite to better match the needs of the community, so now I'm looking for more information about platform and distribution preferences in the lead-up to release. (Those of you who opted in for the beta trial are still on the list, I'll be in touch soon.) This survey should take less than 4 minutes to complete: https://www.surveymonkey.com/r/JBGCHZR Thanks! -Zak. From p.f.moore at gmail.com Fri Oct 6 04:42:29 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 09:42:29 +0100 Subject: why does memory consumption keep growing? In-Reply-To: References: <59d6d250$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: On 6 October 2017 at 06:51, Chris Angelico wrote: > Cloud computing is the answer. > > If you don't believe me, just watch the sky for a while - new clouds > get added without the sky turning off and on again. The sky reboots every 24 hours, and the maintenance window's about 8-12 hours. Not exactly high availability. But apparently in the arctic regions they are experimenting with a 24x7 version of the sky for short periods of the year... Paul From p.f.moore at gmail.com Fri Oct 6 04:48:36 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 09:48:36 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On 6 October 2017 at 09:36, Peter J. Holzer wrote: > On 2017-10-06 08:09, Steve D'Aprano wrote: >> What are the right ways for a Python script to detect these sorts of >> situations? >> >> (1) Standard input is coming from a pipe; >> >> (2) Stdin is being read from a file; >> >> (3) Stdin is coming from a human at a terminal; >> >> I get these. How did I do? >> >> >> # 1 detect input coming from a pipe. >> import sys >> import os >> from stat import S_ISFIFO >> if S_ISFIFO(os.fstat(0).st_mode): >> print("Reading from a pipe") >> >> >> # 2 detect input coming from a regular file. >> from stat import S_ISREG >> if S_ISREG(os.fstat(0).st_mode): >> print("Reading from a file.") >> >> # 3 detect a terminal, hopefully with a human typing at it >> if os.isatty(0): >> print("Oy noddy, wake up and type something, I'm waiting for you!") > > I'd do it the same way. > >> I feel a bit weird about using the magic constant 0 here. Is that guaranteed >> to be stdin on all platforms? > > It is guaranteed on POSIX compatible OSs. I think it is also guaranteed > on Windows, Don't know about VMS or the IBM OSs. All of these work on Windows, I just tested. Which surprised me, as I thought things like S_ISFIFO were Unix specific. Today I learned... (I'd probably still use sys.stdin.fileno() as it's more self-documenting). Paul From darcy at VybeNetworks.com Fri Oct 6 05:05:50 2017 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Fri, 6 Oct 2017 05:05:50 -0400 Subject: why does memory consumption keep growing? In-Reply-To: References: Message-ID: <22e0d5a8-6cf1-332c-1811-d61d5bef3934@VybeNetworks.com> On 10/05/2017 05:42 PM, Fetchinson . via Python-list wrote: > On 10/5/17, Chris Angelico wrote: >> On Fri, Oct 6, 2017 at 8:06 AM, Fetchinson . via Python-list >> wrote: >>> import mystuff >>> mystuff.some_more_expensive_stuff( x ) >>> del mystuff >>> del x >> You're not actually deleting anything. When you say "del x", all >> you're doing is removing the *name* x. Especially, deleting an >> imported module basically does nothing; it's a complete waste of time. >> Modules are kept in their own special cache. > > Meaning that if mystuff has some leaky stuff in it, there is no way > for me to recover? Even if deleting mystuff actually removed it the objects that it created would still exist. Import mystuff once at the top of your script. Importing over and over just costs CPU and could, in fact, be contributing to your leak. Also, I am pretty sure that the two del statements don't do anything that ending the loop doesn't already do anyway. Your best option is to find the leak. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From marko at pacujo.net Fri Oct 6 05:14:48 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 12:14:48 +0300 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: <87d160skmv.fsf@elektro.pacujo.net> Chris Angelico : > On Fri, Oct 6, 2017 at 7:09 PM, Steve D'Aprano > wrote: >> What are the right ways for a Python script to detect these sorts of >> situations? >> >> (1) Standard input is coming from a pipe; >> (2) Stdin is being read from a file; >> (3) Stdin is coming from a human at a terminal; >> >> I get these. How did I do? >> >> # 3 detect a terminal, hopefully with a human typing at it >> if os.isatty(0): >> print("Oy noddy, wake up and type something, I'm waiting for you!") > > This ought to be the only one that matters. It's the closest thing you > have to "we're working in interactive mode". Generally speaking, you > shouldn't care about the difference between a pipe and a file; and > remember, you can have stdin be anything else, too (eg a Unix or TCP > socket). Generally, you shouldn't condition the program too much on such environmental details, although it is done. For example, the "ls" command outputs the directory listing in a (colorful) multi-column format when stdout is a terminal and in a (b/w) one-file-per-line format otherwise. Since such guesswork often goes wrong, the program should provide command-line options to specify the operating mode explicitly. The "ls" command has "--color" and "--format". The "ssh" command has "-o BatchMode=yes" and so on. Again, the Unix way is to preferably stay silent by default. If you want the program to chat, there is "--verbose". Marko From flebber.crue at gmail.com Fri Oct 6 05:21:50 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Fri, 6 Oct 2017 02:21:50 -0700 (PDT) Subject: Suggestions on storing, caching, querying json In-Reply-To: <45d21aad-3db7-4d56-ae07-85f9797a3469@googlegroups.com> References: <45d21aad-3db7-4d56-ae07-85f9797a3469@googlegroups.com> Message-ID: On Thursday, 5 October 2017 15:13:43 UTC+11, Sayth Renshaw wrote: > HI > > Looking for suggestions around json libraries. with Python. I am looking for suggestions around a long term solution to store and query json documents across many files. > > I will be accessing an api and downloading approx 20 json files from an api a week. Having downloaded this year I have over 200 files already. So it will grow at a reasonable rate. > > What I have initially done is store them into a mongo db. Now I am wondering if this is useful or prudent since other than querying the json I wont have much use of other mongo features. > > When querying the json files though queries will utilise multiple json files at once, not just retrieving a single record. The usage is for data analysis. > > Is there a good json storage option, with caching and optimal querying etc. > > Regarding querying I did find a library for json searching called ObjectPath written in Python http://objectpath.org/reference.html > > Looking to leverage your experience. > > Cheers > > Sayth There is a new extension for redis ReJson and redis-py for using redis and python as a json store. http://rejson.io/ and https://github.com/andymccurdy/redis-py. Not sure if this has much more upside than mongo other than having a more fmailiar query language like JsonPath http://rejson.io/path/ Cheers Sayth From p.f.moore at gmail.com Fri Oct 6 05:27:42 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 10:27:42 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <87d160skmv.fsf@elektro.pacujo.net> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> Message-ID: On 6 October 2017 at 10:14, Marko Rauhamaa wrote: > Generally, you shouldn't condition the program too much on such > environmental details, although it is done. For example, the "ls" > command outputs the directory listing in a (colorful) multi-column > format when stdout is a terminal and in a (b/w) one-file-per-line format > otherwise. Agreed that any behaviour of the program should be explicitly controllable by command line arguments and/or configuration. But IMO, it's perfectly OK for the *default* behaviour to be affected by the environment, as long as that's done in a way that provides a good user experience. Of course, what constitutes a "good UX" is a judgement call... (Personally I think ls goes too far in how different it is in the interactive case, for example). Paul > > Since such guesswork often goes wrong, the program should provide > command-line options to specify the operating mode explicitly. The "ls" > command has "--color" and "--format". The "ssh" command has "-o > BatchMode=yes" and so on. > > Again, the Unix way is to preferably stay silent by default. If you want > the program to chat, there is "--verbose". > > > Marko > -- > https://mail.python.org/mailman/listinfo/python-list From alister.ware at ntlworld.com Fri Oct 6 06:28:24 2017 From: alister.ware at ntlworld.com (alister) Date: Fri, 06 Oct 2017 10:28:24 GMT Subject: stop/start windows services -python command References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> Message-ID: On Thu, 05 Oct 2017 17:37:11 -0700, Prabu T.S. wrote: > On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: >> On 2017-10-05 23:32, Prabu T.S. wrote: >> > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: >> >> hello all,what is the command to stop and start windows services ? i >> >> can't install win32serviceutil bec am using latest python version. >> > >> > Please advice on this >> > >> Ask Google: windows services start stop command line > > asking for python. if you know the command line call to shut down the service then you can use the subprocess module to call it -- In the long run, every program becomes rococco, and then rubble. -- Alan Perlis From bc at freeuk.com Fri Oct 6 06:32:12 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 11:32:12 +0100 Subject: The "loop and a half" In-Reply-To: <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: On 06/10/2017 02:07, Steve D'Aprano wrote: > On Fri, 6 Oct 2017 09:57 am, Marko Rauhamaa wrote: > Waiting for input isn't "hangs". That's an ignorant and foolish thing to say, > more suited for a wet-behind-the-ears newbie than somebody who claims to be a > long-time old-school programmer. That's what it looks like. Is it doing something, or waiting for me to do something, or did it just hang? > The Unix commandline interface is not designed to be user-friendly for newbies > (or experts, for that matter). It is terse, often *painfully* so (would it > have killed the author to have spelled `umount` correctly?), the UI is > inconsistent, and it has a very steep learning curve -- a lot of effort is > required to make a little bit of progress. OK. I'm not out to change Unix (not right now). But this style of utility was suggested as a way to write interactive input loops in Python programs, which you might expect to be friendlier. That it is, to assume from the outset that you're reading from a file, not a keyboard. And the user typing in data also has to pretend they are entering data in a file, complete with an EOF marker. That is what I object to. It is anyway not really acceptable these days for a long list of data to simply be typed in like that without any feedback at all. And 100% dependent on your typing Ctrl-D at the end and not Ctrl-C by mistake. This is not still the 1970s. > But in fairness, if the author of the `sort` command had a commitment to > friendliness in their programs, they could have `sort` only print a message > when it is reading from stdin and writing to stdout, much as `ls` defaults to > outputting control characters but automatically swaps to replacing them > with ? when writing to a terminal. Surely this is a common problem that has long since been solved? You have a interactive program - even with proper prompts and feedback - and one day you want to run it as a script. (And properly, by being given the same of an actual file rather than using crude redirection.) But you don't want hundreds of dialogue lines scrolling up the screen while it's doing so. -- bartc From ben.usenet at bsb.me.uk Fri Oct 6 06:33:22 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 06 Oct 2017 11:33:22 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: <87zi9436rx.fsf@bsb.me.uk> Chris Angelico writes: > On Fri, Oct 6, 2017 at 7:09 PM, Steve D'Aprano > wrote: >> What are the right ways for a Python script to detect these sorts of >> situations? >> >> (1) Standard input is coming from a pipe; >> >> (2) Stdin is being read from a file; >> >> (3) Stdin is coming from a human at a terminal; >> >> I get these. How did I do? >> >> # 3 detect a terminal, hopefully with a human typing at it >> if os.isatty(0): >> print("Oy noddy, wake up and type something, I'm waiting for you!") > > This ought to be the only one that matters. It's the closest thing you > have to "we're working in interactive mode". Generally speaking, you > shouldn't care about the difference between a pipe and a file; and > remember, you can have stdin be anything else, too (eg a Unix or TCP > socket). Yes. I'd say the key differences is whether the input is seekable or not. A program might legitimately choose different algorithms based on that property of the input, but whether it's an actual file or an actual pipe is less likely to be interesting. >> I feel a bit weird about using the magic constant 0 here. Is that guaranteed >> to be stdin on all platforms? Or should I be using >> sys.stdin.fileno()? A general solution to the (rather odd) complaint about silent waiting should really check any input fileno to see if a prompt is needed. You could argue, though, that anyone who's re-arranged a program's input so that some non-zero input fileno is attached to a terminal won't need the prompt! -- Ben. From steve+python at pearwood.info Fri Oct 6 06:41:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 21:41:08 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> Message-ID: <59d75dc6$0$14973$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 09:33 pm, Ben Bacarisse wrote: > A general solution to the (rather odd) complaint about silent waiting > should really check any input fileno to see if a prompt is needed. ?You > could argue, though, that anyone who's re-arranged a program's input so > that some non-zero input fileno is attached to a terminal won't need the > prompt! I'm afraid I don't quite know if I'm understanding you or not. I think you mean to say I should look at sys.stdin.fileno(), and if it is 0, then write a prompt, and if not, just read from stdin (possibly blocking, waiting for input). Is that right? But aren't there circumstances where fileno 0 isn't attached to a terminal, and writing a prompt would be inappropriate? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Fri Oct 6 06:57:51 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 6 Oct 2017 11:57:51 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 9:48 AM, Paul Moore wrote: > On 6 October 2017 at 09:36, Peter J. Holzer wrote: >> On 2017-10-06 08:09, Steve D'Aprano wrote: >> >>> # 1 detect input coming from a pipe. >>> import sys >>> import os >>> from stat import S_ISFIFO >>> if S_ISFIFO(os.fstat(0).st_mode): >>> print("Reading from a pipe") >>> >>> # 2 detect input coming from a regular file. >>> from stat import S_ISREG >>> if S_ISREG(os.fstat(0).st_mode): >>> print("Reading from a file.") >>> >>> # 3 detect a terminal, hopefully with a human typing at it >>> if os.isatty(0): >>> print("Oy noddy, wake up and type something, I'm waiting for you!") [...] > All of these work on Windows, I just tested. Which surprised me, as I > thought things like S_ISFIFO were Unix specific. Today I learned... The fstat implementation for Windows calls GetFileType to look for FILE_TYPE_CHAR (i.e. S_IFCHR) and FILE_TYPE_PIPE (i.e. S_IFIFO). The C runtime's isatty() on Windows checks for for a character device, not a console. If we're using this to test for interactive mode, we'll get a false positive on Windows if StandardInput is redirected to the DOS "NUL" device (i.e. NT "\Device\Null") or a COM port (rarely). As to using fd 0 instead of stdin.fileno(), for a non-console process (e.g. pythonw.exe), the C runtime doesn't initialize the standard file descriptors beyond their static definitions, and isatty(0) will return true even though it's not a valid file descriptor (i.e. the mapped Windows handle is INVALID_HANDLE_VALUE). We need to rely on the C stdin FILE stream being invalid, in which case sys.stdin is set to None. From rhodri at kynesim.co.uk Fri Oct 6 07:04:48 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Fri, 6 Oct 2017 12:04:48 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> On 05/10/17 19:45, bartc wrote: > Yes, I tried typing 'sort' in Linux, where it apparently hangs (same on > Windows actually). The reason: because it might have killed someone to > have added a message saying what you are expected to type and how to end > it. (Namely, press Ctrl-D start at the start of a line in Linux, and > Ctrl-Z followed by Enter, I think also at the start, in Windows.) Actually it might. Linux tools are written not to assume that stdin and stdout are the console, because they frequently aren't. Extra puff written to stdout at best makes it harder to use in a pipeline, and at worst makes it useless; tools that do that tend not to get used. -- Rhodri James *-* Kynesim Ltd From rosuav at gmail.com Fri Oct 6 07:35:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 22:35:45 +1100 Subject: why does memory consumption keep growing? In-Reply-To: <22e0d5a8-6cf1-332c-1811-d61d5bef3934@VybeNetworks.com> References: <22e0d5a8-6cf1-332c-1811-d61d5bef3934@VybeNetworks.com> Message-ID: On Fri, Oct 6, 2017 at 8:05 PM, D'Arcy Cain wrote: > On 10/05/2017 05:42 PM, Fetchinson . via Python-list wrote: >> >> On 10/5/17, Chris Angelico wrote: >>> >>> On Fri, Oct 6, 2017 at 8:06 AM, Fetchinson . via Python-list >>> wrote: >>>> >>>> import mystuff >>>> mystuff.some_more_expensive_stuff( x ) >>>> del mystuff >>>> del x >>> >>> You're not actually deleting anything. When you say "del x", all >>> you're doing is removing the *name* x. Especially, deleting an >>> imported module basically does nothing; it's a complete waste of time. >>> Modules are kept in their own special cache. >> >> >> Meaning that if mystuff has some leaky stuff in it, there is no way >> for me to recover? > > > Even if deleting mystuff actually removed it the objects that it created > would still exist. Import mystuff once at the top of your script. Importing > over and over just costs CPU and could, in fact, be contributing to your > leak. Importing an already-imported module is as simple as: mystuff = sys.modules["mystuff"] So it's not contributing to the leak, but it's also not helping anything. ChrisA From bc at freeuk.com Fri Oct 6 07:41:48 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 12:41:48 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 12:04, Rhodri James wrote: > On 05/10/17 19:45, bartc wrote: >> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >> on Windows actually). The reason: because it might have killed someone >> to have added a message saying what you are expected to type and how >> to end it. (Namely, press Ctrl-D start at the start of a line in >> Linux, and Ctrl-Z followed by Enter, I think also at the start, in >> Windows.) > > Actually it might.? Linux tools are written not to assume that stdin and > stdout are the console, because they frequently aren't.? Extra puff > written to stdout at best makes it harder to use in a pipeline, and at > worst makes it useless; tools that do that tend not to get used. A lot of people aren't getting it. The internal utilities used within an operating system, primarily intended for file or redirected i/o with no live interaction, should be distinct from those designed to be used directly with a live user. Or is it against the rules of Unix to have two different versions of a program? Then you might have 'sort' for the non-interactive version, and 'isort' or whatever for the interactive. Except you simply wouldn't use an interactive version of any program that reads an arbitrary long list of uninterrupted data, no matter how the ending is indicated. You'd use a text editor, enter the lines at your leisure, go back and forth to check and edit as needed, THEN you would submit that file to 'sort'. As an actual input file. So I can't see the point of using that same pattern of input usage (reading from stream, file, pipe whatever until interrupted with an EOF signal) for a true interactive program that is used in a sensible manner. (Say, an FTP program, which has a prompt, lots of commands, and usually instant feedback. An actual dialogue. A program like 'python' might fit the bill too. Here, you can give it the name of a file, OR say nothing, and it will take input from stdin. Funnily enough, you now get a message, and a prompt. And when entering multiple lines of code, you get a prompt before each line. You can also terminate the sequence by entering a blank line. So it IS possible to do it properly after all!) -- bartc From rosuav at gmail.com Fri Oct 6 07:42:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 22:42:39 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> Message-ID: On Fri, Oct 6, 2017 at 8:27 PM, Paul Moore wrote: > On 6 October 2017 at 10:14, Marko Rauhamaa wrote: >> Generally, you shouldn't condition the program too much on such >> environmental details, although it is done. For example, the "ls" >> command outputs the directory listing in a (colorful) multi-column >> format when stdout is a terminal and in a (b/w) one-file-per-line format >> otherwise. > > Agreed that any behaviour of the program should be explicitly > controllable by command line arguments and/or configuration. But IMO, > it's perfectly OK for the *default* behaviour to be affected by the > environment, as long as that's done in a way that provides a good user > experience. Of course, what constitutes a "good UX" is a judgement > call... (Personally I think ls goes too far in how different it is in > the interactive case, for example). Yeah, agreed. That said, though, there are even more ways for interactive mode to be configured. Your .bashrc quite probably has something like: alias ls='ls --color=auto' So if you type "ls" at the shell prompt, you get colour, but if you type "/bin/ls", you don't. The "=auto" part does what you're saying (using the tty-ness of stdout to control whether or not colour is used), but it's not even active unless you're using bash aliases. Generally, you should not have to worry about the behaviour of a program being drastically different if you append "| cat" to the command line. Which means you don't want TOO much difference between interactive mode and non-interactive mode, which in turn limits the extent of these changes to the defaults. It wants to be small changes only. Everything else should be controlled with options, not magic. ChrisA From rosuav at gmail.com Fri Oct 6 07:45:23 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 22:45:23 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 6, 2017 at 9:32 PM, bartc wrote: > (And properly, by being given the same of an actual file rather than using > crude redirection.) Why do you call redirection "crude"? Do you not understand the value of generic solutions that work with all programs, rather than having every program implement its own "take input from file" parameter? You can disagree with the Unix philosophy all you like, but when you insult it, you just make yourself look like an idiot. ChrisA From rosuav at gmail.com Fri Oct 6 07:51:31 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 6 Oct 2017 22:51:31 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On Fri, Oct 6, 2017 at 10:41 PM, bartc wrote: > On 06/10/2017 12:04, Rhodri James wrote: >> >> On 05/10/17 19:45, bartc wrote: >>> >>> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same on >>> Windows actually). The reason: because it might have killed someone to have >>> added a message saying what you are expected to type and how to end it. >>> (Namely, press Ctrl-D start at the start of a line in Linux, and Ctrl-Z >>> followed by Enter, I think also at the start, in Windows.) >> >> >> Actually it might. Linux tools are written not to assume that stdin and >> stdout are the console, because they frequently aren't. Extra puff written >> to stdout at best makes it harder to use in a pipeline, and at worst makes >> it useless; tools that do that tend not to get used. > > > A lot of people aren't getting it. > > The internal utilities used within an operating system, primarily intended > for file or redirected i/o with no live interaction, should be distinct from > those designed to be used directly with a live user. > > Or is it against the rules of Unix to have two different versions of a > program? > > Then you might have 'sort' for the non-interactive version, and 'isort' or > whatever for the interactive. > > Except you simply wouldn't use an interactive version of any program that > reads an arbitrary long list of uninterrupted data, no matter how the ending > is indicated. You'd use a text editor, enter the lines at your leisure, go > back and forth to check and edit as needed, THEN you would submit that file > to 'sort'. As an actual input file. > > So I can't see the point of using that same pattern of input usage (reading > from stream, file, pipe whatever until interrupted with an EOF signal) for a > true interactive program that is used in a sensible manner. What you REALLY mean is that you can't see the point of an interactive sort command. It doesn't fit into your workflow. And that's fine. It's not something you'd use very often. There are other programs, however, that behave exactly the same whether used in batch mode or interactive mode, and where you would do exactly the same thing as I described - provide input, and hit Ctrl-D to mark that you're done. And since every one of these programs is written to read from stdin until EOF, you can use them all in exactly the same way - type at keyboard, hit Ctrl-D when done. > A program like 'python' might fit the bill too. Here, you can give it the > name of a file, OR say nothing, and it will take input from stdin. > > Funnily enough, you now get a message, and a prompt. And when entering > multiple lines of code, you get a prompt before each line. You can also > terminate the sequence by entering a blank line. > > So it IS possible to do it properly after all!) Actually, Python's interactive mode is completely different from its file input mode. You're not really comparing like things here. Python's incremental interpreter is *by nature* interactive, and redirecting its input does not produce the same result as running "python3 filename.py" would. Can you find yourself a better example? ChrisA From p.f.moore at gmail.com Fri Oct 6 08:11:59 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 13:11:59 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> Message-ID: On 6 October 2017 at 12:42, Chris Angelico wrote: > Generally, you should not have to worry about the behaviour of a > program being drastically different if you append "| cat" to the > command line. Which means you don't want TOO much difference between > interactive mode and non-interactive mode, which in turn limits the > extent of these changes to the defaults. It wants to be small changes > only. Everything else should be controlled with options, not magic. Yep. My real beef with ls is multi-column vs single-column. Paul From steve+python at pearwood.info Fri Oct 6 08:22:04 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 06 Oct 2017 23:22:04 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> Message-ID: <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 11:11 pm, Paul Moore wrote: > On 6 October 2017 at 12:42, Chris Angelico wrote: >> Generally, you should not have to worry about the behaviour of a >> program being drastically different if you append "| cat" to the >> command line. Which means you don't want TOO much difference between >> interactive mode and non-interactive mode, which in turn limits the >> extent of these changes to the defaults. It wants to be small changes >> only. Everything else should be controlled with options, not magic. > > Yep. My real beef with ls is multi-column vs single-column. > Paul You don't think multiple columns in interactive mode is useful? I'm surprised, because I find it invaluable. I would hate for `ls` to default to printing everything in one long column. I suppose I could define an alias, but then every time I'm on a different computer or running as a different user, I'd end up with the annoying default single column again. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjol at tjol.eu Fri Oct 6 08:31:26 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 6 Oct 2017 14:31:26 +0200 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <87zi9436rx.fsf@bsb.me.uk> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> Message-ID: <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> On 2017-10-06 12:33, Ben Bacarisse wrote: > Chris Angelico writes: > >> On Fri, Oct 6, 2017 at 7:09 PM, Steve D'Aprano >> wrote: >>> What are the right ways for a Python script to detect these sorts of >>> situations? >>> >>> (1) Standard input is coming from a pipe; >>> >>> (2) Stdin is being read from a file; >>> >>> (3) Stdin is coming from a human at a terminal; >>> >>> I get these. How did I do? >>> >>> # 3 detect a terminal, hopefully with a human typing at it >>> if os.isatty(0): >>> print("Oy noddy, wake up and type something, I'm waiting for you!") >> >> This ought to be the only one that matters. It's the closest thing you >> have to "we're working in interactive mode". Generally speaking, you >> shouldn't care about the difference between a pipe and a file; and >> remember, you can have stdin be anything else, too (eg a Unix or TCP >> socket). > > Yes. I'd say the key differences is whether the input is seekable or > not. A program might legitimately choose different algorithms based on > that property of the input, but whether it's an actual file or an actual > pipe is less likely to be interesting. > >>> I feel a bit weird about using the magic constant 0 here. Is that guaranteed >>> to be stdin on all platforms? Or should I be using >>> sys.stdin.fileno()? > > A general solution to the (rather odd) complaint about silent waiting > should really check any input fileno to see if a prompt is needed. You > could argue, though, that anyone who's re-arranged a program's input so > that some non-zero input fileno is attached to a terminal won't need the > prompt! stdin is ALWAYS fileno 0, whether the input is attached to a tty or not. The only situation where sys.stdin.fileno() != 0 is when sys.stdin has been reassigned from within python. $ python -c 'import sys; print(sys.stdin.fileno())' < /dev/zero 0 This should be true for all platforms, or at least all platforms python supports. > > > -- Thomas Jollans From christopher_reimer at icloud.com Fri Oct 6 08:36:06 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Fri, 06 Oct 2017 05:36:06 -0700 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> <59d691ee$0$786$e4fe514c@news.xs4all.nl> Message-ID: <751B5F5E-F484-4CA9-8393-54AD097C376F@icloud.com> On Oct 5, 2017, at 3:34 PM, Christopher Reimer wrote: > >> On Oct 5, 2017, at 1:11 PM, Irmen de Jong wrote: >> >>> On 10/05/2017 04:23 AM, Christopher Reimer wrote: >>> >>> I'm leaning towards installing the latest minor version of each available major version, running tox to run the unit tests against each one, and seeing what blows up. >> >> Perhaps you can use the service of Travis (travis-ci.org) to avoid installing the Python >> versions yourself. They have lots of older versions available to run tests on. >> >> Irmen >> -- >> https://mail.python.org/mailman/listinfo/python-list > > Travis might be a bit of an overkill for my project. Someone suggested docker containers and there is a docker plugin for tox that looks promising. > > Chris R. > -- > https://mail.python.org/mailman/listinfo/python-list So I got tox and tox-docker installed. When I went to install Docker for Windows, it wouldn't work because Hyper-V wasn't available on Windows 10 Home. After paying Microsoft $99 for the privilege, I got Windows 10 Pro installed and Docker started working. I've read that all the cool programming kids are using Docker these days. I certainly hope it was worth the cost. O_o Chris R. From bc at freeuk.com Fri Oct 6 08:38:05 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 13:38:05 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 12:51, Chris Angelico wrote: > On Fri, Oct 6, 2017 at 10:41 PM, bartc wrote: > What you REALLY mean is that you can't see the point of an interactive > sort command. It doesn't fit into your workflow. And that's fine. It's > not something you'd use very often. There are other programs, however, > that behave exactly the same whether used in batch mode or interactive > mode, and where you would do exactly the same thing as I described - > provide input, and hit Ctrl-D to mark that you're done. Examples? And is there any reason why you wouldn't use a text editor to capture your input first? I can see myself noticing an error I'd made 10 lines up, which is now too late to change, and I've still got 100 lines to go. What to do? I just can't anyone wanting to use programs that work in the way you suggest. Not outside of a student exercise (read N numbers and display the average), where getting the input correct isn't so important. > And since > every one of these programs is written to read from stdin until EOF, > you can use them all in exactly the same way - type at keyboard, hit > Ctrl-D when done. So they're all at it! That doesn't mean it's a good way of doing things. >> So it IS possible to do it properly after all!) > > Actually, Python's interactive mode is completely different from its > file input mode. You're not really comparing like things here. > Python's incremental interpreter is *by nature* interactive, and > redirecting its input does not produce the same result as running > "python3 filename.py" would. Can you find yourself a better example? I gave the FTP example. (Which actually, in my version, doesn't work properly when trying redirect input into it. You have to give an actual file name.) But what's wrong with the Python example? It detects when it's given a file (as as actual parameter, or redirected), and when it's used interactively, and adapts its behaviour accordingly. It just needs a bit of effort. -- bartc From dvl at psu.edu Fri Oct 6 08:44:34 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Fri, 6 Oct 2017 08:44:34 -0400 Subject: Introducing the "for" loop In-Reply-To: mailman.1425.1507262275.2818.python-list@python.org References: Message-ID: <1507293873l.25886794l.0l@psu.edu> On Thu, Oct 5, 2017 7:45 PM, breamoreboy at gmail.com wrote: > On Friday, October 6, 2017 at 2:05:58 AM UTC+1, Irv Kalb wrote: >> >> The range function is discussed after that. >> > >FWIW range isn't a function in Python 3. From https://docs.python.org/3/library/functions.html#func-range "Rather than being a function, range is actually an immutable sequence type, as documented in Ranges and Sequence Types ? list, tuple, range.". > > > Despite the documentation, I would still be tempted to say that range is a function. Taking duck-typing to the meta-level, every time I use range, I use its name followed by a pair of parentheses enclosing one to three parameters, and I get back an immutable sequence object. It sure looks like a function to me. I would similarly say that map is a function, or an iterator generator I write myself with yield statements is a function, both of which also return sequences. It is not clear to me what the difference really is between my conception and the official definition -- is it a question about whether it returns a first-class object? Or more simply, what is the clear and obvious distinction that I can give to my non-scientific laypeople about why range isn't a function, if it looks like one? ------------ Oh, and since I think this was snipped from the thread mentioning textbook usage, I'll just toss in my note that I am using an on-line textbook developed by zyBooks.com, and those guys are nice enough to actually collaborate with the instructor about what goes into the text, and in what order. I actually like that they choose to present lists and dictionaries before any control structures, because a lot of interesting simple programming problems can be solved by making use of those structures -- like simple table-lookup applications. A C++ teacher really cannot present arrays before control structures, because there is nothing in the language to manipulate the arrays (without appealing to the libraries). But with these objects as built-in's, we can encourage nice and simple code without unnecessarily complicated control structures. Unfortunately, it seems the zyBooks authors still succumb to the same temptations as any other programmer who learned control structures before data structures. Soon after introducing those control structures, they have an exercise to do a string translation, using an ugly 12-way if-else construct nested within a loop where a dictionary would very easily trim the if-else and str.translate() would eliminate the loop. (This is fresh in my mind because I plan to illustrate both their solution and mine in my very next class) Roger Christman Pennsylvania State University From cfkaran2 at gmail.com Fri Oct 6 08:49:15 2017 From: cfkaran2 at gmail.com (CFK) Date: Fri, 6 Oct 2017 08:49:15 -0400 Subject: OT I before E [was Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5b964$0$14964$b1db1813$d948b532@news.astraweb.com> Message-ID: On Oct 5, 2017 2:02 PM, "Roel Schroeven" wrote: Thomas Jollans schreef op 5/10/2017 10:30: On 2017-10-05 06:47, Steve D'Aprano wrote: > >> On Thu, 5 Oct 2017 02:54 pm, Chris Angelico wrote: >> >>> (There are exceptions even to the longer form of the rule, but only a >>> handful. English isn't a tidy language.) >>> >> Even with the longer version of the rule, there are so few applicable >> cases, >> and enough difficulty in applying the rule correctly, that the rule is not >> worth the breath it takes to say it. >> > > Maybe we should just all switch to Dutch on this list. Might be easier. > Certainly more consistent. > Although that way may not be obvious at first unless you're Dutch. (or Flemish) There are a lot of exceptions, weird rules and other difficulties in Dutch as well; we native speakers just don't notice them that much. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeve Esperanto. https://en.m.wikipedia.org/wiki/Esperanto_grammar Thanks, Cem Karan From bc at freeuk.com Fri Oct 6 08:56:46 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 13:56:46 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: <24LBB.1729028$Ld2.1169900@fx17.am4> On 06/10/2017 12:45, Chris Angelico wrote: > On Fri, Oct 6, 2017 at 9:32 PM, bartc wrote: >> (And properly, by being given the same of an actual file rather than using >> crude redirection.) > > Why do you call redirection "crude"? Do you not understand the value > of generic solutions that work with all programs, rather than having > every program implement its own "take input from file" parameter? > > You can disagree with the Unix philosophy all you like, but when you > insult it, you just make yourself look like an idiot. Redirection is used on Windows too. I use output redirection quite frequently (to capture the output of 'dir' for example). I don't rely on > and < in my own programs. Where there's complex, mixed output, the bulk of it - the data - needs to go to a proper file, while the screen shows messages. If you don't like the word 'crude', try 'lazy'. Take this example of the gcc C compiler: > gcc -E program.c This preprocesses the code and shows the result. Typical programs will have many thousands of lines of output, but it just dumps it to the console. You /have/ to use '>' to use it practically (Windows doesn't really have a working '|' system.) Another compiler might simply write the output to a file 'program.i'. BTW if I try: > gcc Message-ID: <1507295023l.16187446l.0l@psu.edu> On Thu, Oct 5, 2017 8:18 PM, Ben Bacarisse wrote: > Steve D'Aprano writes: > >> There's no link to the original paper, only to secondary sources that discuss >> it, e.g.: >> >> http://phys.org/pdf128266927.pdf > > >> [1] Anecdotes are not data, but for what it is worth, just in the last two >> days I came across two examples of this. Teaching a boy in Year 10 maths >> about logarithms, he struggled with purely algebraic questions involving >> solving exponential equations by using logs, but when given a concrete >> problem involving an investment he was able to solve it immediately. >> >> The second example involved a girl in Year 8 maths, who again struggled with >> abstract questions about adding and multiplying fractions. In particular, she >> overgeneralised from fraction multiplication to addition, thinking that 1/4 + >> 1/4 must add to 2/8. But when put into concrete geometric terms, showing >> physical shapes divided into quarters, she could instantly tell that 1/4 plus >> 1/4 must be 1/2. >> >> As I said, anecdotes are not data, but when research claims to show that >> apples fall upwards in contradiction to anecdotal evidence that they fall >> downwards, we would be wise to be cautious before accepting the research as >> fact. > >I think the paper is this one: > >http://faculty.psy.ohio-state.edu/sloutsky/pdf/KSH-published.pdf > >(You can find more recent papers by searching the Ohio State University >site.) > >>From what I've read, your anecdotes are not in contradiction to the >paper's claims. > > Thank you for finding the paper, since I too was skeptical that the abstraction-first approach would really he profitable. I remember all those years in the public schools where everyone dreaded the 'word problems' in the math text. The abstractions rarely translated intuitively into the concrete for many of my classmates. And after reading this paper, I remain skeptical about its results. I cannot help but think that the experiment it constructs is inherently flawed. The claim is that this is an exercise in equivalence classes, obeying the same rules as modulo-3 arithmetic. There is the symbolic abstract model with the circles and diamonds, the fabricated children's game, and the 'concrete' problems with the liquid containers (shown in the paper), tennis balls, and pizza slices (not explicitly described). The paper argues that those who were given the concrete models to start with did not understand the children's game, simply because they could not transfer their knowledge to the new environment without the abstract thought. That's not a flaw in the student; that's a flaw in the model. Looking at the liquid picture, one can very easily see the concrete values 1, 2, and 3, and can easily recognize the modular arithmetic involved. It sounds to me that the same thing would be true for the tennis balls and pizza slices. And then he tells this group that the children's game is exactly analogous to what they have already experienced. So here they are, all ready to do some more modulo-3 addition, and the rug gets pulled out from under them. There is no visible quantity of 1, 2, or 3 in those circles and diamonds; and probably no 1, 2, or 3 in the children's game either. How is it any surprise that they did not figure out the children's game as quickly? Roger Christman Pennsylvania State University From hjp-usenet3 at hjp.at Fri Oct 6 09:06:03 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 15:06:03 +0200 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06 11:41, bartc wrote: > On 06/10/2017 12:04, Rhodri James wrote: >> On 05/10/17 19:45, bartc wrote: >>> Yes, I tried typing 'sort' in Linux, where it apparently hangs (same >>> on Windows actually). The reason: because it might have killed someone >>> to have added a message saying what you are expected to type and how >>> to end it. (Namely, press Ctrl-D start at the start of a line in >>> Linux, and Ctrl-Z followed by Enter, I think also at the start, in >>> Windows.) >> >> Actually it might.? Linux tools are written not to assume that stdin and >> stdout are the console, because they frequently aren't.? Extra puff >> written to stdout at best makes it harder to use in a pipeline, and at >> worst makes it useless; tools that do that tend not to get used. > > A lot of people aren't getting it. > > The internal utilities used within an operating system, primarily > intended for file or redirected i/o with no live interaction, should be > distinct from those designed to be used directly with a live user. > > Or is it against the rules of Unix to have two different versions of a > program? > > Then you might have 'sort' for the non-interactive version, and 'isort' > or whatever for the interactive. How would that stop your clueless user from invoking sort and thinking that it hangs? Short of putting it into /usr/lib or so, but that would make it hard to use in a pipe. I can see some merit in the idea that filters could print a short help message when reading from a terminal, but creating a second "interactive" version of each filter with a different name seems to be utterly pointless to me. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From neilc at norwich.edu Fri Oct 6 09:10:39 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 6 Oct 2017 13:10:39 +0000 (UTC) Subject: Pedagogical style [was Re: The "loop and a half"] References: <59d403b1$0$14959$b1db1813$d948b532@news.astraweb.com> <59d4590b$0$14941$b1db1813$d948b532@news.astraweb.com> <877ewb6qi4.fsf@bsb.me.uk> <59d66637$0$14949$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-05, Steve D'Aprano wrote: > In fairness to Stefan, we should also include iteration and > assignment as concepts being learned, but in this case they are > directly connected to the immediate problem rather than at best > only indirectly, and distantly, related. Teaching the iteration protocol before using and being comfortable with iterating creates major burdern. The student doesn't know what it supposed to *do*. On the other hand, if iteration is already second nature, then the protocal can be comprehended, and hopefully understood as a building block of something the student relies on. I had this exact experience learning C++ with Accelerated C++ by Koenig & Moo. Pointers, for example, usually taught long before the STL, here became just another form of random access iterator and the aura of mystery they formerly held in my mind evaporated. > So the bottom line is: > > - Stefan's approach requires the student to learn fifteen concepts > to get poor-quality Python code; > > - the idiomatic approach requires the student to learn five concepts > to get high-quality Python code (for this toy problem). The Koenig & Moo method was to first teach high-level, generalizable tools and much later show how to implement them yourself. I can't think of anything else as correct after having experienced it, but nevertheless other approaches have been used with success for years. And Stefan's way of thinking about and asking questions about Python has been of great interest to me, and provided entertainment and enlightenment. -- Neil Cerutti From hjp-usenet3 at hjp.at Fri Oct 6 09:11:26 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 15:11:26 +0200 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06 12:38, bartc wrote: > On 06/10/2017 12:51, Chris Angelico wrote: > > What you REALLY mean is that you can't see the point of an interactive > > sort command. It doesn't fit into your workflow. And that's fine. It's > > not something you'd use very often. There are other programs, however, > > that behave exactly the same whether used in batch mode or interactive > > mode, and where you would do exactly the same thing as I described - > > provide input, and hit Ctrl-D to mark that you're done. > > Examples? > > And is there any reason why you wouldn't use a text editor to capture > your input first? I can see myself noticing an error I'd made 10 lines > up, which is now too late to change, and I've still got 100 lines to go. > What to do? > > I just can't anyone wanting to use programs that work in the way you > suggest. Not outside of a student exercise (read N numbers and display > the average), where getting the input correct isn't so important. I regularly use at least cat, wc and od this way (plus a few of my own utilities like utf8dump). I'm sure I've used sort this way, too, though rather rarely. I usually don't type the input but paste it in, but the program can't distinguish that: All it sees is some input from the terminal at stdin. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From p.f.moore at gmail.com Fri Oct 6 09:12:15 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 14:12:15 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: On 6 October 2017 at 13:22, Steve D'Aprano wrote: >> Yep. My real beef with ls is multi-column vs single-column. >> Paul > > You don't think multiple columns in interactive mode is useful? I'm surprised, > because I find it invaluable. Interactively, I use ls -l 99.9% of the time. When I use raw ls, the column format is OK, but the fact that the number of columns varies depending on the filename length is really annoying (one really long filename can really mess the layout up). > I would hate for `ls` to default to printing everything in one long column. I > suppose I could define an alias, but then every time I'm on a different > computer or running as a different user, I'd end up with the annoying default > single column again. And that's precisely why carefully defining the defaults is both crucial and hard :-) I don't think the designers of ls necessarily got it wrong. But I'm one of the (small, presumably) group who find it sub-optimal. That's OK - you can't please all of the people all of the time and all that :-) What *really* bugs me is colour settings that default to dark blues on a black background. Someone, presumably an admin who set the system up, worked on a light-background system, and defined defaults that are good for them. And which are illegible for every single one of the actual users who have black-background ssh clients. Add the fact that I work on shared admin accounts, where setting non-default preferences is considered an antisocial act (even when they are "better" ;-)) and I spend my life squinting at screens, or typing "unalias ls" to remove the --color setting. Luckily (for everyone who has to listen to me rant), this is "just" annoyingly badly configured systems, and not baked in program defaults. Paul From bc at freeuk.com Fri Oct 6 09:15:19 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 14:15:19 +0100 Subject: newb question about @property In-Reply-To: <59d63014$0$14973$b1db1813$d948b532@news.astraweb.com> References: <59d0ad4f$0$14974$b1db1813$d948b532@news.astraweb.com> <59d36975$0$14940$b1db1813$d948b532@news.astraweb.com> <59d4728d$0$14962$b1db1813$d948b532@news.astraweb.com> <5L6BB.1362664$ji4.1213053@fx10.am4> <59d63014$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: On 05/10/2017 14:13, Steve D'Aprano wrote: > On Thu, 5 Oct 2017 10:51 pm, bartc wrote: > >> Am I allowed to say that it all seems a bit of a mess? > > > You may or may not be pleased to learn that there's a push to create a "record > like" or "struct like" datatype for Python 3.7 or 3.8, tentatively called > a "Data Class" for now. .... > https://www.python.org/dev/peps/pep-0557/ Yeah, maybe (looks a bit over-ambitious to me; why is it necessary to have typed fields? Or to do relative compares?) But it reminds me a bit of the xkcd cartoon about the 14 competing standards becoming 15... -- bartc From bc at freeuk.com Fri Oct 6 09:24:16 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 14:24:16 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 14:11, Peter J. Holzer wrote: > On 2017-10-06 12:38, bartc wrote: >> On 06/10/2017 12:51, Chris Angelico wrote: >>> What you REALLY mean is that you can't see the point of an interactive >>> sort command. It doesn't fit into your workflow. And that's fine. It's >>> not something you'd use very often. There are other programs, however, >>> that behave exactly the same whether used in batch mode or interactive >>> mode, and where you would do exactly the same thing as I described - >>> provide input, and hit Ctrl-D to mark that you're done. >> >> Examples? >> >> And is there any reason why you wouldn't use a text editor to capture >> your input first? I can see myself noticing an error I'd made 10 lines >> up, which is now too late to change, and I've still got 100 lines to go. >> What to do? >> >> I just can't anyone wanting to use programs that work in the way you >> suggest. Not outside of a student exercise (read N numbers and display >> the average), where getting the input correct isn't so important. > > I regularly use at least cat, wc and od this way (plus a few of my own > utilities like utf8dump). I'm sure I've used sort this way, too, though > rather rarely. I usually don't type the input but paste it in, Exactly. Probably no one ever uses these programs with actual live input with the possibility of uncorrectable errors getting through. So not a good reason to use that coding pattern to write programs which ARE designed for live, interactive input: print ("Enter blank expression to quit.") while 1: buffer=input("Expr: ") if buffer == "": break try: print (eval(buffer)) except: print ("Error") print ("Done") -- bartc From p.f.moore at gmail.com Fri Oct 6 09:26:23 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 14:26:23 +0100 Subject: Introducing the "for" loop In-Reply-To: <1507293873l.25886794l.0l@psu.edu> References: <1507293873l.25886794l.0l@psu.edu> Message-ID: On 6 October 2017 at 13:44, ROGER GRAYDON CHRISTMAN wrote: > Despite the documentation, I would still be tempted to say that range is a > function. > Taking duck-typing to the meta-level, every time I use range, I use its name > followed > by a pair of parentheses enclosing one to three parameters, and I get back an > immutable sequence object. It sure looks like a function to me. > > I would similarly say that map is a function, or an iterator generator I write > myself > with yield statements is a function, both of which also return sequences. > It is not clear to me what the difference really is between my conception > and the official definition -- is it a question about whether it returns a > first-class object? > > Or more simply, what is the clear and obvious distinction that I can give to my > non-scientific laypeople about why range isn't a function, if it looks like one? Technically, range (and the other examples you mentioned) is a "callable". The concept of a callable is closer to the intuitive meaning of "function" than the things Python technically calls functions. And most code accepts callables rather than functions (i.e., there's little if anything that rejects range because it's a callable not a function). Duck typing means that you can take class C: pass c = C() and replace it with class C_impl: pass def C(): return C_impl() c = C() and there's little or no difference. In the first, C is a class. In the second it's a (factory) function. You can tell the difference, via introspection. But it's unlikely you'd care in practice. As to the technical differences, you can use type() to tell: >>> def f(): pass ... >>> class F: pass ... >>> type(f) >>> type(F) >>> type(range) >>> type(map) >>> type(open) Note the last - built in functions are implemented in C, and behave slightly differently than *either* functions or classes. But you're unlikely to ever care. One example: >>> f.attr = 1 >>> open.attr = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'builtin_function_or_method' object has no attribute 'attr' functions defined in Python can have user-defined attributes, builtins can't. Embrace duck typing, and only care about what you can do, not what type of object you're doing it to ;-) Paul From p.f.moore at gmail.com Fri Oct 6 09:35:49 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Oct 2017 14:35:49 +0100 Subject: The "loop and a half" In-Reply-To: <24LBB.1729028$Ld2.1169900@fx17.am4> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> Message-ID: On 6 October 2017 at 13:56, bartc wrote: > If you don't like the word 'crude', try 'lazy'. Take this example of the gcc > C compiler: > > > gcc -E program.c > > This preprocesses the code and shows the result. Typical programs will have > many thousands of lines of output, but it just dumps it to the console. You > /have/ to use '>' to use it practically (Windows doesn't really have a > working '|' system.) No you don't. Ignoring the fact that "windows doesn't really have a working '|' system" (which is an oversimplification, by the way) the following all work: Python: data = subprocess.check_output(["gcc", "-E", "program.c"]) Powershell: $x = (gcc -E program.c) cmd: for /f %i in ('gcc -E program.c') do ... If gcc -E wrote its output to a file, you'd have to read that file, manage the process of deleting it after use (and handle possible deletion of it if an error occurred), etc. Writing to stdout is very often a good design. Not always, but nothing is ever 100% black and white, but sufficiently often that building an OS based on the idea (Unix) was pretty successful :-) Paul From as at sci.fi Fri Oct 6 09:58:41 2017 From: as at sci.fi (Anssi Saari) Date: Fri, 06 Oct 2017 16:58:41 +0300 Subject: Good virtualenv and packaging tutorials for beginner? References: <11a7c587-fd67-2a7c-e165-54c77d83ee43@gmail.com> Message-ID: Leam Hall writes: > Folks on IRC have suggested using virtualenv to test code under > different python versions. Sadly, I've not found a virtualenv tutorial > I understand. Anyone have a link to a good one? I recently used http://www.simononsoftware.com/virtualenv-tutorial-part-2/ to set up one. I was mostly trying to see if I could use it to install current Python 3 to an oldish Debian system. Worked fine. From tjol at tjol.eu Fri Oct 6 10:00:50 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 6 Oct 2017 16:00:50 +0200 Subject: The "loop and a half" In-Reply-To: <362ftclud5cqu045l5n37nvmcurrss6v4g@4ax.com> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <362ftclud5cqu045l5n37nvmcurrss6v4g@4ax.com> Message-ID: <1fd3dda9-40a3-e091-393f-49ddabe18e38@tjol.eu> On 2017-10-06 15:48, Dennis Lee Bieber wrote: > {Okay -- I've probably inspired someone to create an OS where one uses a > leading "-" to "undo" the normal action of the following command} For clarity, I think it should be +mount! and -mount! to mount and unmount, and +mount? to see what's mounted. +dir! is mkdir -dir! is rmdir dir? is ls Beautiful. +move! and -move! do the same thing, but with the arguments reversed. For clarity. From bc at freeuk.com Fri Oct 6 10:04:45 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 15:04:45 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> Message-ID: On 06/10/2017 14:35, Paul Moore wrote: > On 6 October 2017 at 13:56, bartc wrote: >> If you don't like the word 'crude', try 'lazy'. Take this example of the gcc >> C compiler: >> >> > gcc -E program.c >> >> This preprocesses the code and shows the result. Typical programs will have >> many thousands of lines of output, but it just dumps it to the console. You >> /have/ to use '>' to use it practically (Windows doesn't really have a >> working '|' system.) > > No you don't. Ignoring the fact that "windows doesn't really have a > working '|' system" (which is an oversimplification, by the way) the > following all work: > > Python: > data = subprocess.check_output(["gcc", "-E", "program.c"]) > Powershell: > $x = (gcc -E program.c) > cmd: > for /f %i in ('gcc -E program.c') do ... > > If gcc -E wrote its output to a file, you'd have to read that file, > manage the process of deleting it after use (and handle possible > deletion of it if an error occurred), etc. But if I use the -S option (show assembly listing) that DOES output to a file (program.s). If I use -fdump-tree-all, that also goes to a file (various). > > Writing to stdout is very often a good design. Not always, but nothing > is ever 100% black and white, but sufficiently often that building an > OS based on the idea (Unix) was pretty successful :-) The first computer I used was via a teletype printing on paper. If every operation producing an arbitrarily open-ended amount of output defaulted to the terminal, then it would have been totally impractical (in time taken and the amount of paper used). Even with today's consoles, 80,000 lines whizzing up the screen (when program.c contains '#include '), while not taking quite that long, still seems crazy to have as the default operating mode. But you will say, Ah but you will never see it because it will nearly always be piped into another program. And that reinforces the point I keep making: many of these are internal utilities never intended to be used as properly interactive commands. Fine, but why keep bringing them up as models of how to write true interactive programs? -- bartc From hjp-usenet3 at hjp.at Fri Oct 6 10:05:18 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 16:05:18 +0200 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06 13:24, bartc wrote: > On 06/10/2017 14:11, Peter J. Holzer wrote: >> On 2017-10-06 12:38, bartc wrote: >>> On 06/10/2017 12:51, Chris Angelico wrote: >>>> What you REALLY mean is that you can't see the point of an interactive >>>> sort command. It doesn't fit into your workflow. And that's fine. It's >>>> not something you'd use very often. There are other programs, however, >>>> that behave exactly the same whether used in batch mode or interactive >>>> mode, and where you would do exactly the same thing as I described - >>>> provide input, and hit Ctrl-D to mark that you're done. >>> >>> Examples? >>> >>> And is there any reason why you wouldn't use a text editor to capture >>> your input first? I can see myself noticing an error I'd made 10 lines >>> up, which is now too late to change, and I've still got 100 lines to go. >>> What to do? >>> >>> I just can't anyone wanting to use programs that work in the way you >>> suggest. Not outside of a student exercise (read N numbers and display >>> the average), where getting the input correct isn't so important. >> >> I regularly use at least cat, wc and od this way (plus a few of my own >> utilities like utf8dump). I'm sure I've used sort this way, too, though >> rather rarely. I usually don't type the input but paste it in, > > Exactly. Probably no one ever uses these programs with actual live input > with the possibility of uncorrectable errors getting through. > > So not a good reason to use that coding pattern to write programs which > ARE designed for live, interactive input: But you aren't arguing about programs which are designed for live, interactive input, you are arguing about sort. Sort is designed as a filter. It should be able to read from stdin (otherwise you couldn't use it in a pipe) and the fact that this also works when stdin is a terminal is just a bonus. The authors of sort would have to go out of their way to prevent that and they didn't. I think that's a good thing: It's occassionally useful that I can paste something directly into sort and don't have to write it into a file first, and the rest of the time it does no harm. > print ("Enter blank expression to quit.") > > while 1: > > buffer=input("Expr: ") > if buffer == "": break > try: > print (eval(buffer)) > except: > print ("Error") > > print ("Done") I hope you don't think this a good design for interactive input: It's much too easy to hit enter once too often. Ctrl-D is much harder to hit accidentally. In addition you could add a "user-friendly" command like "exit" or "quit". Although I hate it when Ctrl-D doesn't work: Ctrl-D - does nothing exit - syntax error quit - syntax error bye - program exits. There are way too many english words which a user might want to use to indicate their intention to close a program. None of them is therefore likely to work everywhere (and I'm not even talking about stuff like "\q" or "~."). EOF is unambiguous (and Ctrl-D for signalling EOF is an OS feature - the application knows nothing about it). hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From eryksun at gmail.com Fri Oct 6 10:34:44 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 6 Oct 2017 15:34:44 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> Message-ID: On Fri, Oct 6, 2017 at 2:35 PM, Paul Moore wrote: > > cmd: > for /f %i in ('gcc -E program.c') do ... Note that CMD processes the output as decoded Unicode text instead of encoded bytes. This is often a source of mojibake. It runs the above command with stdout redirected to a pipe, and it decodes the output line-by-line using the console's output codepage from GetConsoleOutputCP(). The console defaults to the system locale's OEM codepage. If CMD is run without a console (detached), it defaults to the system locale's ANSI codepage. Commonly, the program being run might write its output as OEM, ANSI, or UTF-8 text. For example, Python defaults to ANSI on Windows, but %PYTHONIOENCODING% could override this as UTF-8. You can change the console's input and output codepages to another codepage (but not separate values) via chcp.com. For example, `chcp 65001` sets it to UTF-8. But don't change it permanently, and certainly don't leave it in its (buggy) UTF-8 mode. Instead, a batch script should save the previous codepage and switch back when it's done. From rosuav at gmail.com Fri Oct 6 10:55:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 01:55:24 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On Fri, Oct 6, 2017 at 11:38 PM, bartc wrote: > And is there any reason why you wouldn't use a text editor to capture your > input first? I can see myself noticing an error I'd made 10 lines up, which > is now too late to change, and I've still got 100 lines to go. What to do? As has already been mentioned, this is frequently used with pasted text. But if I'm typing it manually, it's because there aren't a hundred lines to go, there might be a dozen. What to do? Hit Ctrl-D, let it spam its output, and then start over, probably by highlighting the previous text and using middle-button paste. > I just can't anyone wanting to use programs that work in the way you > suggest. Not outside of a student exercise (read N numbers and display the > average), where getting the input correct isn't so important. And since you've never personally worked this way, you scorn the concept. Have you ever worked on a slow remote session where a GUI is completely impracticable (or maybe even unavailable), and redrawing the screen is too expensive to do all the time? You absolutely have to work with line-by-line content. What text editor do you use there? >> And since >> every one of these programs is written to read from stdin until EOF, >> you can use them all in exactly the same way - type at keyboard, hit >> Ctrl-D when done. > > So they're all at it! That doesn't mean it's a good way of doing things. Actually it does, because you learn the system once and use it for everything. > On 06/10/2017 12:45, Chris Angelico wrote: >> Why do you call redirection "crude"? Do you not understand the value >> of generic solutions that work with all programs, rather than having >> every program implement its own "take input from file" parameter? >> >> You can disagree with the Unix philosophy all you like, but when you >> insult it, you just make yourself look like an idiot. > > Redirection is used on Windows too. I use output redirection quite > frequently (to capture the output of 'dir' for example). > > I don't rely on > and < in my own programs. Where there's complex, mixed > output, the bulk of it - the data - needs to go to a proper file, while the > screen shows messages. That's why we have the stderr stream as well as stdout. You can redirect stdout and still have messages go to the screen (via stderr); or you can redirect stderr as well, sending those messages to a logging facility. How do you do THAT with your program? Also: can you use your program to write to a file on a different computer? I can pipe a program's output into SSH. You can make a crude intercom system like this: ssh flurble arecord | aplay & arecord | ssh flurble aplay More practically, I've done remote backups by triggering something on a remote system and then transferring it to the system I'm running the script on. In one particular case, the CPU load of compressing the data was impacting the (single-core) source computer, so I piped it uncompressed to the recipient, deflating it on arrival: ssh flurble dump_stuff | gzip >some.dump > If you don't like the word 'crude', try 'lazy'. Take this example of the gcc > C compiler: > > > gcc -E program.c > > This preprocesses the code and shows the result. Typical programs will have > many thousands of lines of output, but it just dumps it to the console. You > /have/ to use '>' to use it practically (Windows doesn't really have a > working '|' system.) I'm not sure what you mean by a working pipe system. Yes, the one cmd.exe gives you is not nearly as flexible as what bash gives you, but for the purposes of the examples given so far, I'm pretty sure the Windows cmd.exe facilities are fine. I don't know about the actual OS facilities, but unless you're trying to redirect input/output from/to something other than a file or another process, that should be fine too. And as has already been said in this thread, it should not be a problem to explicitly redirect that. > BTW if I try: > > > gcc > it doesn't work (this on Linux). What happened to the generic solution? "It doesn't work" is a terrible way to report this. What's the problem? Is it possibly what's described fairly clearly in the error message? gcc: error: -E or -x required when input is from standard input Oh look! "In the face of ambiguity, refuse the temptation to guess." Since GCC is the GNU Compiler *Collection*, it won't assume that it knows what language you want it to compile. When you give it a file name with an extension, it will presume that that's correct (you can override with "-x", but I don't remember *ever* needing to do that), but otherwise, it doesn't guess, so you have to specify with "-x c" or similar. ChrisA From eryksun at gmail.com Fri Oct 6 11:01:04 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 6 Oct 2017 16:01:04 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On Fri, Oct 6, 2017 at 1:31 PM, Thomas Jollans wrote: > On 2017-10-06 12:33, Ben Bacarisse wrote: > >> A general solution to the (rather odd) complaint about silent waiting >> should really check any input fileno to see if a prompt is needed. You >> could argue, though, that anyone who's re-arranged a program's input so >> that some non-zero input fileno is attached to a terminal won't need the >> prompt! > > stdin is ALWAYS fileno 0, whether the input is attached to a tty or not. > The only situation where sys.stdin.fileno() != 0 is when sys.stdin has > been reassigned from within python. > > $ python -c 'import sys; print(sys.stdin.fileno())' < /dev/zero > 0 > > This should be true for all platforms, or at least all platforms python > supports. POSIX defines STDIN_FILENO as 0, and the Windows C runtime reserves FD 0 to map to the native StandardInput handle. But as I noted in a previous message, on Windows isatty(0) returns true if a process isn't executed with a valid StandardInput. In this case sys.stdin will be None, so call sys.stdin.fileno() and handle the exception if that fails. If you really need to know that stdin is interactive for something critical, then isatty() is the wrong function on Windows. You need to check for a console, which is most easily done by using ctypes to call GetConsoleMode. For example: import os if os.name == 'posix': from os import isatty elif os.name == 'nt': import ctypes import msvcrt kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) def isatty(fd): """Return True if the fd is connected to a console.""" try: handle = ctypes.c_void_p(msvcrt.get_osfhandle(fd)) except (OSError, IOError): return False mode = ctypes.c_ulong() success = kernel32.GetConsoleMode(handle, ctypes.byref(mode)) return bool(success) From tjol at tjol.eu Fri Oct 6 11:27:06 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 6 Oct 2017 17:27:06 +0200 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On 2017-10-06 17:01, eryk sun wrote: > On Fri, Oct 6, 2017 at 1:31 PM, Thomas Jollans wrote: >> On 2017-10-06 12:33, Ben Bacarisse wrote: >> >>> A general solution to the (rather odd) complaint about silent waiting >>> should really check any input fileno to see if a prompt is needed. You >>> could argue, though, that anyone who's re-arranged a program's input so >>> that some non-zero input fileno is attached to a terminal won't need the >>> prompt! >> >> stdin is ALWAYS fileno 0, whether the input is attached to a tty or not. >> The only situation where sys.stdin.fileno() != 0 is when sys.stdin has >> been reassigned from within python. >> >> $ python -c 'import sys; print(sys.stdin.fileno())' < /dev/zero >> 0 >> >> This should be true for all platforms, or at least all platforms python >> supports. > > POSIX defines STDIN_FILENO as 0, and the Windows C runtime reserves FD > 0 to map to the native StandardInput handle. But as I noted in a > previous message, on Windows isatty(0) returns true if a process isn't > executed with a valid StandardInput. In this case sys.stdin will be > None, so call sys.stdin.fileno() and handle the exception if that > fails. Seriously? sys.stdin can be None? That's terrifying. > > If you really need to know that stdin is interactive for something > critical, then isatty() is the wrong function on Windows. You need to > check for a console, which is most easily done by using ctypes to call > GetConsoleMode. For example: > > import os > > if os.name == 'posix': > from os import isatty > > elif os.name == 'nt': > import ctypes > import msvcrt > kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) > > def isatty(fd): > """Return True if the fd is connected to a console.""" > try: > handle = ctypes.c_void_p(msvcrt.get_osfhandle(fd)) > except (OSError, IOError): > return False > mode = ctypes.c_ulong() > success = kernel32.GetConsoleMode(handle, ctypes.byref(mode)) > return bool(success) > From tjreedy at udel.edu Fri Oct 6 12:11:06 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 6 Oct 2017 12:11:06 -0400 Subject: Introducing the "for" loop In-Reply-To: <1507293873l.25886794l.0l@psu.edu> References: <1507293873l.25886794l.0l@psu.edu> Message-ID: On 10/6/2017 8:44 AM, ROGER GRAYDON CHRISTMAN wrote: > Despite the documentation, I would still be tempted to say that range is a > function. It is, *according* to the documentation. Built-in classes are included in Library Reference, Ch. 2, Built-in Functions. Changing that to "Built-in Functions and Classes" has been proposed (perhaps by me) and rejected. > Taking duck-typing to the meta-level, every time I use range, I use its name > followed > by a pair of parentheses enclosing one to three parameters, and I get back an > immutable sequence object. It sure looks like a function to me. In the standard mathematical sense it is, more so than instances of classes or I would similarly say that map is a function, or an iterator generator > I write myself with yield statements is a function, Functions with 'yield' are 'generator functions' > both of which also return sequences. Iterators are mutable representatives of sequences, but cannot be indexed. > It is not clear to me what the difference really is between my conception > and the official definition -- is it a question about whether it returns a > first-class object? No. All python callables (functions in the broad sense) return objects. > Or more simply, what is the clear and obvious distinction that I can give to my > non-scientific laypeople about why range isn't a function, if it looks like one? It is a function, in the usual sense not specific to Python, that is specifically a Python class. It is not just a function. It is a function that returns an instance of itself when called. This is the distinguishing feature of classes as functions (callables). -- Terry Jan Reedy From grant.b.edwards at gmail.com Fri Oct 6 13:05:35 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 17:05:35 +0000 (UTC) Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On 2017-10-06, Thomas Jollans wrote: > Seriously? sys.stdin can be None? That's terrifying. Why? Unix daemons usually run with no stdin, stderr, or stdout. And yes, people do write Unix daemons in Python. -- Grant Edwards grant.b.edwards Yow! VICARIOUSLY experience at some reason to LIVE!! gmail.com From rhodri at kynesim.co.uk Fri Oct 6 13:07:07 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Fri, 6 Oct 2017 18:07:07 +0100 Subject: callable values In-Reply-To: References: <1507293873l.25886794l.0l@psu.edu> Message-ID: On 06/10/17 17:25, Stefan Ram wrote: > Terry Reedy writes: >> On 10/6/2017 8:44 AM, ROGER GRAYDON CHRISTMAN wrote: >>> Despite the documentation, I would still be tempted to say that range is a >>> function. >> It is, *according* to the documentation. Built-in classes are included >> in Library Reference, Ch. 2, Built-in Functions. Changing that to >> "Built-in Functions and Classes" has been proposed (perhaps by me) and >> rejected. > > FWIW, in my course notes, I have coined a special word for > this: > > A /prelate/ (German: "Pr?lat") is a callable value (object). I think I'll continue to call them callables. That way I won't burst into giggles when I accidentally think of them as church dignitaries. -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Fri Oct 6 13:13:43 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 18:13:43 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 15:55, Chris Angelico wrote: > On Fri, Oct 6, 2017 at 11:38 PM, bartc wrote: > Have you ever worked on a slow remote session where a GUI is > completely impracticable (or maybe even unavailable), and redrawing > the screen is too expensive to do all the time? You absolutely have to > work with line-by-line content. Not since about 1977, using 110 baud teletypes and VDUs, on a mainframe not geared to interactive use so it was unresponsive anyway. By 1981 however I was using a memory-mapped text display (of my home-made computer) which even with its 8-bit processor could update the screen at 200,000 characters per second. In other words, instantly. So what's the excuse for an unresponsive text display in 2017? > What text editor do you use there? That depends; are we still in the 1970s? > That's why we have the stderr stream as well as stdout. You can > redirect stdout and still have messages go to the screen (via stderr); > or you can redirect stderr as well, sending those messages to a > logging facility. How do you do THAT with your program? I said the output was complex. Then there might be more than one output file, and one or more of the outputs need to be in an actual file. Just blithely sending /everything/ to stdout, even mixed up with stderr, might be something I'd do for a quick throwaway program, not a major application. (And since I mainly use my own languages, actually getting hold of 'stdin', 'stdout' and 'stderr' is not trivial. Creating actual named files is much easier.) > Also: can you use your program to write to a file on a different > computer? I can pipe a program's output into SSH. You can make a crude > intercom system like this: No. I've got no idea about formal networking. (Actually, I walked out of the computer networks exam in my CS course, as I realised I knew nothing.) But as it happens, I could make computers talk to each when I was working with microprocessors, using home-made interfaces, rs232 or rs423. I wouldn't know how to do it now because it depends on other people's over-complex tech. > I'm not sure what you mean by a working pipe system. Yes, the one > cmd.exe gives you is not nearly as flexible as what bash gives you, > but for the purposes of the examples given so far, I just don't work that way. The OS is there to launch applications, or to provide a basic API for common services such as a file system. I don't need its complicated shells. >> BTW if I try: >> >> > gcc > >> it doesn't work (this on Linux). What happened to the generic solution? > > "It doesn't work" is a terrible way to report this. What's the > problem? Is it possibly what's described fairly clearly in the error > message? It says 'no input files'. Presumably it's one of those programs that only takes input instructions from the command line, and does not look at stdin. In which case you wouldn't be able to pipe source code into it. You might actually have to tell it the name of a discrete file you want to compile. -- bartc From rosuav at gmail.com Fri Oct 6 13:32:06 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 04:32:06 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On Sat, Oct 7, 2017 at 4:05 AM, Grant Edwards wrote: > On 2017-10-06, Thomas Jollans wrote: > >> Seriously? sys.stdin can be None? That's terrifying. > > Why? > > Unix daemons usually run with no stdin, stderr, or stdout. > > And yes, people do write Unix daemons in Python. Hmm, but usually I would expect them still to HAVE those streams, they're just connected to /dev/null or something. I don't think they would actually fail to exist, would they? ChrisA From rosuav at gmail.com Fri Oct 6 13:42:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 04:42:24 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On Sat, Oct 7, 2017 at 4:13 AM, bartc wrote: > On 06/10/2017 15:55, Chris Angelico wrote: >> >> On Fri, Oct 6, 2017 at 11:38 PM, bartc wrote: > > >> Have you ever worked on a slow remote session where a GUI is >> completely impracticable (or maybe even unavailable), and redrawing >> the screen is too expensive to do all the time? You absolutely have to >> work with line-by-line content. > > > Not since about 1977, using 110 baud teletypes and VDUs, on a mainframe not > geared to interactive use so it was unresponsive anyway. > > By 1981 however I was using a memory-mapped text display (of my home-made > computer) which even with its 8-bit processor could update the screen at > 200,000 characters per second. In other words, instantly. > > So what's the excuse for an unresponsive text display in 2017? Got it. You assume that a system is a coherent computer with its peripherals, rather than being a networked collection of computers, all of them controlled from your laptop in response to a panicked email saying the web site is down. Your terminal exists on your laptop. The programs are being run on any of a number of your servers, possibly several of them concurrently. How do you handle that? >> What text editor do you use there? > > That depends; are we still in the 1970s? Nope. We're in 2017, where the "system" is often a networked collection of computers. > (And since I mainly use my own languages, actually getting hold of 'stdin', > 'stdout' and 'stderr' is not trivial. Creating actual named files is much > easier.) More blub happening right there. You start out by assuming that the standard streams are unimportant, therefore you think that providing them is pointless. It's self-perpetuating. >> Also: can you use your program to write to a file on a different >> computer? I can pipe a program's output into SSH. You can make a crude >> intercom system like this: > > No. I've got no idea about formal networking. (Actually, I walked out of the > computer networks exam in my CS course, as I realised I knew nothing.) It's 2017. You should understand at least a bit about the internet and how to use it. > But as it happens, I could make computers talk to each when I was working > with microprocessors, using home-made interfaces, rs232 or rs423. I wouldn't > know how to do it now because it depends on other people's over-complex > tech. I don't know if you're an idiot or a troll. Using TCP/IP networking is pretty simple (at least if you're using a real language - your own toy languages might have made it unnecessarily hard, for all I know), hardly "over-complex" by comparison to RS-232 programming. >> I'm not sure what you mean by a working pipe system. Yes, the one >> cmd.exe gives you is not nearly as flexible as what bash gives you, >> but for the purposes of the examples given so far, > > > I just don't work that way. The OS is there to launch applications, or to > provide a basic API for common services such as a file system. I don't need > its complicated shells. Blub rears its head again. (Also, the shell isn't part of the OS. It's a separate tool. But still.) >>> BTW if I try: >>> >>> > gcc >> >>> it doesn't work (this on Linux). What happened to the generic solution? >> >> >> "It doesn't work" is a terrible way to report this. What's the >> problem? Is it possibly what's described fairly clearly in the error >> message? > > > It says 'no input files'. Presumably it's one of those programs that only > takes input instructions from the command line, and does not look at stdin. Presumably you don't know that a large number of programs accept "-" as a pseudo-filename that means "stdin" or "stdout" depending on context. For instance, you can diff a file with stdin, or send wget's output to stdout. At what point will you acknowledge that there are things you do not understand that are actually useful? ChrisA From grant.b.edwards at gmail.com Fri Oct 6 13:54:02 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 17:54:02 +0000 (UTC) Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On 2017-10-06, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 4:05 AM, Grant Edwards wrote: >> On 2017-10-06, Thomas Jollans wrote: >> >>> Seriously? sys.stdin can be None? That's terrifying. >> >> Why? >> >> Unix daemons usually run with no stdin, stderr, or stdout. >> >> And yes, people do write Unix daemons in Python. > > Hmm, but usually I would expect them still to HAVE those streams, > they're just connected to /dev/null or something. I don't think they > would actually fail to exist, would they? That's a good point. The basic approach is to fork and then just close all file descriptors. Since what is normally the std{in,out,err} descriptors can be re-used, you could end up with some naive code (e.g. something in a library) writing to a file/socket that it shouldn't be writing to. The defensive approach is to open /dev/null and use dup2() to make fd 0 1 2 refer to that. In that case, they do exist, but stdin always reads empty and stdout stderr write data is discarded. That's probably the more common approach. -- Grant Edwards grant.b.edwards Yow! Could I have a drug at overdose? gmail.com From marko at pacujo.net Fri Oct 6 13:55:25 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 20:55:25 +0300 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <87k208w48i.fsf@elektro.pacujo.net> bartc : > The internal utilities used within an operating system, primarily > intended for file or redirected i/o with no live interaction, should be > distinct from those designed to be used directly with a live user. > > Or is it against the rules of Unix to have two different versions of a > program? > > Then you might have 'sort' for the non-interactive version, and 'isort' > or whatever for the interactive. You are right that interactive programs exist and operate very differently from what are known as *tools*. Interactive Linux programs include: LibreOffice NetworkManager Firefox Skype XBoard Emacs GIMP top gdb etc. I don't know if anybody has seen a market/need for an interactive sort program, but there's nothing preventing you from writing one. > So I can't see the point of using that same pattern of input usage > (reading from stream, file, pipe whatever until interrupted with an > EOF signal) for a true interactive program that is used in a sensible > manner. So go ahead and write an interactive sort program. It might employ a mouse, joystick, iris scanner, touchscreen, sound effects or whatever facilities you would think would be useful for the human user. > A program like 'python' might fit the bill too. Here, you can give it > the name of a file, OR say nothing, and it will take input from stdin. Personally, I think stdin is a bit lame as a stimulus source for an interactive program. That's not even what stdin is primarily meant for; stdin is meant to be the input data for a job. Similarly, stdout is meant to be the result of the computation. Stderr, then, is used to deliver optional diagnostic messages, ie, metainfo about the computation. For interaction with a human, you should look into frameworks like Qt or curses. > Funnily enough, you now get a message, and a prompt. And when entering > multiple lines of code, you get a prompt before each line. You can > also terminate the sequence by entering a blank line. > > So it IS possible to do it properly after all!) The Unix terminal can operate in two main modes: canonical and non-canonical. The latter is typically used for interaction. See: Marko From marko at pacujo.net Fri Oct 6 14:01:51 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 21:01:51 +0300 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: <87fuaww3xs.fsf@elektro.pacujo.net> Chris Angelico : > On Sat, Oct 7, 2017 at 4:05 AM, Grant Edwards wrote: >> On 2017-10-06, Thomas Jollans wrote: >>> Seriously? sys.stdin can be None? That's terrifying. >> >> Why? >> >> Unix daemons usually run with no stdin, stderr, or stdout. >> >> And yes, people do write Unix daemons in Python. > > Hmm, but usually I would expect them still to HAVE those streams, > they're just connected to /dev/null or something. I don't think they > would actually fail to exist, would they? The reason a daemon usually opens dummy file descriptors for the 0, 1 and 2 slots is to avoid accidents. Some library might assume the existence of those file descriptors. For example, I often see GTK print out diagnositic messages. It would be awkward if those file descriptor slots were assigned to, say, a database or a socket. Marko From grant.b.edwards at gmail.com Fri Oct 6 14:02:04 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 18:02:04 +0000 (UTC) Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06, bartc wrote: > So what's the excuse for an unresponsive text display in 2017? Well, they've invented this thing called "networking". Not everybody in the world sits alone working with nothing other than the computer in front of them. Some people have to deal with remote systems connected via satellites links, low-bandwidth RF modems, and whatnot. Even cellular network connections often have high latency and low bandwidth. -- Grant Edwards grant.b.edwards Yow! BELA LUGOSI is my at co-pilot ... gmail.com From hjp-usenet3 at hjp.at Fri Oct 6 14:16:21 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 20:16:21 +0200 Subject: The "loop and a half" References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> <726ftchvm14pjl9mflceb3fpjbfvicqoc8@4ax.com> Message-ID: On 2017-10-06 15:00, Dennis Lee Bieber wrote: > On Fri, 6 Oct 2017 15:04:45 +0100, bartc declaimed the > following: >>But if I use the -S option (show assembly listing) that DOES output to a >>file (program.s). >> > > The assembler file is created anyway -- but is normally a temporary > file, deleted after the assembly pass of gcc. The -S option tells gcc to > /stop before the assembly pass/, leaving the assembly file in the > directory. On modern gcc, no: The assembly file is normally created in /tmp or (with -pipe) not at all. With -S the assembly file needs to be created with the proper name in the current directory, which is not the default behaviour. Your description may have been true for the original portable c compiler. But if I remember correctly, that one also wrote the output of the preprocessor to a temporary file which was then read by cc1. In any case, that -E writes to stdout and -S to file is an inconsistency which looks more like a historical accident than a planned feature to me. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From prabu.ts at gmail.com Fri Oct 6 14:23:03 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Fri, 6 Oct 2017 11:23:03 -0700 (PDT) Subject: stop/start windows services -python command In-Reply-To: References: <576f2168-9f50-42ff-a245-270533e7370f@googlegroups.com> <9da6710a-aa2d-4d12-9d33-89f9b57e24e8@googlegroups.com> <8a1d168e-8f46-7f3a-a556-ea94d9697c64@mrabarnett.plus.com> <5d589da4-b6a6-4c34-a745-62b241fde0ef@googlegroups.com> Message-ID: On Friday, October 6, 2017 at 4:29:48 AM UTC-4, Paul Moore wrote: > On 6 October 2017 at 04:52, Prabu T.S. wrote: > > On Thursday, October 5, 2017 at 9:00:19 PM UTC-4, MRAB wrote: > >> On 2017-10-06 01:37, Prabu T.S. wrote: > >> > On Thursday, October 5, 2017 at 8:33:02 PM UTC-4, MRAB wrote: > >> >> On 2017-10-05 23:32, Prabu T.S. wrote: > >> >> > On Thursday, October 5, 2017 at 6:16:44 PM UTC-4, Prabu T.S. wrote: > >> >> >> hello all,what is the command to stop and start windows services ? > >> >> >> i can't install win32serviceutil bec am using latest python version. > >> >> > > >> >> > Please advice on this > >> >> > > >> >> Ask Google: windows services start stop command line > >> > > >> > asking for python. > >> > > >> Again, ask Google: windows services start stop python > >> > >> Those results talk about "win32serviceutil", which is not part of the > >> standard library, but part of pywin32, which you can download. > > > > i tried pywin32, but its not compatible with python 3.6. Is there anyway i can implement start and stop services in python 3.6 version. > > pywin32 *is* available for Python 3.6. Either from > https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/ (a > wininst installer, which is not compatible with pip but which > nevertheless can be installed in your system Python) or from > http://www.lfd.uci.edu/~gohlke/pythonlibs/ which hosts a lot of wheels > for Windows,or as pypiwin32 from PyPI > (https://pypi.python.org/pypi/pypiwin32/220). > > It's possible to find at least some of these via Google searches, too. > Paul thank Pauls. I have to rename 3.6 python in registry to 3.6-32 to make it work. Thanks again. From rosuav at gmail.com Fri Oct 6 14:25:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 05:25:05 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On Sat, Oct 7, 2017 at 4:54 AM, Grant Edwards wrote: > On 2017-10-06, Chris Angelico wrote: >> On Sat, Oct 7, 2017 at 4:05 AM, Grant Edwards wrote: >>> On 2017-10-06, Thomas Jollans wrote: >>> >>>> Seriously? sys.stdin can be None? That's terrifying. >>> >>> Why? >>> >>> Unix daemons usually run with no stdin, stderr, or stdout. >>> >>> And yes, people do write Unix daemons in Python. >> >> Hmm, but usually I would expect them still to HAVE those streams, >> they're just connected to /dev/null or something. I don't think they >> would actually fail to exist, would they? > > That's a good point. The basic approach is to fork and then just > close all file descriptors. Since what is normally the > std{in,out,err} descriptors can be re-used, you could end up with some > naive code (e.g. something in a library) writing to a file/socket that > it shouldn't be writing to. > > The defensive approach is to open /dev/null and use dup2() to make fd > 0 1 2 refer to that. In that case, they do exist, but stdin always > reads empty and stdout stderr write data is discarded. > > That's probably the more common approach. Yeah. I usually see daemonized processes that still have *some* handles in 0/1/2, either /dev/null or to some sort of logging facility (the latter being extremely helpful where it's available). But I just tested it, and sure enough, Python does actually stash None into sys.stdin/out/err if the corresponding file handles were closed prior to exec. #include #define IGNORE1 \ """ "; int main() { close(0); close(1); close(2); execlp("python3", "python3", "nostreams.c", NULL); } #define IGNORE2 \ " """; #\ import sys #\ with open("nostreams.log", "w") as f: #\ print("stdin: ", sys.stdin, file=f) #\ print("stdout:", sys.stdout, file=f) #\ print("stderr:", sys.stderr, file=f) So I suppose you'd have to do something like: if sys.stdout and sys.stdout.isatty(): # be interactive else: # perform in batch mode just to be on the safe side. ChrisA From hjp-usenet3 at hjp.at Fri Oct 6 14:29:15 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 20:29:15 +0200 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06 15:08, Dennis Lee Bieber wrote: > On Fri, 6 Oct 2017 15:06:03 +0200, "Peter J. Holzer" > declaimed the following: > > >>I can see some merit in the idea that filters could print a short help >>message when reading from a terminal, but creating a second >>"interactive" version of each filter with a different name seems to be >>utterly pointless to me. >> > Knowing UNIX practices -- it would still be the same executable with a > different name hard-linked to it, and would differentiate by reading the > name used to invoke it. That, or a system wide alias that expands to the > base sort command with a default argument specifying interactive mode. I'm not worried about code duplication or disk space. I think that the creation of such "interactive" variants is pointless because those people who would - in bartc's opinion - benefit from them would never find out about them: Would a newbie who wants to sort something and hasn't read a book or other documentation type "sort" or "isort"? If they type "linux sort file" into Google, will they find lots of search results telling them to use "sort" or "isort"? If they do read a book, will it recommend "sort" or "isort"? > {I was tempted to make that "isor", pronounced eye-sore} :-) hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From grant.b.edwards at gmail.com Fri Oct 6 14:33:59 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 18:33:59 +0000 (UTC) Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> <87fuaww3xs.fsf@elektro.pacujo.net> Message-ID: On 2017-10-06, Marko Rauhamaa wrote: > The reason a daemon usually opens dummy file descriptors for the 0, 1 > and 2 slots is to avoid accidents. Some library might assume the > existence of those file descriptors. For example, I often see GTK print > out diagnositic messages. I run a lot of GTK programs from the command line, and I would say 90% or more of them spew a steady stream of meaningless (to the user) diagnostic messages. That's pretty sloppy programming if you ask me... -- Grant Edwards grant.b.edwards Yow! I've got a COUSIN at who works in the GARMENT gmail.com DISTRICT ... From hjp-usenet3 at hjp.at Fri Oct 6 14:40:01 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 6 Oct 2017 20:40:01 +0200 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On 2017-10-06 17:05, Grant Edwards wrote: > On 2017-10-06, Thomas Jollans wrote: >> Seriously? sys.stdin can be None? That's terrifying. > > Why? > > Unix daemons usually run with no stdin, stderr, or stdout. That's pretty rare. Usually they are just connected to /dev/null or a log file. Completely closing them is dangerous: The next open will use the first free file descriptor and there may be parts of your daemon (e.g. a library function) which assumes that it can write on file descriptor 2: You don't want random error messages or warnings appear in one of your output files (there have been a few security holes because of this). hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From rosuav at gmail.com Fri Oct 6 14:43:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 05:43:51 +1100 Subject: The "loop and a half" In-Reply-To: <87k208w48i.fsf@elektro.pacujo.net> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> Message-ID: On Sat, Oct 7, 2017 at 4:55 AM, Marko Rauhamaa wrote: > Personally, I think stdin is a bit lame as a stimulus source for an > interactive program. That's not even what stdin is primarily meant for; > stdin is meant to be the input data for a job. Similarly, stdout is > meant to be the result of the computation. Stderr, then, is used to > deliver optional diagnostic messages, ie, metainfo about the > computation. > > For interaction with a human, you should look into frameworks like Qt or > curses. Qt is *massive* overkill for a lot of programs. Even curses is more than you'd normally need. Stick to readline and have done with it. Challenge: Take a reasonably competent computer programmer who's never written a single line of Python code, and teach him/her how to use Python in one working week (Mon-Fri). Your student must work mostly solo, and you may meet with him/her once per day for one hour. At 5PM on Friday, your student must demonstrate a fully-working project written in Python; it must have some form of user interaction and perform some kind of useful or semi-useful task. Are you going to recommend that s/he master Qt as well as the Python language itself? And if you say "this task is impossible", that's fine. I'll have to rewrite my history and erase the eighteen or twenty (I think) students of mine that have done exactly that. A number of them used Flask to create a web application (since they already had experience with web app development using Node.js prior to this one-week challenge), one did data analysis with output in matplotlib, and the rest all used nothing but the console. Recently, I had three (out of seven) students building adventure games, and they all had distinct styles and dramatically different code models, but all of them followed a basic "read from stdin, write to stdout" model. Their UIs literally consisted of just input() and print() calls, letting them all concentrate on the guts of the code. Standard streams exist for a reason. ChrisA From marko at pacujo.net Fri Oct 6 14:47:40 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 21:47:40 +0300 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> Message-ID: <8760bsw1tf.fsf@elektro.pacujo.net> Chris Angelico : > On Sat, Oct 7, 2017 at 4:55 AM, Marko Rauhamaa wrote: >> Personally, I think stdin is a bit lame as a stimulus source for an >> interactive program. That's not even what stdin is primarily meant >> for; stdin is meant to be the input data for a job. Similarly, stdout >> is meant to be the result of the computation. Stderr, then, is used >> to deliver optional diagnostic messages, ie, metainfo about the >> computation. > > [...] > > Standard streams exist for a reason. That's exactly what I said. Marko From bc at freeuk.com Fri Oct 6 14:51:38 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 19:51:38 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 18:42, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 4:13 AM, bartc wrote: >> So what's the excuse for an unresponsive text display in 2017? > > Got it. You assume that a system is a coherent computer with its > peripherals, rather than being a networked collection of computers, > all of them controlled from your laptop in response to a panicked > email saying the web site is down. Your terminal exists on your > laptop. The programs are being run on any of a number of your servers, > possibly several of them concurrently. How do you handle that? > >>> What text editor do you use there? >> >> That depends; are we still in the 1970s? > > Nope. We're in 2017, where the "system" is often a networked > collection of computers. Still no excuse. We were talking about the need to use a crude keyboard interface to serially create a file of data a line at a time rather than using a proper application for it (such as a text editor). So which bit do you have access to locally? The keyboard presumably. And the display has to be local unless you it's either got a 50' screen or you use binoculars. Are you saying the bit in the middle is remote and that is what is playing up? This is exactly the situation from the 1970s that I thought we'd long left behind! There really is no excuse for not having the minimum amount of local processing power available to be able to enter and review a few dozen lines of text without needing to involve a temperamental network. If you're stuck, whip out a tablet computer or smartphone (they should still function without connectivity) and use a preloaded text editor. Or just compose and then save an email. Even the simplest should be more sophisticated than just blindly entering text on a Linux terminal screen. >> (And since I mainly use my own languages, actually getting hold of 'stdin', >> 'stdout' and 'stderr' is not trivial. Creating actual named files is much >> easier.) > > More blub happening right there. You start out by assuming that the > standard streams are unimportant, therefore you think that providing > them is pointless. It's self-perpetuating. They were a creation of either C, Unix, or some combination. Certainly I had very little to with any of that for decades. I can't remember that it ever stopped me doing anything I needed to do. My own programs worked like this (showing original '#' symbol I used); print "One" # to console (also using print #0) print #f, "Two" # to file handle f print #s, "Three" # append to string s print #w, "Four" # to window or control w print #b, "Five" # to image b What would I want with stderr?! > It's 2017. You should understand at least a bit about the internet and > how to use it. I can use it but can't fix it when it goes wrong. > I don't know if you're an idiot or a troll. Using TCP/IP networking is > pretty simple (at least if you're using a real language - your own toy > languages might have made it unnecessarily hard, for all I know), > hardly "over-complex" by comparison to RS-232 programming. So how do you use it from Python - without using an existing Python library? Or is it only simple when someone else has made it idiot-proof? (That reminds me, at some point I have to provide a winsock2.h file for my C compiler. I may have to finally learn some of this stuff to test it.) >> I just don't work that way. The OS is there to launch applications, or to >> provide a basic API for common services such as a file system. I don't need >> its complicated shells. > > Blub rears its head again. How so? I said I have little need for most of what an OS does, in my development work, which is perfectly true. Perhaps you haven't noticed that many are using sophisticated IDEs now rather than fiddling about with command lines. (Mine is not sophisticated, but still better than a command line.) > At what point will you acknowledge that there are things you do not > understand that are actually useful? I can understand that people who use Unix and Linux arrange their world so that all these things are apparently indispensable. What you're trying to tell me is that because Unix provides such a tightly knit system system of utilities, stdin, stdout, files, pipes and very specific ways of using the keyword (which seems to consist of avoiding actually using it for entering data as much a possible; just using the logical idea of it), then every other text entry system in the world, in whichever OS or application, must follow exactly the same model. Because the Unix one is so great. Needless to say, I disagree. -- bartc From rosuav at gmail.com Fri Oct 6 14:56:19 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 05:56:19 +1100 Subject: The "loop and a half" In-Reply-To: <8760bsw1tf.fsf@elektro.pacujo.net> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <8760bsw1tf.fsf@elektro.pacujo.net> Message-ID: On Sat, Oct 7, 2017 at 5:47 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Sat, Oct 7, 2017 at 4:55 AM, Marko Rauhamaa wrote: >>> Personally, I think stdin is a bit lame as a stimulus source for an >>> interactive program. That's not even what stdin is primarily meant >>> for; stdin is meant to be the input data for a job. Similarly, stdout >>> is meant to be the result of the computation. Stderr, then, is used >>> to deliver optional diagnostic messages, ie, metainfo about the >>> computation. >> >> [...] >> >> Standard streams exist for a reason. > > That's exactly what I said. And you also described it as "lame" for an interactive program. ChrisA From bc at freeuk.com Fri Oct 6 15:18:33 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 20:18:33 +0100 Subject: The "loop and a half" In-Reply-To: <87k208w48i.fsf@elektro.pacujo.net> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> Message-ID: <_FQBB.1006789$cS2.932107@fx19.am4> On 06/10/2017 18:55, Marko Rauhamaa wrote: > bartc : > >> The internal utilities used within an operating system, primarily >> intended for file or redirected i/o with no live interaction, should be >> distinct from those designed to be used directly with a live user. >> >> Or is it against the rules of Unix to have two different versions of a >> program? >> >> Then you might have 'sort' for the non-interactive version, and 'isort' >> or whatever for the interactive. > > You are right that interactive programs exist and operate very > differently from what are known as *tools*. > > Interactive Linux programs include: > > LibreOffice > NetworkManager > Firefox > Skype > XBoard > Emacs > GIMP > top > gdb > > etc. I meant interactive programs that work with a scrolling display, as suits a program like this: while 1: print (input("?")) Considerations of the 'eof' status of a keyboard would of course be irrelevant for anything more sophisticated, either with a more organised console app, or one using a GUI. > I don't know if anybody has seen a market/need for an interactive sort > program, but there's nothing preventing you from writing one. For sort, there is no real need. You use a text editor to create your data. Then use existing file-based sort. >> So I can't see the point of using that same pattern of input usage >> (reading from stream, file, pipe whatever until interrupted with an >> EOF signal) for a true interactive program that is used in a sensible >> manner. > > So go ahead and write an interactive sort program. It might employ a > mouse, joystick, iris scanner, touchscreen, sound effects or whatever > facilities you would think would be useful for the human user. It wasn't me who brought up 'sort' when the subject was interactive keyboard dialogues (it was Chris I think). >> A program like 'python' might fit the bill too. Here, you can give it >> the name of a file, OR say nothing, and it will take input from stdin. > > Personally, I think stdin is a bit lame as a stimulus source for an > interactive program. That's not even what stdin is primarily meant for; > stdin is meant to be the input data for a job. Similarly, stdout is > meant to be the result of the computation. Stderr, then, is used to > deliver optional diagnostic messages, ie, metainfo about the > computation. You really need punched tape or cards for input and output to get the full effect. Complete with all the sound effects. Sadly when I came into it, we were using mag-tape for input (with a variety of outputs). The clatter of an ASR33 printing your results was also a treat. -- bartc From rosuav at gmail.com Fri Oct 6 15:21:55 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 06:21:55 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On Sat, Oct 7, 2017 at 5:51 AM, bartc wrote: > On 06/10/2017 18:42, Chris Angelico wrote: >> >> On Sat, Oct 7, 2017 at 4:13 AM, bartc wrote: > > >>> So what's the excuse for an unresponsive text display in 2017? >> >> >> Got it. You assume that a system is a coherent computer with its >> peripherals, rather than being a networked collection of computers, >> all of them controlled from your laptop in response to a panicked >> email saying the web site is down. Your terminal exists on your >> laptop. The programs are being run on any of a number of your servers, >> possibly several of them concurrently. How do you handle that? >> >>>> What text editor do you use there? >>> >>> >>> That depends; are we still in the 1970s? >> >> >> Nope. We're in 2017, where the "system" is often a networked >> collection of computers. > > > Still no excuse. We were talking about the need to use a crude keyboard > interface to serially create a file of data a line at a time rather than > using a proper application for it (such as a text editor). > > So which bit do you have access to locally? The keyboard presumably. And the > display has to be local unless you it's either got a 50' screen or you use > binoculars. Are you saying the bit in the middle is remote and that is what > is playing up? > > This is exactly the situation from the 1970s that I thought we'd long left > behind! There really is no excuse for not having the minimum amount of local > processing power available to be able to enter and review a few dozen lines > of text without needing to involve a temperamental network. > > If you're stuck, whip out a tablet computer or smartphone (they should still > function without connectivity) and use a preloaded text editor. Or just > compose and then save an email. Even the simplest should be more > sophisticated than just blindly entering text on a Linux terminal screen. Save an email and then what? You need to make a change to something on a remote server. Maybe your hosts file is borked. Maybe the mail server configuration is slightly off, resulting in delayed delivery. Maybe your bindfile has an outdated record in it so half your users are getting timeouts. Whatever it is, it's almost certainly going to be managed by a text file in /etc, so you need to edit that file. How do you edit a file on a remote computer? How do you compare three instances of that file on different computers to see if one of them is different? >>> (And since I mainly use my own languages, actually getting hold of >>> 'stdin', >>> 'stdout' and 'stderr' is not trivial. Creating actual named files is much >>> easier.) >> >> >> More blub happening right there. You start out by assuming that the >> standard streams are unimportant, therefore you think that providing >> them is pointless. It's self-perpetuating. > > > They were a creation of either C, Unix, or some combination. Certainly I had > very little to with any of that for decades. I can't remember that it ever > stopped me doing anything I needed to do. That's Blub. You've never used it, so you don't understand its value, so you scorn it. > My own programs worked like this (showing original '#' symbol I used); > > print "One" # to console (also using print #0) > print #f, "Two" # to file handle f > print #s, "Three" # append to string s > print #w, "Four" # to window or control w > print #b, "Five" # to image b I'm not sure what printing to a window or image would mean, or how it's useful, but sure. In Python, you could do the same by making those things into file-like objects. > What would I want with stderr?! That's where you print error messages. It is, in fact, the standard place to print error messages. That's what it's for. If you're running a program interactively, stderr goes to the console. If you're running it interactively with output redirected, stderr still goes to the console. If you run it detached as a system service, stderr goes to the system logger. (At least, it should; it does with a lot of service managers.) If you run it from a web application framework, stderr goes into the web server's log. Etcetera. The program doesn't need to know or care - it just writes text to stderr. >> It's 2017. You should understand at least a bit about the internet and >> how to use it. > > I can use it but can't fix it when it goes wrong. Can you use it from a program, though? Not just working in a web browser, mail client, news client, etc - can your applications establish socket connections, listen for incoming connections, etc? >> I don't know if you're an idiot or a troll. Using TCP/IP networking is >> pretty simple (at least if you're using a real language - your own toy >> languages might have made it unnecessarily hard, for all I know), >> hardly "over-complex" by comparison to RS-232 programming. > > > So how do you use it from Python - without using an existing Python library? > Or is it only simple when someone else has made it idiot-proof? The only "existing Python library" you need is the Python stdlib, which is effectively part of the language. Basic socket services are right there in the 'socket' module, and there are various higher-level protocols provided by their own modules. >>> I just don't work that way. The OS is there to launch applications, or to >>> provide a basic API for common services such as a file system. I don't >>> need >>> its complicated shells. >> >> >> Blub rears its head again. > > How so? I said I have little need for most of what an OS does, in my > development work, which is perfectly true. Perhaps you haven't noticed that > many are using sophisticated IDEs now rather than fiddling about with > command lines. (Mine is not sophisticated, but still better than a command > line.) Some are. Many aren't. In fact, the basic terminal has many advantages over purely GUI systems, including that you can scroll back and see exactly what happened previously. I generally recommend my students to use a sophisticated text editor, and then a separate terminal (or, as I tend to do, *many* separate terminals) to run commands in. >> At what point will you acknowledge that there are things you do not >> understand that are actually useful? > > > I can understand that people who use Unix and Linux arrange their world so > that all these things are apparently indispensable. > > What you're trying to tell me is that because Unix provides such a tightly > knit system system of utilities, stdin, stdout, files, pipes and very > specific ways of using the keyword (which seems to consist of avoiding > actually using it for entering data as much a possible; just using the > logical idea of it), then every other text entry system in the world, in > whichever OS or application, must follow exactly the same model. Because the > Unix one is so great. > > Needless to say, I disagree. Yeah. It's called an "ecosystem". And it's called "consistency". They're good things. ChrisA From rosuav at gmail.com Fri Oct 6 15:31:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 06:31:58 +1100 Subject: The "loop and a half" In-Reply-To: <_FQBB.1006789$cS2.932107@fx19.am4> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: On Sat, Oct 7, 2017 at 6:18 AM, bartc wrote: >> I don't know if anybody has seen a market/need for an interactive sort >> program, but there's nothing preventing you from writing one. > > > For sort, there is no real need. You use a text editor to create your data. > Then use existing file-based sort. How about this: you run "du -sb *" to get directory sizes, and then after the info's on screen, you want to show the five largest. You could rerun the command, piped into "sort -n", or you could grab the existing text from your console and sort it. Do you have to switch across to a text editor, save the output there, and then sort it? Or do you invoke "sort -n" with no other args, paste straight from terminal back into the same terminal, hit Ctrl-D, and then read the output? I have done this exact thing multiple times. ChrisA From grant.b.edwards at gmail.com Fri Oct 6 15:37:07 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 19:37:07 +0000 (UTC) Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 2017-10-06, Chris Angelico wrote: > Some are. Many aren't. In fact, the basic terminal has many advantages > over purely GUI systems, including that you can scroll back and see > exactly what happened previously. I generally recommend my students to > use a sophisticated text editor, and then a separate terminal (or, as > I tend to do, *many* separate terminals) to run commands in. Indeed, I find that to be very efficient. I'm always amazed how long it takes people to accomplish simple tasks when they refuse to use anything other than eclipse and a web browser. -- Grant Edwards grant.b.edwards Yow! I'm pretending that at we're all watching PHIL gmail.com SILVERS instead of RICARDO MONTALBAN! From grant.b.edwards at gmail.com Fri Oct 6 15:38:37 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 6 Oct 2017 19:38:37 +0000 (UTC) Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: On 2017-10-06, bartc wrote: > For sort, there is no real need. You use a text editor to create > your data. Then use existing file-based sort. I sort streams on stdin far more often than I sort named files. -- Grant Edwards grant.b.edwards Yow! Do you like "TENDER at VITTLES"? gmail.com From marko at pacujo.net Fri Oct 6 15:41:08 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 22:41:08 +0300 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: <87zi94ukrv.fsf@elektro.pacujo.net> bartc : >> Personally, I think stdin is a bit lame as a stimulus source for an >> interactive program. That's not even what stdin is primarily meant for; >> stdin is meant to be the input data for a job. Similarly, stdout is >> meant to be the result of the computation. Stderr, then, is used to >> deliver optional diagnostic messages, ie, metainfo about the >> computation. > > You really need punched tape or cards for input and output to get the > full effect. Complete with all the sound effects. Sadly when I came into > it, we were using mag-tape for input (with a variety of outputs). > > The clatter of an ASR33 printing your results was also a treat. There you go! You got the idea. Keep that in your head when you are programming with stdin and stdout. Marko From marko at pacujo.net Fri Oct 6 15:55:50 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 06 Oct 2017 22:55:50 +0300 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <87vajsuk3d.fsf@elektro.pacujo.net> Grant Edwards : > I'm always amazed how long it takes people to accomplish simple tasks > when they refuse to use anything other than eclipse and a web browser. Now I can bring this back to Python. I have had a huge task of arranging 1000+ soccer games in a tournament. I could have used a web service for the purpose, but chose to use Python instead -- to great success! It's actually not about combining Unix tools but using programming to express ad hoc relationships between data. I could set up the games and have the program verify the diverse and individualistic boundary conditions are obeyed. Whenever customers come up with impromptu requests and change their minds, the Python program can accommodate the sudden needs with ease. The Web service owner was dismayed that I wasn't using their fancy web service. There was a time in computing when people fancied everybody should know how to program and apply the skill to everyday tasks. I don't think many people have such misconceptions anymore. However, it's a bit depressing because I have had a tremendous advantage several times when I've been able to solve my problems with programming. And Python's the obvious choice for the job. So today at 3:15 pm I was sent an Excel template I should use to deliver the game schedule. By 3:40 pm I had modified my program to generate the desired Excel format (with openpyxl) and sent the schedule to the referee administrator as an attachment. I told him I could send him a different format if he needed it, and advised him he should be prepared for multiple iterations as new requests arrive from customers. I wouldn't have managed the task with the mouse and the clunky web service. Marko From stephanh42 at gmail.com.invalid Fri Oct 6 15:58:05 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 06 Oct 2017 19:58:05 GMT Subject: How to determine lowest version of Python 3 to run? References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> <59d691ee$0$786$e4fe514c@news.xs4all.nl> <751B5F5E-F484-4CA9-8393-54AD097C376F@icloud.com> Message-ID: Op 2017-10-06, Christopher Reimer schreef : > So I got tox and tox-docker installed. When I went to install Docker > for Windows, it wouldn't work because Hyper-V wasn't available on > Windows 10 Home. After paying Microsoft $99 for the privilege, I got > Windows 10 Pro installed and Docker started working. I've read that > all the cool programming kids are using Docker these days. I certainly > hope it was worth the cost. O_o You could have just used a Linux VM in Virtualbox for $0, and run Docker in that. Stephan From bc at freeuk.com Fri Oct 6 16:01:56 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 21:01:56 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: On 06/10/2017 20:38, Grant Edwards wrote: > On 2017-10-06, bartc wrote: > >> For sort, there is no real need. You use a text editor to create >> your data. Then use existing file-based sort. > > I sort streams on stdin far more often than I sort named files. So what's been established is that 'sort' is useful for a text stream that already exists as a file or is the output of another process. What hasn't been established is why how this works this has to influence the design of simple text-based dialogues, real ones where there will likely be an actual user tapping the keys in real time. The only tenuous connection I can see, is that if 'sort' is run without any piped or redirected input, it will resort to the keyboard. Even though that method is not user-friendly and hardly anyone ever uses it in that mode. So that same unfriendly technique should be the model for text dialogues everywhere. -- bartc From tjreedy at udel.edu Fri Oct 6 16:11:16 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 6 Oct 2017 16:11:16 -0400 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On 10/6/2017 1:32 PM, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 4:05 AM, Grant Edwards wrote: >> On 2017-10-06, Thomas Jollans wrote: >> >>> Seriously? sys.stdin can be None? That's terrifying. >> >> Why? >> >> Unix daemons usually run with no stdin, stderr, or stdout. >> >> And yes, people do write Unix daemons in Python. > > Hmm, but usually I would expect them still to HAVE those streams, > they're just connected to /dev/null or something. I don't think they > would actually fail to exist, would they? On Windows, a file run with pythonw.exe (no console) starts with sys.[__]std[in|out|err][__] (6 entries) set to None. The program can reset them as it pleases. In an IDLE user process, the non-dunder names are set to objects that communicate with the IDLE GUI process. >>> import sys; sys.__stdin__, sys.stdin (None, ) GUI programs interact with the OS and thence users through event streams rather than character streams. (Note that terminal escape sequences are character-encoded events.) That is why automating a GUI usually requires a special program than can generate and inject events into the event queue read by the GUI program. -- Terry Jan Reedy From bc at freeuk.com Fri Oct 6 16:32:06 2017 From: bc at freeuk.com (bartc) Date: Fri, 6 Oct 2017 21:32:06 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On 06/10/2017 20:21, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 5:51 AM, bartc wrote: >> If you're stuck, whip out a tablet computer or smartphone (they should still >> function without connectivity) and use a preloaded text editor. Or just >> compose and then save an email. Even the simplest should be more >> sophisticated than just blindly entering text on a Linux terminal screen. > > Save an email and then what? You will have been able to create and review the text using normal editing methods. It will then be saved as a draft somewhere. When the sort program is back up (if that's what you're planning), then retrieve the text with copy&paste. No need to actually send it anywhere. My point is that everyone has the facility to create at least plain text documents off-line. There is no need to start some ancient terminal utility, get no response from it, then blindly type in line after line with no idea if it's doing anything with it. > You need to make a change to something on > a remote server. Maybe your hosts file is borked. Maybe the mail > server configuration is slightly off, resulting in delayed delivery. > Maybe your bindfile has an outdated record in it so half your users > are getting timeouts. Whatever it is, it's almost certainly going to > be managed by a text file in /etc, so you need to edit that file. > > How do you edit a file on a remote computer? How do you compare three > instances of that file on different computers to see if one of them is > different? I've no idea where you're going with this. Remember my original complaint was in treating keyboard entry like a file complete with EOF marker. >> They were a creation of either C, Unix, or some combination. Certainly I had >> very little to with any of that for decades. I can't remember that it ever >> stopped me doing anything I needed to do. > > That's Blub. You've never used it, so you don't understand its value, > so you scorn it. OK, that works both ways as you've never used any of my stuff. >> print #w, "Four" # to window or control w >> print #b, "Five" # to image b > I'm not sure what printing to a window or image would mean, or how > it's useful, It displayed the text at the current position (using the current font, size and colour) and moved the current position to the end of the text. -- bartc From eryksun at gmail.com Fri Oct 6 16:40:02 2017 From: eryksun at gmail.com (eryk sun) Date: Fri, 6 Oct 2017 21:40:02 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: On Fri, Oct 6, 2017 at 4:27 PM, Thomas Jollans wrote: > On 2017-10-06 17:01, eryk sun wrote: >> >> POSIX defines STDIN_FILENO as 0, and the Windows C runtime reserves FD >> 0 to map to the native StandardInput handle. But as I noted in a >> previous message, on Windows isatty(0) returns true if a process isn't >> executed with a valid StandardInput. In this case sys.stdin will be >> None, so call sys.stdin.fileno() and handle the exception if that >> fails. > > Seriously? sys.stdin can be None? That's terrifying. Just check for None or handle the exception. It could be worse in Windows -- like inheriting broken standard I/O. This can happen if a console application is run without a console (i.e. dwCreationFlags=DETACHED_PROCESS) yet console handles are inherited in the standard handles via STARTUPINFO. isatty() will return True, but using the handles will fail. To avoid this problem with subprocess.Popen, if you use creationflags=DETACHED_PROCESS, make sure to set unused standard handles to subprocess.DEVNULL. From rosuav at gmail.com Fri Oct 6 16:49:10 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 07:49:10 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: On Sat, Oct 7, 2017 at 7:32 AM, bartc wrote: > On 06/10/2017 20:21, Chris Angelico wrote: >> >> On Sat, Oct 7, 2017 at 5:51 AM, bartc wrote: > > >>> If you're stuck, whip out a tablet computer or smartphone (they should >>> still >>> function without connectivity) and use a preloaded text editor. Or just >>> compose and then save an email. Even the simplest should be more >>> sophisticated than just blindly entering text on a Linux terminal screen. >> >> >> Save an email and then what? > > > You will have been able to create and review the text using normal editing > methods. It will then be saved as a draft somewhere. When the sort program > is back up (if that's what you're planning), then retrieve the text with > copy&paste. No need to actually send it anywhere. > > My point is that everyone has the facility to create at least plain text > documents off-line. > > There is no need to start some ancient terminal utility, get no response > from it, then blindly type in line after line with no idea if it's doing > anything with it. Saving a text file doesn't actually change that file anywhere. Yes, anyone can edit text files that do nothing. Great. Your task is to actually edit a file that exists on a remote server. You can't do that through a draft email. ChrisA From tjol at tjol.eu Fri Oct 6 16:55:53 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 6 Oct 2017 22:55:53 +0200 Subject: callable values In-Reply-To: References: <1507293873l.25886794l.0l@psu.edu> Message-ID: <86fb642b-6972-ef4c-6313-3e3cf1346223@tjol.eu> On 06/10/17 18:25, Stefan Ram wrote: > Terry Reedy writes: >> On 10/6/2017 8:44 AM, ROGER GRAYDON CHRISTMAN wrote: >>> Despite the documentation, I would still be tempted to say that range is a >>> function. >> It is, *according* to the documentation. Built-in classes are included >> in Library Reference, Ch. 2, Built-in Functions. Changing that to >> "Built-in Functions and Classes" has been proposed (perhaps by me) and >> rejected. > FWIW, in my course notes, I have coined a special word for > this: > > A /prelate/ (German: "Pr?lat") is a callable value (object). Are you just going to let that hanging without justifying your choice of word? > > For, example, I say: > > |When certain prelates are called, one has to write an > |expression into the parentheses, which is called "argument > |expression. In the case of the call ?neg( 9 )?, ?9? is the > |argument expression. > > (In the original German text this is: > > |Beim Aufruf mancher Pr?laten kann oder mu? in den f?r den > |Aufruf verwendeten Klammern ein Ausdruck geschrieben werden, > |der Argumentausdruck genannt wird. Im Aufruf ?neg( 9 )? ist > |?9? der Argumentausdruck. > > .) > From christopher_reimer at icloud.com Fri Oct 6 17:48:53 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Fri, 06 Oct 2017 14:48:53 -0700 Subject: How to determine lowest version of Python 3 to run? In-Reply-To: References: <5D58CE67-376A-407E-93E4-ABAC3C7BA932@icloud.com> <59d691ee$0$786$e4fe514c@news.xs4all.nl> <751B5F5E-F484-4CA9-8393-54AD097C376F@icloud.com> Message-ID: <000EAB8F-706F-42AE-BA21-E389894C0E30@icloud.com> On Oct 6, 2017, at 12:58 PM, Stephan Houben wrote: > > Op 2017-10-06, Christopher Reimer schreef : > >> So I got tox and tox-docker installed. When I went to install Docker >> for Windows, it wouldn't work because Hyper-V wasn't available on >> Windows 10 Home. After paying Microsoft $99 for the privilege, I got >> Windows 10 Pro installed and Docker started working. I've read that >> all the cool programming kids are using Docker these days. I certainly >> hope it was worth the cost. O_o > > You could have just used a Linux VM in Virtualbox for $0, and run Docker > in that. > > Stephan > -- > https://mail.python.org/mailman/listinfo/python-lists My $200 Dell laptop couldn't handle that combo and it would add an extra layer of complexity to my programming setup. Chris R. From steve+python at pearwood.info Fri Oct 6 19:43:00 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 10:43:00 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 12:24 am, bartc wrote: > print ("Enter blank expression to quit.") I *despise* programs that do that, and would cheerfully and unapologetically take their designers, disguise them as a lettuce, and stake them out to be nibbled to death by snails. At the interactive prompt, I am frequently hitting Enter on a blank line, either by accident, or deliberately to break up the calculations into groups, or just to give myself time to think. Blank lines should be treated as "do nothing" and simply ignored, and there should be an explicit QUIT command. Actually two: your OS's version of EOF (Ctrl-D on Unix-like systems, Ctrl-Z ENTER on Windows), plus at least one of the usual exit/quit/bye named commands. (Localised into whatever natural language you are using for the prompts.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 6 19:49:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 10:49:56 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <59d816a7$0$14932$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 01:55 am, Chris Angelico wrote: > Since GCC is the GNU Compiler *Collection* Today I Learned that gcc doesn't mean "Gnu C Compiler". I knew gcc compiles C, C++, Objective C and Objective C++, but today I discovered it also compiles Ada, Fortran, Java and "treelang", whatever the hell that is. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Fri Oct 6 19:50:08 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 07 Oct 2017 00:50:08 +0100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <59d75dc6$0$14973$b1db1813$d948b532@news.astraweb.com> Message-ID: <87d15z6dlb.fsf@bsb.me.uk> Steve D'Aprano writes: > On Fri, 6 Oct 2017 09:33 pm, Ben Bacarisse wrote: > >> A general solution to the (rather odd) complaint about silent waiting >> should really check any input fileno to see if a prompt is needed. ?You >> could argue, though, that anyone who's re-arranged a program's input so >> that some non-zero input fileno is attached to a terminal won't need the >> prompt! > > I'm afraid I don't quite know if I'm understanding you or not. > > I think you mean to say I should look at sys.stdin.fileno(), and if it is 0, > then write a prompt, and if not, just read from stdin (possibly blocking, > waiting for input). > > Is that right? > > But aren't there circumstances where fileno 0 isn't attached to a terminal, > and writing a prompt would be inappropriate? No. The point was that other file descriptors might be associated with a terminal. I could, for example, switch the input to an interactive program that interprets command to be done on some data file so that the supposedly interactive commands come form a file and I type the data at a terminal. The second point was anyone doing this probably does not need a hint about tying something. But this detail about other inputs opened by the program aside, it seems to me that what BartC (and others who want this) needs is a beginners shell that prints a helpful text whenever it starts a program (or pipeline of programs) with stdin attached to the terminal. It's not really something that every individual program should have to tackle. -- Ben. From rosuav at gmail.com Fri Oct 6 19:52:56 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 10:52:56 +1100 Subject: The "loop and a half" In-Reply-To: <59d816a7$0$14932$b1db1813$d948b532@news.astraweb.com> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d816a7$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 7, 2017 at 10:49 AM, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 01:55 am, Chris Angelico wrote: > >> Since GCC is the GNU Compiler *Collection* > > Today I Learned that gcc doesn't mean "Gnu C Compiler". > > I knew gcc compiles C, C++, Objective C and Objective C++, but today I > discovered it also compiles Ada, Fortran, Java and "treelang", whatever the > hell that is. > Yeah. I think it *was* originally "GNU C Compiler", but when all the others got added, they renamed it. I may be wrong on that though. ChrisA From steve+python at pearwood.info Fri Oct 6 19:58:54 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 10:58:54 +1100 Subject: callable values (was: Introducing the "for" loop) References: <1507293873l.25886794l.0l@psu.edu> Message-ID: <59d818c1$0$14941$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 03:25 am, Stefan Ram wrote: > FWIW, in my course notes, I have coined a special word for > this: > > A /prelate/ (German: "Pr?lat") is a callable value (object). In English, prelate is a kind of priest, a senior clergyman and dignitary. I don't know whether German makes a practice of taking arbitrary verbs and adjectives and turning them into nouns, but English does, and so a callable object is just called a "callable". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Fri Oct 6 20:05:50 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 01:05:50 +0100 Subject: The "loop and a half" In-Reply-To: <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 00:43, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 12:24 am, bartc wrote: > >> print ("Enter blank expression to quit.") > > > I *despise* programs that do that, and would cheerfully and unapologetically > take their designers, disguise them as a lettuce, and stake them out to be > nibbled to death by snails. > > At the interactive prompt, I am frequently hitting Enter on a blank line, > either by accident, or deliberately to break up the calculations into groups, > or just to give myself time to think. > > Blank lines should be treated as "do nothing" and simply ignored, and there > should be an explicit QUIT command. Um, that actually follows what interactive Python does. If you type this (I'm using <<< as the prompt as >>> confuses my newsreader's quoting system): <<< def fn(): <<< pass <<< pass At this point, you can break out by pressing the key for 'eof', or by pressing Enter at the start of a line. Even though a blank line is legal Python syntax. (Most of my real interactive programs with a command line interface programs use Escape, quit, exit, q or x to finish. Interactive Python requires quit() or exit(), complete with parentheses. Unless you've redefined quit and exit as something else, then you have to crash out by other means.) -- bartc From ben.usenet at bsb.me.uk Fri Oct 6 20:14:11 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 07 Oct 2017 01:14:11 +0100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> Message-ID: <878tgn6ch8.fsf@bsb.me.uk> bartc writes: > On 06/10/2017 14:35, Paul Moore wrote: >> On 6 October 2017 at 13:56, bartc wrote: >>> If you don't like the word 'crude', try 'lazy'. Take this example of the gcc >>> C compiler: >>> >>> > gcc -E program.c >>> >>> This preprocesses the code and shows the result. Typical programs will have >>> many thousands of lines of output, but it just dumps it to the console. You >>> /have/ to use '>' to use it practically (Windows doesn't really have a >>> working '|' system.) Does the Windows version of gcc not support the -o file option? >> No you don't. Ignoring the fact that "windows doesn't really have a >> working '|' system" (which is an oversimplification, by the way) the >> following all work: >> >> Python: >> data = subprocess.check_output(["gcc", "-E", "program.c"]) >> Powershell: >> $x = (gcc -E program.c) >> cmd: >> for /f %i in ('gcc -E program.c') do ... >> >> If gcc -E wrote its output to a file, you'd have to read that file, >> manage the process of deleting it after use (and handle possible >> deletion of it if an error occurred), etc. > > But if I use the -S option (show assembly listing) that DOES output to > a file (program.s). And a good system lets you alter that. How many labels are generated for some code? Let's see: gcc -S -o /dev/stdout t.c | grep -c '^\.L' Now gcc also permits -o - (it's a common Unix convention) but you can almost always get round less helpful programs using /dev/sdout. -- Ben. From steve+python at pearwood.info Fri Oct 6 20:19:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 11:19:14 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> <87fuaww3xs.fsf@elektro.pacujo.net> Message-ID: <59d81d85$0$14936$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 05:33 am, Grant Edwards wrote: > On 2017-10-06, Marko Rauhamaa wrote: > >> The reason a daemon usually opens dummy file descriptors for the 0, 1 >> and 2 slots is to avoid accidents. Some library might assume the >> existence of those file descriptors. For example, I often see GTK print >> out diagnositic messages. > > I run a lot of GTK programs from the command line, and I would say 90% > or more of them spew a steady stream of meaningless (to the user) > diagnostic messages. That's pretty sloppy programming if you ask > me... Indeed. If you ever start to doubt that the average quality of software is "crap", try running GUI applications from the command line and watch the stream of warnings and errors flow by. They range from relatively trivial warnings that correctly sized icons are missing, to scary looking warnings about uninitialised pointers. Developers: why do you bother to log warnings if you're never going to fix the damn things? They're probably to busy re-doing working programs from scratch every few versions, with a brand new "improved" UI (almost invariably including a kool new design that uses light grey text on an ever so slightly lighter grey background) and deleting any useful functionality that the lead developer personally doesn't use, because "nobody uses that". https://www.jwz.org/doc/cadt.html -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Fri Oct 6 20:25:07 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 07 Oct 2017 01:25:07 +0100 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> Message-ID: <874lrb6bz0.fsf@bsb.me.uk> bartc writes: > It is anyway not really acceptable these days for a long list of data > to simply be typed in like that without any feedback at all. And 100% > dependent on your typing Ctrl-D at the end and not Ctrl-C by > mistake. This is not still the 1970s. It was not acceptable in the 1970s either. No one did it then and no one does it now. What you are actually saying is that it is not acceptable that the option of unprompted input even exists. -- Ben. From steve+python at pearwood.info Fri Oct 6 20:30:28 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 11:30:28 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <8760bsw1tf.fsf@elektro.pacujo.net> Message-ID: <59d82027$0$14964$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 05:56 am, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 5:47 AM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> On Sat, Oct 7, 2017 at 4:55 AM, Marko Rauhamaa wrote: >>>> Personally, I think stdin is a bit lame as a stimulus source for an >>>> interactive program. That's not even what stdin is primarily meant >>>> for; stdin is meant to be the input data for a job. Similarly, stdout >>>> is meant to be the result of the computation. Stderr, then, is used >>>> to deliver optional diagnostic messages, ie, metainfo about the >>>> computation. >>> >>> [...] >>> >>> Standard streams exist for a reason. >> >> That's exactly what I said. > > And you also described it as "lame" for an interactive program. I think what Marko meant is that using stdin ALONE, with no prompts, no help, and no explanation, provides a bare-bones, featureless and underpowered user interface unsuitable for an interactive program. Let's come back to sort again: we all agree that sort reading from stdin is *not* a user-friendly interface suitable for an interactive program. "Lame" is a good word to describe an interface so bleakly featureless, underpowered and user-hostile. But that interface wasn't designed for interactive use. It was designed for passing data in via a file or pipe. The fact that it happens to work at all is a happy accident, and one which experienced users can take advantage of, but the primary intention of sort is that it will read from a pipe or a file, not the keyboard. It works because that's how Unix works, not because it was intentionally designed that way. The sorts of games your students write use stdin to get input from the user, but they don't -- or at least, shouldn't -- merely dump the user at a bare screen with no prompt and no explanation of what they're supposed to do. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Fri Oct 6 20:47:49 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 01:47:49 +0100 Subject: The "loop and a half" In-Reply-To: <878tgn6ch8.fsf@bsb.me.uk> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> <878tgn6ch8.fsf@bsb.me.uk> Message-ID: On 07/10/2017 01:14, Ben Bacarisse wrote: > bartc writes: > >> On 06/10/2017 14:35, Paul Moore wrote: >>> On 6 October 2017 at 13:56, bartc wrote: >>>> If you don't like the word 'crude', try 'lazy'. Take this example of the gcc >>>> C compiler: >>>> >>>> > gcc -E program.c >>>> >>>> This preprocesses the code and shows the result. Typical programs will have >>>> many thousands of lines of output, but it just dumps it to the console. You >>>> /have/ to use '>' to use it practically (Windows doesn't really have a >>>> working '|' system.) > > Does the Windows version of gcc not support the -o file option? I never thought of trying it. But OK, you have to use -o or > to stop the output from swamping the console. (| might work if I could find a program that captured the piped output for the purpose of browsing.) (In my IDE, if a particular C file is highlighted, then typing 'cpp' will invoke the currently selected C compiler's -E equivalent, then instantly show the result in the editor. Looking at the script for that, that does make use of the -o option, for gcc, so I must have known about it.) > And a good system lets you alter that. How many labels are generated > for some code? Let's see: > > gcc -S -o /dev/stdout t.c | grep -c '^\.L' > > Now gcc also permits -o - (it's a common Unix convention) but you can > almost always get round less helpful programs using /dev/sdout. I'm not really arguing against this. I'm disputing the suggestion that because these utilities, which are rarely given data actually typed from the keyboard, work by reading from stdin using an eof-checking loop, that the same method must always be used in programs that really are reading data from the keyboard. I think they are different kinds of applications. (As for the ability to chain these programs together, I have the same opinion of that as doing the same in program code when you use the result of one function/method call to feed in as the argument to the next. They can both end up as long horizontal hard-to-follow (or to intercept) sequences. I'd prefer to break them up using intermediate variables in the language and intermediate files in the shell. And as you can see from my IDE's 'cpp' command, which works with 4 different C compilers (a fifth doesn't properly support -E), there are other approaches developers can use.) -- bartc From rosuav at gmail.com Fri Oct 6 21:09:03 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 12:09:03 +1100 Subject: Warnings (was Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]) Message-ID: On Sat, Oct 7, 2017 at 11:19 AM, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 05:33 am, Grant Edwards wrote: > >> On 2017-10-06, Marko Rauhamaa wrote: >> >>> The reason a daemon usually opens dummy file descriptors for the 0, 1 >>> and 2 slots is to avoid accidents. Some library might assume the >>> existence of those file descriptors. For example, I often see GTK print >>> out diagnositic messages. >> >> I run a lot of GTK programs from the command line, and I would say 90% >> or more of them spew a steady stream of meaningless (to the user) >> diagnostic messages. That's pretty sloppy programming if you ask >> me... > > Indeed. > > If you ever start to doubt that the average quality of software is "crap", try > running GUI applications from the command line and watch the stream of > warnings and errors flow by. They range from relatively trivial warnings that > correctly sized icons are missing, to scary looking warnings about > uninitialised pointers. > > Developers: why do you bother to log warnings if you're never going to fix the > damn things? The warnings usually come from the library, but the bugs are in the application. So the question is: is it right for a library to raise console warnings like that? Under what circumstances and to what destinations should a library report on potential problems? IMO the console belongs to the application, not a library. But that means that the logging of warnings needs to be done by the app. I'm not sure what the solution is. ChrisA From ben.usenet at bsb.me.uk Fri Oct 6 21:29:47 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 07 Oct 2017 02:29:47 +0100 Subject: The "loop and a half" References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> <878tgn6ch8.fsf@bsb.me.uk> Message-ID: <87zi934ues.fsf@bsb.me.uk> bartc writes: > On 07/10/2017 01:14, Ben Bacarisse wrote: >> bartc writes: >> >>> On 06/10/2017 14:35, Paul Moore wrote: >>>> On 6 October 2017 at 13:56, bartc wrote: >>>>> If you don't like the word 'crude', try 'lazy'. Take this example of the gcc >>>>> C compiler: >>>>> >>>>> > gcc -E program.c >>>>> >>>>> This preprocesses the code and shows the result. Typical programs will have >>>>> many thousands of lines of output, but it just dumps it to the console. You >>>>> /have/ to use '>' to use it practically (Windows doesn't really have a >>>>> working '|' system.) >> >> Does the Windows version of gcc not support the -o file option? > > I never thought of trying it. But OK, you have to use -o or > to stop > the output from swamping the console. (| might work if I could find a > program that captured the piped output for the purpose of browsing.) >> And a good system lets you alter that. How many labels are generated >> for some code? Let's see: >> >> gcc -S -o /dev/stdout t.c | grep -c '^\.L' >> >> Now gcc also permits -o - (it's a common Unix convention) but you can >> almost always get round less helpful programs using /dev/sdout. > > I'm not really arguing against this. I'm disputing the suggestion that > because these utilities, which are rarely given data actually typed > from the keyboard, work by reading from stdin using an eof-checking > loop, that the same method must always be used in programs that really > are reading data from the keyboard. And I'm not arguing about that -- I know there is no point. The way Usenet works is that I reply to the parts I want to comment on. No one can see what made me make the above remark because you cut it, but the combined effect of the two remarks of yours that I commented on was to suggest that gcc makes arbitrary choices about output destinations (and I assume gcc was just an example). I was simply saying that a good OS can finesse such matters. -- Ben. From steve+python at pearwood.info Fri Oct 6 21:46:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 12:46:14 +1100 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 06:18 am, bartc wrote: > For sort, there is no real need. You use a text editor to create your > data. Then use existing file-based sort. What you mean is, *you* see no need for sorting interactively, or sorting as part of a pipeline of functions. That speaks more of your lack of imagination and closed-mindedness than any real lack. Personally, if I have a few (say, a dozen or so) lines I want sorted, it is much easier to: - select text from whatever application I'm using; - bring up a terminal (I almost always have one already open); - type `sort ENTER`; - middle-click to paste the lines; - type Ctrl-D; - select the output lines; - paste back into the original application; (seven steps) versus: - select text from whatever application I'm using; - open a text editor; - paste into the text editor; - hit Ctrl-S to save; - think of a file name ("temp123"); - type it in the file save dialog; - click SAVE to save; - invariably the file name is already used from the previous time I needed a temporary file and forgot to delete it afterwards; - so look more carefully at the existing files, and think of a new name; - edit the file name; - click SAVE again; - open an external sort program (do I even have one? what's it called again?); - in the sort program, whatever it is called, type Ctrl-O to open; - navigate to the directory I saved the temp file; - open that file; - sort the file; - save the output; - type Ctrl-Q to quit; - go back to the editor; - select the File > Reload menu command to reload the now sorted file; - select the text; - go back to the application I started with; - paste; - go back to the editor and hit Ctrl-Q to quit; - go to a file manager; - locate the temporary file; - hit backspace to send it to the trash (27 steps). Yes, I can certainly see why you prefer that second work-flow to the first. Its so much more inefficient than memorising "Ctrl-D for EOF". I forget... it is "work harder, not smarter"? Being inefficient is a good thing, right? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 6 21:50:12 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 12:50:12 +1100 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <59d832d4$0$14944$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 06:21 am, Chris Angelico wrote: > I'm not sure what printing to a window or image would mean, or how > it's useful, but sure. Print to window: Print Preview. Print to image: export to pdf or jpg or png. More useful for rich GUI apps than plain old text apps, but the basic principle is still sound. I recently had a HUGE LibreOffice draw document, containing a very long mathematical formula, and the only practical way for me to turn it into an image was by taking 14 screenshots and then merging them by hand. A working export to PNG command would have been really useful. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Oct 6 22:18:46 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 7 Oct 2017 13:18:46 +1100 Subject: The "loop and a half" In-Reply-To: <59d832d4$0$14944$b1db1813$d948b532@news.astraweb.com> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d832d4$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 7, 2017 at 12:50 PM, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 06:21 am, Chris Angelico wrote: > >> I'm not sure what printing to a window or image would mean, or how >> it's useful, but sure. > > Print to window: Print Preview. > > Print to image: export to pdf or jpg or png. Except that that isn't what you get in his programming language. What you seem to get (judging from his response to the post you're quoting) is something that lets you treat a GUI window as if it were a console, printing text to it in some sort of "glass teletype" way. Leaves me wondering if he's ever taken one of his apps to a system that uses a different default font, or anything like that. Unless all his GUIs are built in HTML? > More useful for rich GUI apps than plain old text apps, but the basic > principle is still sound. I recently had a HUGE LibreOffice draw document, > containing a very long mathematical formula, and the only practical way for > me to turn it into an image was by taking 14 screenshots and then merging > them by hand. A working export to PNG command would have been really useful. Yes, absolutely agree. But personally, I'd have looked for a "print to PS" of some sort, using a gigantic 'page' size, and then convert the PS to PNG. I don't know for certain that I can do the latter conversion, but there are a couple of places I'd look, including the image processing module in a language's standard library ("does this support postscript load?") and typing "ps2" in the terminal and hitting Tab twice. ChrisA From steve+python at pearwood.info Sat Oct 7 04:35:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 19:35:08 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 11:05 am, bartc wrote: > On 07/10/2017 00:43, Steve D'Aprano wrote: >> On Sat, 7 Oct 2017 12:24 am, bartc wrote: >> >>> print ("Enter blank expression to quit.") >> >> >> I *despise* programs that do that, and would cheerfully and >> unapologetically take their designers, disguise them as a lettuce, and >> stake them out to be nibbled to death by snails. >> >> At the interactive prompt, I am frequently hitting Enter on a blank line, >> either by accident, or deliberately to break up the calculations into >> groups, or just to give myself time to think. >> >> Blank lines should be treated as "do nothing" and simply ignored, and there >> should be an explicit QUIT command. > > > Um, that actually follows what interactive Python does. What is "that" to which you refer? If you mean, "what I, Bart C, suggested, namely having the program exit on a blank line", then you are wrong. In the Python interactive interpreter, you can enter blank lines, and the interpreter doesn't exit. If you mean "what Steven suggests, namely ignoring blank lines and requiring either an explicit EOF (Ctrl-D on Unix) or quit() or exit() command to exit", then you are correct, Python does do that. There's one priviso: blank lines are only ignored at the primary prompt >>> not the secondary prompt ... where a blank line is used to end the block. But the interpreter doesn't exit. A slight compromise for the sake of convenience at the interactive prompt. > (Most of my real interactive programs with a command line interface > programs use Escape, quit, exit, q or x to finish. And how do you distinguish between calling quit from inside the function, and using quit inside the function? def foo(): pass quit # or exit, q or x for that matter would be a perfectly legal (although rather pointless) function. If typing the letter x alone was enough to exit the block, how do you type `x=1` when the instance you press `x` the interpreter helpfully exits the block? If you require `x` followed by ENTER, then that prohibits legal lines which consist of nothing but `x`. Unfortunately ESCAPE is already used. VT100 (the terminal emulation which is used in just about all terminals) all control sequences begin with ESC. So every time you do something like press an arrow key, the terminal sends ESC followed by other stuff. It would be painful if every time you hit an arrow key, the interpreter took it as "Exit". > Interactive Python requires quit() or exit(), complete with parentheses. > Unless you've redefined quit and exit as something else, then you have > to crash out by other means.) "Crash out", he says. If your intention is to prove that you're a know-nothing ignoramus, you're doing an excellent job of it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 7 04:57:47 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 19:57:47 +1100 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d8970d$0$14964$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 12:12 am, Paul Moore wrote: > What really bugs me is colour settings that default to dark blues on > a black background. Amen to that! Between the very low contrast, and the stereopsis whereby blues appear to recede into the distance (and bright red pull forward, appearing to float on top of the terminal), dark blue text on the background is nearly unreadable. https://en.wikipedia.org/wiki/Chromostereopsis [...] > Add the fact that > I work on shared admin accounts, where setting non-default preferences > is considered an antisocial act (even when they are "better" ;-)) and > I spend my life squinting at screens, or typing "unalias ls" to remove > the --color setting. Luckily (for everyone who has to listen to me > rant), this is "just" annoyingly badly configured systems, and not > baked in program defaults. At my previous job, the solution to that was to ban changing the default settings, but allow admins to create their own personalised defaults. Fred would log in to the shared account, and run source fred to get his personalised settings. When George logged in to the shared account, he'd see the defaults, not Fred's settings, and could run source george to get his own settings. (Obviously Fred and George would have to create the "fred" and "george" config files first.) I don't know if Windows shells have something like bash's `source` command, but if it does, this would be one solution. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 7 05:09:14 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 20:09:14 +1100 Subject: Introducing the "for" loop References: <1507293873l.25886794l.0l@psu.edu> Message-ID: <59d899bb$0$14955$b1db1813$d948b532@news.astraweb.com> On Fri, 6 Oct 2017 11:44 pm, ROGER GRAYDON CHRISTMAN wrote: > Despite the documentation, I would still be tempted to say that range is a > function. > Taking duck-typing to the meta-level, every time I use range, I use its name > followed > by a pair of parentheses enclosing one to three parameters, and I get back > an > immutable sequence object. It sure looks like a function to me. I agree -- range() is a function in the (almost) mathematical sense, something which takes arguments and returns a value. It's also a type (class), in the OOP sense: py> type(range) The term "function" is ambiguous but normally clear from context. Often, the differences make no difference, but when they are important, we can discuss them: - range is a callable (a callable object); - it is also a type/class, and calling it returns an instance; - it looks like, and behaves like, a function; - and is, conceptually, a function; - but it is *not* an instance of FunctionType: py> from types import FunctionType py> def function(): ... pass ... py> isinstance(function, FunctionType) True py> isinstance(range, FunctionType) False It is this last sense (an instance of FunctionType) which people are thinking of when they state that range is not a function. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Sat Oct 7 05:13:48 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 7 Oct 2017 05:13:48 -0400 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: <59d81d85$0$14936$b1db1813$d948b532@news.astraweb.com> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> <87fuaww3xs.fsf@elektro.pacujo.net> <59d81d85$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/6/2017 8:19 PM, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 05:33 am, Grant Edwards wrote: > >> On 2017-10-06, Marko Rauhamaa wrote: >> >>> The reason a daemon usually opens dummy file descriptors for the 0, 1 >>> and 2 slots is to avoid accidents. Some library might assume the >>> existence of those file descriptors. For example, I often see GTK print >>> out diagnositic messages. >> >> I run a lot of GTK programs from the command line, and I would say 90% >> or more of them spew a steady stream of meaningless (to the user) >> diagnostic messages. That's pretty sloppy programming if you ask >> me... > > Indeed. > > If you ever start to doubt that the average quality of software is "crap", try > running GUI applications from the command line and watch the stream of > warnings and errors flow by. They range from relatively trivial warnings that > correctly sized icons are missing, to scary looking warnings about > uninitialised pointers. IDLE does not do that because tkinter and the other stdlib modules it uses does not do that. There are occasionally extraneous messages from tcl when shutting down. > Developers: why do you bother to log warnings if you're never going to fix the > damn things? > > They're probably to busy re-doing working programs from scratch every few > versions, with a brand new "improved" UI (almost invariably including a kool > new design that uses light grey text on an ever so slightly lighter grey > background) and deleting any useful functionality that the lead developer > personally doesn't use, because "nobody uses that". > > https://www.jwz.org/doc/cadt.html > > > -- Terry Jan Reedy From steve+python at pearwood.info Sat Oct 7 05:43:02 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 20:43:02 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> Message-ID: <59d8a1a8$0$14948$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 07:01 am, bartc wrote: > On 06/10/2017 20:38, Grant Edwards wrote: >> On 2017-10-06, bartc wrote: >> >>> For sort, there is no real need. You use a text editor to create >>> your data. Then use existing file-based sort. >> >> I sort streams on stdin far more often than I sort named files. > > So what's been established is that 'sort' is useful for a text stream > that already exists as a file or is the output of another process. Indeed. And "another process" can just as easily be a human at the keyboard, although we must keep in mind the primitive line-oriented editing facilities of stdin (basically backspace and that's it). > What hasn't been established is why how this works this has to influence > the design of simple text-based dialogues, real ones where there will > likely be an actual user tapping the keys in real time. Obviously it doesn't, and as far as I can see, nobody has suggested that such primitive facilities should be treated as the archetype of *user-oriented* application programming. There are other kinds of programs than those aimed primarily at direct use by users. As I interpret what people have been saying in this thread, the claim is far more modest: - sort is designed for use in pipelines, as a typical Unix tool; - the way to exit sort is the same way you exit pretty much everything in the Unix command line, using EOF (Ctrl-D); - Unix admins and even Unix end users learn Ctrl-D just as Windows users learn ESC and Ctrl-Q and Alt-F4 and the other myriad standard ways to exit an application in the Windows world; - as a Unix tool rather than a text-based application with a rich user-interface, it doesn't need verbose output, introduction messages, help screens, and all the other bells and whistles; - although there is a valid argument to be made that sort could be a little more user-friendly in interactive use without badly compromising its primary purpose. But that's a matter of taste. Besides, we know what the nirvana of user-interfaces should look like: https://www.gnu.org/fun/jokes/ed-msg.html > The only tenuous connection I can see, is that if 'sort' is run without > any piped or redirected input, it will resort to the keyboard. It doesn't "resort to the keyboard", that's just how stdin works. If you have a car built to drive on roads, and you drive it up onto the lawn, the car doesn't need a special mode that detects the missing road and "resorts to driving on the lawn". It just happens. Stdin is like that. > Even though that method is not user-friendly True enough. `sort` is not designed to be a end-user application, although it can be used by the end-user. > and hardly anyone ever uses it in that mode. Translation: "I don't use it in that mode". > So that same unfriendly technique should be the model for > text dialogues everywhere. I'm pretty sure that nobody said that. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 7 05:56:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 07 Oct 2017 20:56:03 +1100 Subject: Warnings (was Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]) References: Message-ID: <59d8a4b6$0$14953$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 12:09 pm, Chris Angelico wrote: > So the question is: is it right for a library to raise > console warnings like that? Under what circumstances and to what > destinations should a library report on potential problems? Of course they should -- and applications should be free to disable the warnings. Personally, I think Python gets it right: by default, warnings are only printed once each. Instead of getting a million WARNING: 32x32 icon missing, using 16x16 icon you only see it once. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Sat Oct 7 08:06:23 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 13:06:23 +0100 Subject: The "loop and a half" In-Reply-To: <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 02:46, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 06:18 am, bartc wrote: > >> For sort, there is no real need. You use a text editor to create your >> data. Then use existing file-based sort. > > What you mean is, *you* see no need for sorting interactively, or sorting as > part of a pipeline of functions. That speaks more of your lack of imagination > and closed-mindedness than any real lack. Where did I say that? The 'existing file-based sort' works from files and pipes. It's capturing the data to start with that it would be ludicrous to do interactively using such a crude interface. > Personally, if I have a few (say, a dozen or so) lines I want sorted, it is > much easier to: > > - select text from whatever application I'm using; > > - bring up a terminal (I almost always have one already open); > > - type `sort ENTER`; > > - middle-click to paste the lines; > > - type Ctrl-D; > > - select the output lines; > > - paste back into the original application; That's not entering the data interactively (such as typing 'sort' then it sits silently recording lines of text (you hope) until it sees EOF). You say the lines already exist in a text editor. Exactly what I said; you start with text that has already been entered or generated by other means. However it does seem to expose a flaw in the ability of command line tools to work with non-command line tools. So I have to copy 33,000 lines from a document, get to the terminal (keeping that document open because I'm not done with it), start 'sort', and paste those 33,000 lines into the console, then type Ctrl-D. Which then promptly prints the newly sorted 33,000 lines onto the console. Then you have to select those 33,000 lines (not easy as you now have find the midpoint of 66,000 lines of output), copy it, and paste it back into the app. Put that way, it doesn't sound very sophisticated does it? (Now I expect you're going to bombard with workarounds to fix all those issues.) > (seven steps) > > > versus: That's not what I meant at all, which is about creating the data from scratch. Here, the data STILL already magically exists somewhere! Is it possible that you guys have never had to create anything original? (How would /I/ sort a block of text in my editor? There's no built-in function, so using your approach: * Select the block * Ctrl KWfileEnter to write it to 'file' * Elsewhere, sort file2 * Back in editor, Ctrl KY Ctrl KRfile2Enter to delete then insert file2 Only four steps? Something went amiss in your analysis I think. And even these could be simplified by Using Ctrl KS and Ctrl KL to use a predefined clipboard-like filename for the intermediate. However, how hard would it for the editor to do its own sorting? Not very, as it turns out, only half a dozen lines of code (see below after sig). Now the sequence is: * Select the block * Ctrl KH How many steps did you have, seven?) > I forget... it is "work harder, not smarter"? Being inefficient is a good > thing, right? Okay.... -- bartc -------------------------------------------------- proc cmd_ctrlkh <"med:*ck*ch "> (td) = if td.blockstart then isort(td.text[td.blockstart..td.blockend]) pageupdate(td) fi end From bc at freeuk.com Sat Oct 7 08:34:30 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 13:34:30 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d832d4$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: <9R3CB.1032610$JP7.710125@fx09.am4> On 07/10/2017 03:18, Chris Angelico wrote: > On Sat, Oct 7, 2017 at 12:50 PM, Steve D'Aprano > wrote: >> On Sat, 7 Oct 2017 06:21 am, Chris Angelico wrote: >> >>> I'm not sure what printing to a window or image would mean, or how >>> it's useful, but sure. >> >> Print to window: Print Preview. >> >> Print to image: export to pdf or jpg or png. > > Except that that isn't what you get in his programming language. What > you seem to get (judging from his response to the post you're quoting) > is something that lets you treat a GUI window as if it were a console, > printing text to it in some sort of "glass teletype" way. Leaves me > wondering if he's ever taken one of his apps to a system that uses a > different default font, or anything like that. Unless all his GUIs are > built in HTML? (That particular language was one I was using over 20 years ago and it was an integrated part of a CAD application (and it started off pre-Windows). The app also supported printer/plotter/image destinations including PostScript. The same language could take a marked-up text file, convert it to a series of drawing elements, then render and print that page on a PS printer. Then the same with the next page. I think I did a 350-page manual like that. With pictures. My current language doesn't link its 'print' to graphic displays or images.) -- bartc From bc at freeuk.com Sat Oct 7 08:54:36 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 13:54:36 +0100 Subject: The "loop and a half" In-Reply-To: <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: <_74CB.1635456$tu4.672402@fx35.am4> On 07/10/2017 09:35, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 11:05 am, bartc wrote: >> Um, that actually follows what interactive Python does. > > > What is "that" to which you refer? > > If you mean, "what I, Bart C, suggested, namely having the program exit on a > blank line", then you are wrong. In the Python interactive interpreter, you > can enter blank lines, and the interpreter doesn't exit. > There's one priviso: blank lines are only ignored at the primary prompt >>> > not the secondary prompt ... where a blank line is used to end the block. But > the interpreter doesn't exit. A slight compromise for the sake of convenience > at the interactive prompt. That's what I mean. Enter is used to end one state and get back to another. The same thing with my loop evaluating expressions. When you're done with with expressions, you might exit in the trivial example, or you might get back to a main command loop. Just like Python. > >> (Most of my real interactive programs with a command line interface >> programs use Escape, quit, exit, q or x to finish. > > And how do you distinguish between calling quit from inside the function, and > using quit inside the function? Huh? Most interactive programs aren't language interpreters like Python, where everything needs to be considered to be a line in the language syntax. > Unfortunately ESCAPE is already used. VT100 (the terminal emulation which is > used in just about all terminals) all control sequences begin with ESC. So > every time you do something like press an arrow key, the terminal sends ESC > followed by other stuff. It would be painful if every time you hit an arrow > key, the interpreter took it as "Exit". Yes, I discovered this. So my programs that use Escape on Windows needed to use Escape Escape on Linux to get around that. >> Interactive Python requires quit() or exit(), complete with parentheses. >> Unless you've redefined quit and exit as something else, then you have >> to crash out by other means.) > > "Crash out", he says. > > If your intention is to prove that you're a know-nothing ignoramus, you're > doing an excellent job of it. Yes, crash out. I've always considered control sequences as something out of the ordinary used when things go wrong. Eg: <<< while 1: <<< pass <<< Here it appears to hang, and nothing you type does anything except (on Windows) Ctrl-C. That's what I call crashing out. Although that stays within the interpreter. If I press Ctrl-Break instead, it aborts the interpreter too. What exactly is your problem, that anyone who has a different take on things must be an 'ignoramus'? And, to boot, an ignoramus who knows nothing? -- bartc From steve+python at pearwood.info Sat Oct 7 09:19:47 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 00:19:47 +1100 Subject: The "loop and a half" References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 11:06 pm, bartc wrote: > That's not entering the data interactively (such as typing 'sort' then > it sits silently recording lines of text (you hope) until it sees EOF). I manually hit paste, that's just as much a form of data entry as typing characters one at a time. And then I hit Ctrl-D, and the program responds by sorting the input. That's a manual, interactive process. My example only specified copying data out of another application because that's my usual use-case. Perhaps I'm writing an email, or using a news reader (as I'm doing now). But that part isn't critical. Perhaps I haven't typed them yet. Since there's only a few of them, a dozen or so, its trivial to type or paste them into stdin. Because I'm not a caveman, I use all the tools at my disposal, and that includes the ability to copy and paste from one application to another. Maybe I even type some lines, and paste others. Its 2017 and I've been using computers long enough to know how to copy and paste from one application to another like a Boss. You ought to see me, I drag the mouse over text like a virtuoso, and middle-click like a world-champion. The fact that stdin is a pretty feature-poor line based editor is neither in dispute nor is it a problem. If I make a mistake, its no big deal. I just re-type the line, sort the lot including the erroneous line, then edit out the bad line after I've pasted it into the email. YOU, not me, is the one talking in absolutes: according to you, nobody could possibly need anything less than a full-featured user-friendly application with prompts and all the bells and whistles in order to sort even half a dozen lines of text. You make absolute statements about how we ought to work: "You use a text editor to create your data. Then use existing file-based sort." No we don't. Not always. Sometimes we might, but its not compulsory and sometimes the task is small enough that having sort read from stdin is all we need. There's no need for saving text into a file. I've given you a real-world actual use-case where your solution is inconvenient, complicated and difficult. This is a case where I don't need all the bells and whistles you claim are indispensable. I'm not saying there's "never" a use for such an application. Of course there are cases where entering data into stdin is inconvenient and underpowered. Nobody disputes that. YOU are arguing against some imaginary, exaggerated position where we say that nobody ever, under any circumstances, needs a UI more friendly than bare-bones stdin. Nobody said that. You are projecting your own absolutism onto us. We're explaining why the sort program is designed the way it is. That doesn't mean that other designs don't have their uses too. > You say the lines already exist in a text editor. I said "whatever application I'm using", which is not necessarily a text editor. Perhaps it is my email client. Perhaps it is a graphics program, and I want to enter some sorted text. Perhaps its a web form. Or perhaps I haven't actually typed the lines yet. > Exactly what I said; > you start with text that has already been entered or generated by other > means. Unless I don't. > However it does seem to expose a flaw in the ability of command line > tools to work with non-command line tools. > > So I have to copy 33,000 lines from a document, Don't be daft. Nobody says that stdin is a sufficient interface for a heavy-weight task like that. With 33000 lines of text, I absolutely would save them to a permanent file on disk, because I wouldn't want to risk having the application crash or the power go off just as I've typed line 32999 and lose the lot. For 33000 lines, having one extra temporary file floating around is a cost worth paying. For 33 lines, it is not. You want a one-size fits all solution. Are you capable of understanding that different tasks and different scenarios are often best solved using different techniques? [...] > Put that way, it doesn't sound very sophisticated does it? Nobody said it was sophisticated. That's the whole point: having sort read from stdin as you type into the terminal is the LEAST sophisticated, simplest, most bare-bones, basic technique that works. If I want to prise open the lid of a paint can, a screwdriver will do the job. I don't need to purchase a $50 gold-plated electric paint lid opening machine that requires ten minutes to set up. Now maybe if I needed to open 33000 paint tins, it would be worth using a more heavyweight solution. But I only need to open two tins. A screwdriver is fine. > (Now I expect you're going to bombard with workarounds to fix all those > issues.) Not at all. [...] > You've got data in an email client. You've already typed it out, and then you think "I want this sorted". What do you do? You have to copy the lines into a text editor, save to disk, then launch your file-based sort program, tell it to sort the saved file, reload the saved file, copy the lines back again, then delete the now-unneeded file. If you can see some steps which are unnecessary, then do tell. > That's not what I meant at all, which is about creating the data from > scratch. Here, the data STILL already magically exists somewhere! > > Is it possible that you guys have never had to create anything original? > > (How would /I/ sort a block of text in my editor? There's no built-in > function, so using your approach: > > * Select the block > > * Ctrl KWfileEnter to write it to 'file' Oops, you've just overwritten some data in 'file' that you needed. Damn. That's three steps: Ctrl-K to enter "operate on selected text" mode, or equivalent; W to Save enter a file name > * Elsewhere, sort file2 > > * Back in editor, Ctrl KY Ctrl KRfile2Enter to delete then insert file2 Ctrl-K to enter "operate on selected text" mode; Y to Delete Ctrl-K to enter "operate on selected text" mode; R to Read from a file (at last an actual mnemonic command!) enter a file name That's five steps. And now you have two "temporary" files, file and file2, which need to be deleted. Two more steps that you conveniently don't count. And you think that memorising these non-standard keyboard shortcuts Ctrl-KW Ctrl-KY Ctrl-KR is better than memorising Ctrl-D which works across thousands of applications? Good for you. You probably would love Emacs, except other people use it, and therefore you will hate it. [...] > However, how hard would it for the editor to do its own sorting? Yes, it is a mystery to me why so few editors include a "sort lines" function. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Sat Oct 7 09:49:41 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 14:49:41 +0100 Subject: The "loop and a half" In-Reply-To: <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 14:19, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 11:06 pm, bartc wrote: > Ctrl-K to enter "operate on selected text" mode; > Y to Delete > Ctrl-K to enter "operate on selected text" mode; > R to Read from a file (at last an actual mnemonic command!) > enter a file name > > That's five steps. Are we counting steps or keystrokes? > And now you have two "temporary" files, file and file2, which need to be > deleted. Two more steps that you conveniently don't count. Actually I keep a set of 9 scratch file names just for such purposes. So overwriting and deleting don't matter. (Apparently the names are offensive to some so I won't mention them, but they're easy to type.) > And you think that memorising these non-standard keyboard shortcuts Ctrl-KW > Ctrl-KY Ctrl-KR I believe these used to be WordStar commands, if anyone can remember that. is better than memorising Ctrl-D which works across thousands > of applications? Good for you. You probably would love Emacs, except other > people use it, and therefore you will hate it. > > > [...] >> However, how hard would it for the editor to do its own sorting? > > Yes, it is a mystery to me why so few editors include a "sort lines" function. I don't know if you're being sarcastic here or not, so I don't know if you mean few editors have 'sort' or most of them do. Neither do I get the point you're making. But it's a little ironic that it took less time to add such a feature than I spent writing about it in a post! -- bart From tjol at tjol.eu Sat Oct 7 09:53:29 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 7 Oct 2017 15:53:29 +0200 Subject: callable values In-Reply-To: <59d818c1$0$14941$b1db1813$d948b532@news.astraweb.com> References: <1507293873l.25886794l.0l@psu.edu> <59d818c1$0$14941$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/17 01:58, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 03:25 am, Stefan Ram wrote: > >> FWIW, in my course notes, I have coined a special word for >> this: >> >> A /prelate/ (German: "Pr?lat") is a callable value (object). > > In English, prelate is a kind of priest, a senior clergyman and dignitary. ... which is, of course, exactly what ?Pr?lat? means. > > I don't know whether German makes a practice of taking arbitrary verbs and > adjectives and turning them into nouns, but English does, and so a callable > object is just called a "callable". > > > From saxri89 at gmail.com Sat Oct 7 10:13:13 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 7 Oct 2017 07:13:13 -0700 (PDT) Subject: python multiprocessing Message-ID: I have three functions in the python that each one puts an image (image path) as input and makes a simple image processing and creates a new image (image path) as output. in the example below, one function depends on the other, ie: the function of alg2 takes as input the image that generates the function of alg and the function of alg3 assign as input the image that generates the function of alg2 which depends on the function of alg1. (I hope you do not mind basically) because of their relatively high execution time (image processing is that) I would like to ask if I can to parallelize them using python multiprocessing. I have read about multiprocessing map and pool but I was pretty confused . whenever I summarize I have three interdependent functions and I would like to run them together if done. I would also like to know how I would perform these three functions in a contemporary way if they were not interdependent, ie each was autonomous. def alg1(input_path_image,output_path_image): start = timeit.default_timer() ###processing###) stop = timeit.default_timer() print stop - start return output_path_image def alg1(output_path_image,output_path_image1): start = timeit.default_timer() ###processing### stop = timeit.default_timer() print stop - start return output_path_image1 def alg3(output_path_image1,output_path_image2): start = timeit.default_timer() ###processing### stop = timeit.default_timer() print stop - start return output_path_image2 if __name__ == '__main__': alg1(output_path_image,output_path_image) alg2(output_path_image1,output_path_image1) alg3(output_path_image2,output_path_image2) From bc at freeuk.com Sat Oct 7 10:15:32 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 15:15:32 +0100 Subject: The "loop and a half" In-Reply-To: <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 14:19, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 11:06 pm, bartc wrote: >> So I have to copy 33,000 lines from a document, > > Don't be daft. Nobody says that stdin is a sufficient interface for a > heavy-weight task like that. With 33000 lines of text, I absolutely would > save them to a permanent file on disk, because I wouldn't want to risk having > the application crash or the power go off just as I've typed line 32999 and > lose the lot. You're missing something I think. The 33K lines already exist elsewhere, and have been copied to the clipboard. So you won't lose anything if the power goes off, assuming the original are secure. > For 33000 lines, having one extra temporary file floating around is a cost > worth paying. For 33 lines, it is not. I was demonstrating the crudeness of copying and pasting to a console window rather than between proper applications. That was easier to show with a larger example which highlighted some of the practical aspects. > You want a one-size fits all solution. Are you capable of understanding that > different tasks and different scenarios are often best solved using different > techniques? > > > [...] >> Put that way, it doesn't sound very sophisticated does it? > > Nobody said it was sophisticated. That's the whole point: having sort read > from stdin as you type into the terminal is the LEAST sophisticated, > simplest, most bare-bones, basic technique that works. You do remember this was about using programs /like/ sort as a model for writing true interactive scrolling text apps? I said it was a poor model because sort is normally used with files and pipes. You're trying to keep it as a viable model because, sometimes, sort can also be used with pasted text which sort reads as as though it had been typed in real time (not from a pipe or file anyway). OK, I'm not suggesting doing away with that. But it is still a bad model of text application to use elsewhere, even if extra prompts were added. For one thing, it only reads one block of data, then it stops. A more typical text application will be doing different things requiring different entry modes. -- bartc -- bartc From steve+python at pearwood.info Sat Oct 7 10:40:36 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 01:40:36 +1100 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> Message-ID: <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> On Sat, 7 Oct 2017 11:54 pm, bartc wrote: > So my programs that use Escape on Windows needed > to use Escape Escape on Linux to get around that. Or you could just follow the expected Unix interface instead of inventing your own. Back in the days when I used a Mac (long before OS X), I used to hate it when Windows developers would port their software to Mac. With the exception of a few big software companies like Microsoft, who had actual Mac teams, they would do what you do: completely ignore the Apple UI guidelines and port their unfamiliar and arbitrary user interfaces to the Mac software, making it essentially unusable. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From grant.b.edwards at gmail.com Sat Oct 7 10:45:23 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 7 Oct 2017 14:45:23 +0000 (UTC) Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-07, bartc wrote: > Interactive Python requires quit() or exit(), complete with parentheses. Nonsense. On Unix you can just press ctrl-D (or whatever you have configured as eof) at the command prompt. On windows, it's Ctrl-Z . > Unless you've redefined quit and exit as something else, then you have > to crash out by other means.) Admit it, you're just trolling. Plonk. From grant.b.edwards at gmail.com Sat Oct 7 10:57:24 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 7 Oct 2017 14:57:24 +0000 (UTC) Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> <87fuaww3xs.fsf@elektro.pacujo.net> <59d81d85$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-07, Steve D'Aprano wrote: > They're probably to busy re-doing working programs from scratch every > few versions, with a brand new "improved" UI (almost invariably > including a kool new design that uses light grey text on an ever so > slightly lighter grey background) Yes! That! > and deleting any useful functionality that the lead developer > personally doesn't use, because "nobody uses that". > > https://www.jwz.org/doc/cadt.html >From the above: "Why not be honest and resign yourself to the fact that version 0.8 is followed by version 0.8, which is then followed by version 0.8?" I think he's being overly optimistic. If you want to be honest about a lot of open-source GUI stuff, version 0.8 should generally be followed by 0.7. It seems that the state of affairs in non-GUI open source programs is far, far better -- maintainers often actually care whether the program works correclty and does useful things. -- Grant From eryksun at gmail.com Sat Oct 7 11:04:18 2017 From: eryksun at gmail.com (eryk sun) Date: Sat, 7 Oct 2017 16:04:18 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 7, 2017 at 1:06 PM, bartc wrote: > > So I have to copy 33,000 lines from a document, get to the terminal (keeping > that document open because I'm not done with it), start 'sort', and paste > those 33,000 lines into the console, then type Ctrl-D. Which then promptly > prints the newly sorted 33,000 lines onto the console. Then you have to > select those 33,000 lines (not easy as you now have find the midpoint of > 66,000 lines of output), copy it, and paste it back into the app. Just redirect sort's output to the clipboard. In X11 a common program for this is xclip, e.g. `sort | xclip`, which defaults to the "primary" clipboard but can also use the "secondary" and "clipboard" clipboards. In Windows it's clip.exe, e.g. `sort /u /c | clip`, where /u and /c are undocumented sort switches to use UTF-16 output and a case-sensitive sort. That said, I wouldn't use sort.exe to read input from the Windows console since it uses the legacy codepage. Instead I'd use the new Linux subsystem in Windows 10, which as of the Creators update supports piping between Linux and Windows programs and reads Unicode from the console. Run `bash` and then `sort | clip.exe`. Finish with Ctrl+D, and Bob's your uncle. Using MSYS bash.exe and sort.exe (e.g. from Git for Windows) is another option. From bc at freeuk.com Sat Oct 7 11:06:40 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 16:06:40 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 15:45, Grant Edwards wrote: > On 2017-10-07, bartc wrote: > >> Interactive Python requires quit() or exit(), complete with parentheses. > > Nonsense. On Unix you can just press ctrl-D (or whatever you have > configured as eof) at the command prompt. On windows, it's Ctrl-Z > . Steve spoke about the 'usual quit/exit/bye' commands. If you type 'quit' in interactive Python, then it says: Use quit() or Ctrl-Z plus Return to exit Same for exit. So in Python, IF you want to use quit or exit to terminate, you have to use quit() or exit() instead. So please explain how what I wrote was nonsense. >> Unless you've redefined quit and exit as something else, then you have >> to crash out by other means.) > > Admit it, you're just trolling. FFS, NOW what's wrong? IF you DO redefine those names, then you DO have to use other means to terminate. I happen to call those means 'crashing out', because it's like yanking the plug rather than using the on/off switch. Especially on Windows where the usual Ctrl C doesn't work, so you resort to Ctrl-Break will which actually abort it. Ctrl Z is uncommon. I suspect it's you trying to deliberately wind ME up. I'm getting fed up with this thread now. From steve+python at pearwood.info Sat Oct 7 11:19:51 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 02:19:51 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d8f099$0$14929$b1db1813$d948b532@news.astraweb.com> On Sun, 8 Oct 2017 12:49 am, bartc wrote: > On 07/10/2017 14:19, Steve D'Aprano wrote: >> On Sat, 7 Oct 2017 11:06 pm, bartc wrote: > >> Ctrl-K to enter "operate on selected text" mode; >> Y to Delete >> Ctrl-K to enter "operate on selected text" mode; >> R to Read from a file (at last an actual mnemonic command!) >> enter a file name >> >> That's five steps. > > Are we counting steps or keystrokes? Steps. If I was counting keystrokes, entering the filename "file" would have counted as five. It didn't. >> And now you have two "temporary" files, file and file2, which need to be >> deleted. Two more steps that you conveniently don't count. > > Actually I keep a set of 9 scratch file names just for such purposes. So that's yet another thing for you to memorise rather than Ctrl-D for EOF. Bart, its cool that you have your own personal, idiosyncratic way of working that suits you. (I'm not being sarcastic.) If it works for you, great. Nobody says you have to change your own personal workflow. You've got a bunch of personal habits and customised tools that do just what you want them to do and how YOU want them to do, and that's great for you. We accept that. Please have the decency to return the favour. Just because other people follow a different workflow and use tools that make other choices than yours, doesn't make them wrong, and it doesn't make those other tools substandard. You're old enough and supposedly mature enough that you ought to know that before you criticise a technical solution, you should at least try to understand what problem it is solving before deciding it is wrong. [...] >> And you think that memorising these non-standard keyboard shortcuts Ctrl-KW >> Ctrl-KY Ctrl-KR > > I believe these used to be WordStar commands, if anyone can remember that. Never used it, but I remember when WordPerfect for Mac came with a keyboard overlay with WordStar commands on it. But WordStar or not, you still have to memorise a bunch of commands to get a task done, commands which don't work anywhere else. You're happy to do it, which is great. Vim and Emacs users make the same choice, to memorise many special, idiosyncratic ways of working in a text editor which don't apply anywhere else. We don't judge: some people love Vim, some love Emacs, some prefer lightweight GUI editors, some use heavy-duty IDEs. >> [...] >>> However, how hard would it for the editor to do its own sorting? >> >> Yes, it is a mystery to me why so few editors include a "sort lines" >> function. > > I don't know if you're being sarcastic here or not, so I don't know if > you mean few editors have 'sort' or most of them do. Neither do I get > the point you're making. I'm not being sarcastic, and I'm agreeing with you. Sorting a few lines is not a hard task, and yet not one of my GUI editors have it as either a built-in command or even a default plug-in. (There may be third-party plugins I don't know of.) I daresay that Emacs, at least, will have a sort command, but as they say about Emacs, it would be the perfect operating system if only its standard text editor wasn't so bad. (Hey, just because I respect the right of people to choose Emacs, doesn't mean I can't make fun of its size and bloatedness.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Oct 7 12:00:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Oct 2017 03:00:48 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 2:06 AM, bartc wrote: > On 07/10/2017 15:45, Grant Edwards wrote: >> Admit it, you're just trolling. > > FFS, NOW what's wrong? > > IF you DO redefine those names, then you DO have to use other means to > terminate. I happen to call those means 'crashing out', because it's like > yanking the plug rather than using the on/off switch. Especially on Windows > where the usual Ctrl C doesn't work, so you resort to Ctrl-Break will which > actually abort it. Ctrl Z is uncommon. > > I suspect it's you trying to deliberately wind ME up. > > I'm getting fed up with this thread now. It's when you call that "crashing out" that people think you're just trolling. Why on earth would that be a crash? Don't you know of any other way to perform an orderly shutdown in a Python script? You talk about Ctrl-C/Ctrl-Break as if it were an orderly shutdown, yet ignore all of the other methods and call them "crashing". Is there any surprise that people call you out for that? ChrisA From torriem at gmail.com Sat Oct 7 12:08:33 2017 From: torriem at gmail.com (Michael Torrie) Date: Sat, 7 Oct 2017 10:08:33 -0600 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: <2388ab67-3968-01dd-6136-823576f55500@gmail.com> On 10/06/2017 07:24 AM, bartc wrote: > On 06/10/2017 14:11, Peter J. Holzer wrote: >> I regularly use at least cat, wc and od this way (plus a few of my own >> utilities like utf8dump). I'm sure I've used sort this way, too, though >> rather rarely. I usually don't type the input but paste it in, > > Exactly. Probably no one ever uses these programs with actual live input > with the possibility of uncorrectable errors getting through. I regularly use cat in an interactive way. Most Unix users do. cat >somefile.txt sometext ^D It's a very fast way of moving stuff from the clipboard to a file without having to fire up an editor. I'm not sure a prompt would benefit very much here, since this is such a common operation once you learn it. In MS-DOS to create a file you do "copy con: file" and then you also get a blinking cursor with no prompt. Yet this was just something we did with MS-DOS. Not sure a prompt would have gained much. From steve+python at pearwood.info Sat Oct 7 12:28:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 03:28:06 +1100 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d90099$0$14942$b1db1813$d948b532@news.astraweb.com> On Sun, 8 Oct 2017 01:15 am, bartc wrote: > On 07/10/2017 14:19, Steve D'Aprano wrote: >> On Sat, 7 Oct 2017 11:06 pm, bartc wrote: > >>> So I have to copy 33,000 lines from a document, >> >> Don't be daft. Nobody says that stdin is a sufficient interface for a >> heavy-weight task like that. With 33000 lines of text, I absolutely would >> save them to a permanent file on disk, because I wouldn't want to risk >> having the application crash or the power go off just as I've typed line >> 32999 and lose the lot. > > You're missing something I think. The 33K lines already exist elsewhere, > and have been copied to the clipboard. So you won't lose anything if the > power goes off, assuming the original are secure. If I have the 33K lines in a file already, why would I copy them and paste them into stdin (possibly overflowing the input buffer in the terminal) when I can just call sort on the file? I can even sort in place: sort -o filename filename You seem to have decided that it is our position that the one and only acceptable way to use the sort command is to launch it with no options, then laboriously type (or copy and paste) all the lines into stdin. We're not idiots. We have many options, and we use the one which is most appropriate for the task at hand. >> [...] >>> Put that way, it doesn't sound very sophisticated does it? >> >> Nobody said it was sophisticated. That's the whole point: having sort read >> from stdin as you type into the terminal is the LEAST sophisticated, >> simplest, most bare-bones, basic technique that works. > > You do remember this was about using programs /like/ sort as a model for > writing true interactive scrolling text apps? I don't remember any such thing. I remember you *claiming* that, but if anyone actually did make that claim, I haven't seen it. There are reasons why sort defaults to not providing a richer UI. Its not intended as a "Sort Stuff App". It is designed to be used in Unix command pipelines, where an introductory message would be a nuisance. sort *could* detect when it is reading from stdin interactively and give an introductory message. I already said this. The fact that it doesn't is a matter of the personal taste of the author, who presumably doesn't think it is necessary. Simplicity is itself a virtue, although not necessarily to beginners. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From prabu.ts at gmail.com Sat Oct 7 12:38:00 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Sat, 7 Oct 2017 09:38:00 -0700 (PDT) Subject: exception should not stop program. Message-ID: I would like to continue to second function invocation "checkServiceStatus('AdobeARMservice')" even if the first checkServiceStatus('Tomcat9') has any exception.Please advice.Second function invocation not getting executed if any exception occurs in first.Please advice. import psutil def checkServiceStatus(server): status = None try: service = psutil.win_service_get(server) status = service.as_dict() print("%s (%s)" % (status['name'], status['display_name'])) print("status: %s, start: %s, username: %s, pid: %s" % ( status['status'], status['start_type'], status['username'], status['pid'])) print("binpath: %s" % status['binpath']) print(" ") except Exception: pass return status checkServiceStatus('Tomcat9') checkServiceStatus('AdobeARMservice') From prabu.ts at gmail.com Sat Oct 7 12:41:32 2017 From: prabu.ts at gmail.com (Prabu T.S.) Date: Sat, 7 Oct 2017 09:41:32 -0700 (PDT) Subject: exception should not stop program. In-Reply-To: References: Message-ID: <320bcb66-28a9-4a5c-9027-b304de0ec348@googlegroups.com> On Saturday, October 7, 2017 at 12:38:11 PM UTC-4, Prabu T.S. wrote: > I would like to continue to second function invocation "checkServiceStatus('AdobeARMservice')" even if the first > checkServiceStatus('Tomcat9') has any exception.Please advice.Second function invocation not getting executed if any exception occurs in first.Please advice. > > > import psutil > > def checkServiceStatus(server): > status = None > try: > service = psutil.win_service_get(server) > status = service.as_dict() > print("%s (%s)" % (status['name'], status['display_name'])) > print("status: %s, start: %s, username: %s, pid: %s" % ( > status['status'], status['start_type'], status['username'], status['pid'])) > print("binpath: %s" % status['binpath']) > print(" ") > except Exception: > pass > return status > > checkServiceStatus('Tomcat9') > checkServiceStatus('AdobeARMservice') ignore this for now. From grant.b.edwards at gmail.com Sat Oct 7 12:52:11 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 7 Oct 2017 16:52:11 +0000 (UTC) Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> <59d90099$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-07, Steve D'Aprano wrote: > sort *could* detect when it is reading from stdin interactively and give an > introductory message. There are some command-line utilities that try to do that: they modify their behavior when they think that stdin or stdout is connected to a "terminal" instead of a pipe. Personally, I find that annoying and inconvenient. I hate it when I run a program, look at the output and decide I want to do some processing on it, so I hit 'ctrl-P' and pipe the output to sed, grep, awk, sort, whatever. And it doesn't work. After wasting _way_ too much time trying to figure out what I did wrong in my sed/grep/awk/whatever command, I finally realize that the @#$!% program changes its output format when stdout is a pipe, and the rest of the pipeline was designed to process the stdout-is-a-terminal output format. And don't get me started on command-line utilities that use colored output that's only usable if yor terminal is configured with a black background. Hint to gentoo portage authors: yellow-on-white text is _not_ readable. -- Grant From jlgimeno71 at gmail.com Sat Oct 7 12:55:09 2017 From: jlgimeno71 at gmail.com (Jorge Gimeno) Date: Sat, 7 Oct 2017 09:55:09 -0700 Subject: exception should not stop program. In-Reply-To: References: Message-ID: Catching all exceptions in a try-except block is almost always a bad idea. You can't tell the difference between an exception that you are looking to handle and an exception that should cause your program to crash and burn (because something is wrong). It's best to catch a specific exception. What condition are you looking to handle here? -Jorge On Sat, Oct 7, 2017 at 9:38 AM, Prabu T.S. wrote: > I would like to continue to second function invocation > "checkServiceStatus('AdobeARMservice')" even if the first > checkServiceStatus('Tomcat9') has any exception.Please advice.Second > function invocation not getting executed if any exception occurs in > first.Please advice. > > > import psutil > > def checkServiceStatus(server): > status = None > try: > service = psutil.win_service_get(server) > status = service.as_dict() > print("%s (%s)" % (status['name'], status['display_name'])) > print("status: %s, start: %s, username: %s, pid: %s" % ( > status['status'], status['start_type'], status['username'], > status['pid'])) > print("binpath: %s" % status['binpath']) > print(" ") > except Exception: > pass > return status > > checkServiceStatus('Tomcat9') > checkServiceStatus('AdobeARMservice') > > -- > https://mail.python.org/mailman/listinfo/python-list > From tjreedy at udel.edu Sat Oct 7 12:58:03 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 7 Oct 2017 12:58:03 -0400 Subject: Introducing the "for" loop In-Reply-To: <59d899bb$0$14955$b1db1813$d948b532@news.astraweb.com> References: <1507293873l.25886794l.0l@psu.edu> <59d899bb$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/7/2017 5:09 AM, Steve D'Aprano wrote: > On Fri, 6 Oct 2017 11:44 pm, ROGER GRAYDON CHRISTMAN wrote: > >> Despite the documentation, I would still be tempted to say that range is a >> function. >> Taking duck-typing to the meta-level, every time I use range, I use its name >> followed >> by a pair of parentheses enclosing one to three parameters, and I get back >> an >> immutable sequence object. It sure looks like a function to me. > > I agree -- range() is a function in the (almost) mathematical sense, something > which takes arguments and returns a value. It's also a type (class), in the > OOP sense: > > > py> type(range) > > > > The term "function" is ambiguous but normally clear from context. Often, the > differences make no difference, but when they are important, we can discuss > them: > > - range is a callable (a callable object); > > - it is also a type/class, and calling it returns an instance; > > - it looks like, and behaves like, a function; > > - and is, conceptually, a function; > > - but it is *not* an instance of FunctionType: > > > py> from types import FunctionType > py> def function(): > ... pass > ... > py> isinstance(function, FunctionType) > True > py> isinstance(range, FunctionType) > False No built-in function is an instance of FunctionType >>> isinstance(compile, FunctionType) False >>> isinstance(print, FunctionType) False >>> type(compile) >>> type(int.bit_length) FunctionType == function defined by def statement or lambda expression. These are a subset of functions defined by Python code. > It is this last sense (an instance of FunctionType) which people are thinking > of when they state that range is not a function. Unless one means 'function defined by def or class', excluding all functions defined in the interpreter implementation language, which can change as modules are recoded one way or the other, and some functions defined in Python, FunctionType is too narrow. The predecate would have to be at least BuiltinFunction = type(compile) MethonDescriptor = type(int.bit_length) isinstance(x, (FunctionType, BuiltinFunction, MethodDescriptor)) but that still excludes bound methods and partial functions. I have the impression that classes being functions is somewhat peculiar to Python but I have not been exposed to enough OOP languages to know. The other sense in which 'range is not a function' makes some sense is when it means 'range is not *just* a function'. This is akin to when 'people are not animals' means 'people are not (or should not be) *just* animals'. Do people speaking other than English say subcategory Y of category X is special by saying 'Ys are not Xs'? -- Terry Jan Reedy From bc at freeuk.com Sat Oct 7 13:01:54 2017 From: bc at freeuk.com (bartc) Date: Sat, 7 Oct 2017 18:01:54 +0100 Subject: The "loop and a half" In-Reply-To: <59d90099$0$14942$b1db1813$d948b532@news.astraweb.com> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> <59d90099$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 17:28, Steve D'Aprano wrote: > On Sun, 8 Oct 2017 01:15 am, bartc wrote: >> You do remember this was about using programs /like/ sort as a model for >> writing true interactive scrolling text apps? > > I don't remember any such thing. I remember you *claiming* that, but if anyone > actually did make that claim, I haven't seen it. bart: > This doesn't make sense. For interactive use, you wouldn't bother > testing for eof, as you'd be testing the eof status of the keyboard. > ChrisA: > You mean the way heaps and heaps of Unix programs work, processing > until EOF of stdin? Yeah, totally makes no sense, man, no sense at > all. bart: > If you're referring to the ability to redirect stdin so that input can > come from a file as well as from a live keyboard, then you're doing > file handling; it's NOT interactive. ChrisA: > How would you write a sort program? How would you tell it that you're > done entering data? From tjreedy at udel.edu Sat Oct 7 13:09:02 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 7 Oct 2017 13:09:02 -0400 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/7/2017 10:45 AM, Grant Edwards wrote: > On 2017-10-07, bartc wrote: > >> Interactive Python requires quit() or exit(), complete with parentheses. > > Nonsense. On Unix you can just press ctrl-D (or whatever you have > configured as eof) at the command prompt. On windows, it's Ctrl-Z > . IDLE's shell quits on ^D at a prompt on all systems. (This became true before I worked on IDLE.) Clicking the [X] button should close any window on all systems. -- Terry Jan Reedy From grant.b.edwards at gmail.com Sat Oct 7 13:53:03 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 7 Oct 2017 17:53:03 +0000 (UTC) Subject: exception should not stop program. References: Message-ID: On 2017-10-07, Jorge Gimeno wrote: > Catching all exceptions in a try-except block is almost always a bad > idea. Catching it and ignoring it as the OP was doing (or assuming it's some particular exception) certainly is. If you know (or suspect) that stderr isn't going anywhere that it will be seen, then catching all exceptions at the top of your program and logging them and exiting with an error status is a reasonable thing to do. #!/usr/bin/python import sys,syslog,traceback def main(): [...] try: main() except: syslog.syslog("%s: %s\n" % (sys.argv[0], traceback.format_exc())) sys.exit(1) If it's a GUI program, then popping up an error dialog instead of sending it to syslog might make more sense -- if you can be reasonably sure that the GUI framework is still operational. -- Grant Edwards grant.b.edwards Yow! Am I accompanied by a at PARENT or GUARDIAN? gmail.com From python at mrabarnett.plus.com Sat Oct 7 14:44:36 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 7 Oct 2017 19:44:36 +0100 Subject: exception should not stop program. In-Reply-To: References: Message-ID: On 2017-10-07 17:38, Prabu T.S. wrote: > I would like to continue to second function invocation "checkServiceStatus('AdobeARMservice')" even if the first > checkServiceStatus('Tomcat9') has any exception.Please advice.Second function invocation not getting executed if any exception occurs in first.Please advice. > > > import psutil > > def checkServiceStatus(server): > status = None > try: > service = psutil.win_service_get(server) > status = service.as_dict() > print("%s (%s)" % (status['name'], status['display_name'])) > print("status: %s, start: %s, username: %s, pid: %s" % ( > status['status'], status['start_type'], status['username'], status['pid'])) > print("binpath: %s" % status['binpath']) > print(" ") > except Exception: > pass > return status > > checkServiceStatus('Tomcat9') > checkServiceStatus('AdobeARMservice') > Catch the exception that's raised, like you're already doing in checkServiceStatus. From rosuav at gmail.com Sat Oct 7 14:47:18 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Oct 2017 05:47:18 +1100 Subject: Introducing the "for" loop In-Reply-To: References: <1507293873l.25886794l.0l@psu.edu> <59d899bb$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 3:58 AM, Terry Reedy wrote: > No built-in function is an instance of FunctionType >>>> isinstance(compile, FunctionType) > False >>>> isinstance(print, FunctionType) > False >>>> type(compile) > >>>> type(int.bit_length) > > > > FunctionType == function defined by def statement or lambda expression. > These are a subset of functions defined by Python code. > > Unless one means 'function defined by def or class', excluding all functions > defined in the interpreter implementation language, which can change as > modules are recoded one way or the other, and some functions defined in > Python, FunctionType is too narrow. The predecate would have to be at least > > BuiltinFunction = type(compile) > MethonDescriptor = type(int.bit_length) > isinstance(x, (FunctionType, BuiltinFunction, MethodDescriptor)) > > but that still excludes bound methods and partial functions. The three types lack a concrete base class... but fortunately, they have an abstract one. >>> isinstance(print, collections.abc.Callable) True >>> isinstance(int.bit_length, collections.abc.Callable) True >>> isinstance(lambda: 1, collections.abc.Callable) True >>> isinstance(range, collections.abc.Callable) True Now I know my ABCs. :) ChrisA From pavol.lisy at gmail.com Sat Oct 7 16:59:44 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Sat, 7 Oct 2017 22:59:44 +0200 Subject: Creating a Dictionary In-Reply-To: References: Message-ID: On 10/5/17, Chris Angelico wrote: > On Thu, Oct 5, 2017 at 12:24 PM, Stefan Ram wrote: >> One might wish to implement a small language with these commands: >> >> F - move forward >> B - move backward >> L - larger stepsize >> S - smaller stepsize >> >> . One could start with the following pseudocode for a dictionary: >> >> { 'F': lambda: myturtle.forward( s ), >> 'B': lambda: myturtle.backward( s ), >> 'L': lambda: global s; s *= 2, >> 'S': lambda: global s; s /= 2 } >> >> . But lambda-expressions cannot contain statements. >> >> Any other suggestions? > > There are a few options here. One is to make use of a simple > "collector decorator": > > commands = {} > def cmd(f): > commands[f.__name__.upper()] = f > return f > > @cmd > def f(): > myturtle.forward( s ) > > @cmd > def b(): > myturtle.backward( s ) > > @cmd > def l(): > global siz > size *= 2 > > @cmd > def s(): > global siz > size //= 2 > > (Also untested, but the pattern is one I've used many times.) > > Another is to deploy the whole thing as a class, with no globals: > > class Turtle: > def __init__(self): > self.size = ... > self.turtle = ... > def cmd_F(self): > self.turtle.forward(self.size) > def cmd_B(self): > self.turtle.backward(self.size) > def cmd_L(self): > self.size *= 2 > def cmd_S(self): > self.size //= 2 > > Note that I've added "cmd_" in front of the names, which means you can > safely getattr(turtle, "cmd_" + letter) and be confident you won't > accidentally grab something that isn't a command. > > (Note also that I used // for division. If your size is actually a > float, then change those back to single slashes.) > > Either of these patterns would work fairly well. There are others, > too, but I'd be inclined to use one of these. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list size = 2*size if size else 1 From pavol.lisy at gmail.com Sat Oct 7 17:37:34 2017 From: pavol.lisy at gmail.com (Pavol Lisy) Date: Sat, 7 Oct 2017 23:37:34 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/5/17, Steve D'Aprano wrote: > The A and E in the word "are" are not vowels, since they are silent. Interesting! :) Is then R (half?) silent in word "Brazil"? From gengyangcai at gmail.com Sat Oct 7 19:42:10 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 7 Oct 2017 16:42:10 -0700 (PDT) Subject: Finding Old Posts on Python Message-ID: <5a7cf063-0a4f-469d-80b4-ff379a598017@googlegroups.com> Hello, Does anyone know of a way to find all my old posts about Python ? Thanks a lot! GengYang From Richard at Damon-Family.org Sat Oct 7 19:55:35 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Sat, 7 Oct 2017 19:55:35 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59d5a568$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/4/17 11:22 PM, Steve D'Aprano wrote: > The A and E in the word "are" are not vowels, since they are silent. The A is clearly not silent, unless you have some strange pronunciation. The fact that are is pronounced just like the NAME of the letter R doesn't mean it is silent. Compare the sound of the word are (which begins like the 'pirate' phrase arrgh) to me, to a word that begins with a letter R like rat or ring. There is a clear aspiration in the former that isn't in the latter. From saxri89 at gmail.com Sat Oct 7 20:40:53 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 7 Oct 2017 17:40:53 -0700 (PDT) Subject: django project avoid reload page where work algorithm Message-ID: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> I have a website where the user can be put three numbers on my html template and get some results from my personal mathematical algorithm. the result save at user personal table on my database and can see in specific tab in my website. my problem is where the algorithm to calculate result maybe take time between 5-10 minutes in this time the browser stay on reload. if user change tab or close browser or maybe have problem with internet connection then loose that request and need again to put the numbers and again wait for results. I want after the user request from my form in html to keep this request and work my algorithm without reload the page and where the algorithm finish then to send the user some email or message or just need the user visit the tab of results to see new results. that I want to avoid the problems where the algorithm is in running and user loose data or request or time. is easy to do that using suproccess,celery or RabbitMQ ? any idea ? here the code views.py def math_alg(request): if request.method == "POST": test = request.POST.get('no1') test = request.POST.get('no3') test = request.POST.get('no3') #start algorith calc_math(no1,no1,no3,result) instance = rmodel.objects.create(user=request.user,rfield=result) instance.save return render(request, 'page.html', {'result': result}) html :
{% csrf_token %} op math calculate:

{{result }}
From python at mrabarnett.plus.com Sat Oct 7 21:56:28 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 8 Oct 2017 02:56:28 +0100 Subject: django project avoid reload page where work algorithm In-Reply-To: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> Message-ID: <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> On 2017-10-08 01:40, Xristos Xristoou wrote: > I have a website where the user can be put three numbers on my html template and get some results from my personal mathematical algorithm. the result save at user personal table on my database and can see in specific tab in my website. > my problem is where the algorithm to calculate result maybe take time between 5-10 minutes in this time the browser stay on reload. if user change tab or close browser or maybe have problem with internet connection then loose that request and need again to put the numbers and again wait for results. > I want after the user request from my form in html to keep this request and work my algorithm without reload the page and where the algorithm finish then to send the user some email or message or just need the user visit the tab of results to see new results. > that I want to avoid the problems where the algorithm is in running and user loose data or request or time. > is easy to do that using suproccess,celery or RabbitMQ ? > any idea ? [snip] You mentioned email, so why not just ask for an email address to which you will send the result? From formisc at gmail.com Sat Oct 7 23:07:45 2017 From: formisc at gmail.com (Andrew Z) Date: Sat, 7 Oct 2017 23:07:45 -0400 Subject: "comprehend" into a single value Message-ID: Hello, i wonder how can i accomplish the following as a one liner: dict= {10: ['a',1,'c'], 20: ['d',2,'f']} p = 0 for i in dict: p += dict[i][1] Thank you From steve+python at pearwood.info Sat Oct 7 23:09:22 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 14:09:22 +1100 Subject: Introducing the "for" loop References: <1507293873l.25886794l.0l@psu.edu> <59d899bb$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d996e3$0$14943$b1db1813$d948b532@news.astraweb.com> On Sun, 8 Oct 2017 03:58 am, Terry Reedy wrote: > No built-in function is an instance of FunctionType > >>> isinstance(compile, FunctionType) > False Ah yes, thanks Terry, I forgot that builtin functions have a distinct type. [...] > FunctionType == function defined by def statement or lambda expression. > These are a subset of functions defined by Python code. Indeed. >> It is this last sense (an instance of FunctionType) which people are >> thinking of when they state that range is not a function. > > Unless one means 'function defined by def or class', excluding all > functions defined in the interpreter implementation language, which can > change as modules are recoded one way or the other, and some functions > defined in Python, FunctionType is too narrow. Yes, I forgot about the other ways functions can be created, such as partial objects and builtins and callable instances. (C++ calls them functors.) However, people often distinguish between functions and methods, since methods carry their own implicit state ("self") and traditionally functions don't. But of course the line between them is blurred: - closures (a kind of function) do carry implicit state; - static methods in Python (a kind of method) don't carry implicit state. [...] > I have the impression that classes being functions is somewhat peculiar > to Python but I have not been exposed to enough OOP languages to know. Many languages use function call syntax MyClass(args) (or something quite close to function call syntax) for instantiating a class. In the sense that a function is defined by the use of function call syntax, then classes are functions. But in the sense that in some languages functions are values but classes are not (e.g. Java), then classes should be considered distinct from functions. As always, it depends on what you mean by function. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nhilterbrand at gmail.com Sat Oct 7 23:17:08 2017 From: nhilterbrand at gmail.com (Nathan Hilterbrand) Date: Sat, 7 Oct 2017 23:17:08 -0400 Subject: "comprehend" into a single value In-Reply-To: References: Message-ID: dict= {10: ['a',1,'c'], 20: ['d',2,'f']} p = sum([dict[i][1] for i in dict]) Something like that? On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote: > Hello, > i wonder how can i accomplish the following as a one liner: > > dict= {10: ['a',1,'c'], 20: ['d',2,'f']} > p = 0 > for i in dict: > p += dict[i][1] > > > Thank you > -- > https://mail.python.org/mailman/listinfo/python-list > From bgailer at gmail.com Sat Oct 7 23:30:15 2017 From: bgailer at gmail.com (bob gailer) Date: Sat, 7 Oct 2017 23:30:15 -0400 Subject: "comprehend" into a single value In-Reply-To: References: Message-ID: <62312c4b-86c6-576f-fb16-79f7df0a7c2d@gmail.com> On 10/7/2017 11:17 PM, Nathan Hilterbrand wrote: > dict= {10: ['a',1,'c'], 20: ['d',2,'f']} > p = sum([dict[i][1] for i in dict]) > > Something like that? Ah, but that's 2 lines. sum(val[1] for val in? {10: ['a',1,'c'], 20: ['d',2,'f']}.values()) > On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote: > >> Hello, >> i wonder how can i accomplish the following as a one liner: >> >> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >> p = 0 >> for i in dict: >> p += dict[i][1] >> >> >> Thank you >> -- >> https://mail.python.org/mailman/listinfo/python-list >> -- Image and video hosting by TinyPic From formisc at gmail.com Sun Oct 8 00:37:34 2017 From: formisc at gmail.com (Andrew Z) Date: Sun, 8 Oct 2017 00:37:34 -0400 Subject: "comprehend" into a single value In-Reply-To: <62312c4b-86c6-576f-fb16-79f7df0a7c2d@gmail.com> References: <62312c4b-86c6-576f-fb16-79f7df0a7c2d@gmail.com> Message-ID: Nathan, Bob - on the money. Thank you ! On Sat, Oct 7, 2017 at 11:30 PM, bob gailer wrote: > On 10/7/2017 11:17 PM, Nathan Hilterbrand wrote: > >> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >> p = sum([dict[i][1] for i in dict]) >> >> Something like that? >> > Ah, but that's 2 lines. > > sum(val[1] for val in {10: ['a',1,'c'], 20: ['d',2,'f']}.values()) > > On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote: >> >> Hello, >>> i wonder how can i accomplish the following as a one liner: >>> >>> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >>> p = 0 >>> for i in dict: >>> p += dict[i][1] >>> >>> >>> Thank you >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >>> > -- > Image and video hosting by TinyPic > From formisc at gmail.com Sun Oct 8 01:00:38 2017 From: formisc at gmail.com (Andrew Z) Date: Sun, 8 Oct 2017 01:00:38 -0400 Subject: "comprehend" into a single value In-Reply-To: References: <62312c4b-86c6-576f-fb16-79f7df0a7c2d@gmail.com> Message-ID: and how about adding "IF" into the mix ? as in : a=0 dict= {10: ['a',1,'c'], 20: ['d',2,'f']} for i in dict: p+= 10 if dict[i][1] in [1,2,3,4,5] else 0 can i "squish" "for" and "if" together ? or will it be too perl-ish ? On Sun, Oct 8, 2017 at 12:37 AM, Andrew Z wrote: > Nathan, Bob - on the money. Thank you ! > > On Sat, Oct 7, 2017 at 11:30 PM, bob gailer wrote: > >> On 10/7/2017 11:17 PM, Nathan Hilterbrand wrote: >> >>> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >>> p = sum([dict[i][1] for i in dict]) >>> >>> Something like that? >>> >> Ah, but that's 2 lines. >> >> sum(val[1] for val in {10: ['a',1,'c'], 20: ['d',2,'f']}.values()) >> >> On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote: >>> >>> Hello, >>>> i wonder how can i accomplish the following as a one liner: >>>> >>>> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >>>> p = 0 >>>> for i in dict: >>>> p += dict[i][1] >>>> >>>> >>>> Thank you >>>> -- >>>> https://mail.python.org/mailman/listinfo/python-list >>>> >>>> >> -- >> Image and video hosting by TinyPic >> > > From breamoreboy at gmail.com Sun Oct 8 03:06:23 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 8 Oct 2017 00:06:23 -0700 (PDT) Subject: Finding Old Posts on Python In-Reply-To: <5a7cf063-0a4f-469d-80b4-ff379a598017@googlegroups.com> References: <5a7cf063-0a4f-469d-80b4-ff379a598017@googlegroups.com> Message-ID: On Sunday, October 8, 2017 at 12:42:19 AM UTC+1, Cai Gengyang wrote: > Hello, > > Does anyone know of a way to find all my old posts about Python ? Thanks a lot! > > GengYang Make a site specific search for your name here https://mail.python.org/pipermail/python-list/ -- Kindest regards. Mark Lawrence. From steve+python at pearwood.info Sun Oct 8 05:12:26 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 08 Oct 2017 20:12:26 +1100 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> On Sun, 8 Oct 2017 02:06 am, bartc wrote: >> On 2017-10-07, bartc wrote: >> >>> Interactive Python requires quit() or exit(), complete with parentheses. .......................^^^^^^^^^^^^^^^^^^^^^^^^^ >> Nonsense. On Unix you can just press ctrl-D (or whatever you have >> configured as eof) at the command prompt. On windows, it's Ctrl-Z >> . > > Steve spoke about the 'usual quit/exit/bye' commands. As well as Ctrl-D, EOF, which is a standard way to exit most Unix programs. (Or as close as anything in Unix comes to a standard UI.) > If you type 'quit' in interactive Python, then it says: > > Use quit() or Ctrl-Z plus Return to exit > > Same for exit. So in Python, IF you want to use quit or exit to > terminate, you have to use quit() or exit() instead. > > So please explain how what I wrote was nonsense. Do you still believe that quit/exit is REQUIRED? To be pedantic, we have at least seven ways: - EOF (Ctrl-D on Unix, Ctrl-Z ENTER on Windows); - calling quit() or exit(); - raise SystemExit; - call sys.exit() - call os._exit(); - call os.abort(); - write your own custom quitter object that exits on simply being printed, e.g.: class Bye: def __repr__(self): sys.exit(0) bye = Bye() Four of these are intended for programmatic use, but they work interactively as well. At least two of them shouldn't be used unless you know what you are doing (os._exit and os.abort). >>> Unless you've redefined quit and exit as something else, then you have >>> to crash out by other means.) >> >> Admit it, you're just trolling. > > FFS, NOW what's wrong? "Crash out". > IF you DO redefine those names, then you DO have to use other means to > terminate. I happen to call those means 'crashing out', because it's > like yanking the plug rather than using the on/off switch. os._exit and os.abort (especially the second) could legitimately be described as "crashing out". The others, not within a million miles of it. > Especially on > Windows where the usual Ctrl C doesn't work, so you resort to Ctrl-Break > will which actually abort it. Ctrl Z is uncommon. Thousands of Python programmers on Windows successfully learned to use Ctrl-Z ENTER back in the days of Python 1.5, before quit/exit were added as a convenience for beginners, and many of them probably still use it. Yet again you assume that because YOU don't do something, nobody else in the world could possibly do it. > I'm getting fed up with this thread now. This thread would be a lot less frustrating if you would enter into it with a spirit of open-minded enquiry rather than an overbearing sense of superiority that anything others do that you don't must be worthless. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From j.wilde at hotmail.co.uk Sun Oct 8 05:24:59 2017 From: j.wilde at hotmail.co.uk (Joe Wilde) Date: Sun, 8 Oct 2017 09:24:59 +0000 Subject: IDLE help. Message-ID: Hello, I am having trouble getting IDLE (Python 3.6 - 64-bit) to open for Windows 10. When I try and run IDLE, nothing happens. It works fine for Python 2.7, but won't open for Python 3.6. I have tried repairing the install but to no avail. I'm fairly new at this kind of thing, so I don't have a lot of technical knowledge. Thanks in advance, Joe. From saxri89 at gmail.com Sun Oct 8 06:27:23 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 8 Oct 2017 03:27:23 -0700 (PDT) Subject: django project avoid reload page where work algorithm In-Reply-To: References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> Message-ID: <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> > You mentioned email, so why not just ask for an email address to which > you will send the result? i have user email from register i need to do all automate,how to use celery for this ? From bc at freeuk.com Sun Oct 8 06:36:15 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 11:36:15 +0100 Subject: The "loop and a half" In-Reply-To: <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 08/10/2017 10:12, Steve D'Aprano wrote: > On Sun, 8 Oct 2017 02:06 am, bartc wrote: > Thousands of Python programmers on Windows successfully learned to use Ctrl-Z > ENTER back in the days of Python 1.5, before quit/exit were added as a > convenience for beginners, and many of them probably still use it. Actually I NEVER normally use Python in interactive mode. Only to test how it works interactively. When I use Python, it's from my IDE. >> I'm getting fed up with this thread now. > > This thread would be a lot less frustrating if you would enter into it with a > spirit of open-minded enquiry rather than an overbearing sense of superiority > that anything others do that you don't must be worthless. Frustrating for whom? It seems to me that it's pretty much everyone here who has an overbearing sense of superiority in that everything that Unix or Linux does is a million times better than anything else. Even with things like building applications (eg. trying to build CPython from sources), they are designed from the ground up to be inextricably linked to Linux scripts, utilities, makefiles, installation schemes, or designed to work with the Linux-centric gcc C compiler. Then when they don't work as well anywhere else, it's because Linux is so much better! No, it's because they were non-portably designed around Linux and therefore designed NOT to work well anywhere else. It is also slightly frustrating for me when I see how Python is developing, with layer after layer and library after library piled on to achieve some fantastically complicated solution (one of 48 different ones to choose from) to replicate some basic functionality that could have been done in 5 minutes if GvR had decided to add it right at the start. But this is a Python group and I have to restrain myself from such comments to avoid getting lynched. There is nothing wrong with Python! -- bartc From marko at pacujo.net Sun Oct 8 07:13:03 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 08 Oct 2017 14:13:03 +0300 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: <8760bpriyo.fsf@elektro.pacujo.net> bartc : > It seems to me that it's pretty much everyone here who has an > overbearing sense of superiority in that everything that Unix or Linux > does is a million times better than anything else. People's opinions don't matter here. Point is, if you are writing software for Linux, you need to do it according to (or at least understanding) the general Linux system requirements. I don't do Windows programming, but if I did, I'd try to write the programs in the way Windows programs are intended to be written. Similarly with other operating systems. If you wrote your own OS, you'd get to dictate the proper philosophy for application programming. > Even with things like building applications (eg. trying to build > CPython from sources), they are designed from the ground up to be > inextricably linked to Linux scripts, utilities, makefiles, > installation schemes, or designed to work with the Linux-centric gcc C > compiler. Then when they don't work as well anywhere else, it's > because Linux is so much better! No, it's because they were > non-portably designed around Linux and therefore designed NOT to work > well anywhere else. Ok. Clearly Python has its roots in Linux. > It is also slightly frustrating for me when I see how Python is > developing, with layer after layer and library after library piled on > to achieve some fantastically complicated solution (one of 48 > different ones to choose from) to replicate some basic functionality > that could have been done in 5 minutes if GvR had decided to add it > right at the start. Maybe. I don't know if that would count as constructive criticism, though, because you can't change your past. > But this is a Python group and I have to restrain myself from such > comments to avoid getting lynched. There is nothing wrong with Python! Once you have expressed your opinion and frustration, you don't have to repeat it at every turn. Do you have a concrete enhancement proposal? Or is Python a lost cause for you? Marko From p.f.moore at gmail.com Sun Oct 8 07:22:26 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 8 Oct 2017 12:22:26 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 8 October 2017 at 11:36, bartc wrote: > Frustrating for whom? Well, me as well as Steve, if we're counting votes for who finds your attitude frustrating... > It seems to me that it's pretty much everyone here who has an overbearing > sense of superiority in that everything that Unix or Linux does is a million > times better than anything else. As a Windows user, I would like to make it clear that your views don't in any way represent me. > Even with things like building applications (eg. trying to build CPython > from sources), they are designed from the ground up to be inextricably > linked to Linux scripts, utilities, makefiles, installation schemes, or > designed to work with the Linux-centric gcc C compiler. Then when they don't > work as well anywhere else, it's because Linux is so much better! No, it's > because they were non-portably designed around Linux and therefore designed > NOT to work well anywhere else. When developing scripts, applications, or any form of code, I use good ideas from anywhere, as I doubt that I have the monopoly on knowing the perfect way to write code. Some of those good ideas come from Unix-based systems. That's not "because Linux is so much better", it's because someone other than me had a good idea, and I acknowledge the fact. Paul From ned at nedbatchelder.com Sun Oct 8 07:40:23 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 8 Oct 2017 07:40:23 -0400 Subject: "comprehend" into a single value In-Reply-To: References: <62312c4b-86c6-576f-fb16-79f7df0a7c2d@gmail.com> Message-ID: <5932a800-a5e4-4e76-63df-43eb7c9cf302@nedbatchelder.com> On 10/8/17 1:00 AM, Andrew Z wrote: > and how about adding "IF" into the mix ? > > as in : > > a=0 > > dict= {10: ['a',1,'c'], 20: ['d',2,'f']} > > for i in dict: > > p+= 10 if dict[i][1] in [1,2,3,4,5] else 0 > > > can i "squish" "for" and "if" together ? or will it be too perl-ish ? ??? sum(10 for v in dict.values() if v[1] in [1,2,3,4,5]) or: ??? sum((10 if v[1] in [1,2,3,4,5] else 0) for v in dict.values()) (in case you actually need something other than 0.) Also, note that if your list of five values is actually much longer than five, then you want a set: ??? sum((10 if v[1] in {1,2,3,4,5} else 0) for v in dict.values()) --Ned. > > > > > On Sun, Oct 8, 2017 at 12:37 AM, Andrew Z wrote: > >> Nathan, Bob - on the money. Thank you ! >> >> On Sat, Oct 7, 2017 at 11:30 PM, bob gailer wrote: >> >>> On 10/7/2017 11:17 PM, Nathan Hilterbrand wrote: >>> >>>> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >>>> p = sum([dict[i][1] for i in dict]) >>>> >>>> Something like that? >>>> >>> Ah, but that's 2 lines. >>> >>> sum(val[1] for val in {10: ['a',1,'c'], 20: ['d',2,'f']}.values()) >>> >>> On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote: >>>> Hello, >>>>> i wonder how can i accomplish the following as a one liner: >>>>> >>>>> dict= {10: ['a',1,'c'], 20: ['d',2,'f']} >>>>> p = 0 >>>>> for i in dict: >>>>> p += dict[i][1] >>>>> >>>>> >>>>> Thank you >>>>> -- >>>>> https://mail.python.org/mailman/listinfo/python-list >>>>> >>>>> >>> -- >>> Image and video hosting by TinyPic >>> >> From rosuav at gmail.com Sun Oct 8 07:41:24 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Oct 2017 22:41:24 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 10:22 PM, Paul Moore wrote: > On 8 October 2017 at 11:36, bartc wrote: >> Even with things like building applications (eg. trying to build CPython >> from sources), they are designed from the ground up to be inextricably >> linked to Linux scripts, utilities, makefiles, installation schemes, or >> designed to work with the Linux-centric gcc C compiler. Then when they don't >> work as well anywhere else, it's because Linux is so much better! No, it's >> because they were non-portably designed around Linux and therefore designed >> NOT to work well anywhere else. > > When developing scripts, applications, or any form of code, I use good > ideas from anywhere, as I doubt that I have the monopoly on knowing > the perfect way to write code. Some of those good ideas come from > Unix-based systems. That's not "because Linux is so much better", it's > because someone other than me had a good idea, and I acknowledge the > fact. It's also worth noting that "Linux" and "Unix" are not synonymous, and that a lot of these ideas have come from BSD, or some other OS. In fact, a good few of the ideas being pooh-poohed in this thread have been around longer than Linux has. The idea that most tools should work with stdin and stdout (while sending errors to stderr) dates back *at least* to the earliest Unix, and quite possibly earlier. So they're not "because Linux is so much better" - they're "because Unix has worked this way for longer than Windows or Linux or Mac OS has been around". And by "worked", I don't just mean that this is how it is - I also mean that it *works*, it is useful, it is a viable solution to real-world problems. ChrisA From bc at freeuk.com Sun Oct 8 07:46:48 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 12:46:48 +0100 Subject: The "loop and a half" In-Reply-To: <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: On 07/10/2017 15:40, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 11:54 pm, bartc wrote: > >> So my programs that use Escape on Windows needed >> to use Escape Escape on Linux to get around that. > > > Or you could just follow the expected Unix interface instead of inventing your > own. Your job is to port an editor that people have been using for 30 years to Linux. The first thing you do is to change all the commands and shortcuts to match what is typical on Linux? So that no-one who was familiar with it as it was can actually use it? > Back in the days when I used a Mac (long before OS X), I used to hate it when > Windows developers would port their software to Mac. With the exception of a > few big software companies like Microsoft, who had actual Mac teams, they > would do what you do: completely ignore the Apple UI guidelines and port > their unfamiliar and arbitrary user interfaces to the Mac software, making it > essentially unusable. What is it with those who write OSes in that they have to tell everyone how to run the show? An OS's job is to run programs and do whatever the program requests. BTW, how did the Apple UI guidelines come about; where they copying existing practice, or did /they/ decide to come up with something new and incompatible with anything else? And if the former, then you ask the same question of Xerox or whoever. Just look at any interactive page on the web, they all work differently. People are used to it. And it allows innovation. -- bartc From rosuav at gmail.com Sun Oct 8 08:05:11 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 8 Oct 2017 23:05:11 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 10:46 PM, bartc wrote: > On 07/10/2017 15:40, Steve D'Aprano wrote: >> >> On Sat, 7 Oct 2017 11:54 pm, bartc wrote: >> >>> So my programs that use Escape on Windows needed >>> to use Escape Escape on Linux to get around that. >> >> >> >> Or you could just follow the expected Unix interface instead of inventing >> your >> own. > > > Your job is to port an editor that people have been using for 30 years to > Linux. The first thing you do is to change all the commands and shortcuts to > match what is typical on Linux? So that no-one who was familiar with it as > it was can actually use it? Is it graphical? Does it use any well-known toolkit? If so, then yeah, the first thing - and the automatic thing - is that it will respond to the host platform's keys and so on. Which is more important: for someone familiar with Program X on Windows to be able to use Program X on Linux, or for every GUI program on my computer to respond to Ctrl-Insert by copying something to the clipboard? Adobe picked the former for earlier versions of Acrobat Reader, and they suck because of it. (I don't know if they've fixed that since, as I threw them over years ago.) >> Back in the days when I used a Mac (long before OS X), I used to hate it >> when >> Windows developers would port their software to Mac. With the exception of >> a >> few big software companies like Microsoft, who had actual Mac teams, they >> would do what you do: completely ignore the Apple UI guidelines and port >> their unfamiliar and arbitrary user interfaces to the Mac software, making >> it >> essentially unusable. > > > What is it with those who write OSes in that they have to tell everyone how > to run the show? An OS's job is to run programs and do whatever the program > requests. An OS's job is to let the human interface with the computer. Same human, same interface. > BTW, how did the Apple UI guidelines come about; where they copying existing > practice, or did /they/ decide to come up with something new and > incompatible with anything else? And if the former, then you ask the same > question of Xerox or whoever. I don't know about those specifically, but a lot of what I see today in Linux is derived from the CUA guidelines [1]. They unified a bunch of disparate UI controls into what we now expect to see everywhere. Does Alt-F4 close a program on your system? Does it close pretty much any program? You can thank CUA for that. Or would you prefer that every program have its own unique way of being closed? Oh, and if you say "click on the X in the corner", well, common UI guidelines is what gave us those, too. Or would you prefer every program to put a close button in a different place? > Just look at any interactive page on the web, they all work differently. > People are used to it. And it allows innovation. No, they don't all work differently. They actually all work largely the same way, because their fundamental controls are provided by the host platform (in this case, the web browser). And where they *don't* all work the same way (date pickers, ugh ugh ugh), they're a pain in the behind. Maybe it's just that you're not old enough to have worked with text editors that predate CUA? That might explain your confusion about what should be standardized. ChrisA remembers WordStar but none of its bindings, and PC-Type and most of its command keys [1] https://en.wikipedia.org/wiki/IBM_Common_User_Access From greg.ewing at canterbury.ac.nz Sun Oct 8 08:07:22 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 01:07:22 +1300 Subject: The "loop and a half" In-Reply-To: References: <59d4ffb5$0$14949$b1db1813$d948b532@news.astraweb.com> <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> Message-ID: bartc wrote: > Then you might have 'sort' for the non-interactive version, and 'isort' > or whatever for the interactive. It's pretty rare that you'd want to use 'sort' interactively, which is why your hypothetical 'isort' doesn't exist. However, it *is* common to use it as part of a pipeline, which is why it defaults to reading from stdin. A consequence of that is that if you run it on its own, without any arguments, it stops and waits for input from the terminal. But it's not specifically designed to be used that way, so it doesn't bother providing any prompts in that situation. > Except you simply wouldn't use an interactive version of any program > that reads an arbitrary long list of uninterrupted data, no matter how > the ending is indicated. You'd use a text editor, enter the lines at > your leisure, go back and forth to check and edit as needed, THEN you > would submit that file to 'sort'. As an actual input file. In which case it won't 'hang', and you will be happy. :-) > A program like 'python' might fit the bill too. Here, you can give it > the name of a file, OR say nothing, and it will take input from stdin. > > Funnily enough, you now get a message, and a prompt. And when entering > multiple lines of code, you get a prompt before each line. You can also > terminate the sequence by entering a blank line. Python goes to all that trouble because it's designed to be used interactively, i.e. you can have a true conversation with it. You can't really have a conversation with 'sort', since it needs all the input before it can do anything. -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 8 08:15:22 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 01:15:22 +1300 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] In-Reply-To: <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > You don't think multiple columns in interactive mode is useful? The issue is not whether single vs. multi column mode is better, but whether it should automatically switch based on what kind of thing is connected to stdin. Personally I find that behavour surprising and would be happy for it to use multi-column always unless explicitly told otherwise, but it's not a big issue. The thing that *really* annoys me is Linux insisting on colourising the output to a tty, since it invariably seems to pick an undreadable colour scheme. And the case-insensitive sorting... there's a reason Makefile starts with a capital M, dammit! -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 8 08:28:38 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 01:28:38 +1300 Subject: The "loop and a half" In-Reply-To: <24LBB.1729028$Ld2.1169900@fx17.am4> References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> Message-ID: bartc wrote: > This preprocesses the code and shows the result. Typical programs will > have many thousands of lines of output, but it just dumps it to the > console. You /have/ to use '>' to use it practically (Windows doesn't > really have a working '|' system.) This may be why you're having trouble grokking unix. The primary purpose of the unix convention of reading from stdin and writing to stdout by default is to make it easy to use programs in pipelines. If you haven't experienced that way of working, you may not appreciate how useful it can be. > BTW if I try: > > > gcc > it doesn't work (this on Linux). What happened to the generic solution? I agree that the C compiler is unusual in that respect. The reason may be that it relies on filename suffixes to figure out what to do with its inputs: if it's .c, compile it; if it's .s, assemble it; if it's .o, link it; etc. It wouldn't know what to do with something fed into stdin. There may be ways to tell it, though. Stack Overflow suggests: flex -t lexer.l | gcc -x c -c -o lexer.o - -- Greg From bc at freeuk.com Sun Oct 8 08:33:02 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 13:33:02 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: <1WoCB.965336$HN.410384@fx21.am4> On 08/10/2017 12:22, Paul Moore wrote: > When developing scripts, applications, or any form of code, I use good > ideas from anywhere, as I doubt that I have the monopoly on knowing > the perfect way to write code. Some of those good ideas come from > Unix-based systems. That's not "because Linux is so much better", it's > because someone other than me had a good idea, and I acknowledge the > fact. When developing code I use whatever tools and techniques I like. But if I want to someone else to build one of my applications, I make it as easy as possible for them, even to to the extent of translating it to a language that they will be able to build. Here, for example, is a C compiler (which, actually, is not written in C, but is presented as a C program): www.bcas.freeuk.com/mcc64.c One file only. But it is only for 64 bits. Build on Windows or Linux (for Linux it is 'gcc -m64 mcc64.c -omcc -lm'). Runs on either (but generates code for win64 ABI). Another: www.bcas.freeuk.com/pcc64.c, a byte-code interpreter, builds and runs on either OS, one file. That needs a separate byte-code compiler, www.bcas.freeuk.com/qcc64.c. Again, one file. Again, runs on either OS. And I believe each file works with any C compiler. (Actually, I just found it had a dependency on a Windows time function; I've removed that temporarily for this upload.) See the pattern here? Simplicity not complexity. Consideration for others by making life easier.) -- bartc From leamhall at gmail.com Sun Oct 8 08:35:16 2017 From: leamhall at gmail.com (leam hall) Date: Sun, 8 Oct 2017 08:35:16 -0400 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 8:15 AM, Gregory Ewing wrote: > > The thing that *really* annoys me is Linux insisting on colourising > the output to a tty, since it invariably seems to pick an undreadable > colour scheme. And the case-insensitive sorting... there's a reason > Makefile starts with a capital M, dammit! > Actually, it doesn't. The color output may come from /etc/profile.d/colorls.sh configs. Either dump that or "unalias ls" in your ~/.bash_profile. Colorized ls is something the distrobution people like and they put it in. Others of us don't care for it. But it's not "Linux", is the profile. Easy to customize. Leam From greg.ewing at canterbury.ac.nz Sun Oct 8 08:45:17 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 01:45:17 +1300 Subject: Interactive scripts (back on topic for once) [was Re: The "loop and a half"] In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87zi9436rx.fsf@bsb.me.uk> <94e48042-6099-efc3-eea2-eb31773239af@tjol.eu> Message-ID: Chris Angelico wrote: > Hmm, but usually I would expect them still to HAVE those streams, > they're just connected to /dev/null or something. I don't think they > would actually fail to exist, would they? On unix there's nothing to stop you launching a process with fds 0, 1 and 2 all closed. It would be a very unusual thing to do, but not impossible. I expect it's possible on Windows as well. -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 8 08:53:09 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 01:53:09 +1300 Subject: The "loop and a half" In-Reply-To: References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> <726ftchvm14pjl9mflceb3fpjbfvicqoc8@4ax.com> Message-ID: Peter J. Holzer wrote: > In any case, that -E writes to stdout and -S to file is an inconsistency > which looks more like a historical accident than a planned feature to > me. A possible reason is that with -S there is an obvious choice for the output file name, i.e. .s, but there is no conventional way of naming a preprocessed C source file, so it's best to make the user specify it. -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 8 09:06:29 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 02:06:29 +1300 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: bartc wrote: > Interactive Python requires quit() or exit(), complete with parentheses. Er, what? Ctrl-D works fine for me to exit Python when not in the midst of entering a block. -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 8 09:16:42 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 02:16:42 +1300 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d832d4$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: Chris Angelico wrote: > But personally, I'd have looked for a "print to > PS" of some sort, using a gigantic 'page' size, and then convert the > PS to PNG. I don't know for certain that I can do the latter > conversion, It would be easy on MacOSX. Anything you can print can be sent to a PDF file, and Preview lets you save any page from a PDF file as an image in your format of choice. -- Greg From lele at metapensiero.it Sun Oct 8 09:24:03 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Sun, 08 Oct 2017 15:24:03 +0200 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <87lgklkc24.fsf@metapensiero.it> Gregory Ewing writes: > The thing that *really* annoys me is Linux insisting on colourising > the output to a tty, since it invariably seems to pick an undreadable > colour scheme. And the case-insensitive sorting... there's a reason > Makefile starts with a capital M, dammit! As most things, that's configurable, for example I have the following in one of my teenager .bashxxx scripts: # Old-school ordering, upcase letters before lowercase export LC_COLLATE=C # Format dates accordingly to the language export TIME_STYLE=locale As others said already, it's difficult-to-impossible to get a one-size-fits-all configuration that satisfies everybody (where the cardinality of "everybody" exceeds 1, of course). ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From python at mrabarnett.plus.com Sun Oct 8 09:38:22 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 8 Oct 2017 14:38:22 +0100 Subject: django project avoid reload page where work algorithm In-Reply-To: <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> Message-ID: <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> On 2017-10-08 11:27, Xristos Xristoou wrote: > >> You mentioned email, so why not just ask for an email address to which >> you will send the result? > > i have user email from register i need to do all automate,how to use celery for this ? > You can use the smtplib module to send an email. From hjp-usenet3 at hjp.at Sun Oct 8 09:41:17 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sun, 8 Oct 2017 15:41:17 +0200 Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-07 08:35, Steve D'Aprano wrote: > Unfortunately ESCAPE is already used. VT100 (the terminal emulation which is > used in just about all terminals) all control sequences begin with ESC. So > every time you do something like press an arrow key, the terminal sends ESC > followed by other stuff. It would be painful if every time you hit an arrow > key, the interpreter took it as "Exit". The usual solution to this problem is to use a timeout: If ESC is not followed by another character which could continue the sequence within e.g. 0.1 seconds it is interpreted as a standalone key press instead of the start of an escape sequence. This is not 100% reliable: If you are on a jittery network connection, the O in the cursor-up sequence might arrive more than 0.1 seconds after the ESC - or the the three distinct keystrokes ESC O A might be delivered in the same packet. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From eryksun at gmail.com Sun Oct 8 09:52:49 2017 From: eryksun at gmail.com (eryk sun) Date: Sun, 8 Oct 2017 14:52:49 +0100 Subject: The "loop and a half" In-Reply-To: <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 10:12 AM, Steve D'Aprano wrote: > On Sun, 8 Oct 2017 02:06 am, bartc wrote: > >> Especially on >> Windows where the usual Ctrl C doesn't work, so you resort to Ctrl-Break >> will which actually abort it. Ctrl Z is uncommon. > > Thousands of Python programmers on Windows successfully learned to use Ctrl-Z > ENTER back in the days of Python 1.5, before quit/exit were added as a > convenience for beginners, and many of them probably still use it. Using Ctrl+Z (0x1A) isn't specific to Python. The Windows CRT's text-mode I/O inherits this from MS-DOS, which took it from CP/M. It's obvious in Python 2: >>> open('test.txt', 'w').write('123\x1a456') >>> open('test.txt', 'r').read() '123' >>> open('test.txt', 'rb').read() '123\x1a456' This has been misreported as a bug more than once. Python 3 opens CRT file descriptors in binary mode. Its text mode (TextIOWraper) doesn't implement the legacy Ctrl+Z behavior. However, Ctrl+Z at the start of a line is manually supported in the REPL and raw _WindowsConsoleIO. From hjp-usenet3 at hjp.at Sun Oct 8 10:04:17 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sun, 8 Oct 2017 16:04:17 +0200 Subject: The "loop and a half" References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <87k208w48i.fsf@elektro.pacujo.net> <_FQBB.1006789$cS2.932107@fx19.am4> <59d831e7$0$14944$b1db1813$d948b532@news.astraweb.com> <59d8d474$0$14928$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-07 13:19, Steve D'Aprano wrote: > On Sat, 7 Oct 2017 11:06 pm, bartc wrote: >> However it does seem to expose a flaw in the ability of command line >> tools to work with non-command line tools. >> >> So I have to copy 33,000 lines from a document, > > Don't be daft. Nobody says that stdin is a sufficient interface for a > heavy-weight task like that. With 33000 lines of text, I absolutely would > save them to a permanent file on disk, I think you got sucked into bartc's way of thinking here. Of course stdin is perfectly fine for reading 33000 lines of text. You probably do that all the time. Think of pipes like "... | grep ... | sort | uniq | ...": No reason to save the output of one stage to a temporary file before invoking the next just because it might be large. It's the terminal which is not a good interface for entering 33000 lines of text, even though pasting 33000 lines from the clipboard into xterm works (I vaguely recall that there is or was a size limit, but 33000 shortish lines doesn't seem to be enough to reach it). hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From saxri89 at gmail.com Sun Oct 8 10:27:22 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 8 Oct 2017 07:27:22 -0700 (PDT) Subject: django project avoid reload page where work algorithm In-Reply-To: References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> Message-ID: <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> > You can use the smtplib module to send an email. ok but how to avoid pong reload on page ?can you help me ? From hjp-usenet3 at hjp.at Sun Oct 8 10:31:01 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sun, 8 Oct 2017 16:31:01 +0200 Subject: Python platforms without long long Message-ID: I just stumbled about this note in https://docs.python.org/3/library/array.html: | 2. The 'q' and 'Q' type codes are available only if the platform C | compiler used to build Python supports C long long, or, on Windows, | __int64. | | New in version 3.3. The long long type was standardized in C99 (and was a popular extension before that). Python 3.3 was released in 2012 - 13 years later. Were any of the supported platforms of Python3 really still missing this type at that time? hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From bc at freeuk.com Sun Oct 8 11:01:23 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 16:01:23 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> Message-ID: <65rCB.121006$Hs3.57077@fx16.am4> On 08/10/2017 13:05, Chris Angelico wrote: > On Sun, Oct 8, 2017 at 10:46 PM, bartc wrote: >> Just look at any interactive page on the web, they all work differently. >> People are used to it. And it allows innovation. > Maybe it's just that you're not old enough to have worked with text > editors that predate CUA? That might explain your confusion about what > should be standardized. I first used line editors in the 1970s. When I started writing graphics apps, graphics was very new, at least in the low-end business computer world (the one that was soon dominated by the IBM PC). So there was little existing practice to copy from. (And actually, I had my hands full either designing the graphics boards, or writing the drivers, graphics libraries, providing the fonts and so on (plus the languages), all the underlying stuff needed to make it all work.) However as graphics became more mainstream then yes I did adopt some commonly expected styles (menubars for example). As for Alt-F4, if that generates a WM_CLOSE message for example, then I would be obliged to deal with it. -- bartc From python at mrabarnett.plus.com Sun Oct 8 11:35:12 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 8 Oct 2017 16:35:12 +0100 Subject: django project avoid reload page where work algorithm In-Reply-To: <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> Message-ID: <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> On 2017-10-08 15:27, Xristos Xristoou wrote: > >> You can use the smtplib module to send an email. > > > ok but how to avoid pong reload on page ?can you help me ? > Do you mean "long reload"? If you have the user's email address from when they registered, you could have the user log in to get the results. If the user has closed the window, or the session has expired, well.... they'd have to log in again. From hjp-usenet3 at hjp.at Sun Oct 8 11:41:04 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sun, 8 Oct 2017 17:41:04 +0200 Subject: The "loop and a half" References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <24LBB.1729028$Ld2.1169900@fx17.am4> <726ftchvm14pjl9mflceb3fpjbfvicqoc8@4ax.com> Message-ID: On 2017-10-08 12:53, Gregory Ewing wrote: > Peter J. Holzer wrote: >> In any case, that -E writes to stdout and -S to file is an inconsistency >> which looks more like a historical accident than a planned feature to >> me. > > A possible reason is that with -S there is an obvious choice > for the output file name, i.e. .s, but there is > no conventional way of naming a preprocessed C source file, > so it's best to make the user specify it. Actually, .i seems to be pretty common, and at least gcc recognizes .i files as C source files which should not be preprocessed. Same for DEC's c89 compiler (ca. 1992). The manual page for the PC/IX cc (ca. 1984) doesn't mention .i, so I guess it didn't recognize that extension (but it already had the -E option). hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From grant.b.edwards at gmail.com Sun Oct 8 11:48:11 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 8 Oct 2017 15:48:11 +0000 (UTC) Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-08, eryk sun wrote: > On Sun, Oct 8, 2017 at 10:12 AM, Steve D'Aprano > wrote: >> On Sun, 8 Oct 2017 02:06 am, bartc wrote: >> >>> Especially on >>> Windows where the usual Ctrl C doesn't work, so you resort to Ctrl-Break >>> will which actually abort it. Ctrl Z is uncommon. >> >> Thousands of Python programmers on Windows successfully learned to use Ctrl-Z >> ENTER back in the days of Python 1.5, before quit/exit were added as a >> convenience for beginners, and many of them probably still use it. > > Using Ctrl+Z (0x1A) isn't specific to Python. The Windows CRT's > text-mode I/O inherits this from MS-DOS, which took it from CP/M. Which took it from RSX-11. Or probably more specifically from FILES-11. I woldn't be surprised if the enineers at DEC got it from somewhere else before that. -- Grant From saxri89 at gmail.com Sun Oct 8 11:58:06 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 8 Oct 2017 08:58:06 -0700 (PDT) Subject: django project avoid reload page where work algorithm In-Reply-To: References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> Message-ID: ?? ???????, 8 ????????? 2017 - 6:35:28 ?.?. UTC+3, ? ??????? MRAB ??????: > On 2017-10-08 15:27, Xristos Xristoou wrote: > Do you mean "long reload"? user can put some numbers in my html template and i take that numbers and i sue it in specific mathematical algorithm that algorithm to create result need time between 5-10 minutes in that time browser stay on reload...that i mean From rosuav at gmail.com Sun Oct 8 11:59:11 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 02:59:11 +1100 Subject: Python platforms without long long In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 1:31 AM, Peter J. Holzer wrote: > I just stumbled about this note in > https://docs.python.org/3/library/array.html: > > | 2. The 'q' and 'Q' type codes are available only if the platform C > | compiler used to build Python supports C long long, or, on Windows, > | __int64. > | > | New in version 3.3. > > The long long type was standardized in C99 (and was a popular extension > before that). Python 3.3 was released in 2012 - 13 years later. Were any > of the supported platforms of Python3 really still missing this type at > that time? At what point did MSVC support long long? That's usually the guiding factor with C standard usage. ChrisA From cl at isbd.net Sun Oct 8 12:02:03 2017 From: cl at isbd.net (Chris Green) Date: Sun, 8 Oct 2017 17:02:03 +0100 Subject: Pandas/dataexplore, how do you get data into them? Message-ID: I am looking at dataexplore and Pandas, they look as if they may provide useful tools but at the moment I can't quite understand how you get data into them. How do you load a large table into dataexplore? Ultimetely I want to get data from a database table but any help would be useful. -- Chris Green ? From rosuav at gmail.com Sun Oct 8 12:13:38 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 03:13:38 +1100 Subject: The "loop and a half" In-Reply-To: <65rCB.121006$Hs3.57077@fx16.am4> References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: On Mon, Oct 9, 2017 at 2:01 AM, bartc wrote: > However as graphics became more mainstream then yes I did adopt some > commonly expected styles (menubars for example). As for Alt-F4, if that > generates a WM_CLOSE message for example, then I would be obliged to deal > with it. Yes, it usually does generate that. Why? Because your desktop manager translates concrete user actions into abstract events like "close window" - and does so according to a number of standards. In case you haven't noticed, those standards are not 100% consistent across platforms. So that means that... On Sun, Oct 8, 2017 at 10:46 PM, bartc wrote: > On 07/10/2017 15:40, Steve D'Aprano wrote: >> Or you could just follow the expected Unix interface instead of inventing >> your own. > > Your job is to port an editor that people have been using for 30 years to > Linux. The first thing you do is to change all the commands and shortcuts to > match what is typical on Linux? So that no-one who was familiar with it as > it was can actually use it? ... yeah, you absolutely *should* follow your OS's conventions, and you automatically *will* if you're using a properly-designed GUI toolkit. Why should it be different with the console? For instance, anyone on Linux will understand what this prompt notation means: Use config file: [~/.flurblerc] Yep, definitely follow your host platform's conventions. ChrisA From marko at pacujo.net Sun Oct 8 12:43:35 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 08 Oct 2017 19:43:35 +0300 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <87a811io94.fsf@elektro.pacujo.net> leam hall : > Colorized ls is something the distrobution people like and they put it > in. Others of us don't care for it. But it's not "Linux", is the > profile. Easy to customize. Easy and easy... "Linux" means so many things to people. For example, the recent "Linux Subsystem on Windows 10" is funny in that Linux is the one thing it most definitely doesn't have. Closer to home, systemd has taken a central role in the main Linux distributions. I think it would be more accurate to call them "systemd distros" than "Linux distros". It is not at all easy for the Linux user to figure out what configuration options there are, and which ones are intended for end-user configuration. More and more, such tuning needs to be done via systemd unit files (or applicable GUI facilities) and the classical configuration files are deprecated. For example, how can a programmer get a core file of a crashing program? Why, you need to use the systemd-coredump service, of course: BTW, I'm not ranting against systemd here, just stating the tectonic shift that is undergoing in the Linux world. As far as "ls" goes, its man page states: The LS_COLORS environment variable can change the settings. Use the dircolors command to set it. Yes, my distro does contain "/etc/profile.d/colorls.sh" but nothing indicates whether the system administrator should or should not touch the file. I can't seem to locate a mention of it here: Also, you need some detective work to derive the luser configuration interface from "/etc/profile.d/colorls.sh". Marko From bc at freeuk.com Sun Oct 8 12:50:59 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 17:50:59 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: On 08/10/2017 17:13, Chris Angelico wrote: > On Mon, Oct 9, 2017 at 2:01 AM, bartc wrote: >> However as graphics became more mainstream then yes I did adopt some >> commonly expected styles (menubars for example). As for Alt-F4, if that >> generates a WM_CLOSE message for example, then I would be obliged to deal >> with it. > > Yes, it usually does generate that. Why? Because your desktop manager > translates concrete user actions into abstract events like "close > window" - and does so according to a number of standards. In case you > haven't noticed, those standards are not 100% consistent across > platforms. So that means that... > > On Sun, Oct 8, 2017 at 10:46 PM, bartc wrote: >> On 07/10/2017 15:40, Steve D'Aprano wrote: >>> Or you could just follow the expected Unix interface instead of inventing >>> your own. >> >> Your job is to port an editor that people have been using for 30 years to >> Linux. The first thing you do is to change all the commands and shortcuts to >> match what is typical on Linux? So that no-one who was familiar with it as >> it was can actually use it? > > ... yeah, you absolutely *should* follow your OS's conventions, and > you automatically *will* if you're using a properly-designed GUI > toolkit. Why should it be different with the console? For instance, > anyone on Linux will understand what this prompt notation means: > > Use config file: [~/.flurblerc] > > Yep, definitely follow your host platform's conventions. Yeah, well, some people like to be sheep, others like to be individuals**. I start in computing at a time when an application was the only thing running on a computer, at least, when people had personal computers of their own. Then it really didn't matter what went on outside, as nothing did. (That approach seems to have become popular again with tablets and things usually having one application occupying the screen at a time.) And within an application, it can do what it likes. With regards to editing, there are some common conventions that I absolutely hate: * Left and right keys, and backspace and delete keys, that don't regard the left and right ends of a line as hard stops; they just keep going. OK-ish for word processing, but not for line-oriented code. * Clicking on a box containing text and the whole thing being highlighted. Now you're on tenterhooks as the slightest move and everything disappears. You might be able able to press Ctrl Z to get it back (yes, the same Ctrl Z that's supposed to terminate applications!) but it's still highlighted. * Clicking backspace on a web page doing the same as the Back button. Now, backspace is used for editing text within forms. It's very annoying if, after half an hour filling in a form, somehow the current box loses the focus (the cursor), but you proceed to press backspace space several time before noticing. Like my first complaint, but in spades. * Under Windows, if you press Shift while Caps Lock is on, you get lower case letters. I've never, ever wanted to do this (probably no one else has). My own editor doesn't obey that convention: shift-A will always come out as 'A' whatever the caps lock setting. There are dozens more, yet you are surprised why sometimes I prefer doing things my own way? There are good reasons! (**I was in the audience of a Michael Palin interview a couple of weeks back. (You know, an actual Python!) Before he came on the audience was programmed to respond to the word 'individuals' by all saying 'Yes, we are all individuals!'. Apart from me, obviously.) -- bartc From leamhall at gmail.com Sun Oct 8 13:20:59 2017 From: leamhall at gmail.com (Leam Hall) Date: Sun, 8 Oct 2017 13:20:59 -0400 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] In-Reply-To: <87a811io94.fsf@elektro.pacujo.net> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> <87a811io94.fsf@elektro.pacujo.net> Message-ID: <70070bf1-6fd4-8f24-0626-7ea8f2c9ecc1@gmail.com> On 10/08/2017 12:43 PM, Marko Rauhamaa wrote: > leam hall : > "Linux" means so many things to people. Yes, but just because someone can spell it doesn't mean they can redefine it. :) > Closer to home, systemd has taken a central role in the main Linux > distributions. I think it would be more accurate to call them "systemd > distros" than "Linux distros". I have other words for them; mostly ones I try not to say. > It is not at all easy for the Linux user to figure out what > configuration options there are, and which ones are intended for > end-user configuration. Agree! I had to look this up and I've been doing Linux for a few years. > Marko I knew your e-mail address was familiar; saluton! Leam -- Who hasn't practiced that language in years. Sadly... From cl at isbd.net Sun Oct 8 13:51:13 2017 From: cl at isbd.net (Chris Green) Date: Sun, 8 Oct 2017 18:51:13 +0100 Subject: Pandas/dataexplore, how do you get data into them? References: Message-ID: Stefan Ram wrote: > Chris Green writes: > >How do you load a large table into dataexplore? > > I have not tested the following lines, I have no experience > with "dataexplore", this is just from what I heard: > > from pandastable.app import DataExplore > app = DataExplore() > table = app.getCurrentTable() > > You can now supposedly use ?table.importCSV? to import CSV. > Read the docs for more details. It might not even work as I > wrote, but maybe just the names of those callables can get > you started. > > Also check out ?app.load_dataframe? which supposedly can > load a "dataframe" (interactively?). > Thanks, those might get me going. -- Chris Green ? From tjreedy at udel.edu Sun Oct 8 13:56:05 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 8 Oct 2017 13:56:05 -0400 Subject: IDLE help. In-Reply-To: References: Message-ID: On 10/8/2017 5:24 AM, Joe Wilde wrote: > I am having trouble getting IDLE (Python 3.6 - 64-bit) to open for Windows 10. When I try and run IDLE, nothing happens. It works fine for Python 2.7, but won't open for Python 3.6. Give more information. How did you install Python? Did you select the option to install tkinter and IDLE? How are you trying to run IDLE? Do you know what Command Prompt and a console are? -- Terry Jan Reedy From rosuav at gmail.com Sun Oct 8 13:57:26 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 04:57:26 +1100 Subject: The "loop and a half" In-Reply-To: References: <59d579f0$0$14951$b1db1813$d948b532@news.astraweb.com> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> Message-ID: On Mon, Oct 9, 2017 at 3:37 AM, Stefan Ram wrote: > Gregory Ewing writes: >>bartc wrote: >>>Interactive Python requires quit() or exit(), complete with parentheses. >>Er, what? Ctrl-D works fine for me to exit Python when not >>in the midst of entering a block. > > Under Microsoft Windows, one can press > > Alt-Space > > to open the window menu and then > > C > > to close the window. Since this already works with nearly > every other window, it seems natural to use this also with > the Python console. Also works in any other standards-compliant OS. > (Disclaimer: I don't whether Microsoft is retiring the > Windows menu in Windows 10.) See above comment :| ChrisA From rosuav at gmail.com Sun Oct 8 14:10:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 05:10:37 +1100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: On Mon, Oct 9, 2017 at 3:50 AM, bartc wrote: > Yeah, well, some people like to be sheep, others like to be individuals**. Yeah, well, some people like to be standards-compliant, others like to be irrelevant morons. > I start in computing at a time when an application was the only thing > running on a computer, at least, when people had personal computers of their > own. Then it really didn't matter what went on outside, as nothing did. > > (That approach seems to have become popular again with tablets and things > usually having one application occupying the screen at a time.) > > And within an application, it can do what it likes. It's 2017. Even on a phone/tablet, where there isn't enough room to usefully display more than one app *at a time*, you still have multiple apps. > With regards to editing, > there are some common conventions that I absolutely hate: > > * Under Windows, if you press Shift while Caps Lock is on, you get lower > case letters. I've never, ever wanted to do this (probably no one else has). > My own editor doesn't obey that convention: shift-A will always come out as > 'A' whatever the caps lock setting. > > There are dozens more, yet you are surprised why sometimes I prefer doing > things my own way? There are good reasons! Yep. Good reasons like that you're a moron. You assume that since *you* have never needed to produce one lower-case letter in a block of upper-case, that "probably no one else has", and then you make it impossible to do that in your editor. I have wanted to produce a lower-case letter by holding Shift. I have also used this behaviour to detect and recognize faults of various sorts. Do you understand the concept of debugging a system by getting more information, not less? > (**I was in the audience of a Michael Palin interview a couple of weeks > back. (You know, an actual Python!) Before he came on the audience was > programmed to respond to the word 'individuals' by all saying 'Yes, we are > all individuals!'. Apart from me, obviously.) Obviously. Which meant that the crowd created a joke, and you simply didn't take part in it. You became irrelevant. ChrisA From marko at pacujo.net Sun Oct 8 14:26:04 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 08 Oct 2017 21:26:04 +0300 Subject: The "loop and a half" References: <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: <87wp45fqdf.fsf@elektro.pacujo.net> Chris Angelico : > On Mon, Oct 9, 2017 at 3:50 AM, bartc wrote: >> Yeah, well, some people like to be sheep, others like to be >> individuals**. > > Yeah, well, some people like to be standards-compliant, others like to > be irrelevant morons. Even being called a sheep doesn't justify that kind of response, Chris. >> There are dozens more, yet you are surprised why sometimes I prefer >> doing things my own way? There are good reasons! > > Yep. Good reasons like that you're a moron. You can think that but shouldn't post it. Simply plonk Bart if you have to. Marko From bc at freeuk.com Sun Oct 8 15:19:40 2017 From: bc at freeuk.com (bartc) Date: Sun, 8 Oct 2017 20:19:40 +0100 Subject: The "loop and a half" In-Reply-To: References: <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: On 08/10/2017 19:10, Chris Angelico wrote: > On Mon, Oct 9, 2017 at 3:50 AM, bartc wrote: > You assume that since > *you* have never needed to produce one lower-case letter in a block of > upper-case, that "probably no one else has", and then you make it > impossible to do that in your editor. Only when caps-lock is stuck on (then the feature may have be of some use at last). However my editor also makes it very easy to reverse or change case of existing text. The advantage of Shift working one-way is that it is guaranteed to give you a capital letter without you needing to check the caps-lock status or checking the screen to see that it was typed as capitals. That is 100 times more useful than the rare time you have to add a lower case letter in a sea of capitals and can't be bothered to turn off caps lock. I have wanted to produce a > lower-case letter by holding Shift. I've read that other people have had exactly the same trouble. I don't believe that my pattern of typing English text is that different from most other people's, and I've ALWAYS found that 'feature' annoying, so from that I might infer that plenty of others do too. In fact, if I start off MS Word, with caps lock unknowingly on, and type Shifted-T, unshifted-H, unshifted-E, that will temporarily display 'tHE' before it gets auto-corrected to the intended 'The'. I wonder why it does that? According to you, people WANT to type tHE. BTW, all the typewriters I've used do exactly what I want. If caps lock is on (shift-lock there), then pressing Shift doesn't reverse it. So, what happened to existing practice there, typewriters hadn't been around long enough? > I have also used this behaviour to > detect and recognize faults of various sorts. Do you understand the > concept of debugging a system by getting more information, not less? I've no idea what you're talking about there. > Yep. Good reasons like that you're a moron. > Yeah, well, some people like to be standards-compliant, others like to > be irrelevant morons. I started to be angry when I saw these insults now I'm just rather depressed. -- bartc From tjol at tjol.eu Sun Oct 8 15:45:01 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 8 Oct 2017 21:45:01 +0200 Subject: Pandas/dataexplore, how do you get data into them? In-Reply-To: References: Message-ID: On 08/10/17 18:02, Chris Green wrote: > I am looking at dataexplore and Pandas, they look as if they may > provide useful tools but at the moment I can't quite understand how > you get data into them. > > How do you load a large table into dataexplore? > > Ultimetely I want to get data from a database table but any help would > be useful. > I know nothing about dataexplore, but pandas has tonnes of handy I/O functions: http://pandas.pydata.org/pandas-docs/version/0.20/io.html I mostly use pandas.read_csv to get data in, but there is also a read_sql function that you might find useful -- Thomas From Richard at Damon-Family.org Sun Oct 8 15:48:27 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 8 Oct 2017 15:48:27 -0400 Subject: django project avoid reload page where work algorithm In-Reply-To: References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> Message-ID: On 10/8/17 11:58 AM, Xristos Xristoou wrote: > ?? ???????, 8 ????????? 2017 - 6:35:28 ?.?. UTC+3, ? ??????? MRAB ??????: >> On 2017-10-08 15:27, Xristos Xristoou wrote: > >> Do you mean "long reload"? > > user can put some numbers in my html template and i take that numbers and i sue it in specific mathematical algorithm that algorithm to create result need time between 5-10 minutes in that time browser stay on reload...that i mean > It sounds like the fundamental problem is that you are doing the calculation in the web page handler. This means that the browser will be stuck in the page load until the calculation finishes, and that if the user aborts the access (or loses connection) then the web page handler is aborted. What you need to do is rather than doing the calculation in the page handler, you need to kick off an independent process to do the calculation, and then immediately finish the page (maybe giving the use a link to a page they can check to see if a result is available). From saxri89 at gmail.com Sun Oct 8 16:46:59 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 8 Oct 2017 13:46:59 -0700 (PDT) Subject: django project avoid reload page where work algorithm In-Reply-To: References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> Message-ID: <547684af-f5f5-4d75-b95b-ffb82f2deaf0@googlegroups.com> ?? ???????, 8 ????????? 2017 - 10:48:38 ?.?. UTC+3, ? ??????? Richard Damon ??????: > On 10/8/17 11:58 AM, Xristos Xristoou wrote: > > ?? ???????, 8 ????????? 2017 - 6:35:28 ?.?. UTC+3, ? ??????? MRAB ??????: > >> On 2017-10-08 15:27, Xristos Xristoou wrote: > > > >> Do you mean "long reload"? > > > > user can put some numbers in my html template and i take that numbers and i sue it in specific mathematical algorithm that algorithm to create result need time between 5-10 minutes in that time browser stay on reload...that i mean > > > > It sounds like the fundamental problem is that you are doing the > calculation in the web page handler. This means that the browser will be > stuck in the page load until the calculation finishes, and that if the > user aborts the access (or loses connection) then the web page handler > is aborted. > > What you need to do is rather than doing the calculation in the page > handler, you need to kick off an independent process to do the > calculation, and then immediately finish the page (maybe giving the use > a link to a page they can check to see if a result is available). yes this is nice can you help me to do that? From torriem at gmail.com Sun Oct 8 17:41:32 2017 From: torriem at gmail.com (Michael Torrie) Date: Sun, 8 Oct 2017 15:41:32 -0600 Subject: django project avoid reload page where work algorithm In-Reply-To: <547684af-f5f5-4d75-b95b-ffb82f2deaf0@googlegroups.com> References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> <547684af-f5f5-4d75-b95b-ffb82f2deaf0@googlegroups.com> Message-ID: <79e355b8-0381-a8fe-f23e-753c0ba40c87@gmail.com> On 10/08/2017 02:46 PM, Xristos Xristoou wrote: > ?? ???????, 8 ????????? 2017 - 10:48:38 ?.?. UTC+3, ? ??????? Richard Damon ??????: >> It sounds like the fundamental problem is that you are doing the >> calculation in the web page handler. This means that the browser will be >> stuck in the page load until the calculation finishes, and that if the >> user aborts the access (or loses connection) then the web page handler >> is aborted. >> >> What you need to do is rather than doing the calculation in the page >> handler, you need to kick off an independent process to do the >> calculation, and then immediately finish the page (maybe giving the use >> a link to a page they can check to see if a result is available). > > yes this is nice can you help me to do that? I'm sure google can give you some ideas on how to implement this. Have you done a google search? Maybe start with the terms "django long-running process" and see what comes up. Good luck. Learning is a fun adventure! From Richard at Damon-Family.org Sun Oct 8 17:50:29 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 8 Oct 2017 17:50:29 -0400 Subject: django project avoid reload page where work algorithm In-Reply-To: <547684af-f5f5-4d75-b95b-ffb82f2deaf0@googlegroups.com> References: <22667bf4-4eed-4dfb-8ded-45e868c33a0d@googlegroups.com> <33367873-0107-b297-d5b7-e87decc867f8@mrabarnett.plus.com> <3469af48-e592-41f6-a4b0-6e5b8636d3f3@googlegroups.com> <8dff2da0-730c-e506-cd02-4e72de7034df@mrabarnett.plus.com> <75ad8325-178a-4255-9115-fe6db9ed23fc@googlegroups.com> <4494d8a5-ea35-25fd-0e60-bddd90b1bc40@mrabarnett.plus.com> <547684af-f5f5-4d75-b95b-ffb82f2deaf0@googlegroups.com> Message-ID: On 10/8/17 4:46 PM, Xristos Xristoou wrote: > ?? ???????, 8 ????????? 2017 - 10:48:38 ?.?. UTC+3, ? ??????? Richard Damon ??????: >> On 10/8/17 11:58 AM, Xristos Xristoou wrote: >>> ?? ???????, 8 ????????? 2017 - 6:35:28 ?.?. UTC+3, ? ??????? MRAB ??????: >>>> On 2017-10-08 15:27, Xristos Xristoou wrote: >>> >>>> Do you mean "long reload"? >>> >>> user can put some numbers in my html template and i take that numbers and i sue it in specific mathematical algorithm that algorithm to create result need time between 5-10 minutes in that time browser stay on reload...that i mean >>> >> >> It sounds like the fundamental problem is that you are doing the >> calculation in the web page handler. This means that the browser will be >> stuck in the page load until the calculation finishes, and that if the >> user aborts the access (or loses connection) then the web page handler >> is aborted. >> >> What you need to do is rather than doing the calculation in the page >> handler, you need to kick off an independent process to do the >> calculation, and then immediately finish the page (maybe giving the use >> a link to a page they can check to see if a result is available). > > yes this is nice can you help me to do that? > I haven't done it myself, but you may want to look at the various os.spawn* methods. From greg.ewing at canterbury.ac.nz Sun Oct 8 17:57:04 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 10:57:04 +1300 Subject: The "loop and a half" In-Reply-To: References: <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d891be$0$14972$b1db1813$d948b532@news.astraweb.com> <_74CB.1635456$tu4.672402@fx35.am4> <59d8e765$0$14944$b1db1813$d948b532@news.astraweb.com> <65rCB.121006$Hs3.57077@fx16.am4> Message-ID: bartc wrote: > And within an application, it can do what it likes. With regards to > editing, there are some common conventions that I absolutely hate: In other words, you would like all authors of text editors to adopt a certain set of conventions for these things. So much for each program "doing what it likes". -- Greg From cl at isbd.net Sun Oct 8 18:17:16 2017 From: cl at isbd.net (Chris Green) Date: Sun, 8 Oct 2017 23:17:16 +0100 Subject: Pandas/dataexplore, how do you get data into them? References: Message-ID: Thomas Jollans wrote: > On 08/10/17 18:02, Chris Green wrote: > > I am looking at dataexplore and Pandas, they look as if they may > > provide useful tools but at the moment I can't quite understand how > > you get data into them. > > > > How do you load a large table into dataexplore? > > > > Ultimetely I want to get data from a database table but any help would > > be useful. > > > I know nothing about dataexplore, but pandas has tonnes of handy I/O > functions: > > http://pandas.pydata.org/pandas-docs/version/0.20/io.html > > I mostly use pandas.read_csv to get data in, but there is also a > read_sql function that you might find useful > Ah, thank you, this is getting more interesting! :-) -- Chris Green ? From robertvstepp at gmail.com Sun Oct 8 20:37:23 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 8 Oct 2017 19:37:23 -0500 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sun, Oct 8, 2017 at 5:36 AM, bartc wrote: > > On 08/10/2017 10:12, Steve D'Aprano wrote: >> >> On Sun, 8 Oct 2017 02:06 am, bartc wrote: >>> I'm getting fed up with this thread now. >> >> >> This thread would be a lot less frustrating if you would enter into it with a >> spirit of open-minded enquiry rather than an overbearing sense of superiority >> that anything others do that you don't must be worthless. > > > Frustrating for whom? > > It seems to me that it's pretty much everyone here who has an overbearing sense of superiority in that everything that Unix or Linux does is a million times better than anything else. I follow this list in an effort to learn as much as I can even though I am usually a fish out of water here. But this thread in all its twists and turns and various subject line changes seems to have gotten totally out of hand. Even though I am quoting only part of this one message, there are actually many others that I am responding to here. In my opinion (Honestly admitting my lack of technical competence.), this insatiable thirst on this list to get every bit of technical minutiae exactly correct is doing a severe disservice to the friendliness of this community. Yes, gently correct the (perceived) errors, because we all want to have it right, but please cease this incessant pounding of points (and people) into the ground to prove we are right and they are wrong! I doubt any of this is going to change Bart's mind. Why can we not allow him to make his points, respond to them appropriately, and then let it go when it is clear he has strongly held opinions that are not likely to change? And Bart, when large numbers of technical experts in their fields have spent many hours/months/years, yea, even several decades, developing these software systems, why do you think that you, all by yourself, know better? Can you not see how frustrating this is for people who have spent good chunks of their lives trying to do the best they can on these software systems? Don't you think it is a better approach to perhaps do some self-examination and approach things from a more humble learner's perspective? And BTW, there are many users of non-*nix systems on this list, or who do work on multiple operating system platforms. Can we not let people be who they are, perceived warts (valid or not) and all, and after responding (hopefully gently) to technical errors just let them be??? Peace. -- boB From ryan.xgamer99 at gmail.com Sun Oct 8 20:38:26 2017 From: ryan.xgamer99 at gmail.com (Ryan Holmes) Date: Sun, 8 Oct 2017 17:38:26 -0700 (PDT) Subject: How to track usage within a desktop python application Message-ID: <4cb2e619-8057-452d-b150-638fb4630aac@googlegroups.com> I maintain a desktop python application that is used by a decent number of folks (I would assume 10k+, though it's hard to know since it's based on number of downloads rather than number of unique users). I would like to integrate some sort of usage tracking that would enable me to determine number of users along with startups, which features are used/not used, performance metrics, exceptions, etc. Since it's an open-source project, I am looking for a free service (or a service that provides free licenses to open source projects) that has a python client library. Can anyone recommend something? From wrw at mac.com Sun Oct 8 22:43:00 2017 From: wrw at mac.com (William Ray Wing) Date: Sun, 08 Oct 2017 22:43:00 -0400 Subject: How to track usage within a desktop python application In-Reply-To: <4cb2e619-8057-452d-b150-638fb4630aac@googlegroups.com> References: <4cb2e619-8057-452d-b150-638fb4630aac@googlegroups.com> Message-ID: > On Oct 8, 2017, at 8:38 PM, Ryan Holmes wrote: > > I maintain a desktop python application that is used by a decent number of folks (I would assume 10k+, though it's hard to know since it's based on number of downloads rather than number of unique users). I would like to integrate some sort of usage tracking that would enable me to determine number of users along with startups, which features are used/not used, performance metrics, exceptions, etc. Since it's an open-source project, I am looking for a free service (or a service that provides free licenses to open source projects) that has a python client library. > You do know, of course, that most folks frown on applications that ?phone home? without asking first, and if you do ask your users for permission, many of them (perhaps most) will say no. So, you still won?t really have good statistics. Bill > Can anyone recommend something? > -- > https://mail.python.org/mailman/listinfo/python-list From douglashf.mec at gmail.com Sun Oct 8 23:25:34 2017 From: douglashf.mec at gmail.com (douglashf.mec at gmail.com) Date: Sun, 8 Oct 2017 20:25:34 -0700 (PDT) Subject: Python GUI application embedding a web browser - Options? In-Reply-To: References: <1c4488d1-aafa-41c3-a2aa-8bd01837dc2f@googlegroups.com> <8ea166e0-28ed-4289-b88b-b6f03ff4f6aa@googlegroups.com> Message-ID: Did you find out the answer for that? From mikhailwas at gmail.com Mon Oct 9 00:35:15 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Mon, 9 Oct 2017 06:35:15 +0200 Subject: The "loop and a half" Message-ID: bartc wrote: >> But as it happens, I could make computers talk to each when I was working >> with microprocessors, using home-made interfaces, rs232 or rs423. I wouldn't >> know how to do it now because it depends on other people's over-complex >> tech. Chris Angelico wrote: > I don't know if you're an idiot or a troll. Using TCP/IP networking is > pretty simple (at least if you're using a real language - your own toy > languages might have made it unnecessarily hard, for all I know), > hardly "over-complex" by comparison to RS-232 programming. I suppose he is a programmer who is just not much interested in networking and what you can do in beloved console. And if you have remote editing you still need to work in a line by line input? Just for people like me who know nothing about networking, can you popularly explain the : > Have you ever worked on a slow remote session where a GUI is > completely impracticable (or maybe even unavailable), and redrawing > the screen is too expensive to do all the time? So where does the redrawing happen? The machine youre sitting on (let's call it 'A') and send remote commands or retrieving text files? Or the redrawing must be synced on both A and the remote machine? If so, then why so? How does the bandwidth implies that you must edit stuff in the console on A? And not in a nice editor with normal fonts? Am i missing something or your 'A' machine cannot use graphics? Even on 386 computers there was graphics and keybord&mouse input. That is definitely what I would want for editing files. Yes I've tried line by line eding back in DOS times and that really sucks. Mikhail From rosuav at gmail.com Mon Oct 9 00:49:29 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 15:49:29 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 3:35 PM, Mikhail V wrote: >> Have you ever worked on a slow remote session where a GUI is >> completely impracticable (or maybe even unavailable), and redrawing >> the screen is too expensive to do all the time? > > So where does the redrawing happen? The machine youre sitting on (let's > call it 'A') and send remote commands or retrieving text files? Or the > redrawing must be synced on both A and > the remote machine? If so, then why so? > How does the bandwidth implies that you must edit stuff in the console on > A? > And not in a nice editor with normal fonts? > Am i missing something or your 'A' machine cannot use graphics? Even on 386 > computers > there was graphics and keybord&mouse input. That is definitely what I would > want > for editing files. Yes I've tried line by line eding back in DOS times and > that really sucks. Mostly, I use an SSH session without X11 forwarding, so everything happens on that link. Redrawing happens on "A", and the program runs on "B". It is technologically possible to have a GUI (that's what X11 forwarding is for), but it's a lot more fiddliness and bandwidth, and it requires that "B" have the appropriate GUI libraries installed, so I often don't or can't do that. Generally, my preferred editor is nano, since it lives within those requirements but still has a decent UI. It's not always available though, and it's useful to know how to manage without it. But even though you won't always be doing this sort of thing, it's definitely something that a *programming language designer* should be aware of. Basic networking is critical to any modern language. ChrisA From greg.ewing at canterbury.ac.nz Mon Oct 9 01:15:14 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 09 Oct 2017 18:15:14 +1300 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: Grant Edwards wrote: > Which took it from RSX-11. Or probably more specifically from > FILES-11. I woldn't be surprised if the enineers at DEC got it from > somewhere else before that. Quite possibly it goes back to the very earliest DEC OS that had files, whatever that was. The reason for it was that the file system only kept track of file sizes in blocks, not bytes, so some way was needed to mark the end of a text file part way through a block. -- Greg From ian.g.kelly at gmail.com Mon Oct 9 01:17:37 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 8 Oct 2017 23:17:37 -0600 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Sun, Oct 8, 2017 at 10:49 PM, Chris Angelico wrote: > On Mon, Oct 9, 2017 at 3:35 PM, Mikhail V wrote: >>> Have you ever worked on a slow remote session where a GUI is >>> completely impracticable (or maybe even unavailable), and redrawing >>> the screen is too expensive to do all the time? >> >> So where does the redrawing happen? The machine youre sitting on (let's >> call it 'A') and send remote commands or retrieving text files? Or the >> redrawing must be synced on both A and >> the remote machine? If so, then why so? >> How does the bandwidth implies that you must edit stuff in the console on >> A? >> And not in a nice editor with normal fonts? >> Am i missing something or your 'A' machine cannot use graphics? Even on 386 >> computers >> there was graphics and keybord&mouse input. That is definitely what I would >> want >> for editing files. Yes I've tried line by line eding back in DOS times and >> that really sucks. > > Mostly, I use an SSH session without X11 forwarding, so everything > happens on that link. Redrawing happens on "A", and the program runs > on "B". It is technologically possible to have a GUI (that's what X11 > forwarding is for), but it's a lot more fiddliness and bandwidth, and > it requires that "B" have the appropriate GUI libraries installed, so > I often don't or can't do that. Or you could use a GUI editor that runs locally and has the capability to edit files remotely over ssh. From rosuav at gmail.com Mon Oct 9 01:24:31 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 16:24:31 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 4:17 PM, Ian Kelly wrote: > On Sun, Oct 8, 2017 at 10:49 PM, Chris Angelico wrote: >> On Mon, Oct 9, 2017 at 3:35 PM, Mikhail V wrote: >>>> Have you ever worked on a slow remote session where a GUI is >>>> completely impracticable (or maybe even unavailable), and redrawing >>>> the screen is too expensive to do all the time? >>> >>> So where does the redrawing happen? The machine youre sitting on (let's >>> call it 'A') and send remote commands or retrieving text files? Or the >>> redrawing must be synced on both A and >>> the remote machine? If so, then why so? >>> How does the bandwidth implies that you must edit stuff in the console on >>> A? >>> And not in a nice editor with normal fonts? >>> Am i missing something or your 'A' machine cannot use graphics? Even on 386 >>> computers >>> there was graphics and keybord&mouse input. That is definitely what I would >>> want >>> for editing files. Yes I've tried line by line eding back in DOS times and >>> that really sucks. >> >> Mostly, I use an SSH session without X11 forwarding, so everything >> happens on that link. Redrawing happens on "A", and the program runs >> on "B". It is technologically possible to have a GUI (that's what X11 >> forwarding is for), but it's a lot more fiddliness and bandwidth, and >> it requires that "B" have the appropriate GUI libraries installed, so >> I often don't or can't do that. > > Or you could use a GUI editor that runs locally and has the capability > to edit files remotely over ssh. That's also a possibility, but I have yet to find one that can SSH to a server as a non-root user and then sudo to edit the files. If I simply run everything over SSH, I can "sudo -e /etc/some-file" and it'll manage that side of things for me. Of course, there is another option, and one that I'm using increasingly often these days: edit files locally, commit to git, and then "git pull" on the remote system. :) ChrisA From lele at metapensiero.it Mon Oct 9 02:00:34 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 09 Oct 2017 08:00:34 +0200 Subject: The "loop and a half" References: Message-ID: <87tvz87tdp.fsf@metapensiero.it> Chris Angelico writes: >> Or you could use a GUI editor that runs locally and has the capability >> to edit files remotely over ssh. > > That's also a possibility, but I have yet to find one that can SSH to > a server as a non-root user and then sudo to edit the files. If it's just a matter of "finding one", look no further and try out Emacs's TRAMP :-) ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From arj.python at gmail.com Mon Oct 9 02:40:01 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 9 Oct 2017 10:40:01 +0400 Subject: on = style In-Reply-To: References: Message-ID: hi just a quick question, why is my_pens = 4 my_pencils = 5 is preffered to my_pens = 4 my_pencils = 5 *referring to = symbol alignment tk ! Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com From rosuav at gmail.com Mon Oct 9 02:49:14 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 17:49:14 +1100 Subject: on = style In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 5:40 PM, Abdur-Rahmaan Janhangeer wrote: > hi just a quick question, why is > > my_pens = 4 > my_pencils = 5 > > is preffered to > > my_pens = 4 > my_pencils = 5 > > *referring to = symbol alignment Because when you add a new variable: my_mousepads = 6 you then have to add extra spaces to each of the other lines. That's spurious changes in a diff, unnecessary edits that you have to make, and lots of pointless work. ChrisA From hjp-usenet3 at hjp.at Mon Oct 9 03:06:04 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Mon, 9 Oct 2017 09:06:04 +0200 Subject: The "loop and a half" References: Message-ID: On 2017-10-09 04:35, Mikhail V wrote: > Just for people like me who know nothing about networking, > can you popularly explain the : > >> Have you ever worked on a slow remote session where a GUI is >> completely impracticable (or maybe even unavailable), and redrawing >> the screen is too expensive to do all the time? > > So where does the redrawing happen? The machine youre sitting on (let's > call it 'A') and send remote commands or retrieving text files? Or the > redrawing must be synced on both A and > the remote machine? If so, then why so? Because you want to see what you are typing. Imagine you are logged into a server on the other side of the world. You want to edit a file on that machine (for example a configuration file for the web server). You invoke the editor on that server. Now everything the editor wants to display must be sent to your machine so that it can display it on your screen, and everything you do (hit a key, move the mouse) must be sent to the remote machine so that the editor can obey your commands. It is extremely irritating if there is a noticable delay between action and response, so the data must be transferred in both directions quickly. > How does the bandwidth implies that you must edit stuff in the console on > A? I'm not sure what you mean be "in the console on A". What you usually do is invoke an SSH client on A ("ssh" on Linux, "putty" on Windows) to connect to the server and then invoke the editor there. > And not in a nice editor with normal fonts? > Am i missing something or your 'A' machine cannot use graphics? Your A machine can use graphics. The server may not be able to (it's a server, why would anyone install a GUI on it?), and even if it could, streaming the screen contents of a rich GUI around the world may be not be possible for bandwidth or delay reasons. What you could do is copy the file from the server to your computer, edit it there and then copy it back. But that's two extra steps and it may not even be possible in some cases (maybe you can't reach the server directly, but only over one or more bastion hosts). > Even on 386 computers there was graphics and keybord&mouse input. That > is definitely what I would want for editing files. Yes I've tried line > by line eding back in DOS times and that really sucks. There is a wide gap between line editing and graphical editors. Text-terminal oriented editors (like vi, emacs, nano in the Unix world or KEdit or e4 in the MS-DOS world) are not line based. They use the full screen, and they can often also use the mouse. They are just restricted to a rectangular grid of characters for display purposes. Not much of a restriction if you want to edit only text. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From dieter at handshake.de Mon Oct 9 03:08:23 2017 From: dieter at handshake.de (dieter) Date: Mon, 09 Oct 2017 09:08:23 +0200 Subject: python multiprocessing References: Message-ID: <87376sbxy0.fsf@handshake.de> Xristos Xristoou writes: > I have three functions in the python that each one puts an image (image path) as input and makes a simple image processing and creates a new image (image path) as output. In order to make effective use of multiprocessing, you need to split your complete task into (mostly) independent subtasks and let them be processed by subprocesses. For this to be effective, the subtasks must need significant processing. If dependencies remain, the processes must communicate which makes things more complicated and less efficient. I agree with Stefan Ram: if you have essentially a single image which gets processed in order by different alorithms, then you have a sequential process which can not profit from multiprocessing. However, if you have a set of input images, then you can process each image in a separate process -- and potentially gain significantly in speed. From steve+python at pearwood.info Mon Oct 9 03:46:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 09 Oct 2017 18:46:56 +1100 Subject: on = style References: Message-ID: <59db2971$0$14948$b1db1813$d948b532@news.astraweb.com> On Mon, 9 Oct 2017 05:56 pm, Stefan Ram wrote: > Abdur-Rahmaan Janhangeer writes: >>hi just a quick question, why is > > When you post a new question, you should start a new thread. > Your post already has many other posts in the References header. > >>my_pens = 4 >>my_pencils = 5 >>is preffered to >>my_pens = 4 >>my_pencils = 5 > > "preferred" > > The rationale is not clear to me. Could be: > > abc = 2 > def = 3 > ghi = 4 > jkl = 5 > mno = 6 > pqr = 7 > stu = 8 > vwx = 9 > abcdefghijklmnopqrstuvwxyzabcdefghij = 0 And then when you add another line: abcdefghijklmnopqrstuvwxyzabcdefghij???????????? = 1 you have to waste time lining everything up again: abcdefghijklmnopqrstuvwxyzabcdefghij[????????????] = 1 abc = 2 def = 3 ghi = 4 jkl = 5 mno = 6 pqr = 7 stu = 8 vwx = 9 abcdefghijklmnopqrstuvwxyzabcdefghij = 0 And then later you decide that's a damn ridiculously long line, so you split it over two: abcdefghijklmnopqrstuvwxyzabcdefghij[ ????????????] = 1 abc = 2 def = 3 ghi = 4 jkl = 5 mno = 6 pqr = 7 stu = 8 vwx = 9 abcdefghijklmnopqrstuvwxyzabcdefghij = 0 so now you waste MORE time aligning things: abcdefghijklmnopqrstuvwxyzabcdefghij[ ????????????] = 1 abc = 2 def = 3 ghi = 4 jkl = 5 mno = 6 pqr = 7 stu = 8 vwx = 9 abcdefghijklmnopqrstuvwxyzabcdefghij = 0 and so it continues. Not only are wasting time, but you make the version control diffs less useful. And the end result is, it becomes much easier to misread one of the lines and accidentally read "mno = 7" instead of 6 and get yourself confused. Assignment values should be close to their variable, not waaaaaaaaaaaaaaaaaay over on the far side of the line. > The distance between ?jkl? and ?5? might make it more > difficult to read which value belongs to which name. There is no "might", it definitely makes reading harder. > Also, sometimes people like to use automated formatting > tools, and they will usually remove such formatting anyway. > > And possibly some people think that it must take time > of the programmer to create and maintain such an alignment, > time in which he is not programming "productively". There's no question about that. I've done it myself, spent five minutes carefully lining up (for example) a dict and arranging things Just Perfectly: mydict = dict(a = 1234, b = 123, c = 123, de = 12, fgh = 1, ij = 12345, klm = 123456789, n = 12345678901, opq = 1, ) and then half an hour later, decide that, no, it is better to sort the items in rows, not columns, so then I spend another ten minutes re-arranging my Perfectly Laid Out dict: mydict = dict(a = 1234, de = 12, klm = 123456789, b = 123, fgh = 1, n = 12345678901, c = 123, ij = 12345, opq = 1, } and then the next day I decide that, no, it was better the first way, so I spend another ten minutes re-aligning the rows and columns. And *then* I decide that some of the keys are badly named, and I need to change them, and that requires another re-alignment. And then I decide that I need another key, so its no longer a perfect 3x3 square. Should I re-arrange it to a 5x2 or 2x5 rectangle? Obviously I must try both. (If you think I am exaggerating or making this up, I'm not.) Eventually, after spending multiple hours (spread over a couple of days) dicking around, I finally cannot avoid the truth any more, and have to admit to myself that I'm just dicking around and this isn't productive work, it's busy-work to make me look busy while actually avoiding doing anything important but necessary, like fixing a bug. Its even less productive and less useful than spending hours dicking around on Usenet arguing about the number of angels that can dance on a reference. Aligning assignments like this is an easy no-brainer activity. You are avoiding doing anything *hard* like fixing bugs, writing documentation, or adding new features, while actually doing something mindlessly simple that takes no real effort. I truly believe that, with *very* few exceptions (*ALL* of which are some form of in-line data table, and even then only rarely) any programmer who worries about lining up assignments on different lines like this is just engaged in a time-wasting exercise to make themselves look busy while actually taking it easy. It is a pure avoidance activity. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From mikhailwas at gmail.com Mon Oct 9 04:05:47 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Mon, 9 Oct 2017 10:05:47 +0200 Subject: The "loop and a half" Message-ID: bartc wrote: >> Your job is to port an editor that people have been using for 30 years to >> Linux. The first thing you do is to change all the commands and shortcuts to >> match what is typical on Linux? So that no-one who was familiar with it as >> it was can actually use it? Chris Angelico wrote: > Is it graphical? Does it use any well-known toolkit? If so, then yeah, > the first thing - and the automatic thing - is that it will respond to > the host platform's keys and so on. In case of a toolkit probably, but generally there is no such thing as "host platform's keys" in context of a desktop application. There are keys which the application defines. OS is used only to run the application, (some system keys, e.g. Alt-Tab, Ctrl-Alt-del, are of course better left alone). The first thing a developer should provide - the keys and mouse input should be *customizable* by the user. It is so by most serious application I have ever used. (I don't know though if it is so in Bart's application) Anything that can not - is for me 'below the bar' usability straightaway. (With the exception when the defaults are so good that there is no much need to redefine them, or it is a very simple app with only few input commands) Some frameworks do not provide such customization and that is simply bad design. The second thing - the default keys should be optimized for the input efficiency and ergonomics. Other issues, e.g. whether the keys are same as in some other app or not, becomes irrelevant if the app fulfils the first two criteria (see above). If not, I would consider looking for an alternative app with customizable keys. bartc wrote: >> * Under Windows, if you press Shift while Caps Lock is on, you get lower >> case letters. I've never, ever wanted to do this (probably no one else has). >> My own editor doesn't obey that convention: shift-A will always come out as >> 'A' whatever the caps lock setting. >> >> There are dozens more, yet you are surprised why sometimes I prefer doing >> things my own way? There are good reasons! Chris Angelico wrote: >Yep. Good reasons like that you're a moron. You assume that > since *you* have never needed to produce one lower-case letter > in a block of upper-case, that "probably no one else has", > and then you make it impossible to do that in your editor. > I have wanted to produce a lower-case letter by holding Shift. I think I've never advocated anybody on the list, but here I really don't understand why you are attacking Bart. In this particular Caps lock example - it's pretty clear for me that the input of lowercase letters with Shift is historically there just for the 'symmetry'. Obviously there is zero practical need for that in general case, so "probably no one else need this" is quite appropriate statement. > I have wanted to produce a lower-case letter by holding Shift. Ok, once in my life I have wanted too, just to see what happens, which does not mean that this feature has any value. Mikhail From p.f.moore at gmail.com Mon Oct 9 04:20:12 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Oct 2017 09:20:12 +0100 Subject: OT again sorry [Re: Interactive scripts (back on topic for once) [was Re: The "loop and a half"]] In-Reply-To: <87a811io94.fsf@elektro.pacujo.net> References: <87y3opw6c0.fsf@elektro.pacujo.net> <59d6d76d$0$14947$b1db1813$d948b532@news.astraweb.com> <878tgp3vlj.fsf@bsb.me.uk> <59d73a2e$0$14941$b1db1813$d948b532@news.astraweb.com> <87d160skmv.fsf@elektro.pacujo.net> <59d7756d$0$14931$b1db1813$d948b532@news.astraweb.com> <87a811io94.fsf@elektro.pacujo.net> Message-ID: On 8 October 2017 at 17:43, Marko Rauhamaa wrote: > It is not at all easy for the Linux user to figure out what > configuration options there are, and which ones are intended for > end-user configuration. More and more, such tuning needs to be > done via systemd unit files (or applicable GUI facilities) and the > classical configuration files are deprecated. For example, how can a > programmer get a core file of a crashing program? Why, you need to use > the systemd-coredump service, of course: One of the things I liked about Debian many years ago when I played with Linux (when the options available were Debian, Red Hat, Slackware and SuSE and that was about it) was that they typically "fixed" the defaults of programs in the build options, so that there were almost no config files in the default install. That made it pretty easy for a user - you just set any extra options you want. With the other distros, they tended to make changes via config files, which was probably more transparent and easier to understand, but meant that a naive user like me couldn't tell what I was "allowed" to change (and by "allowed" I don't mean permission, more "if I change this, will I end up spending days trying to work out what weird interaction with other tools'expectations I just broke"). Sadly, those simpler days are long gone, and nowadays all Linux distros as far as I can see have a mass of predefined config (and the inevitable "config manager" tools to manage them). Not that I can complain about this, as a Windows user, but I do have fond memories of those simpler times :-) Obligatory xkcd: https://xkcd.com/297/ Paul From p.f.moore at gmail.com Mon Oct 9 04:24:24 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Oct 2017 09:24:24 +0100 Subject: Python GUI application embedding a web browser - Options? In-Reply-To: References: <1c4488d1-aafa-41c3-a2aa-8bd01837dc2f@googlegroups.com> <8ea166e0-28ed-4289-b88b-b6f03ff4f6aa@googlegroups.com> Message-ID: On 9 October 2017 at 04:25, wrote: > Did you find out the answer for that? Nothing much beyond the pointer to PyQt (which basically said "a lot of the info on the web is out of date" so I should check the latest docs). I didn't take it much further, though, as it was a hobby project and the learning curve for PyQt (any GUI framework, really) was a bit high for the amount of spare time I had at the time. Paul From alister.ware at ntlworld.com Mon Oct 9 05:15:58 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 09 Oct 2017 09:15:58 GMT Subject: The "loop and a half" Message-ID: On Mon, 09 Oct 2017 08:00:34 +0200, Lele Gaifax wrote: > Chris Angelico writes: > >>> Or you could use a GUI editor that runs locally and has the capability >>> to edit files remotely over ssh. >> >> That's also a possibility, but I have yet to find one that can SSH to a >> server as a non-root user and then sudo to edit the files. > > If it's just a matter of "finding one", look no further and try out > Emacs's TRAMP :-) > > ciao, lele. or if you want the luxury of a GUI editor simply ssh to the remote machine & run the editor there (using X forwarding to route the display to you local PC) -- The more you sweat in peace, the less you bleed in war. From rosuav at gmail.com Mon Oct 9 05:19:35 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 20:19:35 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 7:05 PM, Mikhail V wrote: > The first thing a developer should provide - the keys and mouse input > should be > *customizable* by the user. It is so by most serious application I have > ever used. And they most certainly are. Often, in something in the host platform. For instance, Xfce allows me to change the keys used for various purposes, by going into the menu and choosing "Settings" and then "Keyboard". (Or, of course, by editing the config files.) The most obvious and dramatic change you can do is to choose a specific keyboard layout - Qwerty vs Dvorak, US vs German, etc, etc, etc. Do you think that sort of thing should be in the hands of the application, or the platform? (Note that I'm not saying "OS" here. Usually the platform is at a higher level than that; in my case, I would consider Xfce (my desktop manager) to be that.) ChrisA From rosuav at gmail.com Mon Oct 9 05:23:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 20:23:16 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 8:15 PM, alister via Python-list wrote: > On Mon, 09 Oct 2017 08:00:34 +0200, Lele Gaifax wrote: > >> Chris Angelico writes: >> >>>> Or you could use a GUI editor that runs locally and has the capability >>>> to edit files remotely over ssh. >>> >>> That's also a possibility, but I have yet to find one that can SSH to a >>> server as a non-root user and then sudo to edit the files. >> >> If it's just a matter of "finding one", look no further and try out >> Emacs's TRAMP :-) >> >> ciao, lele. > > or if you want the luxury of a GUI editor simply ssh to the remote > machine & run the editor there (using X forwarding to route the display > to you local PC) That assumes that you have a GUI editor installed on the server, with all the libraries it requires. Not always the case - especially if you have multiple admins, some who like GNOME and some who like KDE... how many editors and how many subsystems are you going to install? ChrisA From marko at pacujo.net Mon Oct 9 05:38:38 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 09 Oct 2017 12:38:38 +0300 Subject: The "loop and a half" References: Message-ID: <87po9whd9d.fsf@elektro.pacujo.net> alister : > or if you want the luxury of a GUI editor simply ssh to the remote > machine & run the editor there (using X forwarding to route the > display to you local PC) I could be doing it at this very moment. However X11 networking is so clunky that I choose to do it in an SSH text terminal. There is barely any difference in using emacs in text mode than graphically. Marko From mikhailwas at gmail.com Mon Oct 9 05:44:26 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Mon, 9 Oct 2017 11:44:26 +0200 Subject: The "loop and a half" Message-ID: >>> Have you ever worked on a slow remote session where a GUI is >>> completely impracticable (or maybe even unavailable), and redrawing >>> the screen is too expensive to do all the time? >> >> So where does the redrawing happen? The machine youre sitting on (let's >> call it 'A') and send remote commands or retrieving text files? Or the >> redrawing must be synced on both A and >> the remote machine? If so, then why so? Peter J. Holzer wrote: > Because you want to see what you are typing. Imagine you are logged into > a server on the other side of the world. You want to edit a file on that > machine (for example a configuration file for the web server). You > invoke the editor on that server. I've never done this, but if one told me to edit a file, my first attempt would be like: - get read/write access to the file system - browse it (e.g. some file browser plugin?) - get the file to the local machine - edit locally - write it back to the remote machine >> And not in a nice editor with normal fonts? >> Am i missing something or your 'A' machine cannot use graphics? > ... The server may not be able to > (it's a server, why would anyone install a GUI on it?) If I ever work on it (locally) why would I want a GUI on it? o_O I'm not sure if I'm getting you. You mean probably a server which is never worked on locally? If it has a display and a keyb, and I must do something on it, even seldom, then certainly I want a GUI on it (not just to see a desktop wallpaper ;). > streaming the screen contents of a rich GUI around the world may be not > be possible for bandwidth or delay reasons. Sure, streaming video consumes a lot, but that is what one tries to avoid if under limitations so streaming messages only is faster. Mikhail From rosuav at gmail.com Mon Oct 9 06:07:43 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 9 Oct 2017 21:07:43 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 8:44 PM, Mikhail V wrote: >> ... The server may not be able to >> (it's a server, why would anyone install a GUI on it?) > > If I ever work on it (locally) why would I want a GUI on it? (Presuming you mean "wouldn't" here) > o_O I'm not sure if I'm getting you. > You mean probably a server which is never worked on locally? > If it has a display and a keyb, and I must do something on it, even > seldom, then certainly I want a GUI on it (not just to see > a desktop wallpaper ;). If it's a server that also is used directly at its own console, then sure! Have a GUI installed. Right now, for instance, I'm typing this up on my main computer "Sikorsky", who is a full desktop system with four monitors, two webcams, a keyboard, a mouse, and several dozen figurines (they aren't USB-connected or anything, but without Alice, Elsa, Kaylee, and Joy, this just wouldn't be the same). He happens to be a server, providing various facilities to the local network, but he is a client too, so he has a GUI. (Though if I remote in to him from the laptop, I'm *still* going to stick to the terminal, just because it's easier that way.) But there are plenty of servers that don't. You can rent yourself a cheap box on Amazon's EC2 for less than a cent per hour - not much of a box, but enough to do some testing on. It's an easy and cheap way to try things out. But since that box has something like half a gig of RAM, you don't want to waste any of it on a GUI; you want to work with the simplest, lightest-weight user interface you can. That means the "glass teletype" of the terminal; all the fancy features are provided by your client, where you run "ssh some-server" in a local terminal emulator, and the server just has to worry about running the programs themselves. ChrisA From bc at freeuk.com Mon Oct 9 06:43:16 2017 From: bc at freeuk.com (bartc) Date: Mon, 9 Oct 2017 11:43:16 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: <4pICB.954012$284.440000@fx20.am4> On 09/10/2017 06:15, Gregory Ewing wrote: > Grant Edwards wrote: >> Which took it from RSX-11.? Or probably more specifically from >> FILES-11.? I woldn't be surprised if the enineers at DEC got it from >> somewhere else before that. > > Quite possibly it goes back to the very earliest DEC OS > that had files, whatever that was. > > The reason for it was that the file system only kept track > of file sizes in blocks, not bytes, so some way was needed > to mark the end of a text file part way through a block. Ctrl-Z was just code 26 or [what I've always called] ETX or end-of-text. I'd used DEC quite a bit (including rsx-11m) but only became aware of ETX when using CP/M. (Where it had a annoying habit, IIRC, of copying files in text mode so that it stopped at the first ETX byte.) And actually, until recently, I added ETX (followed by 0 for good measure) as a sentinel in a function reading a file into a block of memory (now I just use 0 for that purpose). But I don't consider that these internal uses, some of which are archaic, belong in a user interface. [Looking it up now, actual ETX is code 3, or Ctrl-C, and EOT is code 4. So how 26 - apparently code SUB, whatever that means - ended up being used to mark the end of text files, I don't know.] -- bartc From mal at europython.eu Mon Oct 9 07:51:50 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Mon, 9 Oct 2017 13:51:50 +0200 Subject: EuroPython 2017: Videos for Tuesday available online Message-ID: <55812807-44f8-90db-7195-3bab6c9d7865@europython.eu> We are pleased to announce the second batch of cut videos for EuroPython 2017. To see the new videos, please head over to our EuroPython YouTube channel and select the "EuroPython 2017" playlist. The new videos start at entry 31 in the playlist. * EuroPython 2017 Videos * http://europython.tv/ In the coming weeks, we will continue to release the other videos currently marked as ?private?, in batches of one conference day per week. Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/916690691224473601 Thanks. From bc at freeuk.com Mon Oct 9 08:43:19 2017 From: bc at freeuk.com (bartc) Date: Mon, 9 Oct 2017 13:43:19 +0100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On 09/10/2017 05:49, Chris Angelico wrote: > Generally, my preferred editor is nano, since it lives within those > requirements but still has a decent UI. It's not always available > though, and it's useful to know how to manage without it. But even > though you won't always be doing this sort of thing, it's definitely > something that a *programming language designer* should be aware of. > Basic networking is critical to any modern language. That's an odd opinion given that modern languages don't even want to have basic input and output as part of the language; they're offloaded to a library. And in your example, which I don't really understand, it seems more like the headache of the application. (Where a language allows elements of a program to be disseminated across a network, that's rather different, and an advanced feature that only some languages might have. Not all.) -- bartc From rosuav at gmail.com Mon Oct 9 09:08:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 00:08:58 +1100 Subject: The "loop and a half" In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 11:43 PM, bartc wrote: > On 09/10/2017 05:49, Chris Angelico wrote: > >> Generally, my preferred editor is nano, since it lives within those >> requirements but still has a decent UI. It's not always available >> though, and it's useful to know how to manage without it. But even >> though you won't always be doing this sort of thing, it's definitely >> something that a *programming language designer* should be aware of. >> Basic networking is critical to any modern language. > > > That's an odd opinion given that modern languages don't even want to have > basic input and output as part of the language; they're offloaded to a > library. And in your example, which I don't really understand, it seems more > like the headache of the application. > > (Where a language allows elements of a program to be disseminated across a > network, that's rather different, and an advanced feature that only some > languages might have. Not all.) The standard libraries of most application languages include both console I/O and networking. ("Application language" excludes such things as JavaScript in a web browser, which for security reasons is not permitted file system access, and has limited console and network features. But Node.js has all of the above.) The distinction between language syntax and the standard library is not overly significant here; things that don't need to be syntax are best implemented with the standard library. We're not talking about third-party libraries here - with a vanilla CPython 3.6 and no external libraries, you can access the console and you can open a socket. (And you get higher-level facilities too - protocols like HTTP, FTP, DNS, and SMTP are handled too.) ChrisA From grant.b.edwards at gmail.com Mon Oct 9 10:16:49 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 9 Oct 2017 14:16:49 +0000 (UTC) Subject: The "loop and a half" References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-09, Gregory Ewing wrote: > Grant Edwards wrote: >> Which took it from RSX-11. Or probably more specifically from >> FILES-11. I woldn't be surprised if the enineers at DEC got it from >> somewhere else before that. > > Quite possibly it goes back to the very earliest DEC OS > that had files, whatever that was. > > The reason for it was that the file system only kept track > of file sizes in blocks, not bytes, so some way was needed > to mark the end of a text file part way through a block. A "feature" that CP/M copied almost verbatim. If you looked at the CP/M FCB (file-control-block) structure it looked almost identical to that used by early DEC OSes. DOS then copied CP/M's file i/o scheme even more exactly than CP/M had copied DEC. It was years later than DOS finally copied some Unix features like nested directories, opening files via path strings, etc. -- Grant Edwards grant.b.edwards Yow! Do you think the at "Monkees" should get gas on gmail.com odd or even days? From grant.b.edwards at gmail.com Mon Oct 9 10:22:44 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 9 Oct 2017 14:22:44 +0000 (UTC) Subject: The "loop and a half" References: Message-ID: On 2017-10-09, alister via Python-list wrote: > or if you want the luxury of a GUI editor simply ssh to the remote > machine & run the editor there (using X forwarding to route the display > to you local PC) AFAICT, most modern GUI toolkits are no longer usable via X forwarding at sub-gigabit network speeds. The toolkit designers have botched things up so that even the most trivial operation requires hundreds of round-trips between server and client. -- Grant Edwards grant.b.edwards Yow! Did YOU find a at DIGITAL WATCH in YOUR box gmail.com of VELVEETA? From marko at pacujo.net Mon Oct 9 10:27:27 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 09 Oct 2017 17:27:27 +0300 Subject: The "loop and a half" References: Message-ID: <87po9wflbk.fsf@elektro.pacujo.net> Grant Edwards : > On 2017-10-09, alister via Python-list wrote: > >> or if you want the luxury of a GUI editor simply ssh to the remote >> machine & run the editor there (using X forwarding to route the >> display to you local PC) > > AFAICT, most modern GUI toolkits are no longer usable via X forwarding > at sub-gigabit network speeds. The toolkit designers have botched > things up so that even the most trivial operation requires hundreds of > round-trips between server and client. Yep. Funny thing is, xterm runs nicely over a 9,600-baud line, but there's no hope to get Firefox, Evince or the like to run over a 1,500,000 bps connection. Latency is more of an issue than throughput, indicating that those round-trips are sequential. X11 was designed to be pipelined but the toolkits can't pipeline themselves. Marko From hcchen5600 at gmail.com Mon Oct 9 10:30:15 2017 From: hcchen5600 at gmail.com (H.C. Chen) Date: Mon, 9 Oct 2017 07:30:15 -0700 (PDT) Subject: A programmable python debugger. Set one breakpoint to x-ray everything Message-ID: <7542d617-5ea9-4977-b733-abd973a2bfca@googlegroups.com> peforth If the below explanations would be messy please directly refer to README.md @ https://github.com/hcchengithub/peforth A programmable python debugger. Set one breakpoint to x-ray everything. You guys know how to bebug already. We all do. But when it comes to Machine Learning and Tensorflow or the likes, things are getting annoying if we were still using traditional debuggers. A programmable debugger is what in my mind and probably in yours too. One breakpoint to investigate about everything with procedures that we come out at the point depend on variant needs of emerged ideas good or bad. Debug commands in FORTH syntax So now we need to choose an interactive UI and its syntax that is light weight, reliable and flexible so we won't regret of choosing it someday, has been there for decades so many people don't need to learn about another new language although we are only to use some debug commands, yet easy enough for new users, that's FORTH. Install peforth: pip install peforth Run peforth: Print "Hello World!" Microsoft Windows [Version 10.0.15063] (c) 2017 Microsoft Corporation. All rights reserved. c:\Users\your-working-folder>python -m peforth .' Hello World!!' cr bye Hello World!! c:\Users\your-working-folder> so your peforth has been working fine. To your application, import peforth as usual to bring in the debugger: c:\Users\your-working-folder>python Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import peforth p e f o r t h v1.07 source code http://github.com/hcchengithub/peforth Type 'peforth.ok()' to enter forth interpreter, 'exit' to come back. >>> The greeing message tells us how to enter the FORTH interpreter for your debugging or investigating and how to come back to continue running your code. Let's try to debug a program # 100.py sum = 0 for i in range(100): sum += i print("The sum of 1..100 is ", sum) Run it: c:\Users\your-working-folder>python 100.py The sum of 1..100 is 4950 c:\Users\your-working-folder> The result should be 5050 but it's not! Let's drop a breakpoint to see what's wrong: # 100.py with breakpoing .----- Specify an unique command prompt to indicate where | the breakpoint is from if there are many of them import peforth | .----- pass locals() at the breakpoint sum = 0 | | to our debugger for i in range(100): | | .------- use a FORTH constant sum += i | | | to represent the locals() peforth.ok('my first breakpoint> ',loc=locals(),cmd="constant locals-after-the-for-loop") print("The sum of 1..100 is ", sum) Run again: c:\Users\your-working-folder>python 100.py p e f o r t h v1.07 source code http://github.com/hcchengithub/peforth Type 'peforth.ok()' to enter forth interpreter, 'exit' to come back. .--------------- at the breakpoint, type in 'words' | command to see what have we got my first breakpoint> words .-------- It's a long list of 'words' ... snip ....... | or available commands. Don't worry, we'll use only some of them. expected_rstack expected_stack test-result [all-pass] *** all-pass [r r] [d d] [p p] WshShell inport OK dir keys --- locals-after-the-for-loop | The last one is what ------' we have just created throuth the breakpoint statement , named "locals-after-the-for-loop" Let's see it: print a carriage return at the end -------. print the thing -----. | | | my first breakpoint> locals-after-the-for-loop . cr ({'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x000001DD2D737710>, '__spec__': None, '__annotations__': {}, '__builtins__': , '__file__': '100.py', '__cached__': None, 'peforth': , 'sum': 4950, 'i': 99}, {}, 'my first breakpoint> ') my first breakpoint> | | | | | '--- our command our sum -----' | prompt | indicates where the 99 instead of 100 ----------' breakpoint is from this is the problem !! Now leave the breakpoint and let the program continue: my first breakpoint> exit my first breakpoint> The sum of 1..100 is 4950 c:\Users\your-working-folder> Investigate by running experiments right at a breakpoint When at a breakpoint in Tensorfow tutorials, I always want to make some experiments on those frustrating tf.something(tf.something(...),...) things to have a clearer understanding of them without leaving the underlying tutorial. Let's use the above example again in another way to demonstrate how to do that with peforth: Run peforth: Microsoft Windows [Version 10.0.15063] (c) 2017 Microsoft Corporation. All rights reserved. c:\Users\hcche\Downloads>python Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import peforth p e f o r t h v1.07 source code http://github.com/hcchengithub/peforth Type 'peforth.ok()' to enter forth interpreter, 'exit' to come back. >>> peforth.ok() OK <-------- Default FORTH command prompt OK Assume we are at a breakpoint and we need a procedure to add 1..100 to get the sum of them. We are not sure if the procedure is correct so we need to try. Now copy the procedure from your text editor. The ... tells the debugger that the code within is a block of in-line python. The outport() function outports the given locals() to the FORTH environment outside the in-line python block. sum = 0 for i in range(100): sum += i print("The sum of 1..100 is ", sum) outport(locals()) It's a block of multiple-line text strings so we press Ctrl-D to start a multiple-line input, copy-paste, and press another Ctrl-D to end the multiple-line block. Like this: OK OK ^D sum = 0 for i in range(100): sum += i print("The sum of 1..100 is ", sum) outport(locals()) ^D The sum of 1..100 is 4950 OK Now use the 'words' command to see what have we got: OK words code end-code \ // bye /// immediate stop compyle trim indent -indent words . cr help interpret-only compile-only literal reveal privacy (create) : ; ( BL CR word ' , [compile] py: py> py:~ py>~ 0branch here! here swap ! @ ? >r r> r@ drop dup over 0< + * - / 1+ 2+ 1- 2- compile if then compiling char last version execute cls private nonprivate (space) exit ret rescan-word-hash (') branch bool and or not (forget) AND OR NOT XOR true false "" [] {} none >> << 0= 0> 0<> 0<= 0>= = == > < != >= <= abs max min doVar doNext depth pick roll space [ ] colon-word create (marker) marker next abort alias <> public nip rot -rot 2drop 2dup invert negate within ['] allot for begin until again ahead never repeat aft else while ?stop ?dup variable +! chars spaces .( ." .' s" s' s` does> count accept accept2 nop refill [else] [if] [then] (::) (:>) :: :> ::~ :>~ "msg"abort abort" "msg"?abort ?abort" ' () (constant) constant value to tib. >t t@ t> [begin] [again] [until] [for] [next] modules int float drops dropall char>ASCII ASCII>char ASCII .s (*debug*) *debug* readTextFile writeTextFile tib.insert sinclude include type obj>keys obj2dict stringify toString .literal .function (dump) dump dump2ret d (see) .members .source see dos cd slice description expected_rstack expected_stack test-result [all-pass] *** all-pass [r r] [d d] [p p] WshShell inport OK dir keys --- i sum OK And the end of the long list after the --- marker we found i and sum. They are all locals() at the point in the in-line python block. Let's see them: OK i . cr 99 OK sum . cr 4950 OK Again, we found the root cause of why the sum is not 5050 because i didn't reach to 100 as anticipated. That's exactly how the python range() works and that has actually confused me many times. Visit this project's Wiki pages for more examples about how to view MNIST handwritten digit images at the half way of your investigating in a Tensorflow tutorial, for example, and the usages of this programmable debugger. Have fun! H.C. Chen, FigTaiwan hcchen_1471 at hotmail.com Just undo it! From alister.ware at ntlworld.com Mon Oct 9 10:35:19 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 09 Oct 2017 14:35:19 GMT Subject: The "loop and a half" References: <87po9wflbk.fsf@elektro.pacujo.net> Message-ID: On Mon, 09 Oct 2017 17:27:27 +0300, Marko Rauhamaa wrote: > Grant Edwards : > >> On 2017-10-09, alister via Python-list wrote: >> >>> or if you want the luxury of a GUI editor simply ssh to the remote >>> machine & run the editor there (using X forwarding to route the >>> display to you local PC) >> >> AFAICT, most modern GUI toolkits are no longer usable via X forwarding >> at sub-gigabit network speeds. The toolkit designers have botched >> things up so that even the most trivial operation requires hundreds of >> round-trips between server and client. > > Yep. > > Funny thing is, xterm runs nicely over a 9,600-baud line, but there's no > hope to get Firefox, Evince or the like to run over a 1,500,000 bps > connection. > > Latency is more of an issue than throughput, indicating that those > round-trips are sequential. X11 was designed to be pipelined but the > toolkits can't pipeline themselves. > > > Marko works fine over my wifi which is considerably less than gigabit speeds then again I only run applications remotely that are sensible to run remotely such as text editors (geany) & file managers (thunar) I cant see any reason why I would want to run a web browser remotely I can see that even this would be too slow on some connections but it invariably works better that Remote desktop which seems to be the preferred approach in the windows world where they don't have much choice. -- It's very glamorous to raise millions of dollars, until it's time for the venture capitalist to suck your eyeballs out. -- Peter Kennedy, chairman of Kraft & Kennedy. From marko at pacujo.net Mon Oct 9 10:47:21 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 09 Oct 2017 17:47:21 +0300 Subject: The "loop and a half" References: <87po9wflbk.fsf@elektro.pacujo.net> Message-ID: <87lgkkfkee.fsf@elektro.pacujo.net> alister : > I cant see any reason why I would want to run a web browser remotely I have had that need occasionally. More recently, I wanted to print a PDF document using evince. It took forever for evince to respond over the WAN. I had to resort to other means. > I can see that even this would be too slow on some connections but it > invariably works better that Remote desktop which seems to be the > preferred approach in the windows world where they don't have much > choice. I can't comment on Windows. However, X11's remote access is hardly usable. The problem is it's too low-level ("mechanism, not a policy"). What we'd need is this setup: +---------+ | client | +---------+ | toolkit | | RPC | +----+----+ | | TCP | +----+----+ | toolkit | | server | +----+----+ | | Local | +------+------+ | Wayland | | compositor | +-------------+ Unfortunately, what we wil be given is: +---------+ | client | +---------+ | toolkit | | lib | +----+----+ | | Local | +------+------+ | Wayland | | compositor | +-------------+ Which will get rid of the network transparency altogether. Marko From grant.b.edwards at gmail.com Mon Oct 9 10:50:04 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 9 Oct 2017 14:50:04 +0000 (UTC) Subject: The "loop and a half" References: <87po9wflbk.fsf@elektro.pacujo.net> Message-ID: On 2017-10-09, Marko Rauhamaa wrote: > Grant Edwards : > >> On 2017-10-09, alister via Python-list wrote: >> >>> or if you want the luxury of a GUI editor simply ssh to the remote >>> machine & run the editor there (using X forwarding to route the >>> display to you local PC) >> >> AFAICT, most modern GUI toolkits are no longer usable via X forwarding >> at sub-gigabit network speeds. The toolkit designers have botched >> things up so that even the most trivial operation requires hundreds of >> round-trips between server and client. > > Yep. > > Funny thing is, xterm runs nicely over a 9,600-baud line, but there's no > hope to get Firefox, Evince or the like to run over a 1,500,000 bps > connection. Most of the "old-school" apps that use athena or motif widgets work fine. There are a a couple more modern toolkits that still work right: the last time I tried an fltk, it seemed OK. But, both GTK and QT are hopeless pigs. Xemacs can be built so that it works fine remotely -- but if you're an emacs user you generally don't care. It's simpler to just run emacs on the remote machine via a normal "tty" connection. > Latency is more of an issue than throughput, That's true, I should have been more precise. > indicating that those round-trips are sequential. X11 was designed > to be pipelined but the toolkits can't pipeline themselves. -- Grant Edwards grant.b.edwards Yow! Am I SHOPLIFTING? at gmail.com From grant.b.edwards at gmail.com Mon Oct 9 11:34:19 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 9 Oct 2017 15:34:19 +0000 (UTC) Subject: The "loop and a half" References: <87po9wflbk.fsf@elektro.pacujo.net> <87lgkkfkee.fsf@elektro.pacujo.net> Message-ID: On 2017-10-09, Marko Rauhamaa wrote: > I can't comment on Windows. However, X11's remote access is hardly > usable. 15 years ago it worked great over 256Kbps DSL links. Even on dialup links it was usable (if a bit clumsy). It's the toolkits that are broken when it comes to remote access. > The problem is it's too low-level ("mechanism, not a policy"). What > we'd need is this setup: > > +---------+ > | client | > +---------+ > | toolkit | > | RPC | > +----+----+ > | > | TCP > | > +----+----+ > | toolkit | > | server | > +----+----+ > | > | Local > | > +------+------+ > | Wayland | > | compositor | > +-------------+ But then you need to re-impliment that for each and every toolkit. With the old X11 scheme, it only need to be implimented once. > Unfortunately, what we wil be given is: > > +---------+ > | client | > +---------+ > | toolkit | > | lib | > +----+----+ > | > | Local > | > +------+------+ > | Wayland | > | compositor | > +-------------+ > > > Which will get rid of the network transparency altogether. For all practial purposes, X11 network transparancy has been gone for years: it only works for apps that nobody cares about. I still use it occasionally just to verify that a remotely upgraded/installed wxPython app will start up, but it's way too slow to actually _use_ the app. -- Grant Edwards grant.b.edwards Yow! I want the presidency at so bad I can already taste gmail.com the hors d'oeuvres. From marko at pacujo.net Mon Oct 9 11:56:08 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 09 Oct 2017 18:56:08 +0300 Subject: The "loop and a half" References: <87po9wflbk.fsf@elektro.pacujo.net> <87lgkkfkee.fsf@elektro.pacujo.net> Message-ID: <87fuasfh7r.fsf@elektro.pacujo.net> Grant Edwards : > On 2017-10-09, Marko Rauhamaa wrote: >> The problem is it's too low-level ("mechanism, not a policy"). What >> we'd need is this setup: >> >> +---------+ >> | client | >> +---------+ >> | toolkit | >> | RPC | >> +----+----+ >> | >> | TCP >> | >> +----+----+ >> | toolkit | >> | server | >> +----+----+ >> | >> | Local >> | >> +------+------+ >> | Wayland | >> | compositor | >> +-------------+ > > But then you need to re-impliment that for each and every toolkit. > With the old X11 scheme, it only need to be implimented once. The solution is that the toolkit RPC sends (or is prepared to send) the toolkit server plugin to the generic remote server. In fact, that's how web pages are implemented using JavaScript (with the roles of the server and the client reversed). Marko From jblack at nopam.com Mon Oct 9 12:22:45 2017 From: jblack at nopam.com (John Black) Date: Mon, 9 Oct 2017 11:22:45 -0500 Subject: Is there a way to globally set the print function separator? Message-ID: I want sep="" to be the default without having to specify it every time I call print. Is that possible? John Black From ned at nedbatchelder.com Mon Oct 9 12:38:17 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 9 Oct 2017 12:38:17 -0400 Subject: Is there a way to globally set the print function separator? In-Reply-To: References: Message-ID: On 10/9/17 12:22 PM, John Black wrote: > I want sep="" to be the default without having to specify it every time I > call print. Is that possible? > > There isn't a way to change the default for print(sep="") globally. Generally when you want better control over the output, you use string formatting.? Instead of: ??? print("X is ", x, " and y is ", y, sep="") use: ??? print("X is {} and y is {}".format(x, y)) If you show us your actual print lines, we can give more concrete advice. --Ned. From python at example.invalid Mon Oct 9 12:40:28 2017 From: python at example.invalid (Python) Date: Mon, 9 Oct 2017 18:40:28 +0200 Subject: Is there a way to globally set the print function separator? References: Message-ID: Le 09/10/2017 ? 18:22, John Black a ?crit?: > I want sep="" to be the default without having to specify it every time I > call print. Is that possible? >>> oldprint = print >>> def print(*args,**kwargs): ... oldprint(*args,**kwargs,sep='') ... >>> print(1,2,3) 123 From arj.python at gmail.com Mon Oct 9 12:40:50 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 9 Oct 2017 20:40:50 +0400 Subject: on = style In-Reply-To: References: Message-ID: hum i see clearly with all my python experience, i have more to learn. i never aligned the = but i found it neat looking but discouraged by pep8. glad i did not switch to it and maintained it for sometimes ! thanks for answers ! Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 9 Oct 2017 10:40, "Abdur-Rahmaan Janhangeer" wrote: > hi just a quick question, why is > > my_pens = 4 > my_pencils = 5 > > is preffered to > > my_pens = 4 > my_pencils = 5 > > *referring to = symbol alignment > > tk ! > > Abdur-Rahmaan Janhangeer, > Mauritius > abdurrahmaanjanhangeer.wordpress.com > From p.f.moore at gmail.com Mon Oct 9 12:41:29 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 9 Oct 2017 17:41:29 +0100 Subject: Is there a way to globally set the print function separator? In-Reply-To: References: Message-ID: On 9 October 2017 at 17:22, John Black wrote: > I want sep="" to be the default without having to specify it every time I > call print. Is that possible? def myprint(*args, **kw): print(*args, sep="", **kw) If you want, assign print=myprint. Paul From __peter__ at web.de Mon Oct 9 12:44:59 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 09 Oct 2017 18:44:59 +0200 Subject: Is there a way to globally set the print function separator? References: Message-ID: John Black wrote: > I want sep="" to be the default without having to specify it every time I > call print. Is that possible? No, but you can replace the print function with your own: >>> print = functools.partial(print, sep="") >>> print("I", "recommend", "you", "choose", "another", "name", "and", "preserve", "your", "sanity") Irecommendyouchooseanothernameandpreserveyoursanity And everybody else's. From tjreedy at udel.edu Mon Oct 9 12:46:53 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 9 Oct 2017 12:46:53 -0400 Subject: Is there a way to globally set the print function separator? In-Reply-To: References: Message-ID: On 10/9/2017 12:22 PM, John Black wrote: > I want sep="" to be the default without having to specify it every time I > call print. Is that possible? > > John Black Define a replacement print function that makes this the default. Something like (untested, a detail may be wrong): _print = print def print(*args, **kwargs): _print(*args, {'sep':''}.update(kwargs)) -- Terry Jan Reedy From arj.python at gmail.com Mon Oct 9 12:49:46 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 9 Oct 2017 20:49:46 +0400 Subject: Finding Old Posts on Python In-Reply-To: <5a7cf063-0a4f-469d-80b4-ff379a598017@googlegroups.com> References: <5a7cf063-0a4f-469d-80b4-ff379a598017@googlegroups.com> Message-ID: download the archive in compressed format. then use notepad++ or sublimetext with option search in files. search for your name ! Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 8 Oct 2017 03:45, "Cai Gengyang" wrote: > Hello, > > Does anyone know of a way to find all my old posts about Python ? Thanks a > lot! > > GengYang > -- > https://mail.python.org/mailman/listinfo/python-list > From nntp.fpp at spamgourmet.com Mon Oct 9 12:50:26 2017 From: nntp.fpp at spamgourmet.com (fpp) Date: 09 Oct 2017 16:50:26 GMT Subject: Python GUI application embedding a web browser - Options? References: <1c4488d1-aafa-41c3-a2aa-8bd01837dc2f@googlegroups.com> <8ea166e0-28ed-4289-b88b-b6f03ff4f6aa@googlegroups.com> Message-ID: Paul Moore said : > On 9 October 2017 at 04:25, wrote: >> Did you find out the answer for that? > > Nothing much beyond the pointer to PyQt (which basically said "a lot > of the info on the web is out of date" so I should check the latest > docs). I didn't take it much further, though, as it was a hobby > project and the learning curve for PyQt (any GUI framework, really) > was a bit high for the amount of spare time I had at the time. > > Paul Have you looked at CEFpython ? It seems to work with all major GUIs (Qt, wx, Tk, gtk..) https://github.com/cztomczak/cefpython/tree/master/examples From bc at freeuk.com Mon Oct 9 15:32:42 2017 From: bc at freeuk.com (bartc) Date: Mon, 9 Oct 2017 20:32:42 +0100 Subject: on = style In-Reply-To: References: Message-ID: On 09/10/2017 07:40, Abdur-Rahmaan Janhangeer wrote: > hi just a quick question, why is > > my_pens = 4 > my_pencils = 5 > > is preffered to > > my_pens = 4 > my_pencils = 5 > > *referring to = symbol alignment I will sometimes line things up, if a block of assignments are related: red = 1 green = 2 blue = 3 There might sometimes be extra maintenance if you then add: turquoise = 4 and have to readjust the others, or you add: yellow = 201 and prefer the number should be aligned right-justified: red = 1 green = 2 blue = 3 turquoise = 4 yellow = 201 but it's not hard. Although it might not be worth doing until a program is finished. Compare to: red = 1 green = 2 blue = 3 turquoise = 4 yellow = 201 -- bartc From jblack at nopam.com Mon Oct 9 16:40:42 2017 From: jblack at nopam.com (John Black) Date: Mon, 9 Oct 2017 15:40:42 -0500 Subject: Is there a way to globally set the print function separator? References: Message-ID: In article , __peter__ at web.de says... > > John Black wrote: > > > I want sep="" to be the default without having to specify it every time I > > call print. Is that possible? > > No, but you can replace the print function with your own: > > >>> print = functools.partial(print, sep="") > >>> print("I", "recommend", "you", "choose", "another", "name", "and", > "preserve", "your", "sanity") > Irecommendyouchooseanothernameandpreserveyoursanity > > And everybody else's. print=functools.partial(print, sep="") NameError: name 'functools' is not defined John Black From rosuav at gmail.com Mon Oct 9 16:47:25 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 07:47:25 +1100 Subject: Is there a way to globally set the print function separator? In-Reply-To: References: Message-ID: On Tue, Oct 10, 2017 at 7:40 AM, John Black wrote: > In article , > __peter__ at web.de says... >> >> John Black wrote: >> >> > I want sep="" to be the default without having to specify it every time I >> > call print. Is that possible? >> >> No, but you can replace the print function with your own: >> >> >>> print = functools.partial(print, sep="") >> >>> print("I", "recommend", "you", "choose", "another", "name", "and", >> "preserve", "your", "sanity") >> Irecommendyouchooseanothernameandpreserveyoursanity >> >> And everybody else's. > > print=functools.partial(print, sep="") > NameError: name 'functools' is not defined It's a module you can import. ChrisA From jblack at nopam.com Mon Oct 9 17:04:36 2017 From: jblack at nopam.com (John Black) Date: Mon, 9 Oct 2017 16:04:36 -0500 Subject: Is there a way to globally set the print function separator? References: Message-ID: In article , python at example.invalid says... > > Le 09/10/2017 ? 18:22, John Black a ?crit?: > > I want sep="" to be the default without having to specify it every time I > > call print. Is that possible? > > >>> oldprint = print > >>> def print(*args,**kwargs): > ... oldprint(*args,**kwargs,sep='') > ... > >>> print(1,2,3) > 123 Winner! Thanks all. I want to make sure I understand what this line is doing: > oldprint = print Experimenting, I find this is not a rename because I can use both function names. It looks it literally copies the function "print" to another function called "oldprint". But now, I have a way to modify the builtin funciton "print" by referencing oldprint. Without oldprint, I have no way to directly modify print? For example, based on your post, I tried: def print(*args, **kw): print(*args, sep='', **kw) meaning print calls print (itself) with sep=''. But this failed and I guess the reason is that it would keep calling itself recursively adding sep='' each time? Thanks. John Black From grant.b.edwards at gmail.com Mon Oct 9 17:19:52 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 9 Oct 2017 21:19:52 +0000 (UTC) Subject: Is there a way to globally set the print function separator? References: Message-ID: On 2017-10-09, John Black wrote: > I want to make sure I understand what this line is doing: > >> oldprint = print > > Experimenting, I find this is not a rename because I can use both > function names. Right it's not _changing_ a name. It's _adding_ a name. > It looks it literally copies the function "print" to > another function called "oldprint". No, it binds a new name of 'oldprint' to the function to which 'print' is bound. It does not create a second function, nor does it remove the binding of the name 'print' to that function. It just adds a second name that's also bound to that function. > But now, I have a way to modify the builtin funciton "print" by > referencing oldprint. Without oldprint, I have no way to directly > modify print? Well... you can still access the builtin print via the module that contains it: >>> def print(*args, **kw): ... __builtins__.print(*args, sep='', **kw) ... >>> print(1,2,3) However, anytime you find yourself using double underscore stuff you should think to yourself "I should find a better way to do that." > def print(*args, **kw): > print(*args, sep='', **kw) > > meaning print calls print (itself) with sep=''. But this failed and I > guess the reason is that it would keep calling itself recursively adding > sep='' each time? Yep. Just define a new function with a new name: def myprint(*args, **kw): print(*args, sep='', **kw) Redefining builtins is just going get you sworn at down the road a bit. If not by yourself, then by somebody else... -- Grant Edwards grant.b.edwards Yow! Used staples are good at with SOY SAUCE! gmail.com From hjp-usenet3 at hjp.at Mon Oct 9 17:43:22 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Mon, 9 Oct 2017 23:43:22 +0200 Subject: The "loop and a half" References: Message-ID: On 2017-10-09 09:44, Mikhail V wrote: >>>> Have you ever worked on a slow remote session where a GUI is >>>> completely impracticable (or maybe even unavailable), and redrawing >>>> the screen is too expensive to do all the time? >>> >>> So where does the redrawing happen? The machine youre sitting on (let's >>> call it 'A') and send remote commands or retrieving text files? Or the >>> redrawing must be synced on both A and >>> the remote machine? If so, then why so? > > Peter J. Holzer wrote: > >> Because you want to see what you are typing. Imagine you are logged into >> a server on the other side of the world. You want to edit a file on that >> machine (for example a configuration file for the web server). You >> invoke the editor on that server. > > I've never done this, but if one told me to edit a file, > my first attempt would be like: > - get read/write access to the file system > - browse it (e.g. some file browser plugin?) > - get the file to the local machine > - edit locally > - write it back to the remote machine I've mentioned that possibility, but as I've also mentioned it may not even be possible, and even if it is, it feels clunky to me. I'm already on the server because I'm investigating an issue. I can simply edit the file there. Why should I switch to a different tool (e.g. WinSCP), open another connection, navigate to a directory again, copy a file, edit it, copy it back (ok, the actual copying back and forth may happen automatically, so I don't have to do that)? Of course I am assuming that I have an editor I'm familiar with on the server. For me this is almost always the case (I use vi (or rather vim) - yes, I'm currently writing this posting with vim). For a Windows user this will usually not be the case: There will be no Notepad++, Atom or PyCharm installed on a typical Linux server. (Of course there are those who maintain that you should never ever log in to server: All interaction should be through APIs and management systems. That may be true if you have thousands of servers. If you have a few dozen I think that to get a management system into place which is even close to the linux CLI tools in veratility is way too much work.) >>> And not in a nice editor with normal fonts? >>> Am i missing something or your 'A' machine cannot use graphics? > >> ... The server may not be able to >> (it's a server, why would anyone install a GUI on it?) > > If I ever work on it (locally) why would I want a GUI on it? > o_O I'm not sure if I'm getting you. > You mean probably a server which is never worked on locally? Typically yes. T usually am sitting in front of a server just long enough to make sure I have a network connection. Then I go back to my office and continue from there. Server rooms are loud, drafty, and either too cold or too warm. But it really doesn't matter to me: On the text console I can use exactly the same programs that I would use remotely over ssh. The only differences are that there is only one window and I can't resize it. >> streaming the screen contents of a rich GUI around the world may be not >> be possible for bandwidth or delay reasons. > Sure, streaming video consumes a lot, but that is what one tries to > avoid if under limitations so streaming messages only is faster. Streaming video is basically what RDP or VNC are. A command line shell is closer to streaming messages. X11 is really message-based but it's slow. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From eryksun at gmail.com Mon Oct 9 17:47:11 2017 From: eryksun at gmail.com (eryk sun) Date: Mon, 9 Oct 2017 22:47:11 +0100 Subject: Is there a way to globally set the print function separator? In-Reply-To: References: Message-ID: On Mon, Oct 9, 2017 at 10:04 PM, John Black wrote: > In article , python at example.invalid says... >> >> Le 09/10/2017 ? 18:22, John Black a ?crit : >> > I want sep="" to be the default without having to specify it every time I >> > call print. Is that possible? >> >> >>> oldprint = print >> >>> def print(*args,**kwargs): >> ... oldprint(*args,**kwargs,sep='') >> ... >> >>> print(1,2,3) >> 123 > > Winner! Thanks all. functools.partial(print, sep='') is the winner, IMO. Or even code that use kwargs.setdefault('sep', ''). The above code is wrong in a way that leads to a TypeError. Can you see the problem? From jblack at nopam.com Tue Oct 10 00:05:33 2017 From: jblack at nopam.com (John Black) Date: Mon, 9 Oct 2017 23:05:33 -0500 Subject: Is there a way to globally set the print function separator? References: Message-ID: In article , eryksun at gmail.com says... > > On Mon, Oct 9, 2017 at 10:04 PM, John Black wrote: > > In article , python at example.invalid says... > >> > >> Le 09/10/2017 ? 18:22, John Black a ?crit : > >> > I want sep="" to be the default without having to specify it every time I > >> > call print. Is that possible? > >> > >> >>> oldprint = print > >> >>> def print(*args,**kwargs): > >> ... oldprint(*args,**kwargs,sep='') > >> ... > >> >>> print(1,2,3) > >> 123 > > > > Winner! Thanks all. > > functools.partial(print, sep='') is the winner, IMO. Ok, I got that working too after someone told me I have to import functools. Can you explain what it means? It looks like it is not defining another function like the other solutions but is truly changing the defaults for the existing function print? John Black From steve+python at pearwood.info Tue Oct 10 02:45:42 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 10 Oct 2017 17:45:42 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> On Tue, 10 Oct 2017 06:06 am, Stefan Ram wrote: > In his book about programming, Bjarne Stroustrup writes: > > |We try hard to avoid "white lies"; that is, we refrain from > |oversimplified explanations that are clear and easy to > |understand, but not true in the context of real languages and > |real problems. Bjarne Stroustrup is famous for designing one of the most heavyweight, baraque, hard-to-understand, difficult-to-use programming languages in common use. While C++ has many excellent features, and is constrained by the need to be compatible with C, I don't think many people believe that it is a well-designed language. But even if it were the best language in the world, and Stroustrup the greatest language designer in the history of computing, what makes you think that he knows anything about teaching? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 10 03:01:15 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 10 Oct 2017 18:01:15 +1100 Subject: Is there a way to globally set the print function separator? References: Message-ID: <59dc703c$0$14970$b1db1813$d948b532@news.astraweb.com> On Tue, 10 Oct 2017 03:05 pm, John Black wrote: >> functools.partial(print, sep='') is the winner, IMO. > > Ok, I got that working too after someone told me I have to import > functools. Can you explain what it means? It looks like it is not > defining another function like the other solutions but is truly changing > the defaults for the existing function print? No. You are absolutely not changing the defaults for the actual built-in print function. There is no support for that in Python. What is happening is a bit more complex, but simplified: - functools.partial is creating a new function-like object, almost identical to the built-in print in every way that matters except that it uses sep=''; - this special function-like object actually calls the true built-in print (which is why we can trust it will be almost identical); - but you have changed the name "print" to refer to the customised version instead of the original. (That change of name will only apply to the current module. Other modules will still see the original print function.) One way to convince yourself that this is the case is to look at the ID of the print function before and after: py> import functools py> id(print) 19 py> print = functools.partial(print, sep='') py> id(print) 20 The ID numbers are different, which proves they are different objects. (If you try this on your computer, you will almost certainly get different ID numbers than I did. The ID numbers are opaque integers with no inherent meaning, and will vary according to the version of Python you have, and even the history of what you have run in the current interpreter session.) We can dig a little deeper, and inspect the new "print" function: py> print.func It is holding onto a reference to the original (which is how it can call the original). We can confirm this: py> id(print.func) 19 Same ID as the original. (Note that this test is not *quite* definitive: ID numbers are only guaranteed to be unique between objects which exist at the same time. If you delete an object, so the garbage collector reclaims its memory and reuses it, it is possible that the ID number may be reused as well.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From flebber.crue at gmail.com Tue Oct 10 03:21:24 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Tue, 10 Oct 2017 00:21:24 -0700 (PDT) Subject: pathlib PurePosixPath Message-ID: Hi How do I create a valid file name and directory with pathlib? When I create it using PurePosixPath I end up with an OSError due to an obvously invlaid path being created. import pathlib for dates in fullUrl: # print(dates) time.sleep(0.3) r = requests.get(dates) data = r.json() if data["RaceDay"] is not None: file_name = data["RaceDay"]["Meetings"][0]["VenueName"] + data["RaceDay"]["MeetingDate"] + '.json' result_path = pathlib.PurePosixPath(r'C:\Users\Sayth\Projects\results', file_name) with open(result_path, 'a') as f: f.write(data) ##Output C:\Users\Sayth\Anaconda3\envs\json\python.exe C:/Users/Sayth/PycharmProjects/ubet_api_mongo/json_download.py Traceback (most recent call last): File "C:/Users/Sayth/PycharmProjects/ubet_api_mongo/json_download.py", line 40, in with open(result_path, 'a') as f: OSError: [Errno 22] Invalid argument: 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' Process finished with exit code 1 Not sure exactly which way to fix it. Cheers Sayth From rosuav at gmail.com Tue Oct 10 03:29:17 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 18:29:17 +1100 Subject: pathlib PurePosixPath In-Reply-To: References: Message-ID: On Tue, Oct 10, 2017 at 6:21 PM, Sayth Renshaw wrote: > Hi > > How do I create a valid file name and directory with pathlib? > > When I create it using PurePosixPath I end up with an OSError due to an obvously invlaid path being created. You're on Windows. The rules for POSIX paths don't apply to your file system, and... > OSError: [Errno 22] Invalid argument: 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' ... the colon is invalid on Windows file systems. You'll have to replace those with something else. ChrisA From flebber.crue at gmail.com Tue Oct 10 03:44:20 2017 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Tue, 10 Oct 2017 00:44:20 -0700 (PDT) Subject: pathlib PurePosixPath In-Reply-To: References: Message-ID: > > Hi > > > > How do I create a valid file name and directory with pathlib? > > > > When I create it using PurePosixPath I end up with an OSError due to an obvously invlaid path being created. > > You're on Windows. The rules for POSIX paths don't apply to your file > system, and... > > > OSError: [Errno 22] Invalid argument: 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' > > ... the colon is invalid on Windows file systems. You'll have to > replace those with something else. > > ChrisA Thanks. Updated the script. But shouldn't it create the file if it doesn't exist? Which none of them will. for dates in fullUrl: # print(dates) time.sleep(0.3) r = requests.get(dates) data = r.json() if data["RaceDay"] is not None: a = data["RaceDay"]["MeetingDate"] b = a[:7] file_name = data["RaceDay"]["Meetings"][0]["VenueName"] + '_' + b + '.json' result_path = pathlib.PurePath(r'C:\Users\Sayth\Projects\results', file_name) with open(result_path, 'a') as f: f.write(data) ##Output File "C:/Users/Sayth/PycharmProjects/ubet_api_mongo/json_download.py", line 42, in with open(result_path, 'a') as f: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Sayth\\Projects\\results\\Warwick Farm_2017-09.json' Process finished with exit code 1 Cheers Sayth From p.f.moore at gmail.com Tue Oct 10 04:09:05 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Oct 2017 09:09:05 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 9 October 2017 at 01:37, boB Stepp wrote: > I follow this list in an effort to learn as much as I can even though > I am usually a fish out of water here. But this thread in all its > twists and turns and various subject line changes seems to have gotten > totally out of hand. Even though I am quoting only part of this one > message, there are actually many others that I am responding to here. > > In my opinion (Honestly admitting my lack of technical competence.), > this insatiable thirst on this list to get every bit of technical > minutiae exactly correct is doing a severe disservice to the > friendliness of this community. Yes, gently correct the (perceived) > errors, because we all want to have it right, but please cease this > incessant pounding of points (and people) into the ground to prove we > are right and they are wrong! > > I doubt any of this is going to change Bart's mind. Why can we not > allow him to make his points, respond to them appropriately, and then > let it go when it is clear he has strongly held opinions that are not > likely to change? > > And Bart, when large numbers of technical experts in their fields have > spent many hours/months/years, yea, even several decades, developing > these software systems, why do you think that you, all by yourself, > know better? Can you not see how frustrating this is for people who > have spent good chunks of their lives trying to do the best they can > on these software systems? Don't you think it is a better approach to > perhaps do some self-examination and approach things from a more > humble learner's perspective? And BTW, there are many users of > non-*nix systems on this list, or who do work on multiple operating > system platforms. > > Can we not let people be who they are, perceived warts (valid or not) > and all, and after responding (hopefully gently) to technical errors > just let them be??? +1 Thank you for saying this. Paul From blindanagram at nowhere.com Tue Oct 10 04:24:05 2017 From: blindanagram at nowhere.com (BlindAnagram) Date: Tue, 10 Oct 2017 09:24:05 +0100 Subject: pathlib PurePosixPath In-Reply-To: References: Message-ID: On 10/10/2017 08:44, Sayth Renshaw wrote: > >>> Hi >>> >>> How do I create a valid file name and directory with pathlib? >>> >>> When I create it using PurePosixPath I end up with an OSError due to an obvously invlaid path being created. >> >> You're on Windows. The rules for POSIX paths don't apply to your file >> system, and... >> >>> OSError: [Errno 22] Invalid argument: 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' >> >> ... the colon is invalid on Windows file systems. You'll have to >> replace those with something else. >> >> ChrisA > > Thanks. Updated the script. But shouldn't it create the file if it doesn't exist? Which none of them will. The documentation says "Pure path objects provide path-handling operations which don?t actually access a filesystem". But the "Concrete paths" subclass is described later with support for a number of file system operations. From tjol at tjol.eu Tue Oct 10 04:26:46 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 10 Oct 2017 10:26:46 +0200 Subject: pathlib PurePosixPath In-Reply-To: References: Message-ID: <8444ac64-630d-35cb-8bed-62318ea9868b@tjol.eu> On 2017-10-10 09:44, Sayth Renshaw wrote: > >>> Hi >>> >>> How do I create a valid file name and directory with pathlib? >>> >>> When I create it using PurePosixPath I end up with an OSError due to an obvously invlaid path being created. >> >> You're on Windows. The rules for POSIX paths don't apply to your file >> system, and... >> >>> OSError: [Errno 22] Invalid argument: 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' >> >> ... the colon is invalid on Windows file systems. You'll have to >> replace those with something else. >> >> ChrisA > > Thanks. Updated the script. But shouldn't it create the file if it doesn't exist? Which none of them will It should, but only if the directory exists. Does it? > > for dates in fullUrl: > # print(dates) > time.sleep(0.3) > r = requests.get(dates) > data = r.json() > if data["RaceDay"] is not None: > a = data["RaceDay"]["MeetingDate"] > b = a[:7] > file_name = data["RaceDay"]["Meetings"][0]["VenueName"] + '_' + b + '.json' > result_path = pathlib.PurePath(r'C:\Users\Sayth\Projects\results', file_name) > with open(result_path, 'a') as f: > f.write(data) > > ##Output > File "C:/Users/Sayth/PycharmProjects/ubet_api_mongo/json_download.py", line 42, in > with open(result_path, 'a') as f: > FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Sayth\\Projects\\results\\Warwick Farm_2017-09.json' > > Process finished with exit code 1 > > Cheers > > Sayth > -- Thomas Jollans From __peter__ at web.de Tue Oct 10 04:49:50 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 10 Oct 2017 10:49:50 +0200 Subject: pathlib PurePosixPath References: Message-ID: Sayth Renshaw wrote: > Thanks. Updated the script. But shouldn't it create the file if it doesn't > exist? Which none of them will. > pathlib.PurePath(r'C:\Users\Sayth\Projects\results', file_name) > with open(result_path, 'a') as f: > f.write(data) > ##Output > File "C:/Users/Sayth/PycharmProjects/ubet_api_mongo/json_download.py", > line 42, in > with open(result_path, 'a') as f: > FileNotFoundError: [Errno 2] No such file or directory: > 'C:\\Users\\Sayth\\Projects\\results\\Warwick Farm_2017-09.json' Read the error message carefully. My crystal ball says that the directory C:\Users\Sayth\Projects\results does not exist. From __peter__ at web.de Tue Oct 10 04:57:23 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 10 Oct 2017 10:57:23 +0200 Subject: Introducing the "for" loop References: <1507207046l.3997912l.0l@psu.edu> Message-ID: Stefan Ram wrote: > Which advice do you refer to? Teach the parts that are most useful first, i. e. for loops over anything but range rather than while loops. From mail at timgolden.me.uk Tue Oct 10 05:22:30 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 10 Oct 2017 10:22:30 +0100 Subject: pathlib PurePosixPath In-Reply-To: References: Message-ID: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> On 2017-10-10 08:29, Chris Angelico wrote: > On Tue, Oct 10, 2017 at 6:21 PM, Sayth Renshaw > wrote: >> Hi >> >> How do I create a valid file name and directory with pathlib? >> >> When I create it using PurePosixPath I end up with an OSError due to >> an obvously invlaid path being created. > > You're on Windows. The rules for POSIX paths don't apply to your file > system, and... > >> OSError: [Errno 22] Invalid argument: >> 'C:\\Users\\Sayth\\Projects\\results/Warwick >> Farm2017-09-06T00:00:00.json' > > ... the colon is invalid on Windows file systems. You'll have to > replace those with something else. I haven't followed closely, so this may well not be an issue here, but the colon is valid on a Windows file system; it introduces an alternate data stream: https://msdn.microsoft.com/en-us/library/windows/desktop/aa364404(v=vs.85).aspx So this is perfectly valid on Windows: with open("temp.txt:abc", "w") as f: f.write("abc") TJG From rosuav at gmail.com Tue Oct 10 05:28:36 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 20:28:36 +1100 Subject: pathlib PurePosixPath In-Reply-To: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> References: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> Message-ID: On Tue, Oct 10, 2017 at 8:22 PM, Tim Golden wrote: > On 2017-10-10 08:29, Chris Angelico wrote: >> >> On Tue, Oct 10, 2017 at 6:21 PM, Sayth Renshaw >> wrote: >>> >>> Hi >>> >>> How do I create a valid file name and directory with pathlib? >>> >>> When I create it using PurePosixPath I end up with an OSError due to an >>> obvously invlaid path being created. >> >> >> You're on Windows. The rules for POSIX paths don't apply to your file >> system, and... >> >>> OSError: [Errno 22] Invalid argument: >>> 'C:\\Users\\Sayth\\Projects\\results/Warwick Farm2017-09-06T00:00:00.json' >> >> >> ... the colon is invalid on Windows file systems. You'll have to >> replace those with something else. > > > I haven't followed closely, so this may well not be an issue here, but the > colon is valid on a Windows file system; it introduces an alternate data > stream: > > https://msdn.microsoft.com/en-us/library/windows/desktop/aa364404(v=vs.85).aspx > > So this is perfectly valid on Windows: > > with open("temp.txt:abc", "w") as f: > f.write("abc") Have you tested that? My understanding of the document you linked to is that the colon still has special meaning, and thus you can't use it in arbitrary file names. (Now, if the question were "can I ever see a colon in a file name", then you'd be correct to use this as proof that you can; but trying to use a file name such as the OP's is not thus proven.) And no, I haven't tested my theory on this either. Any Windows guinea pigs want to give this a try? ChrisA From mail at timgolden.me.uk Tue Oct 10 05:56:58 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 10 Oct 2017 10:56:58 +0100 Subject: pathlib PurePosixPath In-Reply-To: References: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> Message-ID: <3107045cd70515e2dc72f59f54d70a0e@timgolden.me.uk> On 2017-10-10 10:28, Chris Angelico wrote: > On Tue, Oct 10, 2017 at 8:22 PM, Tim Golden > wrote: >> On 2017-10-10 08:29, Chris Angelico wrote: >>> >>> On Tue, Oct 10, 2017 at 6:21 PM, Sayth Renshaw >>> >>> wrote: >>>> >>>> Hi >>>> >>>> How do I create a valid file name and directory with pathlib? >>>> >>>> When I create it using PurePosixPath I end up with an OSError due to >>>> an >>>> obvously invlaid path being created. >>> >>> >>> You're on Windows. The rules for POSIX paths don't apply to your file >>> system, and... >>> >>>> OSError: [Errno 22] Invalid argument: >>>> 'C:\\Users\\Sayth\\Projects\\results/Warwick >>>> Farm2017-09-06T00:00:00.json' >>> >>> >>> ... the colon is invalid on Windows file systems. You'll have to >>> replace those with something else. >> >> >> I haven't followed closely, so this may well not be an issue here, but >> the >> colon is valid on a Windows file system; it introduces an alternate >> data >> stream: >> >> https://msdn.microsoft.com/en-us/library/windows/desktop/aa364404(v=vs.85).aspx >> >> So this is perfectly valid on Windows: >> >> with open("temp.txt:abc", "w") as f: >> f.write("abc") > > Have you tested that? Yes. I ran it before posting. > My understanding of the document you linked to > is that the colon still has special meaning, and thus you can't use it > in arbitrary file names. In fact its presence in that filename creates a (usually hidden) data stream piggybacked onto that file which has the name "abc" into which the data is written. So, following on, the follow works: assert open("temp.txt:abc").read() == "abc" TJG From rosuav at gmail.com Tue Oct 10 05:58:43 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 10 Oct 2017 20:58:43 +1100 Subject: pathlib PurePosixPath In-Reply-To: <3107045cd70515e2dc72f59f54d70a0e@timgolden.me.uk> References: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> <3107045cd70515e2dc72f59f54d70a0e@timgolden.me.uk> Message-ID: On Tue, Oct 10, 2017 at 8:56 PM, Tim Golden wrote: >> My understanding of the document you linked to >> is that the colon still has special meaning, and thus you can't use it >> in arbitrary file names. > > > In fact its presence in that filename creates a (usually hidden) data stream > piggybacked onto that file which has the name "abc" into which the data is > written. > > So, following on, the follow works: > > assert open("temp.txt:abc").read() == "abc" Cool. Does it require that temp.txt exist first? And if you have multiple colons (as in the OP's), does the part after the second colon have to be a type indicator? ChrisA From mail at timgolden.me.uk Tue Oct 10 06:18:26 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 10 Oct 2017 11:18:26 +0100 Subject: pathlib PurePosixPath In-Reply-To: References: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> <3107045cd70515e2dc72f59f54d70a0e@timgolden.me.uk> Message-ID: <727d80056273d4da47b83797dedd4c1e@timgolden.me.uk> On 2017-10-10 10:58, Chris Angelico wrote: > On Tue, Oct 10, 2017 at 8:56 PM, Tim Golden > wrote: >>> My understanding of the document you linked to >>> is that the colon still has special meaning, and thus you can't use >>> it >>> in arbitrary file names. >> >> >> In fact its presence in that filename creates a (usually hidden) data >> stream >> piggybacked onto that file which has the name "abc" into which the >> data is >> written. >> >> So, following on, the follow works: >> >> assert open("temp.txt:abc").read() == "abc" > > Cool. Does it require that temp.txt exist first? And if you have > multiple colons (as in the OP's), does the part after the second colon > have to be a type indicator? > > ChrisA No. temp.txt is created empty. Multiple colons *does* appear to be a syntax error in the filename. TJG From rhodri at kynesim.co.uk Tue Oct 10 06:44:01 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 10 Oct 2017 11:44:01 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <363aa468-0d92-e299-b01c-ad106fa082f9@kynesim.co.uk> On 09/10/17 20:06, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >> Steve D'Aprano writes: >>> At various stages of education, we teach many lies-to-children, including: >> Many of those lies can be perfectly true in some sense. >> I pick some examples: > > Another noun phrase with "lie" is "white lie". > > In his book about programming, Bjarne Stroustrup writes: > > |We try hard to avoid "white lies"; that is, we refrain from > |oversimplified explanations that are clear and easy to > |understand, but not true in the context of real languages and > |real problems. That would go a long way to explaining why I tried and failed to learn C++ three times from Stroustrup's books. -- Rhodri James *-* Kynesim Ltd From rhodri at kynesim.co.uk Tue Oct 10 06:52:51 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 10 Oct 2017 11:52:51 +0100 Subject: on = style In-Reply-To: <59db2971$0$14948$b1db1813$d948b532@news.astraweb.com> References: <59db2971$0$14948$b1db1813$d948b532@news.astraweb.com> Message-ID: On 09/10/17 08:46, Steve D'Aprano wrote: > I truly believe that, with*very* few exceptions (*ALL* of which are some form > of in-line data table, and even then only rarely) any programmer who worries > about lining up assignments on different lines like this is just engaged in a > time-wasting exercise to make themselves look busy while actually taking it > easy. It is a pure avoidance activity. I'm inclined to disagree, at least in as much as I think the exceptions are more common than you do. For me at least there are two opposed principles: * Columnar data is easier to read. * Big gaps in the middle of a line make it hard to read. I fairly often line up initialisers height = 5 width = 10 length = 2 density = 0.75 ...or other related groups of assignments, but that takes a minute at most. When there is a huge mismatch in name length, aligning the columns is more tedious and gives you less visual advantage, so I don't bother. Somewhere in the middle is a tipping point. -- Rhodri James *-* Kynesim Ltd From BILL_NOSPAM at whoknows.net Tue Oct 10 07:09:00 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Tue, 10 Oct 2017 07:09:00 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <363aa468-0d92-e299-b01c-ad106fa082f9@kynesim.co.uk> Message-ID: Rhodri James wrote: > On 09/10/17 20:06, Stefan Ram wrote: >> ram at zedat.fu-berlin.de (Stefan Ram) writes: >>> Steve D'Aprano writes: >>>> At various stages of education, we teach many lies-to-children, >>>> including: >>> Many of those lies can be perfectly true in some sense. >>> I pick some examples: >> >> Another noun phrase with "lie" is "white lie". >> >> In his book about programming, Bjarne Stroustrup writes: >> >> |We try hard to avoid "white lies"; that is, we refrain from >> |oversimplified explanations that are clear and easy to >> |understand, but not true in the context of real languages and >> |real problems. > > That would go a long way to explaining why I tried and failed to learn > C++ three times from Stroustrup's books. > He is surely one of the best authors in computer science (at least based upon "The C++ Programming Language", 3rd ed.). He indicates that he assumes that the reader has some experience developing serious software. Beazley's, "Python: Essential Reference" is one of the best books I've seen since. It doesn't have the same depth, but it (too) provides articulate answers, head-on. Both books have 5-star rankings on Amazon.com. That doesn't mean that either of them is right for everybody. Come back to Stroustrup's book "after" you learn C++ somewhere else, and maybe you'll enjoy it more. Bill From sjmsoft at gmail.com Tue Oct 10 08:12:42 2017 From: sjmsoft at gmail.com (sjmsoft at gmail.com) Date: Tue, 10 Oct 2017 05:12:42 -0700 (PDT) Subject: on = style In-Reply-To: References: Message-ID: <5804f7e6-2787-4c05-9918-91ec55a2728b@googlegroups.com> Being an old-timer, I was brought up on languages like COBOL, where programmers love to line up MOVE, COMPUTE and some other statements. I still do that when I write COBOL, as it is expected by other programmers and somehow just seems right. But I resist the urge in Python and never do it. Previous posters have given good reasons for this. My main reason is to keep diffs as small as possible. Small diffs would apply to COBOL too, of course. Even programmers are not always completely rational. Tradition and common practice have their say, too. (And COBOL is so verbose that perhaps it's a lost cause!) Cheers, Steve J. Martin From bc at freeuk.com Tue Oct 10 08:44:13 2017 From: bc at freeuk.com (bartc) Date: Tue, 10 Oct 2017 13:44:13 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 09/10/2017 01:37, boB Stepp wrote: > On Sun, Oct 8, 2017 at 5:36 AM, bartc wrote: > And Bart, when large numbers of technical experts in their fields have > spent many hours/months/years, yea, even several decades, developing > these software systems, why do you think that you, all by yourself, > know better? In some fields, then yes! The problem with experts is that they do like to pile layer upon layer, plus another layer to deal with the complexity created by the previous layers! For example, I write compilers, including a C compiler. But I couldn't start compiling the CPython sources (with any compiler actually), because the first step was running a 30,000-line configure script, for Linux. I work on Windows. However there is now a solution using VS2015. There was a thread on this subject in May 2017, and there I recounted my attempts at using VS2015 to build CPython. A copy of that I've put here: https://pastebin.com/raw/V76WP1Ha (Raw text so no adverts.) CPython is 250K lines I think; a fast compiler ought to be able to deal with a program like that in one second (depending on how many files it's split up into). Now look at that link. Experts.... Building big cumbersome, complicated systems is easy. Making them small and simple is harder. > Can you not see how frustrating this is for people who > have spent good chunks of their lives trying to do the best they can > on these software systems? Only if they concede I might have a point. I haven't seen much sign of that! I doubt anyone using *nix systems, which are massively popular, is going to start feeling insecure due to the opinions of one guy. You say (in a bit I see I've snipped), that non *nix people read these forums. But my experience is that the majority of people who post (in c.l.p and comp.lang.c) are using *nix. It's unbalanced. -- bartc From p.f.moore at gmail.com Tue Oct 10 09:20:23 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Oct 2017 14:20:23 +0100 Subject: The "loop and a half" In-Reply-To: References: <21e3e300-2652-a256-b1ae-688259089511@kynesim.co.uk> <59d81507$0$14932$b1db1813$d948b532@news.astraweb.com> <59d9ebfb$0$14937$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10 October 2017 at 13:44, bartc wrote: >> Can you not see how frustrating this is for people who >> have spent good chunks of their lives trying to do the best they can >> on these software systems? > > Only if they concede I might have a point. I haven't seen much sign of that! You have a point, if you're willing to work in a simplified environment. Like most older, larger systems, a lot of the complexity comes from trying to support a large number of environments/configurations, many more than anyone could have envisioned at the start. Code gets layered on top of older code to support extra situations. Refactoring and simplification is possible, but often really hard because not all of the peculiar situations that prompted the complexity can be reproduced (or even remembered, much of the time) by the developers. Strict "no fix is allowed without a failing test demonstrating the bug" policies can help with that, but in practice such policies are *really* onerous, and tend to risk preventing necessary fixes being applied. The problem is that someone coming in saying "it could be so much simpler" is probably only considering a very small subset of the situations that have come up over the lifetime of the project. Summary: "Things don't need to be this complicated" is likely true. But the cost of simplifying is likely to either be massive coding effort, or reintroduction of obscure bugs that were previously fixed. So I've conceded that you might have a point. Are *you* willing to concede that you may have missed something when making your assertions? Paul From eryksun at gmail.com Tue Oct 10 09:40:45 2017 From: eryksun at gmail.com (eryk sun) Date: Tue, 10 Oct 2017 14:40:45 +0100 Subject: pathlib PurePosixPath In-Reply-To: <727d80056273d4da47b83797dedd4c1e@timgolden.me.uk> References: <8cc96e0e4c229cb2f50151588538aee3@timgolden.me.uk> <3107045cd70515e2dc72f59f54d70a0e@timgolden.me.uk> <727d80056273d4da47b83797dedd4c1e@timgolden.me.uk> Message-ID: On Tue, Oct 10, 2017 at 11:18 AM, Tim Golden wrote: > On 2017-10-10 10:58, Chris Angelico wrote: >> On Tue, Oct 10, 2017 at 8:56 PM, Tim Golden wrote: >> >>> In fact its presence in that filename creates a (usually hidden) data >>> stream piggybacked onto that file which has the name "abc" into which the >>> data is written. >>> >>> So, following on, the follow works: >>> >>> assert open("temp.txt:abc").read() == "abc" >> >> Cool. Does it require that temp.txt exist first? And if you have >> multiple colons (as in the OP's), does the part after the second colon >> have to be a type indicator? >> >> ChrisA > > No. temp.txt is created empty. > > Multiple colons *does* appear to be a syntax error in the filename. Colon in a file path (not a device name) is a reserved character that's used to reference NTFS file streams. The colon is not part of the name of either the base file/directory or the stream. You can check whether a volume supports named streams by calling GetVolumeInformation. If they're supported, then lpFileSystemFlags will contain the flag FILE_NAMED_STREAMS. The complete spec is "FileName:StreamName:StreamType". When a regular file is opened, NTFS defaults to opening the anonymous data stream, "FileName::$DATA". When a directory is opened, NTFS defaults to opening "DirName:$I30:$INDEX_ALLOCATION", i.e. the $I30 stream of type $INDEX_ALLOCATION. A directory can also have named $DATA streams, but it can't have an anonymous $DATA stream because that would be ambiguous in general. Other stream types used internally in NTFS include $FILE_NAME, $REPARSE_POINT, $OBJECT_ID, $ATTRIBUTE_LIST, $INDEX_ROOT, and $BITMAP. But these are of little interest unless you specialize in forensics or are supporting NTFS on another OS. From xyheme at gmail.com Tue Oct 10 10:37:29 2017 From: xyheme at gmail.com (xieyuheng) Date: Tue, 10 Oct 2017 07:37:29 -0700 (PDT) Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' Message-ID: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> in CPython, instead of trying each object and function, is there any rules can help me answer the following questions ? or is there any third part documentations [community driven docs ? wiki ?] that can answer them ? 1. 'setattr(o, name, value)' can be used for what kind of objects ? section '2. Built-in Functions' of the official documentation says : > The function assigns the value to the attribute, provided the object allows it. 2. what kind of functions does not have signature, so that 'inspect.signature(f)' can be used for them ? section '29.12.3. Introspecting callables with the Signature object' of the official documentation says : > Some callables may not be introspectable in certain implementations of Python. > For example, in CPython, some built-in functions defined in C > provide no metadata about their arguments. this is depends on implementation, so I ask for CPython. ------ xyh From xyheme at gmail.com Tue Oct 10 10:39:26 2017 From: xyheme at gmail.com (xieyuheng) Date: Tue, 10 Oct 2017 07:39:26 -0700 (PDT) Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' Message-ID: <83fa1cf0-17da-41e9-9390-7aadd7acece3@googlegroups.com> in CPython, instead of trying each object and function, is there any rules can help me answer the following questions ? or is there any third part documentations [community driven docs ? wiki ?] that can answer them ? 1. 'setattr(o, name, value)' can be used for what kind of objects ? section '2. Built-in Functions' of the official documentation says : > The function assigns the value to the attribute, provided the object allows it. 2. what kind of functions does not have signature, so that 'inspect.signature(f)' can be used for them ? section '29.12.3. Introspecting callables with the Signature object' of the official documentation says : > Some callables may not be introspectable in certain implementations of Python. > For example, in CPython, some built-in functions defined in C > provide no metadata about their arguments. this is depends on implementation, so I ask for CPython. ------ xyh From oliver.schoenborn at gmail.com Tue Oct 10 11:07:44 2017 From: oliver.schoenborn at gmail.com (oliver) Date: Tue, 10 Oct 2017 15:07:44 +0000 Subject: Python GUI application embedding a web browser - Options? In-Reply-To: References: <1c4488d1-aafa-41c3-a2aa-8bd01837dc2f@googlegroups.com> <8ea166e0-28ed-4289-b88b-b6f03ff4f6aa@googlegroups.com> Message-ID: Can you elaborate what is not sufficient with Qt's web components? http://doc.qt.io/qt-5/topics-web-content.html. This allows for HTML, CSS, Javascript, via either native C++ Qt, QML, or Python (PyQt) or mixtures of these 3. That said, I have not yet used Qt`s web engine so I don`t know how full-featured it is, how robust, etc. I have used PyQt for several years in production grade project and it is very solid and well documented and easy to program with and easy to write tests for. Oliver On Mon, 9 Oct 2017 at 12:59 fpp wrote: > Paul Moore said : > > > On 9 October 2017 at 04:25, wrote: > >> Did you find out the answer for that? > > > > Nothing much beyond the pointer to PyQt (which basically said "a lot > > of the info on the web is out of date" so I should check the latest > > docs). I didn't take it much further, though, as it was a hobby > > project and the learning curve for PyQt (any GUI framework, really) > > was a bit high for the amount of spare time I had at the time. > > > > Paul > > Have you looked at CEFpython ? > It seems to work with all major GUIs (Qt, wx, Tk, gtk..) > https://github.com/cztomczak/cefpython/tree/master/examples > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Oliver My StackOverflow contributions My CodeProject articles My Github projects My SourceForget.net projects From p.f.moore at gmail.com Tue Oct 10 11:15:50 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Oct 2017 16:15:50 +0100 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' In-Reply-To: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> Message-ID: On 10 October 2017 at 15:37, xieyuheng wrote: > 1. 'setattr(o, name, value)' can be used for what kind of objects ? > > section '2. Built-in Functions' > of the official documentation says : > > > The function assigns the value to the attribute, provided the object allows it. Anything for which o. = value will work. However, user defined types can choose to raise an exception if you try to assign to certain attributes, types with slots will raise an exception if you try to assign to an attribute not defined in __slots__, etc. These are all run-time behaviours, and so there's no way you can check for them ahead of time. If you want to be sure setattr is allowed, you need to handle possible exceptions: try: setattr(o, name, value) except Exception: # deal with the problem Example: >>> class C: ... __slots__ = ['a', 'b'] ... >>> c = C() >>> c.c = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'C' object has no attribute 'c' >>> setattr(c, 'c', 1) Traceback (most recent call last): File "", line 1, in AttributeError: 'C' object has no attribute 'c' >>> try: ... setattr(c, 'c', 1) ... except AttributeError: ... pass ... >>> > 2. what kind of functions does not have signature, > so that 'inspect.signature(f)' can be used for them ? > > section '29.12.3. Introspecting callables with the Signature object' > of the official documentation says : > > > Some callables may not be introspectable in certain implementations of Python. > > For example, in CPython, some built-in functions defined in C > > provide no metadata about their arguments. > > this is depends on implementation, so I ask for CPython. Again, it's a runtime issue, so the answer is the same "it's easier to ask for forgiveness than permission" - try the operation and handle any exception: >>> import inspect >>> inspect.signature(type) Traceback (most recent call last): File "", line 1, in File "C:\Users\me\AppData\Local\Programs\Python\Python36\Lib\inspect.py", line 3033, in signature return Signature.from_callable(obj, follow_wrapped=follow_wrapped) File "C:\Users\me\AppData\Local\Programs\Python\Python36\Lib\inspect.py", line 2783, in from_callable follow_wrapper_chains=follow_wrapped) File "C:\Users\me\AppData\Local\Programs\Python\Python36\Lib\inspect.py", line 2262, in _signature_from_callable skip_bound_arg=skip_bound_arg) File "C:\Users\me\AppData\Local\Programs\Python\Python36\Lib\inspect.py", line 2087, in _signature_from_builtin raise ValueError("no signature found for builtin {!r}".format(func)) ValueError: no signature found for builtin >>> try: ... sig = inspect.signature(type) ... except ValueError: ... sig = None ... >>> print(repr(sig)) None Paul From p.f.moore at gmail.com Tue Oct 10 11:18:24 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 10 Oct 2017 16:18:24 +0100 Subject: Python GUI application embedding a web browser - Options? In-Reply-To: References: <1c4488d1-aafa-41c3-a2aa-8bd01837dc2f@googlegroups.com> <8ea166e0-28ed-4289-b88b-b6f03ff4f6aa@googlegroups.com> Message-ID: On 10 October 2017 at 16:07, oliver wrote: > Can you elaborate what is not sufficient with Qt's web components? I can't, sorry. Douglas was resurrecting a thread from a year ago. At the time I was trying to do a quick proof of concept project and asked for help on here. The project never really went anywhere, because it ended up being more complex than I had time for. I don't recall the details any more. Paul From mikhailwas at gmail.com Tue Oct 10 11:53:46 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Tue, 10 Oct 2017 17:53:46 +0200 Subject: Keyboard Input [was: The "loop and a half"] Message-ID: Chris Angelico wrote: > On Mon, Oct 9, 2017 at 7:05 PM, Mikhail V wrote: > > The first thing a developer should provide - the keys and mouse input > > should be > > *customizable* by the user. It is so by most serious application I have > > ever used. > > And they most certainly are. Often, in something in the host platform. > For instance, Xfce allows me to change the keys used for various > purposes, by going into the menu and choosing "Settings" and then > "Keyboard". (Or, of course, by editing the config files.) The most > obvious and dramatic change you can do is to choose a specific > keyboard layout - Qwerty vs Dvorak, US vs German, etc, etc, etc. Do > you think that sort of thing should be in the hands of the > application, or the platform? (Note that I'm not saying "OS" here. > Usually the platform is at a higher level than that; in my case, I > would consider Xfce (my desktop manager) to be that.) I am not sure I correctly understand your question. Do you ask whether the application should bypass the OS selected layout and use own character input tables? It is of course possible, but since the OS provides input events, why not use those? We were speaking about a text editor, so yes this will be useful if one wants multilingual text input. Also I am almost sure I can programmatically switch the layout pages from Python (some Winapi calls I suppose) So theoretically, based only on keyb scancodes, one can use own LUTS and flags to replicate the unicode input. I personally never wrote apps with multilingual text input though. Seems that for keyboard input everything is already invented and well-established. Below is a minimal app showing pressed keys. Try switching layouts and see 'unicode' field changing for same key scancode. import pygame pygame.init() window_w = 300 # display width window_h = 200 # display height pygame.display.set_mode((window_w, window_h)) clock = pygame.time.Clock( ) FPS = 15 MainLoop = True while MainLoop : clock.tick(FPS) for E in pygame.event.get(): if E.type == pygame.QUIT: MainLoop = False if E.type == pygame.KEYDOWN: print (E) if E.key == pygame.K_ESCAPE: MainLoop = False pygame.quit( ) Mikhail From rvail at presidio.com Tue Oct 10 12:00:53 2017 From: rvail at presidio.com (Vail, Rick) Date: Tue, 10 Oct 2017 16:00:53 +0000 Subject: Printing to a file and a terminal at the same time Message-ID: <5565394e43ef4e08a8bfa16e1ce6f0a0@sva-ex13-svr03.Presidio.Corp> I have a script for Cisco devices that will do configuration or any CLI command. What I would like to do is print the output to my terminal(windows) and to a file. I can come up with stdout parameters To print to a file but not to the screen and when I remove the stdout part it prints to the screen only. This prints to a file filename = open("outputfile",'w') sys.stdout = filename print ("Anything printed will go to the output file") Remove the above and it prints to the terminal, help Rick Vail | Network Engineer Presidio (NASDAQ: PSDO) | www.presidio.com 12272 Hancock St, Carmel, IN 46032 D: 317.428.6399 | C: 317.847.0493 | rvail at presidio.com [Future. Built.] Follow us: [Follow Presidio on Twitter] This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments. Please be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. From tjreedy at udel.edu Tue Oct 10 12:06:52 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 10 Oct 2017 12:06:52 -0400 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' In-Reply-To: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> Message-ID: On 10/10/2017 10:37 AM, xieyuheng wrote: > 2. what kind of functions does not have signature, > so that 'inspect.signature(f)' can be used for them ? When .signature was added, it may not have been usable with *any* C-coded function. About the same, a mechanism was added to make signatures available with C-coded functions. Using the mechanism requires recoding for each file. By now, many CPython functions have been recoded. > section '29.12.3. Introspecting callables with the Signature object' > of the official documentation says : > > > Some callables may not be introspectable in certain implementations of Python. So 'some' is a gradually shrinking set. > > For example, in CPython, some built-in functions defined in C > > provide no metadata about their arguments. > > this is depends on implementation, so I ask for CPython. -- Terry Jan Reedy From rvail at presidio.com Tue Oct 10 12:09:46 2017 From: rvail at presidio.com (Vail, Rick) Date: Tue, 10 Oct 2017 16:09:46 +0000 Subject: FW: Printing to a file and a terminal at the same time Message-ID: Rick Vail | Network Engineer Presidio (NASDAQ: PSDO) | www.presidio.com 12272 Hancock St, Carmel, IN 46032 D: 317.428.6399 | C: 317.847.0493 | rvail at presidio.com [Future. Built.] Follow us: [Follow Presidio on Twitter] From: Vail, Rick Sent: Tuesday, October 10, 2017 7:04 AM To: 'python-list at python.org' Subject: Printing to a file and a terminal at the same time I have a script for Cisco devices that will do configuration or any CLI command. What I would like to do is print the output to my terminal(windows) and to a file. I can come up with stdout parameters To print to a file but not to the screen and when I remove the stdout part it prints to the screen only. This prints to a file filename = open("outputfile",'w') sys.stdout = filename print ("Anything printed will go to the output file") Remove the above and it prints to the terminal, help This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments. Please be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. From tjol at tjol.eu Tue Oct 10 12:14:22 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 10 Oct 2017 18:14:22 +0200 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' In-Reply-To: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> Message-ID: <2d6d1bfb-464e-d80b-6e3c-575b34e16eb1@tjol.eu> On 2017-10-10 16:37, xieyuheng wrote: > > Some callables may not be introspectable in certain implementations of Python. > > For example, in CPython, some built-in functions defined in C > > provide no metadata about their arguments. > > this is depends on implementation, so I ask for CPython. There's not really much more to say than what the docs you're quoting say. As far as I'm aware, all callables defined in Python will have a signature, while the default for "built-in" functions (anything written in C) in CPython is for them not to have a signature. This applies to __builtins__ and other modules written in C equally. In __builtins__, most callables have NO signature as of Python 3.6. However, around 3/4 of the *functions* there (i.e. not types) DO have signatures. In particular none of the built-in exceptions have signatures. >>> import inspect >>> no_sig = [] >>> has_sig = [] >>> for name, val in __builtins__.__dict__.items(): ... try: ... tmp = inspect.signature(val) ... except TypeError: ... pass # not callable ... except ValueError: ... no_sig.append(name) ... else: ... has_sig.append(name) ... >>> >>> len(no_sig) 101 >>> len(has_sig) 40 Look at only "functions": >>> [name for name in no_sig if type(getattr(__builtins__, name)) == type(print)] ['__build_class__', '__import__', 'dir', 'getattr', 'iter', 'max', 'min', 'next', 'print', 'round', 'vars'] >>> len([name for name in no_sig if type(getattr(__builtins__, name)) == type(print)]) 11 >>> [name for name in has_sig if type(getattr(__builtins__, name)) == type(print)] ['abs', 'all', 'any', 'ascii', 'bin', 'callable', 'chr', 'compile', 'delattr', 'divmod', 'eval', 'exec', 'format', 'globals', 'hasattr', 'hash', 'hex', 'id', 'input', 'isinstance', 'issubclass', 'len', 'locals', 'oct', 'ord', 'pow', 'repr', 'setattr', 'sorted', 'sum', 'open'] >>> len([name for name in has_sig if type(getattr(__builtins__, name)) == type(print)]) 31 Take out the exceptions and warnings: >>> [name for name in no_sig if name[0] == name[0].lower()] ['__build_class__', '__import__', 'dir', 'getattr', 'iter', 'max', 'min', 'next', 'print', 'round', 'vars', 'bool', 'bytearray', 'bytes', 'classmethod', 'complex', 'dict', 'enumerate', 'filter', 'float', 'frozenset', 'property', 'int', 'list', 'map', 'range', 'reversed', 'set', 'slice', 'staticmethod', 'str', 'super', 'tuple', 'type', 'zip'] >>> len([name for name in no_sig if name[0] == name[0].lower()]) 35 >>> [name for name in has_sig if name[0] == name[0].lower()] ['__loader__', 'abs', 'all', 'any', 'ascii', 'bin', 'callable', 'chr', 'compile', 'delattr', 'divmod', 'eval', 'exec', 'format', 'globals', 'hasattr', 'hash', 'hex', 'id', 'input', 'isinstance', 'issubclass', 'len', 'locals', 'oct', 'ord', 'pow', 'repr', 'setattr', 'sorted', 'sum', 'memoryview', 'object', 'open', 'quit', 'exit', 'copyright', 'credits', 'license', 'help'] >>> len([name for name in has_sig if name[0] == name[0].lower()]) 40 >>> -- Thomas Jollans From tiglathsuriol at gmail.com Tue Oct 10 12:47:25 2017 From: tiglathsuriol at gmail.com (Tiglath Suriol) Date: Tue, 10 Oct 2017 09:47:25 -0700 (PDT) Subject: test Message-ID: <450a3842-8f83-4f6f-98b4-d7b8539d831c@googlegroups.com> test From ned at nedbatchelder.com Tue Oct 10 13:36:32 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 10 Oct 2017 13:36:32 -0400 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' In-Reply-To: References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> Message-ID: On 10/10/17 11:00 AM, Stefan Ram wrote: > xieyuheng writes: >> 1. 'setattr(o, name, value)' can be used for what kind of objects ? > It takes three objects as arguments. > > The first object should have an attribute named by the value > of ?name? that should allow this attribute to be set to > ?value? as if by > > o.=value > > where is the value of ?name? (which should be a string). > One tweak to this: the object doesn't need to have the attribute initially.? setattr() can create new attributes that don't already exist. --Ned. From mcuddehe at strategicga.com Tue Oct 10 16:37:51 2017 From: mcuddehe at strategicga.com (Michael Cuddehe) Date: Tue, 10 Oct 2017 15:37:51 -0500 Subject: Unable to run pip in Windows 10 Message-ID: <005701d34207$a52165b0$ef643110$@com> I have tried multiple versions, 32 & 64 bit. Same problem. "This app can't run on your PC. To find a version for your PC, check with the software publisher." From python at mrabarnett.plus.com Tue Oct 10 19:27:10 2017 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 11 Oct 2017 00:27:10 +0100 Subject: Printing to a file and a terminal at the same time In-Reply-To: <5565394e43ef4e08a8bfa16e1ce6f0a0@sva-ex13-svr03.Presidio.Corp> References: <5565394e43ef4e08a8bfa16e1ce6f0a0@sva-ex13-svr03.Presidio.Corp> Message-ID: <057f57e6-659e-f5a2-d2a7-e33b1bac9489@mrabarnett.plus.com> On 2017-10-10 17:00, Vail, Rick wrote: > I have a script for Cisco devices that will do configuration or any CLI command. What I would like to do is print the output to my terminal(windows) and to a file. I can come up with stdout parameters > To print to a file but not to the screen and when I remove the stdout part it prints to the screen only. > > This prints to a file > filename = open("outputfile",'w') > sys.stdout = filename > print ("Anything printed will go to the output file") > > Remove the above and it prints to the terminal, help > Why not define a function that does both? def my_print(*args, **kwargs): print(*args, **kwargs) print(*args, **kwargs, file=output_file) From BILL_NOSPAM at whoknows.net Tue Oct 10 20:48:26 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Tue, 10 Oct 2017 20:48:26 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Tue, 10 Oct 2017 06:06 am, Stefan Ram wrote: > >> In his book about programming, Bjarne Stroustrup writes: >> >> |We try hard to avoid "white lies"; that is, we refrain from >> |oversimplified explanations that are clear and easy to >> |understand, but not true in the context of real languages and >> |real problems. > > Bjarne Stroustrup is famous for designing one of the most heavyweight, > baraque, hard-to-understand, difficult-to-use programming languages in common > use. While C++ has many excellent features, and is constrained by the need to > be compatible with C, I don't think many people believe that it is a > well-designed language. It is a well-designed language. It is and was carefully thought out. One could argue that there are perhaps "too many ways" to do a given thing in Python (one could say it's features are "not orthogonal"). I'm sure you are familiar with where the language drew its name. I'm not here to "cast stones", I like Python. I just think that you shouldn't cast stones at C/C++. People started programming in C in the late 70's, and before that some were programming in B ("B Programming Language"), if I recall correctly. Python generally runs "further ways from the hardware" than these other languages, and in some sense, it still would, *even if* you were able to statically compile it to machine language. To me if feels like Python runs like an application. I don't wish to debate this as I have other needs this week. But I felt compelled to try to explain why maybe you shouldn't be casting stones at C/C++. One thing you didn't bring up at all, is that the audiences for the languages appears to be different. You still need folks who can encode data structures and write device drivers, from scratch. And "woe" if you need performance, such as applications involving AI. Cheers, Bill > > But even if it were the best language in the world, and Stroustrup the > greatest language designer in the history of computing, what makes you think > that he knows anything about teaching? > > From steve+python at pearwood.info Tue Oct 10 20:59:58 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 11 Oct 2017 11:59:58 +1100 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> Message-ID: <59dd6d11$0$14936$b1db1813$d948b532@news.astraweb.com> On Wed, 11 Oct 2017 02:15 am, Paul Moore wrote: > These are all run-time behaviours, and so there's no way you can check > for them ahead of time. If you want to be sure setattr is allowed, you > need to handle possible exceptions: > > try: > setattr(o, name, value) > except Exception: > # deal with the problem I would say that you should only catch AttributeError here. Anything else is, in my opinion, a bug in the object o that needs to be spotted and fixed. Even catching AttributeError is a bit... suspicious. Why exactly are we trying to attach attributes to arbitrary objects like None or "Hello World"? But certainly if you get something like UnicodeDecodeError or ImportError from trying to set an attribute, that's a bug in o.__setattr__ that needs fixing. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From torriem at gmail.com Tue Oct 10 23:42:16 2017 From: torriem at gmail.com (Michael Torrie) Date: Tue, 10 Oct 2017 21:42:16 -0600 Subject: FW: Printing to a file and a terminal at the same time In-Reply-To: References: Message-ID: On 10/10/2017 10:09 AM, Vail, Rick wrote: > I have a script for Cisco devices that will do configuration or any CLI command. What I would like to do is print the output to my terminal(windows) and to a file. I can come up with stdout parameters > To print to a file but not to the screen and when I remove the stdout part it prints to the screen only. > > This prints to a file > filename = open("outputfile",'w') > sys.stdout = filename > print ("Anything printed will go to the output file") Is this script interactive? I'm guessing not. In any case, the recommended course for any Unix OS is to just use the tee command to redirect the output both to the terminal and to a file. Works with anything that outputs text to standard out. As for doing it in Python, maybe you can implement your own print-like function that prints everything out twice. Once to the file and once to std out. From BILL_NOSPAM at whoknows.net Wed Oct 11 00:21:46 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 11 Oct 2017 00:21:46 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <1b3rtct57h13vh78abcgssk184cqof0v1b@4ax.com> Message-ID: Dennis Lee Bieber wrote: > On Tue, 10 Oct 2017 20:48:26 -0400, Bill > declaimed the following: > >> cast stones at C/C++. People started programming in C in the late 70's, >> and before that some were programming in B ("B Programming Language"), > Preceded by BCPL (which leads to the joke that the language that > supplants C will be named P) PL-I has already been taken. That is a pretty free-wheeling language compared to Fortran, where, IIRC, you had to start in the 7th column. Oops, hand me another punch card... :) From soyeomul at doraji.xyz Wed Oct 11 02:09:43 2017 From: soyeomul at doraji.xyz (Byung-Hee HWANG =?utf-8?B?KO2Zqeuzke2drCwg6buD?= =?utf-8?B?54Kz54aZKQ==?=) Date: Wed, 11 Oct 2017 15:09:43 +0900 Subject: [Q] days -> months Message-ID: In real life, i am breeding Hanwoo(Korean cattle), about 100 head of cattle. About 50 days ago, i got one young cattle. The name is "Bullseye". I love Bullseye. Until the Bullseye is released by butchery, i would like to maintain him with his age. So thus i did write some humble code to identify his age [1]. By the way i did fail to chage the format "days" to "months". How? Of course i want Python way! Sincerely, [1] https://raw.githubusercontent.com/soyeomul/Gnus/MaGnus/thanks-bullseye-age.rb.gnus -- ^????? _????_ ?????_^))// From marko at pacujo.net Wed Oct 11 02:38:33 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Oct 2017 09:38:33 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: <874lr689zq.fsf@elektro.pacujo.net> Bill : > Steve D'Aprano wrote: >> Bjarne Stroustrup is famous for designing one of the most >> heavyweight, baraque, hard-to-understand, difficult-to-use >> programming languages in common use. While C++ has many excellent >> features, and is constrained by the need to be compatible with C, I >> don't think many people believe that it is a well-designed language. > > It is a well-designed language. It is and was carefully thought out. > One could argue that there are perhaps "too many ways" to do a given > thing in Python (one could say it's features are "not orthogonal"). I'm > sure you are familiar with where the language drew its name. I'm not > here to "cast stones", I like Python. I just think that you shouldn't > cast stones at C/C++. One is allowed to have opinions and express them. We are not talking about religion here. I don't like everything in Python. On the whole, though, it is an excellent language. In practice, it is the best available tool for most programming needs I face. There are situations where C++ is suitable. Its primary advantage over C is the automatic generation of the virtual table. However, having programmed in C++ for a decade or so, it has left a bad taste in my mouth. Its core philosophy (ie, do the utmost at compile time) is cumbersome and misplaced. Until very recently, C++ didn't offer a good way to implement callback functions, for example. > People started programming in C in the late 70's, For the most part, I like C much better than C++, even though the standards committees have been working hard to sabotage it. > One thing you didn't bring up at all, is that the audiences for the > languages appears to be different. You still need folks who can encode > data structures and write device drivers, from scratch. And "woe" if > you need performance, such as applications involving AI. For performance-critical stuff, I use C. For the rest, I use Python and bash. There are other excellent languages, but they don't have a real market niche between C and Python. Marko From p.f.moore at gmail.com Wed Oct 11 04:17:27 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 11 Oct 2017 09:17:27 +0100 Subject: Unable to run pip in Windows 10 In-Reply-To: <005701d34207$a52165b0$ef643110$@com> References: <005701d34207$a52165b0$ef643110$@com> Message-ID: On 10 October 2017 at 21:37, Michael Cuddehe wrote: > I have tried multiple versions, 32 & 64 bit. Same problem. > > "This app can't run on your PC. To find a version for your PC, check with > the software publisher." It's difficult to know what to say - it runs fine for me (Windows 10, Python 3.6, 64-bit, downloaded from python.org). Maybe check that the file you downloaded is complete, by confirming the size/checksum matches? Do you have any antivirus software that could be interfering? Paul From p.f.moore at gmail.com Wed Oct 11 04:18:51 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 11 Oct 2017 09:18:51 +0100 Subject: about 'setattr(o, name, value)' and 'inspect.signature(f)' In-Reply-To: <59dd6d11$0$14936$b1db1813$d948b532@news.astraweb.com> References: <3b41c1df-43ce-4014-aea2-001b4f919f54@googlegroups.com> <59dd6d11$0$14936$b1db1813$d948b532@news.astraweb.com> Message-ID: Agreed. I was being lazy and didn't check precisely which exception was raised before writing the code. "Making this code production ready is left as an exercise for the reader" :-) On 11 October 2017 at 01:59, Steve D'Aprano wrote: > On Wed, 11 Oct 2017 02:15 am, Paul Moore wrote: > >> These are all run-time behaviours, and so there's no way you can check >> for them ahead of time. If you want to be sure setattr is allowed, you >> need to handle possible exceptions: >> >> try: >> setattr(o, name, value) >> except Exception: >> # deal with the problem > > > I would say that you should only catch AttributeError here. Anything else is, > in my opinion, a bug in the object o that needs to be spotted and fixed. > > Even catching AttributeError is a bit... suspicious. Why exactly are we trying > to attach attributes to arbitrary objects like None or "Hello World"? > > But certainly if you get something like UnicodeDecodeError or ImportError from > trying to set an attribute, that's a bug in o.__setattr__ that needs fixing. > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list From tjol at tjol.eu Wed Oct 11 04:24:48 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 11 Oct 2017 10:24:48 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <363aa468-0d92-e299-b01c-ad106fa082f9@kynesim.co.uk> Message-ID: <4e66191e-aafb-1cef-6f08-6de7464afdab@tjol.eu> On 2017-10-10 13:09, Bill wrote: > (at least based upon "The C++ Programming Language", 3rd ed.) My impression when I read "The C++ Programming Language" as a teenager (many years ago) was that the 1st edition was an excellent, if somewhat dense book, while the 3rd edition was completely impenetrable without first having read the 1st edition. -- Thomas Jollans From tjol at tjol.eu Wed Oct 11 04:30:08 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 11 Oct 2017 10:30:08 +0200 Subject: Unable to run pip in Windows 10 In-Reply-To: <005701d34207$a52165b0$ef643110$@com> References: <005701d34207$a52165b0$ef643110$@com> Message-ID: <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> On 2017-10-10 22:37, Michael Cuddehe wrote: > I have tried multiple versions, 32 & 64 bit. Same problem. > > "This app can't run on your PC. To find a version for your PC, check with > the software publisher." > You're going to have to give us some more information for anybody to be able to really help you here. We just don't know what's going on. - What exactly did you install? - Can you start the Python interpreter? * How exactly did you go about this - How exactly do you try to run pip? - What exactly happens when you try? You quoted an error message, and that's great. What's missing is really any information on how you GOT this error message. -- Thomas Jollans From soyeomul at doraji.xyz Wed Oct 11 07:26:30 2017 From: soyeomul at doraji.xyz (Byung-Hee HWANG =?utf-8?B?KO2Zqeuzke2drCwg6buD?= =?utf-8?B?54Kz54aZKQ==?=) Date: Wed, 11 Oct 2017 20:26:30 +0900 Subject: Solved (Was: Re: [Q] days -> months) References: Message-ID: Oh never mind it, after so many trial and error, i did make months format with success, thanks!!! Sincerely, Byung-Hee. -- ^????? _????_ ?????_^))// From rhodri at kynesim.co.uk Wed Oct 11 08:54:09 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 11 Oct 2017 13:54:09 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: On 11/10/17 01:48, Bill wrote: > Steve D'Aprano wrote: >> On Tue, 10 Oct 2017 06:06 am, Stefan Ram wrote: >> >>> In his book about programming, Bjarne Stroustrup writes: >>> >>> |We try hard to avoid "white lies"; that is, we refrain from >>> |oversimplified explanations that are clear and easy to >>> |understand, but not true in the context of real languages and >>> |real problems. >> >> Bjarne Stroustrup is famous for designing one of the most heavyweight, >> baraque, hard-to-understand, difficult-to-use programming languages in >> common >> use. While C++ has many excellent features, and is constrained by the >> need to >> be compatible with C, I don't think many people believe that it is a >> well-designed language. > > > It is a well-designed language.? It is and was carefully thought out. I was manfully trying not to head off on another OT trail, but this is simply not true. C++ is designed, true, but well designed? It has a fundamental flaw; it wants to be both a high-level language and compatible with C, under the mistaken impression that C is a high level language. Since C is actually an excellent macro-assembler, this dooms the exercise from the very start. C++ lives in the no-man's land between programming languages that care quite a lot what processor they are running on and programming languages that wouldn't recognise hardware if it came up and bit them. It can be used either way, but comes with all the baggage for both. I am yet to see a C++ program that wasn't more comprehensible when rendered as either C or Python (or the high-level language of your choice, I imagine). -- Rhodri James *-* Kynesim Ltd From marko at pacujo.net Wed Oct 11 09:16:33 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Oct 2017 16:16:33 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: <87shepddu6.fsf@elektro.pacujo.net> Rhodri James : > C++ is designed, true, but well designed? It has a fundamental flaw; > it wants to be both a high-level language and compatible with C, under > the mistaken impression that C is a high level language. Since C is > actually an excellent macro-assembler, this dooms the exercise from > the very start. C++ was never compatible with C, any more than Peter Jackson's Hobbit movies were a rendition of JRR Tolkien's "Hobbit, or There and Back Again." Objective C, by contrast, was designed to be compatible with C. That's why its syntax looks so weird. > C++ lives in the no-man's land between programming languages that care > quite a lot what processor they are running on and programming > languages that wouldn't recognise hardware if it came up and bit them. C is the traditional application programming language in the Unix family. It doesn't see any hardware. It only sees function libraries and virtual memory. (Exception: recent memory barrier semantics in C force every C and C++ programmer to be acutely aware of multicore RAM cache behavior the moment you start doing multithreading or multiprocessing. To me, this looks more like a bug than a feature in the languages.) > It can be used either way, but comes with all the baggage for both. I > am yet to see a C++ program that wasn't more comprehensible when > rendered as either C or Python (or the high-level language of your > choice, I imagine). I agree. C++'s main problem is that it tries to solve the wrong problem. A C++ compiler seeks to make sure your program doesn't have bugs. That noble (but futile) goal makes it painful to program in C++. Python and C don't try to protect you. In return, you get syntactic convenience that probably enhances the quality of your programs. Marko From grant.b.edwards at gmail.com Wed Oct 11 09:31:17 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 11 Oct 2017 13:31:17 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-11, Bill wrote: > [...] I'm not here to "cast stones", I like Python. I just think > that you shouldn't cast stones at C/C++. Not while PHP exists. There aren't enough stones in the world... -- Grant Edwards grant.b.edwards Yow! Is it 1974? What's at for SUPPER? Can I spend gmail.com my COLLEGE FUND in one wild afternoon?? From rosuav at gmail.com Wed Oct 11 09:46:56 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 00:46:56 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87shepddu6.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 12, 2017 at 12:16 AM, Marko Rauhamaa wrote: > Rhodri James : > >> C++ is designed, true, but well designed? It has a fundamental flaw; >> it wants to be both a high-level language and compatible with C, under >> the mistaken impression that C is a high level language. Since C is >> actually an excellent macro-assembler, this dooms the exercise from >> the very start. > > C++ was never compatible with C, any more than Peter Jackson's Hobbit > movies were a rendition of JRR Tolkien's "Hobbit, or There and Back > Again." The places where C++ is not a superset of C are mostly things you wouldn't want to be doing anyway. You can generally take C code and compile it with a C++ compiler, and it'll have the same semantics. ChrisA From bc at freeuk.com Wed Oct 11 10:14:35 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 15:14:35 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87shepddu6.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> Message-ID: <5HpDB.6442$cW1.5993@fx07.am4> On 11/10/2017 14:16, Marko Rauhamaa wrote: > C++'s main problem is that it tries to solve the wrong problem. A C++ > compiler seeks to make sure your program doesn't have bugs. That noble > (but futile) goal makes it painful to program in C++. It's painful to program for lots of reasons, I don't think that is the main one. The ideas behind C++'s many features individually make perfectly good sense - on paper. Until you look at ghastly examples of C++ source code and it really hurts your eyes. > Python and C don't try to protect you. In return, you get syntactic > convenience that probably enhances the quality of your programs. Python, maybe. C syntax isn't as painful as C++ but I still have a lot of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The name of the variable can be found lurking in that lot somewhere, but what's the type?) Not so convenient. -- bartc From rosuav at gmail.com Wed Oct 11 10:36:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 01:36:16 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <5HpDB.6442$cW1.5993@fx07.am4> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: > Python, maybe. C syntax isn't as painful as C++ but I still have a lot of > trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The > name of the variable can be found lurking in that lot somewhere, but what's > the type?) Not so convenient. People love showcasing stupid examples like that. But how often do you REALLY make declarations that complex? That's not technically strawmanning, since C syntax does indeed include that, but you're cherry-picking the most extreme example. ChrisA From mcuddehe at strategicga.com Wed Oct 11 10:46:48 2017 From: mcuddehe at strategicga.com (Michael Cuddehe) Date: Wed, 11 Oct 2017 09:46:48 -0500 Subject: Unable to run pip in Windows 10 In-Reply-To: <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> Message-ID: <001b01d3429f$c4b38910$4e1a9b30$@com> - What exactly did you install? >> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32 >> Downloaded from python.org. - Can you start the Python interpreter? >> Yes...works fine. * How exactly did you go about this >> ?? - How exactly do you try to run pip? >> Command prompt: c:\Users\FFC>pip import (module) >> - What exactly happens when you try? >> Windows error message: "This app can't run on your PC...." -----Original Message----- From: Python-list [mailto:python-list-bounces+mcuddehe=strategicga.com at python.org] On Behalf Of Thomas Jollans Sent: Wednesday, October 11, 2017 3:30 AM To: python-list at python.org Subject: Re: Unable to run pip in Windows 10 On 2017-10-10 22:37, Michael Cuddehe wrote: > I have tried multiple versions, 32 & 64 bit. Same problem. > > "This app can't run on your PC. To find a version for your PC, check > with the software publisher." > You're going to have to give us some more information for anybody to be able to really help you here. We just don't know what's going on. - What exactly did you install? - Can you start the Python interpreter? * How exactly did you go about this - How exactly do you try to run pip? - What exactly happens when you try? You quoted an error message, and that's great. What's missing is really any information on how you GOT this error message. -- Thomas Jollans -- https://mail.python.org/mailman/listinfo/python-list --- This email has been checked for viruses by AVG. http://www.avg.com From breamoreboy at gmail.com Wed Oct 11 10:52:53 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 11 Oct 2017 07:52:53 -0700 (PDT) Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <5HpDB.6442$cW1.5993@fx07.am4> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On Wednesday, October 11, 2017 at 3:14:51 PM UTC+1, bartc wrote: > On 11/10/2017 14:16, Marko Rauhamaa wrote: > > > Python and C don't try to protect you. In return, you get syntactic > > convenience that probably enhances the quality of your programs. > > Python, maybe. C syntax isn't as painful as C++ but I still have a lot > of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. > The name of the variable can be found lurking in that lot somewhere, but > what's the type?) Not so convenient. > > -- > bartc https://cdecl.org/ tells me that your variable declaration is a syntax error so maybe not much of an example. -- Kindest regards. Mark Lawrence. From p.f.moore at gmail.com Wed Oct 11 11:10:29 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 11 Oct 2017 16:10:29 +0100 Subject: Unable to run pip in Windows 10 In-Reply-To: <001b01d3429f$c4b38910$4e1a9b30$@com> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> Message-ID: On 11 October 2017 at 15:46, Michael Cuddehe wrote: > - What exactly did you install? >>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC > v.1900 64 bit (AMD64)] on win32 >>> Downloaded from python.org. > - Can you start the Python interpreter? >>> Yes...works fine. > * How exactly did you go about this >>> ?? > - How exactly do you try to run pip? >>> Command prompt: c:\Users\FFC>pip import (module) >>> > - What exactly happens when you try? >>> Windows error message: "This app can't run on your PC...." There's no "pip import" command - I guess you meant "pip install"? Rather than transcribing what you did, can you include a copy and paste of the actual text in your command window? I also googled for the message you quoted, and I see that it's a Windows message (not one from pip or Python). The most obvious suggestion from that was "you're running a 64-bit application on a 32-bit Windows". I note from the above: > Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC > v.1900 64 bit (AMD64)] Are you running 32-bit Windows? If so, you got the wrong version of Python (you need the 32-bit version). Paul From marko at pacujo.net Wed Oct 11 11:43:56 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Oct 2017 18:43:56 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> Message-ID: <87lgkhd70j.fsf@elektro.pacujo.net> Chris Angelico : > The places where C++ is not a superset of C are mostly things you > wouldn't want to be doing anyway. You can generally take C code and > compile it with a C++ compiler, and it'll have the same semantics. Here's a C/C++ program: ======================================================================== #include int main() { struct {} s; printf("%d\n", (int) sizeof 'a'); printf("%d\n", (int) sizeof s); return 0; } ======================================================================== When compiled (with gcc) as a C program, the output is: 4 0 When the same program is compiled (with gcc) as a C++ program, the output is: 1 1 That is not immediately all that significant but points to subtle incompatibilities between the data models of C and C++. Then we have syntactic problems: ======================================================================== int main() { void *s = "hello"; char *t = s; return 0; } ======================================================================== which, as a C program, makes gcc perfectly happy, but a C++ compilation complains: test.C: In function ?int main()?: test.C:5:15: error: invalid conversion from ?const void*? to ?void*? \ [-fpermissive] void *s = "hello"; ^~~~~~~ test.C:6:15: error: invalid conversion from ?void*? to ?char*? [-fper\ missive] char *t = s; ^ (The first one should trigger an error message even in C compilation, but string literals were standardized to be "semiconstant" in C.) Marko From neilc at norwich.edu Wed Oct 11 11:45:47 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Wed, 11 Oct 2017 15:45:47 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 2017-10-11, Marko Rauhamaa wrote: > Bill : >> Steve D'Aprano wrote: >>> Bjarne Stroustrup is famous for designing one of the most >>> heavyweight, baraque, hard-to-understand, difficult-to-use >>> programming languages in common use. While C++ has many excellent >>> features, and is constrained by the need to be compatible with C, I >>> don't think many people believe that it is a well-designed language. >> >> It is a well-designed language. It is and was carefully >> thought out. One could argue that there are perhaps "too many >> ways" to do a given thing in Python (one could say it's >> features are "not orthogonal"). I'm sure you are familiar with >> where the language drew its name. I'm not here to "cast >> stones", I like Python. I just think that you shouldn't cast >> stones at C/C++. > > One is allowed to have opinions and express them. We are not talking > about religion here. > > I don't like everything in Python. On the whole, though, it is an > excellent language. In practice, it is the best available tool for most > programming needs I face. > > There are situations where C++ is suitable. Its primary > advantage over C is the automatic generation of the virtual > table. However, having programmed in C++ for a decade or so, it > has left a bad taste in my mouth. Its core philosophy (ie, do > the utmost at compile time) is cumbersome and misplaced. Until > very recently, C++ didn't offer a good way to implement > callback functions, for example. Some other cool stuff in C++ and how it relates to a Python features as I understand them: Support for Resource Acquisition is Initialization (RAII) with constructors and destructors is a big advantage. Python's got this now, too, with with. C++'s big idea of allowing you to define you own data types and make them work precisely as you wish is a great idea, only it's too bad it requires such experience and skill to get it right. Python provides a similar feature with dynamic typing, though the urge to implement my own type that acts just like one of the built-in types hardly ever comes to me in Python, thanks to its fabulous standard library, which regularly uses these features. C++ compile-time polymorphism using templates is awesome and vital, but unfortunately (or fortunately?) abusable for compile-time data processing and foster-parent of horrid compiler error message bloat. Run-time polymorphism using inheritance is a feature that in both Python and C++ is easy to overcomplicate with the multiple inheritance feature. Alternatives are almost always preferable. Some people also appreciate C++'s improvements upon C's type sytem enough to compile all their C programs with C++. I dig const qualifiers, even though I'm comletely fine with their absence from Python. I'd use C++ for stuff--but Python is perfect for my needs, meanwhile the advantages I'd get from using C++ wouldn't be relevant to my work. If I ever need to write a device driver I'm really screwed. -- Neil Cerutti From bc at freeuk.com Wed Oct 11 11:47:38 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 16:47:38 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 11/10/2017 15:52, breamoreboy at gmail.com wrote: > On Wednesday, October 11, 2017 at 3:14:51 PM UTC+1, bartc wrote: >> On 11/10/2017 14:16, Marko Rauhamaa wrote: >> >>> Python and C don't try to protect you. In return, you get syntactic >>> convenience that probably enhances the quality of your programs. >> >> Python, maybe. C syntax isn't as painful as C++ but I still have a lot >> of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. >> The name of the variable can be found lurking in that lot somewhere, but >> what's the type?) Not so convenient. > https://cdecl.org/ tells me that your variable declaration is a syntax error so maybe not much of an example. Perhaps you didn't write or paste it properly. The site tells me that: char(*(*x[3])())[5] (with or without a trailing semicolon) means: declare x as array 3 of pointer to function returning pointer to array 5 of char (Example taken from page 122 of the C book "K&R2", in a section about writing a program to make sense of complex declarations.) Anyway that fact you either tripped up on typing it, or that you had to use a special tool to find out what it meant, sort of reinforces my point... -- bartc From p.f.moore at gmail.com Wed Oct 11 11:50:13 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 11 Oct 2017 16:50:13 +0100 Subject: Unable to run pip in Windows 10 In-Reply-To: <001c01d342a7$ab331660$01994320$@com> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> <001c01d342a7$ab331660$01994320$@com> Message-ID: "Access is denied" sounds like you've installed Python for all users (i.e. in C:\Program Files) and you're not using an elevated prompt to run pip. If you have Python in C:\Program Files, you either need to install modules in your user environment (via pip install --user) which means they'll only be visible to you, or you need to be running as a system administrator to install them for anyone (which means running pip from a command prompt that was opened via "run as administrator") Paul On 11 October 2017 at 16:43, Michael Cuddehe wrote: > Running 64 bit Windows 10. > Re-installed Python 3.6.3 - Download Windows x86-64 web-based installer. Same problem. > Screen capture attached. > > > -----Original Message----- > From: Paul Moore [mailto:p.f.moore at gmail.com] > Sent: Wednesday, October 11, 2017 10:10 AM > To: Michael Cuddehe > Cc: Thomas Jollans; python-list at python.org > Subject: Re: Unable to run pip in Windows 10 > > On 11 October 2017 at 15:46, Michael Cuddehe wrote: >> - What exactly did you install? >>>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) >>>> [MSC >> v.1900 64 bit (AMD64)] on win32 >>>> Downloaded from python.org. >> - Can you start the Python interpreter? >>>> Yes...works fine. >> * How exactly did you go about this >>>> ?? >> - How exactly do you try to run pip? >>>> Command prompt: c:\Users\FFC>pip import (module) >>>> >> - What exactly happens when you try? >>>> Windows error message: "This app can't run on your PC...." > > There's no "pip import" command - I guess you meant "pip install"? > > Rather than transcribing what you did, can you include a copy and paste of the actual text in your command window? > > I also googled for the message you quoted, and I see that it's a Windows message (not one from pip or Python). The most obvious suggestion from that was "you're running a 64-bit application on a 32-bit Windows". I note from the above: > >> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) >> [MSC >> v.1900 64 bit (AMD64)] > > Are you running 32-bit Windows? If so, you got the wrong version of Python (you need the 32-bit version). > Paul > > > --- > This email has been checked for viruses by AVG. > http://www.avg.com From torriem at gmail.com Wed Oct 11 11:54:46 2017 From: torriem at gmail.com (Michael Torrie) Date: Wed, 11 Oct 2017 09:54:46 -0600 Subject: Unable to run pip in Windows 10 In-Reply-To: <001b01d3429f$c4b38910$4e1a9b30$@com> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> Message-ID: <62d964cc-79dc-62a0-dfa2-4ccedb31c922@gmail.com> On 10/11/2017 08:46 AM, Michael Cuddehe wrote: > - What exactly did you install? >>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC > v.1900 64 bit (AMD64)] on win32 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ So your OS is 32 bit? If so, you can't run 64-bit software on it. This v.1900 was downloaded from python.org? Or is it from a different web site? From bc at freeuk.com Wed Oct 11 12:01:23 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 17:01:23 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 11/10/2017 15:36, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: >> Python, maybe. C syntax isn't as painful as C++ but I still have a lot of >> trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The >> name of the variable can be found lurking in that lot somewhere, but what's >> the type?) Not so convenient. > > People love showcasing stupid examples like that. But how often do you > REALLY make declarations that complex? That's not technically > strawmanning, since C syntax does indeed include that, but you're > cherry-picking the most extreme example. Sure. Statistically most declarations are going to be things like 'int' or 'char*. But more complicated ones (usually not as bad as the example), crop up often enough to be a nuisance. I may see them more than others because I very often need to interface one of my languages with some API defined in C, and I need to exactly understand what the types are so that I can create compatible ones. Anyone else can just include the apposite headers and be done with it without needing to see what's inside. While on the subject of C syntax, here are some fun ambiguities: f(x); // call function with arg x, or declare x of type f? a*b; // multiply a by b, or declare b of type pointer to a? (a)*b // multiply a by b, or cast *b to type a? I understand that in C++, you also have things like this, but in spades. -- bartc From BILL_NOSPAM at whoknows.net Wed Oct 11 12:07:03 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 11 Oct 2017 12:07:03 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: Grant Edwards wrote: > On 2017-10-11, Bill wrote: > > >> [...] I'm not here to "cast stones", I like Python. I just think >> that you shouldn't cast stones at C/C++. > Not while PHP exists. There aren't enough stones in the world... > PHP seems (seemed?) popular for laying out web pages. Are their vastly superior options? I'm a little naive in this matter, thus my question. From jonathanccast at fastmail.fm Wed Oct 11 12:16:45 2017 From: jonathanccast at fastmail.fm (Jonathan Cast) Date: Wed, 11 Oct 2017 11:16:45 -0500 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <5HpDB.6442$cW1.5993@fx07.am4> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: <1507738605.3238.7.camel@fastmail.fm> On Wed, 2017-10-11 at 15:14 +0100, bartc wrote: > On 11/10/2017 14:16, Marko Rauhamaa wrote: > > Python and C don't try to protect you. In return, you get syntactic > > convenience that probably enhances the quality of your programs. > > Python, maybe. C syntax isn't as painful as C++ but I still have a lot > of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. > The name of the variable can be found lurking in that lot somewhere, but > what's the type?) Not so convenient. I believe the type of any variable in C is the same as its declaration, but with the variable name deleted. So: char (*(*[3])())[5] That is, an array of 3 pointers to functions that return pointers to arrays of 5 characters. Jonathan From rhodri at kynesim.co.uk Wed Oct 11 12:29:05 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Wed, 11 Oct 2017 17:29:05 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 11/10/17 15:36, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: >> Python, maybe. C syntax isn't as painful as C++ but I still have a lot of >> trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The >> name of the variable can be found lurking in that lot somewhere, but what's >> the type?) Not so convenient. > > People love showcasing stupid examples like that. But how often do you > REALLY make declarations that complex? That's not technically > strawmanning, since C syntax does indeed include that, but you're > cherry-picking the most extreme example. That's only really one level more complex than declarations I use fairly regularly (I am an embedded system programmer most of the time). On the other hand, I never actually do declare things in that way: typedef is your friend, and makes your C code much easier to read. -- Rhodri James *-* Kynesim Ltd From mikhailwas at gmail.com Wed Oct 11 13:25:34 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Wed, 11 Oct 2017 19:25:34 +0200 Subject: Lies in education [was Re: The "loop and a half"] Message-ID: > >> [...] I'm not here to "cast stones", I like Python. I just think > >> that you shouldn't cast stones at C/C++. > > Not while PHP exists. There aren't enough stones in the world... > > > > PHP seems (seemed?) popular for laying out web pages. Are their vastly > superior options? Python? Superior syntax for sure From rosuav at gmail.com Wed Oct 11 13:41:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 04:41:16 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87lgkhd70j.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 12, 2017 at 2:43 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> The places where C++ is not a superset of C are mostly things you >> wouldn't want to be doing anyway. You can generally take C code and >> compile it with a C++ compiler, and it'll have the same semantics. > > Here's a C/C++ program: > > ======================================================================== > #include > > int main() > { > struct {} s; > printf("%d\n", (int) sizeof 'a'); > printf("%d\n", (int) sizeof s); > return 0; > } > ======================================================================== > > When compiled (with gcc) as a C program, the output is: > > 4 > 0 > > When the same program is compiled (with gcc) as a C++ program, the > output is: > > 1 > 1 > > That is not immediately all that significant but points to subtle > incompatibilities between the data models of C and C++. Indeed - their handling of empty structs is different. But that doesn't invalidate my point; how often do you declare a variable that has nothing in it? Remember, a Python object with no attributes still has an identity, a type, and so on; in C, data has no type or identity, so this truly has no data attributes. This is legal but extremely esoteric, and the difference in sizeof is less about an incompatible data model and more about the definition of an empty struct. > Then we have syntactic problems: > > ======================================================================== > int main() > { > void *s = "hello"; > char *t = s; > return 0; > } > ======================================================================== > > which, as a C program, makes gcc perfectly happy, but a C++ compilation > complains: > > test.C: In function ?int main()?: > test.C:5:15: error: invalid conversion from ?const void*? to ?void*? \ > [-fpermissive] > void *s = "hello"; > ^~~~~~~ > test.C:6:15: error: invalid conversion from ?void*? to ?char*? [-fper\ > missive] > char *t = s; > ^ > > (The first one should trigger an error message even in C compilation, > but string literals were standardized to be "semiconstant" in C.) Also something you should not be using. For hysterical raisins, certain things in C are legal-yet-highly-inadvisable, and one of them is putting string literals into non-const pointers. (C didn't originally even _have_ const.) You'll probably find that some compilers give a warning (at least if -Wall is active), but even if not, you shouldn't be doing this. So I stand by my original statement. Yes, there are incompatibilities, but it's easy enough to write code that's compatible with both. Hmm, something tells me I've heard this sort of thing before... ChrisA From tjreedy at udel.edu Wed Oct 11 13:56:48 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Oct 2017 13:56:48 -0400 Subject: Unable to run pip in Windows 10 In-Reply-To: <62d964cc-79dc-62a0-dfa2-4ccedb31c922@gmail.com> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> <62d964cc-79dc-62a0-dfa2-4ccedb31c922@gmail.com> Message-ID: On 10/11/2017 11:54 AM, Michael Torrie wrote: > On 10/11/2017 08:46 AM, Michael Cuddehe wrote: >> - What exactly did you install? >>>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC >> v.1900 64 bit (AMD64)] on win32 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is exactly what I see from 64 bit python installed on 64 bit Windows. 'win32' refers to the Windows platform, versus linux, unix, darwin, etc. Yes, it is confusing. > So your OS is 32 bit? The next line says that this binary runs fine. So 'No'. The problem is specific to trying to run pip. > This v.1900 was downloaded from python.org? MSC v.1900 is the latest Microsoft C compiler, which was used to compile the binary. It is not downloaded. -- Terry Jan Reedy From tjreedy at udel.edu Wed Oct 11 14:00:14 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Oct 2017 14:00:14 -0400 Subject: Unable to run pip in Windows 10 In-Reply-To: <001b01d3429f$c4b38910$4e1a9b30$@com> References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> Message-ID: On 10/11/2017 10:46 AM, Michael Cuddehe wrote: > - What exactly did you install? >>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC > v.1900 64 bit (AMD64)] on win32 >>> Downloaded from python.org. > - Can you start the Python interpreter? >>> Yes...works fine. > * How exactly did you go about this >>> ?? > - How exactly do you try to run pip? >>> Command prompt: c:\Users\FFC>pip import (module) Try running instead ...> python -m pip install or to specifically ensure that you run 3.5, ...> py -3.5 -m pip install and if that fails, copy the entire response. > - What exactly happens when you try? >>> Windows error message: "This app can't run on your PC...." -- Terry Jan Reedy From grant.b.edwards at gmail.com Wed Oct 11 14:22:09 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 11 Oct 2017 18:22:09 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 2017-10-11, bartc wrote: > On 11/10/2017 15:36, Chris Angelico wrote: >> On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: >>> Python, maybe. C syntax isn't as painful as C++ but I still have a lot of >>> trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The >>> name of the variable can be found lurking in that lot somewhere, but what's >>> the type?) Not so convenient. >> >> People love showcasing stupid examples like that. But how often do you >> REALLY make declarations that complex? That's not technically >> strawmanning, since C syntax does indeed include that, but you're >> cherry-picking the most extreme example. > > Sure. Statistically most declarations are going to be things like 'int' > or 'char*. But more complicated ones (usually not as bad as the > example), crop up often enough to be a nuisance. The easiest way to make stuff like that readable is to unroll them into a sequence of typedefs. But, a lot of people never really learn how to do that... -- Grant Edwards grant.b.edwards Yow! ... I don't like FRANK at SINATRA or his CHILDREN. gmail.com From grant.b.edwards at gmail.com Wed Oct 11 14:24:59 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 11 Oct 2017 18:24:59 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-11, Bill wrote: > Grant Edwards wrote: >> On 2017-10-11, Bill wrote: >> >> >>> [...] I'm not here to "cast stones", I like Python. I just think >>> that you shouldn't cast stones at C/C++. >> Not while PHP exists. There aren't enough stones in the world... >> > > PHP seems (seemed?) popular for laying out web pages. It's not really used for "laying out" web pages. Thats what HTML and CSS do. PHP is for server-side generation of dynamic content and handling of submitted forms and uploaded data. > Are their vastly superior options? Yes: Python. There are other choices that while superior to PHP aren't as good as Python. :) > I'm a little naive in this matter, thus my question. -- Grant Edwards grant.b.edwards Yow! Those people look at exactly like Donnie and gmail.com Marie Osmond!! From rosuav at gmail.com Wed Oct 11 14:30:07 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 05:30:07 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On Thu, Oct 12, 2017 at 5:22 AM, Grant Edwards wrote: > On 2017-10-11, bartc wrote: >> On 11/10/2017 15:36, Chris Angelico wrote: >>> On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: >>>> Python, maybe. C syntax isn't as painful as C++ but I still have a lot of >>>> trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The >>>> name of the variable can be found lurking in that lot somewhere, but what's >>>> the type?) Not so convenient. >>> >>> People love showcasing stupid examples like that. But how often do you >>> REALLY make declarations that complex? That's not technically >>> strawmanning, since C syntax does indeed include that, but you're >>> cherry-picking the most extreme example. >> >> Sure. Statistically most declarations are going to be things like 'int' >> or 'char*. But more complicated ones (usually not as bad as the >> example), crop up often enough to be a nuisance. > > The easiest way to make stuff like that readable is to unroll them > into a sequence of typedefs. But, a lot of people never really > learn how to do that... The most complexity you'll usually see is a function that accepts/returns a pointer, and those generally should be typedef'd. But even if not, they're still not anything like as bad as the mythical examples that get touted as "why C is bad" or "why variable declarations are bad" or "why type declarations are bad". ChrisA From marko at pacujo.net Wed Oct 11 14:35:53 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 11 Oct 2017 21:35:53 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> Message-ID: <87r2u97cs6.fsf@elektro.pacujo.net> Chris Angelico : > On Thu, Oct 12, 2017 at 2:43 AM, Marko Rauhamaa wrote: >> That is not immediately all that significant but points to subtle >> incompatibilities between the data models of C and C++. > > Indeed - their handling of empty structs is different. But that > doesn't invalidate my point; how often do you declare a variable that > has nothing in it? In my case, this particular problem caused a bug once some years back. > Remember, a Python object with no attributes still > has an identity, a type, and so on; in C, data has no type or > identity, so this truly has no data attributes. This is legal but > extremely esoteric, and the difference in sizeof is less about an > incompatible data model and more about the definition of an empty > struct. Here's a tiny example from the Linux kernel: ======================================================================== struct snd_ac97_build_ops { int (*build_3d) (struct snd_ac97 *ac97); int (*build_specific) (struct snd_ac97 *ac97); int (*build_spdif) (struct snd_ac97 *ac97); int (*build_post_spdif) (struct snd_ac97 *ac97); #ifdef CONFIG_PM void (*suspend) (struct snd_ac97 *ac97); void (*resume) (struct snd_ac97 *ac97); #endif void (*update_jacks) (struct snd_ac97 *ac97); /* for jack-sharing */ }; ======================================================================== Such #ifdef'ed structures abound in the Linux kernel. We had a similar situation with the *whole* structure consisting of #ifdef'ed segments. I had naively assumed that word alignment was preserved regardless of the preprocessing conditions. > So I stand by my original statement. Yes, there are incompatibilities, > but it's easy enough to write code that's compatible with both. Hmm, > something tells me I've heard this sort of thing before... Python++ FTW? Marko From rosuav at gmail.com Wed Oct 11 15:01:38 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 06:01:38 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87r2u97cs6.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <87r2u97cs6.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 12, 2017 at 5:35 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Thu, Oct 12, 2017 at 2:43 AM, Marko Rauhamaa wrote: >>> That is not immediately all that significant but points to subtle >>> incompatibilities between the data models of C and C++. >> >> Indeed - their handling of empty structs is different. But that >> doesn't invalidate my point; how often do you declare a variable that >> has nothing in it? > > In my case, this particular problem caused a bug once some years back. Once. Probably in your entire career. In my case, never, and I spent about fifteen years of my life writing mostly either C or C++ code. This is NOT common. ChrisA From bc at freeuk.com Wed Oct 11 15:16:26 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 20:16:26 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <1507738605.3238.7.camel@fastmail.fm> Message-ID: <46uDB.18957$Jk.1555@fx30.am4> On 11/10/2017 17:16, Jonathan Cast wrote: > On Wed, 2017-10-11 at 15:14 +0100, bartc wrote: >> On 11/10/2017 14:16, Marko Rauhamaa wrote: >>> Python and C don't try to protect you. In return, you get syntactic >>> convenience that probably enhances the quality of your programs. >> >> Python, maybe. C syntax isn't as painful as C++ but I still have a lot >> of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. >> The name of the variable can be found lurking in that lot somewhere, but >> what's the type?) Not so convenient. > > I believe the type of any variable in C is the same as its declaration, > but with the variable name deleted. Yes, I think we got that... > So: > > char (*(*[3])())[5] ..which doesn't help, and in fact makes things worse, as now you don't have a start point at which to start unravelling it. You have to do it from the inside out. > That is, an array of 3 pointers to functions that return pointers to > arrays of 5 characters. But you left out the dozen steps needed to get from that to this! Anyway if such a type can be more clearly expressed like this, why doesn't a language simply allow that, or near enough? Why does it need to be cryptic, or require an external tool to encode and decode (there is a reason that CDECL exists) or require the programmer to apply an algorithm? -- bartc From rosuav at gmail.com Wed Oct 11 15:30:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 06:30:58 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On Thu, Oct 12, 2017 at 3:29 AM, Rhodri James wrote: > On 11/10/17 15:36, Chris Angelico wrote: >> >> On Thu, Oct 12, 2017 at 1:14 AM, bartc wrote: >>> >>> Python, maybe. C syntax isn't as painful as C++ but I still have a lot of >>> trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. The >>> name of the variable can be found lurking in that lot somewhere, but >>> what's >>> the type?) Not so convenient. >> >> >> People love showcasing stupid examples like that. But how often do you >> REALLY make declarations that complex? That's not technically >> strawmanning, since C syntax does indeed include that, but you're >> cherry-picking the most extreme example. > > > That's only really one level more complex than declarations I use fairly > regularly (I am an embedded system programmer most of the time). On the > other hand, I never actually do declare things in that way: typedef is your > friend, and makes your C code much easier to read. I wouldn't consider embedded systems to be the most common kind of C coding out there, particularly when people compare against C++ (I don't remember ever hearing of anyone doing embedded work in C++, though I'm sure it does happen). Nevertheless, you're exactly right about the typedefs. Writing crazily complicated type declarations without typedefs is like writing massively nested list comprehensions without intermediate variables. Hey look, Python's terrible! Or maybe they're just non-idiomatic examples. ChrisA From bc at freeuk.com Wed Oct 11 15:49:35 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 20:49:35 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: <9BuDB.15425$uS.4321@fx24.am4> On 11/10/2017 19:30, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 5:22 AM, Grant Edwards >> The easiest way to make stuff like that readable is to unroll them >> into a sequence of typedefs. But, a lot of people never really >> learn how to do that... > > The most complexity you'll usually see is a function that > accepts/returns a pointer, and those generally should be typedef'd. > But even if not, they're still not anything like as bad as the > mythical examples that get touted as "why C is bad" or "why variable > declarations are bad" or "why type declarations are bad". IME typedefs can make things worse, especially if used even for basic types. Then you spend your time tracing back typedefs to see what they mean (was it a char, or struct or what?). C can combine the abstruseness of its type declarations, overuse of macros, and overuse of #if/ifdef blocks to render a lot of code a complete nightmare to try and understand. Or debug. Or maintain. Or port. Or implement (if testing a compiler). I've looked at some of my generated C code, and here's one common example, not a complex type: tokenrec * (*)[] But I couldn't figure that out from the top of my head. Bear in mind this was in a parameter list that can change the meaning of []. I looked at the original source and that type is written like this: ref [] ref tokenrec 'ref' means 'pointer to', and it can also be written like this: pointer [] pointer tokenrec In English, 'pointer to array of pointer to tokenrec'. See how the source language corresponds almost exactly to the English, and both are written left to right. Now go back and look at the C version - where do you even start? Are those parentheses significant? (I've no idea.) The most complex of /my/ types to appear in generated C is probably this, used as a cast in this assignment: pcptr = (*(int64 * (**) (void))pcptr)(); (Original source is this: pcptr := ref ref function=>ref int64(pcptr)^^() In English, the cast is 'pointer to pointer to function returning pointer to int64'. The line includes two dereferences (^^) which /I think/ account for two of the "*" in the C version. Which * are dereferences, and which are part of the type, or indeed whether only one * is the dereference, I have absolutely no idea. Great language...) -- bartc From bc at freeuk.com Wed Oct 11 15:53:35 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 20:53:35 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 11/10/2017 20:30, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 3:29 AM, Rhodri James wrote: >> On 11/10/17 15:36, Chris Angelico wrote: >> That's only really one level more complex than declarations I use fairly >> regularly (I am an embedded system programmer most of the time). On the >> other hand, I never actually do declare things in that way: typedef is your >> friend, and makes your C code much easier to read. > > I wouldn't consider embedded systems to be the most common kind of C > coding out there, particularly when people compare against C++ (I > don't remember ever hearing of anyone doing embedded work in C++, > though I'm sure it does happen). Nevertheless, you're exactly right > about the typedefs. Writing crazily complicated type declarations > without typedefs is like writing massively nested list comprehensions > without intermediate variables. Hey look, Python's terrible! Or maybe > they're just non-idiomatic examples. Look at my last example posted a few minutes before this. You'd say the C needed typedefs. But in the original language, type declarations are clear enough that they don't need typedefs. The typedefs help to mitigate a problem in one language that doesn't exist in the other. Surely it's better to have neither the cryptic type nor the typedef. Everything would be much cleaner. -- bartc From BILL_NOSPAM at whoknows.net Wed Oct 11 16:07:14 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 11 Oct 2017 16:07:14 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: Message-ID: Mikhail V wrote: >>>> [...] I'm not here to "cast stones", I like Python. I just think >>>> that you shouldn't cast stones at C/C++. >>> Not while PHP exists. There aren't enough stones in the world... >>> >> PHP seems (seemed?) popular for laying out web pages. Are their vastly >> superior options? > Python? Superior syntax for sure I believe that. What accounts for the popularity of PHP then? From BILL_NOSPAM at whoknows.net Wed Oct 11 16:16:04 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 11 Oct 2017 16:16:04 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: Grant Edwards wrote: > On 2017-10-11, Bill wrote: >> Grant Edwards wrote: >>> On 2017-10-11, Bill wrote: >>> >>> >>>> [...] I'm not here to "cast stones", I like Python. I just think >>>> that you shouldn't cast stones at C/C++. >>> Not while PHP exists. There aren't enough stones in the world... >>> >> PHP seems (seemed?) popular for laying out web pages. > It's not really used for "laying out" web pages. Thats what HTML and > CSS do. PHP is for server-side generation of dynamic content and > handling of submitted forms and uploaded data. Thank you. I DO appreciate learning about the correct terminology. From grant.b.edwards at gmail.com Wed Oct 11 16:27:09 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 11 Oct 2017 20:27:09 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: Message-ID: On 2017-10-11, Bill wrote: > Mikhail V wrote: >>>>> [...] I'm not here to "cast stones", I like Python. I just think >>>>> that you shouldn't cast stones at C/C++. >>>> Not while PHP exists. There aren't enough stones in the world... >>>> >>> PHP seems (seemed?) popular for laying out web pages. Are their vastly >>> superior options? >> Python? Superior syntax for sure > > I believe that. What accounts for the popularity of PHP then? I ask myself that everytime I have to work with it. -- Grant Edwards grant.b.edwards Yow! FOOLED you! Absorb at EGO SHATTERING impulse gmail.com rays, polyester poltroon!! From rosuav at gmail.com Wed Oct 11 16:27:47 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 07:27:47 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: Message-ID: On Thu, Oct 12, 2017 at 7:07 AM, Bill wrote: > Mikhail V wrote: >>>>> >>>>> [...] I'm not here to "cast stones", I like Python. I just think >>>>> that you shouldn't cast stones at C/C++. >>>> >>>> Not while PHP exists. There aren't enough stones in the world... >>>> >>> PHP seems (seemed?) popular for laying out web pages. Are their vastly >>> superior options? >> >> Python? Superior syntax for sure > > > I believe that. What accounts for the popularity of PHP then? It's popular, therefore people use it. It's purely self-perpetuating. There was a time (a time forever gone) when you could get extremely cheap web hosting with PHP support, but other scripting languages were only available if you paid more. That created a culture of low-grade sites gravitating to PHP and MySQL, which meant that software aimed at them (WordPress, various forum software, MediaWiki) would be written in PHP. And that means that anyone who wants to mod them (WordPress particularly) has to learn PHP. It's an endless cycle. But since it's the lowest-end sites that have traditionally driven that demand for PHP, there's a general tendency for low-grade programmers to gravitate to it, so there's a lot of really REALLY bad code out there. Yes, someone's going to chime in with (a) "You can write good code in PHP" and/or (b) "Big sites like Facebook and Wikipedia use PHP, so it must be fine". I don't care. About either. They're irrelevant to someone who's looking over the job postings... the chances that a PHP job will involve bad code are far higher than, say, Haskell jobs, which will be rarer but much less hit-and-miss. There's basically no reason to use PHP unless you're working in existing PHP code (as mentioned, things like WordPress plugins). Also: http://thedailywtf.com/articles/are-you-down-with-php- ChrisA From ben.usenet at bsb.me.uk Wed Oct 11 16:42:04 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 11 Oct 2017 21:42:04 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: Message-ID: <87po9tmn6r.fsf@bsb.me.uk> Bill writes: > Mikhail V wrote: >>>>> [...] I'm not here to "cast stones", I like Python. I just think >>>>> that you shouldn't cast stones at C/C++. >>>> Not while PHP exists. There aren't enough stones in the world... >>>> >>> PHP seems (seemed?) popular for laying out web pages. Are their vastly >>> superior options? >> Python? Superior syntax for sure > > I believe that. What accounts for the popularity of PHP then? Two things, probably. First, it was almost always pre-installed even on low-cost hosting. Second, you could start very simply because it was designed to be used embedded. Tiny little bits of code could so something -- not need for a "framework". Other languages were not always installed (I'm sure it's better these days) and those that were (Perl almost always was) could not, by default, be used embedded -- you had to generate the whole page. What is (or are) the Python way (or ways) to do it? -- Ben. From rosuav at gmail.com Wed Oct 11 16:50:58 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 07:50:58 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87po9tmn6r.fsf@bsb.me.uk> References: <87po9tmn6r.fsf@bsb.me.uk> Message-ID: On Thu, Oct 12, 2017 at 7:42 AM, Ben Bacarisse wrote: > Bill writes: > >> Mikhail V wrote: >>>>>> [...] I'm not here to "cast stones", I like Python. I just think >>>>>> that you shouldn't cast stones at C/C++. >>>>> Not while PHP exists. There aren't enough stones in the world... >>>>> >>>> PHP seems (seemed?) popular for laying out web pages. Are their vastly >>>> superior options? >>> Python? Superior syntax for sure >> >> I believe that. What accounts for the popularity of PHP then? > > Two things, probably. First, it was almost always pre-installed even on > low-cost hosting. Second, you could start very simply because it was > designed to be used embedded. Tiny little bits of code could so > something -- not need for a "framework". > > Other languages were not always installed (I'm sure it's better these > days) and those that were (Perl almost always was) could not, by > default, be used embedded -- you had to generate the whole page. > > What is (or are) the Python way (or ways) to do it? Check out Django and Flask, the two most popular ways. I quite like Flask. ChrisA From breamoreboy at gmail.com Wed Oct 11 16:52:16 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 11 Oct 2017 13:52:16 -0700 (PDT) Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On Wednesday, October 11, 2017 at 4:47:43 PM UTC+1, bartc wrote: > On 11/10/2017 15:52, wrote: > > On Wednesday, October 11, 2017 at 3:14:51 PM UTC+1, bartc wrote: > >> On 11/10/2017 14:16, Marko Rauhamaa wrote: > >> > >>> Python and C don't try to protect you. In return, you get syntactic > >>> convenience that probably enhances the quality of your programs. > >> > >> Python, maybe. C syntax isn't as painful as C++ but I still have a lot > >> of trouble with it. (Eg. the variable declaration 'char(*(*x[3])())[5]'. > >> The name of the variable can be found lurking in that lot somewhere, but > >> what's the type?) Not so convenient. > > > https://cdecl.org/ tells me that your variable declaration is a syntax error so maybe not much of an example. > > Perhaps you didn't write or paste it properly. The site tells me that: > > char(*(*x[3])())[5] > > (with or without a trailing semicolon) means: > > declare x as array 3 of pointer to function returning pointer to > array 5 of char > > (Example taken from page 122 of the C book "K&R2", in a section about > writing a program to make sense of complex declarations.) > > Anyway that fact you either tripped up on typing it, or that you had to > use a special tool to find out what it meant, sort of reinforces my point... > > -- > bartc Don't know what happened there but what the heck. More importantly is the fact that due to your magnificent performance recently you have been promoted to be the General Manager of my Dream Team. You can of course cement your place when you explain how, in your language, converting an invalid piece of user input, which should be an integer, is always converted to zero, and how you handle the inevitable divide by zero errors that will always, eventually, occur. -- Kindest regards. Mark Lawrence. From bc at freeuk.com Wed Oct 11 17:35:58 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 22:35:58 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: On 11/10/2017 21:52, breamoreboy at gmail.com wrote: >> More importantly is the fact that due to your magnificent performance recently you have > been promoted to be the General Manager of my Dream Team. Thanks, I guess. > You can of course cement your place when you explain how, in your language, converting an invalid piece of user input, which should be an integer, is always converted to zero, and how you handle the inevitable divide by zero errors that will always, eventually, occur. You mean it shouldn't do what happens here (Py3): a = input("? ").split() x = int(a[0]) y = int(a[1]) print (x,y) print (x/y) and somebody enters '10 0' ? I don't think you can do much about that. However since that thread I've tweaked the way I do this, so that here [non-Python code]: print "? " readln a, b # read each as int, float or string println a/b # floating point divide this produces these a/b results for various inputs: 10 20 # 0.500000 10,20 # 0.500000 10skjhf 20 # error, divide string by int 17.9 2 # 8.950000 10 # error, divide int by string ("") # (blank) error, divide string by string .1e10 1e5 # 10000.000000 ten twenty # error, divide string by string For throwaway programs, or for testing, or for trusted input, this is perfectly reasonable. For perfect error checking, you need to do a bit more work on either verifying input, or using more sophisticated parsing. -- bartc From mikhailwas at gmail.com Wed Oct 11 17:45:06 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Wed, 11 Oct 2017 23:45:06 +0200 Subject: Lies in education [was Re: The "loop and a half"] Message-ID: Bill wrote: > Mikhail V wrote: > > Python? Superior syntax for sure > > I believe that. What accounts for the popularity of PHP then? I can't tell for PHP for sure... As in many cases in software world, there is a principle of "who was the first there to solve some task". Probably also it was bundled with many Web solutions at the time when Web market started to grow, IDK. PS Off-topic: I have a related observation regarding popularity of software. There is such a program "VLC", which is a video player. Some would think it is sort of best free player, etc. I was also under impression, but then I've found a video player called "MPC-HC". I have started to use it and quickly understood that it is by far more superior player than VLC, literally by all core parameters - performance, codec support, customizable key bindings with a lot of internal commands. (Remark for Linux users: there is no Linux version of that player, it is Windows-only) So, I think one key to popularity is mostly related to *marketing*, in case of MPC, there is also a problem of a *good name* for the software. Once I was trying to remember, what was that nice player and just couldnt remember. MPC, MTC, HTC... So if it was from the beginning somthing more catchy like "Vulcano" or "Tornado" then I bet it would had way bigger user community now. And surprisingly, VLC seems to be more popular (if we take Windows), despite it can't even read some video formats, which MPC can. So go figure. Most important, one should not rely on those trashy "Top-10 ..." articles, and don't judge by the fancy GUI, but try the software yourself for some time. From greg.ewing at canterbury.ac.nz Wed Oct 11 17:55:36 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 12 Oct 2017 10:55:36 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> Message-ID: bartc wrote: > While on the subject of C syntax, here are some fun ambiguities: > > f(x); // call function with arg x, or declare x of type f? > > a*b; // multiply a by b, or declare b of type pointer to a? > > (a)*b // multiply a by b, or cast *b to type a? Technically these are not ambiguous in C, because the context always makes it clear whether you're dealing with a declaration or an expression. The indications can be subtle sometimes, but they're there. In C++ they really are ambiguous in some cases, and some arbitrary rules had to be added to disambiguate them. (I think the rule is "if it could be a declaration, then it is" or something like that.) -- Greg From greg.ewing at canterbury.ac.nz Wed Oct 11 18:03:39 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 12 Oct 2017 11:03:39 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <9BuDB.15425$uS.4321@fx24.am4> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> Message-ID: bartc wrote: > tokenrec * (*)[] > > the original source and that type is written like this: > > ref [] ref tokenrec The idiomatic way to write that type in C would be tokenrec ** -- Greg From greg.ewing at canterbury.ac.nz Wed Oct 11 18:10:33 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 12 Oct 2017 11:10:33 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: Neil Cerutti wrote: > I dig > const qualifiers, even though I'm comletely fine with their > absence from Python. Out of curiosity, do you have any insights into why you like them in C++, if you don't miss them in Python? -- Greg From bc at freeuk.com Wed Oct 11 18:39:24 2017 From: bc at freeuk.com (bartc) Date: Wed, 11 Oct 2017 23:39:24 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> Message-ID: On 11/10/2017 23:03, Gregory Ewing wrote: > bartc wrote: > >> ??? tokenrec * (*)[] > > >> the original source and that type is written like this: >> >> ??? ref [] ref tokenrec > > The idiomatic way to write that type in C would be > > ?? tokenrec ** The original has an extra pointer so idiomatic C might be more: tokenrec *** But this still defines a different type, namely: pointer to pointer to pointer to tokenrec not: pointer to array of pointer to tokenrec If you want to lose all array information, and really don't care if you're dealing with a pointer, or an array (and don't mind changing how such a value is passed, and how it is accessed) then this is fine for you. You just have to access a 'tokenrec ***args' parameter as ***args. Or **args[i]. Or *args[i][j]. **(args[i]). Or ***args. Or args[i][j][k]. Yes, any combination will work! Only one will be correct. In the original source, it can only be accessed one way - the correct way. Using the non-idiomatic 'tokenrec *(*)[]' in C imposes some extra constraints, but doesn't do the full job. However my complaint was about the syntax; the type system of C is another subject. (How I ended up talking about C in this group I don't know. But yesterday I mentioned Python in the C group, so ...) -- bartc From ben.usenet at bsb.me.uk Wed Oct 11 18:44:04 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 11 Oct 2017 23:44:04 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> Message-ID: <87h8v5mhjf.fsf@bsb.me.uk> Chris Angelico writes: > On Thu, Oct 12, 2017 at 7:42 AM, Ben Bacarisse wrote: >> Bill writes: >> >>> Mikhail V wrote: >>>>>>> [...] I'm not here to "cast stones", I like Python. I just think >>>>>>> that you shouldn't cast stones at C/C++. >>>>>> Not while PHP exists. There aren't enough stones in the world... >>>>>> >>>>> PHP seems (seemed?) popular for laying out web pages. Are their vastly >>>>> superior options? >>>> Python? Superior syntax for sure >>> >>> I believe that. What accounts for the popularity of PHP then? >> >> Two things, probably. First, it was almost always pre-installed even on >> low-cost hosting. Second, you could start very simply because it was >> designed to be used embedded. Tiny little bits of code could so >> something -- not need for a "framework". >> >> Other languages were not always installed (I'm sure it's better these >> days) and those that were (Perl almost always was) could not, by >> default, be used embedded -- you had to generate the whole page. >> >> What is (or are) the Python way (or ways) to do it? > > Check out Django and Flask, the two most popular ways. I quite like > Flask. I see. Both appear to be frameworks (I'd heard of Django). Do you know if they widely available on low-cost hosting packages? (I don't think they are on mine, but that's dirt-cheap because I don't use it for anything important!) One thing that helped PHP was that it could be used (and learned) in an incremental way. You could add a "this page last updated on..." text in a line or two to an existing page. Then a button to change the layout or theme. Then a simple form and so on. Many professionals started that way. In the early days, there ware few other routes into the technical side of web authoring. That helped to cement PHP as the dominant technology because it was what they knew. -- Ben. From ben.usenet at bsb.me.uk Wed Oct 11 18:53:26 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Wed, 11 Oct 2017 23:53:26 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> Message-ID: <87d15tmh3t.fsf@bsb.me.uk> Gregory Ewing writes: > bartc wrote: > >> tokenrec * (*)[] >> >> the original source and that type is written like this: >> >> ref [] ref tokenrec > > The idiomatic way to write that type in C would be > > tokenrec ** That's a different type. I think you mean that a human writing C (rather than bartc's code generator) would probably design the code to use tokenrec ** then I agree, but the latter is not just a different way to write the former. -- Ben. From christopher_reimer at icloud.com Wed Oct 11 19:32:57 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Wed, 11 Oct 2017 16:32:57 -0700 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> Message-ID: <379DBE2E-D89C-4C3E-80F1-8C794F676430@icloud.com> On Oct 11, 2017, at 9:07 AM, Bill wrote: > > Grant Edwards wrote: >> On 2017-10-11, Bill wrote: >> >> >>> [...] I'm not here to "cast stones", I like Python. I just think >>> that you shouldn't cast stones at C/C++. >> Not while PHP exists. There aren't enough stones in the world... >> > > PHP seems (seemed?) popular for laying out web pages. Are their vastly superior options? I'm a little naive in this matter, thus my question. > -- > https://mail.python.org/mailman/listinfo/python-list AFAIK, JavaScript frameworks has largely replaced PHP. I personally use Pelican to generate static web pages and use JavaScript sparingly. Chris R. From rosuav at gmail.com Wed Oct 11 19:33:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 10:33:39 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87h8v5mhjf.fsf@bsb.me.uk> References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> Message-ID: On Thu, Oct 12, 2017 at 9:44 AM, Ben Bacarisse wrote: > Chris Angelico writes: >> Check out Django and Flask, the two most popular ways. I quite like >> Flask. > > I see. Both appear to be frameworks (I'd heard of Django). Do you know > if they widely available on low-cost hosting packages? (I don't think > they are on mine, but that's dirt-cheap because I don't use it for > anything important!) Generally, low-cost hosting will offer Python, and then you can install Flask/Django from there. For example, you can deploy to Heroku (zero-dollar hosting for small sites), and as long as you provide a file called "requirements.txt", they'll install whatever you depend on. > One thing that helped PHP was that it could be used (and learned) in an > incremental way. You could add a "this page last updated on..." text in > a line or two to an existing page. Then a button to change the layout > or theme. Then a simple form and so on. > > Many professionals started that way. In the early days, there ware few > other routes into the technical side of web authoring. That helped to > cement PHP as the dominant technology because it was what they knew. Yeah. The trouble is that this is a really REALLY bad way to design something. Have you seen a city that grew one house at a time, and had streets added to service those houses? Not good. The end result is that PHP is still bound by this "just a bit of scripting inside your HTML" structure, which has annoying consequences in certain areas (like whitespace before the first "") def show_article(article): ... Have a think about what the PHP setup implies. Arbitrary files in your web server tree are not just automatically available (as they'd be if you use flat-file hosting like GitHub Pages), but they are *executable*. You don't have to set the file mode to say "this is executable", you just have to have the file there with the appropriate name. So PHP-based web sites end up having to monitor their uploads and downloads lest someone slip something in and then run it on the server... with full permissions. With Flask, in contrast, you can make an uploads directory that's automatically downloadable, but nothing in it will ever be executed. ChrisA From ben.usenet at bsb.me.uk Wed Oct 11 20:21:33 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 01:21:33 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> Message-ID: <87zi8xkygi.fsf@bsb.me.uk> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Ben Bacarisse writes: >>That's a different type. I think you mean that a human writing C >>(rather than bartc's code generator) would probably design the code to >>use tokenrec ** then I agree, but the latter is not just a different way >>to write the former. > > That most difficult thing in C in the realm of type > declarations for me is: > > Define a function ?g? with a parameter ?x? of type ?int?, so > that this function ?g? returns a pointer to another function. > This other function has a parameter of type ?char? and returns > a double value. > > /Without/ a typedef. You read a C type from the "inside out", going right if you can and left when you can't. That rule can be reversed to built up the type: You know it has (read "g is a function taking an int") g(int x) and since g returns a pointer you will have *g(int x) But it returns a pointer to a function taking a char so we must add (char) on the right but the brackets can't go here: *g(int x)(char) because then you would be read "function taking a char" before the pointer to. We need extra brackets to stop us reading right until we've read left: (*g(int x))(char) This forces "g is a function taking int returning a pointer to...". Finally, we just need the type of the function whose pointer is being returned: double (*g(int x))(char) Check with you finger on the name and reading right when you can and left when you can't (because of brackets). And then you re-write it using a typedef. Knowing how is simply interesting, not useful. -- Ben. From robertvstepp at gmail.com Wed Oct 11 20:38:17 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 11 Oct 2017 19:38:17 -0500 Subject: OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]] Message-ID: On Wed, Oct 11, 2017 at 4:45 PM, Mikhail V wrote: > > PS Off-topic: > I have a related observation regarding popularity of software. > There is such a program "VLC", which is a video player. Some would > think it is sort of best free player, etc. I was also under impression, > but then I've found a video player called "MPC-HC". > I have started to use it and quickly understood that it is by far more > superior player than VLC, literally by all core parameters - performance, > codec support, customizable key bindings with a lot of internal commands. > (Remark for Linux users: there is no Linux version of that player, it > is Windows-only) I went to https://mpc-hc.org/2017/07/16/1.7.13-released-and-farewell/ and it says: July 16, 2017 XhmikosR v1.7.13, the latest, and probably the last release of our project? For quite a few months now, or even years, the number of active developers has been decreasing and has inevitably reached zero. This, unfortunately, means that the project is officially dead and this release would be the last one. ?Unless some people step up that is. So, if someone?s willing to really contribute and has C/C++ experience, let me know on IRC or via e-mail. Otherwise, all things come to an end and life goes on. It?s been a nice journey and I?m personally pretty overwhelmed having to write this post. Thanks to everyone who has contributed in any way all these years; Remember, MPC-HC is an 11-year old project. -- boB From rosuav at gmail.com Wed Oct 11 20:51:07 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 11:51:07 +1100 Subject: OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: References: Message-ID: On Thu, Oct 12, 2017 at 11:38 AM, boB Stepp wrote: > On Wed, Oct 11, 2017 at 4:45 PM, Mikhail V wrote: > >> >> PS Off-topic: >> I have a related observation regarding popularity of software. >> There is such a program "VLC", which is a video player. Some would >> think it is sort of best free player, etc. I was also under impression, >> but then I've found a video player called "MPC-HC". >> I have started to use it and quickly understood that it is by far more >> superior player than VLC, literally by all core parameters - performance, >> codec support, customizable key bindings with a lot of internal commands. >> (Remark for Linux users: there is no Linux version of that player, it >> is Windows-only) > > I went to https://mpc-hc.org/2017/07/16/1.7.13-released-and-farewell/ > and it says: > > > > July 16, 2017 XhmikosR > > v1.7.13, the latest, and probably the last release of our project? > > For quite a few months now, or even years, the number of active > developers has been decreasing and has inevitably reached zero. This, > unfortunately, means that the project is officially dead and this > release would be the last one. > > ?Unless some people step up that is. > > So, if someone?s willing to really contribute and has C/C++ > experience, let me know on IRC or via e-mail. > > Otherwise, all things come to an end and life goes on. It?s been a > nice journey and I?m personally pretty overwhelmed having to write > this post. > > Thanks to everyone who has contributed in any way all these years; > Remember, MPC-HC is an 11-year old project. > > If it wants new life, it's probably going to need a Linux version, because that's where a lot of developers hang out. The reality is that open source developers are much more likely to develop on Linux than on Windows; you can maintain a Windows port of a Linux program with fewer Windows experts than maintaining the entire program on Windows. The other option, though, would be for the useful parts to become feature suggestions for VLC. ChrisA From ben.usenet at bsb.me.uk Wed Oct 11 20:55:30 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 01:55:30 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> Message-ID: <87vajlkwvx.fsf@bsb.me.uk> Chris Angelico writes: > On Thu, Oct 12, 2017 at 9:44 AM, Ben Bacarisse wrote: >> Chris Angelico writes: >>> Check out Django and Flask, the two most popular ways. I quite like >>> Flask. >> >> I see. Both appear to be frameworks (I'd heard of Django). Do you know >> if they widely available on low-cost hosting packages? (I don't think >> they are on mine, but that's dirt-cheap because I don't use it for >> anything important!) > > Generally, low-cost hosting will offer Python, and then you can > install Flask/Django from there. For example, you can deploy to Heroku > (zero-dollar hosting for small sites), and as long as you provide a > file called "requirements.txt", they'll install whatever you depend > on. > >> One thing that helped PHP was that it could be used (and learned) in an >> incremental way. You could add a "this page last updated on..." text in >> a line or two to an existing page. Then a button to change the layout >> or theme. Then a simple form and so on. >> >> Many professionals started that way. In the early days, there ware few >> other routes into the technical side of web authoring. That helped to >> cement PHP as the dominant technology because it was what they knew. > > Yeah. The trouble is that this is a really REALLY bad way to design > something. Yes, I know that's the way of the devil, just I'm explaining why PHP is popular! > Have you seen a city that grew one house at a time, and had > streets added to service those houses? Not good. The end result is > that PHP is still bound by this "just a bit of scripting inside your > HTML" structure, which has annoying consequences in certain areas > (like whitespace before the first " situations, and the way "include" works), plus it binds your URLs to > the concrete file system. That may not seem like too much of a > problem, but it's a pretty big limitation; you can't have URLs like > "https://en.wikipedia.org/wiki/Foo" without some help from the web > server, eg Apache's mod_rewrite. I don't follow this. Your "can't" and "big limitation" suggests something inevitable, but I don't see it as an intrinsic problem with the language. I'm sure PHP is not as flexible as the frameworks you mention, but you are not tied to URLs mapping to files. Maybe you meant that this is what often happens, or what most people do, with PHP. > In contrast, Python and Flask would > handle that like this: > > @app.route("/wiki/
") > def show_article(article): > ... > > Have a think about what the PHP setup implies. Arbitrary files in your > web server tree are not just automatically available (as they'd be if > you use flat-file hosting like GitHub Pages), but they are > *executable*. You don't have to set the file mode to say "this is > executable", you just have to have the file there with the appropriate > name. So PHP-based web sites end up having to monitor their uploads > and downloads lest someone slip something in and then run it on the > server... with full permissions. With Flask, in contrast, you can make > an uploads directory that's automatically downloadable, but nothing in > it will ever be executed. Yes, it's fraught with security issues. -- Ben. From rosuav at gmail.com Wed Oct 11 21:06:38 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 12:06:38 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87vajlkwvx.fsf@bsb.me.uk> References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> Message-ID: On Thu, Oct 12, 2017 at 11:55 AM, Ben Bacarisse wrote: > Chris Angelico writes: >> it binds your URLs to >> the concrete file system. That may not seem like too much of a >> problem, but it's a pretty big limitation; you can't have URLs like >> "https://en.wikipedia.org/wiki/Foo" without some help from the web >> server, eg Apache's mod_rewrite. > > I don't follow this. Your "can't" and "big limitation" suggests > something inevitable, but I don't see it as an intrinsic problem with > the language. I'm sure PHP is not as flexible as the frameworks you > mention, but you are not tied to URLs mapping to files. Maybe you meant > that this is what often happens, or what most people do, with PHP. How would you, with PHP itself, handle database-provided URLs? The only way I've ever seen it done is at an external level - such as mod_rewrite - which means that someone else, *not* the PHP script, is managing your URLs. They're pushed to some external config file somewhere. That's okay for just one URL pattern, but it doesn't scale well, which is why (for example) Wikipedia's editing pages are "/w/index.php?...." instead of, say, "/wiki/Foo/edit" or "/wiki/edit/Foo". Unless you know something I don't? ChrisA From steve+python at pearwood.info Wed Oct 11 21:15:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 12 Oct 2017 12:15:33 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> Message-ID: <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> On Thu, 12 Oct 2017 02:43 am, Marko Rauhamaa wrote: > Chris Angelico : > >> The places where C++ is not a superset of C are mostly things you >> wouldn't want to be doing anyway. You can generally take C code and >> compile it with a C++ compiler, and it'll have the same semantics. > > Here's a C/C++ program: > > ======================================================================== > #include > > int main() > { > struct {} s; > printf("%d\n", (int) sizeof 'a'); > printf("%d\n", (int) sizeof s); > return 0; > } > ======================================================================== > > When compiled (with gcc) as a C program, the output is: > > 4 > 0 > > When the same program is compiled (with gcc) as a C++ program, the > output is: > > 1 > 1 > > That is not immediately all that significant but points to subtle > incompatibilities between the data models of C and C++. I don't think anyone should expect that platform specific details like the size of a char should be precisely the same between C and C++. Even two different C compilers could return different values. > Then we have syntactic problems: [...] I don't believe that anyone meant to imply that C++ is an exact superset of C. I know I didn't, although I acknowledge that my terminology was lazy. C++ is broadly-speaking compatible with C, with just enough differences to act as landmines to the unwary. Good enough? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Wed Oct 11 21:19:25 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 02:19:25 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> Message-ID: <87inflkvs2.fsf@bsb.me.uk> Chris Angelico writes: > On Thu, Oct 12, 2017 at 11:55 AM, Ben Bacarisse wrote: >> Chris Angelico writes: >>> it binds your URLs to >>> the concrete file system. That may not seem like too much of a >>> problem, but it's a pretty big limitation; you can't have URLs like >>> "https://en.wikipedia.org/wiki/Foo" without some help from the web >>> server, eg Apache's mod_rewrite. >> >> I don't follow this. Your "can't" and "big limitation" suggests >> something inevitable, but I don't see it as an intrinsic problem with >> the language. I'm sure PHP is not as flexible as the frameworks you >> mention, but you are not tied to URLs mapping to files. Maybe you meant >> that this is what often happens, or what most people do, with PHP. > > How would you, with PHP itself, handle database-provided URLs? The > only way I've ever seen it done is at an external level - such as > mod_rewrite - which means that someone else, *not* the PHP script, is > managing your URLs. They're pushed to some external config file > somewhere. That's okay for just one URL pattern, but it doesn't scale > well, which is why (for example) Wikipedia's editing pages are > "/w/index.php?...." instead of, say, "/wiki/Foo/edit" or > "/wiki/edit/Foo". > > Unless you know something I don't? Provided some early part of the URL is handled by PHP, the rest of the URL path is provided to PHP in $_SERVER["PATH_INFO"]. -- Ben. From keishbby at gmail.com Wed Oct 11 21:19:44 2017 From: keishbby at gmail.com (keishbby at gmail.com) Date: Wed, 11 Oct 2017 18:19:44 -0700 (PDT) Subject: Test Bank for Introduction to Sociology 10th Edition by Anthony Giddens In-Reply-To: References: Message-ID: <7070c03b-90a9-4112-97a3-4c6b8cc6007b@googlegroups.com> On Wednesday, July 12, 2017 at 5:15:39 PM UTC-4, Test Banks wrote: > Greetings, > > You can get Test Bank for " Introduction to Sociology 10th Edition by Anthony Giddens, Mitchell Duneier, Richard P. Appelbaum, Deborah Carr " at very reasonable price. Our team is available 24/7 and 365 days / year to respond your requests. Send us an email at pro.fast(@)hotmail(dot)com > > Place your order at PRO.FAST(@)HOTMAIL(DOT)COM > > ISBN Numbers for this book (ISBN-13: 9780393289688) > > INTRODUCTION TO SOCIOLOGY 10TH EDITION BY GIDDENS TEST BANK > > You can also email for other Sociology books Solutions and Test Bank at low prices and our team will try to get all resources you need. > > Do not post your reply here. Simply send us an email at PRO.FAST (AT) HOTMAIL (DOT) COM > > Cheers, From steve+python at pearwood.info Wed Oct 11 21:26:07 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 12 Oct 2017 12:26:07 +1100 Subject: Looping [was Re: Python and the need for speed] References: <768f9a44-5d90-4756-8a99-821253a491f8@googlegroups.com> <4277c149-e136-4d33-91d5-d011df9149f1@googlegroups.com> <58edcdd2$0$1501$c3e8da3$5496439d@news.astraweb.com> <54f1685d-1d2d-44b5-869e-a510569ba9d1@googlegroups.com> <1d352948-9029-4b86-b2f1-2b52608109ea@googlegroups.com> <616ebaec-f140-491c-897e-f2c4b8368d28@googlegroups.com> <58f2dc43$0$1583$c3e8da3$5496439d@news.astraweb.com> <_6JIA.1041096$Pm3.778529@fx42.am4> <87shl7mrtf.fsf@elektro.pacujo.net> <87k26jvvpn.fsf@elektro.pacujo.net> <87efwqoqtz.fsf@bsb.me.uk> <87d1caubg2.fsf@elektro.pacujo.net> <87y3uymuuq.fsf@bsb.me.uk> Message-ID: <59dec4b0$0$14935$b1db1813$d948b532@news.astraweb.com> On Wed, 11 Oct 2017 10:57 pm, Stefan Ram wrote: > FWIW, in is book "Touch of Class" (2009) Bertrand Meyer writes: > > |Such instructions are just the old goto in sheep's clothing. > |Treat them the same way as the original: > | > |/Touch of Methodology/: > | Sticking to one-entry, one-exit building blocks > |Stay away from any "break" or similar control mechanism. I have a great deal of respect for Meyer, but in this case I think he has it badly wrong on both counts (`break` and "one-entry, one-exit"). Unrestricted GOTO (as in BASIC, circa 1970, where you could jump to any line in the code, or as in assembly) is rightly considered harmful (albeit necessary in assembly). But if Meyer means *any* sort of jump ("similar control mechanism"), then that would rule out not just `break` and `continue` but also: * while loops * for loops * if...else * case/switch statements * calling subroutines (functions or procedures) * exception handling Remember that a function call is basically a GOTO in disguise: execution jumps to the function, and jumps back when the function returns. In BASIC, there was a GOSUB intermediate between a GOTO and a procedure call. Surely Meyer doesn't mean to say we should never call functions or use `while`, `for` or `if`. So he has to distinguish between kinds of jumps: Good jumps (I presume): * function calls * if...else * looping Evil jumps: * unrestricted BASIC-style GOTO/GOSUB any line number * break/continue Not sure: * GOTO where there are restrictions on where you can jump * COMEFROM (I kid: I'm sure Meyer would oppose COMEFROM, and I expect that even Pascal-style restricted GOTO would be on his "evil" list to avoid.) So the question becomes, why are such harmless, simple to understand, innocuous jumps like `break` and `continue` in the evil list, when they not only simplify code but make it more efficient? # with break for i in range(2**64): if isprime(i): print(i, "is prime") break # without break still_searching = True for i in range(2**64): if still_searching and isprime(i): print(i, "is prime") still_searching = False # without break, attempt number 2 still_searching = True i = 0 while still_searching and i < 2**64: if isprime(i): print(i, "is prime") still_searching = False Unrestricted jumps *into* a subroutine are hard to reason about. In general, subroutines should have a single entry point. But the requirement for one exit is too strict. From the caller's perspective, there is no way to tell how many exit point a function has: subroutines are black boxes that the caller cannot see into, and the difference between a single exit and multiple exits is invisible. But from the point of view of the subroutines, the rule "one exit" is like the rule "no break" for loops: it makes the code more complex and less efficient. If you're done, you're done, and you might as well return out of the function rather than write boilerplate code to pad it out until you get to the very end. With only a single condition to test, there's not much difference between the two: # with single exit # with multiple exits def foo(): def foo(): if condition: if condition: result = 1 return 1 else: return 2 result = 2 return result but as the number of decision points increase, the complexity required to keep a single exit also increases. Ironically, after telling us to stick to code with one entry and one exit, Meyer then contradicts himself by recommending exceptions: > |You can use exception handling as a technique of last resort > |to handle unexpected events for which the normal control > |structures let you down. Even more ironically, exception handling is most similar to a COMEFROM, which was invented as a joke. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Wed Oct 11 21:27:46 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 02:27:46 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> Message-ID: <87efq9kve5.fsf@bsb.me.uk> Steve D'Aprano writes: > On Thu, 12 Oct 2017 02:43 am, Marko Rauhamaa wrote: > >> Chris Angelico : >> >>> The places where C++ is not a superset of C are mostly things you >>> wouldn't want to be doing anyway. You can generally take C code and >>> compile it with a C++ compiler, and it'll have the same semantics. >> >> Here's a C/C++ program: It's not a C program in the sense that it's undefined in C. A struct with no members is a constraint violation. >> ======================================================================== >> #include >> >> int main() >> { >> struct {} s; >> printf("%d\n", (int) sizeof 'a'); >> printf("%d\n", (int) sizeof s); >> return 0; >> } >> ======================================================================== >> >> When compiled (with gcc) as a C program, the output is: >> >> 4 >> 0 >> >> When the same program is compiled (with gcc) as a C++ program, the >> output is: >> >> 1 >> 1 >> >> That is not immediately all that significant but points to subtle >> incompatibilities between the data models of C and C++. > > I don't think anyone should expect that platform specific details like the > size of a char should be precisely the same between C and C++. Even two > different C compilers could return different values. The size of (in the sense of sizeof) an expression of type char is defined to be 1. All conforming C and C++ compilers must return 1 in such a case. The difference being highlighted here is that, in C, 'a' is an integer expression. In C++ it's of type char. -- Ben. From formisc at gmail.com Wed Oct 11 22:02:41 2017 From: formisc at gmail.com (Andrew Z) Date: Wed, 11 Oct 2017 22:02:41 -0400 Subject: Logging from files doesn't work Message-ID: Hello, apparently my reading comprehension is nose diving these days. After reading python cookbook and a few other tutorials i still can't get a simple logging from a few files to work. I suspected my file organization - all files are in the same directory, causing problem. But it appears it is not. Anyway, in the example below, only logging from main.py works. I also want to have login from "test1/test.py" to write into the same common log file. And how can i accomplish the same if test.py is in the same directory as main.py? dir structure: src/ |- main.py |-test/ |-test.py code: test.py: import logging # neither of below produce any results log = logging.getLogger("test1.test") # log = logging.getLogger(__name__) def fun(): print("DADADA") log.debug(" DADADADA " ) main.py: from test1.test import fun def main(): log = logging.getLogger(__name__) log.setLevel(logging.DEBUG) fh = logging.FileHandler("nja_" + datetime.now().strftime("%Y_%b_%d_%H_%M_%S") +".log") formatter = logging.Formatter('%(levelname)s - %(asctime)s - %(funcName)10s() %(lineno)s - %(message)s') fh.setFormatter(formatter) log.addHandler(fh) log.debug("Yes, this line is in the log file") fun() log.debug("And this one is too") this is 3.5 version. Thank you in advance. From formisc at gmail.com Wed Oct 11 22:15:22 2017 From: formisc at gmail.com (Andrew Z) Date: Wed, 11 Oct 2017 22:15:22 -0400 Subject: Logging from files doesn't work In-Reply-To: References: Message-ID: if i change print statements in both files to print out "__name__": __main__ test1.test On Wed, Oct 11, 2017 at 10:02 PM, Andrew Z wrote: > Hello, > > apparently my reading comprehension is nose diving these days. After > reading python cookbook and a few other tutorials i still can't get a > simple logging from a few files to work. > I suspected my file organization - all files are in the same directory, > causing problem. But it appears it is not. > > Anyway, in the example below, only logging from main.py works. I also want > to have login from "test1/test.py" to write into the same common log file. > > And how can i accomplish the same if test.py is in the same directory as > main.py? > > dir structure: > src/ > |- main.py > |-test/ > |-test.py > > code: > test.py: > > import logging > # neither of below produce any results > > log = logging.getLogger("test1.test") > # log = logging.getLogger(__name__) > > def fun(): > print("DADADA") > log.debug(" DADADADA " ) > > > main.py: > > from test1.test import fun > > def main(): > > log = logging.getLogger(__name__) > log.setLevel(logging.DEBUG) > > fh = logging.FileHandler("nja_" + datetime.now().strftime("%Y_%b_%d_%H_%M_%S") +".log") > formatter = logging.Formatter('%(levelname)s - %(asctime)s - %(funcName)10s() %(lineno)s - %(message)s') > fh.setFormatter(formatter) > log.addHandler(fh) > > log.debug("Yes, this line is in the log file") > > fun() > > log.debug("And this one is too") > > > > this is 3.5 version. > > Thank you in advance. > From formisc at gmail.com Wed Oct 11 22:27:21 2017 From: formisc at gmail.com (Andrew Z) Date: Wed, 11 Oct 2017 22:27:21 -0400 Subject: Logging from files doesn't work In-Reply-To: References: Message-ID: aha. So the issue is that main.py's __name__ attribute == "__main__" and test.py is "test1.test". if i manually assign names: main.py - > log = logging.getLogger("MAIN") test.py - > log = logging.getLogger("MAIN.test1.test") then logging is working perfectly well. This brings me to the question - what is wrong with me file naming/structure that confuses the logging module? I'm not sure i really want to have each little file in it's own directory too.. but i'm open for suggestions. On Wed, Oct 11, 2017 at 10:15 PM, Andrew Z wrote: > if i change print statements in both files to print out "__name__": > __main__ > test1.test > > On Wed, Oct 11, 2017 at 10:02 PM, Andrew Z wrote: > >> Hello, >> >> apparently my reading comprehension is nose diving these days. After >> reading python cookbook and a few other tutorials i still can't get a >> simple logging from a few files to work. >> I suspected my file organization - all files are in the same directory, >> causing problem. But it appears it is not. >> >> Anyway, in the example below, only logging from main.py works. I also >> want to have login from "test1/test.py" to write into the same common log >> file. >> >> And how can i accomplish the same if test.py is in the same directory as >> main.py? >> >> dir structure: >> src/ >> |- main.py >> |-test/ >> |-test.py >> >> code: >> test.py: >> >> import logging >> # neither of below produce any results >> >> log = logging.getLogger("test1.test") >> # log = logging.getLogger(__name__) >> >> def fun(): >> print("DADADA") >> log.debug(" DADADADA " ) >> >> >> main.py: >> >> from test1.test import fun >> >> def main(): >> >> log = logging.getLogger(__name__) >> log.setLevel(logging.DEBUG) >> >> fh = logging.FileHandler("nja_" + datetime.now().strftime("%Y_%b_%d_%H_%M_%S") +".log") >> formatter = logging.Formatter('%(levelname)s - %(asctime)s - %(funcName)10s() %(lineno)s - %(message)s') >> fh.setFormatter(formatter) >> log.addHandler(fh) >> >> log.debug("Yes, this line is in the log file") >> >> fun() >> >> log.debug("And this one is too") >> >> >> >> this is 3.5 version. >> >> Thank you in advance. >> > > From dvl at psu.edu Wed Oct 11 22:47:21 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Wed, 11 Oct 2017 22:47:21 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: mailman.1190.1507771203.12136.python-list@python.org References: Message-ID: <1507776441l.38207678l.0l@psu.edu> Actually, FORTRAN and COBOL and Algol (for its control structures) Trying to support both of the first two was entertaining -- when you declared a variable, it wasn't enough to say it was an Integer: you had to also declare whether it was represented in Binary or Decimal, and also specify the desired precision. The IBM/360 architecture supported both binary and decimal integers, where the decimals were stored as BCD nybbles of arbitrary fixed length (as opposed to binary integers matching the machine word size) The world migrated away from PL/I back in those days because of the one-size fits none consequences of trying to do everything. So I always find myself looking askance when language designers try to repeat the exercise. You know, like designing a largely-interpreted object-oriented language with libraries supporting a functional programming style. I think I've seen a language like that somewhere around this forum. But I like it anyway Roger Christman Pennsylvania State University On Wed, Oct 11, 2017 12:07 PM, Dennis Lee Bieber wrote: > On Wed, 11 Oct 2017 00:21:46 -0400, Bill >declaimed the following: > >>PL-I has already been taken. That is a pretty free-wheeling language > > I think I once heard PL-1 described as trying to combine FORTRAN and >COBOL... > > > From cs at cskk.id.au Wed Oct 11 23:29:49 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 12 Oct 2017 14:29:49 +1100 Subject: Logging from files doesn't work In-Reply-To: References: Message-ID: <20171012032949.GA76681@cskk.homeip.net> On 11Oct2017 22:27, Andrew Z wrote: >aha. So the issue is that main.py's __name__ attribute == "__main__" and >test.py is "test1.test". Yeah. If you invoke a module as "python -m module_name" its __name__ field is "__main__". That makes the boilerplate work, but breaks your expectation that __name__ is usually module_name. >if i manually assign names: >main.py - > > >log = logging.getLogger("MAIN") > >test.py - > >log = logging.getLogger("MAIN.test1.test") > >then logging is working perfectly well. > >This brings me to the question - what is wrong with me file >naming/structure that confuses the logging module? I'm not sure i >really want to have each little file in it's own directory too.. I confess to not using the logging module in the module-named-based fashion that seems to be the default recommendation. I usually prefer to set up the root logger to log somewhere suitable to the main program (typically just stderr by default), and set up special logging for other things as needed. As what may be a poor example, I've got a mail filing program which monitors maildirs for arriving messages to refile. So I've got a class associated with each monitored maildir with these methods: @property def logdir(self): ''' The pathname of the directory in which log files are written. ''' varlog = cs.env.LOGDIR(self.environ) return os.path.join(varlog, 'mailfiler') This essentially computes "$HOME/var/log/mailfiler" as the place where all the logfiles are saved. And this: def folder_logfile(self, folder_path): ''' Return path to log file associated with the named folder. TODO: base on relative path from folder root, not just basename. ''' return os.path.join(self.logdir, '%s.log' % (os.path.basename(folder_path))) which computes "$HOME/var/log/mailfiler/spool.log" as the logfile when working on my maildir "$HOME/mail/spool". And then off it goes with a FileHandler for that path for the filing actions for that maildir. So you're not contrained to drop log files all through your source tree (ewww!) My opinion is that you should decide where your logfiles _should_ live; it generally has nothing (or little) to do with the module name. I keep mine, generally, in various subdirectories of "$HOME/var/log". Cheers, Cameron Simpson (formerly cs at zip.com.au) From rosuav at gmail.com Thu Oct 12 01:31:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 16:31:12 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87inflkvs2.fsf@bsb.me.uk> References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> Message-ID: On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: > Chris Angelico writes: > >> On Thu, Oct 12, 2017 at 11:55 AM, Ben Bacarisse wrote: >>> Chris Angelico writes: >>>> it binds your URLs to >>>> the concrete file system. That may not seem like too much of a >>>> problem, but it's a pretty big limitation; you can't have URLs like >>>> "https://en.wikipedia.org/wiki/Foo" without some help from the web >>>> server, eg Apache's mod_rewrite. >>> >>> I don't follow this. Your "can't" and "big limitation" suggests >>> something inevitable, but I don't see it as an intrinsic problem with >>> the language. I'm sure PHP is not as flexible as the frameworks you >>> mention, but you are not tied to URLs mapping to files. Maybe you meant >>> that this is what often happens, or what most people do, with PHP. >> >> How would you, with PHP itself, handle database-provided URLs? The >> only way I've ever seen it done is at an external level - such as >> mod_rewrite - which means that someone else, *not* the PHP script, is >> managing your URLs. They're pushed to some external config file >> somewhere. That's okay for just one URL pattern, but it doesn't scale >> well, which is why (for example) Wikipedia's editing pages are >> "/w/index.php?...." instead of, say, "/wiki/Foo/edit" or >> "/wiki/edit/Foo". >> >> Unless you know something I don't? > > Provided some early part of the URL is handled by PHP, the rest of the > URL path is provided to PHP in $_SERVER["PATH_INFO"]. Is it possible to do that without having ".php" visible in the path? ChrisA From grant.b.edwards at gmail.com Thu Oct 12 01:35:00 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 05:35:00 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: Message-ID: On 2017-10-11, Chris Angelico wrote: > But since it's the lowest-end sites that have traditionally driven > that demand for PHP, there's a general tendency for low-grade > programmers to gravitate to it, so there's a lot of really REALLY bad > code out there. And there are a lot of people providing really REALLY bad answers on various support fora. I learned very quickly to pay no attention to people providing answers to PHP questions on any web-based forum. I've probably been spoiled by the level of discorse in this list/group, but I was absolutely dumbfounded by the wrong answers people drooled onto the interwebs. PHP is the poster child for cargo-cult programming where people cut/paste chunks of garbage without really understanding what they're doing. Even the official PHP documentation is pretty bad. It's not usually outright wrong, but it's very incomplete, and leaves a _lot_ of things undefined. Of course the PHP lanugage itself is such a mess, it's often not very easy to understand what's going on. -- Grant From grant.b.edwards at gmail.com Thu Oct 12 01:39:20 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 05:39:20 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 2017-10-11, Gregory Ewing wrote: > Neil Cerutti wrote: >> I dig >> const qualifiers, even though I'm comletely fine with their >> absence from Python. > > Out of curiosity, do you have any insights into why you > like them in C++, if you don't miss them in Python? I like them in C because it allows the linker to place them in ROM with the code. It also _sometimes_ provides useful diagnostics when you pass a pointer to something which shouldn't be modified to something that is going to try to modify it. -- Grant From grant.b.edwards at gmail.com Thu Oct 12 01:41:19 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 05:41:19 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-12, Steve D'Aprano wrote: > I don't think anyone should expect that platform specific details like the > size of a char should be precisely the same between C and C++. I don't knwo about that. > Even two > different C compilers could return different values. Nope. If sizeof char is not 1, then it's not C. That doesn't mean that a char can't be 9, 11, 16, 20, 32, or 64 bits, but sizeof char is 1. -- Grant From greg.ewing at canterbury.ac.nz Thu Oct 12 02:03:13 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 12 Oct 2017 19:03:13 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> Message-ID: bartc wrote: > The original has an extra pointer so idiomatic C might be more: > > tokenrec *** I was going by your English description, which when translated into C gives only two pointers. But I think you're right that the C version really has 3 levels of indirection, so the meaning of "array" in the original language must be something different from what it is in C. My point was that C makes very little distinction between "pointer to foo" and "pointer to first element of array of foo", so it's not usual practice to write declarations as convoluted as "* (*)[]". > If you want to lose all array information, and really don't care if > you're dealing with a pointer, or an array (and don't mind changing how > such a value is passed, and how it is accessed) then this is fine for you. You lose that information in C anyway. There's no way for the function to tell whether it was passed a pointer to a single value or a pointer to an array, and if the latter, how many elements are in the array. If you want that information passed, you have to arrange it separately. You don't get it just by writing "* (*)[]" instead of "***" for the argument type. -- Greg From greg.ewing at canterbury.ac.nz Thu Oct 12 02:07:38 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 12 Oct 2017 19:07:38 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87d15tmh3t.fsf@bsb.me.uk> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> Message-ID: Ben Bacarisse wrote: > That's a different type. I think you mean that a human writing C > (rather than bartc's code generator) would probably design the code to > use tokenrec ** then I agree, but the latter is not just a different way > to write the former. Yes, I was translating his English description of the type into C, using C's meaning of the word "array". It seems that arrays in the original language (Algol? One of Bart's inventions?) are somewhat richer things. -- Greg From marko at pacujo.net Thu Oct 12 03:22:07 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Oct 2017 10:22:07 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: <8760bkde5c.fsf@elektro.pacujo.net> Grant Edwards : > I like [const qualifiers] in C because it allows the linker to place > them in ROM with the code. It also _sometimes_ provides useful > diagnostics when you pass a pointer to something which shouldn't be > modified to something that is going to try to modify it. Unfortunately, "const" is so tacky in practice that the language and the standard libraries have rendered it useless. One example is the surprising fact that string literals in C are "char *" and not "const char *". Additionally, you can launder any constant string into a nonconstant string with strstr(3): const char *cs = "hello"; char *s = strstr(cs, ""); s[0] = 'y'; Also, you'd expect execve(2) to take in: const char *const argv[] Instead, it wants: char *const argv[] which makes little semantic sense. The reason for all of this weirdness is that the proper use of "const" in C is very painful to do: you have to sprinkle your code with explicit casts. Tons of legacy code would be riddled with compiler error messages. The solution was to pay lip service to "const." It would have been better not to import "const" into the language. (C++ has its own special issue with "const," which forced C++ to introduce the ugly concept of const_cast<>().) Marko From BILL_NOSPAM at whoknows.net Thu Oct 12 03:48:25 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Thu, 12 Oct 2017 03:48:25 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <8760bkde5c.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> Message-ID: Marko Rauhamaa wrote: > Grant Edwards : > >> I like [const qualifiers] in C because it allows the linker to place >> them in ROM with the code. It also _sometimes_ provides useful >> diagnostics when you pass a pointer to something which shouldn't be >> modified to something that is going to try to modify it. > Unfortunately, "const" is so tacky in practice that the language and the > standard libraries have rendered it useless. > > One example is the surprising fact that string literals in C are "char > *" and not "const char *". If not, you couldn't pass a string literal to a function having prototype void f(char *s); Of course, if f tries to modify *s, there will be a run time error as character string literals are stored in a "special place", and are not mutable. This improves performance. From obulesu.t at gmail.com Thu Oct 12 04:08:41 2017 From: obulesu.t at gmail.com (T Obulesu) Date: Thu, 12 Oct 2017 01:08:41 -0700 (PDT) Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python Message-ID: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Hello all, I want to send some frames defined by me{Example, [0x45,0x43,0x32]} to the raspberry pi from any macine(Desktop/Laptop/other raspberry pi). But I want to send those frames over wifi or use wlan0 using python Any suggestions? From rosuav at gmail.com Thu Oct 12 04:14:49 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 19:14:49 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <8760bkde5c.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 12, 2017 at 6:22 PM, Marko Rauhamaa wrote: > Grant Edwards : > >> I like [const qualifiers] in C because it allows the linker to place >> them in ROM with the code. It also _sometimes_ provides useful >> diagnostics when you pass a pointer to something which shouldn't be >> modified to something that is going to try to modify it. > > Unfortunately, "const" is so tacky in practice that the language and the > standard libraries have rendered it useless. > > One example is the surprising fact that string literals in C are "char > *" and not "const char *". Additionally, you can launder any constant > string into a nonconstant string with strstr(3): > > const char *cs = "hello"; > char *s = strstr(cs, ""); > s[0] = 'y'; Well hey, if you want that, you can just cast the pointer. C is not about preventing you from doing things. The 'const' keyword is a useful protection (and one that would be more useful if string literals were const), but it's not meant to be a straitjacket. ChrisA From mail at timgolden.me.uk Thu Oct 12 04:16:20 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 12 Oct 2017 09:16:20 +0100 Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python In-Reply-To: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> References: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Message-ID: On 12/10/2017 09:08, T Obulesu wrote: > Hello all, I want to send some frames defined by me{Example, > [0x45,0x43,0x32]} to the raspberry pi from any > macine(Desktop/Laptop/other raspberry pi). But I want to send those > frames over wifi or use wlan0 using python Any suggestions? > Are you talking about literal network frames? https://en.wikipedia.org/wiki/Frame_(networking) Or merely data which represents some kind of "frame" in your application? ie are you looking for low-level control of the network device? Or merely a way to transmit data across a network generally? TJG From rosuav at gmail.com Thu Oct 12 04:19:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 19:19:51 +1100 Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python In-Reply-To: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> References: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Message-ID: On Thu, Oct 12, 2017 at 7:08 PM, T Obulesu wrote: > Hello all, I want to send some frames defined by me{Example, [0x45,0x43,0x32]} to the raspberry pi from any macine(Desktop/Laptop/other raspberry pi). But I want to send those frames over wifi or use wlan0 using python Any suggestions? > -- Since you use the word "frame", I'm thinking you possibly mean at a very low level, and not packaged up into UDP/IP packets. I'm curious as to why you'd need that; for most purposes, the easiest way would be either TCP or UDP. But sending raw packets is entirely possible in Python - though you have to bear in mind that many OSes restrict this capability to administrative programs. ChrisA From __peter__ at web.de Thu Oct 12 04:20:53 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 12 Oct 2017 10:20:53 +0200 Subject: Logging from files doesn't work References: Message-ID: Andrew Z wrote: > Hello, > > apparently my reading comprehension is nose diving these days. After > reading python cookbook and a few other tutorials i still can't get a > simple logging from a few files to work. > I suspected my file organization - all files are in the same directory, > causing problem. But it appears it is not. > > Anyway, in the example below, only logging from main.py works. I also want > to have login from "test1/test.py" to write into the same common log file. > > And how can i accomplish the same if test.py is in the same directory as > main.py? > > dir structure: > src/ > |- main.py > |-test/ Shouldn't that be |-test1/ ? > |-test.py > > code: > test.py: > > import logging > # neither of below produce any results > > log = logging.getLogger("test1.test") > # log = logging.getLogger(__name__) > > def fun(): > print("DADADA") > log.debug(" DADADADA " ) > > > main.py: > > from test1.test import fun > > def main(): > > log = logging.getLogger(__name__) > log.setLevel(logging.DEBUG) > > fh = logging.FileHandler("nja_" + > datetime.now().strftime("%Y_%b_%d_%H_%M_%S") +".log") > formatter = logging.Formatter('%(levelname)s - %(asctime)s - > %(funcName)10s() %(lineno)s - %(message)s') > fh.setFormatter(formatter) > log.addHandler(fh) > > log.debug("Yes, this line is in the log file") > > fun() > > log.debug("And this one is too") You have already worked out why the names are what they are. You can avoid the problem if you attach the handler to the root logger, e. g. def main(): root_log = logging.getLogger() root_log.setLevel(logging.DEBUG) fh = logging.FileHandler( "nja_" + datetime.now().strftime("%Y_%b_%d_%H_%M_%S") +".log" ) formatter = logging.Formatter( '%(levelname)s - %(asctime)s - %(funcName)10s()' '%(lineno)s - %(message)s' ) fh.setFormatter(formatter) root_log.addHandler(fh) log = logging.getLogger(__name__) log.debug("Yes, this line is in the log file") fun() log.debug("And this one is too") "And this one is too") That way you can capture messages from the whole hierarchy regardless of its actual layout. From auriocus at gmx.de Thu Oct 12 04:23:57 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 12 Oct 2017 10:23:57 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> Message-ID: Am 12.10.17 um 01:15 schrieb Stefan Ram: > Define a function ?g? with a parameter ?x? of type ?int?, so > that this function ?g? returns a pointer to another function. > This other function has a parameter of type ?char? and returns > a double value. Ok > /Without/ a typedef. > And WHY would you do that? Typedefs do not cost money. Try this: typedef double (*countdown_rate_func) (char letter); countdown_rate_func select_rate_func(int level); // I've substituted 'x' by 'level' // and 'g' by 'select_rate_func' for clarity I claim that the above is quite clear, it declares a function in a Countdown game engine, which, depending on the level, returns a different function to rate the weight of the individual letters. Your exercise results in the line noise double (*g(int x))(char) (thanks to Ben for doing the exercise), which is not extremely bad, but still does tell nothing about the intent of that line. Writing the typedef is easy for humans, stick * in front of the function name for a regular declaration and add (). The compiler can then figure out the complete type on its own, it's its job, not mine. Christian From tjol at tjol.eu Thu Oct 12 04:32:58 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 10:32:58 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> Message-ID: <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> On 2017-10-12 07:31, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >> Provided some early part of the URL is handled by PHP, the rest of the >> URL path is provided to PHP in $_SERVER["PATH_INFO"]. > > Is it possible to do that without having ".php" visible in the path? Just like with Python-based frameworks, this requires a few lines of web server configuration. On Apache, you might use mod_wsgi to tell the server how to run the code in one case, and a combination of mod_php and mod_rewrite in the other. If you're using FastCGI with nginx or lighttpd, I believe the configuration would look pretty similar in both cases. Then again, I don't do much web programming any more and generally stay away from PHP, so I may be misremembering. -- Thomas Jollans From tjol at tjol.eu Thu Oct 12 04:49:57 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 10:49:57 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> Message-ID: <5741869a-930a-c8ba-036b-0fe531d2d3e2@tjol.eu> On 2017-10-12 01:33, Chris Angelico wrote: > Have you seen a city that grew one house at a time, and had > streets added to service those houses? Not good. Actually, that's more or less how most cities grew historically. Nowadays these organically grown city centres tend to be much more people-friendly than "modern" cities (mis-)designed around the road and the motorcar. Of course they've had centuries to mature. They've seen constant maintenance and refactoring (fortifying the city, digging canals, moving fortifications to accommodate growth, building railway lines, replacing key canals with roads, ...) to adapt to changing demands and challenges of the environment. Yes, yes, there are plenty of examples of cities with more recent histories of ad hoc growth that don't function as well as London, Budapest or Jerusalem - yet. No, the comparison is not relevant. For starters, people tend to care more about their home than about their PHP code. -- Thomas Jollans From tjol at tjol.eu Thu Oct 12 05:08:48 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 11:08:48 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <379DBE2E-D89C-4C3E-80F1-8C794F676430@icloud.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <379DBE2E-D89C-4C3E-80F1-8C794F676430@icloud.com> Message-ID: On 2017-10-12 01:32, Christopher Reimer wrote: > On Oct 11, 2017, at 9:07 AM, Bill wrote: >> >> Grant Edwards wrote: >>> On 2017-10-11, Bill wrote: >>> >>> >>>> [...] I'm not here to "cast stones", I like Python. I just think >>>> that you shouldn't cast stones at C/C++. >>> Not while PHP exists. There aren't enough stones in the world... >>> >> >> PHP seems (seemed?) popular for laying out web pages. Are their vastly superior options? I'm a little naive in this matter, thus my question. >> -- >> https://mail.python.org/mailman/listinfo/python-list > > AFAIK, JavaScript frameworks has largely replaced PHP. I personally use Pelican to generate static web pages and use JavaScript sparingly. "Remember Ruby on Rails?" -- Thomas Jollans From marko at pacujo.net Thu Oct 12 05:11:10 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Oct 2017 12:11:10 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> Message-ID: <87zi8wbuj5.fsf@elektro.pacujo.net> Bill : > Marko Rauhamaa wrote: >> One example is the surprising fact that string literals in C are >> "char *" and not "const char *". > > If not, you couldn't pass a string literal to a function having > prototype void f(char *s); That *ought* to be prevented. That's the whole point. Marko From tjol at tjol.eu Thu Oct 12 05:12:27 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 11:12:27 +0200 Subject: OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: References: Message-ID: <7eb848fb-5bed-8766-cbb8-c191f8f7acec@tjol.eu> On 2017-10-12 02:51, Chris Angelico wrote: > If it wants new life, it's probably going to need a Linux version, > because that's where a lot of developers hang out. The reality is that > open source developers are much more likely to develop on Linux than > on Windows; you can maintain a Windows port of a Linux program with > fewer Windows experts than maintaining the entire program on Windows. > > The other option, though, would be for the useful parts to become > feature suggestions for VLC. It's the year of the Linux desktop! (No, actually, that was a few years ago, but nobody noticed at the time) -- Thomas Jollans From marko at pacujo.net Thu Oct 12 05:20:03 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Oct 2017 12:20:03 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> Message-ID: <87vajkbu4c.fsf@elektro.pacujo.net> Chris Angelico : > On Thu, Oct 12, 2017 at 6:22 PM, Marko Rauhamaa wrote: >> Additionally, you can launder any constant string into a nonconstant >> string with strstr(3): >> >> const char *cs = "hello"; >> char *s = strstr(cs, ""); >> s[0] = 'y'; > > Well hey, if you want that, you can just cast the pointer. Point is, there is no legitimate way to implement the strstr(3) prototype. Somebody must be lying through their teeth. The idea of "const" (and other type declaration paraphernalia) is to prevent accidental bugs at compile time. The noble idea of "const" has been undermined by its sloppy use by the standard libraries and the language itself. BTW, C++ tries to be a bit stricter about "const". It declares two separate prototypes: const char *strstr(const char *, const char *); char *strstr(char *, const char *); Also, in C++, string literals are "const char *". Marko From bc at freeuk.com Thu Oct 12 05:45:30 2017 From: bc at freeuk.com (bartc) Date: Thu, 12 Oct 2017 10:45:30 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> Message-ID: On 12/10/2017 09:23, Christian Gollwitzer wrote: > Am 12.10.17 um 01:15 schrieb Stefan Ram: >> ?? Define a function ?g? with a parameter ?x? of type ?int?, so >> ?? that this function ?g? returns a pointer to another function. >> ?? This other function has a parameter of type ?char? and returns >> ?? a double value. > > Ok > >> ?? /Without/ a typedef. >> > > And WHY would you do that? Typedefs do not cost money. They cost extra effort in devising them, writing them, and having to trace them when someone else is reading the code. > Try this: > typedef double (*countdown_rate_func) (char letter); > countdown_rate_func select_rate_func(int level); > // I've substituted 'x' by 'level' > // and 'g' by 'select_rate_func' for clarity Those substitutions don't help, as I'm trying to see how well it matches the original spec. If I change them back (not easy to tell which bit is whic): typedef double (*F) (char); F g(int x); No, sorry, it doesn't really cut it. It is still cryptic. It doesn't help that C doesn't use a keyword to introduce a function (like, say, 'function'). You still need to know that double (*F) is a pointer to a function returning double, and that double *F is a function returning a pointer to a double. It also introduces this arbitrary name 'F', which if you encounter it in source, means you know have to search for this typedef. (And the requirement was a definition not a declaration.) > I claim that the above is quite clear, it declares a function in a > Countdown game engine, which, depending on the level, returns a > different function to rate the weight of the individual letters. Renaming 'g' and 'x' to be more meaningful, and introducing a parameter name that is not needed, is a different subject. And actually I found your 'countdown_rate_func' and 'select_rate_func' names confusing (the two are actually of different rank, but both end in _func). > Your exercise results in the line noise > ????double (*g(int x))(char) > (thanks to Ben for doing the exercise), which is not extremely bad, but > still does tell nothing about the intent of that line. Using any sane syntax for type (AND function) declarations, writing Stefan's function is straightforward. For example**: function g(int x) => ref function(char)real = return nil end Just transcribed exactly as written (I could have used 'x:int' maybe). If I translate this to C, then it gives me: static double (*g(int32 x)) (char) { return 0; } It's a piece of cake. It shows how important the right syntax can be; you write what you have to write, then get on with the rest of the coding. Oh, and it's also very easy to read. It's a mystery to me actually; there is zero cost to devising, implementing and using the most helpful syntax, and actually there are real benefits. So why use something so bizarre? (**This is my actual language but as someone guessed, this part is derived from Algol68.) -- bartc From iranna.gani28 at gmail.com Thu Oct 12 05:45:48 2017 From: iranna.gani28 at gmail.com (Iranna Mathapati) Date: Thu, 12 Oct 2017 15:15:48 +0530 Subject: how to replace maltipal char from string and substitute new char Message-ID: Hi Team, How to replace multipal char from string and substitute with new char with one line code Ex: str = "9.0(3)X7(2) " ===========> 9.0.3.X7.2 need to replace occurrence of '(',')' with dot(.) chars output: 9.0.3.X7.2 Thanks, From bc at freeuk.com Thu Oct 12 05:57:54 2017 From: bc at freeuk.com (bartc) Date: Thu, 12 Oct 2017 10:57:54 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 12/10/2017 06:39, Grant Edwards wrote: > On 2017-10-11, Gregory Ewing wrote: >> Neil Cerutti wrote: >>> I dig >>> const qualifiers, even though I'm comletely fine with their >>> absence from Python. >> >> Out of curiosity, do you have any insights into why you >> like them in C++, if you don't miss them in Python? > > I like them in C because it allows the linker to place them in ROM > with the code. It also _sometimes_ provides useful diagnostics when > you pass a pointer to something which shouldn't be modified to > something that is going to try to modify it. It's one of these features that on paper sound good. In practice, it's just useless extra clutter. It's used to: (1) Define named constants; except (in C) they can't be used like constant expressions, you can take their addresses, and accidentally or maliciously change their values. (2) Declare data to be put into read-only memory as you say. That's fine with a 'const int * p', but what about a 'int * const p'? (Or is it the other way around? Whatever...). Or any other hybrid type where some elements are const and some aren't. (3) Pass data that is supposed to be read-only, but with any even slightly complex type, it won't work (the head of a linked list for example, where the node elements need to be writeable). It gives a false sense of security. -- bartc From lie.1296 at gmail.com Thu Oct 12 06:43:14 2017 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 12 Oct 2017 03:43:14 -0700 (PDT) Subject: FW: Printing to a file and a terminal at the same time In-Reply-To: References: Message-ID: <3f33eaf2-bbb3-4c98-88d9-9a2dbdd87d8c@googlegroups.com> It wouldn't be too difficult to write a file object wrapper that emulates tee, for instance (untested): class tee(object): def __init__(self, file_objs, autoflush=True): self._files = file_objs self._autoflush = autoflush def write(self, buf): for f in self._files: f.write(buf) if self._autoflush: self.flush() def flush(self): for f in self._files: f.flush() use like so: sys.stdout = tee([sys.stdout, open("myfile.txt", "a")]) Another approach is by bsandrow (https://github.com/bsandrow/pytee), which uses fork and os.pipe(), this more closely resembles what the actual tee command are actually doing. From ben.usenet at bsb.me.uk Thu Oct 12 07:01:03 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 12:01:03 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> Message-ID: <87wp40k4uo.fsf@bsb.me.uk> Chris Angelico writes: > On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >> Chris Angelico writes: >> >>> On Thu, Oct 12, 2017 at 11:55 AM, Ben Bacarisse wrote: >>>> Chris Angelico writes: >>>>> it binds your URLs to >>>>> the concrete file system. That may not seem like too much of a >>>>> problem, but it's a pretty big limitation; you can't have URLs like >>>>> "https://en.wikipedia.org/wiki/Foo" without some help from the web >>>>> server, eg Apache's mod_rewrite. >>>> >>>> I don't follow this. Your "can't" and "big limitation" suggests >>>> something inevitable, but I don't see it as an intrinsic problem with >>>> the language. I'm sure PHP is not as flexible as the frameworks you >>>> mention, but you are not tied to URLs mapping to files. Maybe you meant >>>> that this is what often happens, or what most people do, with PHP. >>> >>> How would you, with PHP itself, handle database-provided URLs? The >>> only way I've ever seen it done is at an external level - such as >>> mod_rewrite - which means that someone else, *not* the PHP script, is >>> managing your URLs. They're pushed to some external config file >>> somewhere. That's okay for just one URL pattern, but it doesn't scale >>> well, which is why (for example) Wikipedia's editing pages are >>> "/w/index.php?...." instead of, say, "/wiki/Foo/edit" or >>> "/wiki/edit/Foo". >>> >>> Unless you know something I don't? >> >> Provided some early part of the URL is handled by PHP, the rest of the >> URL path is provided to PHP in $_SERVER["PATH_INFO"]. > > Is it possible to do that without having ".php" visible in the path? Yes, though because that will depend on how the server is configured I should perhaps say "usually yes"! -- Ben. From breamoreboy at gmail.com Thu Oct 12 07:04:56 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 12 Oct 2017 04:04:56 -0700 (PDT) Subject: how to replace maltipal char from string and substitute new char In-Reply-To: References: Message-ID: <932818a9-0c40-46d3-aff7-c85b5bb7d75d@googlegroups.com> On Thursday, October 12, 2017 at 10:46:03 AM UTC+1, Iranna Mathapati wrote: > Hi Team, > > > How to replace multipal char from string and substitute with new char with > one line code > > Ex: > > str = "9.0(3)X7(2) " ===========> 9.0.3.X7.2 > > need to replace occurrence of '(',')' with dot(.) chars > > output: > > 9.0.3.X7.2 > > > Thanks, >>> help(str.replace) replace(...) S.replace(old, new[, count]) -> str Return a copy of S with all occurrences of substring old replaced by new. If the optional argument count is given, only the first count occurrences are replaced. Hence:- >>> '9.0(3)X7(2)'.replace('(','.').replace(')', '.') '9.0.3.X7.2.' -- Kindest regards. Mark Lawrence. From willow.pine.2011 at gmail.com Thu Oct 12 07:21:33 2017 From: willow.pine.2011 at gmail.com (will) Date: Thu, 12 Oct 2017 04:21:33 -0700 (PDT) Subject: Why the CLI hang using pyjwt ? Message-ID: <2a98e37d-08f4-4ea4-9548-981fc348a49d@googlegroups.com> Not sure why the CLI command "pyjwt decode --no-verify ..." will hang at sys.stdin.read() even though I provided all the input. Any ideas on how to work around the problem? $ pyjwt -v pyjwt 1.5.3 $ pyjwt decode --no-verify eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb21lIjoicGF5bG9hZCJ9.4twFt5NiznN84AWoo1d7KO1T_yoc0Z6XOpOVswacPZg ^CTraceback (most recent call last): File "/usr/local/bin/pyjwt", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/dist-packages/jwt/main.py", line 157, in main output = arguments.func(arguments) File "/usr/local/lib/python2.7/dist-packages/jwt/main.py", line 58, in decode_payload token = sys.stdin.read() $ pyjwt decode --no-verify eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb21lIjoicGF5bG9hZCJ9.4twFt5NiznN84AWoo1d7KO1T_yoc0Z6XOpOVswacPZg ^CTraceback (most recent call last): File "/Users/v612996/anaconda/bin/pyjwt", line 11, in sys.exit(main()) File "/Users/v612996/anaconda/lib/python3.6/site-packages/jwt/main.py", line 157, in main output = arguments.func(arguments) File "/Users/v612996/anaconda/lib/python3.6/site-packages/jwt/main.py", line 58, in decode_payload token = sys.stdin.read() KeyboardInterrupt $ python --version Python 3.6.0 :: Anaconda custom (x86_64) But the same token decoded perfectly under python 2.6 interpreter: jwt.decode(encoded,verify=False) {u'some': u'payload'} What is the work around of "pyjwt decode --no-verify" hang in the CLI? From rosuav at gmail.com Thu Oct 12 07:27:32 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 22:27:32 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> Message-ID: On Thu, Oct 12, 2017 at 7:32 PM, Thomas Jollans wrote: > On 2017-10-12 07:31, Chris Angelico wrote: >> On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >>> Provided some early part of the URL is handled by PHP, the rest of the >>> URL path is provided to PHP in $_SERVER["PATH_INFO"]. >> >> Is it possible to do that without having ".php" visible in the path? > > Just like with Python-based frameworks, this requires a few lines of web > server configuration. > > On Apache, you might use mod_wsgi to tell the server how to run the code > in one case, and a combination of mod_php and mod_rewrite in the other. > If you're using FastCGI with nginx or lighttpd, I believe the > configuration would look pretty similar in both cases. > > Then again, I don't do much web programming any more and generally stay > away from PHP, so I may be misremembering. Normally, with a Python-based framework, you don't need _any_ web server configuration. You simply define your URL routing within the Python code. The only thing the web server needs to know is where to find the web app, and that's sufficiently standard that it can be done off-the-shelf; for instance, you push your code to Heroku, and they set everything up to pass requests to your app. Not possible with PHP, since you need *custom* web server config to manage your rewrite rules. ChrisA From rosuav at gmail.com Thu Oct 12 07:29:51 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 22:29:51 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87vajkbu4c.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87vajkbu4c.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 12, 2017 at 8:20 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Thu, Oct 12, 2017 at 6:22 PM, Marko Rauhamaa wrote: >>> Additionally, you can launder any constant string into a nonconstant >>> string with strstr(3): >>> >>> const char *cs = "hello"; >>> char *s = strstr(cs, ""); >>> s[0] = 'y'; >> >> Well hey, if you want that, you can just cast the pointer. > > Point is, there is no legitimate way to implement the strstr(3) > prototype. Somebody must be lying through their teeth. > > The idea of "const" (and other type declaration paraphernalia) is to > prevent accidental bugs at compile time. The noble idea of "const" has > been undermined by its sloppy use by the standard libraries and the > language itself. > > BTW, C++ tries to be a bit stricter about "const". It declares two > separate prototypes: > > const char *strstr(const char *, const char *); > char *strstr(char *, const char *); > > > > Also, in C++, string literals are "const char *". So basically, C++ fixed some problems in C, in the same way that Python 3 fixed some problems in Python 2. Yet for some reason Python 3 is killing Python, but C++ isn't killing C. Not sure how that works. ChrisA From rosuav at gmail.com Thu Oct 12 07:32:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 12 Oct 2017 22:32:45 +1100 Subject: OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: <7eb848fb-5bed-8766-cbb8-c191f8f7acec@tjol.eu> References: <7eb848fb-5bed-8766-cbb8-c191f8f7acec@tjol.eu> Message-ID: On Thu, Oct 12, 2017 at 8:12 PM, Thomas Jollans wrote: > On 2017-10-12 02:51, Chris Angelico wrote: >> If it wants new life, it's probably going to need a Linux version, >> because that's where a lot of developers hang out. The reality is that >> open source developers are much more likely to develop on Linux than >> on Windows; you can maintain a Windows port of a Linux program with >> fewer Windows experts than maintaining the entire program on Windows. >> >> The other option, though, would be for the useful parts to become >> feature suggestions for VLC. > > > It's the year of the Linux desktop! > > (No, actually, that was a few years ago, but nobody noticed at the time) I'm typing this up from my primary Linux system. Beside me, my laptop also runs Linux. I use these computers for basically everything - coding, testing, work, gaming, the lot. Just finished playing an episode of The Walking Dead, streamed to Twitch.tv; yes, that game wasn't released for Linux, but thanks to Wine, I can run the Windows version, and it's flawless. It's the year of the Linux desktop alright. Has been for some time, gonna still be for a while yet. ChrisA From bc at freeuk.com Thu Oct 12 07:42:22 2017 From: bc at freeuk.com (bartc) Date: Thu, 12 Oct 2017 12:42:22 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 12/10/2017 11:39, Stefan Ram wrote: > bartc writes: >> (1) Define named constants; except (in C) they can't be used like >> constant expressions, you can take their addresses, and accidentally or >> maliciously change their values. > > When I think of ?const?, I do not think of ROM. > > ?const? makes code more readable, because it conveys the > intentions of the author and simplifies the mental variable > model of the reader. > > ?const? helps to find inadvertend modifications. > > void f( const int i ){ if( i = 4 )putchar( 'x' ); } That's two undesirable language features: (1) Having to use 'const' in front of every simple parameter, so that 'const' now dominates every program; (2) Mixing up '=' and '=='. You're saying it's OK to use both as they sort of cancel each other out! (Other languages solve the =/== problem by either not allowing assignment within an expression, or by using a symbol for it that isn't so easily mistaken for equality.) -- bartc From steve+python at pearwood.info Thu Oct 12 08:02:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 12 Oct 2017 23:02:56 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> Message-ID: <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> On Thu, 12 Oct 2017 04:41 pm, Grant Edwards wrote: >> Even two >> different C compilers could return different values. > > Nope. If sizeof char is not 1, then it's not C. Today I Learned. Thank you to everyone who corrected me, even the person who said I was not educated. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From breamoreboy at gmail.com Thu Oct 12 08:11:17 2017 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 12 Oct 2017 05:11:17 -0700 (PDT) Subject: OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]] In-Reply-To: References: <7eb848fb-5bed-8766-cbb8-c191f8f7acec@tjol.eu> Message-ID: On Thursday, October 12, 2017 at 12:33:09 PM UTC+1, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 8:12 PM, Thomas Jollans wrote: > > On 2017-10-12 02:51, Chris Angelico wrote: > >> If it wants new life, it's probably going to need a Linux version, > >> because that's where a lot of developers hang out. The reality is that > >> open source developers are much more likely to develop on Linux than > >> on Windows; you can maintain a Windows port of a Linux program with > >> fewer Windows experts than maintaining the entire program on Windows. > >> > >> The other option, though, would be for the useful parts to become > >> feature suggestions for VLC. > > > > > > It's the year of the Linux desktop! > > > > (No, actually, that was a few years ago, but nobody noticed at the time) > > I'm typing this up from my primary Linux system. Beside me, my laptop > also runs Linux. I use these computers for basically everything - > coding, testing, work, gaming, the lot. Just finished playing an > episode of The Walking Dead, streamed to Twitch.tv; yes, that game > wasn't released for Linux, but thanks to Wine, I can run the Windows > version, and it's flawless. > > It's the year of the Linux desktop alright. Has been for some time, > gonna still be for a while yet. > > ChrisA I have to agree as I've now moved to Linux, Ubuntu as it happens. I wish I'd done it years ago as it feels as if I've had a major hardware upgrade as it's so much more responsive. The most noticeable thing is that when watching videos from YouTube the sound and picture always stay in sync. Not a chance of that with Windows 10. -- Kindest regards. Mark Lawrence. From obulesu.t at gmail.com Thu Oct 12 08:14:04 2017 From: obulesu.t at gmail.com (T Obulesu) Date: Thu, 12 Oct 2017 05:14:04 -0700 (PDT) Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python In-Reply-To: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> References: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Message-ID: <27644acf-fad1-4009-a6c8-f7c586e3893a@googlegroups.com> On Thursday, 12 October 2017 13:38:55 UTC+5:30, T Obulesu wrote: > Hello all, I want to send some frames defined by me{Example, [0x45,0x43,0x32]} to the raspberry pi from any macine(Desktop/Laptop/other raspberry pi). But I want to send those frames over wifi or use wlan0 using python Any suggestions? Yes.. Actully we are sending and receiving ZigBee frames through the serial port. But I want to do the same thing through the Ethernet port/ Wlan0 port rather serial port. My Zigbee Frame Format looks like this: 0x7EA88X00S8899800131A2000 My setup is as follows: My raspberry pi and my laptop both are connected to the same WiFi and ports are wlan0. And it can be directly connected to the laptop using Ethernet cable. If this is the case, I wnat to change the port to the eth0 and send the packets From tjol at tjol.eu Thu Oct 12 08:19:48 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 14:19:48 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <8760bkde5c.fsf@elektro.pacujo.net> <87vajkbu4c.fsf@elektro.pacujo.net> Message-ID: <4b143fac-1033-5f0f-a394-da49e438fedc@tjol.eu> On 2017-10-12 14:01, Stefan Ram wrote: > Many of the quotations are from the previous decade. Thanks Stefan, that was fun. > I must say that C++ has improved in this decade (the 2010s), > and there also is a rennaisance of C and C++ (compared to > "coffee languages") usage because single-thread processor > speeds do not grow anymore ("the free lunch is over") and so > in the 2010s C and C++ are more valued as efficiency > languages (also saving energy in the data centers). This is precisely the motivation of languages like Rust, Go and (in a way) Julia: "C++ is horrid, but C is a slog. Python is too slow for [insert specific use case]. There has to be a better way!" -- Thomas Jollans From marko at pacujo.net Thu Oct 12 08:30:53 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Oct 2017 15:30:53 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87vajkbu4c.fsf@elektro.pacujo.net> Message-ID: <87lgkgblaa.fsf@elektro.pacujo.net> Chris Angelico : > On Thu, Oct 12, 2017 at 8:20 PM, Marko Rauhamaa wrote: >> BTW, C++ tries to be a bit stricter about "const". It declares two >> separate prototypes: >> >> const char *strstr(const char *, const char *); >> char *strstr(char *, const char *); >> >> >> >> Also, in C++, string literals are "const char *". > > So basically, C++ fixed some problems in C, in the same way that > Python 3 fixed some problems in Python 2. Yet for some reason Python 3 > is killing Python, but C++ isn't killing C. Not sure how that works. I have bitten the bullet and do my Python development in Python3. There's no going back. As far as C++ goes, though, it didn't succeed in becoming the "fulfillment" of C programming. In fact, it started a whole new religion. C and C++ share the same Old Testament but have completely different virtues and tenets. I'm not convinced by the tenets of C++. Marko From dvl at psu.edu Thu Oct 12 09:10:39 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Thu, 12 Oct 2017 09:10:39 -0400 Subject: how to replace multiple char from string and substitute new char In-Reply-To: mailman.1302.1507809902.12136.python-list@python.org References: Message-ID: <1507813839l.17629246l.0l@psu.edu> On Thu, Oct 12, 2017, Iranna Mathapati wrote: > >Hi Team, > > >How to replace multipal char from string and substitute with new char with >one line code > >Ex: > >str = "9.0(3)X7(2) " ===========> 9.0.3.X7.2 > >need to replace occurrence of '(',')' with dot(.) chars > >output: > > 9.0.3.X7.\ My first gut response was to key on the words "replace" and "string" and think there is a very handy method already existing that can help with that. It would be one line of code, if you don't mind having wo function calls in that one line of code. If you want only one function call and you want to do even more substitutions than just those below, there is another handy functionin the string object that can do so very nicely. What did you try? Roger Christman Pennsylvania State University From ben.usenet at bsb.me.uk Thu Oct 12 09:16:24 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 14:16:24 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> Message-ID: <87sheojyl3.fsf@bsb.me.uk> Gregory Ewing writes: > Ben Bacarisse wrote: >> That's a different type. I think you mean that a human writing C >> (rather than bartc's code generator) would probably design the code to >> use tokenrec ** then I agree, but the latter is not just a different way >> to write the former. > > Yes, I was translating his English description of the type > into C, using C's meaning of the word "array". It seems that > arrays in the original language (Algol? One of Bart's > inventions?) are somewhat richer things. Probably, but you can have pointers to array types in C which is what the posted type used. Humans pass arrays in C by passing a pointer to the first element. Pointers to arrays therefore crop up when passing 2D (or higher) arrays, even when the code is hand written by a person. -- Ben. From neilc at norwich.edu Thu Oct 12 09:54:33 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Thu, 12 Oct 2017 13:54:33 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 2017-10-11, Gregory Ewing wrote: > Neil Cerutti wrote: >> I dig const qualifiers, even though I'm comletely fine with >> their absence from Python. > > Out of curiosity, do you have any insights into why you like > them in C++, if you don't miss them in Python? I can tell at a glance if a parameter is expected to be modifiable just by looking at the function signature. Also using reference types feels more comfortable when it's easy to know it won't be changed. This isn't technical, but it feels somehow satifying the way const propogates seemlessly through a well-designed interface. When programming Python, the only constants I care about are magic numbers and lookup tables defined at the top of a program or outside the program. If there are enough of them I'll put them in a module or class/namespace. They don't exist in interfaces, so I don't think about them in that sense. -- Neil Cerutti From neilc at norwich.edu Thu Oct 12 09:56:31 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Thu, 12 Oct 2017 13:56:31 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> Message-ID: On 2017-10-12, Marko Rauhamaa wrote: > Bill : > >> Marko Rauhamaa wrote: >>> One example is the surprising fact that string literals in C >>> are "char *" and not "const char *". >> >> If not, you couldn't pass a string literal to a function >> having prototype void f(char *s); > > That *ought* to be prevented. That's the whole point. I'm far less experienced in C, but I threw up my hands and stopped bothering with const qualifiers in C due to such headaches. When in Rome, program without const qualifiers in C. -- Neil Cerutti From ben.usenet at bsb.me.uk Thu Oct 12 10:09:15 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 15:09:15 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> Message-ID: <87o9pcjw50.fsf@bsb.me.uk> Chris Angelico writes: > On Thu, Oct 12, 2017 at 7:32 PM, Thomas Jollans wrote: >> On 2017-10-12 07:31, Chris Angelico wrote: >>> On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >>>> Provided some early part of the URL is handled by PHP, the rest of the >>>> URL path is provided to PHP in $_SERVER["PATH_INFO"]. >>> >>> Is it possible to do that without having ".php" visible in the path? >> >> Just like with Python-based frameworks, this requires a few lines of web >> server configuration. >> >> On Apache, you might use mod_wsgi to tell the server how to run the code >> in one case, and a combination of mod_php and mod_rewrite in the other. >> If you're using FastCGI with nginx or lighttpd, I believe the >> configuration would look pretty similar in both cases. >> >> Then again, I don't do much web programming any more and generally stay >> away from PHP, so I may be misremembering. > > Normally, with a Python-based framework, you don't need _any_ web > server configuration. You simply define your URL routing within the > Python code. The only thing the web server needs to know is where to > find the web app, and that's sufficiently standard that it can be done > off-the-shelf; for instance, you push your code to Heroku, and they > set everything up to pass requests to your app. Not possible with PHP, > since you need *custom* web server config to manage your rewrite > rules. That's at odds with what I've read online which admittedly may be all junk. I wanted to try Flask so I installed the Ubuntu packages but then got stuck on a huge document that suggested I needed to install things called Nginx and Gunicorn. You've now mentioned another: Heroku. I'm sure the complex instructions I found are not really required -- it was probably just the usual "this is what I did so this is how it's done" document, but I'm having trouble finding the simpler way to do it. Since no web server configuration is needed (I have a working Apache installation that mirrors, as closely as possible, what my hosting provider uses) it should be relatively easy. Can you tell me, or can you point me to a resource that tells me, where to put the app? I don't yet know what "push your code to Heroku" means. -- Ben. From tjol at tjol.eu Thu Oct 12 10:12:59 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 12 Oct 2017 16:12:59 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87sheojyl3.fsf@bsb.me.uk> References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> <87sheojyl3.fsf@bsb.me.uk> Message-ID: On 2017-10-12 15:16, Ben Bacarisse wrote: > Gregory Ewing writes: > >> Ben Bacarisse wrote: >>> That's a different type. I think you mean that a human writing C >>> (rather than bartc's code generator) would probably design the code to >>> use tokenrec ** then I agree, but the latter is not just a different way >>> to write the former. >> >> Yes, I was translating his English description of the type >> into C, using C's meaning of the word "array". It seems that >> arrays in the original language (Algol? One of Bart's >> inventions?) are somewhat richer things. > > Probably, but you can have pointers to array types in C which is what > the posted type used. Humans pass arrays in C by passing a pointer to > the first element. Pointers to arrays therefore crop up when passing 2D > (or higher) arrays, even when the code is hand written by a person. > No, actually. Multi-dimensional arrays in C are not arrays of arrays. They look very similar, but when the dimensions of the array are known at compile time, arrays of any rank are a continuous region of memory with the data, and a pointer to the start. Casting int[3][3] to int** will not do what you want it to do. If, say, you have variables of the types: int a[M][N], **b; then a[i][j] is equivalent to *(a+(i*M)+j), while b[i][j] is equivalent to *(*(b+i)+j). Observe: --------------------------------------------------------------------------- #include #include #include int main (int argc, char **argv) { int i,j; /* multi-dimensional array: the compiler knows the dimensions! */ short multi_array[3][3] = { {1, 0, 0}, {0, 1, 0}, {0, 0, 1} }; short *array_array[3]; /* fill array_array elements of multi_array */ for(i=0; i<3; ++i) { array_array[i] = calloc(3, sizeof(short)); /* This works because C arrays are row-major */ memcpy(array_array[i], &multi_array[i][0], 3*sizeof(short)); } /* print out the arrays */ puts("multi_array:"); for (i=0; i<3; ++i) { for (j=0; j<3; ++j) { printf("%d ", multi_array[i][j]); } puts(""); } puts("array_array:"); for (i=0; i<3; ++i) { for (j=0; j<3; ++j) { printf("%d ", array_array[i][j]); } puts(""); } printf("&multi_array = 0x%llX\n", (long long)&multi_array); printf("*multi_array = 0x%llX\n", (long long)(*(short**)multi_array)); printf("&array_array = 0x%llX\n", (long long)&array_array); printf("*array_array = 0x%llX\n", (long long)(*(short**)array_array)); /* clean up */ for (i=0; i<3; ++i) { free(array_array[i]); } return 0; } --------------------------------------------------------------------------- [OUTPUT] multi_array: 1 0 0 0 1 0 0 0 1 array_array: 1 0 0 0 1 0 0 0 1 &multi_array = 0x7FFE0E736470 *multi_array = 0x1 &array_array = 0x7FFE0E736450 *array_array = 0x1A42010 From steve+python at pearwood.info Thu Oct 12 10:19:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 13 Oct 2017 01:19:56 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <8760bkde5c.fsf@elektro.pacujo.net> <87vajkbu4c.fsf@elektro.pacujo.net> Message-ID: <59df7a0d$0$14932$b1db1813$d948b532@news.astraweb.com> On Thu, 12 Oct 2017 11:01 pm, Stefan Ram quoted: > Basically I got sick of every single > aspect of C++ being designed around higher performance > instead of my productivity. Unlike C, where every single aspect of the language is designed around higher performance instead of the developer's productivity. > [...] If I had to write a > high performance application these days I would reach > for C. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jon+usenet at unequivocal.eu Thu Oct 12 10:36:19 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Thu, 12 Oct 2017 14:36:19 -0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> <87o9pcjw50.fsf@bsb.me.uk> Message-ID: On 2017-10-12, Ben Bacarisse wrote: > Chris Angelico writes: >> Normally, with a Python-based framework, you don't need _any_ web >> server configuration. You simply define your URL routing within the >> Python code. The only thing the web server needs to know is where to >> find the web app, and that's sufficiently standard that it can be done >> off-the-shelf; for instance, you push your code to Heroku, and they >> set everything up to pass requests to your app. Not possible with PHP, >> since you need *custom* web server config to manage your rewrite >> rules. > > That's at odds with what I've read online which admittedly may be all > junk. I wanted to try Flask so I installed the Ubuntu packages but then > got stuck on a huge document that suggested I needed to install things > called Nginx and Gunicorn. You've now mentioned another: Heroku. I'm > sure the complex instructions I found are not really required -- it was > probably just the usual "this is what I did so this is how it's done" > document, but I'm having trouble finding the simpler way to do it. > > Since no web server configuration is needed (I have a working Apache > installation that mirrors, as closely as possible, what my hosting > provider uses) it should be relatively easy. Can you tell me, or can > you point me to a resource that tells me, where to put the app? I don't > yet know what "push your code to Heroku" means. "don't need _any_ web server configuration" is rather, er, optimistic. For Apache you'd need the mod_proxy_uwsgi module installed, and the config would be something like this: DocumentRoot /srv/www/appname/appname ProxyPass uwsgi://127.0.0.1:3031/ ProxyPass ! and you need an app container listening on the port defined above, e.g. uwsgi with config like: /etc/uwsgi/apps-available/appname.ini: [uwsgi] plugin = python3 socket = 127.0.0.1:3031 threads = 4 master = 1 chdir = /srv/www/appname module = appname:app # https://github.com/unbit/uwsgi/issues/1126 wsgi-disable-file-wrapper = true and you'll need something to run uwsgi on system startup. From grant.b.edwards at gmail.com Thu Oct 12 10:56:10 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 14:56:10 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> Message-ID: On 2017-10-12, Neil Cerutti wrote: > On 2017-10-12, Marko Rauhamaa wrote: >> Bill : >> >>> Marko Rauhamaa wrote: >>>> One example is the surprising fact that string literals in C >>>> are "char *" and not "const char *". Yep, that's the basis for a lot of the problems with 'const' in C. Unfortunately for historical reasons, a lot of people expect string literals to be mutable. That should have been stamped out decades ago, but it's too late now. >>> If not, you couldn't pass a string literal to a function >>> having prototype void f(char *s); IMO, it should be illegal to pass a string literal to a function with prototype f(char *s). You shouldn't try to modify a string literal, and if f() isn't going to modify the string, it should have been declared f(const char *s). >> That *ought* to be prevented. That's the whole point. > > I'm far less experienced in C, but I threw up my hands and stopped > bothering with const qualifiers in C due to such headaches. When in > Rome, program without const qualifiers in C. Using const with strings in C with amateurish libraries is a headache because _some_people_ will write their declarations so as to require pointers to mutable strings even when they have no intention of mutating them. Those people should be hunted down and slapped with a herring until they understand the error of their ways. The standard library is much better about that. -- Grant Edwards grant.b.edwards Yow! Kids, don't gross me at off ... "Adventures with gmail.com MENTAL HYGIENE" can be carried too FAR! From grant.b.edwards at gmail.com Thu Oct 12 11:01:13 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 15:01:13 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87vajkbu4c.fsf@elektro.pacujo.net> Message-ID: On 2017-10-12, Marko Rauhamaa wrote: > Chris Angelico : > >> On Thu, Oct 12, 2017 at 6:22 PM, Marko Rauhamaa wrote: >>> Additionally, you can launder any constant string into a nonconstant >>> string with strstr(3): >>> >>> const char *cs = "hello"; >>> char *s = strstr(cs, ""); >>> s[0] = 'y'; >> >> Well hey, if you want that, you can just cast the pointer. > > Point is, there is no legitimate way to implement the strstr(3) > prototype. Somebody must be lying through their teeth. That's indeed a problem. Personally, I would just use two prototypes: char *strcstr(const char *s, char *s); const char *cstrstr(const char *s, const char *s); Whether you want to invoke some linker-script magic to make them refer to the same blob of code or not is optional. -- Grant Edwards grant.b.edwards Yow! I want my nose in at lights! gmail.com From grant.b.edwards at gmail.com Thu Oct 12 11:06:37 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 15:06:37 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-12, Steve D'Aprano wrote: > On Thu, 12 Oct 2017 04:41 pm, Grant Edwards wrote: > > >>> Even two >>> different C compilers could return different values. >> >> Nope. If sizeof char is not 1, then it's not C. > > Today I Learned. It sure was an education the first I wrote C code for a machine where 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof double All were 32 bits. Writing protocol code that dealt with the outside world via a serial port was _painful_. -- Grant Edwards grant.b.edwards Yow! ... I want a COLOR at T.V. and a VIBRATING BED!!! gmail.com From grant.b.edwards at gmail.com Thu Oct 12 11:12:55 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 15:12:55 +0000 (UTC) Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python References: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Message-ID: On 2017-10-12, Chris Angelico wrote: > On Thu, Oct 12, 2017 at 7:08 PM, T Obulesu wrote: >> Hello all, I want to send some frames defined by me{Example, [0x45,0x43,0x32]} to the raspberry pi from any macine(Desktop/Laptop/other raspberry pi). But I want to send those frames over wifi or use wlan0 using python Any suggestions? >> -- > > Since you use the word "frame", I'm thinking you possibly mean at a > very low level, and not packaged up into UDP/IP packets. I'm curious > as to why you'd need that; for most purposes, the easiest way would be > either TCP or UDP. But sending raw packets is entirely possible in > Python - though you have to bear in mind that many OSes restrict this > capability to administrative programs. Under Linux, theres a network capability that can be enabled for a program to allow raw packet access w/o having to be root. However, it's not very useful for Python apps, since that capability would have to be set for the Python interpreter executable (e.g. /usr/bin/python). I avoid Windows as much as possible, but in recent years, colleagues who do work with Windows have had to convert all of our applicatoins which used to use raw packets to use UDP instead. I'm told that recent windows versions have made raw packet access from userspace (even by admin apps) virtually impossible. -- Grant Edwards grant.b.edwards Yow! Hello? Enema Bondage? at I'm calling because I want gmail.com to be happy, I guess ... From marko at pacujo.net Thu Oct 12 11:18:44 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 12 Oct 2017 18:18:44 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> Message-ID: <871sm875t7.fsf@elektro.pacujo.net> Grant Edwards : > Using const with strings in C with amateurish libraries is a headache > because _some_people_ will write their declarations so as to require > pointers to mutable strings even when they have no intention of > mutating them. Those people should be hunted down and slapped with a > herring until they understand the error of their ways. Hear, hear. > The standard library is much better about that. You lost me there. Seriously, though. C strings are not the most problematic issue. How about other structures? What about: long ftell(FILE *stream); int fgetpos(FILE *stream, fpos_t *pos); Should that be "const FILE *stream"? In general, C record definitions (opaque structs) that represent encapsulated classes don't take a "const" in any context. They *could* but that isn't the tradition. For example, here's a random function from the Linux kernel: ======================================================================== static bool tcp_fastopen_cookie_gen(struct request_sock *req, struct sk_buff *syn, struct tcp_fastopen_cookie *foc) { if (req->rsk_ops->family == AF_INET) { const struct iphdr *iph = ip_hdr(syn); __be32 path[4] = { iph->saddr, iph->daddr, 0, 0 }; return __tcp_fastopen_cookie_gen(path, foc); } #if IS_ENABLED(CONFIG_IPV6) if (req->rsk_ops->family == AF_INET6) { const struct ipv6hdr *ip6h = ipv6_hdr(syn); struct tcp_fastopen_cookie tmp; if (__tcp_fastopen_cookie_gen(&ip6h->saddr, &tmp)) { struct in6_addr *buf = (struct in6_addr *) tmp.val; int i; for (i = 0; i < 4; i++) buf->s6_addr32[i] ^= ip6h->daddr.s6_addr32[i]; return __tcp_fastopen_cookie_gen(buf, foc); } } #endif return false; } ======================================================================== Note how both "req" and "syn" could well be declared as "const" pointers but are not. Marko From formisc at gmail.com Thu Oct 12 11:18:55 2017 From: formisc at gmail.com (Andrew Z) Date: Thu, 12 Oct 2017 11:18:55 -0400 Subject: Logging from files doesn't work In-Reply-To: <20171012032949.GA76681@cskk.homeip.net> References: <20171012032949.GA76681@cskk.homeip.net> Message-ID: Cameron, Peter, Thank you. Your comments were spot on. Changing root logger got the logs spitting into the file. And i now can org these logs into one directory, instead of the current mess. Thank you! On Oct 11, 2017 23:41, "Cameron Simpson" wrote: > On 11Oct2017 22:27, Andrew Z wrote: > >> aha. So the issue is that main.py's __name__ attribute == "__main__" and >> test.py is "test1.test". >> > > Yeah. If you invoke a module as "python -m module_name" its __name__ field > is "__main__". That makes the boilerplate work, but breaks your expectation > that __name__ is usually module_name. > > if i manually assign names: >> main.py - > >> >> log = logging.getLogger("MAIN") >> >> test.py - > >> log = logging.getLogger("MAIN.test1.test") >> >> then logging is working perfectly well. >> >> This brings me to the question - what is wrong with me file >> naming/structure that confuses the logging module? I'm not sure i >> really want to have each little file in it's own directory too.. >> > > I confess to not using the logging module in the module-named-based > fashion that seems to be the default recommendation. I usually prefer to > set up the root logger to log somewhere suitable to the main program > (typically just stderr by default), and set up special logging for other > things as needed. > > As what may be a poor example, I've got a mail filing program which > monitors maildirs for arriving messages to refile. So I've got a class > associated with each monitored maildir with these methods: > > @property > def logdir(self): > ''' The pathname of the directory in which log files are written. > ''' > varlog = cs.env.LOGDIR(self.environ) > return os.path.join(varlog, 'mailfiler') > > This essentially computes "$HOME/var/log/mailfiler" as the place where all > the logfiles are saved. And this: > > def folder_logfile(self, folder_path): > ''' Return path to log file associated with the named folder. > TODO: base on relative path from folder root, not just basename. > ''' > return os.path.join(self.logdir, '%s.log' % > (os.path.basename(folder_path))) > > which computes "$HOME/var/log/mailfiler/spool.log" as the logfile when > working on my maildir "$HOME/mail/spool". > > And then off it goes with a FileHandler for that path for the filing > actions for that maildir. > > So you're not contrained to drop log files all through your source tree > (ewww!) > > My opinion is that you should decide where your logfiles _should_ live; it > generally has nothing (or little) to do with the module name. I keep mine, > generally, in various subdirectories of "$HOME/var/log". > > Cheers, > Cameron Simpson (formerly cs at zip.com.au) > From grant.b.edwards at gmail.com Thu Oct 12 11:38:44 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 15:38:44 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> <871sm875t7.fsf@elektro.pacujo.net> Message-ID: On 2017-10-12, Marko Rauhamaa wrote: > Grant Edwards : > >> Using const with strings in C with amateurish libraries is a headache >> because _some_people_ will write their declarations so as to require >> pointers to mutable strings even when they have no intention of >> mutating them. Those people should be hunted down and slapped with a >> herring until they understand the error of their ways. > > Hear, hear. > >> The standard library is much better about that. > > You lost me there. I meant that in the include files for the C standard library, functions that aren't going to modify a string paramameter always declare the parameter as "const char *". > Seriously, though. C strings are not the most problematic issue. How > about other structures? What about: > > long ftell(FILE *stream); > int fgetpos(FILE *stream, fpos_t *pos); > > Should that be "const FILE *stream"? IMO, yes. > In general, C record definitions (opaque structs) that represent > encapsulated classes don't take a "const" in any context. They > *could* but that isn't the tradition. In my own code I do try to do that, but (as in the Linux kernel code you posted) you still run into problems using third-party libraries written by the, um, unenlightened. > For example, here's a random function from > the Linux kernel: > >======================================================================== > static bool tcp_fastopen_cookie_gen(struct request_sock *req, > struct sk_buff *syn, > struct tcp_fastopen_cookie *foc) > { [...] > Note how both "req" and "syn" could well be declared as "const" pointers > but are not. Yep. IMO, that's just sloppy programming. -- Grant Edwards grant.b.edwards Yow! Can you MAIL a BEAN at CAKE? gmail.com From dvl at psu.edu Thu Oct 12 11:53:40 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Thu, 12 Oct 2017 11:53:40 -0400 Subject: Python-list Digest, Vol 169, Issue 23 In-Reply-To: mailman.1302.1507809902.12136.python-list@python.org References: Message-ID: <1507823620l.6357002l.0l@psu.edu> On Thu, Oct 12, 2017 08:05 AM, Steve D'Aprano wrote:> On Wed, 11 Oct 2017 10:57 pm, Stefan Ram wrote: > >> FWIW, in is book "Touch of Class" (2009) Bertrand >Meyer writes: >> >> |Such instructions are just the old goto in sheep's clothing. >> |Treat them the same way as the original: >> | >> |/Touch of Methodology/: >> | Sticking to one-entry, one-exit building blocks >> |Stay away from any "break" or similar control mechanism. > >I have a great deal of respect for Meyer, but in this case I think he has it >badly wrong on both counts (`break` and "one-entry, >one-exit"). > >Unrestricted GOTO (as in BASIC, circa 1970, where you could jump to any >line >in the code, or as in assembly) is rightly considered harmful (albeit >necessary in assembly). But if Meyer means *any* sort of jump >("similar >control mechanism"), then that would rule out not just `break` and >`continue` >but also: > > * while loops > * for loops > * if...else > * case/switch statements > * calling subroutines (functions or procedures) > * exception handling > >Remember that a function call is basically a GOTO in disguise: execution jumps >to the function, and jumps back when the function returns. In BASIC, there >was a GOSUB intermediate between a GOTO and a procedure call. > >Surely Meyer doesn't mean to say we should never call functions or use >`while`, `for` or `if`. So he has to distinguish between kinds of jumps: > >Good jumps (I presume): > > * function calls > * if...else > * looping > >Evil jumps: > > * unrestricted BASIC-style GOTO/GOSUB any line number > * break/continue > >Not sure: > > * GOTO where there are restrictions on where you can jump > * COMEFROM > >(I kid: I'm sure Meyer would oppose COMEFROM, and I expect that even >Pascal-style restricted GOTO would be on his "evil" list to >avoid.) This seems like a veritable straw man if any. I am fairly sure that "one entry, one exit" does not precisely mean "no branching whatsoever" Nor does discouraging unrestricted GOTO suggest that either. Clearly a while or for loop without break i is single-entry and single exit. The if-else has a clear single entry at the test condition, and a single exit could be imagined at the program point immediately after the if-else., etc. The case/switch does have a single exit in the same way as the if. Unfortunately, it does in itself sort of violate the single entry idea when a break does not precede the next case. That just leaves the multiple returns in a functionand the exception handling. >So the question becomes, why are such harmless, simple to understand, innocuous jumps like `break` and `continue` in the evil list, when they not only simplify code but make it more efficient? # with break for i in range(2**64): if isprime(i): print(i, "is prime") break # without break still_searching = True for i in range(2**64): if still_searching and isprime(i): print(i, "is prime") still_searching = False # without break, attempt number 2 still_searching = True i = 0 while still_searching and i < 2**64: if isprime(i): print(i, "is prime") still_searching = False # without break, the way it should be: i = 0 while not isprime(i): ... i = i+1 print(i, "is prime") I believe this is far simpler _and_ more efifcient than any of your answers above. Do you really believe that there are no prime numbers less than 2**64? Can you imagine any circumstance when "i < 2**75" would become false in this application? Can you really claim that a solution is more efficient when it repeatedly tests a condition that can never posslbly be false? That is the biggest objection programming teachers like me dislike the break statement -- it leads to very careless programming. Once you imagine that you will use a break to exit your loops, you might stop thinking about what your loop conditions are. And then you find yourself in a corner where the only way out is to break. It's like: I know that I can always call a tow truck whenever my car runs out of gas on the highway, so I'll no longer bother checking or filling my tank. I'll just drive until I'm beyohd fumes, break down, call a tow, and then tell my boss how I was unavoidably late because I broke down on the highway. Ironically, after telling us to stick to code with one entry and one exit, Meyer then contradicts himself by recommending exceptions: > |You can use exception handling as a technique of last resort > |to handle unexpected events for which the normal control > |structures let you down. Even more ironically, exception handling is most similar to a COMEFROM, which was invented as a joke. The exception handler itself has one entry (the exception) and one exit. And, unlike goto's and set_jmp() is easiliy incorporated into other control structures depending on where you want to go after any recovery steps. The code that generated the exception, of course, seemingly has more than one exit, but exceptions are just that -- exceptions. I hope you don't start using counting loops up to 2**64 to visit a 100 element array and rely on IndexError to exit such a loop. Roger Christman Pennsylvania State University From rhodri at kynesim.co.uk Thu Oct 12 11:55:04 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 12 Oct 2017 16:55:04 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> Message-ID: On 12/10/17 16:06, Grant Edwards wrote: > On 2017-10-12, Steve D'Aprano wrote: >> On Thu, 12 Oct 2017 04:41 pm, Grant Edwards wrote: >> >> >>>> Even two >>>> different C compilers could return different values. >>> >>> Nope. If sizeof char is not 1, then it's not C. >> >> Today I Learned. > > It sure was an education the first I wrote C code for > a machine where > > 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof double > > All were 32 bits. Ah yes. In my case it was 16 bits, so sizeof long == 2 just for a little variation. > Writing protocol code that dealt with the outside world via a serial > port was _painful_. Amen, brother. -- Rhodri James *-* Kynesim Ltd From bc at freeuk.com Thu Oct 12 11:57:23 2017 From: bc at freeuk.com (bartc) Date: Thu, 12 Oct 2017 16:57:23 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <871sm875t7.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> <871sm875t7.fsf@elektro.pacujo.net> Message-ID: On 12/10/2017 16:18, Marko Rauhamaa wrote: > Grant Edwards : > >> Using const with strings in C with amateurish libraries is a headache >> because _some_people_ will write their declarations so as to require >> pointers to mutable strings even when they have no intention of >> mutating them. Those people should be hunted down and slapped with a >> herring until they understand the error of their ways. > > Hear, hear. > >> The standard library is much better about that. > > You lost me there. > > Seriously, though. C strings are not the most problematic issue. How > about other structures? What about: > > long ftell(FILE *stream); > int fgetpos(FILE *stream, fpos_t *pos); > > Should that be "const FILE *stream"? > > In general, C record definitions (opaque structs) that represent > encapsulated classes don't take a "const" in any context. They *could* > but that isn't the tradition. For example, here's a random function from > the Linux kernel: > > ======================================================================== > static bool tcp_fastopen_cookie_gen(struct request_sock *req, > struct sk_buff *syn, > struct tcp_fastopen_cookie *foc) > { > if (req->rsk_ops->family == AF_INET) { > const struct iphdr *iph = ip_hdr(syn); > > __be32 path[4] = { iph->saddr, iph->daddr, 0, 0 }; > return __tcp_fastopen_cookie_gen(path, foc); > } > > #if IS_ENABLED(CONFIG_IPV6) > if (req->rsk_ops->family == AF_INET6) { > const struct ipv6hdr *ip6h = ipv6_hdr(syn); > struct tcp_fastopen_cookie tmp; > > if (__tcp_fastopen_cookie_gen(&ip6h->saddr, &tmp)) { > struct in6_addr *buf = (struct in6_addr *) tmp.val; > int i; > > for (i = 0; i < 4; i++) > buf->s6_addr32[i] ^= ip6h->daddr.s6_addr32[i]; > return __tcp_fastopen_cookie_gen(buf, foc); > } > } > #endif > return false; > } > ======================================================================== If you took a working C program and removed all the consts, it would still work. I don't think the language would miss them if they were to disappear. It is anyway too crude a technique for the things people like to apply it to. > > Note how both "req" and "syn" could well be declared as "const" > pointers but are not. const pointer, or pointer to const struct? Or both? With a const struct, you are stopped from directly modifying elements, but if an element is a pointer, nothing stops you writing to what the pointer points to, unless that has a const target too. And then you've going to have problems doing normal updates. This constness just insinuates itself everywhere. -- bartc -- bartc From ben.usenet at bsb.me.uk Thu Oct 12 11:57:45 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 16:57:45 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> <87o9pcjw50.fsf@bsb.me.uk> Message-ID: <87fuaojr46.fsf@bsb.me.uk> Jon Ribbens writes: > On 2017-10-12, Ben Bacarisse wrote: >> Chris Angelico writes: >>> Normally, with a Python-based framework, you don't need _any_ web >>> server configuration. You simply define your URL routing within the >>> Python code. The only thing the web server needs to know is where to >>> find the web app, and that's sufficiently standard that it can be done >>> off-the-shelf; for instance, you push your code to Heroku, and they >>> set everything up to pass requests to your app. Not possible with PHP, >>> since you need *custom* web server config to manage your rewrite >>> rules. >> >> That's at odds with what I've read online which admittedly may be all >> junk. I wanted to try Flask so I installed the Ubuntu packages but then >> got stuck on a huge document that suggested I needed to install things >> called Nginx and Gunicorn. You've now mentioned another: Heroku. I'm >> sure the complex instructions I found are not really required -- it was >> probably just the usual "this is what I did so this is how it's done" >> document, but I'm having trouble finding the simpler way to do it. >> >> Since no web server configuration is needed (I have a working Apache >> installation that mirrors, as closely as possible, what my hosting >> provider uses) it should be relatively easy. Can you tell me, or can >> you point me to a resource that tells me, where to put the app? I don't >> yet know what "push your code to Heroku" means. > > "don't need _any_ web server configuration" is rather, er, optimistic. It did seem so. I could not imagine any way it would "just" work unless it was already set up to "just work". > For Apache you'd need the mod_proxy_uwsgi module installed, and the > config would be something like this: > > DocumentRoot /srv/www/appname/appname > > ProxyPass uwsgi://127.0.0.1:3031/ > > > ProxyPass ! > > > and you need an app container listening on the port defined above, > e.g. uwsgi with config like: > > /etc/uwsgi/apps-available/appname.ini: > > [uwsgi] > plugin = python3 > socket = 127.0.0.1:3031 > threads = 4 > master = 1 > chdir = /srv/www/appname > module = appname:app > # https://github.com/unbit/uwsgi/issues/1126 > wsgi-disable-file-wrapper = true > > and you'll need something to run uwsgi on system startup. I see. If I'm reading this right, the app requests are passed through to another server -- uWSGI. How does this typically work on low-cost hosting? I may be able to set up the ProxyPass locally (i.e. in .htaccess) but I won't be able to write /etc/uwsgi/apps-available/appname.ini. Maybe there are a locally defined .ini files that uwsgi reads? As for running something on startup... I suppose I can ask. Maybe it's usual to run it anyway. -- Ben. From grant.b.edwards at gmail.com Thu Oct 12 12:08:04 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 12 Oct 2017 16:08:04 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-12, Rhodri James wrote: > On 12/10/17 16:06, Grant Edwards wrote: >> On 2017-10-12, Steve D'Aprano wrote: >>> On Thu, 12 Oct 2017 04:41 pm, Grant Edwards wrote: >>> >>>>> Even two different C compilers could return different values. >>>> >>>> Nope. If sizeof char is not 1, then it's not C. >>> >>> Today I Learned. >> >> It sure was an education the first I wrote C code for >> a machine where >> >> 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof double >> >> All were 32 bits. > > Ah yes. In my case it was 16 bits, so sizeof long == 2 just for a > little variation. It does help when a char is an integral number of octets. Working with a machine where everything is 20 bits would be even worse. I wonder if anybody ever designed a CPU where the word size was odd? -- Grant Edwards grant.b.edwards Yow! How many retured at bricklayers from FLORIDA gmail.com are out purchasing PENCIL SHARPENERS right NOW?? From ben.usenet at bsb.me.uk Thu Oct 12 12:25:16 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 17:25:16 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <5HpDB.6442$cW1.5993@fx07.am4> <9BuDB.15425$uS.4321@fx24.am4> <87d15tmh3t.fsf@bsb.me.uk> <87sheojyl3.fsf@bsb.me.uk> Message-ID: <87bmlcjpub.fsf@bsb.me.uk> Thomas Jollans writes: > On 2017-10-12 15:16, Ben Bacarisse wrote: >> Gregory Ewing writes: >> >>> Ben Bacarisse wrote: >>>> That's a different type. I think you mean that a human writing C >>>> (rather than bartc's code generator) would probably design the code to >>>> use tokenrec ** then I agree, but the latter is not just a different way >>>> to write the former. >>> >>> Yes, I was translating his English description of the type >>> into C, using C's meaning of the word "array". It seems that >>> arrays in the original language (Algol? One of Bart's >>> inventions?) are somewhat richer things. >> >> Probably, but you can have pointers to array types in C which is what >> the posted type used. Humans pass arrays in C by passing a pointer to >> the first element. Pointers to arrays therefore crop up when passing 2D >> (or higher) arrays, even when the code is hand written by a person. >> > > No, actually. Multi-dimensional arrays in C are not arrays of arrays. That's exactly what they are (in their simplest form). Other structures using pointers can be accessed in the same way so some people call those multi-dimensional arrays, but that can be a bit confusing. > They look very similar, but when the dimensions of the array are known > at compile time, arrays of any rank are a continuous region of memory > with the data, and a pointer to the start. > > Casting int[3][3] to int** will not do what you want it to do. Exactly. This is why I raised this as an example where you get a pointer to an array type. int ** is something else altogether. When you pass an int[3][3] to a function, it pops up in the function as an int (*)[3]. You can (due to another of C's odd rules) write the parameter as int a[][3], or int a[3][3], but the first array in the declarator is replaced by a pointer (and the size is ignored). > If, say, you have variables of the types: > > int a[M][N], **b; > > then a[i][j] is equivalent to *(a+(i*M)+j), No it isn't. That expression does not even have type int. In fact you will be surprised to learn that a[i][j] it is equivalent to *(*(a+i)+j). > while b[i][j] is equivalent to *(*(b+i)+j). That much is true. The fact that the genuine 2D array access (a[i][j]) is equivalent an expression of the same form as the pointer to pointer access (b[i][j]) looks odd because, as you know, they are doing different things. But they do different things because the types are different. > Observe: > short multi_array[3][3] = { > {1, 0, 0}, > {0, 1, 0}, > {0, 0, 1} > }; > short *array_array[3]; I would not have used that name. This is an array of pointers. > /* fill array_array elements of multi_array */ > for(i=0; i<3; ++i) { > array_array[i] = calloc(3, sizeof(short)); > /* This works because C arrays are row-major */ > memcpy(array_array[i], &multi_array[i][0], 3*sizeof(short)); Some people would find array_array[i] = malloc(sizeof multi_array[i]); memcpy(array_array[i], multi_array[i], sizeof multi_array[i]); to be clearer and more maintainable. You don't repeat the element type and there's no need to reference the (possibly arbitrary) size 3. It makes it very clear that enough space is being allocated for what is being copied. > } > > /* print out the arrays */ > puts("multi_array:"); > for (i=0; i<3; ++i) { > for (j=0; j<3; ++j) { > printf("%d ", multi_array[i][j]); Try *(multi_array+(i*3)+j) here to see what happens (that's your re-write with 'a' and 'M' substituted). > } > puts(""); > } -- Ben. From dvl at psu.edu Thu Oct 12 13:06:16 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Thu, 12 Oct 2017 13:06:16 -0400 Subject: Python-list Digest, Vol 169, Issue 23 In-Reply-To: mailman.1302.1507809902.12136.python-list@python.org References: Message-ID: <1507827975l.37028074l.0l@psu.edu> The opinions I give in this post are not as strong as the opinions I expressed before about the loop claiming to count all the way to 2**64. But I'll give them anyway. I don't mind having return's inside an if statement, when the function ends with that if statement. It becomes very clear then that there is a choice of possible results, and that exactly one of those choices apply. def foo(): ... if condition: ...... return 1 ... else ...... return 2 I would expect this to be the same as the "# with multiple exits" with a possibly very minor increase in clarity. I have used several IDE's that let you mask out chunks of code based on their indentation level. Sometimes I use that concept as an estimate of readability (minimizing how much a reader has to look at to understand the structure of the code. Contrast my multiple-exit with the one I quoted: def foo(): def foo(): if (condition) if (condition0 [...] [...] else: return 2 [...] If someone was only reading lines indented once, and skimming the rest as details unimportant to the structure, he might have a misleading assumption about the return value in the code on the right. On the left, he would look be forced to look one level deeper to find the return's, and find two outcomes. This same level-of-indentation measure also argues against break and continue, since statements would also be further indented, which might make them easier to miss. But, in any case, I do not mind multiple return's clearly marked with if/else. On the other hand, I still recommend against return statements inside of loops. It shouldn't be hard to find the appropriate way to leave the loop, and return afterwards. My position is also in line with one of my hard and fast rules for early programming students: Rule: If the operation does not repeat, do not put it into a loop. I don't even make an exception for return statements. And my justification for not making an exception is not only to simply the pedagogy (absolutism is much easier to explain), but also goes along with the other maxim: The code should be obvious! There are just so many times when I see students expect a return statement within a loop to be able to return more than one value. It happens both when reading code, and when writing code, and even happens for upperclassmen, who should know better. Clearly, if such a misinterpretation is prevalent, the code is not obvious. And that's even ignoring the benefit of being able to set a breakpoint at the very bottom of a function when debugging. Roger Christman Pennsylvania State University On Thu, Oct 12, 2017 12:26 AM, Steve D'Aprano wrote: > > >Message: 3 >Date: Thu, 12 Oct 2017 12:26:07 +1100 >From: Steve D'Aprano >To: python-list at python.org >Subject: Re: Looping [was Re: Python and the need for speed] >Message-ID: <59dec4b0$0$14935$b1db1813$d948b532 at news.astraweb.com> >Content-Type: text/plain; charset=utf-8 > > >But from the point of view of the subroutines, the rule "one exit" is >like the >rule "no break" for loops: it makes the code more complex and less >efficient. >If you're done, you're done, and you might as well return out of the function >rather than write boilerplate code to pad it out until you get to the very >end. With only a single condition to test, there's not much difference >between the two: > ># with single exit # with multiple exits >def foo(): def foo(): > if condition: if condition: > result = 1 return 1 > else: return 2 > result = 2 > return result > > >but as the number of decision points increase, the complexity required to keep >a single exit also increases. > > > From jon+usenet at unequivocal.eu Thu Oct 12 13:15:14 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Thu, 12 Oct 2017 17:15:14 -0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> <87o9pcjw50.fsf@bsb.me.uk> <87fuaojr46.fsf@bsb.me.uk> Message-ID: On 2017-10-12, Ben Bacarisse wrote: > I see. If I'm reading this right, the app requests are passed through > to another server -- uWSGI. Yes. It doesn't have to be uWSGI; it could be gunicorn, or you could probably use Apache's mod_fcgid. As a last resort you could use CGI, which wouldn't involve any long-running processes, which has the benefit of not requiring any special support from your host but the disadvantage of most likely being very slow indeed. > How does this typically work on low-cost hosting? I may be able to set > up the ProxyPass locally (i.e. in .htaccess) but I won't be able to > write /etc/uwsgi/apps-available/appname.ini. Maybe there are a locally > defined .ini files that uwsgi reads? You need to choose a host that supports one of the relevant systems mentioned above. If you already have a host then it's possible they already do, otherwise you may need to choose another. From rosuav at gmail.com Thu Oct 12 13:20:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Oct 2017 04:20:12 +1100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) Message-ID: On Fri, Oct 13, 2017 at 1:09 AM, Ben Bacarisse wrote: > Chris Angelico writes: > >> On Thu, Oct 12, 2017 at 7:32 PM, Thomas Jollans wrote: >>> On 2017-10-12 07:31, Chris Angelico wrote: >>>> On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >>>>> Provided some early part of the URL is handled by PHP, the rest of the >>>>> URL path is provided to PHP in $_SERVER["PATH_INFO"]. >>>> >>>> Is it possible to do that without having ".php" visible in the path? >>> >>> Just like with Python-based frameworks, this requires a few lines of web >>> server configuration. >>> >>> On Apache, you might use mod_wsgi to tell the server how to run the code >>> in one case, and a combination of mod_php and mod_rewrite in the other. >>> If you're using FastCGI with nginx or lighttpd, I believe the >>> configuration would look pretty similar in both cases. >>> >>> Then again, I don't do much web programming any more and generally stay >>> away from PHP, so I may be misremembering. >> >> Normally, with a Python-based framework, you don't need _any_ web >> server configuration. You simply define your URL routing within the >> Python code. The only thing the web server needs to know is where to >> find the web app, and that's sufficiently standard that it can be done >> off-the-shelf; for instance, you push your code to Heroku, and they >> set everything up to pass requests to your app. Not possible with PHP, >> since you need *custom* web server config to manage your rewrite >> rules. > > That's at odds with what I've read online which admittedly may be all > junk. I wanted to try Flask so I installed the Ubuntu packages but then > got stuck on a huge document that suggested I needed to install things > called Nginx and Gunicorn. You've now mentioned another: Heroku. I'm > sure the complex instructions I found are not really required -- it was > probably just the usual "this is what I did so this is how it's done" > document, but I'm having trouble finding the simpler way to do it. > > Since no web server configuration is needed (I have a working Apache > installation that mirrors, as closely as possible, what my hosting > provider uses) it should be relatively easy. Can you tell me, or can > you point me to a resource that tells me, where to put the app? I don't > yet know what "push your code to Heroku" means. I abbreviated that down to nothing, but since you ask, here's a really REALLY simple run-down of how to use Heroku: 1) Create a git repository to manage your code. (You should be doing this anyway.) 2) Install the Heroku CLI for your platform 3) Run "heroku login" and enter your credentials (once) 4) Run "heroku create" from your project repo to generate a URL to use for the project 5) Ensure that you list all your requirements in a file called "requirements.txt". Heroku uses this to (a) recognize that it's a Python project, and (b) install your dependencies, with "pip install -r requirements.txt". 6) Tell Heroku how to find your main file by making a file called "Procfile" 7) Run: "git push heroku master" There are other ways to do things (notably, Heroku will work with GitHub and Travis to do CI/CD just by pushing code to the master branch on GitHub - Travis will run your tests and then push to Heroku for you), but this is about the simplest it gets. Yes, there are a good few steps there, but most of them are simple one-offs, or are things you should do anyway. ChrisA From python at mrabarnett.plus.com Thu Oct 12 13:28:18 2017 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 12 Oct 2017 18:28:18 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <1507776441l.38207678l.0l@psu.edu> References: <1507776441l.38207678l.0l@psu.edu> Message-ID: <1431305e-3855-920d-da2f-e662a029bdc6@mrabarnett.plus.com> On 2017-10-12 03:47, ROGER GRAYDON CHRISTMAN wrote: > Actually, FORTRAN and COBOL and Algol (for its control structures) > Trying to support both of the first two was entertaining -- > when you declared a variable, it wasn't enough to say it was an Integer: > you had to also declare whether it was represented in Binary or Decimal, > and also specify the desired precision. > > The IBM/360 architecture supported both binary and decimal integers, > where the decimals were stored as BCD nybbles of arbitrary fixed length > (as opposed to binary integers matching the machine word size) > > The world migrated away from PL/I back in those days because of > the one-size fits none consequences of trying to do everything. > So I always find myself looking askance when language designers > try to repeat the exercise. > > You know, like designing a largely-interpreted object-oriented > language with libraries supporting a functional programming style. > I think I've seen a language like that somewhere around this forum. > > But I like it anyway > I think the difference is that those other languages tried to be "complete" closed languages. Python, on the other hand, doesn't try to do everything itself, and it's open, so you can add functionality written in other languages and call external programs that already exist. [snip] From ben.usenet at bsb.me.uk Thu Oct 12 18:30:21 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 12 Oct 2017 23:30:21 +0100 Subject: Lies in education [was Re: The "loop and a half"] References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> <87vajlkwvx.fsf@bsb.me.uk> <87inflkvs2.fsf@bsb.me.uk> <5b44ad8a-11b4-0f2b-460f-91422101a1e0@tjol.eu> <87o9pcjw50.fsf@bsb.me.uk> <87fuaojr46.fsf@bsb.me.uk> Message-ID: <87mv4whude.fsf@bsb.me.uk> Jon Ribbens writes: > On 2017-10-12, Ben Bacarisse wrote: >> I see. If I'm reading this right, the app requests are passed through >> to another server -- uWSGI. > > Yes. It doesn't have to be uWSGI; it could be gunicorn, or you could > probably use Apache's mod_fcgid. As a last resort you could use CGI, > which wouldn't involve any long-running processes, which has the > benefit of not requiring any special support from your host but the > disadvantage of most likely being very slow indeed. > >> How does this typically work on low-cost hosting? I may be able to set >> up the ProxyPass locally (i.e. in .htaccess) but I won't be able to >> write /etc/uwsgi/apps-available/appname.ini. Maybe there are a locally >> defined .ini files that uwsgi reads? > > You need to choose a host that supports one of the relevant systems > mentioned above. If you already have a host then it's possible they > already do, otherwise you may need to choose another. Ah, thanks. You've cleared up some of miasma of terms that seems to surround the various Python-for-the-web options. I'd like to try it, but not enough to switch hosting and, probably, spend more money. -- Ben. From alberto at metapensiero.it Thu Oct 12 19:07:38 2017 From: alberto at metapensiero.it (Alberto Berti) Date: Fri, 13 Oct 2017 01:07:38 +0200 Subject: Cooperative class tree filtering Message-ID: <87376o2ced.fsf@ender.lizardnet> Hello, suppose you have you have three classes A, B, C where C is a subclass of B and B of A. They provide a method with signature `filter(element)`. It is called by an external function that collects the elements and then to an instance of the classes A, B and C is given a chance to filter these elements, one at a time. So the `filter(element)` method returns `True` if the element has to be included in the final set and `False` if it doesn't, easy peasy. (This is a simplified example of a real use case). What I want to have is that if an instance of class C has doesn't know what to return about an element, it calls `super()` to return the value from the same method implemented in its superclass, and so on up on the tree. To show it in code: class A: def filter(self, element): # the default implementation always returns True return True class B(A): def filter(self, element): if element == 'foo': return True elif element == 'bar': return False else: return super().filter(element) class C(B): def filter(self, element): if element == 'bar': return True else: return super().filter(element) def collect_elements(instance): "A semplified element collect function" all_elts = {'foo', 'bar', 'baz', 'zoo'} filtered_elts = set(el for el in all_elts if instance.filter(el)) return filtered_elts b = B() c = C() print(collect_elements(b)) print(collect_elements(c)) which run gives the following result: >>> {'foo', 'zoo', 'baz'} {'bar', 'foo', 'zoo', 'baz'} Now, what i ideally want is to get rid of that super() call at the end of the methods in classes B and c and to code that "fallback to what my superclass says" coded into A's implementation, where it kicks in if the method in the target instance returns None. So to write it in code, I would like some like: class A: def _filter_impl(self, element): # the default implementation always returns True return True def filter(self, element): # here a logic that if self._filter_impl(element) returns # None, it defaults to super(type(self), self)._filter_impl(element) # until A._filter_impl() is reached pass class B(A): def _filter_impl(self, element): if element == 'foo': return True elif element == 'bar': return False class C(B): def filter(self, element): if element == 'bar': return True I've tried some variants of the 'super()' trick, that sometimes seems to change behavior if it's written like that or like super(type(self), self) in no clear (to me, and I failed to find extensive doc on super()'s behavior) way, with things that stop working if mixins are involved (even if the mixins do not reimplement the methods involved here). Eventually i ended implementing a decorator: from functools import partial, wraps class delegate_super: """A method decorator that delegates part of the computation to the same method on the ancestor class.""" _name = None _owner = None def __init__(self, meth): self._meth = meth @wraps(meth) def wrapper(*args, **kwargs): return self.delegator(*args, **kwargs) self.wrapper = wrapper def __get__(self, instance, owner): return partial(self.wrapper, instance) def __set_name__(self, owner, name): self._owner = owner self._name = name def delegator(self, instance, *args, **kwargs): result = self._meth(instance, *args, **kwargs) if result is None: result = getattr(super(self._owner, instance), self._name)( *args, **kwargs) return result class A: def filter(self, element): # the default implementation always returns True return True class B(A): @delegate_super def filter(self, element): if element == 'foo': return True elif element == 'bar': return False class C(B): @delegate_super def filter(self, element): if element == 'bar': return True else: return super().filter(element) def collect_elements(instance): "A semplified element collect function" all_elts = {'foo', 'bar', 'baz', 'zoo'} filtered_elts = set(el for el in all_elts if instance.filter(el)) return filtered_elts b = B() c = C() print(collect_elements(b)) print(collect_elements(c)) which has the same result as before: >>> {'foo', 'zoo', 'baz'} {'bar', 'foo', 'zoo', 'baz'} I would really like to find a way to do this that doesn't involve decorating the methods in A subclasses to free the final developer to remember to import the decorator and apply it, just like I don't want him (probably me six months from now) to have to remember to add an `else: super()...` to its computation... Has anyone done this before and has any suggestion about a better way to do it? Am I getting it wrong? cheers, Alberto From ben.usenet at bsb.me.uk Thu Oct 12 19:14:11 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 13 Oct 2017 00:14:11 +0100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: Message-ID: <87infkhscc.fsf@bsb.me.uk> Chris Angelico writes: > On Fri, Oct 13, 2017 at 1:09 AM, Ben Bacarisse wrote: >> Chris Angelico writes: >> >>> On Thu, Oct 12, 2017 at 7:32 PM, Thomas Jollans wrote: >>>> On 2017-10-12 07:31, Chris Angelico wrote: >>>>> On Thu, Oct 12, 2017 at 12:19 PM, Ben Bacarisse wrote: >>>>>> Provided some early part of the URL is handled by PHP, the rest of the >>>>>> URL path is provided to PHP in $_SERVER["PATH_INFO"]. >>>>> >>>>> Is it possible to do that without having ".php" visible in the path? >>>> >>>> Just like with Python-based frameworks, this requires a few lines of web >>>> server configuration. >>>> >>>> On Apache, you might use mod_wsgi to tell the server how to run the code >>>> in one case, and a combination of mod_php and mod_rewrite in the other. >>>> If you're using FastCGI with nginx or lighttpd, I believe the >>>> configuration would look pretty similar in both cases. >>>> >>>> Then again, I don't do much web programming any more and generally stay >>>> away from PHP, so I may be misremembering. >>> >>> Normally, with a Python-based framework, you don't need _any_ web >>> server configuration. You simply define your URL routing within the >>> Python code. The only thing the web server needs to know is where to >>> find the web app, and that's sufficiently standard that it can be done >>> off-the-shelf; for instance, you push your code to Heroku, and they >>> set everything up to pass requests to your app. Not possible with PHP, >>> since you need *custom* web server config to manage your rewrite >>> rules. >> >> That's at odds with what I've read online which admittedly may be all >> junk. I wanted to try Flask so I installed the Ubuntu packages but then >> got stuck on a huge document that suggested I needed to install things >> called Nginx and Gunicorn. You've now mentioned another: Heroku. I'm >> sure the complex instructions I found are not really required -- it was >> probably just the usual "this is what I did so this is how it's done" >> document, but I'm having trouble finding the simpler way to do it. >> >> Since no web server configuration is needed (I have a working Apache >> installation that mirrors, as closely as possible, what my hosting >> provider uses) it should be relatively easy. Can you tell me, or can >> you point me to a resource that tells me, where to put the app? I don't >> yet know what "push your code to Heroku" means. > > I abbreviated that down to nothing, but since you ask, here's a really > REALLY simple run-down of how to use Heroku: I think I see what you mean now. You meant no configuration is needed because you use (or buy?) a cloud service that's all set up for it already? >From this and other posts I think the position is that I do need to do some server configuration (and essentially install a proxy server) to run Python web applications on my typical Apache set-up. And I would then have to shop around for suitable hosting that is already set up for running them. Thanks. That's not quite what I was after but it's good to know how to do that should I want to that later. -- Ben. From steve+python at pearwood.info Thu Oct 12 19:22:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 13 Oct 2017 10:22:56 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> Message-ID: <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 02:06 am, Grant Edwards wrote: > It sure was an education the first I wrote C code for > a machine where > > 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof > double > > All were 32 bits. Does that imply that on that machine 1 byte = 32 bits? I don't suppose you remember the name of the machine do you? > Writing protocol code that dealt with the outside world via a serial > port was _painful_. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From totah1234 at gmail.com Thu Oct 12 19:55:00 2017 From: totah1234 at gmail.com (totah1234 at gmail.com) Date: Thu, 12 Oct 2017 16:55:00 -0700 (PDT) Subject: Solution Manual Test Bank for Financial and Managerial Accounting for MBAs 5th Edition by Easton In-Reply-To: <38202c95-45e3-435d-b4f9-1099e71de542@googlegroups.com> References: <38202c95-45e3-435d-b4f9-1099e71de542@googlegroups.com> Message-ID: <72cec506-c0f8-41a9-86dd-5477bfbb9975@googlegroups.com> Greetings Students, We do have Solution Manuals and Test Bank for FINANCIAL AND MANAGERIAL ACCOUNTING FOR MBAs 5TH EDITION BY EASTON at reasonable price. You can get above mentioned resources by sending email to pro.fast(@)hotmail(dot)com Send your order queries at PRO.FAST(@)HOTMAIL(DOT)COM Below are details given for this book Book Name: FINANCIAL AND MANAGERIAL ACCOUNTING FOR MBAs Authors: Peter D. Easton, Robert F. Halsey, Mary Lea McAnally, Al L. Hartgraves, Wayne J. Morse Edition: 5th E ISBN-10: 1618532324 ISBN-13: 9781618532329 Product Format: MS Word Total Modules: 25 Please mention complete details for your book so we could send you sample accordingly. Best Regards, P.S : Please do not post your reply here. We do not monitor queries here. Simply send us an email directly to PRO.FAST (@) HOTMAIL (DOT) COM I tried to send an email to the email you provided but it's not working. I need the test bank as soon as possible. Please contact me on my email:totah1234 at gmail. From alberto at metapensiero.it Thu Oct 12 19:56:37 2017 From: alberto at metapensiero.it (Alberto Berti) Date: Fri, 13 Oct 2017 01:56:37 +0200 Subject: Cooperative class tree filtering References: <87376o2ced.fsf@ender.lizardnet> Message-ID: <87y3of2a4q.fsf@ender.lizardnet> Sorry, i've made a mistake in the second C body, it's written like: >>>>> "me" == Alberto Berti writes: me> I've tried some variants of the 'super()' trick, that sometimes seems to me> change behavior if it's written like that or like super(type(self), me> self) in no clear (to me, and I failed to find extensive doc on me> super()'s behavior) way, with things that stop working if mixins are me> involved (even if the mixins do not reimplement the methods involved me> here). Eventually i ended implementing a decorator: me> from functools import partial, wraps me> class delegate_super: me> """A method decorator that delegates part of the computation to the same me> method on the ancestor class.""" me> _name = None me> _owner = None me> def __init__(self, meth): me> self._meth = meth me> @wraps(meth) me> def wrapper(*args, **kwargs): me> return self.delegator(*args, **kwargs) me> self.wrapper = wrapper me> def __get__(self, instance, owner): me> return partial(self.wrapper, instance) me> def __set_name__(self, owner, name): me> self._owner = owner me> self._name = name me> def delegator(self, instance, *args, **kwargs): me> result = self._meth(instance, *args, **kwargs) me> if result is None: me> result = getattr(super(self._owner, instance), self._name)( me> *args, **kwargs) me> return result me> class A: me> def filter(self, element): me> # the default implementation always returns True me> return True me> class B(A): me> @delegate_super me> def filter(self, element): me> if element == 'foo': me> return True me> elif element == 'bar': me> return False me> class C(B): me> @delegate_super me> def filter(self, element): me> if element == 'bar': me> return True me> else: me> return super().filter(element) The correct version is: class C(B): @delegate_super def filter(self, element): if element == 'bar': return True From rosuav at gmail.com Thu Oct 12 20:26:13 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Oct 2017 11:26:13 +1100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) In-Reply-To: <87infkhscc.fsf@bsb.me.uk> References: <87infkhscc.fsf@bsb.me.uk> Message-ID: On Fri, Oct 13, 2017 at 10:14 AM, Ben Bacarisse wrote: > Chris Angelico writes: >> I abbreviated that down to nothing, but since you ask, here's a really >> REALLY simple run-down of how to use Heroku: > > I think I see what you mean now. You meant no configuration is needed > because you use (or buy?) a cloud service that's all set up for it > already? Correct - because the setup needed is completely generic. > From this and other posts I think the position is that I do need to do > some server configuration (and essentially install a proxy server) to > run Python web applications on my typical Apache set-up. And I would > then have to shop around for suitable hosting that is already set up for > running them. > > > > Thanks. That's not quite what I was after but it's good to know how to > do that should I want to that later. Yep, it's not too hard. And that's why it's cleaner to work with Python than PHP. To use custom URL routing in PHP, you have to use custom server rules; to use custom URL routing in Python, you use "@app.route(...)" lines inside your app, and perfectly standard server rules. ChrisA From grant.b.edwards at gmail.com Thu Oct 12 20:48:42 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 13 Oct 2017 00:48:42 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-12, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 02:06 am, Grant Edwards wrote: > >> It sure was an education the first I wrote C code for >> a machine where >> >> 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof >> double >> >> All were 32 bits. > > > Does that imply that on that machine 1 byte = 32 bits? Maybe. That depends on what you mean by "byte". There is no such thing as a "byte" in C. The smallest addressable unit of memory in C is a "char". Nothing in C says that incrementing an address by 1 gets you to the next 8-bit chunk of memory. According to the "Byte" Wikipedia article: The size of the byte has historically been hardware dependent and no definitive standards existed that mandated the size -- byte-sizes from 1 to 48 bits are known to have been used in the past. The IEEE standards use the word "octet" to refer to a an 8-bit chunk of memory. When working with an architecture with a 32-bit char, you didn't use the word "byte" if you wanted to avoid confusion. > I don't suppose you remember the name of the machine do you? Texas Instruments TMS32C40. It's pretty common on DSPs to have only a single size for all datatypes. The 'C40 as pretty high-end -- it had floating point. Though I had to write routines to convert between IEE-754 and the native 32-bit format when I wanted to exchange floating point data with the outside world. :) -- Grant From sohcahtoa82 at gmail.com Thu Oct 12 21:16:37 2017 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Thu, 12 Oct 2017 18:16:37 -0700 (PDT) Subject: Want to write a python code for sending and receiving frames over wifi/wlan0 using python In-Reply-To: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> References: <8d4cd323-a84f-4b2d-a6fc-08d887a00844@googlegroups.com> Message-ID: On Thursday, October 12, 2017 at 1:08:55 AM UTC-7, T Obulesu wrote: > Hello all, I want to send some frames defined by me{Example, [0x45,0x43,0x32]} to the raspberry pi from any macine(Desktop/Laptop/other raspberry pi). But I want to send those frames over wifi or use wlan0 using python Any suggestions? Check out a Python library called scapy. It's used for raw packet forgery. But as others mentioned, that kind of activity will require you to run as root. Also, sending raw packets over Wifi specifically usually requires you to switch wlan0 to "monitor" mode, essentially making it unable to do anything else with it while your script is running. Also, not every Wifi adapter even supports entering monitor mode. From grant.b.edwards at gmail.com Thu Oct 12 21:31:51 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 13 Oct 2017 01:31:51 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13, Stefan Ram wrote: > Grant Edwards writes: There is no such >>thing as a "byte" in C. > > ?3.6 > > 1 byte > > addressable unit of data storage large enough to hold > any member of the basic character set of the execution > environment? > > ISO C standard Ah, I had forgotten about that paragraph. Now that I see the phrase "large enough to hold any member of the basic character set of the execution environment", I'm pretty sure I knew that at some point in the past. That means that in the context of the ISO C standard, on that architecture, 1 byte is 32 bits. -- Grant From ctippur at gmail.com Fri Oct 13 00:19:58 2017 From: ctippur at gmail.com (Frustrated learner) Date: Thu, 12 Oct 2017 21:19:58 -0700 (PDT) Subject: Running flask on AWS SAM Message-ID: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> Hello, I have a flask based application which i am able to run locally. $ python swagger_server/app.py * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) I am trying to port this over to aws. I have all the dependencies and code organized in the same folder. Here is the transaction and error: $ SERVER_NAME=0.0.0.0:3000 sam local start-api 2017/10/12 21:02:20 Connected to Docker 1.32 2017/10/12 21:02:20 Fetching lambci/lambda:python3.6 image for python3.6 runtime... python3.6: Pulling from lambci/lambda Digest: sha256:c77ea3986c471c2f93dfa2a86492e6306989505073795da3b22defa2b10846a6 Status: Image is up to date for lambci/lambda:python3.6 Mounting swagger_server.app.app (python3.6) at http://127.0.0.1:3000/current [GET] Mounting static files from public at / You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template. 2017/10/12 21:02:30 Invoking swagger_server.app.app (python3.6) 2017/10/12 21:02:30 Decompressing /Users/shekartippur/playground/cyclopscloud/cyclopsglobal/swagger_server.zip START RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 Version: $LATEST 'SERVER_NAME': KeyError Traceback (most recent call last): File "/var/task/connexion/apps/abstract.py", line 266, in __call__ return self.app(environ, start_response) File "/var/task/flask/app.py", line 1997, in __call__ return self.wsgi_app(environ, start_response) File "/var/task/flask/app.py", line 1977, in wsgi_app ctx = self.request_context(environ) File "/var/task/flask/app.py", line 1938, in request_context return RequestContext(self, environ) File "/var/task/flask/ctx.py", line 242, in __init__ self.url_adapter = app.create_url_adapter(self.request) File "/var/task/flask/app.py", line 1765, in create_url_adapter server_name=self.config['SERVER_NAME']) File "/var/task/werkzeug/routing.py", line 1299, in bind_to_environ wsgi_server_name = environ['SERVER_NAME'] KeyError: 'SERVER_NAME' END RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 REPORT RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 Duration: 3257 ms Billed Duration: 3300 ms Memory Size: 0 MB Max Memory Used: 38 MB I looked up the error and dint find much help. Wondering what I could be missing. From formisc at gmail.com Fri Oct 13 00:23:11 2017 From: formisc at gmail.com (Andrew Z) Date: Fri, 13 Oct 2017 00:23:11 -0400 Subject: unorderable types: list() > int() Message-ID: Hello, pos = {"CLown":10,"BArbie":20} I want to return integer (10) for the keyword that starts with "CL" cl_ = [v for k, v in pos.items() if k.startswith('CL')] cl_pos = cl_[0] if cl_pos > 0: blah.. There are 2 issues with the above: a. ugly - cl_pos = cl_ [0] . I was thinking something like cl_ = [v for k, v in pos.items() if k.startswith('CL')][0] but that is too much for the eyes - in 2 weeks i'll rewrite this into something i can understand without banging against the desk. So what can be a better approach here ? b. in "cl_pos > 0:, cl_pos apparently is still a list. Though the run in a python console has no issues and report cl_pos as int. Appreciate. From greg.ewing at canterbury.ac.nz Fri Oct 13 00:37:40 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 17:37:40 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: bartc wrote: > (2) Declare data to be put into read-only memory as you say. That's fine > with a 'const int * p', but what about a 'int * const p'? If the compiler can tell where p is initially pointing, it could put the pointer in read-only memory. Probably unlikely to happen in real code, though. -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 13 00:50:45 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 17:50:45 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <9BuDB.15425$uS.4321@fx24.am4> Message-ID: Stefan Ram wrote: > void > i_know_i_was_passed_a_pointer_to_an_array_and_how_many_elements_are_in_it > ( char( *a )[ 4 ] ) > { for( int i = 0; i < 4; ++i ) > putchar( ( *a )[ i ]); } Only because you've statically made the array size part of the type. Your original example didn't do that; presumably it was intended to accept arrays of any size and cope with them dynamically. That's usually what you want, and the way you do that in C is to pass a pointer to the first element and convey the size separately. So the kind of declaration you used above is hardly ever seen in idiomatic C code. (I've *never* seen it done in real life.) -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 13 01:09:54 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 18:09:54 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <87shepddu6.fsf@elektro.pacujo.net> <87lgkhd70j.fsf@elektro.pacujo.net> <59dec237$0$14946$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> Message-ID: Grant Edwards wrote: > It sure was an education the first I wrote C code for > a machine where > > 1 == sizeof char == sizeof int == sizeof long == sizeof float == sizeof double > > All were 32 bits. Unicode-ready -- way ahead of its time! -- Greg From steve+python at pearwood.info Fri Oct 13 01:16:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 13 Oct 2017 16:16:27 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: > If the compiler can tell where p is initially pointing, it could > put the pointer in read-only memory. If it's read-only, how can the compiler write to it? (I come from the days when ROM was actual ROM, burned in at the factory.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 13 01:22:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 13 Oct 2017 16:22:38 +1100 Subject: unorderable types: list() > int() References: Message-ID: <59e04da0$0$14938$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 03:23 pm, Andrew Z wrote: > Hello, > pos = {"CLown":10,"BArbie":20} > I want to return integer (10) for the keyword that starts with "CL" > > > cl_ = [v for k, v in pos.items() if k.startswith('CL')] > cl_pos = cl_[0] > if cl_pos > 0: > > blah.. > > > There are 2 issues with the above: > a. ugly - cl_pos = cl_ [0] . I was thinking something like > > > cl_ = [v for k, v in pos.items() if k.startswith('CL')][0] > > but that is too much for the eyes - in 2 weeks i'll rewrite this into > something i can understand without banging against the desk. > So what can be a better approach here ? What you wrote is perfectly fine. But if you don't like it, then use a temporary variable, as you do above. > b. in "cl_pos > 0:, cl_pos apparently is still a list. Though the run in a > python console has no issues and report cl_pos as int. It isn't a list. What makes you think it is? py> pos = {"CLown":10,"BArbie":20} py> cl_ = [v for k, v in pos.items() if k.startswith('CL')] py> cl_pos = cl_[0] py> type(cl_pos) py> cl_pos 10 py> cl_pos > 0 True You probably wrote "cl_ > 0" by mistake. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Oct 13 01:24:31 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Oct 2017 16:24:31 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 13, 2017 at 4:16 PM, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: > >> If the compiler can tell where p is initially pointing, it could >> put the pointer in read-only memory. > > If it's read-only, how can the compiler write to it? > > > (I come from the days when ROM was actual ROM, burned in at the factory.) Code pages (nothing to do with eight-bit character sets, I mean memory pages containing program code) are often - and should always be - read-only by default. The compiler can put constants into the code segment and reference them that way. ChrisA From greg.ewing at canterbury.ac.nz Fri Oct 13 01:28:24 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 18:28:24 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: Grant Edwards wrote: > On 2017-10-13, Stefan Ram wrote: > >> 1 byte >> >> addressable unit of data storage large enough to hold >> any member of the basic character set of the execution >> environment? >> >> ISO C standard Hmmm. So an architecture with memory addressed in octets and Unicode as the basic character set would have a char of 8 bits and a byte of 32 bits? Not only does "byte" not always mean "8 bits", but "char" isn't always short for "character"... -- Greg From rosuav at gmail.com Fri Oct 13 01:36:10 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Oct 2017 16:36:10 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 13, 2017 at 4:28 PM, Gregory Ewing wrote: > Grant Edwards wrote: >> >> On 2017-10-13, Stefan Ram wrote: >> >>> 1 byte >>> >>> addressable unit of data storage large enough to hold >>> any member of the basic character set of the execution >>> environment? >>> >>> ISO C standard > > > Hmmm. So an architecture with memory addressed in octets > and Unicode as the basic character set would have a > char of 8 bits and a byte of 32 bits? > > Not only does "byte" not always mean "8 bits", but > "char" isn't always short for "character"... Certainly not. A byte would be 21 bits! Seriously though, I don't think anyone would design hardware like this. But I'd love to see what happens. ChrisA From vincent.vande.vyvre at telenet.be Fri Oct 13 01:46:06 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 13 Oct 2017 07:46:06 +0200 Subject: Return str to a callback raise a segfault if used in string formating Message-ID: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> Simplified code: ---%<------------------------------------------ class Foo: ??? tasks = [] ??? def process(self, *args): ??????? # process with the file ??????? # save result on disk with a new name ??????? self.out_filename.write(.....) ??? def run_multitasks(self): ??????? while self.tasks: ??????????? t = Thread(target=process, args=self.tasks.pop(0)) ??????????? t.start() ??????????? t.join() ??????????? self.callback(self.out_filename) ??? def add_task(self, fname, callback): ??????? self.tasks.append(fname) ??????? self.callback = callback ??????? if len(self.tasks) == 1: ??????????? self.run_multitasks() def slot(filename): ??? print(filename)???????????????? # Works ??? print("%s created" % filename)? # Segfault ! Files = [list of files] foo = Foo() for f in Files: ??? foo.add_task(f, slot) ---%<------------------------------------------ If I place self.callback() at the end of the func process that doesn't change anything. Regards, Vincent. From greg.ewing at canterbury.ac.nz Fri Oct 13 02:16:04 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 19:16:04 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: > >>If the compiler can tell where p is initially pointing, it could >>put the pointer in read-only memory. > > If it's read-only, how can the compiler write to it? > > (I come from the days when ROM was actual ROM, burned in at the factory.) So, the factory is allowed to write to it. Possibly it's writing data that came from... a compiler? A memory that couldn't be written to at all, ever, would be a bit useless! -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 13 02:22:51 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 13 Oct 2017 19:22:51 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: Chris Angelico wrote: > Certainly not. A byte would be 21 bits! Only if 21 bits were *also* an addressable unit of storage in addition to octets. That would be an interesting architecture indeed. If you really wanted that, it might be easier just to make the memory bit-addressable. In which case a char would be 1 bit! I believe Burroughs built a bit-addressable machine at one point. It was meant to be user-microprogrammable, so you designed an instruction set for what you wanted to do, and then programmed in that. -- Greg From auriocus at gmx.de Fri Oct 13 02:59:58 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 13 Oct 2017 08:59:58 +0200 Subject: OT Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <87po9tmn6r.fsf@bsb.me.uk> <87h8v5mhjf.fsf@bsb.me.uk> Message-ID: Am 12.10.17 um 01:33 schrieb Chris Angelico: > Yeah. The trouble is that this is a really REALLY bad way to design > something. Have you seen a city that grew one house at a time, and had > streets added to service those houses? Not good. I tend to disagree. Many European villages have been built that way, and they are lovely for tourists: https://en.wikipedia.org/wiki/Angerdorf :) Christian From rosuav at gmail.com Fri Oct 13 03:23:04 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 13 Oct 2017 18:23:04 +1100 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> Message-ID: On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre wrote: > Simplified code: > > ---%<------------------------------------------ > class Foo: > tasks = [] > def process(self, *args): > # process with the file > # save result on disk with a new name > self.out_filename.write(.....) > > def run_multitasks(self): > while self.tasks: > t = Thread(target=process, args=self.tasks.pop(0)) > t.start() > t.join() > self.callback(self.out_filename) > > def add_task(self, fname, callback): > self.tasks.append(fname) > self.callback = callback > if len(self.tasks) == 1: > self.run_multitasks() > > def slot(filename): > print(filename) # Works > print("%s created" % filename) # Segfault ! > > Files = [list of files] > foo = Foo() > for f in Files: > foo.add_task(f, slot) > ---%<------------------------------------------ > > If I place self.callback() at the end of the func process that doesn't > change anything. First off, exactly what version of Python are you running this under? With a segfault, you need to be pretty specific - platform, version, word size, as much as you can gather. Secondly: Can you create a version of this that doesn't comment out part of the work? If you can make a script where, any time you run it, Python segfaults, that would be extremely helpful. Your code currently looks a bit weird. You create a thread, start it, and then immediately wait for it (join()). When you add a task, if it's the first task you've added, you process tasks, thus removing that task. So the class isn't actually doing anything, and logically, you could simply process the files directly in the loop. I'm guessing that something in there (probably the threading) is causing the crash, but without a complete and testable demo, it's hard to be sure. ChrisA From __peter__ at web.de Fri Oct 13 03:31:12 2017 From: __peter__ at web.de (Peter Otten) Date: Fri, 13 Oct 2017 09:31:12 +0200 Subject: unorderable types: list() > int() References: Message-ID: Andrew Z wrote: > Hello, > pos = {"CLown":10,"BArbie":20} > I want to return integer (10) for the keyword that starts with "CL" > > > cl_ = [v for k, v in pos.items() if k.startswith('CL')] > cl_pos = cl_[0] > if cl_pos > 0: > > blah.. > > > There are 2 issues with the above: > a. ugly - cl_pos = cl_ [0] . I was thinking something like There's another issue: what do you want to do if there is more than one match or no match at all? If you are OK with a ValueError in both cases you can rewrite > cl_ = [v for k, v in pos.items() if k.startswith('CL')][0] [match] = (v for k, v in pos.items() if k.startswith("CL")) If the values are orderable you can standardise on the minimum or maximum match = min(v for k, v in pos.items() if k.startswith("CL")) or if you always want an arbitrary match you can pick the first one explicitly: match = next(v for k, v in pos.items() if k.startswith("CL")) This will raise a StopIteration if there are no matches. > but that is too much for the eyes - in 2 weeks i'll rewrite this into > something i can understand without banging against the desk. > So what can be a better approach here ? > > > b. in "cl_pos > 0:, cl_pos apparently is still a list. Though the run in a > python console has no issues and report cl_pos as int. > > > Appreciate. From vincent.vande.vyvre at telenet.be Fri Oct 13 05:07:54 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 13 Oct 2017 11:07:54 +0200 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> Message-ID: Le 13/10/17 ? 09:23, Chris Angelico a ?crit?: > On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre > wrote: >> Simplified code: >> >> ---%<------------------------------------------ >> ... >> ---%<------------------------------------------ >> >> If I place self.callback() at the end of the func process that doesn't >> change anything. > First off, exactly what version of Python are you running this under? > With a segfault, you need to be pretty specific - platform, version, > word size, as much as you can gather. > > Secondly: Can you create a version of this that doesn't comment out > part of the work? If you can make a script where, any time you run it, > Python segfaults, that would be extremely helpful. > > Your code currently looks a bit weird. You create a thread, start it, > and then immediately wait for it (join()). When you add a task, if > it's the first task you've added, you process tasks, thus removing > that task. So the class isn't actually doing anything, and logically, > you could simply process the files directly in the loop. I'm guessing > that something in there (probably the threading) is causing the crash, > but without a complete and testable demo, it's hard to be sure. > > ChrisA I'm using 3.6.1 in venv It's not easy to write a runnable code because in this example the method process() is too simplified and the code don't reproduce the segfault. The code is an image processing in two parts: the processing himself in CPython and an api in Python. I have written the two parts. In the real code this is the equivalent of the method process(): ---------------------------------------------- ??? def unraw(self, index=0, dest=""): ??????? """Run the demosaication process. ??????? Args: ??????? index -- the index of the image or "all" if there's more than one image ???????????????? into the file ??????? dest -- the absolute file name for the image decoded.? If a file with ??????????????? the same name already exists, it will be overwritten. ??????? Raise IndexError if index >= self.image_count ??????? """ ??????? if not self.is_raw: ??????????? fname = os.path.basename(self.filename) ??????????? raise TypeError("RAW file %s not supported!" % fname) ??????? if index >= self.data["image_count"]: ??????????? raise IndexError("Index of image %s out of range(%s)" ???????????????????????????? %(index, self.data["image_count"])) ??????? multi = 0 ??????? if index == "all": ??????????? multi = 1 ??????????? index = 0 ??????? if not dest: ??????????? dest = self.filename ??????? target = os.path.splitext(dest)[0] ??????? res = self.raw.demosaicate(index, multi, target)??? # This call the CPython code ??????? self.out_filename = self.raw.out_file ??? def run_multitasks(self): ??????? def process(*args): ??????????? fname, idx, oname = args ??????????? self.identify() ??????????? self.unraw(idx, oname) ??????? while self.tasks: ??????????? t = Thread(target=process, args=self.tasks.pop(0)) ??????????? t.start() ??????????? t.join() ??????????? if self.callback: ??????????????? self.callback(self.out_filename) ---------------------------------------------------------------- I use a list of 20 files and all files are converted, one by one, because the CPython part is not thread safe. But the real interesting thing is the behaviour in the slot() function. Examples: ?1 ------------------------------- def slot(name): ??? if os.path.isfile(name): ??????? print("Created:", name, type(name)) ??????? print("%s created" % name) result: add: /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 Scaling with darkness 127, saturation 3712, and multipliers 2.310656 1.000000 1.257116 1.000000 AHD interpolation... Converting to sRGB colorspace... Created: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff Erreur de segmentation (core dumped) ---------------------------------- ?2 -------------------------------- def slot(name): ??? new = name[:] ??? if os.path.isfile(name): ??????? print("Created:", name, type(name)) ??????? print("New:", new) ??????? print("%s created" % new) result: add /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 Scaling with darkness 127, saturation 3712, and multipliers 2.310656 1.000000 1.257116 1.000000 AHD interpolation... Converting to sRGB colorspace... Created: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff New: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff Erreur de segmentation (core dumped) ---------------------------------- The most important, of course is the name of the new file created and I can be satisfied with that, I just need the file name, but it's not a reason to ignore the problem. Vincent From hjp-usenet3 at hjp.at Fri Oct 13 06:06:50 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 13 Oct 2017 12:06:50 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13 05:28, Gregory Ewing wrote: > Grant Edwards wrote: >> On 2017-10-13, Stefan Ram wrote: >>> 1 byte >>> >>> addressable unit of data storage large enough to hold >>> any member of the basic character set of the execution >>> environment? >>> >>> ISO C standard > > Hmmm. So an architecture with memory addressed in octets > and Unicode as the basic character set would have a > char of 8 bits and a byte of 32 bits? No, because a char is also "large enough to store any member of the basic execution character set. (?6.2.5). A "byte" is just the amount of storage a "char" occupies: | The sizeof operator yields the size (in bytes) of its operand [...] | When applied to an operand that has type char, unsigned char, or signed | char, (or a qualified version thereof) the result is 1. (?6.5.3.4) So if a C implementation used Unicode as the base character set, a byte would have to be at least 21 bits, a char the same, and all other types would have to be multiples of that. For any modern architecture that would be rounded up to 32 bits. (I am quite certain that there was at least one computer with a 21 bit word size, but I can't find it: Lots of 18 bit and 24 bit machines, but nothing in between.) An implementation could also choose the BMP as the base character set and the rest of Unicode as the extended character set. That would result in a 16 bit byte and char (and most likely UTF-16 as the multibyte character representation). > Not only does "byte" not always mean "8 bits", but > "char" isn't always short for "character"... True. A character often occupies more space than a char, and you can store non-character data in a char. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From tjol at tjol.eu Fri Oct 13 06:15:33 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 13 Oct 2017 12:15:33 +0200 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> Message-ID: <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> On 2017-10-13 11:07, Vincent Vande Vyvre wrote: > Le 13/10/17 ? 09:23, Chris Angelico a ?crit?: >> On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre >> wrote: >>> Simplified code: >>> >>> ---%<------------------------------------------ >>> ... >>> ---%<------------------------------------------ >>> >>> If I place self.callback() at the end of the func process that doesn't >>> change anything. >> First off, exactly what version of Python are you running this under? >> With a segfault, you need to be pretty specific - platform, version, >> word size, as much as you can gather. >> >> Secondly: Can you create a version of this that doesn't comment out >> part of the work? If you can make a script where, any time you run it, >> Python segfaults, that would be extremely helpful. >> >> Your code currently looks a bit weird. You create a thread, start it, >> and then immediately wait for it (join()). When you add a task, if >> it's the first task you've added, you process tasks, thus removing >> that task. So the class isn't actually doing anything, and logically, >> you could simply process the files directly in the loop. I'm guessing >> that something in there (probably the threading) is causing the crash, >> but without a complete and testable demo, it's hard to be sure. >> >> ChrisA > > I'm using 3.6.1 in venv > > It's not easy to write a runnable code because in this example the > method process() is too simplified and the code don't reproduce the > segfault. That is what you'll have to do, though. If you strip out as much as possible to create a minimal, working, complete example, you will have a much easier time reasoning about what is happening. This will also allow other people to reproduce the problem, and help. > The code is an image processing in two parts: the processing himself in > CPython and an api in Python. If you have custom C code, it's likely that there is a memory management problem there. It's not unusual for incorrect memory management to cause problems in a completely different part of the code, where something tries to access freed memory or something. If removing a call to C fixes it, I recommend you carefully check the memory management logic of your C function. > > I have written the two parts. > > In the real code this is the equivalent of the method process(): > ---------------------------------------------- > ??? def unraw(self, index=0, dest=""): > ??????? """Run the demosaication process. > > ??????? Args: > ??????? index -- the index of the image or "all" if there's more than > one image > ???????????????? into the file > ??????? dest -- the absolute file name for the image decoded.? If a file > with > ??????????????? the same name already exists, it will be overwritten. > > ??????? Raise IndexError if index >= self.image_count > ??????? """ > ??????? if not self.is_raw: > ??????????? fname = os.path.basename(self.filename) > ??????????? raise TypeError("RAW file %s not supported!" % fname) > > ??????? if index >= self.data["image_count"]: > ??????????? raise IndexError("Index of image %s out of range(%s)" > ???????????????????????????? %(index, self.data["image_count"])) > > ??????? multi = 0 > ??????? if index == "all": > ??????????? multi = 1 > ??????????? index = 0 > > ??????? if not dest: > ??????????? dest = self.filename > > ??????? target = os.path.splitext(dest)[0] > ??????? res = self.raw.demosaicate(index, multi, target)??? # This call > the CPython code > ??????? self.out_filename = self.raw.out_file > > ??? def run_multitasks(self): > ??????? def process(*args): > ??????????? fname, idx, oname = args > ??????????? self.identify() > ??????????? self.unraw(idx, oname) > > ??????? while self.tasks: > ??????????? t = Thread(target=process, args=self.tasks.pop(0)) > ??????????? t.start() > ??????????? t.join() > ??????????? if self.callback: > ??????????????? self.callback(self.out_filename) > ---------------------------------------------------------------- > > I use a list of 20 files and all files are converted, one by one, > because the CPython part is not thread safe. > > But the real interesting thing is the behaviour in the slot() function. > > Examples: > ?1 ------------------------------- > def slot(name): > ??? if os.path.isfile(name): > ??????? print("Created:", name, type(name)) > ??????? print("%s created" % name) > > result: > add: /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 > Scaling with darkness 127, saturation 3712, and > multipliers 2.310656 1.000000 1.257116 1.000000 > AHD interpolation... > Converting to sRGB colorspace... > Created: > /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff > > Erreur de segmentation (core dumped) > ---------------------------------- > > ?2 -------------------------------- > def slot(name): > ??? new = name[:] > ??? if os.path.isfile(name): > ??????? print("Created:", name, type(name)) > ??????? print("New:", new) > ??????? print("%s created" % new) > > result: > add /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 > Scaling with darkness 127, saturation 3712, and > multipliers 2.310656 1.000000 1.257116 1.000000 > AHD interpolation... > Converting to sRGB colorspace... > Created: > /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff > > New: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff > Erreur de segmentation (core dumped) > ---------------------------------- > > The most important, of course is the name of the new file created and I > can be satisfied with that, I just need the file name, but it's not a > reason to ignore the problem. > > Vincent > From bc at freeuk.com Fri Oct 13 06:28:49 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 11:28:49 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: On 13/10/2017 07:16, Gregory Ewing wrote: > Steve D'Aprano wrote: >> On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: >> >>> If the compiler can tell where p is initially pointing, it could >>> put the pointer in read-only memory. >> >> If it's read-only, how can the compiler write to it? >> >> (I come from the days when ROM was actual ROM, burned in at the factory.) > > So, the factory is allowed to write to it. Possibly > it's writing data that came from... a compiler? > > A memory that couldn't be written to at all, ever, would > be a bit useless! > It's read-only in the same sense as a printed book; you can't change the marks already on the page. I've programmed EPROMS. Neither the C language nor the concept of 'const' attributes for data ever came into it. While executable code doesn't really need it. It is simply not a necessity. You just arranged for writeable portions of memory to be at different locations than that containing the code and fixed data. Sometimes it was only data. 'const' is anyway the wrong sort of attribute to use, as I've already explained. For example: const int A = rand(); // inside a function A is written to multiple times at runtime. It can't go into actual read-only memory like a ROM. And it might have trouble going into a memory page with read-only attributes. 'const' tries to do too many things, most of them poorly, although it does a very good job at adding clutter. -- bartc From steve+python at pearwood.info Fri Oct 13 06:37:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 13 Oct 2017 21:37:18 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 05:16 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: >> On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: >> >>>If the compiler can tell where p is initially pointing, it could >>>put the pointer in read-only memory. >> >> If it's read-only, how can the compiler write to it? >> >> (I come from the days when ROM was actual ROM, burned in at the factory.) > > So, the factory is allowed to write to it. Possibly > it's writing data that came from... a compiler? The data could come from anywhere, including a scanner: https://hackaday.com/2012/08/24/uncovering-easter-eggs-in-old-mac-roms/ I wasn't questioning where the data came from, but how the compiler can write to READ ONLY MEMORY which might not even be in the same continent as the compiler that generated the code. Read-only memory (ROM) is typically burned into the silicon by the integrated chip manufacturer at the factory: https://en.wikipedia.org/wiki/Mask_ROM or written by a dedicated hardware device: https://en.wikipedia.org/wiki/Programmable_read-only_memory Whether the data is burned into the silicon or electrically written by a dedicated device, it isn't written by the compiler, and once written the data is permanent. > A memory that couldn't be written to at all, ever, would > be a bit useless! Macs used a ROM for at least a decade and probably more. The ROM contained data such as mouse cursors, toolbox routines, icons, sounds, and a bootloader. No Mac was capable of writing to their ROMs any more than they could write to their mouse or a CD-ROM. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.f.moore at gmail.com Fri Oct 13 06:39:12 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 13 Oct 2017 11:39:12 +0100 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> Message-ID: As a specific suggestion, I assume the name of the created file is a string object constructed in the C extension code, somehow. The fact that you're getting the segfault with some uses of that string (specifically, passing it to %-formatting) suggests that there's a bug in the C code that constructs that string. That's where I'd start by looking. Maybe something isn't zero-terminated that should be? Maybe your code doesn't set up the character encoding information correctly? Paul On 13 October 2017 at 11:15, Thomas Jollans wrote: > On 2017-10-13 11:07, Vincent Vande Vyvre wrote: >> Le 13/10/17 ? 09:23, Chris Angelico a ?crit : >>> On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre >>> wrote: >>>> Simplified code: >>>> >>>> ---%<------------------------------------------ >>>> ... >>>> ---%<------------------------------------------ >>>> >>>> If I place self.callback() at the end of the func process that doesn't >>>> change anything. >>> First off, exactly what version of Python are you running this under? >>> With a segfault, you need to be pretty specific - platform, version, >>> word size, as much as you can gather. >>> >>> Secondly: Can you create a version of this that doesn't comment out >>> part of the work? If you can make a script where, any time you run it, >>> Python segfaults, that would be extremely helpful. >>> >>> Your code currently looks a bit weird. You create a thread, start it, >>> and then immediately wait for it (join()). When you add a task, if >>> it's the first task you've added, you process tasks, thus removing >>> that task. So the class isn't actually doing anything, and logically, >>> you could simply process the files directly in the loop. I'm guessing >>> that something in there (probably the threading) is causing the crash, >>> but without a complete and testable demo, it's hard to be sure. >>> >>> ChrisA >> >> I'm using 3.6.1 in venv >> >> It's not easy to write a runnable code because in this example the >> method process() is too simplified and the code don't reproduce the >> segfault. > > That is what you'll have to do, though. > If you strip out as much as possible to create a minimal, working, > complete example, you will have a much easier time reasoning about what > is happening. This will also allow other people to reproduce the > problem, and help. > >> The code is an image processing in two parts: the processing himself in >> CPython and an api in Python. > > If you have custom C code, it's likely that there is a memory management > problem there. It's not unusual for incorrect memory management to cause > problems in a completely different part of the code, where something > tries to access freed memory or something. > > If removing a call to C fixes it, I recommend you carefully check the > memory management logic of your C function. > > >> >> I have written the two parts. >> >> In the real code this is the equivalent of the method process(): >> ---------------------------------------------- >> def unraw(self, index=0, dest=""): >> """Run the demosaication process. >> >> Args: >> index -- the index of the image or "all" if there's more than >> one image >> into the file >> dest -- the absolute file name for the image decoded. If a file >> with >> the same name already exists, it will be overwritten. >> >> Raise IndexError if index >= self.image_count >> """ >> if not self.is_raw: >> fname = os.path.basename(self.filename) >> raise TypeError("RAW file %s not supported!" % fname) >> >> if index >= self.data["image_count"]: >> raise IndexError("Index of image %s out of range(%s)" >> %(index, self.data["image_count"])) >> >> multi = 0 >> if index == "all": >> multi = 1 >> index = 0 >> >> if not dest: >> dest = self.filename >> >> target = os.path.splitext(dest)[0] >> res = self.raw.demosaicate(index, multi, target) # This call >> the CPython code >> self.out_filename = self.raw.out_file >> >> def run_multitasks(self): >> def process(*args): >> fname, idx, oname = args >> self.identify() >> self.unraw(idx, oname) >> >> while self.tasks: >> t = Thread(target=process, args=self.tasks.pop(0)) >> t.start() >> t.join() >> if self.callback: >> self.callback(self.out_filename) >> ---------------------------------------------------------------- >> >> I use a list of 20 files and all files are converted, one by one, >> because the CPython part is not thread safe. >> >> But the real interesting thing is the behaviour in the slot() function. >> >> Examples: >> 1 ------------------------------- >> def slot(name): >> if os.path.isfile(name): >> print("Created:", name, type(name)) >> print("%s created" % name) >> >> result: >> add: /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 >> Scaling with darkness 127, saturation 3712, and >> multipliers 2.310656 1.000000 1.257116 1.000000 >> AHD interpolation... >> Converting to sRGB colorspace... >> Created: >> /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff >> >> Erreur de segmentation (core dumped) >> ---------------------------------- >> >> 2 -------------------------------- >> def slot(name): >> new = name[:] >> if os.path.isfile(name): >> print("Created:", name, type(name)) >> print("New:", new) >> print("%s created" % new) >> >> result: >> add /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2 >> Scaling with darkness 127, saturation 3712, and >> multipliers 2.310656 1.000000 1.257116 1.000000 >> AHD interpolation... >> Converting to sRGB colorspace... >> Created: >> /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff >> >> New: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff >> Erreur de segmentation (core dumped) >> ---------------------------------- >> >> The most important, of course is the name of the new file created and I >> can be satisfied with that, I just need the file name, but it's not a >> reason to ignore the problem. >> >> Vincent >> > > > > -- > https://mail.python.org/mailman/listinfo/python-list From marko at pacujo.net Fri Oct 13 06:47:30 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 13:47:30 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> Message-ID: <871sm7b9z1.fsf@elektro.pacujo.net> "Peter J. Holzer" : > On 2017-10-13 05:28, Gregory Ewing wrote: >> Not only does "byte" not always mean "8 bits", but >> "char" isn't always short for "character"... > > True. Well, it does, in my universe. That was cast in stone 10**-32 seconds after the Big Bang. Marko From marko at pacujo.net Fri Oct 13 06:49:11 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 13:49:11 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> Message-ID: <87wp3z9vbs.fsf@elektro.pacujo.net> bartc : > 'const' tries to do too many things, most of them poorly, although it > does a very good job at adding clutter. +1 Marko From marko at pacujo.net Fri Oct 13 06:51:47 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 13:51:47 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <87wp3z9vbs.fsf@elektro.pacujo.net> Message-ID: <87shen9v7g.fsf@elektro.pacujo.net> Marko Rauhamaa : > bartc : >> 'const' tries to do too many things, most of them poorly, although it >> does a very good job at adding clutter. > > +1 However, I do my best to honor "const" since it's there. I'm even more Catholic than the Pope and declare: int main(int argc, const char *const argv[]) I generally *don't* use "const" with opaque data types. Marko From vincent.vande.vyvre at telenet.be Fri Oct 13 07:18:05 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Fri, 13 Oct 2017 13:18:05 +0200 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> Message-ID: <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Le 13/10/17 ? 12:39, Paul Moore a ?crit?: > As a specific suggestion, I assume the name of the created file is a > string object constructed in the C extension code, somehow. The fact > that you're getting the segfault with some uses of that string > (specifically, passing it to %-formatting) suggests that there's a bug > in the C code that constructs that string. That's where I'd start by > looking. Maybe something isn't zero-terminated that should be? Maybe > your code doesn't set up the character encoding information correctly? > > Paul That was my first idea, because I can verify the instance of PyUnraw is not destroyed when I use the file name, but I was in trouble by the usage of the file name in string formatting. For example I can use the file name into the slot i.e. shutil.copy(fname, "path/renamed.tiff") The file is correctly copied. In fact, I can do anything with the file name except use it in string formatting, then your approach is probably a good way. Into the CPython part I have a c-string pythonized by: ??? temp = self->outfname; ??? self->outfname = PyUnicode_FromString(ofname); ??? Py_XDECREF(temp); and exposed to Python with: static PyMemberDef PyUnraw_members[] = { ??? {"out_file", T_OBJECT_EX, offsetof(PyUnraw, outfname), 0, ???? "Path of the decoded file"}, ??? ... Vincent From hjp-usenet3 at hjp.at Fri Oct 13 07:49:07 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 13 Oct 2017 13:49:07 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13 10:37, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 05:16 pm, Gregory Ewing wrote: >> Steve D'Aprano wrote: >>> On Fri, 13 Oct 2017 03:37 pm, Gregory Ewing wrote: >>> >>>>If the compiler can tell where p is initially pointing, it could >>>>put the pointer in read-only memory. >>> >>> If it's read-only, how can the compiler write to it? >>> >>> (I come from the days when ROM was actual ROM, burned in at the factory.) >> >> So, the factory is allowed to write to it. Possibly >> it's writing data that came from... a compiler? > > The data could come from anywhere, including a scanner: > > https://hackaday.com/2012/08/24/uncovering-easter-eggs-in-old-mac-roms/ > > I wasn't questioning where the data came from, but how the compiler can write > to READ ONLY MEMORY which might not even be in the same continent as the > compiler that generated the code. A compiler rarely writes into the final destination memory (JIT compilers are the exception). It writes into a file. This file often has sections like "code", "read-only data", etc. Another tool (maybe the OS, maybe an EPROM burner) will later read this file and take appropriate actions. "the compiler could put the pointer in read-only memory" is just shorthand for "the compile could put the pointer into one of the read-only sections of the object file". > Read-only memory (ROM) is typically burned into the silicon by the integrated > chip manufacturer at the factory: > > https://en.wikipedia.org/wiki/Mask_ROM In this case there will be tool (or more likely a whole tool-chain) which takes the read-only sections of the object file and converts them into a lithographic mask which will then be used to create chips. The pointer will end up as a bunch of transistors. > or written by a dedicated hardware device: > > https://en.wikipedia.org/wiki/Programmable_read-only_memory And in this case there will be a tool which will read the object file and send the contents of the read-only sections to the device which writes the ((E)E)PROM. And finally, the most frequent case: The OS will will read the executable into RAM, mark those pages from the read-only sections as read-only in the page table and start it. > Whether the data is burned into the silicon or electrically written by a > dedicated device, it isn't written by the compiler, and once written the data > is permanent. Did anyone claim that? >> A memory that couldn't be written to at all, ever, would be a bit >> useless! > > Macs used a ROM for at least a decade and probably more. The ROM contained > data such as mouse cursors, toolbox routines, icons, sounds, and a > bootloader. No Mac was capable of writing to their ROMs any more than they > could write to their mouse or a CD-ROM. Did anyone claim that? hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From p.f.moore at gmail.com Fri Oct 13 08:22:07 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 13 Oct 2017 13:22:07 +0100 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Message-ID: On 13 October 2017 at 12:18, Vincent Vande Vyvre wrote: > Le 13/10/17 ? 12:39, Paul Moore a ?crit : >> >> As a specific suggestion, I assume the name of the created file is a >> string object constructed in the C extension code, somehow. The fact >> that you're getting the segfault with some uses of that string >> (specifically, passing it to %-formatting) suggests that there's a bug >> in the C code that constructs that string. That's where I'd start by >> looking. Maybe something isn't zero-terminated that should be? Maybe >> your code doesn't set up the character encoding information correctly? >> >> Paul > > That was my first idea, because I can verify the instance of PyUnraw is not > destroyed when I use the file name, but I was in trouble by the usage of the > file name in string formatting. > > For example I can use the file name into the slot i.e. shutil.copy(fname, > "path/renamed.tiff") > The file is correctly copied. > > In fact, I can do anything with the file name except use it in string > formatting, then your approach is probably a good way. > > Into the CPython part I have a c-string pythonized by: > temp = self->outfname; > self->outfname = PyUnicode_FromString(ofname); > Py_XDECREF(temp); > > and exposed to Python with: > static PyMemberDef PyUnraw_members[] = { > {"out_file", T_OBJECT_EX, offsetof(PyUnraw, outfname), 0, > "Path of the decoded file"}, OK. I presume ofname is UTF-8 encoded as required by PyUnicode_FromString, and you're not accidentally getting a different encoding? I don't see any obvious issue - but it's still far more likely it's an issue in the C code somewhere. Maybe a refcounting bug - those often trigger peculiar errors as memory gets freed too soon, and something then references freed memory. Paul From greg.ewing at canterbury.ac.nz Fri Oct 13 08:48:44 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 14 Oct 2017 01:48:44 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > I wasn't questioning where the data came from, but how the compiler can write > to READ ONLY MEMORY which might not even be in the same continent as the > compiler that generated the code. I thought it would be fairly obvious that by "put it in read-only memory" I meant "arrange for it to be in a location that is read-only at run time". Obviously it can't be read-only at *compile* time, just as a physical ROM can't be read-only at manufacture time. -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 13 08:54:49 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 14 Oct 2017 01:54:49 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: Neil Cerutti wrote: > I can tell at a glance if a parameter is expected to be > modifiable just by looking at the function signature. The question is why doesn't anyone feel the need to be able to do that for Python functions? Whether a function modifies things passed to it is just as important to know in Python as it is in C. -- Greg From bc at freeuk.com Fri Oct 13 09:00:53 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 14:00:53 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On 13/10/2017 12:49, Peter J. Holzer wrote: > On 2017-10-13 10:37, Steve D'Aprano wrote: >> or written by a dedicated hardware device: >> >> https://en.wikipedia.org/wiki/Programmable_read-only_memory > > And in this case there will be a tool which will read the object file > and send the contents of the read-only sections to the device which > writes the ((E)E)PROM. First time I did this was with a home-made programmer directly connected to a keyboard. I had to press the right key to generate the 7-bit pattern I wanted (I can't remember what I did about the 8th bit), burn it into the current location then step the address counter to the next. No mistakes were tolerated. It worked. > And finally, the most frequent case: The OS will will read the > executable into RAM, mark those pages from the read-only sections as > read-only in the page table and start it. Or as I did it, on a home-made computer with two banks of 16KB RAM, one bank had the editor, compiler and source code, the other the generated code. Just before running it, I would flip a switch to write-protect the first bank in case something went wrong. (I didn't have floppies only slow, unreliable tape, so things had to be memory-resident as much as possible.) And, actually, even now machines don't have that much control: you create a large table of data at runtime, but it is still in writeable memory, and accidental or malicious code could write into it. No matter that the C source may have had a few consts sprinkled about. (It's bit like those incredibly annoying anti-piracy messages and videos you get on DVDS. No actual pirate would ever see them, only honest people who have no intention of copying!) Even if data is actually in write-protected memory, attempts to write to it will cause a crash. On my home-made system, they just did nothing. Much more graceful. -- bartc From rosuav at gmail.com Fri Oct 13 09:16:46 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 00:16:46 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 12:00 AM, bartc wrote: > Even if data is actually in write-protected memory, attempts to write to it > will cause a crash. On my home-made system, they just did nothing. Much more > graceful. The novice thinks his job is to stop the program from crashing. The expert knows that a crash is actually far FAR better than silently doing nothing. ChrisA From marko at pacujo.net Fri Oct 13 09:22:36 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 16:22:36 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: <87mv4v9o83.fsf@elektro.pacujo.net> Gregory Ewing : > Neil Cerutti wrote: >> I can tell at a glance if a parameter is expected to be modifiable >> just by looking at the function signature. > > The question is why doesn't anyone feel the need to be able to do that > for Python functions? Whether a function modifies things passed to it > is just as important to know in Python as it is in C. I often ponder what information should be conveyed by the declaration. For example, some methods are called with lock taken while others not. Should that be indicated in the name of the function? Also ownership and other contractual matters. I occasionally fall into the temptation of coding such aspects in names, but equally often the attempt backfires. Most you get is silly syntactic clutter. Let's just say that the problem remains unsolved, maybe unsolvable, and any attempts at solving the problem seem to cause worse problems. BTW, the original reason for C requiring declarations in the first place wasn't readability. Rather, it was to make compilation possible in the first place. It is interesting that C++ and Java have taken steps to remove such information where the compiler can know or guess it from the context. Marko From bc at freeuk.com Fri Oct 13 09:51:54 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 14:51:54 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On 13/10/2017 14:16, Chris Angelico wrote: > On Sat, Oct 14, 2017 at 12:00 AM, bartc wrote: >> Even if data is actually in write-protected memory, attempts to write to it >> will cause a crash. On my home-made system, they just did nothing. Much more >> graceful. > > The novice thinks his job is to stop the program from crashing. The > expert knows that a crash is actually far FAR better than silently > doing nothing. For a novice, seeing 'Segmentation fault (core dumped)' is better? There, a friendlier, more useful error message is called for ('writing into read-only memory' is a good start). But the idea of having memory which is permanently or temporarily write-protected is also useful: you can do what you like to it and it won't change, and such an attempt wouldn't necessarily be an error. For example, you're drawing a series of points or connected lines into a window to form a shape. But part of the shape is outside the window. So you fix the logic and try again. You don't want it to crash if you gave it coordinates (corresponding to memory beyond the window limits) outside the boundaries. It's just a technique, like greying out certain menu options - clicking them will do nothing, but you won't get an error message and you don't want to allow them anyway, risking more serious consequences. -- bartc From p.f.moore at gmail.com Fri Oct 13 10:03:48 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 13 Oct 2017 15:03:48 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: On 13 October 2017 at 13:54, Gregory Ewing wrote: > Neil Cerutti wrote: >> >> I can tell at a glance if a parameter is expected to be >> modifiable just by looking at the function signature. > > > The question is why doesn't anyone feel the need to be > able to do that for Python functions? Whether a function > modifies things passed to it is just as important to > know in Python as it is in C. While I don't *really* know, my intuition is that const is important in C for guaranteeing that functions don't mess around with the innards of (conceptually) primitive types. Hence const char *, and const pointers/arrays. More complex types like collections - lists, trees, etc - can't be declared as "const". Well, they can, but typically that doesn't guarantee that the collection isn't changed, just that the pointer to it isn't. So in C, for all practical purposes const signifies that an argument is what in languages like C# would be called a value type - an immutable primitive value. In Python and other higher level languages, primitive types are immutable by default, or to put it another way, basic types are always value types, and constness is part of the definition of the *type* rather than of the name referring to it. So there's no need for an explicit "const" annotation. To put it another way, in C const is a property of the variable being declared, not the value assigned to it. In Python, variables aren't declared, and constness is an inherent property of the value (or its type). One interesting question which this does raise is whether there's a place for "const" in type annotations - I suspect not, because it's either trivial (the type is immutable) or too hard to define (you'd need to recursively know for all methods whether they mutate the value). Paul From marko at pacujo.net Fri Oct 13 10:25:38 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 17:25:38 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: <87a80v9lb1.fsf@elektro.pacujo.net> Paul Moore : > To put it another way, in C const is a property of the variable being > declared, not the value assigned to it. In Python, variables aren't > declared, and constness is an inherent property of the value (or its > type). One interesting question which this does raise is whether > there's a place for "const" in type annotations - I suspect not, > because it's either trivial (the type is immutable) or too hard to > define (you'd need to recursively know for all methods whether they > mutate the value). I don't think that's the core issue. User-defined classes (or, rather, objects) are often mutable. The question is who gets to modify them and what modifieds them (methods can have surprising side effects). "Const" would convey some of that information. However, why "const" out of the myriads of restrictions? The nice thing about Python is that I don't get to formally declare those things, and that makes my life easier. I don't have to worry about finding the right combination of these formal comments -- because I can't! Marko From bc at freeuk.com Fri Oct 13 10:27:12 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 15:27:12 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <87mv4v9o83.fsf@elektro.pacujo.net> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <87mv4v9o83.fsf@elektro.pacujo.net> Message-ID: On 13/10/2017 14:22, Marko Rauhamaa wrote: > BTW, the original reason for C requiring declarations in the first place > wasn't readability. Rather, it was to make compilation possible in the > first place. It is interesting that C++ and Java have taken steps to > remove such information where the compiler can know or guess it from the > context. The compiler might be able to, after analysing 100,000 lines of prior code and applying complex algorithms. But what about the poor user reading the code? Or can that now only be done with the aid or a browser that analyses 100,000 lines and applies that same algorithm? We mustn't forget the person writing the code, who may have a certain type in mind for X, but their (I nearly said 'his') analysis may not match the compiler's. Annotations can be useful. -- bartc From marko at pacujo.net Fri Oct 13 10:29:44 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 17:29:44 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <87mv4v9o83.fsf@elektro.pacujo.net> Message-ID: <8760bj9l47.fsf@elektro.pacujo.net> bartc : > But what about the poor user reading the code? Or can that now only be > done with the aid or a browser that analyses 100,000 lines and applies > that same algorithm? > > We mustn't forget the person writing the code, who may have a certain > type in mind for X, but their (I nearly said 'his') analysis may not > match the compiler's. > > Annotations can be useful. The writer of the code is responsible for clear communication. However, I don't believe boilerplate is the answer, quite the contrary. Marko From rosuav at gmail.com Fri Oct 13 10:30:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 01:30:30 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 12:51 AM, bartc wrote: > On 13/10/2017 14:16, Chris Angelico wrote: >> >> On Sat, Oct 14, 2017 at 12:00 AM, bartc wrote: >>> >>> Even if data is actually in write-protected memory, attempts to write to >>> it >>> will cause a crash. On my home-made system, they just did nothing. Much >>> more >>> graceful. >> >> >> The novice thinks his job is to stop the program from crashing. The >> expert knows that a crash is actually far FAR better than silently >> doing nothing. > > > > For a novice, seeing 'Segmentation fault (core dumped)' is better? Better than silently doing nothing? YES. Absolutely it is. > There, a friendlier, more useful error message is called for ('writing into > read-only memory' is a good start). Minor distinction. Yes, it might be helpful to have different error messages for different errors, but it's a trade-off. > But the idea of having memory which is permanently or temporarily > write-protected is also useful: you can do what you like to it and it won't > change, and such an attempt wouldn't necessarily be an error. > > For example, you're drawing a series of points or connected lines into a > window to form a shape. But part of the shape is outside the window. So you > fix the logic and try again. You don't want it to crash if you gave it > coordinates (corresponding to memory beyond the window limits) outside the > boundaries. It's common to emulate an infinite drawable area and then optimizing it by displaying only the part that's outside the window. That's not the same thing as silently doing nothing if something goes wrong - for instance, ignoring one point in the shape, and drawing a shape with fewer points. That would, in my opinion, be utterly unacceptable. And yes, I've seen programs that do that - it's usually called a "glitch". > It's just a technique, like greying out certain menu options - clicking them > will do nothing, but you won't get an error message and you don't want to > allow them anyway, risking more serious consequences. Even there, you often CAN get a report about the failure; clicking on something that's disabled will cause a short beep, unless that's disabled. Maybe you personally don't like that beep, and you'd rather it be suppressed - but that's the end user's choice, NOT the programmer's. "Errors should never pass silently, unless explicitly silenced." ChrisA From steve+python at pearwood.info Fri Oct 13 10:32:47 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 01:32:47 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 11:48 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: >> I wasn't questioning where the data came from, but how the compiler can >> write to READ ONLY MEMORY which might not even be in the same continent as >> the compiler that generated the code. > > I thought it would be fairly obvious that by "put it in > read-only memory" I meant "arrange for it to be in a > location that is read-only at run time". Obviously it > can't be read-only at *compile* time, just as a > physical ROM can't be read-only at manufacture time. No, its not obvious. It is a lot less obvious than my reference to ROM being burnt into hardware at a factory, which you responded to with the irrelevant observation that the data originated from a compiler. Sure, some data in ROMs might come from a compiler -- and some of it might not. Suppose a ROM contains a list of the names of the computer's developers (as some Mac ROMs did). It would be an abuse of language to say "The text editor writes to the ROM" and it is equally an abuse to say that the compiler writes to ROM just because the data burned into the chip is some sort of compiled object code. As Peter Holzer points out, the compiler likely writes to an object file, not the ROM, and as I pointed out, the process manufacturing the ROM might not even be in the same continent as the compiler. So when you talk about Read Only Memory, you're apparently not actually talking about actual Read Only Memory (ROM). Something has to write this data to "read only memory", and it must be able to do it over and over and over again, each time the program runs. How is this *read only*? It's not even "Write Once, Read Many" (WORM). What sort of read only memory can be written to over and over again? (This is not a rhetorical question.) It seems to me that you're not talking about ROM at all, but ordinary RAM. Then what do you mean by "read only"? A block of memory with a flag that says "unprivileged processes are prohibited from writing here"? (That's also not a rhetorical question.) Since the compiler is an unprivileged process, how can it write to memory that unprivileged processes cannot write to? I'm guessing that it can't. Can it? If not, then its probably some privileged process (part of the OS?) which does the writing. Am I close? (Still not a rhetorical question.) Earlier you said: If the compiler can tell where p is initially pointing, it could put the pointer in read-only memory. i.e. that you believe it is the *compiler* doing the writing. If that's not actually what you meant, then what you meant is not sufficiently clear. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 13 10:39:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 01:39:33 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> Message-ID: <59e0d027$0$14942$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 11:54 pm, Gregory Ewing wrote: > Neil Cerutti wrote: >> I can tell at a glance if a parameter is expected to be >> modifiable just by looking at the function signature. > > The question is why doesn't anyone feel the need to be > able to do that for Python functions? Whether a function > modifies things passed to it is just as important to > know in Python as it is in C. Lots of people would like Python to have a "freeze" function that can make immutable objects, it is a moderately common feature request. Some people (myself included) would like a "const" declaration that gives us names that can only be bound to once: const spam = "NOBODY expects the Spanish Inquisition!!!" # Okay spam = "foo" # Error. I don't mind if that is a runtime error, although a compile time error would be nicer. I don't know if either of these (actual immutable pure-Python objects, and un-rebindable names) count as quite the same thing you are asking Neil about. But I trust they're related. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Oct 13 10:51:42 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 01:51:42 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 1:32 AM, Steve D'Aprano wrote: > It seems to me that you're not talking about ROM at all, but ordinary RAM. > Then what do you mean by "read only"? A block of memory with a flag that > says "unprivileged processes are prohibited from writing here"? > > (That's also not a rhetorical question.) When I first learned about Protected Mode (as defined by the Intel 80386 and used in OS/2), there was a real concept of read-only RAM. The program loader would fetch up the executable file (data on the disk) and construct its segments: code, data, and BSS/stack. The data, BSS, and stack all end up as a single segment (data is what comes straight off the disk, BSS is all zeroes initially, and stack is uninitialized initially, but ultimately they're all read/write), and code is in its own segment. When control is handed to the new process, its segment table grants it read/write access to the data/stack segment, but read-only access to its code segment. Within that process, the code really truly is read-only, unless some magic is done. And yes, since it is fully readable, constant data CAN be included alongside actual executable code, but it's never going to be mutable in any way. ChrisA From listes at salort.eu Fri Oct 13 10:59:19 2017 From: listes at salort.eu (Julien Salort) Date: Fri, 13 Oct 2017 16:59:19 +0200 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> <871sm875t7.fsf@elektro.pacujo.net> Message-ID: Le 12/10/2017 ? 17:57, bartc a ?crit?: > With a const struct, you are stopped from directly modifying elements, > but if an element is a pointer, nothing stops you writing to what the > pointer points to, unless that has a const target too. And then you've > going to have problems doing normal updates. This constness just > insinuates itself everywhere. That is not very different from the mutable/immutable concept in Python, is it ? A tuple is immutable, but if it contains a mutable object, nothing prevents you from mutating it. Does it mean you think the mutable/immutable concept of Python is flawed? Julien From alister.ware at ntlworld.com Fri Oct 13 11:11:16 2017 From: alister.ware at ntlworld.com (alister) Date: Fri, 13 Oct 2017 15:11:16 GMT Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, 14 Oct 2017 01:48:44 +1300, Gregory Ewing wrote: > Steve D'Aprano wrote: >> I wasn't questioning where the data came from, but how the compiler can >> write to READ ONLY MEMORY which might not even be in the same continent >> as the compiler that generated the code. > > I thought it would be fairly obvious that by "put it in read-only > memory" I meant "arrange for it to be in a location that is read-only at > run time". Obviously it can't be read-only at *compile* time, just as a > physical ROM can't be read-only at manufacture time. oh yes it can in the past for large quantitys the data in a ROM chip was part of the chip etching mask (unless you consider a blank silicon wafer to be "Programmable" by the etching process)rather than prom which used programmable fuses or prom which could be erased by UV light (assuming the chip was fitted with a window otherwise it was known as one time programmable EPROM) The Apollo lunar lander used a magnetic core store that was hard coded at the time it was "Woven" I doubt that either process is in widespread usage any longer as most manufacturers no incorporate a way to update the firmware of a device (usually with flash memory) -- "I am not sure what this is, but an `F' would only dignify it." -- English Professor From bc at freeuk.com Fri Oct 13 11:15:35 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 16:15:35 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <8760bkde5c.fsf@elektro.pacujo.net> <87zi8wbuj5.fsf@elektro.pacujo.net> <871sm875t7.fsf@elektro.pacujo.net> Message-ID: On 13/10/2017 15:59, Julien Salort wrote: > Le 12/10/2017 ? 17:57, bartc a ?crit?: > >> With a const struct, you are stopped from directly modifying elements, >> but if an element is a pointer, nothing stops you writing to what the >> pointer points to, unless that has a const target too. And then you've >> going to have problems doing normal updates. This constness just >> insinuates itself everywhere. > > That is not very different from the mutable/immutable concept in Python, > is it ? > A tuple is immutable, but if it contains a mutable object, nothing > prevents you from mutating it. > Does it mean you think the mutable/immutable concept of Python is flawed? When you put it like that, then yes! But it depends on how you define the value of a tuple. If that is a recursive definition that includes all nested object levels, then it would be harder to keep the whole thing constant. If you look only one level deep, then it can be fully immutable (never mind that some elements could be functions that will give a different result each time they are evaluated). -- bartc From mal at europython.eu Fri Oct 13 11:27:56 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Fri, 13 Oct 2017 17:27:56 +0200 Subject: EuroPython 2017: Videos for Wednesday available online Message-ID: <6a7482c6-c43f-021b-f35e-ad14752a8551@europython.eu> We are pleased to announce the third batch of cut videos for EuroPython 2017. To see the new videos, please head over to our EuroPython YouTube channel and select the ?EuroPython 2017? playlist. The new videos start at entry 63 in the playlist. * EuroPython 2017 Videos * http://europython.tv/ In the coming two weeks, we will continue to release the other videos currently marked as ?private?, in batches of one conference day per week. Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/918860053070974976 Thanks. From steve+python at pearwood.info Fri Oct 13 11:28:20 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 02:28:20 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e0db96$0$14974$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 01:30 am, Chris Angelico wrote: >> It's just a technique, like greying out certain menu options - clicking >> them will do nothing, but you won't get an error message and you don't want >> to allow them anyway, risking more serious consequences. > > Even there, you often CAN get a report about the failure; clicking on > something that's disabled will cause a short beep, unless that's > disabled. Maybe you personally don't like that beep, and you'd rather > it be suppressed - but that's the end user's choice, NOT the > programmer's. "Errors should never pass silently, unless explicitly > silenced." Over 30 years since Apple first put out their user interface guidelines, and people still get the basics wrong. "Errors should never pass silently..." is excellent advise for programming, but not necessarily UIs. With a program, errors in the code represent bugs that should be fixed, and in principle we can remove the bugs one by one until there are none left, after which the program will be perfect. (Yeah, try not to laugh too much.) But in a UI, that will never be the case. Errors are not bugs, and can't be removed. All you can do is hope the user doesn't do it again, which they will. So UIs should follow a slightly different ideal: "Be tolerant of unimportant errors, and let them pass silently." For example, pressing delete when there is no text to delete should just silently do nothing. Why annoy the user by beeping? They probably just held down the backspace key for a microsecond too long. Surely you wouldn't log the error: "delete pressed but no text to delete". Um, okay, what do you expect me to do about it? Likewise, pressing the left arrow key when you are already at the start of the text should just silently remain at the start, not chastise the user. Or clicking in the body of the window where there are no controls or text fields. (MYOB, I'm looking at you.) But in fact, clicking a disabled button or menu item is not an error. There's no need to chastise the user with a beep or by flashing the screen or logging the event ("warning: user clicked disabled menu"). The point of disabling the menu is to avoid errors by preventing the user from selecting a command that will fail. The reason the menu is disabled rather than hidden is to provide a consistent interface and immediate visual feedback to the user. Clicking a disabled item should simply do nothing at all. If, and *only* if, you cannot rely on visual feedback that the item is disabled (perhaps you're using a text-only interface with no way to visually distinguish items) then you should provide some other feedback, like a beep or flashing the screen, or better (since audible feedback is annoying) display an informational status message that doesn't require explicit acknowledgement from the user. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 13 11:33:55 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 02:33:55 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e0dce4$0$14974$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 01:30 am, Chris Angelico wrote: >> For a novice, seeing 'Segmentation fault (core dumped)' is better? > > Better than silently doing nothing? YES. Absolutely it is. Chris, you forget that for Bart, his user-base is only himself. If he programs his home-made system to silently ignore writes to write-protected memory, that counts as the "...unless explicitly silenced" part of "Errors should never pass silently...". If that means his software is riddled with bugs, it will affect only himself, and no novices will be harmed. But in any case, a seg fault is a pretty crude and dangerous way to report errors. It isn't so much an error report as the *consequences* of the error, a bit like your car engine seizing up because you forgot to put oil in it. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From mcuddehe at strategicga.com Fri Oct 13 11:49:32 2017 From: mcuddehe at strategicga.com (Michael Cuddehe) Date: Fri, 13 Oct 2017 10:49:32 -0500 Subject: Unable to run pip in Windows 10 In-Reply-To: References: <005701d34207$a52165b0$ef643110$@com> <7ff7946a-c515-a18b-da78-1680499ddfcc@tjol.eu> <001b01d3429f$c4b38910$4e1a9b30$@com> Message-ID: <003f01d3443a$dd4588b0$97d09a10$@com> Thank you!!! What finally worked.....> py -m pip install pyperclip -----Original Message----- From: Terry Reedy [mailto:tjreedy at udel.edu] Sent: Wednesday, October 11, 2017 1:00 PM To: python-list at python.org Subject: Re: Unable to run pip in Windows 10 On 10/11/2017 10:46 AM, Michael Cuddehe wrote: > - What exactly did you install? >>> Latest install: Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC > v.1900 64 bit (AMD64)] on win32 >>> Downloaded from python.org. > - Can you start the Python interpreter? >>> Yes...works fine. > * How exactly did you go about this >>> ?? > - How exactly do you try to run pip? >>> Command prompt: c:\Users\FFC>pip import (module) Try running instead ...> python -m pip install or to specifically ensure that you run 3.5, ...> py -3.5 -m pip install and if that fails, copy the entire response. > - What exactly happens when you try? >>> Windows error message: "This app can't run on your PC...." -- Terry Jan Reedy From formisc at gmail.com Fri Oct 13 11:51:51 2017 From: formisc at gmail.com (Andrew Z) Date: Fri, 13 Oct 2017 11:51:51 -0400 Subject: unorderable types: list() > int() In-Reply-To: References: Message-ID: The answer is: The dict returns list - my mistake obviously. I think list.pop(0) is better for sanity than list[0]: Pos= [k,v for ...].pop(0) On Oct 13, 2017 00:23, "Andrew Z" wrote: > Hello, > pos = {"CLown":10,"BArbie":20} > I want to return integer (10) for the keyword that starts with "CL" > > > cl_ = [v for k, v in pos.items() if k.startswith('CL')] > cl_pos = cl_[0] > if cl_pos > 0: > > blah.. > > > There are 2 issues with the above: > a. ugly - cl_pos = cl_ [0] . I was thinking something like > > > cl_ = [v for k, v in pos.items() if k.startswith('CL')][0] > > but that is too much for the eyes - in 2 weeks i'll rewrite this into > something i can understand without banging against the desk. > So what can be a better approach here ? > > > b. in "cl_pos > 0:, cl_pos apparently is still a list. Though the run in a > python console has no issues and report cl_pos as int. > > > Appreciate. > > > > > > From bc at freeuk.com Fri Oct 13 12:19:44 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 17:19:44 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e0dce4$0$14974$b1db1813$d948b532@news.astraweb.com> References: <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0dce4$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: On 13/10/2017 16:33, Steve D'Aprano wrote: > On Sat, 14 Oct 2017 01:30 am, Chris Angelico wrote: > >>> For a novice, seeing 'Segmentation fault (core dumped)' is better? >> >> Better than silently doing nothing? YES. Absolutely it is. > > Chris, you forget that for Bart, his user-base is only himself. If he programs > his home-made system to silently ignore writes to write-protected memory, > that counts as the "...unless explicitly silenced" part of "Errors should > never pass silently...". If that means his software is riddled with bugs, it > will affect only himself, and no novices will be harmed. You're making light of a scheme that was extremely effective in a computer system with otherwise unprotected memory, that could write anywhere, including over all the code and over the OS. Without that write protection, what would have happened? Either it would go completely haywire, or hang, or could subtly change resident programs in dangerous ways. Or I could put that switch in then I could be CERTAIN that essential programs and data were untouched no matter what happened. So if it worked well then without needing to abort and report a message, why can't a scheme like that work now? BTW, when you're developing a new bit of hardware or software, and you're not part of team, then the user-base is normally just yourself. Nothing wrong with that, but you seem to like belittling people with such comments. What was the userbase when GvR started Python? -- Bartc From marko at pacujo.net Fri Oct 13 12:23:05 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 13 Oct 2017 19:23:05 +0300 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: <87infjrp92.fsf@elektro.pacujo.net> alister : > On Sat, 14 Oct 2017 01:48:44 +1300, Gregory Ewing wrote: >> I thought it would be fairly obvious that by "put it in read-only >> memory" I meant "arrange for it to be in a location that is read-only >> at run time". Obviously it can't be read-only at *compile* time, just >> as a physical ROM can't be read-only at manufacture time. > > oh yes it can > [...] > > I doubt that either process is in widespread usage any longer as most > manufacturers no incorporate a way to update the firmware of a device > (usually with flash memory) Then there's the case of FPGA, which can expose parts of itself as memory, whether writable or read-only. You can reprogram its ROM (and other) parts, but not using the regular RAM write mechanisms. In general, memory-mapped I/O may be read-only to software. For example, you could read dip-switch settings or the error counts of a hard disk from "const volatile" memory areas. There, "const" means you can't write into it and "volatile" means it could change "by itself". Marko From steve+python at pearwood.info Fri Oct 13 12:45:06 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 03:45:06 +1100 Subject: Python-list Digest, Vol 169, Issue 23 References: <1507823620l.6357002l.0l@psu.edu> Message-ID: <59e0ed94$0$14938$b1db1813$d948b532@news.astraweb.com> Hi Roger, Unfortunately you seem to have messed up the quoting (*and* the subject line, and the threading) for this post. The perils of replying to digests. I'll try to repair the worst of the misquoted text as I go, but if I accidentally attribute something I said to you, well, that's only because you did so first :-) On Fri, 13 Oct 2017 02:53 am, ROGER GRAYDON CHRISTMAN wrote: > On Thu, Oct 12, 2017 08:05 AM, Steve D'Aprano wrote:> [...] > This seems like a veritable straw man if any. > I am fairly sure that "one entry, one exit" > does not precisely mean "no branching whatsoever" > Nor does discouraging unrestricted GOTO > suggest that either. But Meyer doesn't just prohibit unrestricted GOTO. He prohibits, quote: 'any "break" or similar control mechanism.' Similar control mechanisms include *anything that branches*: they all have in common the fact that they branch, and that they jump from one executable statement to another, potentially far distant statement, rather than executing the next statement unconditionally. Now, you would be right that I was straw-manning *if* I had argued that: "Meyer is wrong because he wishes to prohibit `if` and `for` and function calls." But that is not even close to what I said: paraphrasing slightly, I said that if we take Meyer literally, that is what would be prohibited, but surely Meyer does not intend us to take him literally. That would be unreasonable. Therefore we must infer that Meyer breaks up branching control mechanisms into at least two groups: - those which are allowed, such as (we must presume) function calls, exceptions, loops, `if...else`; - those which are not allowed, such as `break`. The only concrete example of a bad form of jump that he gives is `break`, so we are left to guess at where he draws the line. I made a conservative, reasonable guess that he would want to allow subroutine calls, looping and conditional jumps (`if...else`). So what I did was the *literal opposite* of a straw man: I took a statement which is, on the face of it, obviously wrong, and strengthened it to something much more reasonable and defensible. Setting aside the (likely?) possibility that Meyer clarified his intention in the book, if we judge his position on his quoted words alone, then we must conclude that he wishes to prohibit any branching operation: - he doesn't refer to "unrestricted goto", but to unqualified "goto"; - the one concrete example he gives, "break", is an extremely restricted form of goto: you can only use it to jump from inside a loop to the first instruction immediately outside of the loop. As jumps go, you can't get much more restricted than "break", and yet Meyer is still against it. Judging Meyer's *actual* words, rather than putting words into his mouth, his position is more unreasonable and even weaker than the one I argue against. Hence, the very opposite of a straw man: I am arguing against an iron man, a position stronger than the one Meyer expressed. (One of my pet peeves is people misusing "straw man", but I must say this is the most blatantly wrong example I've seen.) My argument is that `break` and `continue` do not belong in the "bad list" of jumps that should be prohibited, not that Meyer thinks that we shouldn't use subroutines. See further below. Ironically, the one concrete example (that Stefan quotes) of what Meyer considers to be an allowed jump is exception handling, which many people discourage because of its similarity to GOTO or COMEFROM: "I consider exceptions to be no better than ?goto?s?, considered harmful since the 1960s, in that they create an abrupt jump from one point of code to another." https://www.joelonsoftware.com/2003/10/13/13/ "The core problem is the hidden control-flow possibility. [...] Exception handling introduces precisely this kind of hidden control flow possibility, at nearly every significant line of code" http://www.lighterra.com/papers/exceptionsharmful/ "Goto statements went out of style in the 60s, relegated today to be the prototypical example of bad coding. Yet hardly anyone seems to be bothered by exceptions, which do basically the same thing." https://petercai.com/the-case-against-exceptions/ Raymond Chen has written on why exceptions are hard to get right: https://blogs.msdn.microsoft.com/oldnewthing/20040422-00/?p=39683/ https://blogs.msdn.microsoft.com/oldnewthing/20050114-00/?p=36693/ and of course when Rob Pike created Go, he famously left exceptions out of the language, almost uniquely for a 21st century language. So the irony here is that Meyer wishes to prohibit the *least harmful* and *most defensible* form of restricted jump outside of the if/while/for statements, namely the "break" statement; while he would allow one of the more confusing, heavily-criticised, least-restricted jumps still allowed in modern languages (exceptions). In a language with exceptions, every method, function and loop can have multiple exits. Meyer's acceptance of exceptions is in direct contradiction to his stand that code blocks should have a single exit. Personally, I think Meyer's position here is incoherent. It is like he wishes to prohibit Nerf guns as too dangerous, while permitting semi-automatic assault rifles. > Clearly a while or for loop without break i > is single-entry and single exit. Indeed it is, but you have missed that Meyer made two points: - avoid jumps; - code blocks should have a single entry and a single exit; and that I was criticising them both. (Or at least, criticising the strong position that Meyer takes on each of them -- I would agree with a weaker, more nuanced position.) The two points are orthogonal. The first is necessary, otherwise one might jump around inside a single block, turning it into spaghetti code, while still having a single entry and single exit: for i in range(100): do_something() GOTO 20 10: do_more() 20: something_else() if condition: GOTO 10 yet_more() if flag: GOTO 20 # and if we reach here, we jump back to the start of the loop Both Meyer and I would agree this is bad code. But he considers `break` to be just as bad as the use of GOTO, while I don't. I wrote: > # with break > for i in range(2**64): > if isprime(i): > print(i, "is prime") > break > > # without break > still_searching = True > for i in range(2**64): > if still_searching and isprime(i): > print(i, "is prime") > still_searching = False > > # without break, attempt number 2 > still_searching = True > i = 0 > while still_searching and i < 2**64: > if isprime(i): > print(i, "is prime") > still_searching = False to which you (Roger) asked: > Do you really believe that there are no prime numbers > less than 2**64? Of course not. I had hoped that these examples would be interpreted as illustrative of the nature of the boilerplate one must often write, rather than the literal code I would use to find the smallest prime (which, for the record, is 2). I thought that by so clearly making the code so ludicrous, people would focus on the structure of the code snippet rather than the fact I was looking for primes. In hindsight, that was a mistake, and I apologise for my lack of clarity. I should have written something more generic: for attempt in range(number_of_attempts): preprocess() if condition(): print("success") break postprocess() which then can be naively written as: still_trying = True for attempt in range(number_of_attempts): if still_trying: preprocess() if condition(): print("success") still_trying = False if still_trying: postprocess() Please don't try to tell me that nobody would write code that naive, because I have done exactly that. And for exactly the reason Meyer gives: to ensure that the for loop has precisely one entry and one exit. That was a long time ago, when I was young and foolish and more prone to treat programming principles as absolute, universal truths. (By the way: Donald Knuth is, or at least was, one dissenter to the GOTO Considered Harmful movement. In 1974, Knuth argued that GOTO can sometimes lead to better code, and suggested a looser structural constraint: the program's flow chart should be able to be drawn with all forward jumps on one side, all all backward jumps on the other, and no jumps crossing each other.) [...] > That is the biggest objection programming teachers > like me dislike the break statement -- it leads to very > careless programming. That's a very strong statement. Got any studies to back it up? We (the collective programming community) have decades of experience with GOTO, and how *in practice* it leads to spaghetti code. I don't believe that there is any similar experience proving that "break considered harmful". Structured programming conclusively and definitively won the structured versus unstructured programming debate in the 70s and 80s, and consequently, hardly any high-level languages still offer GOTO, and even low-level languages discourage it, and put restrictions on where you can jump into and from. Nevertheless, modern high-level languages do allow many deviations from the structured paradigm, with "break" one of the most common. Wikipedia has a good overview of the more common deviations: https://en.wikipedia.org/wiki/Structured_programming#Common_deviations Peter Ritchie suggests applying Single Exit to OOP is cargo-cult programming: http://blog.peterritchie.com/Single-Entry-2C-Single-Exit-2C-Should-It-Still-Be-Applicable-In-Object-oriented-Languages-3F/ (although I wouldn't go quite that far). > Once you imagine that you will > use a break to exit your loops, you might stop thinking > about what your loop conditions are. And then you find > yourself in a corner where the only way out is to break. It is arguable whether the existence of break encourages that sort of poor code, or whether it causes people to stop thinking about their loop conditions. In any case, even if I accept that break/continue are less than ideal for students, that hardly means that that experienced programmers should avoid them. [...] > The code that generated the exception, of course, seemingly > has more than one exit, but exceptions are just that -- > exceptions. I hope you don't start using counting loops > up to 2**64 to visit a 100 element array and rely on > IndexError to exit such a loop. That is exactly how looping was implemented in early Python (before version 2.2) and it is still available as a fall-back. Of course this was done in the virtual machine, so it was much more efficient than if it were written directly in pure Python code. http://www.effbot.org/zone/python-for-statement.htm -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From dvl at psu.edu Fri Oct 13 14:05:42 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Fri, 13 Oct 2017 14:05:42 -0400 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: mailman.33.1507910404.15564.python-list@python.org References: Message-ID: <1507917942l.13566128l.0l@psu.edu> On Sat, Oct 14, 2017, Gregory Ewing wrote: > Message: 5 >Date: Sat, 14 Oct 2017 01:54:49 +1300 >From: Gregory Ewing >To: python-list at python.org >Subject: Re: Lies in education [was Re: The "loop and a half"] >Message-ID: >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > >Neil Cerutti wrote: >> I can tell at a glance if a parameter is expected to be >> modifiable just by looking at the function signature. > >The question is why doesn't anyone feel the need to be >able to do that for Python functions? Whether a function >modifies things passed to it is just as important to >know in Python as it is in C. > I'm just trying to put myself in the position of the programmer who is in this position of identifying the modifiable parameters in the function signature. Where are you getting the function signature? Are you looking at the source code? Or from the help() that access the module? In both cases, the docstring is an easy solution. The documentation itself in certainly state whether it modifies any parameter. And it is directly visible along with the function signature either way. We don't need any annotations or attributes or whatnot if we have the perfectly normal function documentation. Or is that kind of habit no longer practiced in 'the real world'? Roger Christman Pennsylvania State University From neilc at norwich.edu Fri Oct 13 15:01:03 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Fri, 13 Oct 2017 19:01:03 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e0d027$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 11:54 pm, Gregory Ewing wrote: > >> Neil Cerutti wrote: >>> I can tell at a glance if a parameter is expected to be >>> modifiable just by looking at the function signature. >> >> The question is why doesn't anyone feel the need to be >> able to do that for Python functions? Whether a function >> modifies things passed to it is just as important to >> know in Python as it is in C. > > Lots of people would like Python to have a "freeze" function > that can make immutable objects, it is a moderately common > feature request. > > Some people (myself included) would like a "const" declaration > that gives us names that can only be bound to once: > > const spam = "NOBODY expects the Spanish Inquisition!!!" # > Okay spam = "foo" # Error. > > I don't mind if that is a runtime error, although a compile > time error would be nicer. When there are many constants I tend to create a class called Constant that either doesn't get instanced, or has only one instance (sometimes some of my constants need to be calculated once at runtime). > I don't know if either of these (actual immutable pure-Python > objects, and un-rebindable names) count as quite the same thing > you are asking Neil about. But I trust they're related. Looking at the Python functions I actually write today, I've evolved my style until I simply don't modify function arguments, so it matters not if parameters are declared const. But my oldest programs still in use are different. For example: records = defaultdict(Data) read_unmatched_records(records) And then later code continues screwing around with that dictionary and liberally using C-style constant names, e.g.: for fname in glob.glob(COUNSELING_ACK): process_counseling_acks(fname, records) for fname in glob.glob(PLUS_APP_ACK): process_plus_request(fname, records) Such code feels alien to me now. A small script for reading promissory note acknowledgements grew into a business-vital, 1,000 LOC octopus without my leave. -- Neil Cerutti From hjp-usenet3 at hjp.at Fri Oct 13 15:32:28 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 13 Oct 2017 21:32:28 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13 14:51, Chris Angelico wrote: > On Sat, Oct 14, 2017 at 1:32 AM, Steve D'Aprano > wrote: >> It seems to me that you're not talking about ROM at all, but ordinary RAM. >> Then what do you mean by "read only"? A block of memory with a flag that >> says "unprivileged processes are prohibited from writing here"? >> >> (That's also not a rhetorical question.) > > When I first learned about Protected Mode (as defined by the Intel > 80386 and used in OS/2), there was a real concept of read-only RAM. > The program loader would fetch up the executable file (data on the > disk) and construct its segments: code, data, and BSS/stack. This is still the case. The granularity is just finer now: Instead of segments you use pages. (The 386 was the first x86 processor which supported paging. OS/2 probably used segments because it was originally designed for the 286.) hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From hjp-usenet3 at hjp.at Fri Oct 13 16:04:21 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 13 Oct 2017 22:04:21 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0db96$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13 15:28, Steve D'Aprano wrote: > On Sat, 14 Oct 2017 01:30 am, Chris Angelico wrote: >>> It's just a technique, like greying out certain menu options - clicking >>> them will do nothing, but you won't get an error message and you don't want >>> to allow them anyway, risking more serious consequences. >> >> Even there, you often CAN get a report about the failure; clicking on >> something that's disabled will cause a short beep, unless that's >> disabled. Maybe you personally don't like that beep, and you'd rather >> it be suppressed - but that's the end user's choice, NOT the >> programmer's. "Errors should never pass silently, unless explicitly >> silenced." > > Over 30 years since Apple first put out their user interface guidelines, and > people still get the basics wrong. > > "Errors should never pass silently..." is excellent advise for programming, > but not necessarily UIs. With a program, errors in the code represent bugs > that should be fixed, and in principle we can remove the bugs one by one > until there are none left, after which the program will be perfect. (Yeah, > try not to laugh too much.) > > But in a UI, that will never be the case. Errors are not bugs, and can't be > removed. It is also not a bug if - for example - a file cannot be opened, or is malformed. But you have to tell the user what went wrong. Pretending that all went well won't make the user happy (or at least that happyness will be short-lived). > All you can do is hope the user doesn't do it again, which they > will. So UIs should follow a slightly different ideal: > > "Be tolerant of unimportant errors, and let them pass silently." [...] > Likewise, pressing the left arrow key when you are already at the start of the > text should just silently remain at the start, not chastise the user. [... other examples elided ...] > But in fact, clicking a disabled button or menu item is not an error. If it is not an error, clearly the rule does not apply. So it depends on whether the action is an error or not (actually in the case of the disabled button I would tend to think it is an error: I can't think of a reason why anybody would intentionally click on a disabled button[1]). Pressing the left arrow key when you are already at the start of the line is good example. The user might expect the cursor to jump the end of the previous line, so if your program doesn't do that you probably should give some feedback. Or the user might just have pressed the key a bit too long, then the cursor should just stop there ("vi has two modes: One beeps a lot and the other doesn't"). So it depends on what the user expects. I think Chris a very good point here that this should be configurable unless your user base is very homogenous. hp [1] Except in a demo: "As you can see, this button is now disabled. Nothing happens when I click on it!" Everbody dives under the table just in time before the espresso machine in the corner explodes. -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From rosuav at gmail.com Fri Oct 13 16:07:27 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 07:07:27 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 6:32 AM, Peter J. Holzer wrote: > On 2017-10-13 14:51, Chris Angelico wrote: >> On Sat, Oct 14, 2017 at 1:32 AM, Steve D'Aprano >> wrote: >>> It seems to me that you're not talking about ROM at all, but ordinary RAM. >>> Then what do you mean by "read only"? A block of memory with a flag that >>> says "unprivileged processes are prohibited from writing here"? >>> >>> (That's also not a rhetorical question.) >> >> When I first learned about Protected Mode (as defined by the Intel >> 80386 and used in OS/2), there was a real concept of read-only RAM. >> The program loader would fetch up the executable file (data on the >> disk) and construct its segments: code, data, and BSS/stack. > > This is still the case. The granularity is just finer now: Instead of > segments you use pages. (The 386 was the first x86 processor which > supported paging. OS/2 probably used segments because it was originally > designed for the 286.) ... either that, or I'm just misremembering, it being many MANY years since I did anything that low-level. Let's assume I was mistaken. :) ChrisA From hjp-usenet3 at hjp.at Fri Oct 13 16:15:41 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 13 Oct 2017 22:15:41 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-13 15:11, alister wrote: > On Sat, 14 Oct 2017 01:48:44 +1300, Gregory Ewing wrote: >> Steve D'Aprano wrote: >>> I wasn't questioning where the data came from, but how the compiler can >>> write to READ ONLY MEMORY which might not even be in the same continent >>> as the compiler that generated the code. >> >> I thought it would be fairly obvious that by "put it in read-only >> memory" I meant "arrange for it to be in a location that is read-only at >> run time". Obviously it can't be read-only at *compile* time, just as a >> physical ROM can't be read-only at manufacture time. > > oh yes it can > in the past for large quantitys the data in a ROM chip was part of the > chip etching mask (unless you consider a blank silicon wafer to be > "Programmable" by the etching process)rather than prom which used > programmable fuses or prom which could be erased by UV light (assuming > the chip was fitted with a window otherwise it was known as one time > programmable EPROM) He didn't say "programmable". He said that the is "not read-only". Obviously the wafer is modified during the etching process and afterwards it contains information it didn't before. Would you say a piece of paper is "read-only" because you can't program it using address and data lines? I can write on it, I just need a different tool. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From bc at freeuk.com Fri Oct 13 16:17:16 2017 From: bc at freeuk.com (bartc) Date: Fri, 13 Oct 2017 21:17:16 +0100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e0d027$0$14942$b1db1813$d948b532@news.astraweb.com> References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e0d027$0$14942$b1db1813$d948b532@news.astraweb.com> Message-ID: <1b9EB.10573$JQ7.7375@fx13.am4> On 13/10/2017 15:39, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 11:54 pm, Gregory Ewing wrote: > >> Neil Cerutti wrote: >>> I can tell at a glance if a parameter is expected to be >>> modifiable just by looking at the function signature. >> >> The question is why doesn't anyone feel the need to be >> able to do that for Python functions? Whether a function >> modifies things passed to it is just as important to >> know in Python as it is in C. > > Lots of people would like Python to have a "freeze" function that can make > immutable objects, it is a moderately common feature request. > > Some people (myself included) would like a "const" declaration that gives us > names that can only be bound to once: > > const spam = "NOBODY expects the Spanish Inquisition!!!" # Okay > spam = "foo" # Error. Presumably also: const def f: const class c: const import i > I don't mind if that is a runtime error, although a compile time error would > be nicer. This would be of most use when the byte-code compiler (assuming there is one) knows about them. But many will be inside imported modules whose contents, AIUI, are not visible to the byte-code compiler. And then they would be accessed as: i.spam So this would be a const attribute. -- bartc From ctippur at gmail.com Fri Oct 13 17:25:01 2017 From: ctippur at gmail.com (Frustrated learner) Date: Fri, 13 Oct 2017 14:25:01 -0700 (PDT) Subject: Running flask on AWS SAM In-Reply-To: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> References: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> Message-ID: <4ba14276-aeed-443e-aa89-9ab48f5b3e69@googlegroups.com> Any takers? From ben.usenet at bsb.me.uk Fri Oct 13 17:42:43 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 13 Oct 2017 22:42:43 +0100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: <87infkhscc.fsf@bsb.me.uk> Message-ID: <87mv4ug1ws.fsf@bsb.me.uk> Chris Angelico writes: > On Fri, Oct 13, 2017 at 10:14 AM, Ben Bacarisse wrote: >> Chris Angelico writes: >>> I abbreviated that down to nothing, but since you ask, here's a really >>> REALLY simple run-down of how to use Heroku: >> >> I think I see what you mean now. You meant no configuration is needed >> because you use (or buy?) a cloud service that's all set up for it >> already? > > Correct - because the setup needed is completely generic. > >> From this and other posts I think the position is that I do need to do >> some server configuration (and essentially install a proxy server) to >> run Python web applications on my typical Apache set-up. And I would >> then have to shop around for suitable hosting that is already set up for >> running them. >> >> >> >> Thanks. That's not quite what I was after but it's good to know how to >> do that should I want to that later. > > Yep, it's not too hard. > > And that's why it's cleaner to work with Python than PHP. To use > custom URL routing in PHP, you have to use custom server rules; to use > custom URL routing in Python, you use "@app.route(...)" lines inside > your app, and perfectly standard server rules. That's one way to put it. Another is that to use Python I need to buy a new service that is already configured. If that's the way it's done, fine, but this sub-thread started with someone being surprised by the success of PHP. -- Ben. From rosuav at gmail.com Fri Oct 13 17:48:09 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 08:48:09 +1100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) In-Reply-To: <87mv4ug1ws.fsf@bsb.me.uk> References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: On Sat, Oct 14, 2017 at 8:42 AM, Ben Bacarisse wrote: > Chris Angelico writes: > >> On Fri, Oct 13, 2017 at 10:14 AM, Ben Bacarisse wrote: >>> Chris Angelico writes: >>>> I abbreviated that down to nothing, but since you ask, here's a really >>>> REALLY simple run-down of how to use Heroku: >>> >>> I think I see what you mean now. You meant no configuration is needed >>> because you use (or buy?) a cloud service that's all set up for it >>> already? >> >> Correct - because the setup needed is completely generic. >> >>> From this and other posts I think the position is that I do need to do >>> some server configuration (and essentially install a proxy server) to >>> run Python web applications on my typical Apache set-up. And I would >>> then have to shop around for suitable hosting that is already set up for >>> running them. >>> >>> >>> >>> Thanks. That's not quite what I was after but it's good to know how to >>> do that should I want to that later. >> >> Yep, it's not too hard. >> >> And that's why it's cleaner to work with Python than PHP. To use >> custom URL routing in PHP, you have to use custom server rules; to use >> custom URL routing in Python, you use "@app.route(...)" lines inside >> your app, and perfectly standard server rules. > > That's one way to put it. Another is that to use Python I need to buy a > new service that is already configured. If that's the way it's done, > fine, but this sub-thread started with someone being surprised by the > success of PHP. Thing is, that's exactly the same for both languages these days. You can get cheap (even zero-dollar) hosting that's preconfigured to be able to support either. There USED to be a difference, and everyone's acknowledged this - PHP built up some inertia - but there's now no real reason for it other than "it's popular, therefore people use it". ChrisA From grant.b.edwards at gmail.com Fri Oct 13 18:02:38 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 13 Oct 2017 22:02:38 +0000 (UTC) Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: On 2017-10-13, Chris Angelico wrote: [regarding PHP vs Python capable web-hosting services] > Thing is, that's exactly the same for both languages these days. You > can get cheap (even zero-dollar) hosting that's preconfigured to be > able to support either. There USED to be a difference, and everyone's > acknowledged this - PHP built up some inertia - but there's now no > real reason for it other than "it's popular, therefore people use it". Well, it's said that suffering builds character... What they don't tell you is what sort of character. -- Grant Edwards grant.b.edwards Yow! Am I SHOPLIFTING? at gmail.com From rosuav at gmail.com Fri Oct 13 18:10:48 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 09:10:48 +1100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) In-Reply-To: References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: On Sat, Oct 14, 2017 at 9:02 AM, Grant Edwards wrote: > On 2017-10-13, Chris Angelico wrote: > > [regarding PHP vs Python capable web-hosting services] > >> Thing is, that's exactly the same for both languages these days. You >> can get cheap (even zero-dollar) hosting that's preconfigured to be >> able to support either. There USED to be a difference, and everyone's >> acknowledged this - PHP built up some inertia - but there's now no >> real reason for it other than "it's popular, therefore people use it". > > Well, it's said that suffering builds character... > > What they don't tell you is what sort of character. I think it mostly builds U+1F4A9. ChrisA From sohcahtoa82 at gmail.com Fri Oct 13 18:11:33 2017 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Fri, 13 Oct 2017 15:11:33 -0700 (PDT) Subject: Running flask on AWS SAM In-Reply-To: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> References: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> Message-ID: <6438a158-c7ec-404d-8bad-f880a3f897af@googlegroups.com> On Thursday, October 12, 2017 at 9:20:11 PM UTC-7, Frustrated learner wrote: > Hello, > > I have a flask based application which i am able to run locally. > > $ python swagger_server/app.py > * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) > > I am trying to port this over to aws. I have all the dependencies and code organized in the same folder. > > Here is the transaction and error: > $ SERVER_NAME=0.0.0.0:3000 sam local start-api > 2017/10/12 21:02:20 Connected to Docker 1.32 > 2017/10/12 21:02:20 Fetching lambci/lambda:python3.6 image for python3.6 runtime... > python3.6: Pulling from lambci/lambda > Digest: sha256:c77ea3986c471c2f93dfa2a86492e6306989505073795da3b22defa2b10846a6 > Status: Image is up to date for lambci/lambda:python3.6 > > Mounting swagger_server.app.app (python3.6) at http://127.0.0.1:3000/current [GET] > Mounting static files from public at / > > You can now browse to the above endpoints to invoke your functions. > You do not need to restart/reload SAM CLI while working on your functions, > changes will be reflected instantly/automatically. You only need to restart > SAM CLI if you update your AWS SAM template. > > 2017/10/12 21:02:30 Invoking swagger_server.app.app (python3.6) > 2017/10/12 21:02:30 Decompressing /Users/shekartippur/playground/cyclopscloud/cyclopsglobal/swagger_server.zip > START RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 Version: $LATEST > 'SERVER_NAME': KeyError > Traceback (most recent call last): > File "/var/task/connexion/apps/abstract.py", line 266, in __call__ > return self.app(environ, start_response) > File "/var/task/flask/app.py", line 1997, in __call__ > return self.wsgi_app(environ, start_response) > File "/var/task/flask/app.py", line 1977, in wsgi_app > ctx = self.request_context(environ) > File "/var/task/flask/app.py", line 1938, in request_context > return RequestContext(self, environ) > File "/var/task/flask/ctx.py", line 242, in __init__ > self.url_adapter = app.create_url_adapter(self.request) > File "/var/task/flask/app.py", line 1765, in create_url_adapter > server_name=self.config['SERVER_NAME']) > File "/var/task/werkzeug/routing.py", line 1299, in bind_to_environ > wsgi_server_name = environ['SERVER_NAME'] > KeyError: 'SERVER_NAME' > > END RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 > REPORT RequestId: 9518afce-6cf7-4f20-9a31-0a60907b5467 Duration: 3257 ms Billed Duration: 3300 ms Memory Size: 0 MB Max Memory Used: 38 MB > > I looked up the error and dint find much help. Wondering what I could be missing. Instead of doing: SERVER_NAME=0.0.0.0:3000 sam local start-api try this: export SERVER_NAME="0.0.0.0:3000 sam local start-api" From Irv at furrypants.com Fri Oct 13 18:27:51 2017 From: Irv at furrypants.com (Irv Kalb) Date: Fri, 13 Oct 2017 15:27:51 -0700 Subject: Python 2 -> 3, urllib.urlOpen Message-ID: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> One of the colleges where I teach has just moved from Python 2 to Python 3. I am in the process of converting my beginning Python class from Python 2 to Python 3. Everything has gone smoothly, until I just tried to convert some code that imports and uses urllib.urlOpen to fetch data through an API. I am using an API that I found here: http://www.jarloo.com/yahoo_finance/ As a minimal example, I am trying to get the latest stock price for Apple. The following example works perfectly in Python 2: import urllib # set the Yahoo finance url, set stock name, ask for last price fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' # read all the data response = urllib.urlopen(fullURLWithParameters).read() print 'Response is: ', response This is asking for a stock name (s=) and I am adding in aapl as a stock symbol. I am also adding a "flag" parameter (f=) and setting it to l1 to get the last trade price. When I run this in Python 2, I see: Response is: 156.99 If I take the same program and just modify the print statement to add parentheses, then try to run it in Python 3.6 (on a Mac): import urllib # set the Yahoo finance url, set stock name, ask for last price fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' # read all the data response = urllib.urlopen(fullURLWithParameters).read() print('Response is: ', response) I get the following: Traceback (most recent call last): File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in response = urllib.urlopen(fullURLWithParameters).read() AttributeError: module 'urllib' has no attribute 'urlopen' I've looked at the Python 3.6 documentation for urllib, and I see that certain calls have been changed and others have been eliminated. But my eyes glaze over trying to figure out what to use instead. My question is: Is there a simple (hopefully one or two line) replacement for my call to url lib.urlopen().read() I know that there are other modules out there that handle requests (like the Requests module), but this is a strictly controlled university environment. I cannot download any external packages (other then pygame, which I got special permission for) onto the computers in the school. Therefore, I'm looking for something in the Python 3.6 Standard Library. Thanks in advance, Irv From ben.usenet at bsb.me.uk Fri Oct 13 18:45:42 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 13 Oct 2017 23:45:42 +0100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: <87infifyzt.fsf@bsb.me.uk> Chris Angelico writes: > On Sat, Oct 14, 2017 at 8:42 AM, Ben Bacarisse wrote: >> Chris Angelico writes: >> >>> On Fri, Oct 13, 2017 at 10:14 AM, Ben Bacarisse wrote: >>>> Chris Angelico writes: >>>>> I abbreviated that down to nothing, but since you ask, here's a really >>>>> REALLY simple run-down of how to use Heroku: >>>> >>>> I think I see what you mean now. You meant no configuration is needed >>>> because you use (or buy?) a cloud service that's all set up for it >>>> already? >>> >>> Correct - because the setup needed is completely generic. >>> >>>> From this and other posts I think the position is that I do need to do >>>> some server configuration (and essentially install a proxy server) to >>>> run Python web applications on my typical Apache set-up. And I would >>>> then have to shop around for suitable hosting that is already set up for >>>> running them. >>>> >>>> >>>> >>>> Thanks. That's not quite what I was after but it's good to know how to >>>> do that should I want to that later. >>> >>> Yep, it's not too hard. >>> >>> And that's why it's cleaner to work with Python than PHP. To use >>> custom URL routing in PHP, you have to use custom server rules; to use >>> custom URL routing in Python, you use "@app.route(...)" lines inside >>> your app, and perfectly standard server rules. >> >> That's one way to put it. Another is that to use Python I need to buy a >> new service that is already configured. If that's the way it's done, >> fine, but this sub-thread started with someone being surprised by the >> success of PHP. > > Thing is, that's exactly the same for both languages these days. You > can get cheap (even zero-dollar) hosting that's preconfigured to be > able to support either. There USED to be a difference, and everyone's > acknowledged this - PHP built up some inertia - but there's now no > real reason for it other than "it's popular, therefore people use it". That's good to know, but of course the current success of PHP is based exactly on what used to be the case. It will take a while for the inherent inertia of people, skills, processes and so on to be overcome. -- Ben. From robertvstepp at gmail.com Fri Oct 13 18:46:40 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 13 Oct 2017 17:46:40 -0500 Subject: Python 2 -> 3, urllib.urlOpen In-Reply-To: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: On Fri, Oct 13, 2017 at 5:27 PM, Irv Kalb wrote: > I've looked at the Python 3.6 documentation for urllib, and I see that certain calls have been changed and others have been eliminated. But my eyes glaze over trying to figure out what to use instead. > > My question is: Is there a simple (hopefully one or two line) replacement for my call to url lib.urlopen().read() I don't know much about this, but might you be looking for urllib.request.urlopen()? See https://docs.python.org/3/library/urllib.request.html#module-urllib.request in the docs. > I know that there are other modules out there that handle requests (like the Requests module), but this is a strictly controlled university environment. I cannot download any external packages (other then pygame, which I got special permission for) onto the computers in the school. Therefore, I'm looking for something in the Python 3.6 Standard Library. The above is in the standard library. -- boB From tjol at tjol.eu Fri Oct 13 18:47:42 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 14 Oct 2017 00:47:42 +0200 Subject: Python 2 -> 3, urllib.urlOpen In-Reply-To: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: On 14/10/17 00:27, Irv Kalb wrote: > One of the colleges where I teach has just moved from Python 2 to Python 3. I am in the process of converting my beginning Python class from Python 2 to Python 3. Everything has gone smoothly, until I just tried to convert some code that imports and uses urllib.urlOpen to fetch data through an API. I am using an API that I found here: http://www.jarloo.com/yahoo_finance/ > Hi Irv, That's great! It's always nice to hear about more people moving with the times :-) > [...] > > Traceback (most recent call last): > File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in > response = urllib.urlopen(fullURLWithParameters).read() > AttributeError: module 'urllib' has no attribute 'urlopen' > > > I've looked at the Python 3.6 documentation for urllib, and I see that certain calls have been changed and others have been eliminated. But my eyes glaze over trying to figure out what to use instead. > > My question is: Is there a simple (hopefully one or two line) replacement for my call to url lib.urlopen().read() Python 3 changed a number of things, including some rearrangement of the standard library. The "What's new in Python 3.x" documents are always worth reading, but I think you'll find "What's new in Python 3.0" particularly useful. The library changes are specified in PEP 3108 (this is linked from the "What's new" document) Python 3 has the urllib.request.urlopen function - this is descended from Python 2's urllib2.urlopen, and should be exactly what you need. > > I know that there are other modules out there that handle requests (like the Requests module), but this is a strictly controlled university environment. I cannot download any external packages (other then pygame, which I got special permission for) onto the computers in the school. Therefore, I'm looking for something in the Python 3.6 Standard Library. It's nice that Python comes with batteries included, isn't it? Cheers, Thomas PS: Pay attention to the fact that Python is case sensitive, and the function called urlopen (like in your code) and not urlOpen (like in your subject line) From greg.ewing at canterbury.ac.nz Fri Oct 13 19:58:21 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 14 Oct 2017 12:58:21 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e0db96$0$14974$b1db1813$d948b532@news.astraweb.com> References: <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0db96$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > For example, pressing delete when there is no text to delete should just > silently do nothing. That's really just a matter of deciding what should count as an error and what shouldn't. You've decided that "pressing Delete when there's nothing to delete does nothing" is a reasonable thing to include in the specifications of correct behaviour for the program. -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 13 20:11:47 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 14 Oct 2017 13:11:47 +1300 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: alister wrote: > in the past for large quantitys the data in a ROM chip was part of the > chip etching mask I know, I'm considering the masking process to be a kind of write operation. > The Apollo lunar lander used a magnetic core store that was hard coded at > the time it was "Woven" > > I doubt that either process is in widespread usage any longer as most > manufacturers no incorporate a way to update the firmware of a device Magnetic core technology died out long before that, due to inability to build machines that simulated armies of ladies with sewing needles! -- Greg From farjanaumme1 at gmail.com Fri Oct 13 20:39:34 2017 From: farjanaumme1 at gmail.com (farjanaumme1 at gmail.com) Date: Fri, 13 Oct 2017 17:39:34 -0700 (PDT) Subject: =?UTF-8?Q?Re=3A_Case_Solution=3A_Bratwurst=2C_Beer_and_Business_Plan?= =?UTF-8?Q?ning_for_Growth_at_Wurstk=C3=BCche_by_Thomas_Knapp=2C_Jacqueline_O?= =?UTF-8?Q?rr?= In-Reply-To: <2072594d-1ee6-4261-97b3-a5d5fd5a9921@googlegroups.com> References: <2072594d-1ee6-4261-97b3-a5d5fd5a9921@googlegroups.com> Message-ID: On Thursday, July 27, 2017 at 9:01:17 PM UTC+10, Case Solution & Analysis wrote: > Case Solution and Analysis of Bratwurst, Beer and Business: Planning for Growth at Wurstk?che by Thomas Knapp, Jacqueline Orr is available at a lowest price, send email to casesolutionscentre(at)gmail(dot)com if you want to order the Case Solution. > > Case Study ID: SCG526 > > Get Case Study Solution and Analysis of Bratwurst, Beer and Business: Planning for Growth at Wurstk?che in a FAIR PRICE!! > > Our e-mail address is CASESOLUTIONSCENTRE (AT) GMAIL (DOT) COM. Please replace (at) by @ and (dot) by . > > YOU MUST WRITE THE FOLLOWING WHILE PLACING YOUR ORDER: > Complete Case Study Name > Authors > Case Study ID > Publisher of Case Study > Your Requirements / Case Questions > > Note: Do not REPLY to this post because we do not reply to posts here. If you need any Case Solution please send us an email. We can help you to get it. From python at mrabarnett.plus.com Fri Oct 13 20:39:56 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 14 Oct 2017 01:39:56 +0100 Subject: Python 2 -> 3, urllib.urlOpen In-Reply-To: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: On 2017-10-13 23:27, Irv Kalb wrote: > One of the colleges where I teach has just moved from Python 2 to Python 3. I am in the process of converting my beginning Python class from Python 2 to Python 3. Everything has gone smoothly, until I just tried to convert some code that imports and uses urllib.urlOpen to fetch data through an API. I am using an API that I found here: http://www.jarloo.com/yahoo_finance/ > > As a minimal example, I am trying to get the latest stock price for Apple. > > The following example works perfectly in Python 2: > > import urllib > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.urlopen(fullURLWithParameters).read() > > print 'Response is: ', response > > This is asking for a stock name (s=) and I am adding in aapl as a stock symbol. I am also adding a "flag" parameter (f=) and setting it to l1 to get the last trade price. When I run this in Python 2, I see: > > Response is: 156.99 > > > If I take the same program and just modify the print statement to add parentheses, then try to run it in Python 3.6 (on a Mac): > > > import urllib > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > > I get the following: > > Traceback (most recent call last): > File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in > response = urllib.urlopen(fullURLWithParameters).read() > AttributeError: module 'urllib' has no attribute 'urlopen' > > > I've looked at the Python 3.6 documentation for urllib, and I see that certain calls have been changed and others have been eliminated. But my eyes glaze over trying to figure out what to use instead. > > My question is: Is there a simple (hopefully one or two line) replacement for my call to url lib.urlopen().read() > > I know that there are other modules out there that handle requests (like the Requests module), but this is a strictly controlled university environment. I cannot download any external packages (other then pygame, which I got special permission for) onto the computers in the school. Therefore, I'm looking for something in the Python 3.6 Standard Library. > Try: from urllib import request response = request.urlopen(fullURLWithParameters).read() Note that the response is bytes. From steve+python at pearwood.info Fri Oct 13 21:05:59 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 12:05:59 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e162f9$0$14956$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 07:15 am, Peter J. Holzer wrote: > On 2017-10-13 15:11, alister wrote: >> On Sat, 14 Oct 2017 01:48:44 +1300, Gregory Ewing wrote: >>> Steve D'Aprano wrote: >>>> I wasn't questioning where the data came from, but how the compiler can >>>> write to READ ONLY MEMORY which might not even be in the same continent >>>> as the compiler that generated the code. >>> >>> I thought it would be fairly obvious that by "put it in read-only >>> memory" I meant "arrange for it to be in a location that is read-only at >>> run time". Obviously it can't be read-only at *compile* time, just as a >>> physical ROM can't be read-only at manufacture time. >> >> oh yes it can >> in the past for large quantitys the data in a ROM chip was part of the >> chip etching mask (unless you consider a blank silicon wafer to be >> "Programmable" by the etching process)rather than prom which used >> programmable fuses or prom which could be erased by UV light (assuming >> the chip was fitted with a window otherwise it was known as one time >> programmable EPROM) > > He didn't say "programmable". He said that the is "not read-only". In context, we are talking about a computer program (the compiler) writing data to memory. When we talk about programs writing data to memory, only certain types of actions are included, and a distant factory etching silicon chips is not usually one of them. The question is, does the process of manufacturing a silicon chip count as *writing*? I don't think so. Not every side-effect should be described as writing: if the compiler's output was fed into a robot that used it to assemble a car, I'm sure you would not want to say that the compiler wrote a car. That would be a misuse of language. I think the idea of writing a ROM chip is equally a misuse of language. Your example of writing on paper is a good one, but it argues *against* your position, not for it. If you compile some code, then take a hex dump and print it out: gcc program.c xxd a.out | lp would you describe the process as "the compiler writes to the piece of paper"? I don't think that is justified. The compiler isn't writing to the paper, the printer is printing to the paper. I would accept the description "lp writes to the printer", but that's as far as it goes. Instead of using a printer, perhaps you pipe the output to less and then spend the next hour laboriously hand-writing the hex dump out. (Perhaps you lost a bet, or you are playing a game of Truth Or Dare.) Do you still want to claim that the compiler did the writing? If you email the hex dump to a person on the other side of the world, who then post-processes the file in some unknown fashion, and passes the processed file to another machine, which uses that as input to an industrial process which etches wafers of silicon and manufacturers a ROM chip, do you still think it is reasonable to describe it as the *compiler* doing *writing*? I maintain the choice of words is wrong on both counts: the compiler is not the entity doing the work, and the work of manufacturing a ROM chip should not be described as writing. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 13 21:22:32 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 12:22:32 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0ce91$0$14959$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e166d9$0$14962$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 01:51 am, Chris Angelico wrote: > On Sat, Oct 14, 2017 at 1:32 AM, Steve D'Aprano > wrote: >> It seems to me that you're not talking about ROM at all, but ordinary RAM. >> Then what do you mean by "read only"? A block of memory with a flag that >> says "unprivileged processes are prohibited from writing here"? >> >> (That's also not a rhetorical question.) > > When I first learned about Protected Mode (as defined by the Intel > 80386 and used in OS/2), there was a real concept of read-only RAM. > The program loader would fetch up the executable file (data on the > disk) and construct its segments: code, data, and BSS/stack. The data, > BSS, and stack all end up as a single segment (data is what comes > straight off the disk, BSS is all zeroes initially, and stack is > uninitialized initially, but ultimately they're all read/write), and > code is in its own segment. When control is handed to the new process, > its segment table grants it read/write access to the data/stack > segment, but read-only access to its code segment. > Within that > process, the code really truly is read-only, unless some magic is > done. So... not actually read-only then? You have the program loader writing to it, repeatedly, every time you load a program. And even the unprivileged program itself can write to it, if it knows the correct incantation to use. So its more like Read/Write Memory, or as we normally call it, memory. You know, an awful lot of confusion could have been avoided if people had referred to write-protected memory as write-protected rather than read-only. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 13 21:44:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 12:44:18 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e0db96$0$14974$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e16bf3$0$14952$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 07:04 am, Peter J. Holzer wrote: > On 2017-10-13 15:28, Steve D'Aprano wrote: >> On Sat, 14 Oct 2017 01:30 am, Chris Angelico wrote: >>>> It's just a technique, like greying out certain menu options - clicking >>>> them will do nothing, but you won't get an error message and you don't >>>> want to allow them anyway, risking more serious consequences. >>> >>> Even there, you often CAN get a report about the failure; clicking on >>> something that's disabled will cause a short beep, unless that's >>> disabled. Maybe you personally don't like that beep, and you'd rather >>> it be suppressed - but that's the end user's choice, NOT the >>> programmer's. "Errors should never pass silently, unless explicitly >>> silenced." >> >> Over 30 years since Apple first put out their user interface guidelines, >> and people still get the basics wrong. >> >> "Errors should never pass silently..." is excellent advise for programming, >> but not necessarily UIs. With a program, errors in the code represent bugs >> that should be fixed, and in principle we can remove the bugs one by one >> until there are none left, after which the program will be perfect. (Yeah, >> try not to laugh too much.) >> >> But in a UI, that will never be the case. Errors are not bugs, and can't be >> removed. > > It is also not a bug if - for example - a file cannot be opened, or is > malformed. But you have to tell the user what went wrong. Pretending > that all went well won't make the user happy (or at least that happyness > will be short-lived). I don't disagree with that -- I never said that ALL errors should pass silently. That would be silly. I said that UIs should be tolerant of *unimportant* errors. Being unable to open a file is clearly not unimportant. And besides, that's not really a UI error. Its either a programming error: say, the code tries to call open("filename", "t") or its an unavoidable environmental error (the file actually is corrupt, the disk is failing, the user doesn't have read permission for the file). Either case should be reported. >> All you can do is hope the user doesn't do it again, which they >> will. So UIs should follow a slightly different ideal: >> >> "Be tolerant of unimportant errors, and let them pass silently." > [...] >> Likewise, pressing the left arrow key when you are already at the start of >> the text should just silently remain at the start, not chastise the user. > [... other examples elided ...] >> But in fact, clicking a disabled button or menu item is not an error. > > If it is not an error, clearly the rule does not apply. Indeed. That was my position: since clicking a disable menu item is not an error, there's no need to treat it as an error. The action of disabling the menu item or the button is an explicit way of saying: temporary turn off the function of this item, and make clicking it harmless and once you have done so, there's no error condition to deal with because clicking it does nothing. > So it depends on > whether the action is an error or not (actually in the case of the > disabled button I would tend to think it is an error: I can't think of a > reason why anybody would intentionally click on a disabled button[1]). Why do they need a reason? Maybe they just like to click on things. Since clicking on disabled items is harmless, why elevate it to the state of an error? Maybe they're beginners to computers, and haven't yet learned the difference between active controls that respond when you click them, and inactive ones which do not respond. We all have to learn that, just as we learn which entities respond to voice control, and which do not: "Mummy, I'm thirsty, get me a glass of water please!" "Whiskers, I'm thirsty, get me a glass of water please!" The lack of response from the cat is enough to teach the user that asking your cat is not going to be effective, and the lack of any feedback from a disabled item likewise will quickly teach them that greyed out (disabled) items are inactive and don't respond to clicks. There's no need to treat it as an error and chastise the user, that just makes your program hostile and less usable. > Pressing the left arrow key when you are already at the start of the > line is good example. The user might expect the cursor to jump the end > of the previous line, I certainly do. But that's why I specified the beginning of the text, not the beginning of the line. There's no previous line to go back to. > so if your program doesn't do that you probably > should give some feedback. Or the user might just have pressed the key a > bit too long, then the cursor should just stop there ("vi has two modes: > One beeps a lot and the other doesn't"). Also known as "annoy the user" and "sensible" modes :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From grant.b.edwards at gmail.com Fri Oct 13 22:52:06 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 14 Oct 2017 02:52:06 +0000 (UTC) Subject: Lies in education [was Re: The "loop and a half"] References: <1507148251l.21889118l.0l@psu.edu> <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-14, Gregory Ewing wrote: > >> I doubt that either process is in widespread usage any longer as >> most manufacturers no incorporate a way to update the firmware of a >> device > > Magnetic core technology died out long before that, due to > inability to build machines that simulated armies of ladies > with sewing needles! I interviewed at a company that was still making core memory in 1989. -- Grant From steve+python at pearwood.info Fri Oct 13 23:46:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 14:46:38 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> <871sm7b9z1.fsf@elektro.pacujo.net> Message-ID: <59e188a1$0$14963$b1db1813$d948b532@news.astraweb.com> On Fri, 13 Oct 2017 09:47 pm, Marko Rauhamaa wrote: > "Peter J. Holzer" : > >> On 2017-10-13 05:28, Gregory Ewing wrote: >>> Not only does "byte" not always mean "8 bits", but >>> "char" isn't always short for "character"... >> >> True. > > Well, it does, in my universe. That was cast in stone 10**-32 seconds > after the Big Bang. You've never had char-grilled peppers? *wink* -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Oct 14 00:08:56 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 15:08:56 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e188a1$0$14963$b1db1813$d948b532@news.astraweb.com> References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59df59f2$0$14939$b1db1813$d948b532@news.astraweb.com> <59dff951$0$14972$b1db1813$d948b532@news.astraweb.com> <871sm7b9z1.fsf@elektro.pacujo.net> <59e188a1$0$14963$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 2:46 PM, Steve D'Aprano wrote: > On Fri, 13 Oct 2017 09:47 pm, Marko Rauhamaa wrote: > >> "Peter J. Holzer" : >> >>> On 2017-10-13 05:28, Gregory Ewing wrote: >>>> Not only does "byte" not always mean "8 bits", but >>>> "char" isn't always short for "character"... >>> >>> True. >> >> Well, it does, in my universe. That was cast in stone 10**-32 seconds >> after the Big Bang. > > You've never had char-grilled peppers? *wink* Yeah. They build character. *wink back* ChrisA From dieter at handshake.de Sat Oct 14 02:37:15 2017 From: dieter at handshake.de (dieter) Date: Sat, 14 Oct 2017 08:37:15 +0200 Subject: Return str to a callback raise a segfault if used in string formating References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> Message-ID: <87po9qi6as.fsf@handshake.de> Thomas Jollans writes: > On 2017-10-13 11:07, Vincent Vande Vyvre wrote: >> Le 13/10/17 ? 09:23, Chris Angelico a ?crit?: >>> On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre > ... > If you have custom C code, it's likely that there is a memory management > problem there. It's not unusual for incorrect memory management to cause > problems in a completely different part of the code, where something > tries to access freed memory or something. It is very easy to get something wrong in custom C code. Therefore, I typically use "cython" (a compiler translating extended Python to C) to realize such code. It takes care of almost all subleties. From hjp-usenet3 at hjp.at Sat Oct 14 05:26:44 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sat, 14 Oct 2017 11:26:44 +0200 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e162f9$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-14 01:05, Steve D'Aprano wrote: > On Sat, 14 Oct 2017 07:15 am, Peter J. Holzer wrote: >> On 2017-10-13 15:11, alister wrote: >>> On Sat, 14 Oct 2017 01:48:44 +1300, Gregory Ewing wrote: >>>> Steve D'Aprano wrote: >>>>> I wasn't questioning where the data came from, but how the compiler can >>>>> write to READ ONLY MEMORY which might not even be in the same continent >>>>> as the compiler that generated the code. >>>> >>>> I thought it would be fairly obvious that by "put it in read-only >>>> memory" I meant "arrange for it to be in a location that is read-only at >>>> run time". Obviously it can't be read-only at *compile* time, just as a >>>> physical ROM can't be read-only at manufacture time. >>> >>> oh yes it can >>> in the past for large quantitys the data in a ROM chip was part of the >>> chip etching mask (unless you consider a blank silicon wafer to be >>> "Programmable" by the etching process)rather than prom which used >>> programmable fuses or prom which could be erased by UV light (assuming >>> the chip was fitted with a window otherwise it was known as one time >>> programmable EPROM) >> >> He didn't say "programmable". He said that the is "not read-only". > > In context, we are talking about a computer program (the compiler) writing > data to memory. No, I don't think so. You keep talking about that, even though Gregory hasn't used those exact words and has since clarfified that he didn't mean it that way. Y'know, I'm all for writing as carefully und unambiguously as possible and for discussing questionable wording (especially in a large forum where many (including me) will have a different native language), but we all are sloppy in our writing every now and then and insisting that somebody must have meant X' because they wrote X doesn't really help and gets boring pretty soon. Right now I was replying to alister's claim that "yes [ROMs] can [be read-only at manufacture time]". That doesn't have anything with compilers. Where the information came from is irrelevant to the question whether ROMS are read-only during manufacture. Certainly they are not only read (I don't know if they are read at all, but I guess they are during quality control), but information is put onto them. I think that this process can be called "writing" (It's a lot closer to what has been called "writing" for the last few millenia than what happenes to a RAM chip during "writing"), but even that is a red herring as the word "write" wasn't actually used (only "read"). > Your example of writing on paper is a good one, but it argues *against* your > position, not for it. If you compile some code, then take a hex dump and > print it out: > > gcc program.c > xxd a.out | lp > > would you describe the process as "the compiler writes to the piece of paper"? No, but the printer does. And that means that the paper is not read-only. Which was the point of the exchange between Gregory and alister. The compiler isn't in the game any more. But there is a difference between printing the hex dump of the output of the compiler and loading it into memory to be executed. The latter is the intended purpose of the file. When a compile writes some data into the ".rodata" section of the file, the intention is that this data should be read-only at run-time. Therefore I consider the phrasing "the compiler puts it into read-only memory", while certainly sloppy and not technically correct, perfectly comprehensible. And I can't find anything wrong with his revised phrasing "arrange for it to be in a location that is read-only at run time". That's exactly what the compiler does. That the compiler doesn't actually have control over what happens to the file afterwards is immaterial. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From hjp-usenet3 at hjp.at Sat Oct 14 05:41:08 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sat, 14 Oct 2017 11:41:08 +0200 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: On 2017-10-13 21:42, Ben Bacarisse wrote: > That's one way to put it. Another is that to use Python I need to buy a > new service that is already configured. That's exactly the same for PHP. You can't use that either unless somebody configured to server to use it. The difference is that lots of providers started configuring their servers for use of PHP in the late 1990s, but didn't do that for Python. > If that's the way it's done, fine, but this sub-thread started with > someone being surprised by the success of PHP. Which probably boils down to the question: Why did providers offer PHP and not Python? One reason might be that at the time no suitable web framework for Python existed (Zope was released in 1999, and I remember it to be rather heavy-weight). One reason might be that providers didn't see PHP as a "real" programming language and therefore deemed it safer. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From alister.ware at ntlworld.com Sat Oct 14 05:51:40 2017 From: alister.ware at ntlworld.com (alister) Date: Sat, 14 Oct 2017 09:51:40 GMT Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) Message-ID: On Sat, 14 Oct 2017 11:41:08 +0200, Peter J. Holzer wrote: > On 2017-10-13 21:42, Ben Bacarisse wrote: >> That's one way to put it. Another is that to use Python I need to buy >> a new service that is already configured. > > That's exactly the same for PHP. You can't use that either unless > somebody configured to server to use it. > > The difference is that lots of providers started configuring their > servers for use of PHP in the late 1990s, but didn't do that for Python. > >> If that's the way it's done, fine, but this sub-thread started with >> someone being surprised by the success of PHP. > > Which probably boils down to the question: Why did providers offer PHP > and not Python? One reason might be that at the time no suitable web > framework for Python existed (Zope was released in 1999, and I remember > it to be rather heavy-weight). One reason might be that providers didn't > see PHP as a "real" programming language and therefore deemed it safer. > > hp could it have been that it was simply a default Apache module installed by whatever version of Linux was being used by the host? -- PENGUINICITY!! From steve+python at pearwood.info Sat Oct 14 07:12:07 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 14 Oct 2017 22:12:07 +1100 Subject: Lies in education [was Re: The "loop and a half"] References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e162f9$0$14956$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e1f109$0$14953$b1db1813$d948b532@news.astraweb.com> On Sat, 14 Oct 2017 08:26 pm, Peter J. Holzer wrote: > On 2017-10-14 01:05, Steve D'Aprano wrote: >> In context, we are talking about a computer program (the compiler) writing >> data to memory. > > No, I don't think so. You keep talking about that, even though Gregory > hasn't used those exact words and has since clarfified that he didn't > mean it that way. But without clarifying what he meant. I've had to piece that together from bits and pieces of random comments from other people. I still don't know exactly what Greg intended to say, only what I have inferred. Initially Greg said: "If the compiler can tell where p is initially pointing, it could put the pointer in read-only memory." and when I asked: "If it's read-only, how can the compiler write to it?" instead of clarifying what he meant, Greg defended his misleading choice of words with: "So, the factory is allowed to write to it. Possibly it's writing data that came from... a compiler?" So Greg himself has used the term "write" to describe the scenario he describes, it is not just me. It has taken, what, I guess something like a dozen or more back and forward posts before Chris happened to casually mention that the *program loader* (not compiler!) writes the data to *write-protected memory* (not ROM). If only people had said that in the first place, instead of reinforcing my misunderstanding by arguing whether or not the act of etching paths into a silicon chip at a factory counts as the compiler writing. We're all human and we all sometimes use sloppy phrasing. I know I do. But the least we can do is try to be more aware of when we're causing more confusion than insight, and instead of doubling down and defending the sloppy phrasing, try to explain what we mean. > Y'know, I'm all for writing as carefully und > unambiguously as possible and for discussing questionable wording > (especially in a large forum where many (including me) will have a > different native language), but we all are sloppy in our writing every > now and then and insisting that somebody must have meant X' because they > wrote X doesn't really help and gets boring pretty soon. The problem is not the initial miscommunication. The problem is the insistence on defending that miscommunication even in the face of confusion. Right from my first comment on this topic, I said: "I come from the days when ROM was actual ROM, burned in at the factory." because I already suspected that Greg was using "read-only" to mean something other than read-only, but what I did not know. Now maybe I could have been more explicit by asking "What the hell do you mean by read-only memory, if you're not talking about actual ROM?" and the fact that I didn't is *my* failure. (Mea culpa.) But be reasonable, I had just asked almost exactly the same question not one line earlier: "If it's read-only, how can the compiler write to it?" and (as far as I can see) *nobody* thought to actually answer my question until perhaps a dozen posts later, when Chris more-or-less said: - the compiler doesn't, but the program loader does; - its not so much read-only memory as write-protected memory: privileged code can still write to it. (Have I got the details right? Or at least close enough?) [...] > Therefore I consider the phrasing "the > compiler puts it into read-only memory", while certainly sloppy and not > technically correct, perfectly comprehensible. And here we go again. Instead of holding the author responsible for the poor phrasing, the reader is blamed for not somehow inferring the author's intended meaning. "Somebody misunderstood Greg's sloppy and incorrect words to mean something Greg didn't intend? And when given the opportunity to clarify, Greg failed to do so and continued using the sloppy and incorrect phrasing? Oh, that's not Greg's fault, it is the fault of the reader for not reading Greg's mind and knowing what he really meant. I shall continue to defend the choice of words which are LITERALLY WRONG rather than accept that somebody might have legitimately misunderstood them." > And I can't find anything > wrong with his revised phrasing "arrange for it to be in a location that > is read-only at run time". That's exactly what the compiler does. Except *read-only* is a misleading and poor way to describe it and it too is LITERALLY WRONG. The program loader can write to it, and does so every single time a program is run. Why is it called read-only when it isn't read-only or even Write Once Read Many? I am sure that nobody, including Greg and yourself, would describe /root on a standard Linux/Unix system as "a read only directory" just because non-root processes can't write to it. You'd probably describe it as write-protected, or similar terminology. And if somebody asked "How is it read-only, root can write to it?" I surely hope you wouldn't double-down and insist that, no, it really, truly is read-only even though root can write to it. I don't know, maybe calling this chunk of memory "read-only" is the accepted term of art for those who know about program loaders and the gory details of how C compilers arrange memory and how code is run on x86 hardware. It wouldn't be the first time that the terminology to describe some technology was misleading to those who don't know the jargon, and I'm sure that it won't be the last time. But I'm not one of those people, and I'm sure that was obvious from my questions. Is it really so hard to expect that somebody who did understand what was going on to explain my misapprehension rather than argue over the precise details of what counts as writing and whether or not a compiler can be said to write to a silicon chip being manufactured in a factory half a world away? (By the way, thanks Chris.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben.usenet at bsb.me.uk Sat Oct 14 07:16:02 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 14 Oct 2017 12:16:02 +0100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> Message-ID: <87a80uf099.fsf@bsb.me.uk> "Peter J. Holzer" writes: > On 2017-10-13 21:42, Ben Bacarisse wrote: >> That's one way to put it. Another is that to use Python I need to buy a >> new service that is already configured. > > That's exactly the same for PHP. You can't use that either unless > somebody configured to server to use it. I was not using "I" generically. *I* don't need to buy a service configured to use PHP because: > The difference is that lots of providers started configuring their > servers for use of PHP in the late 1990s, but didn't do that for Python. > >> If that's the way it's done, fine, but this sub-thread started with >> someone being surprised by the success of PHP. > > Which probably boils down to the question: Why did providers offer PHP > and not Python? One reason might be that at the time no suitable web > framework for Python existed (Zope was released in 1999, and I remember > it to be rather heavy-weight). One reason might be that providers didn't > see PHP as a "real" programming language and therefore deemed it > safer. That would be deeply ironic, given the security pain that it has turned out to be! -- Ben. From rosuav at gmail.com Sat Oct 14 08:34:39 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 23:34:39 +1100 Subject: Lies in education [was Re: The "loop and a half"] In-Reply-To: <59e1f109$0$14953$b1db1813$d948b532@news.astraweb.com> References: <59d58de3$0$14956$b1db1813$d948b532@news.astraweb.com> <59dc6c99$0$14955$b1db1813$d948b532@news.astraweb.com> <874lr689zq.fsf@elektro.pacujo.net> <59e04c2c$0$14962$b1db1813$d948b532@news.astraweb.com> <59e09760$0$14950$b1db1813$d948b532@news.astraweb.com> <59e162f9$0$14956$b1db1813$d948b532@news.astraweb.com> <59e1f109$0$14953$b1db1813$d948b532@news.astraweb.com> Message-ID: On Sat, Oct 14, 2017 at 10:12 PM, Steve D'Aprano wrote: > But be reasonable, I had just asked almost exactly the same question not one > line earlier: > > "If it's read-only, how can the compiler write to it?" > > and (as far as I can see) *nobody* thought to actually answer my question > until perhaps a dozen posts later, when Chris more-or-less said: > > - the compiler doesn't, but the program loader does; > > - its not so much read-only memory as write-protected memory: > privileged code can still write to it. > > > (Have I got the details right? Or at least close enough?) Pretty close. There's a lot of levels of indirection involved. When a program asks to manipulate a piece of memory, it uses a segment selector and a memory location (offset); in most modern systems, a flat memory model is used, where all identical offsets mean the same piece of memory, so we tend to think of the offset as being the entire pointer. Those segment selectors refer to something in the segment table for *that process*. That means that location 0x12345678 in my process could be a completely different hunk of memory to location 0x12345678 in your process. (It's also possible that they're the same - loading shared modules (DLL/SO) often just means mapping an already-loaded block of memory into your process.) Your process is completely unaware of any memory that it doesn't have access to. It also has no way to violate the notion of "read-only", because that's a flag set on the memory pages in that process's page table. So when the program loader goes to work, it loads up a page of memory from the disk, then grants this process read-only access to it, by mapping that page into the process's memory space. The program loader, being part of the kernel, isn't just "privileged" in the sense of running as root/Administrator - it is the one that determines what "root" even means. If root is able to manipulate a process's page tables, it's only via kernel APIs. As such, the concept of "read-only" is actually hard-wired in the process's memory map. There's no way to violate that. (I may have some details wrong, as it's a long time since I've worked with any of this. Also, this is massively simplified. But I think it's about right.) > [...] >> Therefore I consider the phrasing "the >> compiler puts it into read-only memory", while certainly sloppy and not >> technically correct, perfectly comprehensible. > > And here we go again. Instead of holding the author responsible for the poor > phrasing, the reader is blamed for not somehow inferring the author's > intended meaning. > > "Somebody misunderstood Greg's sloppy and incorrect words to mean something > Greg didn't intend? And when given the opportunity to clarify, Greg failed to > do so and continued using the sloppy and incorrect phrasing? Oh, that's not > Greg's fault, it is the fault of the reader for not reading Greg's mind and > knowing what he really meant. I shall continue to defend the choice of words > which are LITERALLY WRONG rather than accept that somebody might have > legitimately misunderstood them." Saying that the compiler puts something into read-only memory is skipping a lot of steps in the procedure - notably, it's making a temporal jump right from compilation time to run time. Obviously the data is in some form of read/write storage in between (eg a hard drive), during which it can very well be mutated. So it's somewhat defensible to consider that the compiler sorta-kinda places things into the run-time process's memory (since the compiler (or the linker) is what instructs the program loader how to arrange the process memory tables), while still being reasonable to argue that, since the compiler is writing to it, it must be read/write (which it is - in the compiler's memory space). The distinction here is that there's been a change in POV; each time, we're talking from the POV of the current process, but that's a different process (compiler vs running program). >> And I can't find anything >> wrong with his revised phrasing "arrange for it to be in a location that >> is read-only at run time". That's exactly what the compiler does. > > Except *read-only* is a misleading and poor way to describe it and it too is > LITERALLY WRONG. The program loader can write to it, and does so every single > time a program is run. Why is it called read-only when it isn't read-only or > even Write Once Read Many? It *is* read-only, from the POV of the running process. Nothing (other than actual burned ROM) is truly read-only; possibly the flashable BIOS is read-only from the POV of a running system, but I'm not even sure of that. From the POV of the kernel, all RAM is read/write (I think; there might be some exceptions, but certainly everything we're looking at here is); from the POV of various processes, large slabs of memory are read-only. > I am sure that nobody, including Greg and yourself, would describe /root on a > standard Linux/Unix system as "a read only directory" just because non-root > processes can't write to it. You'd probably describe it as write-protected, > or similar terminology. > > And if somebody asked "How is it read-only, root can write to it?" I surely > hope you wouldn't double-down and insist that, no, it really, truly is > read-only even though root can write to it. Right. Directories are actually read/write, but the nature of Unix file systems is that you can't modify something you don't have write permission for. That's "write-protected". A closer analogy would be chrooting; once a process has been chrooted, it's completely unable to see anything outside its jail (modulo exploits and bugs), and it's as if they don't exist. I believe you can affect a process's mounts such that something is mounted read-only for that process, even though it's read/write for everyone else. That would be similar. > I don't know, maybe calling this chunk of memory "read-only" is the accepted > term of art for those who know about program loaders and the gory details of > how C compilers arrange memory and how code is run on x86 hardware. It > wouldn't be the first time that the terminology to describe some technology > was misleading to those who don't know the jargon, and I'm sure that it won't > be the last time. > > But I'm not one of those people, and I'm sure that was obvious from my > questions. Is it really so hard to expect that somebody who did understand > what was going on to explain my misapprehension rather than argue over the > precise details of what counts as writing and whether or not a compiler can > be said to write to a silicon chip being manufactured in a factory half a > world away? > > (By the way, thanks Chris.) You're welcome. I'm hesitant to post with too much confidence here, as I have a small smattering of experience with these things on a variety of different systems, and in-depth experience on none. But hopefully the above details will clear things up a bit - either by themselves being useful, or by more-experienced people correcting me, which is all to the good :) ChrisA From rosuav at gmail.com Sat Oct 14 08:39:03 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 14 Oct 2017 23:39:03 +1100 Subject: Heroku (was Re: Lies in education [was Re: The "loop and a half"]) In-Reply-To: <87a80uf099.fsf@bsb.me.uk> References: <87infkhscc.fsf@bsb.me.uk> <87mv4ug1ws.fsf@bsb.me.uk> <87a80uf099.fsf@bsb.me.uk> Message-ID: On Sat, Oct 14, 2017 at 10:16 PM, Ben Bacarisse wrote: > "Peter J. Holzer" writes: >> Which probably boils down to the question: Why did providers offer PHP >> and not Python? One reason might be that at the time no suitable web >> framework for Python existed (Zope was released in 1999, and I remember >> it to be rather heavy-weight). One reason might be that providers didn't >> see PHP as a "real" programming language and therefore deemed it >> safer. > > That would be deeply ironic, given the security pain that it has turned > out to be! Yup. And not exactly surprising to any security expert. The history of computing - well, let's face it, the history of mankind - is littered with stories of "this is simple and easy, we don't need to secure it" turning into "this is actually a major problem". Sometimes we can retrofit enough protection onto the system without fundamentally breaking it (eg DNS, where a variety of forms of security have been added); other times, we learn a new best-prac and keep going (eg parameterized queries rather than risking SQL injection, which some people still haven't learned, but a lot have); and other times, we scrap the bad option and start a completely new way of doing things (bye bye Java applets, bye bye Flash, let's do everything with JS), which of course isn't necessarily perfect either, but is usually a big enough advantage to be worth it. ChrisA From thatebart at gmail.com Sat Oct 14 09:43:53 2017 From: thatebart at gmail.com (Bart Thate) Date: Sat, 14 Oct 2017 06:43:53 -0700 (PDT) Subject: BOTLIB #31 released - https://pypi.python.org/pypi/botlib Message-ID: BOTLIB #31 released - https://pypi.python.org/pypi/botlib BOTLIB is a python3 framework to use if you want to program CLI, IRC or XMPP bots. features ======== BOTLIB provides the following features: :: object class save/load to/from a json file. rest server serve saved object?s over http. rss fetcher echo rss feeds to irc channels. udp server udp to bot to irc channel. watcher server run tail -f and have output send to IRC channel. email scanning scan mbox format to searchable botlib objects. json backend objects are stored as json string in files on the fs. db iteration over stored objects. timestamp time based filenames gives logging capabilities future future sensors should provide entry to the logger. install ======= Clone the source: :: bart at okdan:~$ hg clone https://bitbucket.org/bthate/botlib You might need to do one of the following if the bot doesn't work: bart at okdan:~/botlib$ export PYTHONPATH="." bart at okdan:~/botlib$ export PYTHONIOENCODING="utf-8" Another option is to download with pip3 and install globally: :: bart at okdan:~$ pip3 install botlib --upgrade irc === Use -n , -s , -c options to make the bot join the network: :: bart at okdan:~$ bot -i irc -n botlib -s irc.freenode.net -c \#botlib You can use the -w option to write config values to ~/.bot/config/irc xmpp ==== For the xmpp server use a ~/.sleekpass file with the password in it: :: bart at okdan:~$ cat "password" > ~/.sleekpass bart at okdan:~$ bot -i xmpp,rss --room=test at conference.localhost users ===== One needs to add a users origin to be able to give the bot commands. One can add a user with the meet command: :: bart at okdan:~$ bot meet user at server user user at server created To give the user a permission you can use the perm command: :: bart at okdan:~$ bot perm user at server ps ok user at server The u command show the json dump of a user object: :: bart at okdan:~$ bot u user at server { "_container": { "default": "", "path": "/home/bart/.bot/user/2017-10-12/21:05:52.095737", "prefix": "object", "saved": "Thu Oct 12 21:07:03 2017", "signature": "c113c4125f8c2a88d5b312e283792ae019c61a52" }, "_type": "", "origin": "user at server", "perms": [ "USER", "PS" ], "user": "user at server" } The default shell user is root at shell and give all the commands that are available. commands ======== The following commands are available: :: alias key, value alias. announce announce text on all channels in fleet. begin begin stopwatch. cfg edit config files. cmnds show list of commands. deleted show deleted records. delperm delete permissions of an user. dump dump objects matching the given criteria. edit edit and save objects. end stop stopwatch. exit stop the bot. fetcher fetch all rss feeds. find present a list of objects based on prompt input. first show the first record matching the given criteria. fix fix a object by loading and saving it. idle see how long a channel/nick has been idle. last show last objectect matching the criteria. license display BOTLIB license. load force a plugin reload. log log some text. loglevel set loglevel. loud disable silent mode of a bot. ls show subdirs in working directory. man show descriptions of the available commands. mbox convert emails to botlib objects. meet create an user record. nick change bot nick on IRC. perm add/change permissions of an user. permissions show permissions granted to a user. perms show permission of user. pid show pid of the BOTLIB bot. ps show running threads. reboot reboot the bot, allowing statefull reboot (keeping connections alive). reload reload a plugin. restore set deleted=False in selected records. rm set deleted flag on objects. rss add a rss url. save make a kernel dump. shop add a shopitem to the shopping list. show show dumps of basic objects. silent put a bot into silent mode. start start a plugin. stop stop a plugin. synchronize synchronize rss feeds (fetch but don't show). test echo origin. timer timer command to schedule a text to be printed on a given time. stopwatch to measure elapsed time. today show last week's logged objects. todo log a todo item. tomorrow show todo items for tomorrow. u show user selected by userhost. uptime show uptime. version show version. w show user data. watch add a file to watch (monitor and relay to channel). week show last week's logged objects. whoami show origin. yesterday show last week's logged objects. modules ======= The following modules are available: :: botlib.bot bot base class. botlib.cli command line interfacce bot, gives a shell prompt to issue bot commands. botlib.clock timer, repeater and other clock based classes. botlib.cmnds botlib basic commands. botlib.compose construct a object into it?s type. botlib.decorator method decorators botlib.db JSON file db. botlib.engine select.epoll event loop, easily interrup_table esp. versus a blocking event loop. botlib.error botlib exceptions. botlib.event event handling classes. botlib.fleet fleet is a list of bots. botlib.handler schedule events. botlib.irc IRC bot class. botlib.kernel program boot and module loading. botlib.launcher a launcher launches threads (or tasks in this case). botlib.log log module to set standard format of logging. botlib.object JSON file backed object with dotted access. botlib.raw raw output using print. botlib.rss rss module. botlib.selector functions used in code to select what objects to use. botlib.task adapted thread to add extra functionality to threads. botlib.trace functions concering stack trace. botlib.users class to access user records. botlib.xmpp XMPP bot class. botlib.register object with list for multiple values. botlib.rest rest interface. botlib.runner threaded loop to run tasks on. botlib.space central module to store objects in. botlib.static static definitions. botlib.template cfg objects containing default values for various services and plugins. botlib.test plugin containing test commands and classes. botlib.udp relay txt through a udp port listener. botlib.utils lib local helper functions. botlib.url functions that fetch data from url. botlib.watcher watch files. botlib.worker worker thread that handles submitted jobs through Worker.put(func, args, kwargs). botlib.wisdom wijsheid, wijs ! contact ======= | Bart Thate | botfather on #dunkbot irc.freenode.net | bthate at dds.nl, thatebart at gmail.com BOTLIB has a MIT license` From stefan_ml at behnel.de Sat Oct 14 09:59:03 2017 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 14 Oct 2017 15:59:03 +0200 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Message-ID: Vincent Vande Vyvre schrieb am 13.10.2017 um 13:18: > Le 13/10/17 ? 12:39, Paul Moore a ?crit?: >> As a specific suggestion, I assume the name of the created file is a >> string object constructed in the C extension code, somehow. The fact >> that you're getting the segfault with some uses of that string >> (specifically, passing it to %-formatting) suggests that there's a bug >> in the C code that constructs that string. That's where I'd start by >> looking. Absolutely. > That was my first idea, because I can verify the instance of PyUnraw is not > destroyed when I use the file name, but I was in trouble by the usage of > the file name in string formatting. > > For example I can use the file name into the slot i.e. shutil.copy(fname, > "path/renamed.tiff") > The file is correctly copied. > > In fact, I can do anything with the file name except use it in string > formatting, then your approach is probably a good way. > > Into the CPython part I have a c-string pythonized by: > ??? temp = self->outfname; > ??? self->outfname = PyUnicode_FromString(ofname); > ??? Py_XDECREF(temp); > > and exposed to Python with: > static PyMemberDef PyUnraw_members[] = { > ??? {"out_file", T_OBJECT_EX, offsetof(PyUnraw, outfname), 0, > ???? "Path of the decoded file"}, One more suggestion, in case this is actually your own C code: it's much easier to write extension modules in Cython than in plain C and C-API code. Much easier. There is a learning curve, sure, but it unburdens you from so many pitfalls and boilerplate code that it's always worth switching. And you also get faster code as a side-effect. How's that for a tradeoff. :) Stefan (Cython core developer) From vincent.vande.vyvre at telenet.be Sat Oct 14 11:06:03 2017 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Sat, 14 Oct 2017 17:06:03 +0200 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Message-ID: Le 14/10/17 ? 15:59, Stefan Behnel a ?crit?: > Vincent Vande Vyvre schrieb am 13.10.2017 um 13:18: >> Le 13/10/17 ? 12:39, Paul Moore a ?crit?: >>> As a specific suggestion, I assume the name of the created file is a >>> string object constructed in the C extension code, somehow. The fact >>> that you're getting the segfault with some uses of that string >>> (specifically, passing it to %-formatting) suggests that there's a bug >>> in the C code that constructs that string. That's where I'd start by >>> looking. > Absolutely. > > >> That was my first idea, because I can verify the instance of PyUnraw is not >> destroyed when I use the file name, but I was in trouble by the usage of >> the file name in string formatting. >> >> For example I can use the file name into the slot i.e. shutil.copy(fname, >> "path/renamed.tiff") >> The file is correctly copied. >> >> In fact, I can do anything with the file name except use it in string >> formatting, then your approach is probably a good way. >> >> Into the CPython part I have a c-string pythonized by: >> ??? temp = self->outfname; >> ??? self->outfname = PyUnicode_FromString(ofname); >> ??? Py_XDECREF(temp); >> >> and exposed to Python with: >> static PyMemberDef PyUnraw_members[] = { >> ??? {"out_file", T_OBJECT_EX, offsetof(PyUnraw, outfname), 0, >> ???? "Path of the decoded file"}, > One more suggestion, in case this is actually your own C code: it's much > easier to write extension modules in Cython than in plain C and C-API code. > Much easier. There is a learning curve, sure, but it unburdens you from so > many pitfalls and boilerplate code that it's always worth switching. And > you also get faster code as a side-effect. How's that for a tradeoff. :) > > Stefan > (Cython core developer) > Thanks, I know Cython but the code is already in C. This is a lib released as a frontend.? Then usable only in command line. It's not my own code, my job is to reimplement the main() function of the lib in CPython. The work is nearly complete, I'm in the tests part. I think I've found the problem, the string (a file path) is modified in c with "sprintf, snprintf, ..." And I plan to change that with some CPython equivalent function. Vincent From p.f.moore at gmail.com Sat Oct 14 11:28:05 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 14 Oct 2017 16:28:05 +0100 Subject: Return str to a callback raise a segfault if used in string formating In-Reply-To: References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Message-ID: On 14 October 2017 at 16:06, Vincent Vande Vyvre wrote: > I think I've found the problem, the string (a file path) is modified in c > with "sprintf, snprintf, ..." And I plan to change that with some CPython > equivalent function. Nice :-) Glad you found it. Paul From ctippur at gmail.com Sat Oct 14 12:11:48 2017 From: ctippur at gmail.com (Frustrated learner) Date: Sat, 14 Oct 2017 09:11:48 -0700 (PDT) Subject: Running flask on AWS SAM In-Reply-To: <6438a158-c7ec-404d-8bad-f880a3f897af@googlegroups.com> References: <37ce944d-4f97-430a-9aa7-33d27705ba19@googlegroups.com> <6438a158-c7ec-404d-8bad-f880a3f897af@googlegroups.com> Message-ID: <142b85b7-f2ef-452e-9206-827118e3081d@googlegroups.com> export SERVER_NAME="0.0.0.0:3000 sam local start-api" just exports everything and does not execute. I tried export SERVER_NAME="0.0.0.0:3000" sam local start-api comes back with the same error. Shekar From formisc at gmail.com Sat Oct 14 12:14:08 2017 From: formisc at gmail.com (Andrew Z) Date: Sat, 14 Oct 2017 12:14:08 -0400 Subject: Best practise for passing time as arguments In-Reply-To: References: Message-ID: Hello, I wonder what are the "best practises" for passing "time" parameters to functions? I noticed that sometimes i pass "time" as a str and then start "massaging" it into delta or i need this time format or that format. Thats quite annoying and inconsistent. I use word "time" as a general word for date/time/timedelta types. Appreciate. From marko at pacujo.net Sat Oct 14 12:40:03 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 14 Oct 2017 19:40:03 +0300 Subject: Best practise for passing time as arguments References: Message-ID: <87o9p9adjw.fsf@elektro.pacujo.net> Andrew Z : > I wonder what are the "best practises" for passing "time" parameters to > functions? > I noticed that sometimes i pass "time" as a str and then start "massaging" > it into delta or i need this time format or that format. Thats quite > annoying and inconsistent. I do the same thing. It's not all that bad because the choice depends on the needs of the application. It's a slight bit annoying that I have to write parsing functions myself, but it's not all that bad: def parse_date(date): return datetime.date(*map(int, date.split("-"))) def parse_time(time): return datetime.time(*map(int, time.split(":"))) where "date" is expressed as "2017-10-14" and "time" [of day] as "19:39". Marko From Irv at furrypants.com Sat Oct 14 13:40:39 2017 From: Irv at furrypants.com (Irv Kalb) Date: Sat, 14 Oct 2017 10:40:39 -0700 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) In-Reply-To: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: > On Oct 13, 2017, at 3:27 PM, Irv Kalb wrot > If I take the same program and just modify the print statement to add parentheses, then try to run it in Python 3.6 (on a Mac): > > ... > import urllib > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > > I get the following: > > Traceback (most recent call last): > File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in > response = urllib.urlopen(fullURLWithParameters).read() > AttributeError: module 'urllib' has no attribute 'urlopen' > > Thanks for the responses, but I still can't get it to work correctly. With MRAB's suggestion, I've modified the code to be: from urllib import request # set the Yahoo finance url, set stock name, ask for last price fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' # read all the data response = request.urlopen(fullURLWithParameters).read() print('Response is: ', response) But when I run that (Mac Python 3.6.1), I get: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect server_hostname=server_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket _context=self, _session=session) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__ self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Python3 Intro Class/Module 9 - Dictionaries & Internet (not finished)/Module 9 Files/StockQuoteYahooAPIMinimal.py", line 9, in response = request.urlopen(fullURLWithParameters).read() File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 564, in error result = self._call_chain(*args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 756, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: Using Stefan's suggestion from 2to3: import urllib.request, urllib.parse, urllib.error # set the Yahoo finance url, set stock name, ask for last price fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' # read all the data response = urllib.request.urlopen(fullURLWithParameters).read() print('Response is: ', response) I get the exact same traceback as above. Also, if I add a try/except around the call: # read all the data try: response = urllib.request.urlopen(fullURLWithParameters).read() except urllib.error.URLError as e: print(e) I get the same bottom line error from the traceback above: Huh??? I've read a bunch of documentation, and it looks like I'm doing everything right, but I cannot get this to work. Any other suggestions to get this 3 line program to work correctly? Thanks, Irv From marko at pacujo.net Sat Oct 14 13:42:06 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 14 Oct 2017 20:42:06 +0300 Subject: Best practise for passing time as arguments References: Message-ID: <87k1zxaaoh.fsf@elektro.pacujo.net> ram at zedat.fu-berlin.de (Stefan Ram): > Of course, you (the OP), should check out > > datetime.datetime > > and > > datetime.timedelta > > from the Python Library. which he mentioned in his post. Marko From tjol at tjol.eu Sat Oct 14 13:44:18 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 14 Oct 2017 19:44:18 +0200 Subject: Best practise for passing time as arguments In-Reply-To: References: Message-ID: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> On 14/10/17 19:34, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >> a post. Use whatever is appropriate in the special case >> given, or - to write a general library -, learn the design >> of a good existing library, like Time4J, first. > > Though in many cases, an ISO 8601 time string > represented by a (named )Python tuple should > be sufficient for a time stamp. > > E.g., ( year, month, day, hour, minute, seconds, > zone_offset ). > Python provides a datetime (also: date, time, timedelta) type. Use it. https://docs.python.org/3/library/datetime.html When working with time zones, the standard library needs a little help. Luckily, there's a module for that. https://pypi.python.org/pypi/pytz -- Thomas From python at mrabarnett.plus.com Sat Oct 14 14:10:23 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 14 Oct 2017 19:10:23 +0100 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) In-Reply-To: References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: <37cbda38-e51c-ee8d-7c62-6e640a1949d4@mrabarnett.plus.com> On 2017-10-14 18:40, Irv Kalb wrote: > >> On Oct 13, 2017, at 3:27 PM, Irv Kalb wrot >> If I take the same program and just modify the print statement to add parentheses, then try to run it in Python 3.6 (on a Mac): >> >> ... >> import urllib >> >> # set the Yahoo finance url, set stock name, ask for last price >> fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' >> >> # read all the data >> response = urllib.urlopen(fullURLWithParameters).read() >> >> print('Response is: ', response) >> >> I get the following: >> >> Traceback (most recent call last): >> File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in >> response = urllib.urlopen(fullURLWithParameters).read() >> AttributeError: module 'urllib' has no attribute 'urlopen' >> >> > > Thanks for the responses, but I still can't get it to work correctly. With MRAB's suggestion, I've modified the code to be: > > from urllib import request > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = request.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > > > But when I run that (Mac Python 3.6.1), I get: > > > Traceback (most recent call last): > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open > encode_chunked=req.has_header('Transfer-encoding')) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request > self._send_request(method, url, body, headers, encode_chunked) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request > self.endheaders(body, encode_chunked=encode_chunked) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders > self._send_output(message_body, encode_chunked=encode_chunked) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output > self.send(msg) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send > self.connect() > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect > server_hostname=server_hostname) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket > _context=self, _session=session) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__ > self.do_handshake() > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake > self._sslobj.do_handshake() > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake > self._sslobj.do_handshake() > ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749) > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/Python3 Intro Class/Module 9 - Dictionaries & Internet (not finished)/Module 9 Files/StockQuoteYahooAPIMinimal.py", line 9, in > response = request.urlopen(fullURLWithParameters).read() > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen > return opener.open(url, data, timeout) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 532, in open > response = meth(req, response) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 642, in http_response > 'http', request, response, code, msg, hdrs) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 564, in error > result = self._call_chain(*args) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain > result = func(*args) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 756, in http_error_302 > return self.parent.open(new, timeout=req.timeout) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open > response = self._open(req, data) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open > '_open', req) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain > result = func(*args) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open > context=self._context, check_hostname=self._check_hostname) > File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open > raise URLError(err) > urllib.error.URLError: > > > > Using Stefan's suggestion from 2to3: > > import urllib.request, urllib.parse, urllib.error > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.request.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > > I get the exact same traceback as above. > > > Also, if I add a try/except around the call: > > # read all the data > try: > response = urllib.request.urlopen(fullURLWithParameters).read() > except urllib.error.URLError as e: > print(e) > > > I get the same bottom line error from the traceback above: > > > > Huh??? > > > > I've read a bunch of documentation, and it looks like I'm doing everything right, but I cannot get this to work. Any other suggestions to get this 3 line program to work correctly? > I'm on Windows 10 and it's working for me. There's a question on StackOverflow that might help you: https://stackoverflow.com/questions/34503206/ssl-certificate-verify-failed-python From marko at pacujo.net Sat Oct 14 14:20:50 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 14 Oct 2017 21:20:50 +0300 Subject: Best practise for passing time as arguments References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> Message-ID: <87efq5a8vx.fsf@elektro.pacujo.net> Thomas Jollans : > When working with time zones, the standard library needs a little help. > Luckily, there's a module for that. https://pypi.python.org/pypi/pytz Even better: sudo dnf install python3-pytz Marko From ben.usenet at bsb.me.uk Sat Oct 14 15:10:35 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 14 Oct 2017 20:10:35 +0100 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: <8760bhtuj8.fsf@bsb.me.uk> Irv Kalb writes: Lots of detail snipped. I hope it won't matter... > > > Huh??? > > I've read a bunch of documentation, and it looks like I'm doing > everything right, but I cannot get this to work. Any other > suggestions to get this 3 line program to work correctly? Just a data point... It works here: $ python3 t.py Response is: b'156.99\n' $ cat t.py import urllib.request fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' # read all the data response = urllib.request.urlopen(fullURLWithParameters).read() print('Response is: ', response) $ python3 --version Python 3.5.2 Maybe you are missing some crucial certificates? Presumably Python finds them is standard paces, so it would be worth trying other accesses of the URL. For example, here: $ wget -q -O - 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' 156.99 Finally, wget -S shows that the resource has moved. It is now at Location: http://download.finance.yahoo.com/d/quotes.csv?s=aapl&f=l1 I don't think this has anything to do with your problem, but it's worth noting. -- Ben. From breamoreboy at yahoo.co.uk Sat Oct 14 15:25:06 2017 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 14 Oct 2017 20:25:06 +0100 Subject: Python 2 -> 3, urllib.urlOpen In-Reply-To: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> Message-ID: On 13/10/17 23:27, Irv Kalb wrote: > One of the colleges where I teach has just moved from Python 2 to Python 3. I am in the process of converting my beginning Python class from Python 2 to Python 3. Everything has gone smoothly, until I just tried to convert some code that imports and uses urllib.urlOpen to fetch data through an API. I am using an API that I found here: http://www.jarloo.com/yahoo_finance/ > > As a minimal example, I am trying to get the latest stock price for Apple. > > The following example works perfectly in Python 2: > > import urllib > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.urlopen(fullURLWithParameters).read() > > print 'Response is: ', response > > This is asking for a stock name (s=) and I am adding in aapl as a stock symbol. I am also adding a "flag" parameter (f=) and setting it to l1 to get the last trade price. When I run this in Python 2, I see: > > Response is: 156.99 > > > If I take the same program and just modify the print statement to add parentheses, then try to run it in Python 3.6 (on a Mac): > > > import urllib > > # set the Yahoo finance url, set stock name, ask for last price > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > > # read all the data > response = urllib.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > > I get the following: > > Traceback (most recent call last): > File " .... s/StockQuoteYahooAPIMinimal.py", line 9, in > response = urllib.urlopen(fullURLWithParameters).read() > AttributeError: module 'urllib' has no attribute 'urlopen' > > > I've looked at the Python 3.6 documentation for urllib, and I see that certain calls have been changed and others have been eliminated. But my eyes glaze over trying to figure out what to use instead. > > My question is: Is there a simple (hopefully one or two line) replacement for my call to url lib.urlopen().read() > > I know that there are other modules out there that handle requests (like the Requests module), but this is a strictly controlled university environment. I cannot download any external packages (other then pygame, which I got special permission for) onto the computers in the school. Therefore, I'm looking for something in the Python 3.6 Standard Library. > > Thanks in advance, > > Irv > Hopefully this https://docs.python.org/3/howto/pyporting.html will help. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From rosuav at gmail.com Sat Oct 14 16:00:33 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 15 Oct 2017 07:00:33 +1100 Subject: Best practise for passing time as arguments In-Reply-To: <87efq5a8vx.fsf@elektro.pacujo.net> References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> <87efq5a8vx.fsf@elektro.pacujo.net> Message-ID: On Sun, Oct 15, 2017 at 5:20 AM, Marko Rauhamaa wrote: > Thomas Jollans : > >> When working with time zones, the standard library needs a little help. >> Luckily, there's a module for that. https://pypi.python.org/pypi/pytz > > Even better: > > sudo dnf install python3-pytz How is that better? It's the same thing, packaged differently, and thus only available on Red Hat-family systems, and depends on the update cycle of your OS. ChrisA From marko at pacujo.net Sat Oct 14 16:57:27 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 14 Oct 2017 23:57:27 +0300 Subject: Best practise for passing time as arguments References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> <87efq5a8vx.fsf@elektro.pacujo.net> Message-ID: <87zi8tphvs.fsf@elektro.pacujo.net> Chris Angelico : > On Sun, Oct 15, 2017 at 5:20 AM, Marko Rauhamaa wrote: >> Even better: >> >> sudo dnf install python3-pytz > > How is that better? It's the same thing, packaged differently, and > thus only available on Red Hat-family systems, and depends on the > update cycle of your OS. Use the native updater your distro. Several nice things follow from the OS packaging: * You don't have to have *two* separate security update/bug fix streams. Once you've added pytz to your OS package collection, you'll get updates with the routine OS updates. * You have the benefit of a major outside entity vetting your packages. PyPI doesn't have any such oversight: . (Of course, one shouldn't overestimate the security of volunteer-maintained distros, either, but PyPI allows anybody to submit any junk they want.) * If you want to release your software to others, your third-party dependency statement becomes more concise and possible more acceptable to your customer. Also, you don't have to ship the third-party package yourself. Your customer likely knows how to update native distro packages, but may not be familiar with Python and its ecosystem. Depending only on the distro relieves you from educating your customer about PyPI. Marko From rosuav at gmail.com Sat Oct 14 17:06:41 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 15 Oct 2017 08:06:41 +1100 Subject: Best practise for passing time as arguments In-Reply-To: <87zi8tphvs.fsf@elektro.pacujo.net> References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> <87efq5a8vx.fsf@elektro.pacujo.net> <87zi8tphvs.fsf@elektro.pacujo.net> Message-ID: On Sun, Oct 15, 2017 at 7:57 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Sun, Oct 15, 2017 at 5:20 AM, Marko Rauhamaa wrote: >>> Even better: >>> >>> sudo dnf install python3-pytz >> >> How is that better? It's the same thing, packaged differently, and >> thus only available on Red Hat-family systems, and depends on the >> update cycle of your OS. > > Use the native updater your distro. > > Several nice things follow from the OS packaging: > > * You don't have to have *two* separate security update/bug fix > streams. Once you've added pytz to your OS package collection, you'll > get updates with the routine OS updates. > > * You have the benefit of a major outside entity vetting your packages. > PyPI doesn't have any such oversight: formation-technology/2017/09/devs-unknowingly-use-malicious-modules-pu > t-into-official-python-repository/>. > > (Of course, one shouldn't overestimate the security of > volunteer-maintained distros, either, but PyPI allows anybody to > submit any junk they want.) > > * If you want to release your software to others, your third-party > dependency statement becomes more concise and possible more > acceptable to your customer. Also, you don't have to ship the > third-party package yourself. > > Your customer likely knows how to update native distro packages, but > may not be familiar with Python and its ecosystem. Depending only on > the distro relieves you from educating your customer about PyPI. * You get into the habit of posting distro-specific (not just OS-specific) commands to global mailing lists. ChrisA From marko at pacujo.net Sat Oct 14 17:15:35 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 15 Oct 2017 00:15:35 +0300 Subject: Best practise for passing time as arguments References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> <87efq5a8vx.fsf@elektro.pacujo.net> <87zi8tphvs.fsf@elektro.pacujo.net> Message-ID: <87vajhph1k.fsf@elektro.pacujo.net> Chris Angelico : > * You get into the habit of posting distro-specific (not just > OS-specific) commands to global mailing lists. And? I don't mind you posting the instructions for any other Linux distro. Chances are the translation into my distro is somewhat obvious. At least it would encourage me to find out. In no way would I be annoyed by equivalent instructions for alternative operating systems such as OSX or Windows (assuming Python and the relevant packages are available on those operating systems, which I have no knowledge of). Marko From rosuav at gmail.com Sat Oct 14 17:23:44 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 15 Oct 2017 08:23:44 +1100 Subject: Best practise for passing time as arguments In-Reply-To: <87vajhph1k.fsf@elektro.pacujo.net> References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> <87efq5a8vx.fsf@elektro.pacujo.net> <87zi8tphvs.fsf@elektro.pacujo.net> <87vajhph1k.fsf@elektro.pacujo.net> Message-ID: On Sun, Oct 15, 2017 at 8:15 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> * You get into the habit of posting distro-specific (not just >> OS-specific) commands to global mailing lists. > > And? I don't mind you posting the instructions for any other Linux > distro. Chances are the translation into my distro is somewhat obvious. > At least it would encourage me to find out. > > In no way would I be annoyed by equivalent instructions for alternative > operating systems such as OSX or Windows (assuming Python and the > relevant packages are available on those operating systems, which I have > no knowledge of). Or, you could just post a link to PyPI, which applies to all platforms at once, and let people work out their own platform-specific alternatives for themselves. Oh hey, that's what Thomas already did. ChrisA From christopher_reimer at icloud.com Sat Oct 14 17:29:11 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Sat, 14 Oct 2017 14:29:11 -0700 Subject: Best practise for passing time as arguments In-Reply-To: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> References: <515190b1-20b2-b83d-13f2-6ebab9e26a46@tjol.eu> Message-ID: <76AD2073-F8F9-40FE-AC45-1DF83221839A@icloud.com> On Oct 14, 2017, at 10:44 AM, Thomas Jollans wrote: > >> On 14/10/17 19:34, Stefan Ram wrote: >> ram at zedat.fu-berlin.de (Stefan Ram) writes: >>> a post. Use whatever is appropriate in the special case >>> given, or - to write a general library -, learn the design >>> of a good existing library, like Time4J, first. >> >> Though in many cases, an ISO 8601 time string >> represented by a (named )Python tuple should >> be sufficient for a time stamp. >> >> E.g., ( year, month, day, hour, minute, seconds, >> zone_offset ). >> > > Python provides a datetime (also: date, time, timedelta) type. Use it. > > https://docs.python.org/3/library/datetime.html > > When working with time zones, the standard library needs a little help. > Luckily, there's a module for that. https://pypi.python.org/pypi/pytz > > -- Thomas > -- > https://mail.python.org/mailman/listinfo/python-list I wrote a blog post about muddling through the timestamp problem, showing examples for datetime, slice-and-dice text, and pytz. Since I was dealing with time zone-specific timestamps, I went with pytz in the end. https://www.kickingthebitbucket.com/2017/04/04/the-python-time-zone-rabbit-hole/ Chris R. From Irv at furrypants.com Sat Oct 14 20:12:22 2017 From: Irv at furrypants.com (Irv Kalb) Date: Sat, 14 Oct 2017 17:12:22 -0700 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) In-Reply-To: <8760bhtuj8.fsf@bsb.me.uk> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> <8760bhtuj8.fsf@bsb.me.uk> Message-ID: <48E083C7-61A2-47B4-BCBC-1480069D77E8@furrypants.com> Thank you! > On Oct 14, 2017, at 12:10 PM, Ben Bacarisse wrote: > > Irv Kalb writes: > > Lots of detail snipped. I hope it won't matter... > >> >> >> Huh??? >> >> I've read a bunch of documentation, and it looks like I'm doing >> everything right, but I cannot get this to work. Any other >> suggestions to get this 3 line program to work correctly? > > Just a data point... It works here: > > $ python3 t.py > Response is: b'156.99\n' > $ cat t.py > import urllib.request > fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > # read all the data > response = urllib.request.urlopen(fullURLWithParameters).read() > > print('Response is: ', response) > $ python3 --version > Python 3.5.2 > I have not tried this on anything but my Mac. I'm running 3.6.1 > Maybe you are missing some crucial certificates? Presumably Python > finds them is standard paces, so it would be worth trying other accesses > of the URL. I am just using an absolutely standard install from Python.org > For example, here: > > $ wget -q -O - 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' > 156.99 > > Finally, wget -S shows that the resource has moved. It is now at > > Location: http://download.finance.yahoo.com/d/quotes.csv?s=aapl&f=l1 > > I don't think this has anything to do with your problem, but it's worth > noting. > > -- > Ben. > That DID fix it. I changed the URL to add 'download/' and it worked perfectly. Apparently, Python 3 differs from Python 2 in the way that it is handling a missing/forwarding URL, because the original code in Python 2.7 works perfectly. Thanks very much for tracking this down! Irv From ben.usenet at bsb.me.uk Sat Oct 14 21:46:58 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 15 Oct 2017 02:46:58 +0100 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> <8760bhtuj8.fsf@bsb.me.uk> <48E083C7-61A2-47B4-BCBC-1480069D77E8@furrypants.com> Message-ID: <878tgd6v3h.fsf@bsb.me.uk> Irv Kalb writes: > Thank you! You're welcome. >> Just a data point... It works here: >> >> $ python3 t.py >> Response is: b'156.99\n' >> $ cat t.py >> import urllib.request >> fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' >> # read all the data >> response = urllib.request.urlopen(fullURLWithParameters).read() >> >> print('Response is: ', response) >> $ python3 --version >> Python 3.5.2 > > I have not tried this on anything but my Mac. I'm running 3.6.1 >> For example, here: >> >> $ wget -q -O - 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' >> 156.99 >> >> Finally, wget -S shows that the resource has moved. It is now at >> >> Location: http://download.finance.yahoo.com/d/quotes.csv?s=aapl&f=l1 >> >> I don't think this has anything to do with your problem, but it's worth >> noting. > > That DID fix it. I changed the URL to add 'download/' and it worked > perfectly. That's... interesting. > Apparently, Python 3 differs from Python 2 in the way that it is > handling a missing/forwarding URL, because the original code in Python > 2.7 works perfectly. Python 3 works for me. I still suspect it's some system difference rather than being, say, a 3.6.1 vs 3.5.2 difference. What happens if you change the URL to use https rather than http? -- Ben. From steve+python at pearwood.info Sun Oct 15 01:37:54 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 15 Oct 2017 16:37:54 +1100 Subject: An endless loop References: Message-ID: <59e2f435$0$14936$b1db1813$d948b532@news.astraweb.com> On Sun, 15 Oct 2017 01:43 pm, Dennis Lee Bieber wrote: > On 15 Oct 2017 02:10:15 GMT, ram at zedat.fu-berlin.de (Stefan Ram) declaimed > the following: [...] >>def poly( n, length ): >> i = 0 >> while i < n: >> forward( length ) >> left( 360/n ) >> > > A clear example of why a pre-determined loop is better using "for" then > emulated with a non-deterministic "while". > > for i in range(n): > > avoids having to write separate initialization, test, and increment > statements. This, a thousand times this. I'm reminded of an old joke: A mathematician, a physicist, an engineer and a computer programmer are discussing prime numbers and whether or not every odd number is prime. The mathematician says "3 is prime, 5 is prime, 7 is prime. So by induction, all odd numbers must be prime." The physicist says "3 is prime, 5 is prime, 7 is prime, 9 is an experimental error, 11 is prime, 13 is prime... within 83% confidence limits, all odd numbers are prime." The engineer says "3 is prime, 5 is prime, 7 is prime, for safety we better treat 9 as prime, 11 is prime..." The computer programmer says "Listen you guys, you're all doing it wrong. I've written a program to check for primes, it says: 1 is prime, 1 is prime, 1 is prime, 1 is prime ..." -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From premikasharma97 at gmail.com Sun Oct 15 05:35:35 2017 From: premikasharma97 at gmail.com (premikasharma97 at gmail.com) Date: Sun, 15 Oct 2017 02:35:35 -0700 (PDT) Subject: Case Solution: Doing Business in Sierra Leone Graeme Hossie at London Mining (A) by Brian C. Pinkham, Ken Mark In-Reply-To: <82ac1682-9d27-43bc-bdfb-292bd7489c5a@googlegroups.com> References: <353d9194-6a2b-4b4b-875b-4f71d59feba5@googlegroups.com> <82ac1682-9d27-43bc-bdfb-292bd7489c5a@googlegroups.com> Message-ID: <0a2408a3-1a6a-4706-92b1-995e1c0097ad@googlegroups.com> 1. What priorities should Hossie consider? 2. What are some of the concerns around enforcement of the contracts? 3. How should Hossie carry out negotiations? From bc at freeuk.com Sun Oct 15 06:15:22 2017 From: bc at freeuk.com (bartc) Date: Sun, 15 Oct 2017 11:15:22 +0100 Subject: An endless loop In-Reply-To: References: Message-ID: On 15/10/2017 03:10, Stefan Ram wrote: > I made an error I made a thousand times before. > > I had programmed an endless loop. > > But never did I see before so clear why it's called > an endless loop. (Tested in IDLE.) > > from turtle import * > > reset(); reset(); shape( 'turtle' ); showturtle() > > def poly( n, length ): > i = 0 > while i < n: > forward( length ) > left( 360/n ) > > poly( 5, 100 ) > done() I assume you're talking about the while-loop (because on my machine, it hangs just using 'from turtle...' or 'import turtle'). That looks to be a repeat-N-times loop. There isn't a dedicated statement for that, the closest Python feature would be 'for i in range(n)' with i a dummy loop variable. -- bartc From hjp-usenet3 at hjp.at Sun Oct 15 06:23:30 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Sun, 15 Oct 2017 12:23:30 +0200 Subject: An endless loop References: Message-ID: On 2017-10-15 10:15, bartc wrote: > On 15/10/2017 03:10, Stefan Ram wrote: >> I made an error I made a thousand times before. >> >> I had programmed an endless loop. >> >> But never did I see before so clear why it's called >> an endless loop. (Tested in IDLE.) >> >> from turtle import * >> >> reset(); reset(); shape( 'turtle' ); showturtle() >> >> def poly( n, length ): >> i = 0 >> while i < n: >> forward( length ) >> left( 360/n ) >> >> poly( 5, 100 ) >> done() > > I assume you're talking about the while-loop (because on my machine, it > hangs just using 'from turtle...' or 'import turtle'). No graphics? For those of you who can't use turtle graphics and aren't familiar enough with it to see what it does without running the program: It displays a turtle running through the same (pentagon-shaped) loop over and over again. Yes, that's a very nice visualization of an endless loop. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From bintacomputers at gmail.com Sun Oct 15 06:27:31 2017 From: bintacomputers at gmail.com (Umar Yusuf) Date: Sun, 15 Oct 2017 03:27:31 -0700 (PDT) Subject: what is credit in data mining? Message-ID: <502119bd-baae-4a7b-9922-e5768424d88a@googlegroups.com> Hello, Anyone useful idea or material on: credit in data mining? From rosuav at gmail.com Sun Oct 15 07:20:14 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 15 Oct 2017 22:20:14 +1100 Subject: An endless loop In-Reply-To: References: Message-ID: On Sun, Oct 15, 2017 at 9:15 PM, bartc wrote: > On 15/10/2017 03:10, Stefan Ram wrote: >> >> I made an error I made a thousand times before. >> >> I had programmed an endless loop. >> >> But never did I see before so clear why it's called >> an endless loop. (Tested in IDLE.) >> >> from turtle import * >> >> reset(); reset(); shape( 'turtle' ); showturtle() >> >> def poly( n, length ): >> i = 0 >> while i < n: >> forward( length ) >> left( 360/n ) >> >> poly( 5, 100 ) >> done() > > > I assume you're talking about the while-loop (because on my machine, it > hangs just using 'from turtle...' or 'import turtle'). > > That looks to be a repeat-N-times loop. There isn't a dedicated statement > for that, the closest Python feature would be 'for i in range(n)' with i a > dummy loop variable. You can use that or "for _ in range(n)" as a pretty effective form of that loop. I've never had a problem with it. Python doesn't need another type of loop. ChrisA From bc at freeuk.com Sun Oct 15 09:59:54 2017 From: bc at freeuk.com (bartc) Date: Sun, 15 Oct 2017 14:59:54 +0100 Subject: An endless loop In-Reply-To: References: Message-ID: On 15/10/2017 12:20, Chris Angelico wrote: > On Sun, Oct 15, 2017 at 9:15 PM, bartc wrote: >> I assume you're talking about the while-loop (because on my machine, it >> hangs just using 'from turtle...' or 'import turtle'). (Machine was screwed up I think, as I had to restart it shortly after for other reasons. When the turtle graphics worked.) >> That looks to be a repeat-N-times loop. There isn't a dedicated statement >> for that, the closest Python feature would be 'for i in range(n)' with i a >> dummy loop variable. > > You can use that or "for _ in range(n)" as a pretty effective form of > that loop. I've never had a problem with it. Python doesn't need > another type of loop. I could understand that sentiment if there were already dozens. But I believe there's only, what, two loop types? To implement a number of basic looping requirements which, if not that numerous, are somewhat more than two. Not exactly taxing the grey matter which already has those extra loop types in mind, and has to find a way to express them in terms of only two. -- bartc From ned at nedbatchelder.com Sun Oct 15 10:38:13 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 15 Oct 2017 10:38:13 -0400 Subject: An endless loop In-Reply-To: References: Message-ID: <7b8001d8-5c60-90f1-8c24-5f7688fa798b@nedbatchelder.com> On 10/15/17 9:59 AM, bartc wrote: > On 15/10/2017 12:20, Chris Angelico wrote: >> On Sun, Oct 15, 2017 at 9:15 PM, bartc wrote: > >>> I assume you're talking about the while-loop (because on my machine, it >>> hangs just using 'from turtle...' or 'import turtle'). > > (Machine was screwed up I think, as I had to restart it shortly after > for other reasons. When the turtle graphics worked.) > >>> That looks to be a repeat-N-times loop. There isn't a dedicated >>> statement >>> for that, the closest Python feature would be 'for i in range(n)' >>> with i a >>> dummy loop variable. >> >> You can use that or "for _ in range(n)" as a pretty effective form of >> that loop. I've never had a problem with it. Python doesn't need >> another type of loop. > > I could understand that sentiment if there were already dozens. But I > believe there's only, what, two loop types? To implement a number of > basic looping requirements which, if not that numerous, are somewhat > more than two. > > Not exactly taxing the grey matter which already has those extra loop > types in mind, and has to find a way to express them in terms of only > two. > We've covered this before.? A challenge for the group: let's not create yet another 100-reply thread rehashing the design of Python looping... Please? :) --Ned. From steve+python at pearwood.info Sun Oct 15 12:05:18 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 16 Oct 2017 03:05:18 +1100 Subject: An endless loop References: <7b8001d8-5c60-90f1-8c24-5f7688fa798b@nedbatchelder.com> Message-ID: <59e38740$0$14937$b1db1813$d948b532@news.astraweb.com> On Mon, 16 Oct 2017 01:38 am, Ned Batchelder wrote: > We've covered this before.? A challenge for the group: let's not create > yet another 100-reply thread rehashing the design of Python looping... > Please? :) Certainly! So... who thinks that Python should implement tail recursion optimization so that we can forgo iteration altogether? *wicked grin* -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From Irv at furrypants.com Sun Oct 15 14:56:27 2017 From: Irv at furrypants.com (Irv Kalb) Date: Sun, 15 Oct 2017 11:56:27 -0700 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) In-Reply-To: <878tgd6v3h.fsf@bsb.me.uk> References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> <8760bhtuj8.fsf@bsb.me.uk> <48E083C7-61A2-47B4-BCBC-1480069D77E8@furrypants.com> <878tgd6v3h.fsf@bsb.me.uk> Message-ID: > On Oct 14, 2017, at 6:46 PM, Ben Bacarisse wrote: > > Irv Kalb writes: > >> Thank you! > > You're welcome. > > >>> Just a data point... It works here: >>> >>> $ python3 t.py >>> Response is: b'156.99\n' >>> $ cat t.py >>> import urllib.request >>> fullURLWithParameters = 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' >>> # read all the data >>> response = urllib.request.urlopen(fullURLWithParameters).read() >>> >>> print('Response is: ', response) >>> $ python3 --version >>> Python 3.5.2 >> >> I have not tried this on anything but my Mac. I'm running 3.6.1 > >>> For example, here: >>> >>> $ wget -q -O - 'http://finance.yahoo.com/d/quotes.csv?s=aapl&f=l1' >>> 156.99 >>> >>> Finally, wget -S shows that the resource has moved. It is now at >>> >>> Location: http://download.finance.yahoo.com/d/quotes.csv?s=aapl&f=l1 >>> >>> I don't think this has anything to do with your problem, but it's worth >>> noting. >> >> That DID fix it. I changed the URL to add 'download/' and it worked >> perfectly. > > That's... interesting. > >> Apparently, Python 3 differs from Python 2 in the way that it is >> handling a missing/forwarding URL, because the original code in Python >> 2.7 works perfectly. > > Python 3 works for me. I still suspect it's some system difference > rather than being, say, a 3.6.1 vs 3.5.2 difference. What happens if > you change the URL to use https rather than http? > > -- > Ben. > -- > https://mail.python.org/mailman/listinfo/python-list > Trying https (with and without the "download." part) results in the same traceback as I was seeing earlier. Thanks again, Irv From ben.usenet at bsb.me.uk Sun Oct 15 16:06:02 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 15 Oct 2017 21:06:02 +0100 Subject: Python 2 -> 3, urllib.urlopen (corrected the case) References: <524E6B2D-B693-42FE-A040-EE1C29EBAF89@furrypants.com> <8760bhtuj8.fsf@bsb.me.uk> <48E083C7-61A2-47B4-BCBC-1480069D77E8@furrypants.com> <878tgd6v3h.fsf@bsb.me.uk> Message-ID: <87tvz05g7p.fsf@bsb.me.uk> Irv Kalb writes: >> On Oct 14, 2017, at 6:46 PM, Ben Bacarisse wrote: >>>> Finally, wget -S shows that the resource has moved. It is now at >>>> >>>> Location: http://download.finance.yahoo.com/d/quotes.csv?s=aapl&f=l1 >>>> >>>> I don't think this has anything to do with your problem, but it's worth >>>> noting. >>> >>> That DID fix it. I changed the URL to add 'download/' and it worked >>> perfectly. >> >> That's... interesting. >> >>> Apparently, Python 3 differs from Python 2 in the way that it is >>> handling a missing/forwarding URL, because the original code in Python >>> 2.7 works perfectly. >> >> Python 3 works for me. I still suspect it's some system difference >> rather than being, say, a 3.6.1 vs 3.5.2 difference. What happens if >> you change the URL to use https rather than http? >> > > Trying https (with and without the "download." part) results in the > same traceback as I was seeing earlier. It looks like there may be something amiss with your local certificates, at least as far as Python's SSL code in concerned. That's a Dark Art to me so I can't really help (and it might be Mac specific). The error might have showed up in the first test because the redirection was to an https: URL. -- Ben. From formisc at gmail.com Sun Oct 15 22:50:34 2017 From: formisc at gmail.com (Andrew Z) Date: Sun, 15 Oct 2017 22:50:34 -0400 Subject: how to read in the newsreader Message-ID: Gents, how do i get this group in a newsreader? The digest i'm getting is not workable for me - i can't reply , can only read the replies from the members of the group. Or. maybe, it shouldn't be a news reader.... please advise.. P.S. Oh the comp.lang.python is a nightmare because of spam... From torriem at gmail.com Mon Oct 16 00:00:16 2017 From: torriem at gmail.com (Michael Torrie) Date: Sun, 15 Oct 2017 22:00:16 -0600 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> On 10/15/2017 08:50 PM, Andrew Z wrote: > Gents, > how do i get this group in a newsreader? The digest i'm getting is not > workable for me - i can't reply , can only read the replies from the > members of the group. Or. maybe, it shouldn't be a news reader.... > please advise.. > > P.S. Oh the comp.lang.python is a nightmare because of spam... Regardless of what usenet reader you use, com.lang.python will have the same spam everywhere. So maybe reading via usenet isn't that useful anyway. I just subscribe to the mailing list and then have a rule in my email filter to stick all list messages in their own folder (or gmail label). I'm using gmail, so I just use gmail's filtering settings. Then I view email using IMAP and a conventional email reader that supports real threading of messages. Works great. A lot of spam is filtered out this way, also. In fact very little spam gets through to my folder between python.org's filtering and gmail's spam detection. I'm subscribed to maybe a dozen email lists, and I filter all of them into their own folders. From formisc at gmail.com Mon Oct 16 00:19:58 2017 From: formisc at gmail.com (Andrew Z) Date: Mon, 16 Oct 2017 00:19:58 -0400 Subject: how to read in the newsreader In-Reply-To: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: Michael, that's what i use too - gmail. But i get the digest only and can't really reply that way. i was hoping to get the mail.python.org list.... On Mon, Oct 16, 2017 at 12:00 AM, Michael Torrie wrote: > On 10/15/2017 08:50 PM, Andrew Z wrote: > > Gents, > > how do i get this group in a newsreader? The digest i'm getting is not > > workable for me - i can't reply , can only read the replies from the > > members of the group. Or. maybe, it shouldn't be a news reader.... > > please advise.. > > > > P.S. Oh the comp.lang.python is a nightmare because of spam... > > Regardless of what usenet reader you use, com.lang.python will have the > same spam everywhere. So maybe reading via usenet isn't that useful > anyway. > > I just subscribe to the mailing list and then have a rule in my email > filter to stick all list messages in their own folder (or gmail label). > I'm using gmail, so I just use gmail's filtering settings. Then I view > email using IMAP and a conventional email reader that supports real > threading of messages. Works great. A lot of spam is filtered out this > way, also. In fact very little spam gets through to my folder between > python.org's filtering and gmail's spam detection. > > I'm subscribed to maybe a dozen email lists, and I filter all of them > into their own folders. > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Mon Oct 16 00:30:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 16 Oct 2017 15:30:15 +1100 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: > Michael, that's what i use too - gmail. But i get the digest only and can't > really reply that way. i was hoping to get the mail.python.org list.... Turn off digests then. Easy! ChrisA From formisc at gmail.com Mon Oct 16 00:31:17 2017 From: formisc at gmail.com (Andrew Z) Date: Mon, 16 Oct 2017 00:31:17 -0400 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: i'm typing without thinking. Sorry. Let me try again. I subscribed to python-list at python.org. That has 0 spam ( as far as i can see), but i can only get a digest. They do say the list and comp.python group are bidirectional, but - at comp - tons of spam, there - none. The google groups doesn't seemed to filter anything in the comp group. But what you recommend is to access the group using regular email client and use it's spam filtering . Did i get that right ? On Mon, Oct 16, 2017 at 12:19 AM, Andrew Z wrote: > Michael, that's what i use too - gmail. But i get the digest only and > can't really reply that way. i was hoping to get the mail.python.org > list.... > > On Mon, Oct 16, 2017 at 12:00 AM, Michael Torrie > wrote: > >> On 10/15/2017 08:50 PM, Andrew Z wrote: >> > Gents, >> > how do i get this group in a newsreader? The digest i'm getting is not >> > workable for me - i can't reply , can only read the replies from the >> > members of the group. Or. maybe, it shouldn't be a news reader.... >> > please advise.. >> > >> > P.S. Oh the comp.lang.python is a nightmare because of spam... >> >> Regardless of what usenet reader you use, com.lang.python will have the >> same spam everywhere. So maybe reading via usenet isn't that useful >> anyway. >> >> I just subscribe to the mailing list and then have a rule in my email >> filter to stick all list messages in their own folder (or gmail label). >> I'm using gmail, so I just use gmail's filtering settings. Then I view >> email using IMAP and a conventional email reader that supports real >> threading of messages. Works great. A lot of spam is filtered out this >> way, also. In fact very little spam gets through to my folder between >> python.org's filtering and gmail's spam detection. >> >> I'm subscribed to maybe a dozen email lists, and I filter all of them >> into their own folders. >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > From formisc at gmail.com Mon Oct 16 00:34:47 2017 From: formisc at gmail.com (Andrew Z) Date: Mon, 16 Oct 2017 00:34:47 -0400 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: hmm. i did do that. maybe just a delay. I'll see how it will go tomorrow then. Thank you gents. On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: > On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: > > Michael, that's what i use too - gmail. But i get the digest only and > can't > > really reply that way. i was hoping to get the mail.python.org list.... > > Turn off digests then. Easy! > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From bagriarsh50 at gmail.com Mon Oct 16 02:20:33 2017 From: bagriarsh50 at gmail.com (bagriarsh50 at gmail.com) Date: Sun, 15 Oct 2017 23:20:33 -0700 (PDT) Subject: Case Solution: Detroit Bikes Becoming the Biggest Bicycle Manufacturer in North America by Kent Walker, Neda Demiri In-Reply-To: <5b4e810b-e242-4956-ac38-c46e06e51e97@googlegroups.com> References: <5b4e810b-e242-4956-ac38-c46e06e51e97@googlegroups.com> Message-ID: <22d4fa11-1141-49b3-adaf-47cb10dd59c1@googlegroups.com> hello can i get solution From dieter at handshake.de Mon Oct 16 02:24:03 2017 From: dieter at handshake.de (dieter) Date: Mon, 16 Oct 2017 08:24:03 +0200 Subject: Return str to a callback raise a segfault if used in string formating References: <65252491-920b-14cb-428c-ef4b390b0ff8@telenet.be> <781c4bdf-7692-2249-eb8d-f2158651f84e@tjol.eu> <6e1654dd-61e7-28c4-d9c4-8250ab7afd83@telenet.be> Message-ID: <87d15nbofw.fsf@handshake.de> Vincent Vande Vyvre writes: > Le 14/10/17 ? 15:59, Stefan Behnel a ?crit?: > ... > Thanks, I know Cython but the code is already in C. > > This is a lib released as a frontend.? Then usable only in command line. This does not prevent the use of "Cython". In fact, one of its use cases is the creation of bindings - a thin wrapper around a C library to make it easily usable from Python. An example is "lxml", a binding for the C library "libxml2". Another one is "dm.xmlsec.binding", a binding for the C library "libxmlsec". From petef4+usenet at gmail.com Mon Oct 16 02:48:02 2017 From: petef4+usenet at gmail.com (Pete Forman) Date: Mon, 16 Oct 2017 07:48:02 +0100 Subject: how to read in the newsreader References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: Andrew Z writes: > hmm. i did do that. maybe just a delay. > I'll see how it will go tomorrow then. Thank you gents. > > On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: > >> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >> > Michael, that's what i use too - gmail. But i get the digest only >> > and can't really reply that way. i was hoping to get the >> > mail.python.org list.... >> >> Turn off digests then. Easy! If you do stick with a digest then check your newsreader for a feature to expand it. Then you can read and reply as if you were getting individual posts. -- Pete Forman From alfredo.branco at gmail.com Mon Oct 16 03:14:49 2017 From: alfredo.branco at gmail.com (Alfredo Branco) Date: Mon, 16 Oct 2017 09:14:49 +0200 Subject: [Python] [JOB] Python developer at JRC Message-ID: Hello everyone, I writing just to let you know that European Commission Joint Research Center [1] is looking for a developer to work in Ispra (Varese - Italy) The profile required is as follow: **** Very good knowledge of Python and R Good English language knowledge A knowledge of one or more of the following would be a plus : UML, agile development, design patterns; OOP: C#, JAVA, C++;ASP.NET ; SQL, ORACLE, SQL Server; Experience in working with large meteorological / climate datasets; Experience in visualization of large spatio-temporal datasets; For more information and to send a resume please contact Roberto Boca: rboca a arcadiasit.it [1] https://ec.europa.eu/jrc/ -- Alfredo Branco From cl at isbd.net Mon Oct 16 03:57:18 2017 From: cl at isbd.net (Chris Green) Date: Mon, 16 Oct 2017 08:57:18 +0100 Subject: how to read in the newsreader References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: Michael Torrie wrote: > On 10/15/2017 08:50 PM, Andrew Z wrote: > > Gents, > > how do i get this group in a newsreader? The digest i'm getting is not > > workable for me - i can't reply , can only read the replies from the > > members of the group. Or. maybe, it shouldn't be a news reader.... > > please advise.. > > > > P.S. Oh the comp.lang.python is a nightmare because of spam... > > Regardless of what usenet reader you use, com.lang.python will have the > same spam everywhere. So maybe reading via usenet isn't that useful anyway. > The mostly very silly spam is trivial to filter with some very simple rules, most newsreaders have easy ways to specify subjects and/or senders to ignore. I have (I think) just three or four rules that eliminate just about all the junk. -- Chris Green ? From tjol at tjol.eu Mon Oct 16 04:15:38 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 16 Oct 2017 10:15:38 +0200 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: On 2017-10-16 08:48, Pete Forman wrote: > Andrew Z writes: > >> hmm. i did do that. maybe just a delay. >> I'll see how it will go tomorrow then. Thank you gents. >> >> On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: >> >>> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >>>> Michael, that's what i use too - gmail. But i get the digest only >>>> and can't really reply that way. i was hoping to get the >>>> mail.python.org list.... >>> >>> Turn off digests then. Easy! > > If you do stick with a digest then check your newsreader for a feature > to expand it. Then you can read and reply as if you were getting > individual posts. > That exists? How does it work? -- Thomas Jollans From gengyangcai at gmail.com Mon Oct 16 05:01:37 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Mon, 16 Oct 2017 02:01:37 -0700 (PDT) Subject: Searching For Old Posts On Python Message-ID: Does anyone here know a way I can search for and display all my old posts on Python ? Thanks a lot. Gengyang From tjol at tjol.eu Mon Oct 16 05:12:10 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 16 Oct 2017 11:12:10 +0200 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: On 2017-10-16 11:01, Cai Gengyang wrote: > Does anyone here know a way I can search for and display all my old posts on Python ? Thanks a lot. > > Gengyang > You already asked this recently. You received good answers. From lkrupp at nospam.pssw.com.invalid Mon Oct 16 05:19:37 2017 From: lkrupp at nospam.pssw.com.invalid (Louis Krupp) Date: Mon, 16 Oct 2017 03:19:37 -0600 Subject: Searching For Old Posts On Python References: Message-ID: <0cu8uc56snuouip708r6qknkr1j384f8r9@4ax.com> On Mon, 16 Oct 2017 02:01:37 -0700 (PDT), Cai Gengyang wrote: >Does anyone here know a way I can search for and display all my old posts on Python ? Thanks a lot. This might be what you're looking for: https://groups.google.com/forum/#!search/cai$20gengyang$20python Louis From alister.ware at ntlworld.com Mon Oct 16 05:52:42 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 16 Oct 2017 09:52:42 GMT Subject: how to read in the newsreader Message-ID: On Mon, 16 Oct 2017 08:57:18 +0100, Chris Green wrote: > Michael Torrie wrote: >> On 10/15/2017 08:50 PM, Andrew Z wrote: >> > Gents, >> > how do i get this group in a newsreader? The digest i'm getting is >> > not >> > workable for me - i can't reply , can only read the replies from the >> > members of the group. Or. maybe, it shouldn't be a news reader.... >> > please advise.. >> > >> > P.S. Oh the comp.lang.python is a nightmare because of spam... >> >> Regardless of what usenet reader you use, com.lang.python will have the >> same spam everywhere. So maybe reading via usenet isn't that useful >> anyway. >> > The mostly very silly spam is trivial to filter with some very simple > rules, most newsreaders have easy ways to specify subjects and/or > senders to ignore. I have (I think) just three or four rules that > eliminate just about all the junk. Although that is the case I ma finding that because of the amount of junk in this newsgroup I am rapidly running out of space in /dev/null ! -- In Oz, never say "krizzle kroo" to a Woozy. From ned at nedbatchelder.com Mon Oct 16 05:58:59 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 16 Oct 2017 05:58:59 -0400 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: On 10/16/17 12:31 AM, Andrew Z wrote: > i'm typing without thinking. Sorry. Let me try again. > > I subscribed to python-list at python.org. That has 0 spam ( as far as i can > see), but i can only get a digest. You can choose to get individual messages from Python-List. --Ned. From tjreedy at udel.edu Mon Oct 16 06:36:48 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 16 Oct 2017 06:36:48 -0400 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: On 10/15/2017 10:50 PM, Andrew Z wrote: > Gents, > how do i get this group in a newsreader? Point your newsreader to news.gmane.org, group gmane.comp.python.general, which mirrors python-list, among hundreds or thousands of other lists. If you check the headers of this message, you should see that I am posting via gmane. -- Terry Jan Reedy From steve+python at pearwood.info Mon Oct 16 08:12:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 16 Oct 2017 23:12:33 +1100 Subject: Searching For Old Posts On Python References: Message-ID: <59e4a233$0$14945$b1db1813$d948b532@news.astraweb.com> On Mon, 16 Oct 2017 08:12 pm, Thomas Jollans wrote: > On 2017-10-16 11:01, Cai Gengyang wrote: >> Does anyone here know a way I can search for and display all my old posts >> on Python ? Thanks a lot. >> >> Gengyang >> > > You already asked this recently. You received good answers. Yes, but he can't find the answers because he can't search for his old posts :-) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Oct 16 08:19:05 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 16 Oct 2017 23:19:05 +1100 Subject: Searching For Old Posts On Python References: Message-ID: <59e4a3bc$0$14941$b1db1813$d948b532@news.astraweb.com> On Mon, 16 Oct 2017 08:01 pm, Cai Gengyang wrote: > Does anyone here know a way I can search for and display all my old posts on > Python ? Thanks a lot. > > Gengyang Repeating the answers you already got: Make a site specific search for your name here https://mail.python.org/pipermail/python-list/ Or download the archives onto your computer and search them locally. https://mail.python.org/pipermail/python-list/ (download the Gzipped archives). Or try googling: https://duckduckgo.com/html/?q=%22Cai%20Gengyang%22%20python -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From grant.b.edwards at gmail.com Mon Oct 16 10:41:32 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 16 Oct 2017 14:41:32 +0000 (UTC) Subject: how to read in the newsreader References: Message-ID: On 2017-10-16, Terry Reedy wrote: > On 10/15/2017 10:50 PM, Andrew Z wrote: >> Gents, >> how do i get this group in a newsreader? > > Point your newsreader to news.gmane.org, That, IMO, is the only sane way to read mailing lists. If a mailing list isn't carried on gmane, I don't bother with it. -- Grant Edwards grant.b.edwards Yow! Were these parsnips at CORRECTLY MARINATED in gmail.com TACO SAUCE? From p.f.moore at gmail.com Mon Oct 16 10:52:14 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 16 Oct 2017 15:52:14 +0100 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: On 16 October 2017 at 15:41, Grant Edwards wrote: > On 2017-10-16, Terry Reedy wrote: >> On 10/15/2017 10:50 PM, Andrew Z wrote: >>> Gents, >>> how do i get this group in a newsreader? >> >> Point your newsreader to news.gmane.org, > > That, IMO, is the only sane way to read mailing lists. If a mailing > list isn't carried on gmane, I don't bother with it. Unless you work regularly on multiple PCs, as there's no newsreader I know of that maintains your settings (what articles you have read, in particular) across multiple installations. And gmane's UI sucks. For that situation, reading mailing lists as mails in gmail is the best option I've been able to find (not ideal, but adequate). Paul From grant.b.edwards at gmail.com Mon Oct 16 11:07:37 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 16 Oct 2017 15:07:37 +0000 (UTC) Subject: how to read in the newsreader References: Message-ID: On 2017-10-16, Paul Moore wrote: > On 16 October 2017 at 15:41, Grant Edwards wrote: >> On 2017-10-16, Terry Reedy wrote: >>> On 10/15/2017 10:50 PM, Andrew Z wrote: >>>> Gents, >>>> how do i get this group in a newsreader? >>> >>> Point your newsreader to news.gmane.org, >> >> That, IMO, is the only sane way to read mailing lists. If a mailing >> list isn't carried on gmane, I don't bother with it. > > Unless you work regularly on multiple PCs, as there's no newsreader I > know of that maintains your settings (what articles you have read, in > particular) across multiple installations. Ah yes. I solved problem that by writing a wrapper around slrn so that my .newsrc and .score files reside "in the could". [They're actually just sitting in my home directory on a Unix server which I can scp them to/from. > And gmane's UI sucks. AFIAK, it has no UI. The web interface has been completely broken for a couple years now. It was always mediocre at best (the built-in search was useless), but it did provide something that you could search with Google, and it was sometimes handy when you wanted to provide somebody with a link to a particular posting from years gone by. > For that situation, reading mailing lists as mails in gmail is the > best option I've been able to find (not ideal, but adequate). You mean the gmail web UI? I don't even use that e-mail (and I use gmail for all my e-mail). -- Grant Edwards grant.b.edwards Yow! I'm encased in the at lining of a pure pork gmail.com sausage!! From p.f.moore at gmail.com Mon Oct 16 11:21:44 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 16 Oct 2017 16:21:44 +0100 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: On 16 October 2017 at 16:07, Grant Edwards wrote: > On 2017-10-16, Paul Moore wrote: >> Unless you work regularly on multiple PCs, as there's no newsreader I >> know of that maintains your settings (what articles you have read, in >> particular) across multiple installations. > > Ah yes. I solved problem that by writing a wrapper around slrn so > that my .newsrc and .score files reside "in the could". [They're > actually just sitting in my home directory on a Unix server which I > can scp them to/from. Something like that would be great, but not practical for me to write myself. I'm on Windows machines, and Windows newsreaders from what I recall aren't as hackable as Unix ones (I'm able to write the code, but not if the data isn't accessible...). Also, on at least one of my PCs I'm behind a firewall that blocks access to pretty much all "personal cloud storage" sites, so a DIY solution isn't possible. Something like Chrome's "sync my settings" would be ideal, but I've never found anything like that in a newsreader. > You mean the gmail web UI? I don't even use that e-mail (and I use > gmail for all my e-mail). Yep. Same reason, it's the only globally accessible option. (I could use a mail client on the systems that don't block IMAP, but I'd still need the web UI for the others, and for "borrowed" PCs that I have no software on). The cloud is great, but the app UIs are still a lot worse than a dedicated client app, sadly... Paul From torriem at gmail.com Mon Oct 16 11:23:50 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 16 Oct 2017 09:23:50 -0600 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: <056f5ab2-c7ef-6e82-d19b-9c820f532927@gmail.com> On 10/16/2017 08:52 AM, Paul Moore wrote: > For that situation, reading mailing lists as mails in gmail is the > best option I've been able to find (not ideal, but adequate). > Paul Gmail's web interface is completely broken as far as mailing lists are concerned. Conversation view in no way is adequate for the more complex, multi-person threads that are common. Gmail does not preserve the flow and context of such emails. However Gmail over IMAP is adequate. There is the issue of in IMAP gmail will silently through away your replies to messages when they come through the mailing list server, which is a pretty serious and long-standing brokenness. But if you send messages via a different smtp server than gmail's, it doesn't do that. If reading the list from anywhere is important, one can always use a web-based mail client like RoundCube against Gmail's IMAP server. From rhodri at kynesim.co.uk Mon Oct 16 11:36:56 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 16 Oct 2017 16:36:56 +0100 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: On 16/10/17 16:07, Grant Edwards wrote: > Ah yes. I solved problem that by writing a wrapper around slrn so > that my .newsrc and .score files reside "in the could". ^^^^^ Now there's a typo someone should run with :-) -- Rhodri James *-* Kynesim Ltd From xf.lotus at yahoo.com Mon Oct 16 11:37:12 2017 From: xf.lotus at yahoo.com (Xue Feng) Date: Mon, 16 Oct 2017 23:37:12 +0800 Subject: why del is not a function or method? Message-ID: Hi, I wonder why 'del' is not a function or method. Most operations can be used as follows len(team) or team.append("tom") But, I think del team[2] is somewhat uncommon. Why does not it take a syntax we are famillar with? From cl at isbd.net Mon Oct 16 11:44:12 2017 From: cl at isbd.net (Chris Green) Date: Mon, 16 Oct 2017 16:44:12 +0100 Subject: how to read in the newsreader References: Message-ID: Paul Moore wrote: > On 16 October 2017 at 15:41, Grant Edwards wrote: > > On 2017-10-16, Terry Reedy wrote: > >> On 10/15/2017 10:50 PM, Andrew Z wrote: > >>> Gents, > >>> how do i get this group in a newsreader? > >> > >> Point your newsreader to news.gmane.org, > > > > That, IMO, is the only sane way to read mailing lists. If a mailing > > list isn't carried on gmane, I don't bother with it. > > Unless you work regularly on multiple PCs, as there's no newsreader I > know of that maintains your settings (what articles you have read, in > particular) across multiple installations. And gmane's UI sucks. > > For that situation, reading mailing lists as mails in gmail is the > best option I've been able to find (not ideal, but adequate). I read newsgroups using tin, a text-mode/command line newsreader. I always run tin on my home desktop machine, even if I'm away from home by using ssh. So I maintain my settings that way. By the way tin *is* mouse away even though it's a text mode program. For reading mailing lists (and all my mail for that matter) I use mutt which is also a text mode program. It makes reading mailing lists easy because it provides threading etc. I use like I use tin, alway on my desktop machine using ssh where necessary. -- Chris Green ? From marko at pacujo.net Mon Oct 16 11:57:34 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 16 Oct 2017 18:57:34 +0300 Subject: how to read in the newsreader References: Message-ID: <87y3ob6q6p.fsf@elektro.pacujo.net> Chris Green : > I read newsgroups using tin, a text-mode/command line newsreader. I > always run tin on my home desktop machine, even if I'm away from home > by using ssh. So I maintain my settings that way. By the way tin *is* > mouse away even though it's a text mode program. > > For reading mailing lists (and all my mail for that matter) I use mutt > which is also a text mode program. It makes reading mailing lists easy > because it provides threading etc. I use like I use tin, alway on my > desktop machine using ssh where necessary. Same thing, except that I do all of my typing on emacs (GNUS in the case of news and mail). Marko From bc at freeuk.com Mon Oct 16 12:07:31 2017 From: bc at freeuk.com (bartc) Date: Mon, 16 Oct 2017 17:07:31 +0100 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: <3P4FB.14035$d33.6699@fx42.am4> On 16/10/2017 16:58, Stefan Ram wrote: > Xue Feng writes: >> I wonder why 'del' is not a function or method. > > Assume, > > x = 2. > > When a function ?f? is called with the argument ?x?, > this is written as > > f( x ) > > . The function never gets to see the name ?x?, just > its boundee (value) ?2?. So, it cannot delete the > name ?x?. > > Also, the function has no access to the scope of ?x?, > and even more so, it cannot make any changes in it. > > Therefore, even a call such as > > f( 'x' ) > > will not help much. What about del team[2]? There is no name involved here, and even a reference to team[2] won't help. Presumably there is no other way to do an in-place deletion of an element of a list. (Inserting an element is different.) -- Bartc From cl at isbd.net Mon Oct 16 12:10:48 2017 From: cl at isbd.net (Chris Green) Date: Mon, 16 Oct 2017 17:10:48 +0100 Subject: how to read in the newsreader References: <87y3ob6q6p.fsf@elektro.pacujo.net> Message-ID: <892fbe-ing.ln1@esprimo.zbmc.eu> Marko Rauhamaa wrote: > Chris Green : > > > I read newsgroups using tin, a text-mode/command line newsreader. I > > always run tin on my home desktop machine, even if I'm away from home > > by using ssh. So I maintain my settings that way. By the way tin *is* > > mouse away even though it's a text mode program. > > > > For reading mailing lists (and all my mail for that matter) I use mutt > > which is also a text mode program. It makes reading mailing lists easy > > because it provides threading etc. I use like I use tin, alway on my > > desktop machine using ssh where necessary. > > Same thing, except that I do all of my typing on emacs (GNUS in the case > of news and mail). > That's the other advantage of working this way, one always uses the same text editor for composing messages, I use a vi clone (let's not start a vi/emacs war!). -- Chris Green ? From python-oren at ben-kiki.org Mon Oct 16 12:16:54 2017 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 16 Oct 2017 19:16:54 +0300 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: That doesn't explain why `del` isn't a method though. Intuitively, `my_dict.delete(some_key)` makes sense as a method. Of course, you could also make the same case for `len` being a method... and personally I think it would have been cleaner that way in both cases. But it is a minor issue, if at all. I guess the answer is a combination of "historical reasons" and "Guido's preferences"? On Mon, Oct 16, 2017 at 6:58 PM, Stefan Ram wrote: > Xue Feng writes: > >I wonder why 'del' is not a function or method. > > Assume, > > x = 2. > > When a function ?f? is called with the argument ?x?, > this is written as > > f( x ) > > . The function never gets to see the name ?x?, just > its boundee (value) ?2?. So, it cannot delete the > name ?x?. > > Also, the function has no access to the scope of ?x?, > and even more so, it cannot make any changes in it. > > Therefore, even a call such as > > f( 'x' ) > > will not help much. > > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Mon Oct 16 12:24:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Oct 2017 03:24:30 +1100 Subject: why del is not a function or method? In-Reply-To: <3P4FB.14035$d33.6699@fx42.am4> References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: On Tue, Oct 17, 2017 at 3:07 AM, bartc wrote: > On 16/10/2017 16:58, Stefan Ram wrote: >> >> Xue Feng writes: >>> >>> I wonder why 'del' is not a function or method. >> >> >> Assume, >> >> x = 2. >> >> When a function ?f? is called with the argument ?x?, >> this is written as >> >> f( x ) >> >> . The function never gets to see the name ?x?, just >> its boundee (value) ?2?. So, it cannot delete the >> name ?x?. >> >> Also, the function has no access to the scope of ?x?, >> and even more so, it cannot make any changes in it. >> >> Therefore, even a call such as >> >> f( 'x' ) >> >> will not help much. > > > What about del team[2]? > > There is no name involved here, and even a reference to team[2] won't help. > > Presumably there is no other way to do an in-place deletion of an element of > a list. (Inserting an element is different.) team.pop(2) will in-place delete one element. So you have both options. ChrisA From skip.montanaro at gmail.com Mon Oct 16 12:28:40 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 16 Oct 2017 11:28:40 -0500 Subject: why del is not a function or method? In-Reply-To: <3P4FB.14035$d33.6699@fx42.am4> References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: > What about del team[2]? > > There is no name involved here, and even a reference to team[2] won't help. (I'm not sure quite what question is asking. Apologies if my assumption was incorrect.) That is precisely why del is a statement. At byte-compile time, both "team" and "2" are available. The easiest way to see this is to define a simple function: def f(team): del team[2] Now, import the dis module and call dis.dis(f) then check out the byte code assembler statements. Skip From rosuav at gmail.com Mon Oct 16 12:31:08 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Oct 2017 03:31:08 +1100 Subject: why del is not a function or method? In-Reply-To: References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: On Tue, Oct 17, 2017 at 3:18 AM, Stefan Ram wrote: > bartc writes: >>What about del team[2]? > > All arguments of a call are evaluated before the callable > called then is incarnated with the values obtained. Assume, > ?team[ 2 ]? is bound to ?8?. Then, the call > > f( team[ 2 ])? > > is equivalent to > > f( 8 ) > > , and so ?f? doesn't stand a chance of deleting ?team[ 2 ]?. > > The evaluation rules for calls do not apply to the del- > statement, however. Therefore, > > del team[ 2 ] > > is /not/ equivalent to > > del 8 > > , and ?del? can delete ?team[ 2 ]? just fine. (Because the > implementation can use whatever means available to it, > since it is not bound to follow the evaluation rules for > callables.) Right, but you *could* implement a function that takes the two parts and does the assignment or deletion: def f(lst, idx): del lst[idx] f(team, 2) The reason that 'del' absolutely has to be a statement is that it can work on simple names, too: del team There's no way to do THAT part with a function call (it's basically the opposite of assignment), so 'del' has to be a statement. Since it's a statement, it may as well be able to do more than just simple names, which means we have a consistent way to delete items from arbitrary objects: del globals()["foo"] # dicts del sys.argv[1] # lists etc (Although you can't do this with sets. Sets are manipulated exclusively through methods and operators.) ChrisA From __peter__ at web.de Mon Oct 16 12:32:15 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 16 Oct 2017 18:32:15 +0200 Subject: why del is not a function or method? References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: bartc wrote: > On 16/10/2017 16:58, Stefan Ram wrote: >> Xue Feng writes: >>> I wonder why 'del' is not a function or method. >> >> Assume, >> >> x = 2. >> >> When a function ?f? is called with the argument ?x?, >> this is written as >> >> f( x ) >> >> . The function never gets to see the name ?x?, just >> its boundee (value) ?2?. So, it cannot delete the >> name ?x?. >> >> Also, the function has no access to the scope of ?x?, >> and even more so, it cannot make any changes in it. >> >> Therefore, even a call such as >> >> f( 'x' ) >> >> will not help much. > > What about del team[2]? > > There is no name involved here, and even a reference to team[2] won't > help. > > Presumably there is no other way to do an in-place deletion of an > element of a list. (Inserting an element is different.) There is another way: team.pop(2) Stefan's explanation may work for del x if you discard x = None # get rid of the huge object that x was bound to before as a hack, but not for del x[y] or del x.y From ned at nedbatchelder.com Mon Oct 16 12:36:52 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 16 Oct 2017 12:36:52 -0400 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: On 10/16/17 12:16 PM, Oren Ben-Kiki wrote: > That doesn't explain why `del` isn't a method though. Intuitively, > `my_dict.delete(some_key)` makes sense as a method. Of course, you could > also make the same case for `len` being a method... and personally I think > it would have been cleaner that way in both cases. But it is a minor issue, > if at all. > > I guess the answer is a combination of "historical reasons" and "Guido's > preferences"? It would still need to be a statement to allow for: ??? del x since "x.del()" wouldn't affect the name x, it would affect the value x refers to. --Ned. > > > On Mon, Oct 16, 2017 at 6:58 PM, Stefan Ram wrote: > >> Xue Feng writes: >>> I wonder why 'del' is not a function or method. >> Assume, >> >> x = 2. >> >> When a function ?f? is called with the argument ?x?, >> this is written as >> >> f( x ) >> >> . The function never gets to see the name ?x?, just >> its boundee (value) ?2?. So, it cannot delete the >> name ?x?. >> >> Also, the function has no access to the scope of ?x?, >> and even more so, it cannot make any changes in it. >> >> Therefore, even a call such as >> >> f( 'x' ) >> >> will not help much. >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> From python-oren at ben-kiki.org Mon Oct 16 12:55:48 2017 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 16 Oct 2017 19:55:48 +0300 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: True... technically, "Deletion of a name removes the binding of that name from the local or global namespace". Using `x.del()` can't do that. That said, I would hazard to guess that `del x` is pretty rare (I have never felt the need for it myself). Ruby doesn't even have an equivalent operation, and doesn't seem to suffer as a result. If Python used methods instead of global functions for `len` and `del`, and provided a `delete_local_variable('x')` for these rare cases, that could have been a viable solution. So I still think it was a matter of preference rather than a pure technical consideration. But that's all second-guessing, anyway. You'd have to ask Guido what his reasoning was... On Mon, Oct 16, 2017 at 7:36 PM, Ned Batchelder wrote: > On 10/16/17 12:16 PM, Oren Ben-Kiki wrote: > >> That doesn't explain why `del` isn't a method though. Intuitively, >> `my_dict.delete(some_key)` makes sense as a method. Of course, you could >> also make the same case for `len` being a method... and personally I think >> it would have been cleaner that way in both cases. But it is a minor >> issue, >> if at all. >> >> I guess the answer is a combination of "historical reasons" and "Guido's >> preferences"? >> > > It would still need to be a statement to allow for: > > del x > > since "x.del()" wouldn't affect the name x, it would affect the value x > refers to. > > --Ned. > > >> >> On Mon, Oct 16, 2017 at 6:58 PM, Stefan Ram >> wrote: >> >> Xue Feng writes: >>> >>>> I wonder why 'del' is not a function or method. >>>> >>> Assume, >>> >>> x = 2. >>> >>> When a function ?f? is called with the argument ?x?, >>> this is written as >>> >>> f( x ) >>> >>> . The function never gets to see the name ?x?, just >>> its boundee (value) ?2?. So, it cannot delete the >>> name ?x?. >>> >>> Also, the function has no access to the scope of ?x?, >>> and even more so, it cannot make any changes in it. >>> >>> Therefore, even a call such as >>> >>> f( 'x' ) >>> >>> will not help much. >>> >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >>> > -- > https://mail.python.org/mailman/listinfo/python-list > From ian.g.kelly at gmail.com Mon Oct 16 13:09:39 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 16 Oct 2017 11:09:39 -0600 Subject: An endless loop In-Reply-To: References: Message-ID: On Sat, Oct 14, 2017 at 8:10 PM, Stefan Ram wrote: > I made an error I made a thousand times before. > > I had programmed an endless loop. > > But never did I see before so clear why it's called > an endless loop. (Tested in IDLE.) > > from turtle import * > > reset(); reset(); shape( 'turtle' ); showturtle() > > def poly( n, length ): > i = 0 > while i < n: > forward( length ) > left( 360/n ) > > poly( 5, 100 ) > done() I honestly can't remember the last time I programmed an endless loop, and I also can't remember the last time I used a while loop. Those two things are probably related. From bc at freeuk.com Mon Oct 16 13:21:01 2017 From: bc at freeuk.com (bartc) Date: Mon, 16 Oct 2017 18:21:01 +0100 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: On 16/10/2017 17:55, Oren Ben-Kiki wrote: > True... technically, "Deletion of a name removes the binding of that name > from the local or global namespace". Using `x.del()` can't do that. > > That said, I would hazard to guess that `del x` is pretty rare (I have > never felt the need for it myself). Ruby doesn't even have an equivalent > operation, and doesn't seem to suffer as a result. It just seems the peculiar way that Python works: pass # 1 pass # 2 x = 10 # 3 del x # 4 pass # 4 In the above code, x doesn't actually exist in the namespace until line 3. Trying to use x on lines 1 or 2 would cause an 'undefined' error. del x effectively removes it from the namespace so trying to use it on line 4 generates the same 'undefined' error. However, the byte-code still needs to be aware of x: at the time when line 1 is executed, the byte-code for line 3 already exists and it will need to contain a reference to x. This could have been done in other ways by simply having 'x' always in the namespace, but not bound to anything ('undefined). But then the equivalent of del x would still have needed something special, for example: x = anything # any other undefined name which is currently not allowed. -- bartc From lele at metapensiero.it Mon Oct 16 13:33:35 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 16 Oct 2017 19:33:35 +0200 Subject: why del is not a function or method? References: Message-ID: <87efq35768.fsf@metapensiero.it> Oren Ben-Kiki writes: > So I still think it was a matter of preference rather than a pure technical > consideration. But that's all second-guessing, anyway. You'd have to ask > Guido what his reasoning was... A rationale is briefly stated in the design FAQs, see https://docs.python.org/3/faq/design.html#why-does-python-use-methods-for-some-functionality-e-g-list-index-but-functions-for-other-e-g-len-list and the next one. ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From python-oren at ben-kiki.org Mon Oct 16 13:42:00 2017 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 16 Oct 2017 20:42:00 +0300 Subject: why del is not a function or method? In-Reply-To: <87efq35768.fsf@metapensiero.it> References: <87efq35768.fsf@metapensiero.it> Message-ID: The first line says "The major reason is history." :-) But it also gives an explanation: providing functionality for types that, at the time, didn't have methods. On Mon, Oct 16, 2017 at 8:33 PM, Lele Gaifax wrote: > Oren Ben-Kiki writes: > > > So I still think it was a matter of preference rather than a pure > technical > > consideration. But that's all second-guessing, anyway. You'd have to ask > > Guido what his reasoning was... > > A rationale is briefly stated in the design FAQs, see > https://docs.python.org/3/faq/design.html#why-does-python- > use-methods-for-some-functionality-e-g-list-index- > but-functions-for-other-e-g-len-list > and the next one. > > ciao, lele. > -- > nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri > real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. > lele at metapensiero.it | -- Fortunato Depero, 1929. > > -- > https://mail.python.org/mailman/listinfo/python-list > From torriem at gmail.com Mon Oct 16 13:46:16 2017 From: torriem at gmail.com (Michael Torrie) Date: Mon, 16 Oct 2017 11:46:16 -0600 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: On 10/16/2017 11:21 AM, bartc wrote: > del x effectively removes it from the namespace so trying to use it on > line 4 generates the same 'undefined' error. > > However, the byte-code still needs to be aware of x: at the time when > line 1 is executed, the byte-code for line 3 already exists and it will > need to contain a reference to x. I'm not sure why you'd think this, as we all know that code referring to a name involves a dictionary lookup at runtime. So the byte code for line does exist before x is defined, but it does not contain a "reference" to x in the manner you are thinking. Instead, all line three knows is that it must lookup the key "x" (as in string) in the dictionary of local or global objects. If it exists, great, if not, an exception is thrown. So no, there is no reference to x in the byte code before x is assigned. From __peter__ at web.de Mon Oct 16 13:52:14 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 16 Oct 2017 19:52:14 +0200 Subject: why del is not a function or method? References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: Stefan Ram wrote: > Peter Otten <__peter__ at web.de> writes: >>team.pop(2) >>Stefan's explanation may work for >>del x >>if you discard >>x = None # get rid of the huge object that x was bound to before >>as a hack > > ?x = None? observably has not the same effect as ?del x?: > > |>>> x = 2; x = None; x > |>>> x = 2; del x; x > |NameError: name 'x' is not defined That's a minor technical detail. I understood the original question as one regarding language design, and I don't think merely stating what a statement does counts as a reason for its inclusion in the language. x = None covers the memory-micromanaging aspect of del x which makes some marginal sense. I expect to see the NameError (control flow aspect, if you will) only when there's a bug in the program. From petef4+usenet at gmail.com Mon Oct 16 14:02:43 2017 From: petef4+usenet at gmail.com (Pete Forman) Date: Mon, 16 Oct 2017 19:02:43 +0100 Subject: how to read in the newsreader References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: Thomas Jollans writes: > On 2017-10-16 08:48, Pete Forman wrote: >> Andrew Z writes: >> >>> hmm. i did do that. maybe just a delay. >>> I'll see how it will go tomorrow then. Thank you gents. >>> >>> On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: >>> >>>> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >>>>> Michael, that's what i use too - gmail. But i get the digest only >>>>> and can't really reply that way. i was hoping to get the >>>>> mail.python.org list.... >>>> >>>> Turn off digests then. Easy! >> >> If you do stick with a digest then check your newsreader for a feature >> to expand it. Then you can read and reply as if you were getting >> individual posts. >> > That exists? How does it work? The Gnus newsreader in Emacs allows you to type C-d on a digest to run gnus-summary-enter-digest-group. That then behaves the same as if you opened any other summary such as a regular Usenet group. -- Pete Forman From bc at freeuk.com Mon Oct 16 14:38:26 2017 From: bc at freeuk.com (bartc) Date: Mon, 16 Oct 2017 19:38:26 +0100 Subject: An endless loop In-Reply-To: References: Message-ID: On 16/10/2017 18:53, Stefan Ram wrote: > Ian Kelly writes: >> I honestly can't remember the last time I programmed an endless loop, >> and I also can't remember the last time I used a while loop. >> Those two things are probably related. > > My Python installation has a "Lib" directory. > > ?^ +\bwhile\b.*:$? has 1348 hits in this directory, > ?^ +\bfor\b.*:$? has 8713. That's a ratio of 6.46. > > In other words, while-loops are only 13 % of all loops > (while or for). That's a clear minority. But it does > not indicate that while loops are used almost ever. It's presumably a characteristic of the language. And it depends on what the language offers, so if there was no 'while' at all, I guess it would be 100% 'for'. (I just looked through my non-Python language at a couple of projects and got these figures: forall 29% (Equivalent to Python 'for') for 36% (Simple iteration) while 14% (about the same as your figure) repeat-until 3% N-times 11% endless loop 6% 11% N-times loops doesn't sound a lot but it's one in every 9 loops. What the figures don't show is that the N-times and endless loops are probably used more with short test programs than in final applications, so having a very quick way to express them is convenient.) -- bartc From bc at freeuk.com Mon Oct 16 14:48:12 2017 From: bc at freeuk.com (bartc) Date: Mon, 16 Oct 2017 19:48:12 +0100 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: On 16/10/2017 18:46, Michael Torrie wrote: > On 10/16/2017 11:21 AM, bartc wrote: >> del x effectively removes it from the namespace so trying to use it on >> line 4 generates the same 'undefined' error. >> >> However, the byte-code still needs to be aware of x: at the time when >> line 1 is executed, the byte-code for line 3 already exists and it will >> need to contain a reference to x. > > I'm not sure why you'd think this, as we all know that code referring to > a name involves a dictionary lookup at runtime. So the byte code for > line does exist before x is defined, but it does not contain a > "reference" to x in the manner you are thinking. Instead, all line three > knows is that it must lookup the key "x" (as in string) in the > dictionary of local or global objects. Yes, the reference can be a string. So the string "x" will always exist in the byte-code even before the program is run, and even if the bit that uses x is never executed. So if you look a file of Python code and count 288 different global variables, then somehow those 288 have to be identified by name in the binary code. The symbol tables for them are filled in as it goes along. I'm just saying it can be done differently. I think it already is for local variables, otherwise 'STORE_FAST' would require a lookup each time. As it is, accessing a local 'x' after 'del x' says 'referenced before assignment'. -- bartc From gengyangcai at gmail.com Mon Oct 16 16:04:42 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Tue, 17 Oct 2017 04:04:42 +0800 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: I cannot remember where I posted the question .... it was a while ago .. On Mon, Oct 16, 2017 at 5:12 PM, Thomas Jollans wrote: > On 2017-10-16 11:01, Cai Gengyang wrote: > > Does anyone here know a way I can search for and display all my old > posts on Python ? Thanks a lot. > > > > Gengyang > > > > You already asked this recently. You received good answers. > > > From gengyangcai at gmail.com Mon Oct 16 16:10:51 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Mon, 16 Oct 2017 13:10:51 -0700 (PDT) Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: On Tuesday, October 17, 2017 at 4:05:16 AM UTC+8, Cai Gengyang wrote: > I cannot remember where I posted the question .... it was a while ago .. > > On Mon, Oct 16, 2017 at 5:12 PM, Thomas Jollans wrote: > > > On 2017-10-16 11:01, Cai Gengyang wrote: > > > Does anyone here know a way I can search for and display all my old > > posts on Python ? Thanks a lot. > > > > > > Gengyang > > > > > > > You already asked this recently. You received good answers. > > > > > > Oh Ok. The DuckDuckGo thing works ... I can find my old posts now. Cool ... From Karsten.Hilbert at gmx.net Mon Oct 16 16:32:16 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Mon, 16 Oct 2017 22:32:16 +0200 Subject: right list for SIGABRT python binary question ? Message-ID: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> Hi all, is this the right list to ask for help in debugging a SIGABORT (?) happening on shutdown of a Python 2.7 script ? If not, which one is ? Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From tjol at tjol.eu Mon Oct 16 16:54:29 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 16 Oct 2017 22:54:29 +0200 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: On 16/10/17 22:04, Cai Gengyang wrote: > I cannot remember where I posted the question .... it was a while ago .. > > On Mon, Oct 16, 2017 at 5:12 PM, Thomas Jollans > wrote: > > On 2017-10-16 11:01, Cai Gengyang wrote: > > Does anyone here know a way I can search for and display all my > old posts on Python ? Thanks a lot. > > > > Gengyang > > > > You already asked this recently. You received good answers. It was just over one week ago. On this very list. Here is the original thread in the archive: https://mail.python.org/pipermail/python-list/2017-October/727139.html And here is this one: https://mail.python.org/pipermail/python-list/2017-October/727567.html -- Thomas Jollans From tjol at tjol.eu Mon Oct 16 17:57:39 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 16 Oct 2017 23:57:39 +0200 Subject: why del is not a function or method? In-Reply-To: References: <3P4FB.14035$d33.6699@fx42.am4> Message-ID: <50f90389-c753-2d27-f29d-e2ab9049ca54@tjol.eu> On 16/10/17 21:12, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >> ?x = None? observably has not the same effect as ?del x?: > > Paradoxically, /deleting/ a local variable which did not > ever exist, has the effect of creating a ghost of that > local variable which then will hide a global variable: > > Case 1: The global variable ?x? can be used in ?f? just > fine: > > |>>> x = 9 > |>>> def f(): > |... print( x ) > |... > |>>> f() > |9 > > . Case 2: The ?del x? creates a ghost ?local variable 'x'? > which now will hide the global variable ?x?: > > |>>> x = 9 > |>>> def f(): > |... del x > |... print( x ) > |... > |>>> f() > |UnboundLocalError: local variable 'x' referenced before assignment > > . That's not what happens. The UnboundLocalError is in the del statement, not the print call: >>> x = None >>> def f(): ... del x ... >>> f() Traceback (most recent call last): File "", line 1, in File "", line 2, in f UnboundLocalError: local variable 'x' referenced before assignment >>> From tjol at tjol.eu Mon Oct 16 18:00:45 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 17 Oct 2017 00:00:45 +0200 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: <0b465e5b-86ee-30c6-c693-7cc179c449d9@tjol.eu> On 16/10/17 20:02, Pete Forman wrote: > Thomas Jollans writes: > >> On 2017-10-16 08:48, Pete Forman wrote: >>> Andrew Z writes: >>> >>>> hmm. i did do that. maybe just a delay. >>>> I'll see how it will go tomorrow then. Thank you gents. >>>> >>>> On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: >>>> >>>>> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >>>>>> Michael, that's what i use too - gmail. But i get the digest only >>>>>> and can't really reply that way. i was hoping to get the >>>>>> mail.python.org list.... >>>>> >>>>> Turn off digests then. Easy! >>> >>> If you do stick with a digest then check your newsreader for a feature >>> to expand it. Then you can read and reply as if you were getting >>> individual posts. >>> >> That exists? How does it work? > > The Gnus newsreader in Emacs allows you to type C-d on a digest to run > gnus-summary-enter-digest-group. That then behaves the same as if you > opened any other summary such as a regular Usenet group. > Does it set the References header correctly when replying? -- Thomas Jollans From mikhailwas at gmail.com Mon Oct 16 18:55:37 2017 From: mikhailwas at gmail.com (Mikhail V) Date: Tue, 17 Oct 2017 00:55:37 +0200 Subject: how to read in the newsreader Message-ID: Thomas wrote: > > On 16/10/17 20:02, Pete Forman wrote: > > Thomas Jollans writes: > > ... > >>> If you do stick with a digest then check your newsreader for a feature > >>> to expand it. Then you can read and reply as if you were getting > >>> individual posts. > >>> > >> That exists? How does it work? > > > > The Gnus newsreader in Emacs allows you to type C-d on a digest to run > > gnus-summary-enter-digest-group. That then behaves the same as if you > > opened any other summary such as a regular Usenet group. > > > > Does it set the References header correctly when replying? Hi Thomas, regarding the issue with my reply-to header you told me recently - Does this message looks ok in your threaded view now? Now I reply by the href of the message I looked up in pipermail archive, just curious if it looks now correctly. Mikhail From python at mrabarnett.plus.com Mon Oct 16 19:30:20 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 17 Oct 2017 00:30:20 +0100 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: <5aff6eed-4f1d-4210-dec5-752494c40aab@mrabarnett.plus.com> On 2017-10-16 21:04, Cai Gengyang wrote: > I cannot remember where I posted the question .... it was a while ago .. > You posted on Sat, 07 Oct 2017 23:42:10 UTC. > On Mon, Oct 16, 2017 at 5:12 PM, Thomas Jollans wrote: > >> On 2017-10-16 11:01, Cai Gengyang wrote: >> > Does anyone here know a way I can search for and display all my old >> posts on Python ? Thanks a lot. >> > >> > Gengyang >> > >> >> You already asked this recently. You received good answers. >> From mickey.munin at gmail.com Mon Oct 16 19:39:54 2017 From: mickey.munin at gmail.com (=?UTF-8?B?157Xmden15kg157Xldeg15nXnw==?=) Date: Mon, 16 Oct 2017 16:39:54 -0700 Subject: Strange behavior in string interpolation of constants Message-ID: Hello, I am working on an article on python string formatting. As a part of the article I am researching the different forms of python string formatting. While researching string interpolation(i.e. the % operator) I noticed something weird with string lengths. Given two following two functions: def simple_interpolation_constant_short_string(): return "Hello %s" % "World!" def simple_interpolation_constant_long_string(): return "Hello %s. I am a very long string used for research" % "World!" Lets look at the bytecode generated by them using the dis module The first example produces the following bytecode: 9 0 LOAD_CONST 3 ('Hello World!') 2 RETURN_VALUE It seems very normal, it appears that the python compiler optimizes the constant and removes the need for the string interpolation However the output of the second function caught my eye: 12 0 LOAD_CONST 1 ('Hello %s. I am a very long string used for research') 2 LOAD_CONST 2 ('World!') 4 BINARY_MODULO 6 RETURN_VALUE This was not optimized by the compiler! Normal string interpolation was used! Based on some more testing it appears that for strings that would result in more than 20 characters no optimization is done, as evident by these examples: def expected_result(): return "abcdefghijklmnopqrs%s" % "t" Bytecode: 15 0 LOAD_CONST 3 ('abcdefghijklmnopqrst') 2 RETURN_VALUE def abnormal_result(): return "abcdefghijklmnopqrst%s" % "u" Bytecode: 18 0 LOAD_CONST 1 ('abcdefghijklmnopqrst%s') 2 LOAD_CONST 2 ('u') 4 BINARY_MODULO 6 RETURN_VALUE I am using Python 3.6.3 I am curios as to why this happens. Can anyone shed further light on this behaviour? From ned at nedbatchelder.com Mon Oct 16 20:18:22 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 16 Oct 2017 20:18:22 -0400 Subject: Strange behavior in string interpolation of constants In-Reply-To: References: Message-ID: On 10/16/17 7:39 PM, ???? ????? wrote: > Hello, I am working on an article on python string formatting. As a part of > the article I am researching the different forms of python string > formatting. > > While researching string interpolation(i.e. the % operator) I noticed > something weird with string lengths. > > Given two following two functions: > > def simple_interpolation_constant_short_string(): > return "Hello %s" % "World!" > > def simple_interpolation_constant_long_string(): > return "Hello %s. I am a very long string used for research" % "World!" > > > Lets look at the bytecode generated by them using the dis module > > The first example produces the following bytecode: > 9 0 LOAD_CONST 3 ('Hello World!') > 2 RETURN_VALUE > > It seems very normal, it appears that the python compiler optimizes the > constant and removes the need for the string interpolation > > However the output of the second function caught my eye: > > 12 0 LOAD_CONST 1 ('Hello %s. I am a very long > string used for research') > 2 LOAD_CONST 2 ('World!') > 4 BINARY_MODULO > 6 RETURN_VALUE > > This was not optimized by the compiler! Normal string interpolation was > used! > > Based on some more testing it appears that for strings that would result in > more than 20 characters no optimization is done, as evident by these > examples: > > def expected_result(): > return "abcdefghijklmnopqrs%s" % "t" > > Bytecode: > 15 0 LOAD_CONST 3 ('abcdefghijklmnopqrst') > 2 RETURN_VALUE > > def abnormal_result(): > return "abcdefghijklmnopqrst%s" % "u" > > Bytecode: > > 18 0 LOAD_CONST 1 ('abcdefghijklmnopqrst%s') > 2 LOAD_CONST 2 ('u') > 4 BINARY_MODULO > 6 RETURN_VALUE > > I am using Python 3.6.3 > I am curios as to why this happens. Can anyone shed further light on this > behaviour? Optimizers have plenty of heuristics.? This one seems to avoid the constant folding if the string is larger than 20.? The code seems to bear this out (https://github.com/python/cpython/blob/master/Python/peephole.c#L305): ??? } else if (size > 20) { ??????? Py_DECREF(newconst); ??????? return -1; ??? } As to why they chose 20?? There's no clue in the code, and I don't know. --Ned. From steve+python at pearwood.info Mon Oct 16 20:53:27 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 11:53:27 +1100 Subject: why del is not a function or method? References: Message-ID: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 03:16 am, Oren Ben-Kiki wrote: > That doesn't explain why `del` isn't a method though. `del` cannot be a method or a function, because the argument to `del` is the name of the variable, not the contents of the variable. If we write: x = 123 del x then `del` needs to delete the *name* "x", not the value of x, namely 123. If del were a function or method, it would only see the value, 123, and have no idea what the name is. `del` is kind of like an "anti-assignment" in that the argument to `del` must be exactly the same sort of expression that can appear on the left hand side of assignment: 123 = 1+1 # illegal del 123 # also illegal -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From cs at cskk.id.au Mon Oct 16 21:02:04 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 17 Oct 2017 12:02:04 +1100 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: <20171017010204.GA72407@cskk.homeip.net> On 15Oct2017 22:50, Andrew Z wrote: > how do i get this group in a newsreader? The digest i'm getting is not >workable for me - i can't reply , can only read the replies from the >members of the group. Or. maybe, it shouldn't be a news reader.... >please advise.. > >P.S. Oh the comp.lang.python is a nightmare because of spam... Well comp.langpython _is_ the newsgroup. I recommend using the mailing list in individual message mode. It is far better. Many mail readers have good interfaces, and you get to keep all the messages, making it easy to find things later or to read in a nice threaded way. Digests have many problems, some of which you're well aware of. I wish they weren't even an option. Cheers, Cameron Simpson From steve+python at pearwood.info Mon Oct 16 21:04:09 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 12:04:09 +1100 Subject: right list for SIGABRT python binary question ? References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> Message-ID: <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 07:32 am, Karsten Hilbert wrote: > Hi all, > > is this the right list to ask for help in debugging a > SIGABORT (?) happening on shutdown of a Python 2.7 script ? > > If not, which one is ? > > Karsten You should try here first. Please ensure you read and follow this first: http://sscce.org/ and post a *minimum* example of your code. (Sorry for the redundant instructions if you already know this, but you would be surprised how many people don't.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Mon Oct 16 21:06:06 2017 From: bc at freeuk.com (bartc) Date: Tue, 17 Oct 2017 02:06:06 +0100 Subject: why del is not a function or method? In-Reply-To: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> Message-ID: On 17/10/2017 01:53, Steve D'Aprano wrote: > On Tue, 17 Oct 2017 03:16 am, Oren Ben-Kiki wrote: > >> That doesn't explain why `del` isn't a method though. > > > `del` cannot be a method or a function, because the argument to `del` is the > name of the variable, not the contents of the variable. > > If we write: > > x = 123 > del x > > then `del` needs to delete the *name* "x", not the value of x, namely 123. If > del were a function or method, it would only see the value, 123, and have no > idea what the name is. > > `del` is kind of like an "anti-assignment" in that the argument to `del` must > be exactly the same sort of expression that can appear on the left hand side > of assignment: > > > 123 = 1+1 # illegal > del 123 # also illegal Yet in Stefan Ram's example with del applied to a local 'x', it raised an error on: del x # x not yet assigned to but an assignment to x would have been fine. From steve+python at pearwood.info Mon Oct 16 21:07:47 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 12:07:47 +1100 Subject: why del is not a function or method? References: <3P4FB.14035$d33.6699@fx42.am4> <50f90389-c753-2d27-f29d-e2ab9049ca54@tjol.eu> Message-ID: <59e557e4$0$14940$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 08:57 am, Thomas Jollans wrote: > On 16/10/17 21:12, Stefan Ram wrote: >> ram at zedat.fu-berlin.de (Stefan Ram) writes: >>> ?x = None? observably has not the same effect as ?del x?: >> >> Paradoxically, /deleting/ a local variable which did not >> ever exist, has the effect of creating a ghost of that >> local variable which then will hide a global variable: [...] >> . Case 2: The ?del x? creates a ghost ?local variable 'x'? >> which now will hide the global variable ?x?: >> >> |>>> x = 9 >> |>>> def f(): >> |... del x >> |... print( x ) >> |... >> |>>> f() >> |UnboundLocalError: local variable 'x' referenced before assignment >> >> . > > That's not what happens. The UnboundLocalError is in the del statement, > not the print call: That's exactly what happens! Stefan merely explains it poorly. There's no mysterious "ghost", it is just that x is treated as a local variable instead of a global, and you can't delete the local before it is bound to. `del x` is treated as a form of assignment, just like `x = 123`, and makes `x` a local variable. So in this code: def f(): print(x) there is no assignment to x, so it is treated as a global. But if we write: def f(): x = 123 print(x) the Python compiler sees the assignment to x and makes x a local variable. Note that the rules for this are deliberately very, very simple, so this will make x a local: def f(): print(x) # x is local x = 123 and even this will too: def f(): if False: x = 123 # dead code print(x) # but enough to make x a local You can replace the line "x = 123" with "del x" in any of those functions, and you will get the same effect: x will be treated as a local. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ned at nedbatchelder.com Mon Oct 16 22:12:57 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 16 Oct 2017 22:12:57 -0400 Subject: why del is not a function or method? In-Reply-To: References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/16/17 9:06 PM, bartc wrote: > On 17/10/2017 01:53, Steve D'Aprano wrote: >> On Tue, 17 Oct 2017 03:16 am, Oren Ben-Kiki wrote: >> >>> That doesn't explain why `del` isn't a method though. >> >> >> `del` cannot be a method or a function, because the argument to `del` >> is the >> name of the variable, not the contents of the variable. >> >> If we write: >> >> ???? x = 123 >> ???? del x >> >> then `del` needs to delete the *name* "x", not the value of x, namely >> 123. If >> del were a function or method, it would only see the value, 123, and >> have no >> idea what the name is. >> >> `del` is kind of like an "anti-assignment" in that the argument to >> `del` must >> be exactly the same sort of expression that can appear on the left >> hand side >> of assignment: >> >> >> ???? 123 = 1+1? # illegal >> ???? del 123? # also illegal > > Yet in Stefan Ram's example with del applied to a local 'x', it raised > an error on: > > ??? del x??????? # x not yet assigned to > > but an assignment to x would have been fine. Steve meant that syntactically it had to be valid on the left-hand side.? "x" is a syntactically valid LHS, "1+1" is not. --Ned. From ian.g.kelly at gmail.com Mon Oct 16 22:20:40 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 16 Oct 2017 20:20:40 -0600 Subject: Cooperative class tree filtering In-Reply-To: <87376o2ced.fsf@ender.lizardnet> References: <87376o2ced.fsf@ender.lizardnet> Message-ID: On Thu, Oct 12, 2017 at 5:07 PM, Alberto Berti wrote: > Now, what i ideally want is to get rid of that super() call at the end of > the methods in classes B and c and to code that "fallback to what my > superclass says" coded into A's implementation, where it kicks in if the > method in the target instance returns None. So to write it in code, I > would like some like: > > > [SNIP] > > > I've tried some variants of the 'super()' trick, that sometimes seems to > change behavior if it's written like that or like super(type(self), > self) in no clear (to me, and I failed to find extensive doc on > super()'s behavior) way, with things that stop working if mixins are > involved (even if the mixins do not reimplement the methods involved > here). Eventually i ended implementing a decorator: > > from functools import partial, wraps > > > class delegate_super: > """A method decorator that delegates part of the computation to the same > method on the ancestor class.""" > > _name = None > _owner = None > > def __init__(self, meth): > self._meth = meth > @wraps(meth) > def wrapper(*args, **kwargs): > return self.delegator(*args, **kwargs) > self.wrapper = wrapper > > def __get__(self, instance, owner): > return partial(self.wrapper, instance) > > def __set_name__(self, owner, name): > self._owner = owner > self._name = name > > def delegator(self, instance, *args, **kwargs): > result = self._meth(instance, *args, **kwargs) > if result is None: > result = getattr(super(self._owner, instance), self._name)( > *args, **kwargs) > return result > > class A: > def filter(self, element): > # the default implementation always returns True > return True > > > class B(A): > @delegate_super > def filter(self, element): > if element == 'foo': > return True > elif element == 'bar': > return False > > > class C(B): > @delegate_super > def filter(self, element): > if element == 'bar': > return True > else: > return super().filter(element) > > > def collect_elements(instance): > "A semplified element collect function" > all_elts = {'foo', 'bar', 'baz', 'zoo'} > filtered_elts = set(el for el in all_elts if instance.filter(el)) > return filtered_elts My initial reaction is: is this really worth it? This seems like an awful lot of code and added complexity in order to do away with two lines. It's a lot easier to reason about "return super().filter(element)" and verify that it does the right thing than for the complicated descriptor above. > I would really like to find a way to do this that doesn't involve > decorating the methods in A subclasses to free the final developer to > remember to import the decorator and apply it, just like I don't want > him (probably me six months from now) to have to remember to add an > `else: super()...` to its computation... > > Has anyone done this before and has any suggestion about a better way to > do it? Am I getting it wrong? If you don't want an explicit super() call and you don't want the decorator to be explicit either then most likely what you need is a metaclass that will automatically wrap specific methods with your decorator. Metaclasses are inherited, so all you have to do is set it for A and it will automatically apply itselt to B and C as well. From ian.g.kelly at gmail.com Mon Oct 16 22:28:06 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 16 Oct 2017 20:28:06 -0600 Subject: Cooperative class tree filtering In-Reply-To: References: <87376o2ced.fsf@ender.lizardnet> Message-ID: On Mon, Oct 16, 2017 at 8:20 PM, Ian Kelly wrote: > On Thu, Oct 12, 2017 at 5:07 PM, Alberto Berti wrote: >> I would really like to find a way to do this that doesn't involve >> decorating the methods in A subclasses to free the final developer to >> remember to import the decorator and apply it, just like I don't want >> him (probably me six months from now) to have to remember to add an >> `else: super()...` to its computation... >> >> Has anyone done this before and has any suggestion about a better way to >> do it? Am I getting it wrong? > > > If you don't want an explicit super() call and you don't want the > decorator to be explicit either then most likely what you need is a > metaclass that will automatically wrap specific methods with your > decorator. Metaclasses are inherited, so all you have to do is set it > for A and it will automatically apply itselt to B and C as well. Assuming Python 3.6+ (I see you're already using __set_name__) you could probably use __init_subclass__ for this as well. From steve+python at pearwood.info Mon Oct 16 22:35:40 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 13:35:40 +1100 Subject: why del is not a function or method? References: Message-ID: <59e56c7e$0$14958$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 03:55 am, Oren Ben-Kiki wrote: > True... technically, "Deletion of a name removes the binding of that name > from the local or global namespace". Using `x.del()` can't do that. Indeed -- not without introducing magic special syntax into the parser, and effectively treating ".del()" as a special postfix operator that takes the name x given before it. In other words, Python would have had introduce special parsing rules. Instead of always using: DOT and perform an attribute access, Python would first have to check for the special case: DOT "del()" and treat it differently. What if you wrote: x.method().del() or x.del().method() for example? Should that be treated as a syntax error? Special syntactic cases make for difficult to understand parsers, and surprising syntax. > That said, I would hazard to guess that `del x` is pretty rare (I have > never felt the need for it myself). Indeed. `del` is pretty rare. The main reason I use it is to delete temporary variables in repetitious code. For example, in one of my modules I have something close to this snippet to validate a post-condition on two global dictionaries: for d in (dict1, dict2): for x in d.values(): assert x.code == x._calculate_code() del d, x d and x are temporary variables that exist only for the validation, and I don't want them hanging around polluting the module namespace once I'm done with them. > Ruby doesn't even have an equivalent > operation, and doesn't seem to suffer as a result. If Python used methods > instead of global functions for `len` and `del`, The case of `len` is irrelevant. > and provided a `delete_local_variable('x')` for these rare cases, Don't forget delete_nonlocal_variable, delete_global_variable, delete_class_variable, and delete_builtin_variable. And if Python ever added any other scopes (say, a "process global" scope) then you'd need a new delete function to delete from it. In fact, deleting *local* variables is probably the least useful use of del imaginable. Its a local variable -- nothing outside of the current function can access it, so who cares about deleting it? In fairness, you could possibly get away with a single function if, either, you passed the namespace to delete from: delete_variable('foo', globals()) or if it introduced some sort of implementation-specific magic to work out which namespace the variable name was defined in. But that would require the function to know the namespaces of the *caller*, not its own namespaces, and there is no generally supported way to do that. So in other words, it would require significant compiler magic. In general, Python prefers to put the compiler magic in statements, not functions. > that could have been a viable solution. True. It would have been an ugly, inelegant solution, but it would be a solution. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Mon Oct 16 22:38:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 13:38:38 +1100 Subject: why del is not a function or method? References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e56d2e$0$14958$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 01:12 pm, Ned Batchelder wrote: > On 10/16/17 9:06 PM, bartc wrote: >> On 17/10/2017 01:53, Steve D'Aprano wrote: [...] >>> `del` is kind of like an "anti-assignment" in that the argument to >>> `del` must >>> be exactly the same sort of expression that can appear on the left >>> hand side >>> of assignment: >>> >>> >>> 123 = 1+1? # illegal >>> del 123? # also illegal >> >> Yet in Stefan Ram's example with del applied to a local 'x', it raised >> an error on: >> >> del x??????? # x not yet assigned to >> >> but an assignment to x would have been fine. > > Steve meant that syntactically it had to be valid on the left-hand > side.? "x" is a syntactically valid LHS, "1+1" is not. Right. I didn't say that "del x is a compiler declaration that has no runtime effect", because that would have been silly. Of course del x tries to delete the local variable x, and since x doesn't exist yet, it fails with UnboundLocalError. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Mon Oct 16 23:40:14 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 17 Oct 2017 14:40:14 +1100 Subject: why del is not a function or method? References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> Message-ID: <858tga30ip.fsf@benfinney.id.au> Steve D'Aprano writes: > On Tue, 17 Oct 2017 03:16 am, Oren Ben-Kiki wrote: > > > That doesn't explain why `del` isn't a method though. > > `del` cannot be a method or a function, because the argument to `del` > is the name of the variable, not the contents of the variable. Since a Python ?variable? does not have contents, this is IMO another instance where using the term ?variable? is not helpful. >>> x = 123 >>> y = [0, x, 2, x, 4] Neither of the names ?x? nor ?y? have content; they are references to objects. The list itself also has references, which the Python code can use to get at the items. >>> y [0, 123, 2, 123, 4] >>> y[1] 123 So when we give an argument to ?del?, that argument is not always a ?variable?; it is always a reference. We can delete one item from a list, because that item is accessed via a reference; we give the same reference as argument to ?del?:: >>> del y[1] Both ?x? and ?y? remain bound. The reference that was at index 1 of the above list is deleted. >>> x 123 >>> y [0, 2, 123, 4] > then `del` needs to delete the *name* "x", not the value of x, namely > 123. If del were a function or method, it would only see the value, > 123, and have no idea what the name is. Hopefully when one thinks in terms of references ? and the use of a non-name reference, above, may make that distinction clear ? the operation of ?del? is easier to understand. -- \ ?The shortest distance between two points is under | `\ construction.? ?Noelie Alito | _o__) | Ben Finney From llanitedave at birdandflower.com Tue Oct 17 00:06:32 2017 From: llanitedave at birdandflower.com (llanitedave) Date: Mon, 16 Oct 2017 21:06:32 -0700 (PDT) Subject: Logging module stopped working Message-ID: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> I'm building an application that contains some long-running operations in a separate thread from the user interface. I've been using the logging module writing logging.info() statements to a .log file to keep track of the data interactions while it runs. In the midst of a recent run, the logging simply stopped working. The rest of the application continued on as if nothing had happened, but without the log I'm pretty much blind. I set up the logging code at the very beginning of the app, before any other work is done. Here's the relevant code: #!/usr/bin/env python3 import sys import os #import functions_classes from PyQt5 import QtGui, QtCore, QtWidgets from PyQt5.QtGui import * from PyQt5.QtCore import * from PyQt5.QtWidgets import * import sqlite3 import logging import inspect import threading import datetime #import local modules import qt_EnvTabs import globalnames import evocontrol import inputoutput class MainWindow(QMainWindow): def __init__(self): super().__init__() # set up logging logging.basicConfig(format='%(levelname)s:%(message)s', filename="sample.log", level=logging.DEBUG) logging.info("Starting system, MainWindow.__init__, %s", str(datetime.datetime.today())) self.createUI() I also have an earlier draft of the application that I saved into another directory. Its initial code is identical to what I posted here. I opened it, saw the first window activate, and then closed it. I checked for the sample.log file, it existed, and contained the proper message: "INFO:Starting system, MainWindow.__init__, 2017-10-16 20:58:40.988035" I did the same to the current file, and no log file was created at all! Between the time that the logging was working and the time it quit, the only changes I made were to add a couple of logging.info() statements into a downstream module. But that seems irrelevant here, as those modules aren't included in the above test. Is there any behind-the-scenes behavior that I'm missing? I'm stumped. From soyeomul at doraji.xyz Tue Oct 17 00:07:47 2017 From: soyeomul at doraji.xyz (Byung-Hee HWANG =?utf-8?B?KO2Zqeuzke2drCwg6buD?= =?utf-8?B?54Kz54aZKQ==?=) Date: Tue, 17 Oct 2017 13:07:47 +0900 Subject: how to read in the newsreader References: Message-ID: Andrew Z ?? ???, ??? ?? ???: > Gents, > how do i get this group in a newsreader? The digest i'm getting is not > workable for me - i can't reply , can only read the replies from the > members of the group. Or. maybe, it shouldn't be a news reader.... > please advise.. > > P.S. Oh the comp.lang.python is a nightmare because of spam... It would be nice "gmane.comp.python.general" via NNTP, there are no spam. -- ^????? _????_ ?????_^))// From rosuav at gmail.com Tue Oct 17 00:30:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 17 Oct 2017 15:30:16 +1100 Subject: Logging module stopped working In-Reply-To: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> References: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> Message-ID: On Tue, Oct 17, 2017 at 3:06 PM, llanitedave wrote: > I'm building an application that contains some long-running operations in a separate thread from the user interface. I've been using the logging module writing logging.info() statements to a .log file to keep track of the data interactions while it runs. > > In the midst of a recent run, the logging simply stopped working. The rest of the application continued on as if nothing had happened, but without the log I'm pretty much blind. > > I set up the logging code at the very beginning of the app, before any other work is done. Here's the relevant code: > > #!/usr/bin/env python3 > > import sys > import os > #import functions_classes > from PyQt5 import QtGui, QtCore, QtWidgets > from PyQt5.QtGui import * > from PyQt5.QtCore import * > from PyQt5.QtWidgets import * > import sqlite3 > import logging > import inspect > import threading > import datetime > > #import local modules > import qt_EnvTabs > import globalnames > import evocontrol > import inputoutput > > class MainWindow(QMainWindow): > def __init__(self): > super().__init__() > # set up logging > logging.basicConfig(format='%(levelname)s:%(message)s', filename="sample.log", level=logging.DEBUG) > logging.info("Starting system, MainWindow.__init__, %s", str(datetime.datetime.today())) > self.createUI() > > I also have an earlier draft of the application that I saved into another directory. Its initial code is identical to what I posted here. I opened it, saw the first window activate, and then closed it. I checked for the sample.log file, it existed, and contained the proper message: > "INFO:Starting system, MainWindow.__init__, 2017-10-16 20:58:40.988035" > > I did the same to the current file, and no log file was created at all! > > Between the time that the logging was working and the time it quit, the only > changes I made were to add a couple of logging.info() statements into a downstream module. But that seems irrelevant here, as those modules aren't included in the above test. > > Is there any behind-the-scenes behavior that I'm missing? I'm stumped. I'd be suspecting that something reconfigured the logging module, most likely changing the logging level. Audit the modules by either removing some and seeing if it still happens, or by combing through the code. Perhaps something was supposed to construct its own logger, but actually reconfigured the entire module? ChrisA From gengyangcai at gmail.com Tue Oct 17 00:42:30 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Tue, 17 Oct 2017 12:42:30 +0800 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: https://duckduckgo.com/html/?q=%22Cai%20Gengyang%22%20python ---- This seems to be the only method that works, using DuckDuckGo On Tue, Oct 17, 2017 at 4:54 AM, Thomas Jollans wrote: > On 16/10/17 22:04, Cai Gengyang wrote: > > I cannot remember where I posted the question .... it was a while ago .. > > > > On Mon, Oct 16, 2017 at 5:12 PM, Thomas Jollans > > wrote: > > > > On 2017-10-16 11:01, Cai Gengyang wrote: > > > Does anyone here know a way I can search for and display all my > > old posts on Python ? Thanks a lot. > > > > > > Gengyang > > > > > > > You already asked this recently. You received good answers. > > > It was just over one week ago. On this very list. > > Here is the original thread in the archive: > https://mail.python.org/pipermail/python-list/2017-October/727139.html > > And here is this one: > https://mail.python.org/pipermail/python-list/2017-October/727567.html > > > -- > Thomas Jollans > From steve+python at pearwood.info Tue Oct 17 01:07:15 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 16:07:15 +1100 Subject: why del is not a function or method? References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> Message-ID: <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 02:40 pm, Ben Finney wrote: > Steve D'Aprano writes: >> `del` cannot be a method or a function, because the argument to `del` >> is the name of the variable, not the contents of the variable. > > Since a Python ?variable? does not have contents, this is IMO another > instance where using the term ?variable? is not helpful. > > >>> x = 123 > >>> y = [0, x, 2, x, 4] > > Neither of the names ?x? nor ?y? have content; What I meant by "content" was "value". In hindsight, I don't even know why I wrote the word "content". It was not a good choice of words. Mea culpa. Of course the variables "x" and "y" have values: the value of x is 123, and the value of y is the specific list bound to the name "y" in the appropriate namespace. If you don't agree with that, then our terminology is so far apart that I don't believe we can ever agree, or understand each other. The point is, if del were a function, then calling del(x) would pass the *value* of x into the function, not the name 'x' -- unless the interpreter made del a special case, not a regular function. But if the interpreter did that, why bother with function notation and the extraneous parentheses? Why not just make it a statement and avoid surprising the programmer by giving the del function super-powers that no other function has? Which, of course, is exactly what Guido did. > they are references to objects. You are conflating the implementation details of how the Python VM implements name binding (a mapping between names and references to objects) and the high-level semantics of Python code. In CPython, at least, such references are pointers. They're something analogous in Jython and IronPython. It is sometimes useful to peer under the hood and talk about the implementation of the VM, but I don't think this is one of those times. There are too many inconsistencies in how we need to use the word "reference" to make this strategy viable -- see below. [...] > So when we give an argument to ?del?, that argument is not always a > ?variable?; it is always a reference. The argument to del is a comma-separated list of l-values, that is, the kind of expression which is legal on the left-hand side of an assignment statement. That includes: - bare names like `x` - qualified names like `module.x` or `instance.attribute` - items in mappings like `mydict[key]` - items in sequences like `mylist[0]` - lists and tuples of such l-values, e.g. this is legal: py> a, b, c, d = 1, 2, 3, 4 py> del a, [b, c], d as well as more complex examples. But you can't pass an arbitrary "reference" to del and expect something sensible: del math.sin(1) would, if it were legal, evaluate the expression math.sin(1) and return the float 0.8414... (or, if you prefer to talk about the *implementation*, return a reference to the float object) and then try to delete it in some sense. But that's not legal, and you get a syntax error at compile time. If del accepted references in the usual sense, this would be legal but pointless: del would receive the reference to the float object, and delete the reference, leaving the float to possibly be garbage collected. But that's not what del does. > We can delete one item from a list, because that item is accessed via a > reference; we give the same reference as argument to ?del?:: > > >>> del y[1] In context, I was specifically referring to unqualified names because deleting list items *can* be done with a method call. In fact, the above line ends up calling y.__delitem__[1]. In an alternative universe, Guido might have forgone the `del` statement and required us to call: list.pop(1) or similar for dicts and other collections. The OP was already thinking along those lines when he or she asked the question, so I didn't think I needed to cover that case. But the reason del is a statement is that it *also* covers cases where we cannot use a regular method or function, and the simplest such case is unbinding a name. Now, to go back to your comment that "we give the same reference as argument to ?del?" that's incorrect, or at least, in order for it to be correct we must use *two* different meanings to the word "reference", depending on the context. If I write: y = [None, "Aardvark", None, None] print(y[1]) # (A) del y[1] # (B) then the same descriptive text: "the reference y[1]" needs to be understood in two different ways: - in line (A) y[1] is a reference to the string object "Aardvark"; - in line (B) y[1] is *not* a reference to the string object, but stands for the 1st position in the sequence referred to by `y`. So it is misleading, or at least surprising, to use the term "reference" in both situations. It isn't clear whether you mean reference to the value or reference or not. In logic and philosophy, we often need to think about the distinction between *using* a word and *mentioning* the word: https://en.wikipedia.org/wiki/Use%E2%80%93mention_distinction Failing to distinguish the two leads to fallacies: https://www.logicallyfallacious.com/tools/lp/Bo/LogicalFallacies/180/Use-Mention-Error some of which are pretty obvious ("Python is not a programming language, it is a six-letter word"), and others not so obvious. Python has an analogous distinction: (1) Python's evaluation of an expression is normally analogous to *using* a word. For example: print(spam.eggs) the interpreter evaluates the expression `spam.eggs`, returning an object of some kind (often called an "r-value", or just a value), then passes the object to print, which prints it. (2) But binding operations (including unbinding) are analogous to *mentioning* an expression (with the condition that only certain kinds of expressions are legal as assignment targets). For example: del spam.eggs does not evaluate the expression `spam.eggs` and pass the resulting object to del. That's why del cannot be a function: it needs to operate on its arguments before the interpreter evaluates them to objects. Of course, there is a way that we could make del a regular function, even for unqualified names[2]: we could quote the arguments ourselves. So to delete the bare name "x" from the global scope, we could write: delete("x", globals()) To delete an attribute or qualified name, we could write: delete("attribute", instance) # or use delattr(instance, "attribute") delete("x", module) To delete an item from a list: delete(1, mylist) # or use mylist.pop(1) and so forth. All pretty clumsy and ugly, but it could work. So the ultimate reason why del is a statement rather than a function is that Guido wanted to be able to unbind r-values without having to quote them first. [1] Except in the loosest sense that any use of words to refer to a concept is a reference, e.g. the words "Queen Elizabeth" is a reference to the current monarch of the UK, Elizabeth Windsor. [2] Almost. It would be difficult or impossible to delete a local (but why would you want to?), a nonlocal, or a class-level name binding from inside the class. E.g.: class K: x = 1 del x -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 17 01:14:42 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 16:14:42 +1100 Subject: why del is not a function or method? References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <59e591c5$0$14932$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 04:07 pm, Steve D'Aprano wrote: > [1] Except in the loosest sense that any use of words to refer to a concept > is a reference, e.g. the words "Queen Elizabeth" is a reference to the > current monarch of the UK, Elizabeth Windsor. Oops, ignore this footnote! It belonged to a paragraph I ended up deleting. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 17 02:57:46 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 17 Oct 2017 17:57:46 +1100 Subject: Logging module stopped working References: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> Message-ID: <59e5a9ee$0$14943$b1db1813$d948b532@news.astraweb.com> On Tue, 17 Oct 2017 03:06 pm, llanitedave wrote: [...] > I set up the logging code at the very beginning of the app, before any other > work is done. Here's the relevant code: > > #!/usr/bin/env python3 [... snip imports ...] > class MainWindow(QMainWindow): > def __init__(self): > super().__init__() > # set up logging > logging.basicConfig(format='%(levelname)s:%(message)s', > filename="sample.log", level=logging.DEBUG) > logging.info("Starting system, MainWindow.__init__, %s", > str(datetime.datetime.today())) > self.createUI() According to this code, no logging will occur because no MainWindow is created. Nor is there any sort of main event loop. I'm sure that you're not actually so silly that you forgot to create a window at all, but the point is, this example is *not* "the relevant code". It is only *part* of the relevant code. Who knows what else is happening that might be preventing logging from working? We don't know because we can't see the rest of the relevant code. Perhaps you need to focus on that. [...] > Between the time that the logging was working and the time it quit, the only > changes I made were to add a couple of logging.info() statements into a > downstream module. But that seems irrelevant here, as those modules aren't > included in the above test. I doubt that. I expect there must be some other change you have forgotten, and it is *that* which has disabled logging. Maybe you call something which sets the logging level above INFO? I would start with this: import sys import os import logging logging.basicConfig(format='%(levelname)s:%(message)s', filename="sample.log", level=logging.DEBUG) logging.info("Starting module, %s", str(datetime.datetime.today())) # import everything else ... logging.info("Importing complete %s", str(datetime.datetime.today())) class MainWindow(QMainWindow): def __init__(self): super().__init__() logging.info("Creating window, %s", str(datetime.datetime.today())) self.createUI() logging.info("Class created %s", str(datetime.datetime.today())) window = MainWindow() logging.info("Window created %s", str(datetime.datetime.today())) logging.critical("Can you see this? %s", str(datetime.datetime.today())) # plus whatever else is needed to make the application run ... That will show precisely where and when logging stops: 1. Does it work at all, straight out of the logging module? If not, then something broke it before your module even gets loaded. 2. Does it still work after all the other imports? If not, then bisect the imports until you locate which module breaks logging. 3. Do you get the "Class created" and "Window created" messages? If no, then that helps narrow down where the fault may lie. E.g. if you see the first, but not the second, then something in super().__init__ may be disabling logging; if you don't see the first, then look at QMainWindow's metaclass, if it has one. If it doesn't have a metaclass, then that's a mystery -- maybe something in some other thread is messing you about? 4. If you see the CRITICAL message, but not the INFO ones, then something has reset the logging level. 5. What happens if you delete sample.log? Does it get re-created? If not, maybe there's a permissions error. 6. If you're not seeing *any* logging at all, perhaps you need to fall back on calling print() directly. Run your application from a terminal, and see what it prints. If even print isn't working, then follow the three Rs: Reboot, Reinstall, and Resign :-) 7. Perhaps a bit less drastic, you can insert a debugging checkpoint in the code, run the application from a terminal, and when it halts at the debugger, have a look at the state of the logging module. 8. Just to be absolutely sure, check logging.__file__ to ensure you haven't shadowed the real logging module with some sort of mock. But now I'm really clutching at straws, because if that were the case, I'd expect there to be an exception, not just logging failing to work. I'm intrigued by this error, and would love to hear what caused it when you find out. Please respond back on the list with your diagnosis. By the way, this sort of spooky action-at-a-distance is why I prefer to create my own loggers, rather than use the global one. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From Karsten.Hilbert at gmx.net Tue Oct 17 03:30:43 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Tue, 17 Oct 2017 09:30:43 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> Message-ID: <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> On Tue, Oct 17, 2017 at 12:04:09PM +1100, Steve D'Aprano wrote: > > is this the right list to ask for help in debugging a > > SIGABORT (?) happening on shutdown of a Python 2.7 script ? > > > > If not, which one is ? > > You should try here first. > > Please ensure you read and follow this first: > > http://sscce.org/ > > and post a *minimum* example of your code. (Sorry for the redundant > instructions if you already know this, but you would be surprised how many > people don't.) Thanks. I'll work towards a postable example. Running a debug build of py27 gave me a first lead: this Debian system (Testing, upgraded all the way from various releases ago) carries an incompatible mxDateTime which I'll take care of. *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "./bootstrap_gm_db_system.py", line 87, in from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, gmI18N File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", line 34, in from Gnumed.pycommon import gmDateTime File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", line 52, in import mx.DateTime as mxDT File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in from DateTime import * File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in from mxDateTime import * File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in raise ImportError, why ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From tjol at tjol.eu Tue Oct 17 03:45:11 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 17 Oct 2017 09:45:11 +0200 Subject: how to read in the newsreader In-Reply-To: References: Message-ID: <464018d3-5930-fa10-e272-f20015a3643b@tjol.eu> On 17/10/17 00:55, Mikhail V wrote: > Thomas wrote: >> >> On 16/10/17 20:02, Pete Forman wrote: >>> Thomas Jollans writes: >>> ... >>>>> If you do stick with a digest then check your newsreader for a feature >>>>> to expand it. Then you can read and reply as if you were getting >>>>> individual posts. >>>>> >>>> That exists? How does it work? >>> >>> The Gnus newsreader in Emacs allows you to type C-d on a digest to run >>> gnus-summary-enter-digest-group. That then behaves the same as if you >>> opened any other summary such as a regular Usenet group. >>> >> >> Does it set the References header correctly when replying? > > Hi Thomas, regarding the issue with my reply-to header you told me recently - > Does this message looks ok in your threaded view now? > > Now I reply by the href of the message I looked up in pipermail archive, > just curious if it looks now correctly. I'm afraid not. -- Thomas Jollans From tjol at tjol.eu Tue Oct 17 05:16:26 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 17 Oct 2017 11:16:26 +0200 Subject: why del is not a function or method? In-Reply-To: <59e557e4$0$14940$b1db1813$d948b532@news.astraweb.com> References: <3P4FB.14035$d33.6699@fx42.am4> <50f90389-c753-2d27-f29d-e2ab9049ca54@tjol.eu> <59e557e4$0$14940$b1db1813$d948b532@news.astraweb.com> Message-ID: <8ab7a380-2b70-dd07-a292-bf449e855607@tjol.eu> On 2017-10-17 03:07, Steve D'Aprano wrote: > On Tue, 17 Oct 2017 08:57 am, Thomas Jollans wrote: > >> On 16/10/17 21:12, Stefan Ram wrote: >>> ram at zedat.fu-berlin.de (Stefan Ram) writes: >>>> ?x = None? observably has not the same effect as ?del x?: >>> >>> Paradoxically, /deleting/ a local variable which did not >>> ever exist, has the effect of creating a ghost of that >>> local variable which then will hide a global variable: > > [...] >>> . Case 2: The ?del x? creates a ghost ?local variable 'x'? >>> which now will hide the global variable ?x?: >>> >>> |>>> x = 9 >>> |>>> def f(): >>> |... del x >>> |... print( x ) >>> |... >>> |>>> f() >>> |UnboundLocalError: local variable 'x' referenced before assignment >>> >>> . >> >> That's not what happens. The UnboundLocalError is in the del statement, >> not the print call: > > That's exactly what happens! Stefan merely explains it poorly. There's no > mysterious "ghost", it is just that x is treated as a local variable instead > of a global, and you can't delete the local before it is bound to. > > `del x` is treated as a form of assignment, just like `x = 123`, and makes `x` > a local variable. So in this code: You're right of course. While in Stefan's example code the error *was* in the del statement, and trying to delete something that's not either present in the local namespace or explicitly "global" *will* fail, del *does* make a variable local. >>> x = 1 >>> def f(): ... try: ... del x ... except: ... print('del exception caught') ... print(x) ... >>> f() del exception caught Traceback (most recent call last): File "", line 1, in File "", line 6, in f UnboundLocalError: local variable 'x' referenced before assignment >>> def g(): ... global x ... del x ... >>> g() >>> x Traceback (most recent call last): File "", line 1, in NameError: name 'x' is not defined >>> > > def f(): > print(x) > > there is no assignment to x, so it is treated as a global. But if we write: > > def f(): > x = 123 > print(x) > > the Python compiler sees the assignment to x and makes x a local variable. > Note that the rules for this are deliberately very, very simple, so this will > make x a local: > > def f(): > print(x) # x is local > x = 123 > > and even this will too: > > def f(): > if False: > x = 123 # dead code > print(x) # but enough to make x a local > > > You can replace the line "x = 123" with "del x" in any of those functions, and > you will get the same effect: x will be treated as a local. > > > From jasonhihn at gmail.com Tue Oct 17 10:52:17 2017 From: jasonhihn at gmail.com (Jason) Date: Tue, 17 Oct 2017 07:52:17 -0700 (PDT) Subject: multiprocessing shows no benefit Message-ID: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> I've got problem that I thought would scale well across cores. def f(t): return t[0]-d[ t[1] ] d= {k: np.array(k) for k in entries_16k } e = np.array() pool.map(f, [(e, k) for k in d] At the heart of it is a list of ~16k numpy arrays (32 3D points) which are stored in a single dict. Using pool.map() I pass the single item of 32 3D Points to be evaluated again the 16k entries. In theory, this would be a speedup proportional to the number of physical cores, but I see all 4 cores being maxed out and results in a regular map time. How can I use pool.map better? From tomuxiong at gmx.com Tue Oct 17 11:38:04 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Tue, 17 Oct 2017 17:38:04 +0200 Subject: multiprocessing shows no benefit In-Reply-To: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> Message-ID: <4996a311-682d-2eb8-e924-130b6a21060f@gmx.com> Could you post a full code snippet? If the lists of 16k numpy arrays are fixed (say you read them from a file), you could just generate random values that could be fed into the code as your list would. It's hard to say how things could be sped up without a bit more specificity. Cheers, Thomas From tjreedy at udel.edu Tue Oct 17 11:44:43 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Oct 2017 11:44:43 -0400 Subject: why del is not a function or method? In-Reply-To: <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/17/2017 1:07 AM, Steve D'Aprano wrote: > The point is, if del were a function, then calling del(x) would pass the > *value* of x into the function, not the name 'x' So we would have to quote either the entire rest of the statement, or each item separately, > -- unless the interpreter made del a special case, not a regular function. This is what Lisp does. Most functions are 'normal': unquoted argument expressions are evaluated. Some are 'special': at least one of the argument expressions is automatically quoted or treated specially is some way. Users have to memorize which functions are special and which arguments are automatically quoted. Learning the exceptions and quoting rules was for me a barrier to learning Lisp. In Python, the 'special functions' of Lisp are statements. With print now a function, I believe every python statement with arguments does something special with at least one argument. In assignments statements, for instance, everything to the right of the final '=' is evaluated normally. Everything to the left of any '=' gets special treatment. The final 'get object' part of expression evaluation is replace by 'create association' with the corresponding object on the right side. In CPython left-hand expressions are not merely quoted for runtime evaluation and use. They are parsed at compile time and compiled to almost normal bytecode. The difference is that a load bytecode is replace by a store bytecode (that takes a second argument). >>> dis('a[b] = c[d]') 1 0 LOAD_NAME 0 (c) 2 LOAD_NAME 1 (d) 4 BINARY_SUBSCR 6 LOAD_NAME 2 (a) 8 LOAD_NAME 3 (b) 10 STORE_SUBSCR 12 LOAD_CONST 0 (None) 14 RETURN_VALUE >>> dis('a(b).c = d(e).f') 1 0 LOAD_NAME 0 (d) 2 LOAD_NAME 1 (e) 4 CALL_FUNCTION 1 6 LOAD_ATTR 2 (f) 8 LOAD_NAME 3 (a) 10 LOAD_NAME 4 (b) 12 CALL_FUNCTION 1 14 STORE_ATTR 5 (c) 16 LOAD_CONST 0 (None) 18 RETURN_VALUE For statements executed repeatedly, this is more efficient than quoting for runtime evaluation. > But if the interpreter did that, why bother with function notation and the > extraneous parentheses? Why not just make it a statement and avoid surprising > the programmer by giving the del function super-powers that no other function > has? Tcl does not use parentheses for function calls. But with only one syntax, it still needs arcane quoting rules. -- Terry Jan Reedy From marko at pacujo.net Tue Oct 17 12:01:07 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 17 Oct 2017 19:01:07 +0300 Subject: why del is not a function or method? References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <87tvyx9324.fsf@elektro.pacujo.net> Terry Reedy : > On 10/17/2017 1:07 AM, Steve D'Aprano wrote: >> -- unless the interpreter > made del a special case, not a regular function. > > This is what Lisp does. Most functions are 'normal': unquoted argument > expressions are evaluated. Some are 'special': at least one of the > argument expressions is automatically quoted or treated specially is > some way. Users have to memorize which functions are special and which > arguments are automatically quoted. Learning the exceptions and > quoting rules was for me a barrier to learning Lisp. Yes, Lisp (including Scheme) has three kinds of forms: functions: Argument forms are evaluated before passed on to the "callable". The result is returned as is. macros (syntactic forms): Argument forms are passed on to the callable without evaluation. The result is evaluated and then returned. special forms: Argument forms are passed on to the callable without evaluation. The result is returned as is. The biggest ugly spot in the arrangement is that while functions are first-class objects in Lisp, macros and special forms are not. (There are dialects, though, where they all are first class.) Marko From tjreedy at udel.edu Tue Oct 17 12:03:02 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Oct 2017 12:03:02 -0400 Subject: multiprocessing shows no benefit In-Reply-To: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> Message-ID: On 10/17/2017 10:52 AM, Jason wrote: > I've got problem that I thought would scale well across cores. What OS? > def f(t): > return t[0]-d[ t[1] ] > > d= {k: np.array(k) for k in entries_16k } > e = np.array() > pool.map(f, [(e, k) for k in d] *Every* multiprocessing example in the doc intentionally protects multiprocessing calls with if __name__ == '__main__': "Safe importing of main module Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process)." > At the heart of it is a list of ~16k numpy arrays (32 3D points) which are stored in a single dict. Using pool.map() I pass the single item of 32 3D Points to be evaluated again the 16k entries. In theory, this would be a speedup proportional to the number of physical cores, but I see all 4 cores being maxed out and results in a regular map time. > > How can I use pool.map better? Try following the doc and see what happens. -- Terry Jan Reedy From bc at freeuk.com Tue Oct 17 12:19:22 2017 From: bc at freeuk.com (bartc) Date: Tue, 17 Oct 2017 17:19:22 +0100 Subject: why del is not a function or method? In-Reply-To: References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> Message-ID: <74qFB.17573$th3.15570@fx24.am4> On 17/10/2017 16:44, Terry Reedy wrote: > In CPython left-hand expressions are not merely quoted for runtime > evaluation and use.? They are parsed at compile time and compiled to > almost normal bytecode.? The difference is that a load bytecode is > replace by a store bytecode (that takes a second argument). What does the second argument do? I thought Load and Store were of equal rank. -- bartc From llanitedave at birdandflower.com Tue Oct 17 12:22:48 2017 From: llanitedave at birdandflower.com (llanitedave) Date: Tue, 17 Oct 2017 09:22:48 -0700 (PDT) Subject: Logging module stopped working In-Reply-To: <59e5a9ee$0$14943$b1db1813$d948b532@news.astraweb.com> References: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> <59e5a9ee$0$14943$b1db1813$d948b532@news.astraweb.com> Message-ID: <017a94c2-7434-4f67-94ca-9aa337f5f3ee@googlegroups.com> Those are some good suggestions, I've found that I won't be able to work on it today, but I'll definitely follow up tomorrow. As for not showing all the code, the main window and its associated code are definitely there and working. I didn't post it because of all the setup code for fields and buttons and menus that I'd have to filter out. The main point was that I had two files containing identical code (all the differences between the two drafts were in other modules) yet one activated the logging properly and one didn't. It's the silent failure that bothers me, I would have hoped that if I'd done something really boneheaded it would have made some noise. I'll try those suggestions and post back. Thanks for the thoughtful help. From daniel.p.flick at gmail.com Tue Oct 17 12:59:21 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 09:59:21 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 Message-ID: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> I am very new to Python and have been struggling to find some info on processing IP addresses. get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I can do this with netaddr but I am working with Mako templates and ipaddress is a built in module so there are less dependencies. Any ideas? From israel at ravnalaska.net Tue Oct 17 13:26:50 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 17 Oct 2017 09:26:50 -0800 Subject: PEP 249 Compliant error handling Message-ID: I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the string to let them know there was an error, and will allow retrieval of any good data. Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From rgaddi at highlandtechnology.invalid Tue Oct 17 13:36:37 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 17 Oct 2017 10:36:37 -0700 Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: On 10/17/2017 09:59 AM, Daniel Flick wrote: > I am very new to Python and have been struggling to find some info on processing IP addresses. > > get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I can do this with netaddr but I am working with Mako templates and ipaddress is a built in module so there are less dependencies. > > Any ideas? > You mean, other than .split('/')? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From __peter__ at web.de Tue Oct 17 13:45:38 2017 From: __peter__ at web.de (Peter Otten) Date: Tue, 17 Oct 2017 19:45:38 +0200 Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: Daniel Flick wrote: > I am very new to Python and have been struggling to find some info on > processing IP addresses. > > get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I can > do this with netaddr but I am working with Mako templates and ipaddress is > a built in module so there are less dependencies. > > Any ideas? You mean you have >>> import ipaddress >>> n = ipaddress.ip_network("192.168.1.128/25") >>> n IPv4Network('192.168.1.128/25') ? Then one >>> dir(n) [snip private names] '_version', 'address_exclude', 'broadcast_address', 'compare_networks', 'compressed', 'exploded', 'hostmask', 'hosts', 'is_global', 'is_link_local', 'is_loopback', 'is_multicast', 'is_private', 'is_reserved', 'is_unspecified', 'max_prefixlen', 'netmask', 'network_address', 'num_addresses', 'overlaps', 'prefixlen', 'subnets', 'supernet', 'version', 'with_hostmask', 'with_netmask', 'with_prefixlen'] later: >>> n.network_address IPv4Address('192.168.1.128') From python at mrabarnett.plus.com Tue Oct 17 14:35:26 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 17 Oct 2017 19:35:26 +0100 Subject: PEP 249 Compliant error handling In-Reply-To: References: Message-ID: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> On 2017-10-17 18:26, Israel Brewster wrote: > I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the string to l > et them know there was an error, and will allow retrieval of any good data. > > Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. > If a particular text field is corrupted, then raising UnicodeDecodeError when trying to get the contents of that field as a Unicode string seems reasonable to me. Is there a way to get the contents as a bytestring, or to get the contents with a different errors parameter, so that the user has the means to fix it (if it's fixable)? From petef4+usenet at gmail.com Tue Oct 17 14:38:12 2017 From: petef4+usenet at gmail.com (Pete Forman) Date: Tue, 17 Oct 2017 19:38:12 +0100 Subject: how to read in the newsreader References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> <0b465e5b-86ee-30c6-c693-7cc179c449d9@tjol.eu> Message-ID: Thomas Jollans writes: > On 16/10/17 20:02, Pete Forman wrote: >> Thomas Jollans writes: >> >>> On 2017-10-16 08:48, Pete Forman wrote: >>>> Andrew Z writes: >>>> >>>>> hmm. i did do that. maybe just a delay. >>>>> I'll see how it will go tomorrow then. Thank you gents. >>>>> >>>>> On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: >>>>> >>>>>> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >>>>>>> Michael, that's what i use too - gmail. But i get the digest only >>>>>>> and can't really reply that way. i was hoping to get the >>>>>>> mail.python.org list.... >>>>>> >>>>>> Turn off digests then. Easy! >>>> >>>> If you do stick with a digest then check your newsreader for a feature >>>> to expand it. Then you can read and reply as if you were getting >>>> individual posts. >>>> >>> That exists? How does it work? >> >> The Gnus newsreader in Emacs allows you to type C-d on a digest to run >> gnus-summary-enter-digest-group. That then behaves the same as if you >> opened any other summary such as a regular Usenet group. >> > > Does it set the References header correctly when replying? Sorry, I am not in a position to test. The only digest I subscribe to is comp.risks. The only messsage id in that is a single one for the whole digest. Each article only has date, subject and from headers. You would need to look inside a Python digest to see if it carries more headers for the articles. If they are not present then they cannot be used when composing a reply. -- Pete Forman https://payg.pythonanywhere.com From arj.python at gmail.com Tue Oct 17 14:51:27 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 17 Oct 2017 22:51:27 +0400 Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: any idea why this is on spam ? Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 17 Oct 2017 21:00, "Daniel Flick" wrote: > I am very new to Python and have been struggling to find some info on > processing IP addresses. > > get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I > can do this with netaddr but I am working with Mako templates and ipaddress > is a built in module so there are less dependencies. > > Any ideas? > -- > https://mail.python.org/mailman/listinfo/python-list > From israel at ravnalaska.net Tue Oct 17 15:25:25 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 17 Oct 2017 11:25:25 -0800 Subject: PEP 249 Compliant error handling In-Reply-To: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> References: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> Message-ID: <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> > On Oct 17, 2017, at 10:35 AM, MRAB wrote: > > On 2017-10-17 18:26, Israel Brewster wrote: >> I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the s > tring to > l >> et them know there was an error, and will allow retrieval of any good data. >> Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. > If a particular text field is corrupted, then raising UnicodeDecodeError when trying to get the contents of that field as a Unicode string seems reasonable to me. > > Is there a way to get the contents as a bytestring, or to get the contents with a different errors parameter, so that the user has the means to fix it (if it's fixable)? That's certainly a possibility, if that behavior conforms to the DB API "standards". My concern in this front is that in my experience working with other PEP 249 modules (specifically psycopg2), I'm pretty sure that columns designated as type VARCHAR or TEXT are returned as strings (unicode in python 2, although that may have been a setting I used), not bytes. The other complication here is that the 4D database doesn't use the UTF-8 encoding typically found, but rather UTF-16LE, and I don't know how well this is documented. So not only is the bytes representation completely unintelligible for human consumption, I'm not sure the average end-user would know what decoding to use. In the end though, the main thing in my mind is to maintain "standards" compatibility - I don't want to be returning bytes if all other DB API modules return strings, or visa-versa for that matter. There may be some flexibility there, but as much as possible I want to conform to the majority/standard/whatever ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > -- > https://mail.python.org/mailman/listinfo/python-list From nad at python.org Tue Oct 17 15:35:57 2017 From: nad at python.org (Ned Deily) Date: Tue, 17 Oct 2017 15:35:57 -0400 Subject: [RELEASE] Python 3.7.0a2 is now available for testing Message-ID: <77DF3A3A-DC16-46A3-A933-B0CDA771EAB9@python.org> Python 3.7.0a2 is the second of four planned alpha previews of Python 3.7, the next feature release of Python. During the alpha phase, Python 3.7 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview, 3.7.0a3, is planned for 2017-11-27. You can find Python 3.7.0a2 and more information here: https://www.python.org/downloads/release/python-370a2/ -- Ned Deily nad at python.org -- [] From Karsten.Hilbert at gmx.net Tue Oct 17 15:38:22 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Tue, 17 Oct 2017 21:38:22 +0200 Subject: Aw: Re: PEP 249 Compliant error handling In-Reply-To: <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> References: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> Message-ID: > That's certainly a possibility, if that behavior conforms to the DB API "standards". My concern in this front is that in my experience working with other PEP 249 modules (specifically psycopg2), I'm pretty sure that columns designated as type VARCHAR or TEXT are returned as strings (unicode in python 2, although that may have been a setting I used), not bytes. The other complication here is that the 4D database doesn't use the UTF-8 encoding typically found, but rather UTF-16LE, and I don't know how well this is documented. So not only is the bytes representation completely unintelligible for human consumption, I'm not sure the average end-user would know what decoding to use. > > In the end though, the main thing in my mind is to maintain "standards" compatibility - I don't want to be returning bytes if all other DB API modules return strings, or visa-versa for that matter. There may be some flexibility there, but as much as possible I want to conform to the majority/standard/whatever The thing here is that you don't want to return data AS IF it was correct despite it having been "corrected" by some driver logic. What might be interesting to users is to set an attribute on the cursor, say, cursor.faulty_data = unicode(faulty_data, errors='replace') or some such in order to improve error messages to the end user. Karsten From ned at nedbatchelder.com Tue Oct 17 15:48:53 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 17 Oct 2017 15:48:53 -0400 Subject: [RELEASE] Python 3.7.0a2 is now available for testing In-Reply-To: <77DF3A3A-DC16-46A3-A933-B0CDA771EAB9@python.org> References: <77DF3A3A-DC16-46A3-A933-B0CDA771EAB9@python.org> Message-ID: <5a34b330-3852-225e-7203-f748e0c7b9cc@nedbatchelder.com> On 10/17/17 3:35 PM, Ned Deily wrote: > Python 3.7.0a2 is the second of four planned alpha previews of Python 3.7, > the next feature release of Python. During the alpha phase, Python 3.7 > remains under heavy development: additional features will be added > and existing features may be modified or deleted. Please keep in mind > that this is a preview release and its use is not recommended for > production environments. The next preview, 3.7.0a3, is planned for > 2017-11-27. You can find Python 3.7.0a2 and more information here: > > https://www.python.org/downloads/release/python-370a2/ > The coverage.py test suite passes. Keep up the good work! :) --Ned. From nhilterbrand at gmail.com Tue Oct 17 15:52:25 2017 From: nhilterbrand at gmail.com (Nathan Hilterbrand) Date: Tue, 17 Oct 2017 15:52:25 -0400 Subject: Fwd: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: Hit wrong button before. ---------- Forwarded message ---------- From: Nathan Hilterbrand Date: Tue, Oct 17, 2017 at 2:22 PM Subject: Re: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 To: Rob Gaddi I may have misunderstood what you were looking for, but went ahead and cobbled this ugly function together. Takes a string like "192.168.1.128/25" as input, and returns the subnet address. Just slapped together, so not the most robust thing in the world def subnet(inp): addr, bits = inp.split('/') mask = 2 ** (32 - int(bits)) - 1 allones = (2 ** 32) - 1 mask = allones ^ mask addroctets = [int(o) for o in addr.split('.')] addrval = 0 for o in addroctets: addrval = addrval * 256 + o snval = addrval & mask snoctets = [] q = snval for i in range(4): q,r = divmod(q, 256) snoctets.insert(0,str(r)) sn = ".".join(snoctets) return sn On Tue, Oct 17, 2017 at 1:36 PM, Rob Gaddi wrote: > On 10/17/2017 09:59 AM, Daniel Flick wrote: > >> I am very new to Python and have been struggling to find some info on >> processing IP addresses. >> >> get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I >> can do this with netaddr but I am working with Mako templates and ipaddress >> is a built in module so there are less dependencies. >> >> Any ideas? >> >> > You mean, other than .split('/')? > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. > -- > https://mail.python.org/mailman/listinfo/python-list > From python at mrabarnett.plus.com Tue Oct 17 16:02:29 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 17 Oct 2017 21:02:29 +0100 Subject: PEP 249 Compliant error handling In-Reply-To: <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> References: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> Message-ID: On 2017-10-17 20:25, Israel Brewster wrote: > >> On Oct 17, 2017, at 10:35 AM, MRAB > > wrote: >> >> On 2017-10-17 18:26, Israel Brewster wrote: >>> I have written and maintain a PEP 249 compliant (hopefully) DB API >>> for the 4D database, and I've run into a situation where corrupted >>> string data from the database can cause the module to error out. >>> Specifically, when decoding the string, I get a "UnicodeDecodeError: >>> 'utf-16-le' codec can't decode bytes in position 86-87: illegal >>> UTF-16 surrogate" error. This makes sense, given that the string >>> data got corrupted somehow, but the question is "what is the proper >>> way to deal with this in the module?" Should I just throw an error >>> on bad data? Or would it be better to set the errors parameter to >>> something like "replace"? The former feels a bit more "proper" to me >>> (there's an error here, so we throw an error), but leaves the end >>> user dead in the water, with no way to retrieve *any* of the data >>> (from that row at least, and perhaps any rows after it as well). The >>> latter option sort of feels like sweeping the problem under the rug, >>> but does at least leave an error character in the s >> tring to >> l >>> ?et them know there was an error, and will allow retrieval of any >>> good data. >>> Of course, if this was in my own code I could decide on a >>> case-by-case basis what the proper action is, but since this a >>> module that has to work in any situation, it's a bit more complicated. >> If a particular text field is corrupted, then raising >> UnicodeDecodeError when trying to get the contents of that field as a >> Unicode string seems reasonable to me. >> >> Is there a way to get the contents as a bytestring, or to get the >> contents with a different errors parameter, so that the user has the >> means to fix it (if it's fixable)? > > That's certainly a possibility, if that behavior conforms to the DB > API "standards". My concern in this front is that in my experience > working with other PEP 249 modules (specifically psycopg2), I'm pretty > sure that columns designated as type VARCHAR or TEXT are returned as > strings (unicode in python 2, although that may have been a setting I > used), not bytes. The other complication here is that the 4D database > doesn't use the UTF-8 encoding typically found, but rather UTF-16LE, > and I don't know how well this is documented. So not only is the bytes > representation completely unintelligible for human consumption, I'm > not sure the average end-user would know what decoding to use. > > In the end though, the main thing in my mind is to maintain > "standards" compatibility - I don't want to be returning bytes if all > other DB API modules return strings, or visa-versa for that matter. > There may be some flexibility there, but as much as possible I want to > conform to the majority/standard/whatever > The average end-user might not know which encoding is being used, but providing a way to read the underlying bytes will give a more experienced user the means to investigate and possibly fix it: get the bytes, figure out what the string should be, update the field with the correctly decoded string using normal DB instructions. From rosuav at gmail.com Tue Oct 17 16:05:09 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 18 Oct 2017 07:05:09 +1100 Subject: why del is not a function or method? In-Reply-To: <74qFB.17573$th3.15570@fx24.am4> References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> <74qFB.17573$th3.15570@fx24.am4> Message-ID: On Wed, Oct 18, 2017 at 3:19 AM, bartc wrote: > On 17/10/2017 16:44, Terry Reedy wrote: > >> In CPython left-hand expressions are not merely quoted for runtime >> evaluation and use. They are parsed at compile time and compiled to almost >> normal bytecode. The difference is that a load bytecode is replace by a >> store bytecode (that takes a second argument). > > > What does the second argument do? > > I thought Load and Store were of equal rank. CPython's byte code is a stack-based processor. So "load X" means "push X onto the stack", and "store X" means "pop the top of the stack and store it into X". For simple names, those are perfect counterparts. But for dotted or subscripted lookups, it's like this: >>> dis.dis("x[1]=y[2]") 1 0 LOAD_NAME 0 (y) 2 LOAD_CONST 0 (2) 4 BINARY_SUBSCR 6 LOAD_NAME 1 (x) 8 LOAD_CONST 1 (1) 10 STORE_SUBSCR 12 LOAD_CONST 2 (None) 14 RETURN_VALUE (Ignore the last two lines - they're an implicit "return None" at the end of the "function" that I just compiled.) Loading y[2] is done in several steps. First, we get the value of y (which could be an arbitrary expression; in this case, it's just a simple name). Then, we take the constant that we're subscripting with (the integer 2). Finally, we do a BINARY_SUBSCR, which is the square-bracket lookup operator. That leaves us with something on the stack. The load took one argument (the name "y"), and then the subscripting was a separate operation with two arguments (y and 2). Storing into x[1], on the other hand, is collapsed down a bit. We first load up the value of x (again, that could be any expression), and then the subscript (the integer 1). Then the subscripting and assignment are done in one STORE_SUBSCR operation; it takes three arguments off the stack (the new value, the object, and the subscript), and leaves nothing behind. You can also look at it from the POV of the dunder methods that let you customize this. >>> class Foo: def __getitem__(self, item): print("Getting", item) return 42 def __setitem__(self, item, value): print("Setting", item, "to", value) >>> Foo()[1] = Foo()[2] Getting 2 Setting 1 to 42 The setting operation needs to know the object (self), the subscript (item), and the new value. Does that make sense? ChrisA From bc at freeuk.com Tue Oct 17 16:35:01 2017 From: bc at freeuk.com (bartc) Date: Tue, 17 Oct 2017 21:35:01 +0100 Subject: why del is not a function or method? In-Reply-To: References: <59e55489$0$14938$b1db1813$d948b532@news.astraweb.com> <858tga30ip.fsf@benfinney.id.au> <59e59006$0$14931$b1db1813$d948b532@news.astraweb.com> <74qFB.17573$th3.15570@fx24.am4> Message-ID: On 17/10/2017 21:05, Chris Angelico wrote: > On Wed, Oct 18, 2017 at 3:19 AM, bartc wrote: >> On 17/10/2017 16:44, Terry Reedy wrote: >> >>> In CPython left-hand expressions are not merely quoted for runtime >>> evaluation and use. They are parsed at compile time and compiled to almost >>> normal bytecode. The difference is that a load bytecode is replace by a >>> store bytecode (that takes a second argument). >> >> >> What does the second argument do? >> >> I thought Load and Store were of equal rank. > > CPython's byte code is a stack-based processor. So "load X" means > "push X onto the stack", and "store X" means "pop the top of the stack > and store it into X". For simple names, those are perfect > counterparts. But for dotted or subscripted lookups, it's like this: > >>>> dis.dis("x[1]=y[2]") > 1 0 LOAD_NAME 0 (y) > 2 LOAD_CONST 0 (2) > 4 BINARY_SUBSCR > 6 LOAD_NAME 1 (x) > 8 LOAD_CONST 1 (1) > 10 STORE_SUBSCR > 12 LOAD_CONST 2 (None) > 14 RETURN_VALUE > > (Ignore the last two lines - they're an implicit "return None" at the > end of the "function" that I just compiled.) > > Loading y[2] is done in several steps. First, we get the value of y > (which could be an arbitrary expression; in this case, it's just a > simple name). Then, we take the constant that we're subscripting with > (the integer 2). Finally, we do a BINARY_SUBSCR, which is the > square-bracket lookup operator. That leaves us with something on the > stack. The load took one argument (the name "y"), and then the > subscripting was a separate operation with two arguments (y and 2). > > Storing into x[1], on the other hand, is collapsed down a bit. We > first load up the value of x (again, that could be any expression), > and then the subscript (the integer 1). Then the subscripting and > assignment are done in one STORE_SUBSCR operation; it takes three > arguments off the stack (the new value, the object, and the > subscript), and leaves nothing behind. OK, so you're lumping the byte-code arguments (the fixed ones following the byte-code) together with arguments taken off the stack. That's why Store was said to have an extra argument. -- bartc From daniel.p.flick at gmail.com Tue Oct 17 17:17:56 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 14:17:56 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: On Tuesday, October 17, 2017 at 12:37:03 PM UTC-5, Rob Gaddi wrote: > On 10/17/2017 09:59 AM, Daniel Flick wrote: > > I am very new to Python and have been struggling to find some info on processing IP addresses. > > > > get_network returns 192.168.1.128/25 but I need 192.168.1.128 only. I can do this with netaddr but I am working with Mako templates and ipaddress is a built in module so there are less dependencies. > > > > Any ideas? > > > > You mean, other than .split('/')? > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. Rob, maybe that was what I was looking for. New to Python = 4 days new! LOL. I have heard of split before so let me read up on it. From daniel.p.flick at gmail.com Tue Oct 17 17:20:41 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 14:20:41 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: <59a01efc-1797-4264-a12c-e533a4ab727d@googlegroups.com> I was hoping for something more like netaddr.network. Trying to save lots of code as this will be inside a Mako template and I don't want it to get messy. Sounds like the split function will allow me to chop off the /xx from the end of the string which gives me what I need. Off to read documentation on Split..... Thanks! From daniel.p.flick at gmail.com Tue Oct 17 17:24:50 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 14:24:50 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: Peter, I am not following. Are you saying that there is a function that returns the network only? network_address was giving me the mask attached to the end but maybe I was doing something wrong. For an input of LAN_IP=192.168.99.1/24 ipaddress.IPv4Interface(LAN_IP).ip returns 192.168.99.0/24 I need the 192.168.99.0 part only. From daniel.p.flick at gmail.com Tue Oct 17 17:26:23 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 14:26:23 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: On Tuesday, October 17, 2017 at 4:25:02 PM UTC-5, Daniel Flick wrote: > > Peter, I am not following. Are you saying that there is a function that returns the network only? network_address was giving me the mask attached to the end but maybe I was doing something wrong. > > For an input of LAN_IP=192.168.99.1/24 > ipaddress.IPv4Interface(LAN_IP).ip > returns 192.168.99.0/24 > > I need the 192.168.99.0 part only. OOPS! I meant For an input of LAN_IP=192.168.99.1/24 ipaddress.IPv4Interface(LAN_IP).network returns 192.168.99.0/24 From nhilterbrand at gmail.com Tue Oct 17 17:28:57 2017 From: nhilterbrand at gmail.com (Nathan Hilterbrand) Date: Tue, 17 Oct 2017 17:28:57 -0400 Subject: Fwd: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: Absolutely, Stefan! I like yours a lot better. I am an old perl hack that is still learning the ins and outs of Python, and this is just the sort of thing that I like to see. Thanks! On Tue, Oct 17, 2017 at 4:19 PM, Stefan Ram wrote: > Nathan Hilterbrand writes: > >I may have misunderstood what you were looking for, but went ahead and > >cobbled this ugly function together. Takes a string like " > 192.168.1.128/25" > >as input, and returns the subnet address. Just slapped together, so not > >the most robust thing in the world > > The IP address of the OP has some so-called "host bits" set. > Therefore, it might not be considered to be a valid subnet > specification by some parties. > > Your code seems to have the intention to remove those > disturbing host bits. You did this using a bit mask. > It might also be possible to accomplish the same by > a right shift plus a left shift, e.g., > > |>>> IPv4Address( int( IPv4Address( '192.168.1.129' ))>> 25 << 25 ) > |IPv4Address('192.0.0.0') > > . > > -- > https://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Tue Oct 17 18:00:26 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 18 Oct 2017 00:00:26 +0200 Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: Daniel Flick wrote: > On Tuesday, October 17, 2017 at 4:25:02 PM UTC-5, Daniel Flick wrote: >> >> Peter, I am not following. Are you saying that there is a function that >> returns the network only? network_address was giving me the mask >> attached to the end but maybe I was doing something wrong. >> >> For an input of LAN_IP=192.168.99.1/24 >> ipaddress.IPv4Interface(LAN_IP).ip >> returns 192.168.99.0/24 >> >> I need the 192.168.99.0 part only. > > OOPS! I meant > For an input of LAN_IP=192.168.99.1/24 > ipaddress.IPv4Interface(LAN_IP).network > returns 192.168.99.0/24 In the body of your post you had 192.168.1.128/25, so I mistook 192.168.1.129/25 as a typo. However, once you have >>> import ipaddress >>> ipaddress.ip_interface("192.168.99.1/24").network IPv4Network('192.168.99.0/24') you can simply add the step from my first answer: >>> ipaddress.ip_interface("192.168.99.1/24").network.network_address IPv4Address('192.168.99.0') Unless I'm misunderstanding again... From ihamidx0 at gmail.com Tue Oct 17 18:15:18 2017 From: ihamidx0 at gmail.com (ihamidx0 at gmail.com) Date: Tue, 17 Oct 2017 15:15:18 -0700 (PDT) Subject: Test Bank for Campbell Biology 11th Edition by Urry, Cain In-Reply-To: <3b77e8e5-f5e4-4fc6-abc9-73e42a89f429@googlegroups.com> References: <3b77e8e5-f5e4-4fc6-abc9-73e42a89f429@googlegroups.com> Message-ID: On Wednesday, September 27, 2017 at 5:52:23 PM UTC-4, Test Banks wrote: > Greetings, > > You can get Test Bank for " Campbell Biology, 11 Edition by Lisa A. Urry,Michael L. Cain,Steven A. Wasserman,Peter V. Minorsky,Jane B. Reece " at very reasonable price. Our team is available 24/7 and 365 days / year to respond your requests. Send us an email at pro.fast(@)hotmail(dot)com > > Place your order at PRO.FAST(@)HOTMAIL(DOT)COM > > ISBN Numbers for this book (ISBN-10: 0134093410 and ISBN-13: 9780134093413) > > CAMPBELL BIOLOGY 11TH E 11E BY URRY CAIN TEST BANK > > You can also email for other Biology books Solutions and Test Bank at low prices and our team will try to get all resources you need. > > Do not post your reply here. Simply send us an email at PRO.FAST (AT) HOTMAIL (DOT) COM > > Cheers From daniel.p.flick at gmail.com Tue Oct 17 21:24:39 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 18:24:39 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> Message-ID: <99e8e6f0-2c94-4608-89c2-fbd05775ed20@googlegroups.com> On Tuesday, October 17, 2017 at 5:01:13 PM UTC-5, Peter Otten wrote: > Daniel Flick wrote: > > > On Tuesday, October 17, 2017 at 4:25:02 PM UTC-5, Daniel Flick wrote: > >> > >> Peter, I am not following. Are you saying that there is a function that > >> returns the network only? network_address was giving me the mask > >> attached to the end but maybe I was doing something wrong. > >> > >> For an input of LAN_IP=192.168.99.1/24 > >> ipaddress.IPv4Interface(LAN_IP).ip > >> returns 192.168.99.0/24 > >> > >> I need the 192.168.99.0 part only. > > > > OOPS! I meant > > For an input of LAN_IP=192.168.99.1/24 > > ipaddress.IPv4Interface(LAN_IP).network > > returns 192.168.99.0/24 > > In the body of your post you had 192.168.1.128/25, so I mistook > 192.168.1.129/25 as a typo. However, once you have > > >>> import ipaddress > >>> ipaddress.ip_interface("192.168.99.1/24").network > IPv4Network('192.168.99.0/24') > > you can simply add the step from my first answer: > > >>> ipaddress.ip_interface("192.168.99.1/24").network.network_address > IPv4Address('192.168.99.0') > > Unless I'm misunderstanding again... That seems to be spot on. I will try that. I used the /25 to illustrate that that the network may not end in zero. I had asked this question in the Mako forum and I had a few answers to drop the last octet and add zero which was not correct. Thanks for your help! From daniel.p.flick at gmail.com Tue Oct 17 21:26:52 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 18:26:52 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: <99e8e6f0-2c94-4608-89c2-fbd05775ed20@googlegroups.com> References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> <99e8e6f0-2c94-4608-89c2-fbd05775ed20@googlegroups.com> Message-ID: <2169fd67-876f-4391-8302-62ef08ef0ce0@googlegroups.com> On Tuesday, October 17, 2017 at 8:24:52 PM UTC-5, Daniel Flick wrote: > On Tuesday, October 17, 2017 at 5:01:13 PM UTC-5, Peter Otten wrote: > > Daniel Flick wrote: > > > > > On Tuesday, October 17, 2017 at 4:25:02 PM UTC-5, Daniel Flick wrote: > > >> > > >> Peter, I am not following. Are you saying that there is a function that > > >> returns the network only? network_address was giving me the mask > > >> attached to the end but maybe I was doing something wrong. > > >> > > >> For an input of LAN_IP=192.168.99.1/24 > > >> ipaddress.IPv4Interface(LAN_IP).ip > > >> returns 192.168.99.0/24 > > >> > > >> I need the 192.168.99.0 part only. > > > > > > OOPS! I meant > > > For an input of LAN_IP=192.168.99.1/24 > > > ipaddress.IPv4Interface(LAN_IP).network > > > returns 192.168.99.0/24 > > > > In the body of your post you had 192.168.1.128/25, so I mistook > > 192.168.1.129/25 as a typo. However, once you have > > > > >>> import ipaddress > > >>> ipaddress.ip_interface("192.168.99.1/24").network > > IPv4Network('192.168.99.0/24') > > > > you can simply add the step from my first answer: > > > > >>> ipaddress.ip_interface("192.168.99.1/24").network.network_address > > IPv4Address('192.168.99.0') > > > > Unless I'm misunderstanding again... > > That seems to be spot on. I will try that. I used the /25 to illustrate that that the network may not end in zero. I had asked this question in the Mako forum and I had a few answers to drop the last octet and add zero which was not correct. Thanks for your help! SWEET! That worked. Thanks Peter! From daniel.p.flick at gmail.com Tue Oct 17 21:28:43 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Tue, 17 Oct 2017 18:28:43 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: <2169fd67-876f-4391-8302-62ef08ef0ce0@googlegroups.com> References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> <99e8e6f0-2c94-4608-89c2-fbd05775ed20@googlegroups.com> <2169fd67-876f-4391-8302-62ef08ef0ce0@googlegroups.com> Message-ID: On Tuesday, October 17, 2017 at 8:27:03 PM UTC-5, Daniel Flick wrote: > On Tuesday, October 17, 2017 at 8:24:52 PM UTC-5, Daniel Flick wrote: > > On Tuesday, October 17, 2017 at 5:01:13 PM UTC-5, Peter Otten wrote: > > > Daniel Flick wrote: > > > > > > > On Tuesday, October 17, 2017 at 4:25:02 PM UTC-5, Daniel Flick wrote: > > > >> > > > >> Peter, I am not following. Are you saying that there is a function that > > > >> returns the network only? network_address was giving me the mask > > > >> attached to the end but maybe I was doing something wrong. > > > >> > > > >> For an input of LAN_IP=192.168.99.1/24 > > > >> ipaddress.IPv4Interface(LAN_IP).ip > > > >> returns 192.168.99.0/24 > > > >> > > > >> I need the 192.168.99.0 part only. > > > > > > > > OOPS! I meant > > > > For an input of LAN_IP=192.168.99.1/24 > > > > ipaddress.IPv4Interface(LAN_IP).network > > > > returns 192.168.99.0/24 > > > > > > In the body of your post you had 192.168.1.128/25, so I mistook > > > 192.168.1.129/25 as a typo. However, once you have > > > > > > >>> import ipaddress > > > >>> ipaddress.ip_interface("192.168.99.1/24").network > > > IPv4Network('192.168.99.0/24') > > > > > > you can simply add the step from my first answer: > > > > > > >>> ipaddress.ip_interface("192.168.99.1/24").network.network_address > > > IPv4Address('192.168.99.0') > > > > > > Unless I'm misunderstanding again... > > > > That seems to be spot on. I will try that. I used the /25 to illustrate that that the network may not end in zero. I had asked this question in the Mako forum and I had a few answers to drop the last octet and add zero which was not correct. Thanks for your help! > > SWEET! That worked. Thanks Peter! In case anyone needs this as a reference: >>> import ipaddress >>> ipaddress.ip_interface("192.168.99.129/25").network IPv4Network('192.168.99.128/25') >>> ipaddress.ip_interface("192.168.99.129/25").network.network_address IPv4Address('192.168.99.128') >>> ipaddress.ip_interface("192.168.99.154/30").network.network_address IPv4Address('192.168.99.152') From christopher_reimer at icloud.com Wed Oct 18 02:09:24 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Tue, 17 Oct 2017 23:09:24 -0700 Subject: Can't find latest version of 3.4.x on download page Message-ID: Greetings, I'm setting up different test environments for tox. I can't find Windows installer for the latest version of Python 3.4 on the download page. Versions 3.4.5 to 3.4.7 only have the source files available. Version 3.4.4 is the last version with Windows installers. Testing on Python 3.4 might be moot for my program. It choked on a starred expression in a unit test. The starred expression got changed in Python 3.5 and my unit tests pass in Python 3.5/3.6. I don't think I can live without that feature. Thank you, Chris R. From llanitedave at birdandflower.com Wed Oct 18 02:31:58 2017 From: llanitedave at birdandflower.com (llanitedave) Date: Tue, 17 Oct 2017 23:31:58 -0700 (PDT) Subject: Logging module stopped working In-Reply-To: <59e5a9ee$0$14943$b1db1813$d948b532@news.astraweb.com> References: <4cf893cb-6824-4dc6-80d4-7f8712a57e8c@googlegroups.com> <59e5a9ee$0$14943$b1db1813$d948b532@news.astraweb.com> Message-ID: <5beff9c5-bfb3-4ea7-ae63-eabaa5d56cf9@googlegroups.com> On Monday, October 16, 2017 at 11:58:12 PM UTC-7, Steve D'Aprano wrote: > On Tue, 17 Oct 2017 03:06 pm, llanitedave wrote: > > [...] > > I set up the logging code at the very beginning of the app, before any other > > work is done. Here's the relevant code: > > > > #!/usr/bin/env python3 > [... snip imports ...] > > class MainWindow(QMainWindow): > > def __init__(self): > > super().__init__() > > # set up logging > > logging.basicConfig(format='%(levelname)s:%(message)s', > > filename="sample.log", level=logging.DEBUG) > > logging.info("Starting system, MainWindow.__init__, %s", > > str(datetime.datetime.today())) > > self.createUI() > > According to this code, no logging will occur because no MainWindow is > created. Nor is there any sort of main event loop. > > I'm sure that you're not actually so silly that you forgot to create a window > at all, but the point is, this example is *not* "the relevant code". It is > only *part* of the relevant code. Who knows what else is happening that might > be preventing logging from working? We don't know because we can't see the > rest of the relevant code. Perhaps you need to focus on that. > > > [...] > > Between the time that the logging was working and the time it quit, the only > > changes I made were to add a couple of logging.info() statements into a > > downstream module. But that seems irrelevant here, as those modules aren't > > included in the above test. > > I doubt that. I expect there must be some other change you have forgotten, and > it is *that* which has disabled logging. Maybe you call something which sets > the logging level above INFO? > > I would start with this: > > import sys > import os > import logging > logging.basicConfig(format='%(levelname)s:%(message)s', > filename="sample.log", level=logging.DEBUG) > logging.info("Starting module, %s", str(datetime.datetime.today())) > > # import everything else > ... > logging.info("Importing complete %s", str(datetime.datetime.today())) > > class MainWindow(QMainWindow): > def __init__(self): > super().__init__() > logging.info("Creating window, %s", str(datetime.datetime.today())) > self.createUI() > > logging.info("Class created %s", str(datetime.datetime.today())) > window = MainWindow() > logging.info("Window created %s", str(datetime.datetime.today())) > logging.critical("Can you see this? %s", str(datetime.datetime.today())) > > # plus whatever else is needed to make the application run > ... > > That will show precisely where and when logging stops: > > 1. Does it work at all, straight out of the logging module? If not, then > something broke it before your module even gets loaded. > > 2. Does it still work after all the other imports? If not, then bisect the > imports until you locate which module breaks logging. > > 3. Do you get the "Class created" and "Window created" messages? If no, then > that helps narrow down where the fault may lie. > > E.g. if you see the first, but not the second, then something in > super().__init__ may be disabling logging; if you don't see the first, then > look at QMainWindow's metaclass, if it has one. If it doesn't have a > metaclass, then that's a mystery -- maybe something in some other thread is > messing you about? > > 4. If you see the CRITICAL message, but not the INFO ones, then something has > reset the logging level. > > 5. What happens if you delete sample.log? Does it get re-created? If not, > maybe there's a permissions error. > > 6. If you're not seeing *any* logging at all, perhaps you need to fall back on > calling print() directly. Run your application from a terminal, and see what > it prints. > > If even print isn't working, then follow the three Rs: Reboot, Reinstall, and > Resign :-) > > 7. Perhaps a bit less drastic, you can insert a debugging checkpoint in the > code, run the application from a terminal, and when it halts at the debugger, > have a look at the state of the logging module. > > 8. Just to be absolutely sure, check logging.__file__ to ensure you haven't > shadowed the real logging module with some sort of mock. But now I'm really > clutching at straws, because if that were the case, I'd expect there to be an > exception, not just logging failing to work. > > > > I'm intrigued by this error, and would love to hear what caused it when you > find out. Please respond back on the list with your diagnosis. > > By the way, this sort of spooky action-at-a-distance is why I prefer to create > my own loggers, rather than use the global one. > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. Well, I started with your first suggestion, and sure enough, I found the problem right away. What I was calling"downstream modules" really aren't so downstream after all, since they apparently get parsed on some level as soon as they are loaded. I had placed a call to "logging.info()" in the evocontrol module that was imported prior to MainWindow(), and it was after a class definition but before the __init__() function. I learned that if a call comes within the function it's insulated, but outside the function it interferes with the logging module unless the configuration occurs prior to its import. So, live and learn, that's a mistake that won't be repeated. Thanks for your quick and on the money response. There's a reason I lurk here! From tjreedy at udel.edu Wed Oct 18 03:53:38 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 18 Oct 2017 03:53:38 -0400 Subject: Can't find latest version of 3.4.x on download page In-Reply-To: References: Message-ID: On 10/18/2017 2:09 AM, Christopher Reimer wrote: > Greetings, > > I'm setting up different test environments for tox. I can't find Windows installer for the latest version of Python 3.4 on the download page. > > Versions 3.4.5 to 3.4.7 only have the source files available. Correct. These are security releases, mainly for servers and people who would compile themselves. > Version 3.4.4 is the last version with Windows installers. Correct. -- Terry Jan Reedy From ben+python at benfinney.id.au Wed Oct 18 04:11:38 2017 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 18 Oct 2017 19:11:38 +1100 Subject: Can't find latest version of 3.4.x on download page References: Message-ID: <854lqw3mf9.fsf@benfinney.id.au> Christopher Reimer writes: > I'm setting up different test environments for tox. I can't find > Windows installer for the latest version of Python 3.4 on the download > page. Python 3.4 is provided in source form only; the Python core developers no longer generate installers for it. Anyone who wants to run Python 3 older than 3.5, needs to find a way themselves to build and install it; that's the trade-off for wanting an old version :-) > Testing on Python 3.4 might be moot for my program. It choked on a > starred expression in a unit test. The starred expression got changed > in Python 3.5 and my unit tests pass in Python 3.5/3.6. I don't think > I can live without that feature. That sounds reasonable. If you can declare Python 3.4 is not supported by your app, your support job becomes easier. -- \ ?Without cultural sanction, most or all of our religious | `\ beliefs and rituals would fall into the domain of mental | _o__) disturbance.? ?John F. Schumaker | Ben Finney From robin at reportlab.com Wed Oct 18 04:28:34 2017 From: robin at reportlab.com (Robin Becker) Date: Wed, 18 Oct 2017 09:28:34 +0100 Subject: why del is not a function or method? In-Reply-To: References: Message-ID: On 16/10/2017 16:37, Xue Feng via Python-list wrote: > Hi, > ?? I wonder why 'del' is not a function or method. Most operations can be used > as follows > ??????? len(team) > or > ??????? team.append("tom") > But, I think > ??????? del team[2] > is somewhat uncommon. Why does not it take a syntax we are famillar with? It can look like a function >>>> x = 3 >>>> del(x) >>>> x > Traceback (most recent call last): > File "", line 1, in > NameError: name 'x' is not defined >>>> -- Robin Becker From arj.python at gmail.com Wed Oct 18 05:46:45 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 18 Oct 2017 13:46:45 +0400 Subject: PEP 249 Compliant error handling In-Reply-To: References: Message-ID: all corruption systematically ignored but data piece logged in for analysis Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 17 Oct 2017 21:43, "Israel Brewster" wrote: > I have written and maintain a PEP 249 compliant (hopefully) DB API for the > 4D database, and I've run into a situation where corrupted string data from > the database can cause the module to error out. Specifically, when decoding > the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode > bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, > given that the string data got corrupted somehow, but the question is "what > is the proper way to deal with this in the module?" Should I just throw an > error on bad data? Or would it be better to set the errors parameter to > something like "replace"? The former feels a bit more "proper" to me > (there's an error here, so we throw an error), but leaves the end user dead > in the water, with no way to retrieve *any* of the data (from that row at > least, and perhaps any rows after it as well). The latter option sort of > feels like sweeping the problem under the rug, but does at least leave an > error character in the string to l > et them know there was an error, and will allow retrieval of any good > data. > > Of course, if this was in my own code I could decide on a case-by-case > basis what the proper action is, but since this a module that has to work > in any situation, it's a bit more complicated. > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From tjol at tjol.eu Wed Oct 18 07:02:12 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 18 Oct 2017 13:02:12 +0200 Subject: how to read in the newsreader In-Reply-To: References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> <0b465e5b-86ee-30c6-c693-7cc179c449d9@tjol.eu> Message-ID: On 2017-10-17 20:38, Pete Forman wrote: > Thomas Jollans writes: > >> On 16/10/17 20:02, Pete Forman wrote: >>> Thomas Jollans writes: >>> >>>> On 2017-10-16 08:48, Pete Forman wrote: >>>>> Andrew Z writes: >>>>> >>>>>> hmm. i did do that. maybe just a delay. >>>>>> I'll see how it will go tomorrow then. Thank you gents. >>>>>> >>>>>> On Mon, Oct 16, 2017 at 12:30 AM, Chris Angelico wrote: >>>>>> >>>>>>> On Mon, Oct 16, 2017 at 3:19 PM, Andrew Z wrote: >>>>>>>> Michael, that's what i use too - gmail. But i get the digest only >>>>>>>> and can't really reply that way. i was hoping to get the >>>>>>>> mail.python.org list.... >>>>>>> >>>>>>> Turn off digests then. Easy! >>>>> >>>>> If you do stick with a digest then check your newsreader for a feature >>>>> to expand it. Then you can read and reply as if you were getting >>>>> individual posts. >>>>> >>>> That exists? How does it work? >>> >>> The Gnus newsreader in Emacs allows you to type C-d on a digest to run >>> gnus-summary-enter-digest-group. That then behaves the same as if you >>> opened any other summary such as a regular Usenet group. >>> >> >> Does it set the References header correctly when replying? > > Sorry, I am not in a position to test. The only digest I subscribe to is > comp.risks. The only messsage id in that is a single one for the whole > digest. Each article only has date, subject and from headers. You would > need to look inside a Python digest to see if it carries more headers > for the articles. If they are not present then they cannot be used when > composing a reply. > I see. I've never subscribed to anything as a digest so I don't really know what they look like, but judging from https://mail.python.org/pipermail/python-list/2017-October/727421.html (section of the digest quoted at the bottom) it *could* work. -- Thomas Jollans From Karsten.Hilbert at gmx.net Wed Oct 18 07:43:58 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Wed, 18 Oct 2017 13:43:58 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> Message-ID: <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> OK, here's the first bit which might be part of the puzzle of why my Python script SIGABRT's. When run under "normal" py2.7 it runs all the way through but upon shutdown (*after* sys.exit(0)) faulthandler shows a problem (and returns 134 which made me think of SIGABRT): *** Error in `python': free(): invalid pointer: 0x00770b14 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x6738a)[0xb7ccc38a] /lib/i386-linux-gnu/libc.so.6(+0x6dfc7)[0xb7cd2fc7] /lib/i386-linux-gnu/libc.so.6(+0x6e806)[0xb7cd3806] python(PyObject_Free+0xfb)[0x44affb] python(+0x110b51)[0x534b51] python(+0x110b51)[0x534b51] python(+0x110b51)[0x534b51] python(+0xf3ea7)[0x517ea7] python(PyDict_SetItem+0x201)[0x4d4f81] python(_PyModule_Clear+0x150)[0x539630] python(PyImport_Cleanup+0x61d)[0x53927d] python(Py_Finalize+0x9d)[0x53635d] python(Py_Exit+0x14)[0x55cb24] python(+0x136102)[0x55a102] python(PyErr_PrintEx+0x35)[0x559975] python(+0x3dd41)[0x461d41] python(Py_Main+0x753)[0x4d2c83] python(main+0x1b)[0x4d251b] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)[0xb7c7d286] python(+0xae3f3)[0x4d23f3] ======= Memory map: ======== 00424000-00763000 r-xp 00000000 08:01 6285092 /usr/bin/python2.7 00763000-00764000 r--p 0033e000 08:01 6285092 /usr/bin/python2.7 00764000-007c4000 rw-p 0033f000 08:01 6285092 /usr/bin/python2.7 007c4000-007d9000 rw-p 00000000 00:00 0 026f1000-02921000 rw-p 00000000 00:00 0 [heap] b6800000-b6821000 rw-p 00000000 00:00 0 b6821000-b6900000 ---p 00000000 00:00 0 b6995000-b69b0000 r-xp 00000000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 b69b0000-b69b1000 r--p 0001a000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 b69b1000-b69b2000 rw-p 0001b000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 b69f5000-b6b35000 rw-p 00000000 00:00 0 b6b35000-b6b40000 r-xp 00000000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so b6b40000-b6b41000 r--p 0000a000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so b6b41000-b6b42000 rw-p 0000b000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so b6b42000-b6b48000 rw-p 00000000 00:00 0 b6b48000-b6b53000 r-xp 00000000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so b6b53000-b6b54000 r--p 0000a000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so b6b54000-b6b55000 rw-p 0000b000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so b6b64000-b6b6b000 r--p 00000000 08:01 6286752 /usr/share/locale/de/LC_MESSAGES/libpq5-10.mo b6b6b000-b6b72000 r--s 00000000 08:01 6903281 /usr/lib/i386-linux-gnu/gconv/gconv-modules.cache b6b72000-b6b98000 r--p 00000000 08:01 6288754 /usr/share/locale/de/LC_MESSAGES/libc.mo b6b98000-b6c98000 rw-p 00000000 00:00 0 b6c9e000-b6cb4000 r-xp 00000000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so b6cb4000-b6cb5000 r--p 00016000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so b6cb5000-b6cb6000 rw-p 00017000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so b6cb6000-b6cb8000 rw-p 00000000 00:00 0 b6cb8000-b6cc0000 r-xp 00000000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so b6cc0000-b6cc1000 r--p 00007000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so b6cc1000-b6cc2000 rw-p 00008000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so b6cc2000-b6d02000 rw-p 00000000 00:00 0 b6d02000-b6d09000 r-xp 00000000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 b6d09000-b6d0a000 r--p 00006000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 b6d0a000-b6d0b000 rw-p 00007000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 b6d0b000-b6d96000 r-xp 00000000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 b6d96000-b6d97000 r--p 0008a000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 b6d97000-b6d98000 rw-p 0008b000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 b6d98000-b6dcc000 r-xp 00000000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 b6dcc000-b6dcd000 r--p 00033000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 b6dcd000-b6dce000 rw-p 00034000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 b6dce000-b6e08000 r-xp 00000000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 b6e08000-b6e09000 ---p 0003a000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 b6e09000-b6e0a000 r--p 0003a000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 b6e0a000-b6e0b000 rw-p 0003b000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 b6e0b000-b6e1e000 r-xp 00000000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 b6e1e000-b6e1f000 r--p 00012000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 b6e1f000-b6e20000 rw-p 00013000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 b6e20000-b6f8c000 r-xp 00000000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 b6f8c000-b6f8d000 ---p 0016c000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 b6f8d000-b6f8f000 r--p 0016c000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 b6f8f000-b6f90000 rw-p 0016e000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 b6f90000-b6fac000 r-xp 00000000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 b6fac000-b6fad000 r--p 0001b000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 b6fad000-b6fae000 rw-p 0001c000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 b6fae000-b70fe000 r-xp 00000000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 b70fe000-b7104000 r--p 0014f000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 b7104000-b7109000 rw-p 00155000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 b7109000-b710a000 rw-p 00000000 00:00 0 b710a000-b7299000 r-xp 00000000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 b7299000-b72a1000 r--p 0018e000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 b72a1000-b72a2000 rw-p 00196000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 b72a2000-b72a3000 rw-p 00000000 00:00 0 b72a3000-b72bf000 r-xp 00000000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 b72bf000-b72c0000 r--p 0001b000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 b72c0000-b72c1000 rw-p 0001c000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 b72c1000-b72d5000 r-xp 00000000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so b72d5000-b72d6000 r--p 00013000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so b72d6000-b72d7000 rw-p 00014000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so b72d7000-b72d9000 rw-p 00000000 00:00 0 b72d9000-b730b000 r-xp 00000000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 b730b000-b730c000 ---p 00032000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 b730c000-b730d000 r--p 00032000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 b730d000-b730e000 rw-p 00033000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 b730e000-b730f000 rw-p 00000000 00:00 0 b730f000-b73e2000 r-xp 00000000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 b73e2000-b73e3000 ---p 000d3000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 b73e3000-b73e9000 r--p 000d3000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 b73e9000-b73eb000 rw-p 000d9000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 b73eb000-b7441000 r-xp 00000000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 b7441000-b7442000 ---p 00056000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 b7442000-b7443000 r--p 00056000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 b7443000-b7444000 rw-p 00057000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 b7444000-b7445000 rw-p 00000000 00:00 0 b7445000-b7495000 r-xp 00000000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 b7495000-b7496000 r--p 0004f000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 b7496000-b7497000 rw-p 00050000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 b7497000-b74e1000 r-xp 00000000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 b74e1000-b74e3000 r--p 00049000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 b74e3000-b74e4000 rw-p 0004b000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 b74e8000-b74ed000 r-xp 00000000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so b74ed000-b74ee000 r--p 00004000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so b74ee000-b74ef000 rw-p 00005000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so b74ef000-b74f6000 r-xp 00000000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so b74f6000-b74f7000 r--p 00006000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so b74f7000-b74f9000 rw-p 00007000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so b74f9000-b750c000 r-xp 00000000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so b750c000-b750d000 r--p 00012000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so b750d000-b750e000 rw-p 00013000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so b750e000-b7523000 r-xp 00000000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so b7523000-b7524000 r--p 00014000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so b7524000-b7527000 rw-p 00015000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so b7527000-b7559000 r-xp 00000000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so b7559000-b755a000 r--p 00031000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so b755a000-b755e000 rw-p 00032000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so b755e000-b756e000 r-xp 00000000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 b756e000-b756f000 r--p 0000f000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 b756f000-b7570000 rw-p 00010000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 b7576000-b7584000 r-xp 00000000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 b7584000-b7585000 r--p 0000d000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 b7585000-b7586000 rw-p 0000e000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 b7586000-b7589000 r-xp 00000000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 b7589000-b758a000 r--p 00002000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 b758a000-b758b000 rw-p 00003000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 b758b000-b7596000 r-xp 00000000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 b7596000-b7597000 r--p 0000a000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 b7597000-b7598000 rw-p 0000b000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 b7598000-b759f000 r-xp 00000000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so b759f000-b75a0000 r--p 00006000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so b75a0000-b75a1000 rw-p 00007000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so b75a1000-b75b1000 r-xp 00000000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so b75b1000-b75b2000 r--p 0000f000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so b75b2000-b75b3000 rw-p 00010000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so b75b3000-b77fe000 r-xp 00000000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 b77fe000-b77ff000 ---p 0024b000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 b77ff000-b7810000 r--p 0024b000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 b7810000-b7817000 rw-p 0025c000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 b7817000-b781a000 rw-p 00000000 00:00 0 b781a000-b7881000 r-xp 00000000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 b7881000-b7884000 r--p 00066000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 b7884000-b7888000 rw-p 00069000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 b7888000-b7988000 rw-p 00000000 00:00 0 b798a000-b798d000 r-xp 00000000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 b798d000-b798e000 r--p 00002000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 b798e000-b798f000 rw-p 00003000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 b798f000-b7998000 r-xp 00000000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so b7998000-b7999000 r--p 00008000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so b7999000-b799b000 rw-p 00009000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so b799b000-b79fc000 rw-p 00000000 00:00 0 b79fc000-b7a00000 r-xp 00000000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so b7a00000-b7a01000 r--p 00003000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so b7a01000-b7a02000 rw-p 00004000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so b7a02000-b7a05000 r-xp 00000000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so b7a05000-b7a06000 r--p 00002000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so b7a06000-b7a08000 rw-p 00003000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so b7a08000-b7a48000 rw-p 00000000 00:00 0 b7a48000-b7be3000 r--p 00000000 08:01 6299936 /usr/lib/locale/locale-archive b7be3000-b7c65000 rw-p 00000000 00:00 0 b7c65000-b7e16000 r-xp 00000000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so b7e16000-b7e18000 r--p 001b0000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so b7e18000-b7e19000 rw-p 001b2000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so b7e19000-b7e1c000 rw-p 00000000 00:00 0 b7e1c000-b7e6f000 r-xp 00000000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so b7e6f000-b7e70000 r--p 00052000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so b7e70000-b7e71000 rw-p 00053000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so b7e71000-b7e8a000 r-xp 00000000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 b7e8a000-b7e8b000 r--p 00018000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 b7e8b000-b7e8c000 rw-p 00019000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 b7e8c000-b7e8e000 r-xp 00000000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so b7e8e000-b7e8f000 r--p 00001000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so b7e8f000-b7e90000 rw-p 00002000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so b7e90000-b7e93000 r-xp 00000000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so b7e93000-b7e94000 r--p 00002000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so b7e94000-b7e95000 rw-p 00003000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so b7e95000-b7eae000 r-xp 00000000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so b7eae000-b7eaf000 r--p 00018000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so b7eaf000-b7eb0000 rw-p 00019000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so b7eb0000-b7eb2000 rw-p 00000000 00:00 0 b7eb4000-b7ef8000 rw-p 00000000 00:00 0 b7ef8000-b7efb000 r--p 00000000 00:00 0 [vvar] b7efb000-b7efd000 r-xp 00000000 00:00 0 [vdso] b7efd000-b7f20000 r-xp 00000000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so b7f20000-b7f21000 r--p 00022000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so b7f21000-b7f22000 rw-p 00023000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so bf9f5000-bfa16000 rw-p 00000000 00:00 0 [stack] Fatal Python error: Aborted Current thread 0xb7c63d00 (most recent call first): When run under a debug build it sayeth right away (simulated as a minimal working example): root at hermes:~/bin# python2.7-dbg Python 2.7.14 (default, Sep 17 2017, 18:50:44) [GCC 7.2.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in from DateTime import * File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in from mxDateTime import * File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in raise ImportError, why ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 [47287 refs] >>> exit() [21664 refs] root at hermes:~/bin# apt-cache policy python-egenix-mxdatetime python-egenix-mxdatetime: Installiert: 3.2.9-1 Installationskandidat: 3.2.9-1 Versionstabelle: *** 3.2.9-1 990 500 http://httpredir.debian.org/debian stretch/main i386 Packages 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status root at hermes:~/bin# Does this ring a bell or does anyone know what I should further look into ? For good measure I have filed a bug with Debian asking for recompilation of python-egenix-mxdatetime. Next, I'll remove mxDateTime from the script and see what prevails. Thanks, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From tjol at tjol.eu Wed Oct 18 08:07:46 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 18 Oct 2017 14:07:46 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> Message-ID: <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> On 2017-10-18 13:43, Karsten Hilbert wrote: > OK, here's the first bit which might be part of the puzzle of > why my Python script SIGABRT's. > > When run under "normal" py2.7 it runs all the way through but > upon shutdown (*after* sys.exit(0)) faulthandler shows a > problem (and returns 134 which made me think of SIGABRT): > > *** Error in `python': free(): invalid pointer: 0x00770b14 *** > ======= Backtrace: ========= > /lib/i386-linux-gnu/libc.so.6(+0x6738a)[0xb7ccc38a] > /lib/i386-linux-gnu/libc.so.6(+0x6dfc7)[0xb7cd2fc7] > /lib/i386-linux-gnu/libc.so.6(+0x6e806)[0xb7cd3806] > python(PyObject_Free+0xfb)[0x44affb] > python(+0x110b51)[0x534b51] > python(+0x110b51)[0x534b51] > python(+0x110b51)[0x534b51] > python(+0xf3ea7)[0x517ea7] > python(PyDict_SetItem+0x201)[0x4d4f81] > python(_PyModule_Clear+0x150)[0x539630] > python(PyImport_Cleanup+0x61d)[0x53927d] > python(Py_Finalize+0x9d)[0x53635d] > python(Py_Exit+0x14)[0x55cb24] > python(+0x136102)[0x55a102] > python(PyErr_PrintEx+0x35)[0x559975] > python(+0x3dd41)[0x461d41] > python(Py_Main+0x753)[0x4d2c83] > python(main+0x1b)[0x4d251b] > /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6)[0xb7c7d286] > python(+0xae3f3)[0x4d23f3] > ======= Memory map: ======== > 00424000-00763000 r-xp 00000000 08:01 6285092 /usr/bin/python2.7 > 00763000-00764000 r--p 0033e000 08:01 6285092 /usr/bin/python2.7 > 00764000-007c4000 rw-p 0033f000 08:01 6285092 /usr/bin/python2.7 > 007c4000-007d9000 rw-p 00000000 00:00 0 > 026f1000-02921000 rw-p 00000000 00:00 0 [heap] > b6800000-b6821000 rw-p 00000000 00:00 0 > b6821000-b6900000 ---p 00000000 00:00 0 > b6995000-b69b0000 r-xp 00000000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 > b69b0000-b69b1000 r--p 0001a000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 > b69b1000-b69b2000 rw-p 0001b000 08:01 8151085 /lib/i386-linux-gnu/libgcc_s.so.1 > b69f5000-b6b35000 rw-p 00000000 00:00 0 > b6b35000-b6b40000 r-xp 00000000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so > b6b40000-b6b41000 r--p 0000a000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so > b6b41000-b6b42000 rw-p 0000b000 08:01 8151337 /lib/i386-linux-gnu/libnss_files-2.24.so > b6b42000-b6b48000 rw-p 00000000 00:00 0 > b6b48000-b6b53000 r-xp 00000000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so > b6b53000-b6b54000 r--p 0000a000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so > b6b54000-b6b55000 rw-p 0000b000 08:01 8151339 /lib/i386-linux-gnu/libnss_nis-2.24.so > b6b64000-b6b6b000 r--p 00000000 08:01 6286752 /usr/share/locale/de/LC_MESSAGES/libpq5-10.mo > b6b6b000-b6b72000 r--s 00000000 08:01 6903281 /usr/lib/i386-linux-gnu/gconv/gconv-modules.cache > b6b72000-b6b98000 r--p 00000000 08:01 6288754 /usr/share/locale/de/LC_MESSAGES/libc.mo > b6b98000-b6c98000 rw-p 00000000 00:00 0 > b6c9e000-b6cb4000 r-xp 00000000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so > b6cb4000-b6cb5000 r--p 00016000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so > b6cb5000-b6cb6000 rw-p 00017000 08:01 8151334 /lib/i386-linux-gnu/libnsl-2.24.so > b6cb6000-b6cb8000 rw-p 00000000 00:00 0 > b6cb8000-b6cc0000 r-xp 00000000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so > b6cc0000-b6cc1000 r--p 00007000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so > b6cc1000-b6cc2000 rw-p 00008000 08:01 8151335 /lib/i386-linux-gnu/libnss_compat-2.24.so > b6cc2000-b6d02000 rw-p 00000000 00:00 0 > b6d02000-b6d09000 r-xp 00000000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 > b6d09000-b6d0a000 r--p 00006000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 > b6d0a000-b6d0b000 rw-p 00007000 08:01 6899491 /usr/lib/i386-linux-gnu/libffi.so.6.0.4 > b6d0b000-b6d96000 r-xp 00000000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 > b6d96000-b6d97000 r--p 0008a000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 > b6d97000-b6d98000 rw-p 0008b000 08:01 6899509 /usr/lib/i386-linux-gnu/libgmp.so.10.3.2 > b6d98000-b6dcc000 r-xp 00000000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 > b6dcc000-b6dcd000 r--p 00033000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 > b6dcd000-b6dce000 rw-p 00034000 08:01 6899139 /usr/lib/i386-linux-gnu/libhogweed.so.4.3 > b6dce000-b6e08000 r-xp 00000000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 > b6e08000-b6e09000 ---p 0003a000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 > b6e09000-b6e0a000 r--p 0003a000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 > b6e0a000-b6e0b000 rw-p 0003b000 08:01 6897991 /usr/lib/i386-linux-gnu/libnettle.so.6.3 > b6e0b000-b6e1e000 r-xp 00000000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 > b6e1e000-b6e1f000 r--p 00012000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 > b6e1f000-b6e20000 rw-p 00013000 08:01 6898338 /usr/lib/i386-linux-gnu/libtasn1.so.6.5.4 > b6e20000-b6f8c000 r-xp 00000000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 > b6f8c000-b6f8d000 ---p 0016c000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 > b6f8d000-b6f8f000 r--p 0016c000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 > b6f8f000-b6f90000 rw-p 0016e000 08:01 6899693 /usr/lib/i386-linux-gnu/libunistring.so.2.0.0 > b6f90000-b6fac000 r-xp 00000000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 > b6fac000-b6fad000 r--p 0001b000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 > b6fad000-b6fae000 rw-p 0001c000 08:01 6899147 /usr/lib/i386-linux-gnu/libidn2.so.0.3.1 > b6fae000-b70fe000 r-xp 00000000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 > b70fe000-b7104000 r--p 0014f000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 > b7104000-b7109000 rw-p 00155000 08:01 6899036 /usr/lib/i386-linux-gnu/libp11-kit.so.0.3.0 > b7109000-b710a000 rw-p 00000000 00:00 0 > b710a000-b7299000 r-xp 00000000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 > b7299000-b72a1000 r--p 0018e000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 > b72a1000-b72a2000 rw-p 00196000 08:01 6898093 /usr/lib/i386-linux-gnu/libgnutls.so.30.14.7 > b72a2000-b72a3000 rw-p 00000000 00:00 0 > b72a3000-b72bf000 r-xp 00000000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 > b72bf000-b72c0000 r--p 0001b000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 > b72c0000-b72c1000 rw-p 0001c000 08:01 6897853 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25 > b72c1000-b72d5000 r-xp 00000000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so > b72d5000-b72d6000 r--p 00013000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so > b72d6000-b72d7000 rw-p 00014000 08:01 8151343 /lib/i386-linux-gnu/libresolv-2.24.so > b72d7000-b72d9000 rw-p 00000000 00:00 0 > b72d9000-b730b000 r-xp 00000000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 > b730b000-b730c000 ---p 00032000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 > b730c000-b730d000 r--p 00032000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 > b730d000-b730e000 rw-p 00033000 08:01 6898091 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1 > b730e000-b730f000 rw-p 00000000 00:00 0 > b730f000-b73e2000 r-xp 00000000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 > b73e2000-b73e3000 ---p 000d3000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 > b73e3000-b73e9000 r--p 000d3000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 > b73e9000-b73eb000 rw-p 000d9000 08:01 6898964 /usr/lib/i386-linux-gnu/libkrb5.so.3.3 > b73eb000-b7441000 r-xp 00000000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 > b7441000-b7442000 ---p 00056000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 > b7442000-b7443000 r--p 00056000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 > b7443000-b7444000 rw-p 00057000 08:01 6899719 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.10.8 > b7444000-b7445000 rw-p 00000000 00:00 0 > b7445000-b7495000 r-xp 00000000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 > b7495000-b7496000 r--p 0004f000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 > b7496000-b7497000 rw-p 00050000 08:01 6898215 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2 > b7497000-b74e1000 r-xp 00000000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 > b74e1000-b74e3000 r--p 00049000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 > b74e3000-b74e4000 rw-p 0004b000 08:01 6899474 /usr/lib/i386-linux-gnu/libpq.so.5.10 > b74e8000-b74ed000 r-xp 00000000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so > b74ed000-b74ee000 r--p 00004000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so > b74ee000-b74ef000 rw-p 00005000 08:01 6365428 /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so > b74ef000-b74f6000 r-xp 00000000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so > b74f6000-b74f7000 r--p 00006000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so > b74f7000-b74f9000 rw-p 00007000 08:01 6367548 /usr/lib/python2.7/lib-dynload/_csv.i386-linux-gnu.so > b74f9000-b750c000 r-xp 00000000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so > b750c000-b750d000 r--p 00012000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so > b750d000-b750e000 rw-p 00013000 08:01 6367559 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so > b750e000-b7523000 r-xp 00000000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so > b7523000-b7524000 r--p 00014000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so > b7524000-b7527000 rw-p 00015000 08:01 6367569 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so > b7527000-b7559000 r-xp 00000000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so > b7559000-b755a000 r--p 00031000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so > b755a000-b755e000 rw-p 00032000 08:01 6479968 /usr/lib/python2.7/dist-packages/psycopg2/_psycopg.i386-linux-gnu.so > b755e000-b756e000 r-xp 00000000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 > b756e000-b756f000 r--p 0000f000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 > b756f000-b7570000 rw-p 00010000 08:01 8151113 /lib/i386-linux-gnu/libbz2.so.1.0.4 > b7576000-b7584000 r-xp 00000000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 > b7584000-b7585000 r--p 0000d000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 > b7585000-b7586000 rw-p 0000e000 08:01 6897979 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.10.8 > b7586000-b7589000 r-xp 00000000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 > b7589000-b758a000 r--p 00002000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 > b758a000-b758b000 rw-p 00003000 08:01 8151178 /lib/i386-linux-gnu/libkeyutils.so.1.5 > b758b000-b7596000 r-xp 00000000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 > b7596000-b7597000 r--p 0000a000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 > b7597000-b7598000 rw-p 0000b000 08:01 6899192 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1 > b7598000-b759f000 r-xp 00000000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so > b759f000-b75a0000 r--p 00006000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so > b75a0000-b75a1000 rw-p 00007000 08:01 8151344 /lib/i386-linux-gnu/librt-2.24.so > b75a1000-b75b1000 r-xp 00000000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so > b75b1000-b75b2000 r--p 0000f000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so > b75b2000-b75b3000 rw-p 00010000 08:01 6570027 /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so > b75b3000-b77fe000 r-xp 00000000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 > b77fe000-b77ff000 ---p 0024b000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 > b77ff000-b7810000 r--p 0024b000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 > b7810000-b7817000 rw-p 0025c000 08:01 6897994 /usr/lib/i386-linux-gnu/libcrypto.so.1.1 > b7817000-b781a000 rw-p 00000000 00:00 0 > b781a000-b7881000 r-xp 00000000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 > b7881000-b7884000 r--p 00066000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 > b7884000-b7888000 rw-p 00069000 08:01 6899137 /usr/lib/i386-linux-gnu/libssl.so.1.1 > b7888000-b7988000 rw-p 00000000 00:00 0 > b798a000-b798d000 r-xp 00000000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 > b798d000-b798e000 r--p 00002000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 > b798e000-b798f000 rw-p 00003000 08:01 8151240 /lib/i386-linux-gnu/libcom_err.so.2.1 > b798f000-b7998000 r-xp 00000000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so > b7998000-b7999000 r--p 00008000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so > b7999000-b799b000 rw-p 00009000 08:01 6367575 /usr/lib/python2.7/lib-dynload/bz2.i386-linux-gnu.so > b799b000-b79fc000 rw-p 00000000 00:00 0 > b79fc000-b7a00000 r-xp 00000000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so > b7a00000-b7a01000 r--p 00003000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so > b7a01000-b7a02000 rw-p 00004000 08:01 6367555 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so > b7a02000-b7a05000 r-xp 00000000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so > b7a05000-b7a06000 r--p 00002000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so > b7a06000-b7a08000 rw-p 00003000 08:01 6367622 /usr/lib/python2.7/lib-dynload/termios.i386-linux-gnu.so > b7a08000-b7a48000 rw-p 00000000 00:00 0 > b7a48000-b7be3000 r--p 00000000 08:01 6299936 /usr/lib/locale/locale-archive > b7be3000-b7c65000 rw-p 00000000 00:00 0 > b7c65000-b7e16000 r-xp 00000000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so > b7e16000-b7e18000 r--p 001b0000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so > b7e18000-b7e19000 rw-p 001b2000 08:01 8151265 /lib/i386-linux-gnu/libc-2.24.so > b7e19000-b7e1c000 rw-p 00000000 00:00 0 > b7e1c000-b7e6f000 r-xp 00000000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so > b7e6f000-b7e70000 r--p 00052000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so > b7e70000-b7e71000 rw-p 00053000 08:01 8151332 /lib/i386-linux-gnu/libm-2.24.so > b7e71000-b7e8a000 r-xp 00000000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 > b7e8a000-b7e8b000 r--p 00018000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 > b7e8b000-b7e8c000 rw-p 00019000 08:01 8151148 /lib/i386-linux-gnu/libz.so.1.2.8 > b7e8c000-b7e8e000 r-xp 00000000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so > b7e8e000-b7e8f000 r--p 00001000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so > b7e8f000-b7e90000 rw-p 00002000 08:01 8151355 /lib/i386-linux-gnu/libutil-2.24.so > b7e90000-b7e93000 r-xp 00000000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so > b7e93000-b7e94000 r--p 00002000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so > b7e94000-b7e95000 rw-p 00003000 08:01 8151331 /lib/i386-linux-gnu/libdl-2.24.so > b7e95000-b7eae000 r-xp 00000000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so > b7eae000-b7eaf000 r--p 00018000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so > b7eaf000-b7eb0000 rw-p 00019000 08:01 8151342 /lib/i386-linux-gnu/libpthread-2.24.so > b7eb0000-b7eb2000 rw-p 00000000 00:00 0 > b7eb4000-b7ef8000 rw-p 00000000 00:00 0 > b7ef8000-b7efb000 r--p 00000000 00:00 0 [vvar] > b7efb000-b7efd000 r-xp 00000000 00:00 0 [vdso] > b7efd000-b7f20000 r-xp 00000000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so > b7f20000-b7f21000 r--p 00022000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so > b7f21000-b7f22000 rw-p 00023000 08:01 8151050 /lib/i386-linux-gnu/ld-2.24.so > bf9f5000-bfa16000 rw-p 00000000 00:00 0 [stack] > Fatal Python error: Aborted > > Current thread 0xb7c63d00 (most recent call first): > > When run under a debug build it sayeth right away (simulated > as a minimal working example): > > root at hermes:~/bin# python2.7-dbg > Python 2.7.14 (default, Sep 17 2017, 18:50:44) > [GCC 7.2.0] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in > from DateTime import * > File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in > from mxDateTime import * > File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in > raise ImportError, why > ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 I don't really what exactly is going on here, but in general extension modules compiled for non-debug builds won't work with debug builds. The Debian package is almost certainly fine. > For good measure I have filed a bug with Debian asking for > recompilation of python-egenix-mxdatetime. Even if the maintainers do that, it won't help. Check that the module works on its own with the regular Python build and then close the bug if the maintainers don't beat you to it. > When run under "normal" py2.7 it runs all the way through but > upon shutdown (*after* sys.exit(0)) faulthandler shows a > problem (and returns 134 which made me think of SIGABRT): We still don't know what "it" is. Strip down your script as much as possible. It looks like you're using a lot of extension modules, and any one of them could (in theory) be causing problems. -- Thomas Jollans From jasonhihn at gmail.com Wed Oct 18 11:10:37 2017 From: jasonhihn at gmail.com (Jason) Date: Wed, 18 Oct 2017 08:10:37 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> Message-ID: <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> I've read the docs several times, but I still have questions. I've even used multiprocessing before, but not map() from it. I am not sure if map() will let me use a common object (via a manager) and if so, how to set that up. From tomuxiong at gmx.com Wed Oct 18 11:27:09 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Wed, 18 Oct 2017 17:27:09 +0200 Subject: multiprocessing shows no benefit In-Reply-To: <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> Message-ID: On 10/18/2017 05:10 PM, Jason wrote: > I've read the docs several times, but I still have questions. > I've even used multiprocessing before, but not map() from it. > > I am not sure if map() will let me use a common object (via a manager) and if so, how to set that up. > As I said earlier, you should really post the code in question. Fake any data you need so that the problem is computationally the same. Post some file `script.py` so that running `python script.py` works without errors (other than e.g. possible import errors from numpy and stuff like that). This will make it much more likely that someone will be able to help you. Cheers, Thomas From jasonhihn at gmail.com Wed Oct 18 11:46:35 2017 From: jasonhihn at gmail.com (Jason) Date: Wed, 18 Oct 2017 08:46:35 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> Message-ID: <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> #When I change line19 to True to use the multiprocessing stuff it all slows down. from multiprocessing import Process, Manager, Pool, cpu_count from timeit import default_timer as timer def f(a,b): return dict_words[a]-b def f_unpack(args): return f(*args) def init(): global dict_words, dict_keys d = {str(k):k for k in range(100000)} for k in d: dict_words[k] = d[k] dict_keys.append(k) if __name__ == '__main__': print 'CPUs:', cpu_count() # 4 on my machine if False: # make this a zero to use PODs ''' CPUs: 4 pool.map: 47.1494848728 1000000 21209.1394571 map: 49.2051260471 1000000 20323.0858314 ''' manager = Manager() dict_words = manager.dict() dict_keys = manager.list() else: ''' CPUs: 4 pool.map: 3.2546248436 1000000 307255.074872 map: 1.19043779373 1000000 840027.093617 ''' dict_words = {} dict_keys = [] init() pool = Pool(cpu_count()) z=5 funcs = {'pool.map:': pool.map, 'map': map} for name in funcs: start = timer() x = funcs[name](f_unpack, [(k,z) for k in dict_keys]) duration = float(timer() -start) print funcs[name], duration, len(x), len(x) / duration From ian.g.kelly at gmail.com Wed Oct 18 12:13:18 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Oct 2017 10:13:18 -0600 Subject: multiprocessing shows no benefit In-Reply-To: <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> Message-ID: On Wed, Oct 18, 2017 at 9:46 AM, Jason wrote: > #When I change line19 to True to use the multiprocessing stuff it all slows down. > > from multiprocessing import Process, Manager, Pool, cpu_count > from timeit import default_timer as timer > > def f(a,b): > return dict_words[a]-b Since the computation is so simple my suspicion is that the run time is dominated by IPC, in other words the cost of sending objects back and forth outweighs the gains you get from parallelization. What happens if you remove dict_words from the Manager and just pass dict_words[a] across instead of just a? Also, I'm not sure why dict_keys is a managed list to begin with since it only appears to be handled by the main process. From jasonhihn at gmail.com Wed Oct 18 12:21:38 2017 From: jasonhihn at gmail.com (Jason) Date: Wed, 18 Oct 2017 09:21:38 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> Message-ID: <9ed1961d-4f5d-4877-bdbd-714ac0e8abb5@googlegroups.com> On Wednesday, October 18, 2017 at 12:14:30 PM UTC-4, Ian wrote: > On Wed, Oct 18, 2017 at 9:46 AM, Jason wrote: > > #When I change line19 to True to use the multiprocessing stuff it all slows down. > > > > from multiprocessing import Process, Manager, Pool, cpu_count > > from timeit import default_timer as timer > > > > def f(a,b): > > return dict_words[a]-b > > Since the computation is so simple my suspicion is that the run time > is dominated by IPC, in other words the cost of sending objects back > and forth outweighs the gains you get from parallelization. > > What happens if you remove dict_words from the Manager and just pass > dict_words[a] across instead of just a? Also, I'm not sure why > dict_keys is a managed list to begin with since it only appears to be > handled by the main process. You can try that code by changing line 17 :-) It's 10 times faster. Well given the many objects to be iterated over, I was hoping pool.map() would distribute them across the processors. So that each processor gets len(dict_words)/cpu_count() items to process. The actual computation is much longer than a single subtraction, currently I can process about 16k items per second single core. My target is to get those 16k items processed in .25s. From israel at ravnalaska.net Wed Oct 18 12:22:52 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Wed, 18 Oct 2017 08:22:52 -0800 Subject: PEP 249 Compliant error handling In-Reply-To: References: Message-ID: <2A8356A3-13C4-4648-BD97-F078F7D23851@ravnalaska.net> > On Oct 18, 2017, at 1:46 AM, Abdur-Rahmaan Janhangeer wrote: > > all corruption systematically ignored but data piece logged in for analysis Thanks. Can you expound a bit on what you mean by "data piece logged in" in this context? I'm not aware of any logging specifications in the PEP 249, and would think that would be more end-user configured rather than module level. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > > Abdur-Rahmaan Janhangeer, > Mauritius > abdurrahmaanjanhangeer.wordpress.com > > On 17 Oct 2017 21:43, "Israel Brewster" > wrote: > I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the string to l > et them know there was an error, and will allow retrieval of any good data. > > Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > -- > https://mail.python.org/mailman/listinfo/python-list From ian.g.kelly at gmail.com Wed Oct 18 12:31:22 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Oct 2017 10:31:22 -0600 Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> Message-ID: On Wed, Oct 18, 2017 at 10:13 AM, Ian Kelly wrote: > On Wed, Oct 18, 2017 at 9:46 AM, Jason wrote: >> #When I change line19 to True to use the multiprocessing stuff it all slows down. >> >> from multiprocessing import Process, Manager, Pool, cpu_count >> from timeit import default_timer as timer >> >> def f(a,b): >> return dict_words[a]-b > > Since the computation is so simple my suspicion is that the run time > is dominated by IPC, in other words the cost of sending objects back > and forth outweighs the gains you get from parallelization. > > What happens if you remove dict_words from the Manager and just pass > dict_words[a] across instead of just a? Also, I'm not sure why > dict_keys is a managed list to begin with since it only appears to be > handled by the main process. Timings from my system: # Original code without using Manager $ python test.py CPUs: 12 0.0757319927216 100000 1320445.9094 > 0.143120765686 100000 698710.627495 # Original code with Manager $ python test.py CPUs: 12 5.5354039669 100000 18065.5288391 > 4.3253660202 100000 23119.4307101 # Modified code without Manager and avoiding sharing the dict $ python test.py CPUs: 12 0.0657241344452 100000 1521511.09854 > 0.0966320037842 100000 1034853.83811 From israel at ravnalaska.net Wed Oct 18 12:32:48 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Wed, 18 Oct 2017 08:32:48 -0800 Subject: PEP 249 Compliant error handling In-Reply-To: References: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> Message-ID: On Oct 17, 2017, at 12:02 PM, MRAB wrote: > > On 2017-10-17 20:25, Israel Brewster wrote: >> >>> On Oct 17, 2017, at 10:35 AM, MRAB > wrote: >>> >>> On 2017-10-17 18:26, Israel Brewster wrote: >>>> I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the s >>> tring to >>> l >>>> et them know there was an error, and will allow retrieval of any good data. >>>> Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. >>> If a particular text field is corrupted, then raising UnicodeDecodeError when trying to get the contents of that field as a Unicode string seems reasonable to me. >>> >>> Is there a way to get the contents as a bytestring, or to get the contents with a different errors parameter, so that the user has the means to fix it (if it's fixable)? >> >> That's certainly a possibility, if that behavior conforms to the DB API "standards". My concern in this front is that in my experience working with other PEP 249 modules (specifically psycopg2), I'm pretty sure that columns designated as type VARCHAR or TEXT are returned as strings (unicode in python 2, although that may have been a setting I used), not bytes. The other complication here is that the 4D database doesn't use the UTF-8 encoding typically found, but rather UTF-16LE, and I don't know how well this is documented. So not only is the bytes representation completely unintelligible for human consumption, I'm not sure the average end-user would know what decoding to use. >> >> In the end though, the main thing in my mind is to maintain "standards" compatibility - I don't want to be returning bytes if all other DB API modules return strings, or visa-versa for that matter. There may be some flexibility there, but as much as possible I want to conform to the majority/standard/whatever >> > The average end-user might not know which encoding is being used, but providing a way to read the underlying bytes will give a more experienced user the means to investigate and possibly fix it: get the bytes, figure out what the string should be, update the field with the correctly decoded string using normal DB instructions. I agree, and if I was just writing some random module I'd probably go with it, or perhaps with the suggestion offered by Karsten Hilbert. However, neither answer addresses my actual question, which is "how does the STANDARD (PEP 249 in this case) say to handle this, or, baring that (since the standard probably doesn't explicitly say), how do the MAJORITY of PEP 249 compliant modules handle this?" Not what is the *best* way to handle it, but rather what is the normal, expected behavior for a Python DB API module when presented with bad data? That is, how does psycopg2 behave? pyodbc? pymssql (I think)? Etc. Or is that portion of the behavior completely arbitrary and different for every module? It may well be that one of the suggestions *IS* the normal, expected, behavior, but it sounds more like you are suggesting how you think would be best to handle it, which is appreciated but not actually what I'm asking :-) Sorry if I am being difficult. > -- > https://mail.python.org/mailman/listinfo/python-list From ian.g.kelly at gmail.com Wed Oct 18 12:34:44 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Oct 2017 10:34:44 -0600 Subject: multiprocessing shows no benefit In-Reply-To: <9ed1961d-4f5d-4877-bdbd-714ac0e8abb5@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> <9ed1961d-4f5d-4877-bdbd-714ac0e8abb5@googlegroups.com> Message-ID: On Wed, Oct 18, 2017 at 10:21 AM, Jason wrote: > On Wednesday, October 18, 2017 at 12:14:30 PM UTC-4, Ian wrote: >> On Wed, Oct 18, 2017 at 9:46 AM, Jason wrote: >> > #When I change line19 to True to use the multiprocessing stuff it all slows down. >> > >> > from multiprocessing import Process, Manager, Pool, cpu_count >> > from timeit import default_timer as timer >> > >> > def f(a,b): >> > return dict_words[a]-b >> >> Since the computation is so simple my suspicion is that the run time >> is dominated by IPC, in other words the cost of sending objects back >> and forth outweighs the gains you get from parallelization. >> >> What happens if you remove dict_words from the Manager and just pass >> dict_words[a] across instead of just a? Also, I'm not sure why >> dict_keys is a managed list to begin with since it only appears to be >> handled by the main process. > > You can try that code by changing line 17 :-) It's 10 times faster. > > Well given the many objects to be iterated over, I was hoping pool.map() would distribute them across the processors. So that each processor gets len(dict_words)/cpu_count() items to process. The actual computation is much longer than a single subtraction, currently I can process about 16k items per second single core. My target is to get those 16k items processed in .25s. Well, in any case I would try keeping the communication to a minimum. Instead of sending 1 item at a time, break the data set up into batches of 4k and send each child task a batch. Avoid using managers or other shared state. From jasonhihn at gmail.com Wed Oct 18 13:52:09 2017 From: jasonhihn at gmail.com (Jason) Date: Wed, 18 Oct 2017 10:52:09 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> <9ed1961d-4f5d-4877-bdbd-714ac0e8abb5@googlegroups.com> Message-ID: I refactored the map call to break dict_keys into cpu_count() chunks, (so each f() call gets to run continuously over n/cpu_count() items) virtually the same results. pool map is much slower (4x) than regular map, and I don't know why. From tjol at tjol.eu Wed Oct 18 15:15:41 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 18 Oct 2017 21:15:41 +0200 Subject: PEP 249 Compliant error handling In-Reply-To: References: Message-ID: On 17/10/17 19:26, Israel Brewster wrote: > I have written and maintain a PEP 249 compliant (hopefully) DB API for the 4D database, and I've run into a situation where corrupted string data from the database can cause the module to error out. Specifically, when decoding the string, I get a "UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 86-87: illegal UTF-16 surrogate" error. This makes sense, given that the string data got corrupted somehow, but the question is "what is the proper way to deal with this in the module?" Should I just throw an error on bad data? Or would it be better to set the errors parameter to something like "replace"? The former feels a bit more "proper" to me (there's an error here, so we throw an error), but leaves the end user dead in the water, with no way to retrieve *any* of the data (from that row at least, and perhaps any rows after it as well). The latter option sort of feels like sweeping the problem under the rug, but does at least leave an error character in the string to l > et them know there was an error, and will allow retrieval of any good data. > > Of course, if this was in my own code I could decide on a case-by-case basis what the proper action is, but since this a module that has to work in any situation, it's a bit more complicated. The sqlite3 module falls back to returning bytes if there's a decoding error. I don't know what the other modules do. It should be easy enough for you to test this, though! Python 3.5.3 (default, Jan 19 2017, 14:11:04) Type 'copyright', 'credits' or 'license' for more information IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import sqlite3 In [2]: db = sqlite3.connect("malformed-test1.sqlite3") In [3]: db.execute("CREATE TABLE test (txt TEXT)") Out[3]: In [4]: db.execute("INSERT INTO test VALUES(?)", ("utf-8: ?",)) Out[4]: In [5]: db.execute("INSERT INTO test VALUES(?)", ("latin1: ?".encode('latin1'),)) Out[5]: In [6]: db.execute("SELECT * FROM test").fetchall() Out[6]: [('utf-8: ?',), (b'latin1: \xe9',)] In [7]: db.text_factory = bytes # sqlite3 extension to the API In [8]: db.execute("SELECT * FROM test").fetchall() Out[8]: [(b'utf-8: \xc3\xa9',), (b'latin1: \xe9',)] For what it's worth, this is also what os.listdir() does when it encounters filenames in the wrong encoding on operating systems where this is possible (e.g. Linux, but not Windows) If the encoding could be anything, I think you should give the user some kind of choice between using bytes, raising errors, and escaping. In the particular case of UTF-16 (especially if the encoding is always UTF-16), the best solution is almost certainly to use errors='surrogatepass' in both en- and decoding. I believe this is fairly common practice when full interoperability with software that predates UTF-16 (and previously used UCS-2) is required. This should solve all your problems as long as you don't get strings with an odd number of bytes. See: https://en.wikipedia.org/wiki/UTF-16#U.2BD800_to_U.2BDFFF -- Thomas From ian.g.kelly at gmail.com Wed Oct 18 15:38:31 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 18 Oct 2017 13:38:31 -0600 Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: On Mon, Oct 16, 2017 at 10:42 PM, Cai Gengyang wrote: > https://duckduckgo.com/html/?q=%22Cai%20Gengyang%22%20python ---- This > seems to be the only method that works, using DuckDuckGo Or any search engine, really. https://www.google.com/search?q=site%3Apython.org+cai+gengyang From Karsten.Hilbert at gmx.net Wed Oct 18 15:46:45 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Wed, 18 Oct 2017 21:46:45 +0200 Subject: PEP 249 Compliant error handling In-Reply-To: References: <9f74654a-7661-a90b-6882-67559e7192c3@mrabarnett.plus.com> <0980E283-BDDA-4002-843F-E9B66DB02C0A@ravnalaska.net> Message-ID: <20171018194645.3ydqdyzpgzu5jp3g@hermes.hilbert.loc> On Wed, Oct 18, 2017 at 08:32:48AM -0800, Israel Brewster wrote: > actual question, which is "how does the STANDARD (PEP 249 in > this case) say to handle this, or, baring that (since the > standard probably doesn't explicitly say), how do the > MAJORITY of PEP 249 compliant modules handle this?" Not what > is the *best* way to handle it, but rather what is the > normal, expected behavior for a Python DB API module when > presented with bad data? That is, how does psycopg2 behave? > pyodbc? pymssql (I think)? Etc. Or is that portion of the > behavior completely arbitrary and different for every module? For what it is worth, psycopg2 does not give you bad data to the best of my knowledge. In fact, given PostgreSQL's quite tight checking of text data to be "good" psycopg2 hardly has a chance to give you bad data. Most times the database itself will detect the corruption and not even hand the data to psycopg2. IMHO a driver should not hand over to the client any bad data unless explicitely told to do so, which latter case does not seem to be covered by the DB-API specs, regardless of what the majority of drivers might do these days. 2 cent... Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From chololennon at hotmail.com Wed Oct 18 16:59:51 2017 From: chololennon at hotmail.com (Cholo Lennon) Date: Wed, 18 Oct 2017 17:59:51 -0300 Subject: how to read in the newsreader References: <8eded8ec-ad93-f176-ebdd-38ac73ee7859@gmail.com> Message-ID: On 16/10/17 01:00, Michael Torrie wrote: > On 10/15/2017 08:50 PM, Andrew Z wrote: >> Gents, >> how do i get this group in a newsreader? The digest i'm getting is not >> workable for me - i can't reply , can only read the replies from the >> members of the group. Or. maybe, it shouldn't be a news reader.... >> please advise.. >> >> P.S. Oh the comp.lang.python is a nightmare because of spam... > > Regardless of what usenet reader you use, com.lang.python will have the > same spam everywhere. So maybe reading via usenet isn't that useful anyway. I use Mozilla Thunderbird as a newsreader. I am subscribed to 'comp.lang.python' through the free NNTP server 'news.aioe.org'. My spam filter for python only has 6 address! > > I just subscribe to the mailing list and then have a rule in my email > filter to stick all list messages in their own folder (or gmail label). > I'm using gmail, so I just use gmail's filtering settings. Then I view > email using IMAP and a conventional email reader that supports real > threading of messages. Works great. A lot of spam is filtered out this > way, also. In fact very little spam gets through to my folder between > python.org's filtering and gmail's spam detection. > > I'm subscribed to maybe a dozen email lists, and I filter all of them > into their own folders. > In my case I am subscribed (with Thunderbird) to more than 30 newsgroups (in 3 news servers). I have to say that spam is not a problem. Regards -- Cholo Lennon Bs.As. ARG From cs at cskk.id.au Wed Oct 18 17:14:35 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 19 Oct 2017 08:14:35 +1100 Subject: PEP 249 Compliant error handling In-Reply-To: References: Message-ID: <20171018211435.GA66938@cskk.homeip.net> On 17Oct2017 21:38, Karsten Hilbert wrote: >> That's certainly a possibility, if that behavior conforms to the DB API "standards". My concern in this front is that in my experience working with other PEP 249 modules (specifically psycopg2), I'm pretty sure that columns designated as type VARCHAR or TEXT are returned as strings (unicode in python 2, although that may have been a setting I used), not bytes. The other complication here is that the 4D database doesn't use the UTF-8 encoding typically found, but rather UTF-16LE, and I don't know how well this is documented. So not only is the bytes representation completely unintelligible for human consumption, I'm not sure the average end-user would know what decoding to use. >> >> In the end though, the main thing in my mind is to maintain "standards" compatibility - I don't want to be returning bytes if all other DB API modules return strings, or visa-versa for that matter. There may be some flexibility there, but as much as possible I want to conform to the majority/standard/whatever > > >The thing here is that you don't want to return data AS IF it was correct despite it having been >"corrected" by some driver logic. I just want to say that I think this is correct and extremely important. >What might be interesting to users is to set an attribute on the cursor, say, > cursor.faulty_data = unicode(faulty_data, errors='replace') >or some such in order to improve error messages to the end user. Or perhaps more conveniently for the end user, possibly an option supplied at db connect time, though I'd entirely understand wanting a cursor option so that one can pick and choose in a fine grained fashion. Cheers, Cameron Simpson (formerly cs at zip.com.au) From jfong at ms4.hinet.net Wed Oct 18 21:02:32 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 18 Oct 2017 18:02:32 -0700 (PDT) Subject: How to debug an unfired tkinter event? Message-ID: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> In last few days, I tried to experiment with the scrolling table implemented in canvas, started from this example: http://code.activestate.com/recipes/580793-tkinter-table-with-scrollbars/. Everything works fine until I moved the scrolling_area instance (which the canvas is in) from column=0 to column=1. The canvas has a binding: self.canvas.bind('', self._on_canvas_configure) After this movement, the callback was only triggered when dragging the root widget to resize canvas vertically, but not horizontally. Event seems has OS involved(it's MS Windows XP in my case). How to debug this kind of problem, under pdb? Best Regards, Jach Fong From lingmaaki at gmail.com Thu Oct 19 00:35:15 2017 From: lingmaaki at gmail.com (lingmaaki at gmail.com) Date: Wed, 18 Oct 2017 21:35:15 -0700 (PDT) Subject: Python Interview Questions In-Reply-To: <1193768041.349129.26350@v3g2000hsg.googlegroups.com> References: <1193768041.349129.26350@v3g2000hsg.googlegroups.com> Message-ID: <0bddf215-970f-4099-871e-03251a85a7d4@googlegroups.com> Hope this will help you.... http://net-informations.com/python/iq/default.htm From dieter at handshake.de Thu Oct 19 02:32:38 2017 From: dieter at handshake.de (dieter) Date: Thu, 19 Oct 2017 08:32:38 +0200 Subject: right list for SIGABRT python binary question ? References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> Message-ID: <87k1zrfy0p.fsf@handshake.de> Karsten Hilbert writes: > ... > When run under "normal" py2.7 it runs all the way through but > upon shutdown (*after* sys.exit(0)) faulthandler shows a > problem (and returns 134 which made me think of SIGABRT): > > *** Error in `python': free(): invalid pointer: 0x00770b14 *** This indicates some form of memory corruption, usually caused by some C extension. Unfortunately, memory corruption is rarely noticed locally; therefore, the localization is typically complex. Maybe, you are lucky and the "mxdatetime" is to blame. From __peter__ at web.de Thu Oct 19 03:23:39 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 19 Oct 2017 09:23:39 +0200 Subject: How to debug an unfired tkinter event? References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: jfong at ms4.hinet.net wrote: > In last few days, I tried to experiment with the scrolling table > implemented in canvas, started from this example: > http://code.activestate.com/recipes/580793-tkinter-table-with-scrollbars/. > Everything works fine until I moved the scrolling_area instance (which the > canvas is in) from column=0 to column=1. It's not clear to me what you mean with this. Did you place the table from the recipe elsewhere inside a window that you created or did you make changes in the recipe's code? > The canvas has a binding: > self.canvas.bind('', self._on_canvas_configure) > > After this movement, the callback was only triggered when dragging the > root widget to resize canvas vertically, but not horizontally. > > Event seems has OS involved(it's MS Windows XP in my case). How to debug > this kind of problem, under pdb? I don't know, but if you can post a small example script that demonstrates the problem, and you are lucky, someone will see the problem. At the very least we can check if it occurs on your oldish platform (what Python version?) only. From nomail at com.invalid Thu Oct 19 04:11:02 2017 From: nomail at com.invalid (ast) Date: Thu, 19 Oct 2017 10:11:02 +0200 Subject: What happens when a __call__ function is defined in both class and object ? Message-ID: <59e85e1f$0$31612$426a74cc@news.free.fr> Hello, please have a look at following code snippet (python 3.4.4) class Test: a = 1 def __init__(self): self.a = 2 self.f = lambda : print("f from object") self.__call__ = lambda : print("__call__ from object") def __call__(self): print("__call__ from class Test") def f(self): print("f from class Test") test=Test() >>> test.a 2 ok, a is defined in both the class and the object, it is read from the object. This is the expected behavior >>> test.f() f from object ok for the same reason >>> test() __call__ from class Test Surprisingly, __call__ from the class is called, not the one defined in the object. Why ? Regards From tjol at tjol.eu Thu Oct 19 04:22:52 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 19 Oct 2017 10:22:52 +0200 Subject: What happens when a __call__ function is defined in both class and object ? In-Reply-To: <59e85e1f$0$31612$426a74cc@news.free.fr> References: <59e85e1f$0$31612$426a74cc@news.free.fr> Message-ID: <4b6b00b8-d002-f192-a6ad-712383e93b07@tjol.eu> On 2017-10-19 10:11, ast wrote: > Surprisingly, __call__ from the class is called, not the > one defined in the object. Why ? That's just how dunder methods like __call__ work. See https://docs.python.org/3/reference/datamodel.html#special-method-names To quote the docs: > For instance, if a class defines a method named __getitem__(), and x is an instance of this class, then x[i] is roughly equivalent to type(x).__getitem__(x, i). This is also true for __call__. In your case, calling test is more-or-less equivalent to getattr(type(test), '__call__')(test), NOT getattr(test, '__call__')(). -- Thomas Jollans From rosuav at gmail.com Thu Oct 19 04:24:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Oct 2017 19:24:15 +1100 Subject: What happens when a __call__ function is defined in both class and object ? In-Reply-To: <59e85e1f$0$31612$426a74cc@news.free.fr> References: <59e85e1f$0$31612$426a74cc@news.free.fr> Message-ID: On Thu, Oct 19, 2017 at 7:11 PM, ast wrote: > Hello, please have a look at following code snippet > (python 3.4.4) > > class Test: > > a = 1 > > def __init__(self): > self.a = 2 > self.f = lambda : print("f from object") > self.__call__ = lambda : print("__call__ from object") > def __call__(self): > print("__call__ from class Test") > def f(self): > print("f from class Test") > > test=Test() > >>>> test.a > > 2 > ok, a is defined in both the class and the object, it is read from > the object. This is the expected behavior > >>>> test.f() > > f from object > > ok for the same reason > >>>> test() > > __call__ from class Test > > > Surprisingly, __call__ from the class is called, not the > one defined in the object. Why ? That's the way most dunder methods (those with "d"ouble "under"scores before and after their names) work. From rosuav at gmail.com Thu Oct 19 04:26:57 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 19 Oct 2017 19:26:57 +1100 Subject: What happens when a __call__ function is defined in both class and object ? In-Reply-To: References: <59e85e1f$0$31612$426a74cc@news.free.fr> Message-ID: On Thu, Oct 19, 2017 at 7:24 PM, Chris Angelico wrote: > On Thu, Oct 19, 2017 at 7:11 PM, ast wrote: >> Hello, please have a look at following code snippet >> (python 3.4.4) >> >> class Test: >> >> a = 1 >> >> def __init__(self): >> self.a = 2 >> self.f = lambda : print("f from object") >> self.__call__ = lambda : print("__call__ from object") >> def __call__(self): >> print("__call__ from class Test") >> def f(self): >> print("f from class Test") >> >> test=Test() >> >>>>> test.a >> >> 2 >> ok, a is defined in both the class and the object, it is read from >> the object. This is the expected behavior >> >>>>> test.f() >> >> f from object >> >> ok for the same reason >> >>>>> test() >> >> __call__ from class Test >> >> >> Surprisingly, __call__ from the class is called, not the >> one defined in the object. Why ? > > That's the way most dunder methods (those with "d"ouble "under"scores > before and after their names) work. Oops, premature send, sorry! Dunder methods are looked up on the class. In the extremely rare situation where you want a per-instance customization, you can do it yourself with something like this: class Test: def __call__(self): if hasattr(self, "spam_handler"): return self.spam_handler() return generic_spam_handler() It's more efficient if, most of the time, the instance dict is skipped and the search starts with the class. ChrisA From jfong at ms4.hinet.net Thu Oct 19 05:07:53 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 19 Oct 2017 02:07:53 -0700 (PDT) Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: Peter Otten at 2017-10-19 UTC+8 PM 3:24:30 wrote: > It's not clear to me what you mean with this. Did you place the table from > the recipe elsewhere inside a window that you created or did you make > changes in the recipe's code? Thank you, Peter. I am using Python 3.4.4 under WinXP. When running the file directly from download, I get a table scrolling vertically only. If its Table class's __init__ parameter "scroll_horizontally" changed to True, it can be scrolled horizontally also. But there is a problem when scrolled horizontally, the header field will not align with data field anymore. I make some modifications to make it does. So far so good. Later, I want to make the table also has a vertical header. The first step I had taken was to move all 2nd top-level widgets(not much, just four) to right one column further to make room for this new widget. I though it's a simple work, just increase their grid column value by 1. But Murphy's Law comes, the xscrollbar even don't show up when the table was squeezed horizontally. > > The canvas has a binding: > > self.canvas.bind('', self._on_canvas_configure) > > > > After this movement, the callback was only triggered when dragging the > > root widget to resize canvas vertically, but not horizontally. > > > > Event seems has OS involved(it's MS Windows XP in my case). How to debug > > this kind of problem, under pdb? > > I don't know, but if you can post a small example script that demonstrates > the problem, and you are lucky, someone will see the problem. This is a ~5xx lines file and I think it's not fare to ask forum members to look through it. So I decide to ask for help on how to debug it, instead of the solution. I try to change the binding to self.bind_all('', self._on_canvas_configure) and add a line into the callback print(event.widget) I got some info below each time when I squeeze the table: . .50077776 .50077776.50712528 .50077776.50712496 .50077776.50712464 .50077776.50712144 .50077776.50712528.50712560.50782256 .50077776.50712528.50712560.50782256.50783024 .50077776.50712528.50712560.50782256 .50077776.50712528.50712560.50782256.50783024 How to change these number(is it a widget ID?) to a meaning name? --Jach From __peter__ at web.de Thu Oct 19 06:03:57 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 19 Oct 2017 12:03:57 +0200 Subject: How to debug an unfired tkinter event? References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: jfong at ms4.hinet.net wrote: > Peter Otten at 2017-10-19 UTC+8 PM 3:24:30 wrote: >> It's not clear to me what you mean with this. Did you place the table >> from the recipe elsewhere inside a window that you created or did you >> make changes in the recipe's code? > > Thank you, Peter. I am using Python 3.4.4 under WinXP. > > When running the file directly from download, I get a table scrolling > vertically only. If its Table class's __init__ parameter > "scroll_horizontally" changed to True, it can be scrolled horizontally > also. But there is a problem when scrolled horizontally, the header field > will not align with data field anymore. I make some modifications to make > it does. So far so good. > > Later, I want to make the table also has a vertical header. The first step > I had taken was to move all 2nd top-level widgets(not much, just four) to > right one column further to make room for this new widget. I though it's a > simple work, just increase their grid column value by 1. But Murphy's Law > comes, the xscrollbar even don't show up when the table was squeezed > horizontally. Thank you for the clarification; I understand your problem much better now. > >> > The canvas has a binding: >> > self.canvas.bind('', self._on_canvas_configure) >> > >> > After this movement, the callback was only triggered when dragging the >> > root widget to resize canvas vertically, but not horizontally. >> > >> > Event seems has OS involved(it's MS Windows XP in my case). How to >> > debug this kind of problem, under pdb? >> >> I don't know, but if you can post a small example script that >> demonstrates the problem, and you are lucky, someone will see the >> problem. > > This is a ~5xx lines file and I think it's not fare to ask forum members > to look through it. So I decide to ask for help on how to debug it, > instead of the solution. > > I try to change the binding to > self.bind_all('', self._on_canvas_configure) > and add a line into the callback > print(event.widget) > > I got some info below each time when I squeeze the table: > . > .50077776 > .50077776.50712528 > .50077776.50712496 > .50077776.50712464 > .50077776.50712144 > .50077776.50712528.50712560.50782256 > .50077776.50712528.50712560.50782256.50783024 > .50077776.50712528.50712560.50782256 > .50077776.50712528.50712560.50782256.50783024 > > How to change these number(is it a widget ID?) to a meaning name? That's the name of the widget in the underlying tcl/tk. You can specify it in Python: >>> import tkinter as tk >>> print(tk.Button()) .139817813335904 >>> print(tk.Button(name="mybutton")) .mybutton You have to ensure that names are unique. From jfong at ms4.hinet.net Thu Oct 19 08:42:24 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 19 Oct 2017 05:42:24 -0700 (PDT) Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: <5b371b88-63a8-4b05-bf4f-65e92b61d4af@googlegroups.com> Peter Otten? 2017?10?19???? UTC+8??6?04?39???? > jfong at ms4.hinet.net wrote: > > > Peter Otten at 2017-10-19 UTC+8 PM 3:24:30 wrote: > >> It's not clear to me what you mean with this. Did you place the table > >> from the recipe elsewhere inside a window that you created or did you > >> make changes in the recipe's code? > > > > Thank you, Peter. I am using Python 3.4.4 under WinXP. > > > > When running the file directly from download, I get a table scrolling > > vertically only. If its Table class's __init__ parameter > > "scroll_horizontally" changed to True, it can be scrolled horizontally > > also. But there is a problem when scrolled horizontally, the header field > > will not align with data field anymore. I make some modifications to make > > it does. So far so good. > > > > Later, I want to make the table also has a vertical header. The first step > > I had taken was to move all 2nd top-level widgets(not much, just four) to > > right one column further to make room for this new widget. I though it's a > > simple work, just increase their grid column value by 1. But Murphy's Law > > comes, the xscrollbar even don't show up when the table was squeezed > > horizontally. > > Thank you for the clarification; I understand your problem much better now. > > > > >> > The canvas has a binding: > >> > self.canvas.bind('', self._on_canvas_configure) > >> > > >> > After this movement, the callback was only triggered when dragging the > >> > root widget to resize canvas vertically, but not horizontally. > >> > > >> > Event seems has OS involved(it's MS Windows XP in my case). How to > >> > debug this kind of problem, under pdb? > >> > >> I don't know, but if you can post a small example script that > >> demonstrates the problem, and you are lucky, someone will see the > >> problem. > > > > This is a ~5xx lines file and I think it's not fare to ask forum members > > to look through it. So I decide to ask for help on how to debug it, > > instead of the solution. > > > > I try to change the binding to > > self.bind_all('', self._on_canvas_configure) > > and add a line into the callback > > print(event.widget) > > > > I got some info below each time when I squeeze the table: > > . > > .50077776 > > .50077776.50712528 > > .50077776.50712496 > > .50077776.50712464 > > .50077776.50712144 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > > > How to change these number(is it a widget ID?) to a meaning name? > > That's the name of the widget in the underlying tcl/tk. You can specify it > in Python: > > >>> import tkinter as tk > >>> print(tk.Button()) > .139817813335904 > >>> print(tk.Button(name="mybutton")) > .mybutton > > You have to ensure that names are unique. Thank you. It's very helpful information. By the way, I just found this problem can be easily seen by not modifing the original file too much with the following steps: 1. copy and get the file from the link. 2. change the "scroll_horizontally" parameter in the "Table" class's __init__ method to "True" Running it you will see it perform correctly. 3. Change the "column=x" at line 268, 282, 288, 292, 303 to "column=x+1" Now it has the problem. --Jach From robin at reportlab.com Thu Oct 19 10:20:13 2017 From: robin at reportlab.com (Robin Becker) Date: Thu, 19 Oct 2017 15:20:13 +0100 Subject: efficient way to get a sufficient set of identifying attributes Message-ID: <9a789760-515f-659f-6a64-9c1ecd37a33c@chamonix.reportlab.co.uk> Given a list of objects with attributes a0, a1, a2,....an-1 is there an efficient way to find sets of attributes which can be used to distinguish members of the list? As example a list of people might have firstName, lastName, nationality, postcode, phonenumber,.... as attributes. The probe items may have some of these attributes, but which should be used to test. Presumably the information in any attribute is highest if the number of distinct occurrences is the the same as the list length and pairs of attributes are more likely to be unique, but is there some proper way to go about determining what tests to use? A particular problem might be dynamic in that the list may be being constructed from the probes. -- Robin Becker From robin at reportlab.com Thu Oct 19 12:05:57 2017 From: robin at reportlab.com (Robin Becker) Date: Thu, 19 Oct 2017 17:05:57 +0100 Subject: efficient way to get a sufficient set of identifying attributes In-Reply-To: References: <9a789760-515f-659f-6a64-9c1ecd37a33c@chamonix.reportlab.co.uk> Message-ID: On 19/10/2017 16:42, Stefan Ram wrote: > Robin Becker writes: >> Presumably the information in any attribute is highest >> if the number of distinct occurrences is the the same as the list length and >> pairs of attributes are more likely to be unique, but is there some proper way >> to go about determining what tests to use? > > When there is a list > > |>>> list = [ 'b', 'b', 'c', 'd', 'c', 'b' ] > |>>> l = len( list ) > > , the length of its set can be obtained: > > |>>> s = len( set( list )) > > . The entries are unique if the length of the set is the > length of the list > > |>>> l == s > |False > > And the ratio between the length of the set and the length > of the list can be used to quantify the amount of repetiton. > > |>>> s / l > |0.5 ....... this sort of makes sense for single attributes, but ignores the possibility of combining the attributes to make the checks more discerning. -- Robin Becker From daniel.p.flick at gmail.com Thu Oct 19 12:46:48 2017 From: daniel.p.flick at gmail.com (Daniel Flick) Date: Thu, 19 Oct 2017 09:46:48 -0700 (PDT) Subject: Is there a function of ipaddress to get the subnet only from input like 192.168.1.129/25 In-Reply-To: References: <62a5e58a-cf38-457d-a586-d18dc1e4f58c@googlegroups.com> <99e8e6f0-2c94-4608-89c2-fbd05775ed20@googlegroups.com> <2169fd67-876f-4391-8302-62ef08ef0ce0@googlegroups.com> Message-ID: <727caa00-dfbd-4376-9161-c32b4ef118a0@googlegroups.com> The problem was with the validation code. Within the python section of the template, the class IPv4Interface will throw an exception due to the invalid value during the validation process. Therefore, the server rejects the form data and the template is not created. Solution: It would work if you add an error handling to the python section of the template. <%! ## python module-level code import ipaddress %> <%def name="get_address(ip_string)"> <% try: return ipaddress.IPv4Interface(ip_string).ip except Exception: return "FAIL_OR_EMPTY" %> ! Variable Input: ${LAN_IP} ${get_address(LAN_IP)} Explanation: During the server-side validation of the HTML form, the configuration template is rendered with a dummy parameter set to verify the syntax (see file app/forms.py, class ConfigTemplateForm). Your config template is validated with the following parameter set during the form validation: { "hostname": "test", "LAN_IP": "test" } From Karsten.Hilbert at gmx.net Thu Oct 19 13:27:45 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Thu, 19 Oct 2017 19:27:45 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> Message-ID: <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> On Wed, Oct 18, 2017 at 02:07:46PM +0200, Thomas Jollans wrote: > > When run under a debug build it sayeth right away (simulated > > as a minimal working example): > > > > root at hermes:~/bin# python2.7-dbg > > Python 2.7.14 (default, Sep 17 2017, 18:50:44) > > [GCC 7.2.0] on linux2 > > Type "help", "copyright", "credits" or "license" for more information. > > >>> import mx.DateTime > > *** You don't have the (right) mxDateTime binaries installed ! > > Traceback (most recent call last): > > File "", line 1, in > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in > > from DateTime import * > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in > > from mxDateTime import * > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in > > raise ImportError, why > > ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 > > I don't really what exactly is going on here, but in general extension > modules compiled for non-debug builds won't work with debug builds. OK, that helps. I found -dbg builds of mx.DateTime in Debian. > > For good measure I have filed a bug with Debian asking for > > recompilation of python-egenix-mxdatetime. > > Even if the maintainers do that, it won't help. Check that the module > works on its own with the regular Python build and then close the bug if > the maintainers don't beat you to it. Done. > > When run under "normal" py2.7 it runs all the way through but > > upon shutdown (*after* sys.exit(0)) faulthandler shows a > > problem (and returns 134 which made me think of SIGABRT): > > We still don't know what "it" is. > > Strip down your script as much as possible. It looks like you're using a > lot of extension modules, and any one of them could (in theory) be > causing problems. I know. However, stripping down isn't quite so easy (it never is, which is a rather lame excuse for not doing so yet :-) The script itself is but a meager 1843 lines (but using lots of Python modules, both stdlib and my own). What it does is take a large number of SQL files (roughly a thousand) and replaying them against PostgreSQL thereby bootstrapping a database layout from scratch up to the current version (21). The abort-on-exit only happens if "enough" SQL scripts have been run (sounds like a resource leak) and the bootstrapper script exits (having encountered a database problem or not makes no difference). Running various parts of the whole procedure does not make a difference as to whether the fault occurs when exiting, if only "enough" SQL scripts are run. I am currently running the bootstrapper with mxdatetime as a dbg build to see what gives. The only other C extension I am aware of that is in use is psycopg2. However, none of these two (nor any other modules) have been added to the bootstrapper (or updated) recently (in fact, this setup has been in use for, what, a decade?) so it is quite unclear why they should be at fault now. The one thing that did change is PostgreSQL going from 9.6 to 10, effecting a libpq5 recompiled against the newer PG client. I wonder whether psycopg2 needs a recompile against that libpq5 (despite there not having been changes advertised by PG). I guess I'll have to downgrade libpq5 to 9.6 and see what happens. I'll report what I find. (oh, and it's all available on github) Thanks, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From rosuav at gmail.com Thu Oct 19 13:39:06 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Oct 2017 04:39:06 +1100 Subject: efficient way to get a sufficient set of identifying attributes In-Reply-To: References: <9a789760-515f-659f-6a64-9c1ecd37a33c@chamonix.reportlab.co.uk> Message-ID: On Fri, Oct 20, 2017 at 4:21 AM, Stefan Ram wrote: > Dennis Lee Bieber writes: >>Interesting -- that is coming out to be 2^size - 1, which will sure speed >>up calculation for larger sets rather than doing all the factorial stuff. > > A set of size n has 2^n subsets. > > We exclude the empty set for our purpose, so we end at 2^n-1. Why is this? Simple. In each subset, each element is either there or not-there, independently of the others. So each one is 1 bit of information. ChrisA From israel at ravnalaska.net Thu Oct 19 13:40:23 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 19 Oct 2017 09:40:23 -0800 Subject: Efficient counting of results Message-ID: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> I am working on developing a report that groups data into a two-dimensional array based on date and time. More specifically, date is grouped into categories: day, week-to-date, month-to-date, and year-to-date Then, for each of those categories, I need to get a count of records that fall into the following categories: 0 minutes late, 1-5 minutes late, and 6-15 minutes late where minutes late will be calculated based on a known scheduled time and the time in the record. To further complicate things, there are actually two times in each record, so under the day, week-to-date, month-to-date etc groups, there will be two sets of "late" bins, one for each time. In table form it would look something like this: | day | week-to-date | month-to-date | year-to-date | ---------------------------------------------------------------------------------------- t1 0min | t1 1-5 min | ... t1 6-15 min | ... t2 0min | ... t2 1-5 min | ... t2 6-15 min | ... So in the extreme scenario of a record that is for the current day, it will be counted into 8 bins: once each for day, week-to-date, month-to-date and year-to-date under the proper "late" bin for the first time in the record, and once each into each of the time groups under the proper "late" bin for the second time in the record. An older record may only be counted twice, under the year-to-date group. A record with no matching schedule is discarded, as is any record that is "late" by more than 15 minutes (those are gathered into a separate report) My initial approach was to simply make dictionaries for each "row" in the table, like so: t10 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} t15 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} . . t25 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} t215 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} then loop through the records, find the schedule for that record (if any, if not move on as mentioned earlier), compare t1 and t2 against the schedule, and increment the appropriate bin counts using a bunch of if statements. Functional, if ugly. But then I got to thinking: I keep hearing about all these fancy numerical analysis tools for python like pandas and numpy - could something like that help? Might there be a way to simply set up a table with "rules" for the columns and rows, and drop my records into the table, having them automatically counted into the proper bins or something? Or am I over thinking this, and the "simple", if ugly approach is best? ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From Karsten.Hilbert at gmx.net Thu Oct 19 13:49:07 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Thu, 19 Oct 2017 19:49:07 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> Message-ID: <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> On Thu, Oct 19, 2017 at 07:27:45PM +0200, Karsten Hilbert wrote: > I am currently running the bootstrapper with mxdatetime as a > dbg build to see what gives. The only other C extension I am > aware of that is in use is psycopg2. So here's the final console output of that: ==> bootstrapping "v20_fixups-pre_v21" ... ==> dropping pre-existing target database [gnumed_v21] ... ==> cloning [gnumed_v20] (72 MB) as target database [gnumed_v21] ... ==> reindexing target database (can take a while) ... ==> transferring users ... ==> bootstrapping "v20-v21-static" ... ==> bootstrapping "v20-v21-dynamic" ... ==> bootstrapping "v21-fixups" ... ==> setting up auditing ... ==> setting up encounter/episode FKs and IDXs ... ==> setting up encounter/episode FK sanity check triggers ... ==> setting up generic notifications ... ==> upgrading reference data sets ... ==> verifying target database schema ... ==> checking migrated data for plausibility ... Done bootstrapping GNUmed database: We very likely succeeded. log: /home/ncq/Projekte/gm-git/gnumed/gnumed/server/bootstrap/bootstrap-latest.log Debug memory block at address p=0x717b7c: API '' 0 bytes originally requested The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): at p-3: 0x03 *** OUCH at p-2: 0x4e *** OUCH at p-1: 0x00 *** OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 4 pad bytes at tail=0x717b7c are not all FORBIDDENBYTE (0xfb): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH The block was made by call #0 to debug malloc/realloc. Fatal Python error: bad ID: Allocated using API '', verified using API 'o' ./bootstrap-latest.sh: Zeile 80: 28023 Abgebrochen ./bootstrap_gm_db_system.py --log-file=${LOG} --conf-file=${CONF} --${QUIET} Bootstrapping "gnumed_v21" did not finish successfully (134). Aborting. Note there's not faulthandler output here because I don't yet know how to get a debug build of *that* (Debian does not seem to have one and neither does pypi to my knowledge). That'd be needed because of: root at hermes:~/bin# python2.7-dbg Python 2.7.14 (default, Sep 17 2017, 18:50:44) [GCC 7.2.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import faulthandler Traceback (most recent call last): File "", line 1, in ImportError: /usr/lib/python2.7/dist-packages/faulthandler.i386-linux-gnu.so: undefined symbol: Py_InitModule4 [45316 refs] >>> Can anyone give more guidance on what the above python debug output might vaguely point to ? Thanks, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From israel at ravnalaska.net Thu Oct 19 14:04:22 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 19 Oct 2017 10:04:22 -0800 Subject: Efficient counting of results In-Reply-To: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> Message-ID: <4953C7C3-F61A-4440-BFF3-3504BE9F9F6D@ravnalaska.net> > On Oct 19, 2017, at 9:40 AM, Israel Brewster wrote: > > I am working on developing a report that groups data into a two-dimensional array based on date and time. More specifically, date is grouped into categories: > > day, week-to-date, month-to-date, and year-to-date > > Then, for each of those categories, I need to get a count of records that fall into the following categories: > > 0 minutes late, 1-5 minutes late, and 6-15 minutes late > > where minutes late will be calculated based on a known scheduled time and the time in the record. To further complicate things, there are actually two times in each record, so under the day, week-to-date, month-to-date etc groups, there will be two sets of "late" bins, one for each time. In table form it would look something like this: > > | day | week-to-date | month-to-date | year-to-date | > ---------------------------------------------------------------------------------------- > t1 0min | > t1 1-5 min | ... > t1 6-15 min | ... > t2 0min | ... > t2 1-5 min | ... > t2 6-15 min | ... > > So in the extreme scenario of a record that is for the current day, it will be counted into 8 bins: once each for day, week-to-date, month-to-date and year-to-date under the proper "late" bin for the first time in the record, and once each into each of the time groups under the proper "late" bin for the second time in the record. An older record may only be counted twice, under the year-to-date group. A record with no matching schedule is discarded, as is any record that is "late" by more than 15 minutes (those are gathered into a separate report) > > My initial approach was to simply make dictionaries for each "row" in the table, like so: > > t10 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} > t15 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} > . > . > t25 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} > t215 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} > > then loop through the records, find the schedule for that record (if any, if not move on as mentioned earlier), compare t1 and t2 against the schedule, and increment the appropriate bin counts using a bunch of if statements. Functional, if ugly. But then I got to thinking: I keep hearing about all these fancy numerical analysis tools for python like pandas and numpy - could something like that help? Might there be a way to simply set up a table with "rules" for the columns and rows, and drop my records into the table, having them automatically counted into the proper bins or something? Or am I over thinking this, and the "simple", if ugly approach is best? I suppose I should mention: my data source is the results of a psycopg2 query, so a "record" is a tuple or dictionary (depending on how I want to set up the cursor) > > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > -- > https://mail.python.org/mailman/listinfo/python-list From mattschepers at gmail.com Thu Oct 19 14:08:01 2017 From: mattschepers at gmail.com (Matt Schepers) Date: Thu, 19 Oct 2017 12:08:01 -0600 Subject: Vim, ctags jump to python standard library source Message-ID: I prefer to use vim and ctags when developing python, but I'm having trouble getting ctags to index the standard library. Sometimes I would like to see an object's constructor etc... Does anyone know how to do this? Thanks From rdecker at siena.edu Thu Oct 19 14:15:07 2017 From: rdecker at siena.edu (Decker, Ryan C.) Date: Thu, 19 Oct 2017 14:15:07 -0400 Subject: make test on Centos 7.4 fails Message-ID: Hey Python guys, test_socket doesn't seem to be passing on a clean CentOS 7.4 install. strace -s 128 -e trace=%network -o trace ./python -m test -v test_socket -m test_sha256 == CPython 3.6.3 (default, Oct 19 2017, 14:12:01) [GCC 7.1.1 20170526 (Red Hat 7.1.1-2)] == Linux-3.10.0-693.2.2.el7.x86_64-x86_64-with-centos-7.4.1708-Core little-endian == cwd: /tmp/Python-3.6.3/build/test_python_20478 == CPU count: 8 == encodings: locale=UTF-8, FS=utf-8 Run tests sequentially 0:00:00 load avg: 1.03 [1/1] test_socket test_sha256 (test.test_socket.LinuxKernelCryptoAPI) ... ERROR *======================================================================* *ERROR: test_sha256 (test.test_socket.LinuxKernelCryptoAPI)* *----------------------------------------------------------------------* *Traceback (most recent call last):* * File "/tmp/Python-3.6.3/Lib/test/test_socket.py", line 5424, in test_sha256* * op.sendall(b"abc")* *OSError: [Errno 126] Required key not available* ---------------------------------------------------------------------- Ran 1 test in 0.002s FAILED (errors=1) test test_socket failed test_socket failed 1 test failed: test_socket Total duration: 42 ms Tests result: FAILURE Here is the strace: socket(AF_INET6, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 bind(3, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=20479, si_uid=1000, si_status=0, si_utime=0, si_stime=0} --- socket(AF_CAN, SOCK_RAW|SOCK_CLOEXEC, 1) = 3 socket(AF_RDS, SOCK_SEQPACKET|SOCK_CLOEXEC, 0) = -1 EAFNOSUPPORT (Address family not supported by protocol) socket(AF_ALG, SOCK_SEQPACKET|SOCK_CLOEXEC, 0) = 3 socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC, IPPROTO_IP) = 3 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_SCTP) = 3 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_SCTP) = 3 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_SCTP) = 3 socket(AF_ALG, SOCK_SEQPACKET|SOCK_CLOEXEC, 0) = 3 bind(3, {sa_family=AF_ALG, sa_data="hash\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0sha256\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 88) = 0 accept4(3, NULL, NULL, SOCK_CLOEXEC) = 4 sendto(4, "abc", 3, 0, NULL, 0) = -1 ENOKEY (Required key not available) +++ exited with 2 +++ Any ideas? -- Ryan C. Decker '08 Principle Network & Systems Eng. Siena College ITS 515 Loudon Road Loudonville, NY 12211 rdecker at siena.edu CONFIDENTIALITY NOTICE: This e-mail, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, or distribution is prohibited. If you received this e-mail and are not the intended recipient, please inform the sender by e-mail reply and destroy all copies of the original message. From skip.montanaro at gmail.com Thu Oct 19 14:18:19 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 19 Oct 2017 13:18:19 -0500 Subject: Application and package of the same name Message-ID: I'm not understanding something fundamental about absolute/relative imports. Suppose I have an application, fribble.py, and it has a corresponding package full of goodies it relies on, also named fribble. >From the fribble package, the application wants to import the sandwich function from the lunchtime module. At the top level it thus has an import like this: from fribble.lunchtime import sandwich I might have a directory structure like this: example example/fribble.py fribble __init__.py lunchtime.py If I run inside the example directory with PYTHONPATH=.. I can't find/import the fribble package, because the main application directory is prepended to sys.path. Consequently, the import machinery never gets any further down sys.path. It stumbles on the fribble application and tries to find the bits it's interested in, to no avail. This is in Python 2.7, FWIW. What am I missing? Thx, Skip From israel at ravnalaska.net Thu Oct 19 14:28:35 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Thu, 19 Oct 2017 10:28:35 -0800 Subject: Efficient counting of results In-Reply-To: References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> Message-ID: <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> > On Oct 19, 2017, at 10:02 AM, Stefan Ram wrote: > > Israel Brewster writes: >> t10 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} >> increment the appropriate bin counts using a bunch of if statements. > > I can't really completely comprehend your requirements > specification, you might have perfectly described it all and > it's just too complicated for me to comprehend, but I just > would like to add that there are several ways to implement a > "two-dimensional" matrix. You can also imagine your > dictionary like this: > > example = > { 'd10': 0, 'd15': 0, 'd20': 0, 'd215': 0, > 'w10': 0, 'w15': 0, 'w20': 0, 'w215': 0, > 'm10': 0, 'm15': 0, 'm20': 0, 'm215': 0, > 'y10': 0, 'y15': 0, 'y20': 0, 'y215': 0 } > > Then, when the categories are already in two variables, say, > ?a? (?d?, ?w?, ?m?, or ?y?) and ?b? (?10?, ?15?, ?20?, or > ?215?), you can address the appropriate bin as Oh, I probably was a bit weak on the explanation somewhere. I'm still wrapping *my* head around some of the details. That's what makes it fun :-) If it helps, my data would look something like this: [ (date, key, t1, t2), (date, key, t1, t2) . . ] Where the date and the key are what is used to determine what "on-time" is for the record, and thus which "late" bin to put it in. So if the date of the first record was today, t1 was on-time, and t2 was 5 minutes late, then I would need to increment ALL of the following (using your data structure from above): d10, w10, m10, y10, d25, w25, m25 AND y25 Since this record counts not just for the current day, but also for week-to-date, month-to-date and year-to-date. Basically, as the time categories get larger, the percentage of the total records included in that date group also gets larger. The year-to-date group will include all records, grouped by lateness, the daily group will only include todays records. Maybe that will help clear things up. Or not. :-) > > example[ a + b ]+= 1 Not quite following the logic here. Sorry. > > . (And to not have to initialized the entries to zero, > class collections.defaultdict might come in handy.) Yep, those are handy in many places. Thanks for the suggestion. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > > -- > https://mail.python.org/mailman/listinfo/python-list From p.f.moore at gmail.com Thu Oct 19 14:34:56 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 19 Oct 2017 19:34:56 +0100 Subject: Application and package of the same name In-Reply-To: References: Message-ID: On 19 October 2017 at 19:18, Skip Montanaro wrote: > I'm not understanding something fundamental about absolute/relative > imports. Suppose I have an application, fribble.py, and it has a > corresponding package full of goodies it relies on, also named fribble. > From the fribble package, the application wants to import the sandwich > function from the lunchtime module. At the top level it thus has an import > like this: > > from fribble.lunchtime import sandwich > > I might have a directory structure like this: > > example > example/fribble.py > fribble > __init__.py > lunchtime.py > > If I run inside the example directory with PYTHONPATH=.. I can't > find/import the fribble package, because the main application directory is > prepended to sys.path. Consequently, the import machinery never gets any > further down sys.path. It stumbles on the fribble application and tries to > find the bits it's interested in, to no avail. > > This is in Python 2.7, FWIW. What am I missing? My immediate reaction is "you shouldn't name your main program and your package the same". It's not a pattern I've seen commonly used. However, the approaches I've seen used (a __main__.py inside the package, so you can execute it via `python -m fribble`, or a setup.py entry point to generate a script wrapper for the application) may be more common among people focused more on library development than on application development. Paul From __peter__ at web.de Thu Oct 19 15:30:33 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 19 Oct 2017 21:30:33 +0200 Subject: Efficient counting of results References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> Message-ID: Israel Brewster wrote: > >> On Oct 19, 2017, at 10:02 AM, Stefan Ram wrote: >> >> Israel Brewster writes: >>> t10 = {'daily': 0, 'WTD': 0, 'MTD': 0, 'YTD': 0,} >>> increment the appropriate bin counts using a bunch of if statements. >> >> I can't really completely comprehend your requirements >> specification, you might have perfectly described it all and >> it's just too complicated for me to comprehend, but I just >> would like to add that there are several ways to implement a >> "two-dimensional" matrix. You can also imagine your >> dictionary like this: >> >> example = >> { 'd10': 0, 'd15': 0, 'd20': 0, 'd215': 0, >> 'w10': 0, 'w15': 0, 'w20': 0, 'w215': 0, >> 'm10': 0, 'm15': 0, 'm20': 0, 'm215': 0, >> 'y10': 0, 'y15': 0, 'y20': 0, 'y215': 0 } >> >> Then, when the categories are already in two variables, say, >> ?a? (?d?, ?w?, ?m?, or ?y?) and ?b? (?10?, ?15?, ?20?, or >> ?215?), you can address the appropriate bin as > > Oh, I probably was a bit weak on the explanation somewhere. I'm still > wrapping *my* head around some of the details. That's what makes it fun > :-) If it helps, my data would look something like this: > > [ (date, key, t1, t2), > (date, key, t1, t2) > . > . > ] > > Where the date and the key are what is used to determine what "on-time" is > for the record, and thus which "late" bin to put it in. So if the date of > the first record was today, t1 was on-time, and t2 was 5 minutes late, > then I would need to increment ALL of the following (using your data > structure from above): > > d10, w10, m10, y10, d25, w25, m25 AND y25 Start with simpler more generic operations. A def group(rows): ... function that expects rows of the form (date, key, t) can be run twice, once with summary1 = group((date, key, t1) for date, key, t1, t2 in rows) and then with t2. Then only calculate the daily sums as you can derive the weekly, monthly and yearly totals by summing over the respective days (you may also get the yearly totals by summing over the respective months, but I expect this to be an optimisation with negligable effect). > Since this record counts not just for the current day, but also for > week-to-date, month-to-date and year-to-date. Basically, as the time > categories get larger, the percentage of the total records included in > that date group also gets larger. The year-to-date group will include all > records, grouped by lateness, the daily group will only include todays > records. > > Maybe that will help clear things up. Or not. :-) >> >> example[ a + b ]+= 1 > > Not quite following the logic here. Sorry. > >> >> . (And to not have to initialized the entries to zero, >> class collections.defaultdict might come in handy.) > > Yep, those are handy in many places. Thanks for the suggestion. > > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > >> >> -- >> https://mail.python.org/mailman/listinfo/python-list > From tjol at tjol.eu Thu Oct 19 15:36:28 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 19 Oct 2017 21:36:28 +0200 Subject: Efficient counting of results In-Reply-To: <4953C7C3-F61A-4440-BFF3-3504BE9F9F6D@ravnalaska.net> References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <4953C7C3-F61A-4440-BFF3-3504BE9F9F6D@ravnalaska.net> Message-ID: <4495d130-8136-ea6c-f41c-ce15b3b2e0cb@tjol.eu> On 19/10/17 20:04, Israel Brewster wrote: >> then loop through the records, find the schedule for that record (if any, if not move on as mentioned earlier), compare t1 and t2 against the schedule, and increment the appropriate bin counts using a bunch of if statements. Functional, if ugly. But then I got to thinking: I keep hearing about all these fancy numerical analysis tools for python like pandas and numpy - could something like that help? Might there be a way to simply set up a table with "rules" for the columns and rows, and drop my records into the table, having them automatically counted into the proper bins or something? Or am I over thinking this, and the "simple", if ugly approach is best? The numerical packages can do a lot of things; in this kind of case, mostly hiding loops in C code. You'll have to be the judge of whether it's helpful. I'll just try to give you an idea of what you might do. If you had two pandas/numpy series/arrays of numpy datetimes called "scheduled" and "actual", you could get all the delays import pandas as pd delays = pd.Series(actual - scheduled) then index those by the actual time delays.index = actual and select bits of it as you please, e.g. from pandas.tseries.offsets import Week today = pd.to_datetime('today') wtd_delays = delays[today-week:] You can construct boolean mask arrays for certain conditions minute_delta = pd.DateOffset(minutes=1).delta wtd_is_between_1_and_5_min = ((wtd_delays >= 1*minute_delta) & (wtd_delays < 5*minute_delta)) and either get all the affected datapoints wtd_delays[wtd_is_between_1_and_5_min] or count them np.count_nonzero(wtd_is_between_1_and_5_min) If you have a larger table in a pandas DataFrame with more information, and you want to get those rows which fit a particular requirement, you can do that too. Something like some_table = dataframe_conjured_out_of_thin_air() delay = some_table['actual_time'] - some_table['proper_civilised_time'] mtd = today - pd.DateOffset(months=1) data_mtd_5min_or_later = some_table[mtd:][delay[mtd:] >= 5*minute_delta] Or something like that. If you do a lot of this kind of stuff (sifting through largish datasets), learning how to use pandas might be an excellent idea, but it will of course involve a fair amount of scrolling through documentation, googling, and reading stack overflow posts. Some pointers: http://pandas.pydata.org/pandas-docs/stable/ http://pandas.pydata.org/pandas-docs/stable/timeseries.html https://docs.scipy.org/doc/numpy-1.13.0/user/basics.indexing.html#boolean-or-mask-index-arrays > > I suppose I should mention: my data source is the results of a psycopg2 query, so a "record" is a tuple or dictionary (depending on how I want to set up the cursor) In this case, if you want speed, you're far better off doing most of the work in SQL rather than Python! If you want clarity, maybe what you're doing now is already good enough. Or maybe using more complex SQL would actually be clearer. Pandas sits somewhere in between, and IMHO only gives you significant added value (if your data is already in a database) if you want to do some form of statistical analysis, some other kind of more complex computation, or if you want to create plots. -- Thomas From skip.montanaro at gmail.com Thu Oct 19 16:09:45 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 19 Oct 2017 15:09:45 -0500 Subject: Application and package of the same name In-Reply-To: References: Message-ID: > My immediate reaction is "you shouldn't name your main program and > your package the same". It's not a pattern I've seen commonly used. > > However, the approaches I've seen used (a __main__.py inside the > package, so you can execute it via `python -m fribble`, or a setup.py > entry point to generate a script wrapper for the application) may be > more common among people focused more on library development than on > application development. Thanks Paul. The simplest course for me is to change the name of the application, as the package is already in use by other people, and I don't really want to embed the application in the package. Skip From __peter__ at web.de Thu Oct 19 16:36:28 2017 From: __peter__ at web.de (Peter Otten) Date: Thu, 19 Oct 2017 22:36:28 +0200 Subject: How to debug an unfired tkinter event? References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> <5b371b88-63a8-4b05-bf4f-65e92b61d4af@googlegroups.com> Message-ID: jfong at ms4.hinet.net wrote: > Peter Otten? 2017?10?19???? UTC+8??6?04?39???? >> jfong at ms4.hinet.net wrote: >> >> > Peter Otten at 2017-10-19 UTC+8 PM 3:24:30 wrote: >> >> It's not clear to me what you mean with this. Did you place the table >> >> from the recipe elsewhere inside a window that you created or did you >> >> make changes in the recipe's code? >> > >> > Thank you, Peter. I am using Python 3.4.4 under WinXP. >> > >> > When running the file directly from download, I get a table scrolling >> > vertically only. If its Table class's __init__ parameter >> > "scroll_horizontally" changed to True, it can be scrolled horizontally >> > also. But there is a problem when scrolled horizontally, the header >> > field will not align with data field anymore. I make some modifications >> > to make it does. So far so good. >> > >> > Later, I want to make the table also has a vertical header. The first >> > step I had taken was to move all 2nd top-level widgets(not much, just >> > four) to right one column further to make room for this new widget. I >> > though it's a simple work, just increase their grid column value by 1. >> > But Murphy's Law comes, the xscrollbar even don't show up when the >> > table was squeezed horizontally. >> >> Thank you for the clarification; I understand your problem much better >> now. >> >> > >> >> > The canvas has a binding: >> >> > self.canvas.bind('', self._on_canvas_configure) >> >> > >> >> > After this movement, the callback was only triggered when dragging >> >> > the root widget to resize canvas vertically, but not horizontally. >> >> > >> >> > Event seems has OS involved(it's MS Windows XP in my case). How to >> >> > debug this kind of problem, under pdb? >> >> >> >> I don't know, but if you can post a small example script that >> >> demonstrates the problem, and you are lucky, someone will see the >> >> problem. >> > >> > This is a ~5xx lines file and I think it's not fare to ask forum >> > members to look through it. So I decide to ask for help on how to debug >> > it, instead of the solution. >> > >> > I try to change the binding to >> > self.bind_all('', self._on_canvas_configure) >> > and add a line into the callback >> > print(event.widget) >> > >> > I got some info below each time when I squeeze the table: >> > . >> > .50077776 >> > .50077776.50712528 >> > .50077776.50712496 >> > .50077776.50712464 >> > .50077776.50712144 >> > .50077776.50712528.50712560.50782256 >> > .50077776.50712528.50712560.50782256.50783024 >> > .50077776.50712528.50712560.50782256 >> > .50077776.50712528.50712560.50782256.50783024 >> > >> > How to change these number(is it a widget ID?) to a meaning name? >> >> That's the name of the widget in the underlying tcl/tk. You can specify >> it in Python: >> >> >>> import tkinter as tk >> >>> print(tk.Button()) >> .139817813335904 >> >>> print(tk.Button(name="mybutton")) >> .mybutton >> >> You have to ensure that names are unique. > > Thank you. It's very helpful information. > > By the way, I just found this problem can be easily seen by not modifing > the original file too much with the following steps: > > 1. copy and get the file from the link. > 2. change the "scroll_horizontally" parameter in the "Table" class's > __init__ method to "True" > > Running it you will see it perform correctly. > > 3. Change the "column=x" at line 268, 282, 288, 292, 303 to "column=x+1" > > Now it has the problem. There may be other less obvious places that are affected by your modifications. Does changing self.grid_columnconfigure(0, weight=1) to self.grid_columnconfigure(1, weight=1) in Table.__init__() help? From pacqa100 at yahoo.com.au Thu Oct 19 17:46:53 2017 From: pacqa100 at yahoo.com.au (Peter) Date: Fri, 20 Oct 2017 08:46:53 +1100 Subject: Problem with StreamReaderWriter on 3.6.3? Message-ID: I came across this code in Google cpplint.py, a Python script for linting C++ code. I was getting funny results with Python 3.6.3, but it worked fine under 2.7.13 I've tracked the problem to an issue with StreamReaderWriter; the traceback and error never shows under 3. The _cause_ of the error is clear (xrange not in Py3), but I need the raised exception to show. I'm running Python 3.6.3 32bit on Windows 10. I also get the same results on Python 3.5.2 on Ubuntu (under WSL) I'm not super familiar with rebinding stderr with codecs, but I'm guessing they are doing it because of some Unicode issue they may have been having. I have a workaround - drop the rebinding - but it seems like there might be an error in StreamReaderWriter. Do other people see the same behaviour? Is there something I'm not seeing or understanding? Would I raise it on issue tracker? Peter ---------------------------------------------------------- import sys import codecs sys.stderr = codecs.StreamReaderWriter( ??? sys.stderr, codecs.getreader('utf8'), codecs.getwriter('utf8'), 'replace') # This should work fine in Py 2, but raise an exception in Py3 # But instead Py3 "swallows" the exception and it is never seen print(xrange(1, 10)) # Although this line doesn't show in Py 3 (as the script has silently crashed) print("This line doesn't show in Py 3") ---------------------------------------------------------- From python at mrabarnett.plus.com Thu Oct 19 19:19:17 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 20 Oct 2017 00:19:17 +0100 Subject: Problem with StreamReaderWriter on 3.6.3? In-Reply-To: References: Message-ID: <71f94741-1343-e7dc-dd9d-b766c0bd6114@mrabarnett.plus.com> On 2017-10-19 22:46, Peter via Python-list wrote: > I came across this code in Google cpplint.py, a Python script for > linting C++ code. I was getting funny results with Python 3.6.3, but it > worked fine under 2.7.13 > > I've tracked the problem to an issue with StreamReaderWriter; the > traceback and error never shows under 3. The _cause_ of the error is > clear (xrange not in Py3), but I need the raised exception to show. > > I'm running Python 3.6.3 32bit on Windows 10. I also get the same > results on Python 3.5.2 on Ubuntu (under WSL) > > I'm not super familiar with rebinding stderr with codecs, but I'm > guessing they are doing it because of some Unicode issue they may have > been having. > > I have a workaround - drop the rebinding - but it seems like there might > be an error in StreamReaderWriter. > Do other people see the same behaviour? > Is there something I'm not seeing or understanding? > Would I raise it on issue tracker? > > Peter > > ---------------------------------------------------------- > > import sys > import codecs > > sys.stderr = codecs.StreamReaderWriter( > ??? sys.stderr, codecs.getreader('utf8'), codecs.getwriter('utf8'), > 'replace') > > # This should work fine in Py 2, but raise an exception in Py3 > # But instead Py3 "swallows" the exception and it is never seen > print(xrange(1, 10)) > > # Although this line doesn't show in Py 3 (as the script has silently > crashed) > print("This line doesn't show in Py 3") > > ---------------------------------------------------------- > StreamReaderWriter is being passed an encoder which returns bytes (UTF-8), but the output stream that is being passed, to which it will be writing those butes, i.e. the original sys.stderr, expects str. I'd get the underlying byte stream of stderr using .detach(): sys.stderr = codecs.StreamReaderWriter(sys.stderr.detach(), codecs.getreader('utf8'), codecs.getwriter('utf8'), 'replace') From tjreedy at udel.edu Thu Oct 19 19:37:12 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 19 Oct 2017 19:37:12 -0400 Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: On 10/19/2017 5:07 AM, jfong at ms4.hinet.net wrote: > I got some info below each time when I squeeze the table: > . > .50077776 > .50077776.50712528 > .50077776.50712496 > .50077776.50712464 > .50077776.50712144 > .50077776.50712528.50712560.50782256 > .50077776.50712528.50712560.50782256.50783024 > .50077776.50712528.50712560.50782256 > .50077776.50712528.50712560.50782256.50783024 > > How to change these number(is it a widget ID?) to a meaning name? The number strings as names are the defaults that came from tkinter, not tcl/tk. In 3.6, the default names were changed to be versions of the class name. >>> import tkinter as tk >>> r = tk.Tk() >>> b = tk.Button(r) >>> b >>> b2 = tk.Button(r) >>> b2 -- Terry Jan Reedy From rosuav at gmail.com Thu Oct 19 19:44:56 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Oct 2017 10:44:56 +1100 Subject: Application and package of the same name In-Reply-To: References: Message-ID: On Fri, Oct 20, 2017 at 7:09 AM, Skip Montanaro wrote: >> My immediate reaction is "you shouldn't name your main program and >> your package the same". It's not a pattern I've seen commonly used. >> >> However, the approaches I've seen used (a __main__.py inside the >> package, so you can execute it via `python -m fribble`, or a setup.py >> entry point to generate a script wrapper for the application) may be >> more common among people focused more on library development than on >> application development. > > Thanks Paul. The simplest course for me is to change the name of the > application, > as the package is already in use by other people, and I don't really > want to embed the > application in the package. > Have you toyed with "from __future__ import absolute_import" ? Not sure if it'd help or not, but worth a try. ChrisA From skip.montanaro at gmail.com Thu Oct 19 21:08:59 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 19 Oct 2017 20:08:59 -0500 Subject: Application and package of the same name In-Reply-To: References: Message-ID: Have you toyed with "from __future__ import absolute_import" ? Not sure if it'd help or not, but worth a try. Yeah, I did, but it didn't change anything as far as I could tell. S From steve+python at pearwood.info Thu Oct 19 21:18:53 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 20 Oct 2017 12:18:53 +1100 Subject: Efficient counting of results References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> Message-ID: <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> On Fri, 20 Oct 2017 05:28 am, Israel Brewster wrote: > If it helps, my data would look something like this: > > [ (date, key, t1, t2), > (date, key, t1, t2) > . > . > ] > > Where the date and the key are what is used to determine what "on-time" is > for the record, and thus which "late" bin to put it in. It might help if you discuss it in less generic and more concrete terms. E.g. are these invoices? The date field could be the invoice date, the "key" field (aren't they all keys?) might be the due date. What t1 and t2 are, I have no idea. Your code there suggests that they are fields in your data records, but the contents of the fields, who knows? > So if the date of > the first record was today, t1 was on-time, and t2 was 5 minutes late, then > I would need to increment ALL of the following (using your data structure > from above): > > d10, w10, m10, y10, d25, w25, m25 AND y25 Try using descriptive variable names rather than these cryptic codes. I don't understand what is *actually* being computed here -- you say that t1 is "on time" and t2 is "5 minutes late", but that's a contradiction: how can a single record be both on time and 5 minutes late? It also contradicts your statement that it is *date* and *key* that determines which late bin to use. Rather, it seems that date and key are irrelevant and can be ignored, it is only t1 and t2 which determine which late bins to update. Another question: you're talking about *dates*, which implies a resolution of 1 day, but then you talk about records being "five minutes late" which implies a resolution of at least five minutes and probably one minute, if not seconds or milliseconds. Which is it? My guess is that you're probably talking about *timestamps* (datetimes) rather than *dates*. > Since this record counts not just for the current day, but also for > week-to-date, month-to-date and year-to-date. Basically, as the time > categories get larger, the percentage of the total records included in that > date group also gets larger. The year-to-date group will include all > records, grouped by lateness, the daily group will only include todays > records. With the proviso that I have no confidence at all that I understand your specification or requirements, and that I'm *guessing* a spec that makes sense to me, I'll suggest a couple of approaches. Hypotheses: you have data with a timestamp recording when each record becomes "active" in some sense (e.g. when an invoice becomes due for payment), and second timestamp representing the date/time "now" (at the start of the computation?). You want to process all your records, and decide "as of now, how late is each record", and then report *cumulative* subtotals for a number of arbitrary groups: not late yet, five minutes late, one day late, one year late, etc. Suggestion: Start with just the "activation time" and "now", and calculate the difference. If they are both given in seconds, you can just subtract: lateness = now - activation_time to determine how late that record is. If they are Datetime objects, use a Timedelta object. That *single* computed field, the lateness, is enough to determine which subtotals need to be incremented. Start by dividing all of time into named buckets, in numeric order: lateness <= 0: on_time 0 < lateness <= five minutes: five_minutes_late five minutes < lateness <= sixty minutes: one_hour_late sixty minutes < lateness <= 24 hours: one_day_late 24 hours < lateness <= seven days: one_week_late etc. Notice that the buckets don't overlap. (The buckets refer to the *maximum* lateness, which is opposite of standard book-keeping practice which uses the minimum. In accounting and book-keeping, if an invoice is 45 days past the due date, it would fall into the single bucket "30 days past due". When it reaches 60 days past the due date, it moves into the "60 days past due" bucket, where it will stay until it gets to 90 days overdue.) Convert the times to a common unit (seconds), and build a list for each bucket, using the *upper bound* as key: buckets = [(0, on_time), (5*60, five_minutes_late), # i.e. *up to* five minutes late (60*60, one_hour_late), # i.e. *up to* one hour late (24*60*60, one_day_late), ... (float('inf'), unbelievably_late)] The buckets might be lists, to append the record; or integer counts, which you add 1 to, or subtotals. I'll assume that they are lists, as that is the most flexible. Now process your records: for record in records: lateness = now - record.activation_date for end, bucket in buckets: if lateness <= end: bucket.append(record) else: break And you're done! If you want the *number of records* in a particular bucket, you say: len(bucket) If you want the total record amount, you say: sum(record.total for record in bucket) (assuming your records also have a "total" field, if they're invoices say). I hope that's even vaguely helpful. > Maybe that will help clear things up. Or not. :-) Not even a tiny bit :-( -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Thu Oct 19 22:32:12 2017 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 20 Oct 2017 13:32:12 +1100 Subject: Efficient counting of results In-Reply-To: <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> Message-ID: On Fri, Oct 20, 2017 at 12:18 PM, Steve D'Aprano wrote: > On Fri, 20 Oct 2017 05:28 am, Israel Brewster wrote: >> So if the date of >> the first record was today, t1 was on-time, and t2 was 5 minutes late, then >> I would need to increment ALL of the following (using your data structure >> from above): >> >> d10, w10, m10, y10, d25, w25, m25 AND y25 > > Try using descriptive variable names rather than these cryptic codes. > > I don't understand what is *actually* being computed here -- you say that t1 > is "on time" and t2 is "5 minutes late", but that's a contradiction: how can > a single record be both on time and 5 minutes late? t1 and t2 are independent actions/timepoints, AIUI. So t1 could be the time the order was put into the oven, and t2 is the time it got delivered to the person's door. I'm assuming, here, that this database records pizzas, because why not. An order could have been compiled and put into the oven on time, but still delivered late; or it could be ovened slightly late, but thanks to the new high-speed delivery drones, it was actually at the customer's doorstep on time. ChrisA From python at mrabarnett.plus.com Thu Oct 19 23:25:43 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 20 Oct 2017 04:25:43 +0100 Subject: Efficient counting of results In-Reply-To: References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> Message-ID: <773ad296-f32f-8b09-4c54-17be05e44a0e@mrabarnett.plus.com> On 2017-10-20 03:32, Chris Angelico wrote: > On Fri, Oct 20, 2017 at 12:18 PM, Steve D'Aprano > wrote: >> On Fri, 20 Oct 2017 05:28 am, Israel Brewster wrote: >>> So if the date of >>> the first record was today, t1 was on-time, and t2 was 5 minutes late, then >>> I would need to increment ALL of the following (using your data structure >>> from above): >>> >>> d10, w10, m10, y10, d25, w25, m25 AND y25 >> >> Try using descriptive variable names rather than these cryptic codes. >> >> I don't understand what is *actually* being computed here -- you say that t1 >> is "on time" and t2 is "5 minutes late", but that's a contradiction: how can >> a single record be both on time and 5 minutes late? > > t1 and t2 are independent actions/timepoints, AIUI. So t1 could be the > time the order was put into the oven, and t2 is the time it got > delivered to the person's door. I'm assuming, here, that this database > records pizzas, because why not. An order could have been compiled and > put into the oven on time, but still delivered late; or it could be > ovened slightly late, but thanks to the new high-speed delivery > drones, it was actually at the customer's doorstep on time. > Also, AIUI: If it was on-time today, then it was also on-time this week, this month, and this year. If it was on-time yesterday, then it was also on-time this week, this month, and this year, but not today. If it was on-time eight days ago, then it was also on-time this month, and this year, but not this week. Excepting that he's dealing with calendar weeks, calendar months, etc, so yesterday might not be counted as being in this week (or month, or even year!). I think. From jfong at ms4.hinet.net Thu Oct 19 23:26:42 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 19 Oct 2017 20:26:42 -0700 (PDT) Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> <5b371b88-63a8-4b05-bf4f-65e92b61d4af@googlegroups.com> Message-ID: <1870e014-d7c0-4936-9738-f2da626a4d9b@googlegroups.com> Peter Otten? 2017?10?20???? UTC+8??4?37?10???? > jfong at ms4.hinet.net wrote: > > > Peter Otten? 2017?10?19???? UTC+8??6?04?39???? > >> jfong at ms4.hinet.net wrote: > >> > >> > Peter Otten at 2017-10-19 UTC+8 PM 3:24:30 wrote: > >> >> It's not clear to me what you mean with this. Did you place the table > >> >> from the recipe elsewhere inside a window that you created or did you > >> >> make changes in the recipe's code? > >> > > >> > Thank you, Peter. I am using Python 3.4.4 under WinXP. > >> > > >> > When running the file directly from download, I get a table scrolling > >> > vertically only. If its Table class's __init__ parameter > >> > "scroll_horizontally" changed to True, it can be scrolled horizontally > >> > also. But there is a problem when scrolled horizontally, the header > >> > field will not align with data field anymore. I make some modifications > >> > to make it does. So far so good. > >> > > >> > Later, I want to make the table also has a vertical header. The first > >> > step I had taken was to move all 2nd top-level widgets(not much, just > >> > four) to right one column further to make room for this new widget. I > >> > though it's a simple work, just increase their grid column value by 1. > >> > But Murphy's Law comes, the xscrollbar even don't show up when the > >> > table was squeezed horizontally. > >> > >> Thank you for the clarification; I understand your problem much better > >> now. > >> > >> > > >> >> > The canvas has a binding: > >> >> > self.canvas.bind('', self._on_canvas_configure) > >> >> > > >> >> > After this movement, the callback was only triggered when dragging > >> >> > the root widget to resize canvas vertically, but not horizontally. > >> >> > > >> >> > Event seems has OS involved(it's MS Windows XP in my case). How to > >> >> > debug this kind of problem, under pdb? > >> >> > >> >> I don't know, but if you can post a small example script that > >> >> demonstrates the problem, and you are lucky, someone will see the > >> >> problem. > >> > > >> > This is a ~5xx lines file and I think it's not fare to ask forum > >> > members to look through it. So I decide to ask for help on how to debug > >> > it, instead of the solution. > >> > > >> > I try to change the binding to > >> > self.bind_all('', self._on_canvas_configure) > >> > and add a line into the callback > >> > print(event.widget) > >> > > >> > I got some info below each time when I squeeze the table: > >> > . > >> > .50077776 > >> > .50077776.50712528 > >> > .50077776.50712496 > >> > .50077776.50712464 > >> > .50077776.50712144 > >> > .50077776.50712528.50712560.50782256 > >> > .50077776.50712528.50712560.50782256.50783024 > >> > .50077776.50712528.50712560.50782256 > >> > .50077776.50712528.50712560.50782256.50783024 > >> > > >> > How to change these number(is it a widget ID?) to a meaning name? > >> > >> That's the name of the widget in the underlying tcl/tk. You can specify > >> it in Python: > >> > >> >>> import tkinter as tk > >> >>> print(tk.Button()) > >> .139817813335904 > >> >>> print(tk.Button(name="mybutton")) > >> .mybutton > >> > >> You have to ensure that names are unique. > > > > Thank you. It's very helpful information. > > > > By the way, I just found this problem can be easily seen by not modifing > > the original file too much with the following steps: > > > > 1. copy and get the file from the link. > > 2. change the "scroll_horizontally" parameter in the "Table" class's > > __init__ method to "True" > > > > Running it you will see it perform correctly. > > > > 3. Change the "column=x" at line 268, 282, 288, 292, 303 to "column=x+1" > > > > Now it has the problem. > > There may be other less obvious places that are affected by your > modifications. Does changing > > self.grid_columnconfigure(0, weight=1) > > to > > self.grid_columnconfigure(1, weight=1) > > in Table.__init__() help? There is description about those numbers in the "tkinter 8.5 reference manual" section 5.11 Window names. It can be read by w.winfo_name(). After I give every widget a name(using the same Python instance name), the utput looks better. Below is the output of the failed one when squeezed horizontally: .table .table.scrolling_area .table.yscrollbar .table.xscrollbar .table.canvasH .table.scrolling_area.canvas.innerframe .table.scrolling_area.canvas.innerframe._body .table.scrolling_area.canvas.innerframe .table.scrolling_area.canvas.innerframe._body Compared to the output of a good one: .table .table.scrolling_area .table.yscrollbar .table.xscrollbar .table.canvasH .table.scrolling_area.canvas It shows, when the size changed, the failed one fires onto the wrong widgets(the innerframe and innerframe._body), not the canvas. The canvas widget didn't absorb the space changing!! That's prove your intuition is correct. The problem is at the column's weight setting. After does the changes according to your suggestion, the problem is solved. La la! Thank you again, Peter! From jfong at ms4.hinet.net Thu Oct 19 23:28:25 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Thu, 19 Oct 2017 20:28:25 -0700 (PDT) Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: Terry Reedy? 2017?10?20???? UTC+8??7?37?59???? > On 10/19/2017 5:07 AM, jfong at ms4.hinet.net wrote: > > > I got some info below each time when I squeeze the table: > > . > > .50077776 > > .50077776.50712528 > > .50077776.50712496 > > .50077776.50712464 > > .50077776.50712144 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > > > How to change these number(is it a widget ID?) to a meaning name? > > The number strings as names are the defaults that came from tkinter, not > tcl/tk. In 3.6, the default names were changed to be versions of the > class name. > > >>> import tkinter as tk > >>> r = tk.Tk() > >>> b = tk.Button(r) > >>> b > > >>> b2 = tk.Button(r) > >>> b2 > > > -- > Terry Jan Reedy It's very nice to have this improvment. What it will be to the previous number output format, say, .50077776.50712528? Will it become something like this: then, that's really great:-) --Jach From tjreedy at udel.edu Fri Oct 20 00:59:32 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 20 Oct 2017 00:59:32 -0400 Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: On 10/19/2017 11:28 PM, jfong at ms4.hinet.net wrote: > Terry Reedy? 2017?10?20???? UTC+8??7?37?59???? >> On 10/19/2017 5:07 AM, jfong at ms4.hinet.net wrote: >> >>> I got some info below each time when I squeeze the table: >>> . >>> .50077776 >>> .50077776.50712528 >>> .50077776.50712496 >>> .50077776.50712464 >>> .50077776.50712144 >>> .50077776.50712528.50712560.50782256 >>> .50077776.50712528.50712560.50782256.50783024 >>> .50077776.50712528.50712560.50782256 >>> .50077776.50712528.50712560.50782256.50783024 >>> >>> How to change these number(is it a widget ID?) to a meaning name? >> >> The number strings as names are the defaults that came from tkinter, not >> tcl/tk. In 3.6, the default names were changed to be versions of the >> class name. >> >> >>> import tkinter as tk >> >>> r = tk.Tk() >> >>> b = tk.Button(r) >> >>> b >> >> >>> b2 = tk.Button(r) >> >>> b2 >> >> >> -- >> Terry Jan Reedy > > It's very nice to have this improvment. > > What it will be to the previous number output format, say, .50077776.50712528? Will it become something like this: > > I anticipate > then, that's really great:-) -- Terry Jan Reedy From nomail at com.invalid Fri Oct 20 04:30:05 2017 From: nomail at com.invalid (ast) Date: Fri, 20 Oct 2017 10:30:05 +0200 Subject: integer copy Message-ID: <59e9b419$0$3602$426a74cc@news.free.fr> Hello, I tried the following: import copy a = 5 b = copy.copy(a) a is b True I was expecting False I am aware that it is useless to copy an integer (or any immutable type). I know that for small integers, there is always a single integer object in memory, and that for larger one's there may have many. a = 7 b = 7 a is b True a = 56543 b = 56543 a is b False But it seems that Python forbids to have two different small integer objects with same value even if you request it with: a = 5 b = copy.copy(a) any comments ? From robin at reportlab.com Fri Oct 20 04:32:27 2017 From: robin at reportlab.com (Robin Becker) Date: Fri, 20 Oct 2017 09:32:27 +0100 Subject: efficient way to get a sufficient set of identifying attributes In-Reply-To: References: <9a789760-515f-659f-6a64-9c1ecd37a33c@chamonix.reportlab.co.uk> Message-ID: On 19/10/2017 17:50, Stefan Ram wrote: > Robin Becker writes: >> ....... >> this sort of makes sense for single attributes, but ignores the possibility of >> combining the attributes to make the checks more discerning. > > What I wrote also applies to compound attributes > (sets of base attributes). > > When there are n base attributes, one can form 2^n-1 > compound attributes from them, or 2^n-1-n proper compound > attributes. Therefore, a combinatoric explosion might impede > the brute-force approach. A heuristics might start to > explore combinations of keys with the best s/l ratio first > and/or use preferences for certain fields set by a human. > all good > In database design, the keys are usually chosen by a > human database designer using world knowledge. It sounds > as if you want to have the computer make such a choice > using only the information in the table as knowledge. I think I am tending towards the chosen by real world knowledge approach :( > Your "identifying attributes" are called "super keys" > in database science. You probably want minimal > identifying attribute sets (without unneeded attributes), > which are called "candidate keys". > thanks for this and the reference below. > So, now you can find and read literature, such as: > > Journal of Al-Nahrain University > Vol.13 (2), June, 2010, pp.247-255 > Science > 247 > Automatic Discovery Of Candidate In The Relational > Databases Keys By Using Attributes Sets Closure > Yasmeen F. Al-ward > Department of Computer Science, College of Science, > Al-Nahrain University. > > (The title was copied by me as found, the contents is > in the web and makes more sense than the title.) > -- Robin Becker From tomuxiong at gmx.com Fri Oct 20 04:33:45 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Fri, 20 Oct 2017 10:33:45 +0200 Subject: multiprocessing shows no benefit In-Reply-To: <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> Message-ID: <31b2ee18-0824-e3d7-1ce5-4d42a9c8ae16@gmx.com> Correct me if I'm wrong, but at a high level you appear to basically just have a mapping of strings to values and you are then shifting all of those values by a fixed constant (in this case, `z = 5`). Why are you using a dict at all? It would be better to use something like a numpy array or a series from pandas. E.g. something like this without multiprocessing: ----------------------------------------- import pandas as pd from timeit import default_timer as timer s = pd.Series( xrange(100000), index=[str(val) for val in xrange(100000)]) z = 5 start = timer() x = s - 5 duration = float(timer() -start) print duration, len(x), len(x) / duration ----------------------------------------- Then if you wanted to multiprocess it, you could basically just split the series into num_cpu pieces and then concatenate results afterwards. Though I do agree with others here that the operation itself is so simple that IPC might be a drag no matter what. Cheers, Thomas From nomail at com.invalid Fri Oct 20 04:58:03 2017 From: nomail at com.invalid (ast) Date: Fri, 20 Oct 2017 10:58:03 +0200 Subject: integer copy In-Reply-To: <59e9b419$0$3602$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> Message-ID: <59e9baa8$0$10188$426a74cc@news.free.fr> "ast" a ?crit dans le message de news:59e9b419$0$3602$426a74cc at news.free.fr... Neither works for large integers which is even more disturbing a = 6555443 b = copy.copy(a) a is b True From stephen_tucker at sil.org Fri Oct 20 05:09:28 2017 From: stephen_tucker at sil.org (Stephen Tucker) Date: Fri, 20 Oct 2017 10:09:28 +0100 Subject: integer copy In-Reply-To: <59e9baa8$0$10188$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: ast, For what it's worth, After a = 5 b = 5 afloat = float(a) bfloat = float(b) afloat is bfloat returns False. Stephen Tucker. On Fri, Oct 20, 2017 at 9:58 AM, ast wrote: > > "ast" a ?crit dans le message de > news:59e9b419$0$3602$426a74cc at news.free.fr... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > > b = copy.copy(a) > a is b > > True > -- > https://mail.python.org/mailman/listinfo/python-list > From tjol at tjol.eu Fri Oct 20 05:09:53 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 20 Oct 2017 11:09:53 +0200 Subject: integer copy In-Reply-To: <59e9baa8$0$10188$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: On 2017-10-20 10:58, ast wrote: > > "ast" a ?crit dans le message de > news:59e9b419$0$3602$426a74cc at news.free.fr... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > b = copy.copy(a) > a is b > > True Why is this disturbing? As you said, it'd be completely pointless. As to what's going on: copy.copy does not make any attempt to copy immutable types. That's all there is to it. Read the source if you want to know how this is done. https://github.com/python/cpython/blob/master/Lib/copy.py#L111 -- Thomas Jollans From tjol at tjol.eu Fri Oct 20 05:09:53 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 20 Oct 2017 11:09:53 +0200 Subject: integer copy In-Reply-To: <59e9baa8$0$10188$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: On 2017-10-20 10:58, ast wrote: > > "ast" a ?crit dans le message de > news:59e9b419$0$3602$426a74cc at news.free.fr... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > b = copy.copy(a) > a is b > > True Why is this disturbing? As you said, it'd be completely pointless. As to what's going on: copy.copy does not make any attempt to copy immutable types. That's all there is to it. Read the source if you want to know how this is done. https://github.com/python/cpython/blob/master/Lib/copy.py#L111 -- Thomas Jollans From tjol at tjol.eu Fri Oct 20 05:09:53 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 20 Oct 2017 11:09:53 +0200 Subject: integer copy In-Reply-To: <59e9baa8$0$10188$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: On 2017-10-20 10:58, ast wrote: > > "ast" a ?crit dans le message de > news:59e9b419$0$3602$426a74cc at news.free.fr... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > b = copy.copy(a) > a is b > > True Why is this disturbing? As you said, it'd be completely pointless. As to what's going on: copy.copy does not make any attempt to copy immutable types. That's all there is to it. Read the source if you want to know how this is done. https://github.com/python/cpython/blob/master/Lib/copy.py#L111 -- Thomas Jollans From alain at universite-de-strasbourg.fr.invalid Fri Oct 20 05:19:52 2017 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Fri, 20 Oct 2017 11:19:52 +0200 Subject: integer copy References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: <877evqb2h3.fsf@universite-de-strasbourg.fr.invalid> "ast" writes: > "ast" a ?crit dans le message de > news:59e9b419$0$3602$426a74cc at news.free.fr... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > b = copy.copy(a) > a is b > > True In copy.py: | [...] | def _copy_immutable(x): | return x | for t in (type(None), int, long, float, bool, str, tuple, | frozenset, type, xrange, types.ClassType, | types.BuiltinFunctionType, type(Ellipsis), | types.FunctionType, weakref.ref): | d[t] = _copy_immutable | [...] and d[t] is what decides how to copy, via _copy_dispatch. So none of the types listed above will be actually copied. It should be documented more precisely. You should file a documentation-improvement request. Also, the doc says: | This version does not copy types like module, class, function, method, | nor stack trace, stack frame, nor file, socket, window, nor array, nor | any similar types. But who knows what "similar types" are... -- Alain. From tomuxiong at gmx.com Fri Oct 20 05:24:56 2017 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Fri, 20 Oct 2017 11:24:56 +0200 Subject: integer copy In-Reply-To: <59e9b419$0$3602$426a74cc@news.free.fr> References: <59e9b419$0$3602$426a74cc@news.free.fr> Message-ID: <5e3a4855-39a0-ac40-a9f3-d0d7eb973ba4@gmx.com> On 10/20/2017 10:30 AM, ast wrote: > I am aware that it is useless to copy an integer > (or any immutable type). > > ... > > any comments ? > > Why is this a problem for you? Cheers, Thomas From nomail at com.invalid Fri Oct 20 05:28:54 2017 From: nomail at com.invalid (ast) Date: Fri, 20 Oct 2017 11:28:54 +0200 Subject: integer copy In-Reply-To: References: <59e9b419$0$3602$426a74cc@news.free.fr> <5e3a4855-39a0-ac40-a9f3-d0d7eb973ba4@gmx.com> Message-ID: <59e9c1e0$0$9413$426a74cc@news.free.fr> "Thomas Nyberg" a ?crit dans le message de news:mailman.378.1508491267.12137.python-list at python.org... > On 10/20/2017 10:30 AM, ast wrote: >> I am aware that it is useless to copy an integer >> (or any immutable type). >> >> ... >> >> any comments ? >> >> > Why is this a problem for you? > > Cheers, > Thomas It is not. It was a test. From pacqa100 at yahoo.com.au Fri Oct 20 06:46:46 2017 From: pacqa100 at yahoo.com.au (Peter) Date: Fri, 20 Oct 2017 21:46:46 +1100 Subject: Problem with StreamReaderWriter on 3.6.3? SOLVED In-Reply-To: <71f94741-1343-e7dc-dd9d-b766c0bd6114@mrabarnett.plus.com> References: <71f94741-1343-e7dc-dd9d-b766c0bd6114@mrabarnett.plus.com> Message-ID: Thanks MRAB, your solution works a treat. I'm replying to the list so others can know that this solution works. Note that sys.stderr.detach() is only available in >= 3.1, so one might need to do some version checking to get it to work properly in both versions. It also can mess up with the buffering and therefore the order of the output of stderr vs stdout. Thanks again. Peter On 20/10/2017 10:19 AM, MRAB wrote: > On 2017-10-19 22:46, Peter via Python-list wrote: >> I came across this code in Google cpplint.py, a Python script for >> linting C++ code. I was getting funny results with Python 3.6.3, but it >> worked fine under 2.7.13 >> >> I've tracked the problem to an issue with StreamReaderWriter; the >> traceback and error never shows under 3. The _cause_ of the error is >> clear (xrange not in Py3), but I need the raised exception to show. >> >> I'm running Python 3.6.3 32bit on Windows 10. I also get the same >> results on Python 3.5.2 on Ubuntu (under WSL) >> >> I'm not super familiar with rebinding stderr with codecs, but I'm >> guessing they are doing it because of some Unicode issue they may have >> been having. >> >> I have a workaround - drop the rebinding - but it seems like there might >> be an error in StreamReaderWriter. >> Do other people see the same behaviour? >> Is there something I'm not seeing or understanding? >> Would I raise it on issue tracker? >> >> Peter >> >> ---------------------------------------------------------- >> >> import sys >> import codecs >> >> sys.stderr = codecs.StreamReaderWriter( >> ? ??? sys.stderr, codecs.getreader('utf8'), codecs.getwriter('utf8'), >> 'replace') >> >> # This should work fine in Py 2, but raise an exception in Py3 >> # But instead Py3 "swallows" the exception and it is never seen >> print(xrange(1, 10)) >> >> # Although this line doesn't show in Py 3 (as the script has silently >> crashed) >> print("This line doesn't show in Py 3") >> >> ---------------------------------------------------------- >> > StreamReaderWriter is being passed an encoder which returns bytes > (UTF-8), but the output stream that is being passed, to which it will > be writing those butes, i.e. the original sys.stderr, expects str. > > I'd get the underlying byte stream of stderr using .detach(): > > sys.stderr = codecs.StreamReaderWriter(sys.stderr.detach(), > codecs.getreader('utf8'), codecs.getwriter('utf8'), 'replace') > From bc at freeuk.com Fri Oct 20 07:17:14 2017 From: bc at freeuk.com (bartc) Date: Fri, 20 Oct 2017 12:17:14 +0100 Subject: integer copy In-Reply-To: References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: On 20/10/2017 10:09, Thomas Jollans wrote: > Read the source if you want to know how this is done. > https://github.com/python/cpython/blob/master/Lib/copy.py#L111 Good, informative comment block at the top of the type that you don't see often. Usually they concern themselves with licensing or with apportioning credits. Still, copying looks pretty complicated... -- bartc From tjol at tjol.eu Fri Oct 20 08:33:00 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 20 Oct 2017 14:33:00 +0200 Subject: integer copy In-Reply-To: References: <59e9b419$0$3602$426a74cc@news.free.fr> <59e9baa8$0$10188$426a74cc@news.free.fr> Message-ID: <88718f2f-ae43-b468-80f8-d2ad89d76a23@tjol.eu> On 2017-10-20 13:17, bartc wrote: > On 20/10/2017 10:09, Thomas Jollans wrote: > >> Read the source if you want to know how this is done. >> https://github.com/python/cpython/blob/master/Lib/copy.py#L111 > > Good, informative comment block at the top of the type that you don't > see often. Usually they concern themselves with licensing or with > apportioning credits. > > Still, copying looks pretty complicated... > It is usual for Python modules, especially in the stdlib, to start with an informative docstring. But you're right, this one is more detailed and informative than most. -- Thomas Jollans From israel at ravnalaska.net Fri Oct 20 13:05:15 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Fri, 20 Oct 2017 09:05:15 -0800 Subject: Efficient counting of results In-Reply-To: <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> Message-ID: On Oct 19, 2017, at 5:18 PM, Steve D'Aprano wrote: > What t1 and t2 are, I have no idea. Your code there suggests that they are > fields in your data records, but the contents of the fields, who knows? t1 and t2 are *independent* timestamp fields. My apologies - I made the obviously false assumption that it was clear they were timestamps, or at least times based on the fact I was calculating "minutes late" based on them. > > >> d10, w10, m10, y10, d25, w25, m25 AND y25 > > Try using descriptive variable names rather than these cryptic codes. I did. In my original post I showed the table with names like "t1 1-5min". Granted, that's for illustration purposes, not actual code, but still, more descriptive. These codes were just to keep consistent with the alternative data format suggested :-) > > I don't understand what is *actually* being computed here -- you say that t1 > is "on time" and t2 is "5 minutes late", but that's a contradiction: how can > a single record be both on time and 5 minutes late? Easily: because the record contains two DIFFERENT times. Since you want more concrete, we're talking departure and arrival times here. Quite easy to depart on-time, but arrive late, or depart late but arrive on-time. > It also contradicts your statement that it is *date* and *key* that determines > which late bin to use. I never made such a statement. I said they are used to determine "WHAT on-time IS for the record", not WHETHER the record is on-time or not, and certainly not which late bin to use. To put it a different way, those are the key to a lookup table that tells me what T1 and T2 are *supposed* to be in order for *each one* to be on time. So, for example, to completely make up some data (since it doesn't matter in the slightest), date could be 10/5/17 with a key of 42 (Let's say that is a driver ID to keep things concrete for you), and using those values tells me (via the lookup table) that on 10/5/17, 42 should have a T1 of 10:15 and a T2 of 11:30. As we said, those would be departure and arrival times, so what we're saying is that on 10/5, driver #42 was *scheduled* to depart at 10:15 and arrive at their destination at 11:30. So if T1 was *actually* 10:14, and T2 was, say 11:35, then I could say that T1 was on-time (actually, a minute early, but that doesn't matter), while T2 was 5 minutes late. Maybe traffic was horrible, or he had a flat. However, if the date changed to 9/1/17 (with the key still being 42), there could be a completely different schedule, with completely different "late" results, even if the *actual* values of t1 and t2 don't change. Maybe he was supposed to make the run early on that day, say 10:00-11:15, but forgot and left at the same time as he was used to, thereby making him 14 minutes late departing. and really late arriving, or something. > Rather, it seems that date and key are irrelevant and > can be ignored, it is only t1 and t2 which determine which late bins to > update. Except that then we have no way to know what t1 and t2 *should* be. You apparently made the assumption that t1 and t2 should always be some fixed value. In fact, what t1 and t2 should be varies based on date and key (see the driver example above, or Chris Angelico's pizza example also works well). For any given date, there are dozens of different keys with different expected values of t1 and t2 (in the pizza example Chris gave the key might be order number), and for any given key, the expected value of t1 and t2 could vary based on what date it is (say we restart order numbers from 1 each day to make it easy to know how many orders we've done that day, or, of course, same driver different day, depending on which example you prefer). > > Another question: you're talking about *dates*, which implies a resolution of > 1 day, but then you talk about records being "five minutes late" which > implies a resolution of at least five minutes and probably one minute, if not > seconds or milliseconds. Which is it? My guess is that you're probably > talking about *timestamps* (datetimes) rather than *dates*. As stated, the data has two timestamp fields T1 and T2. So yes, the resolution of the data is "one minute" (we ignore sub-minute timings). However (and this addresses your understanding below as well), I am trying to get data for the date, week-to-date, month-to-date, and year-to-date. So there is four different "date" resolution bins in addition to the "minute" resolution bins. Perhaps a better approach to explaining is to pose the question the report is trying to answer: For the given date, how many departures were on time? How many were 1-5 minutes late? 6-15 minutes late? What about this week: how many on-time, 1-5 minutes late, etc? What about this entire month (including the given date)? What about this year (again, including the given date and month)? How about arrivals - same questions. As you can hopefully see now, if a departure happened this week, it probably also happened this month (although that is not necessarily the case, since weeks can cross month boundaries), and if it happened this date or this month, it *definitely* happened this year. As such, a given departure *likely* will be counted in multiple date "groups", if you will. The end result should be a table like the one I posted in the original question: time frame covered on the horizontal axis (YTD, MTD etc.), and "late" groups for T1 and T2 on the vertical. > You want to process all your records, and decide "as of > now, how late is each record", and then report *cumulative* subtotals for a > number of arbitrary groups: not late yet, five minutes late, one day late, > one year late, etc. Just to clarify, as stated, the late groups are not-late, 1-5 minutes late, and 6-15 minutes late. Also as stated in the original message, anything over 15 minutes late is dealt with separately, and therefore ignored for the purposes of this report. > > Suggestion: > > Start with just the "activation time" and "now", and calculate the difference. > If they are both given in seconds, you can just subtract: > > lateness = now - activation_time > > to determine how late that record is. If they are Datetime objects, use a > Timedelta object. > > That *single* computed field, the lateness, is enough to determine which > subtotals need to be incremented. Well, *two* computed fields, one for T1 and one for T2, which are counted separately. > Start by dividing all of time into named > buckets, in numeric order: > > ... > for record in records: > lateness = now - record.activation_date > for end, bucket in buckets: > if lateness <= end: > bucket.append(record) > else: > break > > And you're done! > > If you want the *number of records* in a particular bucket, you say: > > len(bucket) > > If you want the total record amount, you say: > > sum(record.total for record in bucket) > > > (assuming your records also have a "total" field, if they're invoices say). > > > I hope that's even vaguely helpful. In a sense, in that it supports my initial approach. As Stefan Ram pointed out, there is nothing wrong with the solution I have: simply using if statements around the calculated lateness of t1 and t2 to increment the appropriate counters. I was just thinking there might be tools to make the job easier/cleaner/more efficient. From the responses I have gotten, it would seem that that is likely not the case, so I'll just say "thank you all for your time", and let the matter rest. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > > > > >> Maybe that will help clear things up. Or not. :-) > > > Not even a tiny bit :-( > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list From israel at ravnalaska.net Fri Oct 20 13:19:16 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Fri, 20 Oct 2017 09:19:16 -0800 Subject: Save non-pickleable variable? Message-ID: tldr: I have an object that can't be picked. Is there any way to do a "raw" dump of the binary data to a file, and re-load it later? Details: I am using a java (I know, I know - this is a python list. I'm not asking about the java - honest!) library (Jasper Reports) that I access from python using py4j (www.py4j.org ). At one point in my code I call a java function which, after churning on some data in a database, returns an object (a jasper report object populated with the final report data) that I can use (via another java call) to display the results in a variety of formats (HTML, PDF, XLS, etc). At the time I get the object back, I use it to display the results in HTML format for quick display, but the user may or may not also want to get a PDF copy in the near future. Since it can take some time to generate this object, and also since the data may change between when I do the HTML display and when the user requests a PDF (if they do at all), I would like to save this object for potential future re-use. Because it might be large, and there is actually a fairly good chance the user won't need it again, I'd like to save it in a temp file (tat would be deleted when the user logs out) rather than in memory. Unfortunately, since this is an object created by and returned from a java function, not a native python object, it is not able to be pickled (as the suggestion typically is), at least to my knowledge. Given that, is there any way I can write out the "raw" binary data to a file, and read it back in later? Or some other way to be able to save this object? It is theoretically possible that I could do it on the java side, i.e. the library may have some way of writing out the file, but obviously I wouldn't expect anyone here to know anything about that - I'm just asking about the python side :-) ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From python at mrabarnett.plus.com Fri Oct 20 14:28:01 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 20 Oct 2017 19:28:01 +0100 Subject: Save non-pickleable variable? In-Reply-To: References: Message-ID: <91ca71ca-9251-994e-7243-e64db55ddc71@mrabarnett.plus.com> On 2017-10-20 18:19, Israel Brewster wrote: > tldr: I have an object that can't be picked. Is there any way to do a "raw" dump of the binary data to a file, and re-load it later? > > Details: I am using a java (I know, I know - this is a python list. I'm not asking about the java - honest!) library (Jasper Reports) that I access from python using py4j (www.py4j.org ). At one point in my code I call a java function which, after churning on some data in a database, returns an object (a jasper report object populated with the final report data) that I can use (via another java call) to display the results in a variety of formats (HTML, PDF, XLS, etc). At the time I get the object back, I use it to display the results in HTML format for quick display, but the user may or may not also want to get a PDF copy in the near future. > > Since it can take some time to generate this object, and also since the data may change between when I do the HTML display and when the user requests a PDF (if they do at all), I would like to save this object for potential future re-use. Because it might be large, and there is actually a fairly good chance the user won't need it again, I'd like to save it in a temp file (tat would be deleted when the user logs out) rather than in memory. Unfortunately, since this is an object created by and returned from a java function, not a native python object, it is not able to be pickled (as the suggestion typically is), at least to my knowledge. > > Given that, is there any way I can write out the "raw" binary data to a file, and read it back in later? Or some other way to be able to save this object? It is theoretically possible that I could do it on the java side, i.e. the library may have some way of writing out the file, but obviously I wouldn't expect anyone here to know anything about that - I'm just asking about the python side :-) > As far as I can tell, what you're getting is a Python object that's a proxy to the actual Java object in the Java Virtual Machine. The Python side might be taking to the Java side via a socket or a pipe, and not know anything about, or have access to, the internals of the Java object. In fact, you can't even be sure how a particular Python object is laid out in memory without reading the source code. From python at mrabarnett.plus.com Fri Oct 20 14:39:35 2017 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 20 Oct 2017 19:39:35 +0100 Subject: Efficient counting of results In-Reply-To: References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-20 18:05, Israel Brewster wrote:[snip] > In a sense, in that it supports my initial approach. > > As Stefan Ram pointed out, there is nothing wrong with the solution I have: simply using if statements around the calculated lateness of t1 and t2 to increment the appropriate counters. I was just thinking there might be tools to make the job easier/cleaner/more efficient. From the responses I have gotten, it would seem that that is likely not the case, so I'll just say "thank you all for your time", and let the matter rest. > It occurred to me that it might be more efficient to start with the year-to-date first. The reasoning is that over time the number of old entries will increase, so if you see that a timestamp isn't in this period_of_time, then it's not in any smaller_period_of_time either, so you can short-circuit. Compare doing these for an old entry: Day first: this day? no; this week? no; this month? no; this year? no. Year first: this year? no. From jasonhihn at gmail.com Fri Oct 20 16:13:53 2017 From: jasonhihn at gmail.com (Jason) Date: Fri, 20 Oct 2017 13:13:53 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> <31b2ee18-0824-e3d7-1ce5-4d42a9c8ae16@gmx.com> Message-ID: <4277b167-c572-496f-a84c-22e24a39b28f@googlegroups.com> Yes, it is a simplification and I am using numpy at lower layers. You correctly observe that it's a simple operation, but it's not a shift it's actually multidimensional vector algebra in numpy. So the - is more conceptual and takes the place of hundreds of subtractions. But the example dies demonstrate the complexity and how I can divide the problem up conceptually. From israel at ravnalaska.net Fri Oct 20 16:19:09 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Fri, 20 Oct 2017 12:19:09 -0800 Subject: Save non-pickleable variable? In-Reply-To: References: Message-ID: On Oct 20, 2017, at 11:09 AM, Stefan Ram wrote: > > Israel Brewster writes: >> Given that, is there any way I can write out the "raw" binary >> data to a file > > If you can call into the Java SE library, you can try > > docs.oracle.com/javase/9/docs/api/java/io/ObjectOutputStream.html#writeObject-java.lang.Object- > > , e.g.: > > public static void save > ( final java.lang.String path, final java.lang.Object object ) > { try > { final java.io.FileOutputStream fileOutputStream > = new java.io.FileOutputStream( path ); > > final java.io.ObjectOutputStream objectOutputStream > = new java.io.ObjectOutputStream( fileOutputStream ); > > objectOutputStream.writeObject( object ); > > objectOutputStream.close(); } > > catch( final java.io.IOException iOException ) > { /* application-specific code */ }} > > >> , and read it back in later? > > There's a corresponding ?readObject? method in > ?java.io.ObjectInputStream?. E.g., > > public static java.lang.Object load( final java.lang.String path ) > { > java.io.FileInputStream fileInputStream = null; > > java.io.ObjectInputStream objectInputStream = null; > > java.lang.Object object = null; > > try > { fileInputStream = new java.io.FileInputStream( path ); > > objectInputStream = new java.io.ObjectInputStream > ( fileInputStream ); > > object = objectInputStream.readObject(); > > objectInputStream.close(); } > > catch( final java.io.IOException iOException ) > { java.lang.System.out.println( iOException ); } > > catch > ( final java.lang.ClassNotFoundException classNotFoundException ) > { java.lang.System.out.println( classNotFoundException ); } > > return object; } > > However, it is possible that not all objects can be > meaningfully saved and restored in that way. Thanks for the information. In addition to what you suggested, it may be possible that the Java library itself has methods for saving this object - I seem to recall the methods for displaying the data having options to read from files (rather than from the Java object directly like I'm doing), and it wouldn't make sense to load from a file unless you could first create said file by some method. I'll investigate solutions java-side. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- > > -- > https://mail.python.org/mailman/listinfo/python-list From steve+comp.lang.python at pearwood.info Fri Oct 20 20:10:24 2017 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 21 Oct 2017 00:10:24 GMT Subject: Efficient counting of results References: <354DC8E4-1349-4E5F-A043-5D474E06ECEF@ravnalaska.net> <37C332C8-F518-48FF-AF4A-D4B64808E0D4@ravnalaska.net> <59e94efe$0$18588$b1db1813$d948b532@news.astraweb.com> Message-ID: <59ea906e$0$14942$b1db1813$d948b532@news.astraweb.com> On Fri, 20 Oct 2017 09:05:15 -0800, Israel Brewster wrote: > On Oct 19, 2017, at 5:18 PM, Steve D'Aprano > wrote: >> What t1 and t2 are, I have no idea. Your code there suggests that they >> are fields in your data records, but the contents of the fields, who >> knows? > > t1 and t2 are *independent* timestamp fields. My apologies - I made the > obviously false assumption that it was clear they were timestamps, or at > least times based on the fact I was calculating "minutes late" based on > them. It wasn't clear to me whether they were timestamps, flags, or something else. For example, you said: "if the date of the first record was today, t1 was on-time, and t2 was 5 minutes late" which suggested to me that the first record is a timestamp, and t1 and t2 were possibly enums or flags: (date=Date(2017, 10, 21), key=key, t1=ON_TIME, t2=FIVE_MINUTES_LATE) or possibly: (date=Date(2017, 10, 21), key=key, t1=True, t2=False) for example. [...] > Easily: because the record contains two DIFFERENT times. Since you want > more concrete, we're talking departure and arrival times here. Quite > easy to depart on-time, but arrive late, or depart late but arrive > on-time. Ah, the penny drops! If you had called them "arrival" and "departure" instead of "t1" and "t2", it would have been significantly less mysterious. Sometimes a well-chosen variable name is worth a thousand words of explanation. >> It also contradicts your statement that it is *date* and *key* that >> determines which late bin to use. > > I never made such a statement. I said they are used to determine "WHAT > on-time IS for the record", not WHETHER the record is on-time or not, > and certainly not which late bin to use. To put it a different way, > those are the key to a lookup table that tells me what T1 and T2 are > *supposed* to be in order for *each one* to be on time. Ah, that makes sense now. Thank you for explaining. [...] >> Rather, it seems that date and key are irrelevant and can be ignored, >> it is only t1 and t2 which determine which late bins to update. > > Except that then we have no way to know what t1 and t2 *should* be. Yes, that makes sense now. Your example of the driver runs really helped clarify what you are computing. > You > apparently made the assumption that t1 and t2 should always be some > fixed value. I tried to interpret your requirements as best I could from your description. Sorry that I failed so badly. [...] > Perhaps a better approach to explaining is to pose the question the > report is trying to answer: That would have been helpful. [...] > As Stefan Ram pointed out, there is nothing wrong with the solution I > have: simply using if statements around the calculated lateness of t1 > and t2 to increment the appropriate counters. I was just thinking there > might be tools to make the job easier/cleaner/more efficient. From the > responses I have gotten, it would seem that that is likely not the case, > so I'll just say "thank you all for your time", and let the matter rest. No problem. Sorry I couldn't be more helpful and glad you have a working solution. -- Steven D'Aprano From rustompmody at gmail.com Sat Oct 21 00:11:02 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 20 Oct 2017 21:11:02 -0700 (PDT) Subject: grapheme cluster library Message-ID: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Is there a recommended library for manipulating grapheme clusters? In particular, in devanagari ?? + ? = ?? in (pseudo)unicode names KA-letter + I-sign = KI-composite-letter I would like to be able to handle KI as a letter rather than two code-points. Can of course write an automaton to group but guessing that its already available some place? From michele.simionato at gmail.com Sat Oct 21 00:12:33 2017 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 20 Oct 2017 21:12:33 -0700 (PDT) Subject: multiprocessing shows no benefit In-Reply-To: <4277b167-c572-496f-a84c-22e24a39b28f@googlegroups.com> References: <5be8c8be-3f12-4aa4-a3d3-98eb22793b31@googlegroups.com> <76d86fb4-678f-42ae-9c38-6188c438c43a@googlegroups.com> <004c8ee5-08d5-4736-8abf-2faec0e25891@googlegroups.com> <31b2ee18-0824-e3d7-1ce5-4d42a9c8ae16@gmx.com> <4277b167-c572-496f-a84c-22e24a39b28f@googlegroups.com> Message-ID: There is a trick that I use when data transfer is the performance killer. Just save your big array first (for instance on and .hdf5 file) and send to the workers the indices to retrieve the portion of the array you are interested in instead of the actual subarray. Anyway there are cases where multiprocessing will never help, since the operation is too fast with respect to the overhead involved in multiprocessing. In that case just give up and think about ways of changing the original problem. From rosuav at gmail.com Sat Oct 21 02:21:40 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 21 Oct 2017 17:21:40 +1100 Subject: grapheme cluster library In-Reply-To: References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: On Sat, Oct 21, 2017 at 3:25 PM, Stefan Ram wrote: > Rustom Mody writes: >>Is there a recommended library for manipulating grapheme clusters? > > The Python Library has a module "unicodedata", with functions like: > > |unicodedata.normalize( form, unistr ) > | > |Returns the normal form ?form? for the Unicode string ?unistr?. > |Valid values for ?form? are ?NFC?, ?NFKC?, ?NFD?, and ?NFKD?. > > . I don't know whether the transformation you are looking for > is one of those. No, that's at a lower level than grapheme clusters. Rustom, have you looked on PyPI? There are a couple of hits, including one simply called "grapheme". ChrisA From dieter at handshake.de Sat Oct 21 03:31:54 2017 From: dieter at handshake.de (dieter) Date: Sat, 21 Oct 2017 09:31:54 +0200 Subject: right list for SIGABRT python binary question ? References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> Message-ID: <87y3o5gdn9.fsf@handshake.de> Karsten Hilbert writes: > ... > So here's the final console output of that: > ... > Debug memory block at address p=0x717b7c: API '' > 0 bytes originally requested > The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): > at p-3: 0x03 *** OUCH > at p-2: 0x4e *** OUCH > at p-1: 0x00 *** OUCH > Because memory is corrupted at the start, the count of bytes requested > may be bogus, and checking the trailing pad bytes may segfault. > The 4 pad bytes at tail=0x717b7c are not all FORBIDDENBYTE (0xfb): > at tail+0: 0x00 *** OUCH > at tail+1: 0x00 *** OUCH > at tail+2: 0x00 *** OUCH > at tail+3: 0x00 *** OUCH > The block was made by call #0 to debug malloc/realloc. > Fatal Python error: bad ID: Allocated using API '', verified using API 'o' > ... > Can anyone give more guidance on what the above python debug > output might vaguely point to ? It points to a memory corruption. I would approach the problem by means of debugging: put a write breakpoint at the corrupted address "0x717b7c" and check what part of the system accesses it (this assumes you are using a CPU supporting write breakpoints). It may be very tedious as the address might be accessed very often legally before it gets corrupted. Another approach may be to use a tool designed for memory debugging, e.g. "valgrind". From rustompmody at gmail.com Sat Oct 21 06:08:38 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 21 Oct 2017 03:08:38 -0700 (PDT) Subject: grapheme cluster library In-Reply-To: References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: On Saturday, October 21, 2017 at 11:51:57 AM UTC+5:30, Chris Angelico wrote: > On Sat, Oct 21, 2017 at 3:25 PM, Stefan Ram wrote: > > Rustom Mody writes: > >>Is there a recommended library for manipulating grapheme clusters? > > > > The Python Library has a module "unicodedata", with functions like: > > > > |unicodedata.normalize( form, unistr ) > > | > > |Returns the normal form ?form? for the Unicode string ?unistr?. > > |Valid values for ?form? are ?NFC?, ?NFKC?, ?NFD?, and ?NFKD?. > > > > . I don't know whether the transformation you are looking for > > is one of those. > > No, that's at a lower level than grapheme clusters. > > Rustom, have you looked on PyPI? There are a couple of hits, including > one simply called "grapheme". There is this one line solution using regex (or 2 char solution!) Not perfect but a good start >>> from regex import findall >>> veda="""? ???????? ????????? ????????????????????? ???????? ?????????? ????????????????? ? ? ??????? ??????? ??????? ?""" >>> findall(r'\X', veda) ['?', ' ', '??', '??', '?', '?', '??', ' ', '??', '??', '?', '??', '??', ' ', '??', '??', '??', '??', '??', '??', '?', '??', '?', '??', '?', '??', '\n', '??', '??', '?', '??', '?', ' ', '??', '??', '?', '??', '??', '?', ' ', '??', '??', '?', '??', '??', '?', '??', '??', '?', '??', ' ', '?', '\n', '?', ' ', '??', '??', '???', ' ', '??', '??', '???', ' ', '??', '??', '???', ' ', '?'] >>> Compare >>> [x for x in veda] ['?', ' ', '?', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '\n', '?', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '\n', '?', ' ', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', ' ', '?', '?', '?', '?', '?', '?', '?', ' ', '?'] What is not working are the vowel-less consonant-joins: ie ... '??', '?' ... [3,4 element of the findall] should be one '???' But its good enough for me for now I think PS Stefan I dont see your responses unless someone quotes them. Thanks anyway for the inputs From skybuck2000 at hotmail.com Sat Oct 21 08:47:49 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Sat, 21 Oct 2017 05:47:49 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. Message-ID: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> Hello, I just watched this video, it's pretty interesting and somewhat amazing, this guy find a way to find hidden instructions inside processors: https://www.youtube.com/watch?v=KrksBdWcZgQ The software which finds these hidden instructions is available too: https://github.com/xoreaxeaxeax/sandsifter Apperently it's open source python and a little bit of C. It will probably require some kind of python interpreter/executor and probably admin rights to run. I am not yet sure how to run this software it will require installing some additional capstone disassembler. I am curious what would be found on my AMD X2 3800+ from almost 12 years ago... if it will run at all... I think it will run. I would be curious to also here results of other people ! ;) So if you curious as to what secret instructions exist inside your computer give this a run. Later today or in the coming days when I have some time for this I will return to this subject. For now I may have other things to do or maybe not :P :) Bye, Skybuck =D From dstanek at dstanek.com Sat Oct 21 09:08:02 2017 From: dstanek at dstanek.com (David Stanek) Date: Sat, 21 Oct 2017 09:08:02 -0400 Subject: Application and package of the same name In-Reply-To: References: Message-ID: <20171021130802.GC10218@x1> On 19-Oct 19:34, Paul Moore wrote: > On 19 October 2017 at 19:18, Skip Montanaro wrote: > > > > This is in Python 2.7, FWIW. What am I missing? > > My immediate reaction is "you shouldn't name your main program and > your package the same". It's not a pattern I've seen commonly used. > This is actually a common pattern I see when teaching the language. For example, when a student wants to test out a package like requests many seem to initially want to create a requests.py module. Then they become very confused when they get an AttributeError on requests.get(). -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek From skip.montanaro at gmail.com Sat Oct 21 09:27:38 2017 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sat, 21 Oct 2017 08:27:38 -0500 Subject: Application and package of the same name In-Reply-To: <20171021130802.GC10218@x1> References: <20171021130802.GC10218@x1> Message-ID: This is actually a common pattern I see when teaching the language. For example, when a student wants to test out a package like requests many seem to initially want to create a requests.py module. Then they become very confused when they get an AttributeError on requests.get(). That I should fall prey to this after using Python for over 20 years... Should I be humbled, ashamed, delighted that I unwittingly demonstrated something I've seen in others' code? The sequence of events here was that I actually did have my main program inside the package, but wanted to separate the two, since the package is installed for clients of the allocation to use. As I was moving the application into a Docker environment, it made sense to extract the main program from the package and copy it separately into the image. I just git mv'd the file from the package directory to the scripts directory, and it stopped working. I should have realized right then and there what I'd done wrong, but I got hung up thinking I'd done something wrong with absolute imports. Skip From christopher_reimer at icloud.com Sat Oct 21 11:18:35 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Sat, 21 Oct 2017 08:18:35 -0700 Subject: Application and package of the same name In-Reply-To: <20171021130802.GC10218@x1> References: <20171021130802.GC10218@x1> Message-ID: On Oct 21, 2017, at 6:08 AM, David Stanek wrote: > This is actually a common pattern I see when teaching the language. For > example, when a student wants to test out a package like requests many > seem to initially want to create a requests.py module. Then they become > very confused when they get an AttributeError on requests.get(). For my web scraper program, I'm using "requestor.py" for requests. :) Chris R. From python at mrabarnett.plus.com Sat Oct 21 11:52:09 2017 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 21 Oct 2017 16:52:09 +0100 Subject: grapheme cluster library In-Reply-To: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: <1e37e13d-7c9b-de84-80a5-ee49ccf2c38c@mrabarnett.plus.com> On 2017-10-21 05:11, Rustom Mody wrote: > Is there a recommended library for manipulating grapheme clusters? > > In particular, in devanagari > ?? + ? = ?? > in (pseudo)unicode names > KA-letter + I-sign = KI-composite-letter > > I would like to be able to handle KI as a letter rather than two code-points. > Can of course write an automaton to group but guessing that its already > available some place? > You can use the regex module to split a string into graphemes: regex.findall(r'\X', string) From mal at egenix.com Sat Oct 21 13:10:31 2017 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 21 Oct 2017 19:10:31 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> Message-ID: <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> On 17.10.2017 09:30, Karsten Hilbert wrote: > On Tue, Oct 17, 2017 at 12:04:09PM +1100, Steve D'Aprano wrote: > >>> is this the right list to ask for help in debugging a >>> SIGABORT (?) happening on shutdown of a Python 2.7 script ? >>> >>> If not, which one is ? >> >> You should try here first. >> >> Please ensure you read and follow this first: >> >> http://sscce.org/ >> >> and post a *minimum* example of your code. (Sorry for the redundant >> instructions if you already know this, but you would be surprised how many >> people don't.) > > Thanks. I'll work towards a postable example. > > Running a debug build of py27 gave me a first lead: this > Debian system (Testing, upgraded all the way from various > releases ago) carries an incompatible mxDateTime which I'll > take care of. > > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "./bootstrap_gm_db_system.py", line 87, in > from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, gmI18N > File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", line 34, in > from Gnumed.pycommon import gmDateTime > File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", line 52, in > import mx.DateTime as mxDT > File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in > from DateTime import * > File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in > from mxDateTime import * > File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in > raise ImportError, why > ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 This error suggests that you have 32- and 64-bit versions of Python and mxDateTime mixed in your installation. Py_InitModule4 is only available in the 32-bit build of Python. With the 64-bit build, it's called Py_InitModule4_64. Since you're getting the same error from faulthandler, this is where I'd start to investigate. "nm" will list all exported and required symbols. As first step, you should probably check the python binary for its symbols and see whether it exports Py_InitModule* symbols. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Oct 21 2017) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From rustompmody at gmail.com Sat Oct 21 13:18:21 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 21 Oct 2017 10:18:21 -0700 (PDT) Subject: grapheme cluster library In-Reply-To: References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> <1e37e13d-7c9b-de84-80a5-ee49ccf2c38c@mrabarnett.plus.com> Message-ID: On Saturday, October 21, 2017 at 9:22:24 PM UTC+5:30, MRAB wrote: > On 2017-10-21 05:11, Rustom Mody wrote: > > Is there a recommended library for manipulating grapheme clusters? > > > > In particular, in devanagari > > ?? + ? = ?? > > in (pseudo)unicode names > > KA-letter + I-sign = KI-composite-letter > > > > I would like to be able to handle KI as a letter rather than two code-points. > > Can of course write an automaton to group but guessing that its already > > available some place? > > > You can use the regex module to split a string into graphemes: > > regex.findall(r'\X', string) Thanks MRAB Yes as I said I discovered r'\X' Ultimately my code was (effectively) one line! print("".join(map[x] for x in findall(r'\X', l))) with map being a few 100 elements of a dictionary such as map = { ... '?': "OM", ... } $ cat purnam-deva ? ???????? ????????? ????????????????????? ???????? ?????????? ????????????????? ? $ ./devanagari2roman.py purnam-deva OM pUraNamadaH pUraNamidaM pUraNAtpuraNamudachyate pUraNasya pUraNamAdAya pUraNamavAvashiShyate .. OM shAntiH shAntiH shAntiH .. Basically, an inversion of the itrans input method https://en.wikipedia.org/wiki/ITRANS From jfong at ms4.hinet.net Sat Oct 21 13:25:08 2017 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 21 Oct 2017 10:25:08 -0700 (PDT) Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: Terry Reedy at 2017-10-20 UTC+8 AM 7:37:59 wrote: > On 10/19/2017 5:07 AM, jfong at ms4.hinet.net wrote: > > > I got some info below each time when I squeeze the table: > > . > > .50077776 > > .50077776.50712528 > > .50077776.50712496 > > .50077776.50712464 > > .50077776.50712144 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > .50077776.50712528.50712560.50782256 > > .50077776.50712528.50712560.50782256.50783024 > > > > How to change these number(is it a widget ID?) to a meaning name? > > The number strings as names are the defaults that came from tkinter, not > tcl/tk. In 3.6, the default names were changed to be versions of the > class name. > > >>> import tkinter as tk > >>> r = tk.Tk() > >>> b = tk.Button(r) > >>> b > > >>> b2 = tk.Button(r) > >>> b2 > > I have a question about this change. When there are multiple buttons in the same widget hierarchy level and a ".xxx.yyy.!button2" showing up, how to figure out which button it means? By the way, where is the document of this change? Now it doesn't fit the description in the "tkinter 8.5 reference manual" anymore. --Jach From dvl at psu.edu Sat Oct 21 14:29:58 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Sat, 21 Oct 2017 14:29:58 -0400 Subject: Save non-pickleable variable? In-Reply-To: mailman.38.1508601605.29128.python-list@python.org References: Message-ID: <1508610598l.16122004l.0l@psu.edu> On Sat, Oct 21, 2017, Israel Brewster wrote: > tldr: I have an object that can't be picked. Is there any way to do a >"raw" dump of the binary data to a file, and re-load it later? > >Details: I am using a java (I know, I know - this is a python list. I'm >not asking about the java - honest!) library (Jasper Reports) >that I access from python using py4j (www.py4j.org >). At one point in my code I call a java >function which, after churning on some data in a database, returns an object >(a jasper report object populated with the final report data) that I >can use (via another java call) to display the results in a variety >of formats (HTML, PDF, XLS, etc). At the time I get the object back, >I use it to display the results in HTML format for quick display, but the user >may or may not also want to get a PDF copy in the near future. > >Since it can take some time to generate this object, and also since the data >may change between when I do the HTML display and when the user requests a PDF >(if they do at all), I would like to save this object for potential >future re-use. Because it might be large, and there is actually a fairly good >chance the user won't need it again, I'd like to save it in a temp file >(tat would be deleted when the user logs out) rather than in memory. >Unfortunately, since this is an object created by and returned from a java >function, not a native python object, it is not able to be pickled (as the >suggestion typically is), at least to my knowledge. > >Given that, is there any way I can write out the "raw" binary data to >a file, and read it back in later? Or some other way to be able to save this >object? It is theoretically possible that I could do it on the java side, i.e. >the library may have some way of writing out the file, but obviously I wouldn't >expect anyone here to know anything about that - I'm just asking about the >python side :-) > > Since you suggest a file is a suitable medium, perhaps you can try the JSON format I have heard about (but never actually used myself). Apparently there is module support for both Java and Python for this format, which is considered to be language independent. And since the Python view of this data would be a Python data structure, it should also be pickleable. The "Json.org" web site should give you a place to download the modules you would need for each of the two languages. Hope this helps. Roger Christman Pennsylvania State University From tjreedy at udel.edu Sat Oct 21 18:27:55 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 21 Oct 2017 18:27:55 -0400 Subject: How to debug an unfired tkinter event? In-Reply-To: References: <3271455e-12ab-482e-afc5-3125e24aa1a8@googlegroups.com> Message-ID: On 10/21/2017 1:25 PM, jfong at ms4.hinet.net wrote: > Terry Reedy at 2017-10-20 UTC+8 AM 7:37:59 wrote: >> On 10/19/2017 5:07 AM, jfong at ms4.hinet.net wrote: >> >>> I got some info below each time when I squeeze the table: >>> . >>> .50077776 >>> .50077776.50712528 >>> .50077776.50712496 >>> .50077776.50712464 >>> .50077776.50712144 >>> .50077776.50712528.50712560.50782256 >>> .50077776.50712528.50712560.50782256.50783024 >>> .50077776.50712528.50712560.50782256 >>> .50077776.50712528.50712560.50782256.50783024 >>> >>> How to change these number(is it a widget ID?) to a meaning name? >> >> The number strings as names are the defaults that came from tkinter, not >> tcl/tk. In 3.6, the default names were changed to be versions of the >> class name. >> >> >>> import tkinter as tk >> >>> r = tk.Tk() >> >>> b = tk.Button(r) >> >>> b >> >> >>> b2 = tk.Button(r) >> >>> b2 >> >> > I have a question about this change. When there are multiple buttons in the same widget hierarchy level and a ".xxx.yyy.!button2" showing up, how to figure out which button it means? There is an issue about one particular situation with a name clash. This can always be avoided by providing explicit names at least at the leaf. > By the way, where is the document of this change? What New in 3.6 should have something. > Now it doesn't fit the description in the "tkinter 8.5 reference manual" anymore. If you mean the one at nmt.edu, it is not maintained and is slowly becoming out of date. tcl/tk is now at 8.6.7 or .8. 'tkinter 8.5' is something of a misnomer because the tkinter version is the same as the Python version and tries to be somewhat up-to-date with tcl/tk, which being compatible with earlier tcl/tk versions. -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Sat Oct 21 20:13:40 2017 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: 22 Oct 2017 00:13:40 GMT Subject: grapheme cluster library References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: <59ebe2b3$0$18575$b1db1813$d948b532@news.astraweb.com> On Fri, 20 Oct 2017 21:11:02 -0700, Rustom Mody wrote: > Is there a recommended library for manipulating grapheme clusters? Back in July, I asked for anyone interested in grapheme clusters to consider checking out this issue on the bug tracker: http://bugs.python.org/issue30717 My post received at least 170 replies from at least 16 unique people (including you, Rustom). As far as I can see, only two of those people actually registered on the tracker to follow that ticket. >From time to time, there are repeated complaints that the Python standard library doesn't handle graphemes. Are those complaints mostly hot air, or is there actually community interest in having the stdlib deal with this? If there is community interest, the best ways to register that interest are, in order (best to worst): - step up and provide some code; - make a concrete proposal (not just "support graphemes") on the Python-Ideas mailing list; - register a feature request on the tracker; - complain about the lack of such support here; - do nothing. -- Steven D'Aprano From nieuws.pv at xs4all.nl Sun Oct 22 06:24:33 2017 From: nieuws.pv at xs4all.nl (Patrick Vrijlandt) Date: Sun, 22 Oct 2017 12:24:33 +0200 Subject: choice of web-framework Message-ID: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Hello list, I would like your recommendation on the choice of a web framework. The project is completely new, there are no histories to take into account (current solutions are paper-based). The website involves questionnaires that will be developed, filled out and stored. Users are not programmers or developers. They should be authenticated. Version control is required. Internationalization is not an issue. I expect that the project will add additional requirements and complexity later on that I can not foresee yet. I'm targeting a single deployment (maybe a second on a development machine). I usually work on Windows, but Linux can be considered. I'm not afraid to learn a (=one) new framework (that would actually be fun) but trying out a lot of them is not feasible. My current goal is a demonstration version of the project as a proof of concept. I may want to hand it over to a commercial solution at that stage. I'm an experienced python programmer but not an experienced web developer. A few years ago I read some books about Zope and Plone, but never did serious development with those. I currently maintain an intranet site in MoinMoin. I assume Zope could still be a potential choice, but it may have lost the vibrancy of a few years ago. Also, I would not know which version to choose (Zope 4, BlueBream, or something like Grok). The problem seems too complicated for micro frameworks like bottle of Flask. Django could be the next alternative. Finally, for a new project, I would not like to be confined to Python 2.7. What are your ideas? Thanks in advance, -- Patrick From rosuav at gmail.com Sun Oct 22 06:41:17 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Oct 2017 21:41:17 +1100 Subject: choice of web-framework In-Reply-To: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On Sun, Oct 22, 2017 at 9:24 PM, Patrick Vrijlandt wrote: > Hello list, > > I would like your recommendation on the choice of a web framework. > > The project is completely new, there are no histories to take into account > (current solutions are paper-based). The website involves questionnaires > that will be developed, filled out and stored. Users are not programmers or > developers. They should be authenticated. Version control is required. > Internationalization is not an issue. I expect that the project will add > additional requirements and complexity later on that I can not foresee yet. > I'm targeting a single deployment (maybe a second on a development machine). > I usually work on Windows, but Linux can be considered. > > I'm not afraid to learn a (=one) new framework (that would actually be fun) > but trying out a lot of them is not feasible. My current goal is a > demonstration version of the project as a proof of concept. I may want to > hand it over to a commercial solution at that stage. > > I'm an experienced python programmer but not an experienced web developer. A > few years ago I read some books about Zope and Plone, but never did serious > development with those. I currently maintain an intranet site in MoinMoin. I > assume Zope could still be a potential choice, but it may have lost the > vibrancy of a few years ago. Also, I would not know which version to choose > (Zope 4, BlueBream, or something like Grok). The problem seems too > complicated for micro frameworks like bottle of Flask. Django could be the > next alternative. > > Finally, for a new project, I would not like to be confined to Python 2.7. > > What are your ideas? First off, a huge thank-you for laying out your requirements in such detail! My personal choice, under the circumstances, would be Flask. Your project isn't too complicated for it, and it's not too hard to learn (I learned it in a weekend, though YMMV of course). Authentication is a fully-supported plugin (flask-login). I'm not sure what you mean by "version control" - of course your actual Python code can be managed in git/hg, but if it's a project requirement that your *data* be git-managed too, then you'll need to plan something out for that. Not too hard though. Python 3 is fully supported by Flask, so that's not a problem. Windows and Linux are also both viable platforms. I strongly recommend creating a file "requirements.txt" in your project root and recording every package you use there, so you can just run "pip install -r requirements.txt" to grab everything. Using a virtual environment ("python3 -m venv env" in recent-enough Pythons) can help to keep your dependencies clean. If you do that from the start of the project, and avoid using anything that's Windows-specific, you shouldn't have much trouble changing OSes. For the database, I generally use PostgreSQL, unless the job's so simple it can be done with flat files. Using Flask with SQLAlchemy is a popular option, but you can also dive into psycopg2 directly (the "2" doesn't mean "Python 2.7 only", it's fine). The tech stack Python+Flask+SQLAlchemy+PostgreSQL+Linux is an extremely solid one, or you can switch out any part fairly easily. Disclaimer: I use Flask and SQLAlchemy when I'm teaching my students how to build web apps in Python, but I don't have much experience with any other frameworks or ORMs. Other options may very well be viable and/or superior; all I can say is that the above *is* viable. ChrisA From kwpolska at gmail.com Sun Oct 22 06:45:28 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 22 Oct 2017 12:45:28 +0200 Subject: choice of web-framework In-Reply-To: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On 22 October 2017 at 12:24, Patrick Vrijlandt wrote: > Hello list, > > I would like your recommendation on the choice of a web framework. > > The project is completely new, there are no histories to take into account > (current solutions are paper-based). The website involves questionnaires > that will be developed, filled out and stored. Users are not programmers or > developers. They should be authenticated. Version control is required. > Internationalization is not an issue. I expect that the project will add > additional requirements and complexity later on that I can not foresee yet. > I'm targeting a single deployment (maybe a second on a development machine). > I usually work on Windows, but Linux can be considered. If you intend to put this on a server, and you probably do since you?re talking about web frameworks, a Linux machine is your best bet for that. Windows isn?t a good platform for making web servers out of. (Your development machine can probably run Windows.) > I'm not afraid to learn a (=one) new framework (that would actually be fun) > but trying out a lot of them is not feasible. My current goal is a > demonstration version of the project as a proof of concept. I may want to > hand it over to a commercial solution at that stage. > > I'm an experienced python programmer but not an experienced web developer. A > few years ago I read some books about Zope and Plone, but never did serious > development with those. I currently maintain an intranet site in MoinMoin. I > assume Zope could still be a potential choice, but it may have lost the > vibrancy of a few years ago. Also, I would not know which version to choose > (Zope 4, BlueBream, or something like Grok). The problem seems too > complicated for micro frameworks like bottle of Flask. Django could be the > next alternative. Zope is effectively dead these days. IMO your best bet would be Django: * built-in database support * built-in user authentication support * built-in administrator panel * i18n support available for when you need it * it?s a modern, friendly web framework If you went with Flask, you?d end up with a pile of plugins (for auth, for databases, for other things) and reimplement half of Django, badly. -- Chris Warrick PGP: 5EAAEA16 From lele at metapensiero.it Sun Oct 22 07:25:07 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Sun, 22 Oct 2017 13:25:07 +0200 Subject: choice of web-framework References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: <87lgk3v2zw.fsf@metapensiero.it> Chris Warrick writes: > Zope is effectively dead these days. Except it's alive and kicking: https://blog.gocept.com/ :-) ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From kwpolska at gmail.com Sun Oct 22 07:34:25 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 22 Oct 2017 13:34:25 +0200 Subject: choice of web-framework In-Reply-To: <87lgk3v2zw.fsf@metapensiero.it> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <87lgk3v2zw.fsf@metapensiero.it> Message-ID: On 22 October 2017 at 13:25, Lele Gaifax wrote: > Chris Warrick writes: > >> Zope is effectively dead these days. > > Except it's alive and kicking: https://blog.gocept.com/ > > :-) > > ciao, lele. A few people still care, sure. But how alive is a project with 16 (sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on GitHub, and 205 issues on GitHub (since 2013)? -- Chris Warrick PGP: 5EAAEA16 From rosuav at gmail.com Sun Oct 22 07:48:03 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 22 Oct 2017 22:48:03 +1100 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <87lgk3v2zw.fsf@metapensiero.it> Message-ID: On Sun, Oct 22, 2017 at 10:34 PM, Chris Warrick wrote: > On 22 October 2017 at 13:25, Lele Gaifax wrote: >> Chris Warrick writes: >> >>> Zope is effectively dead these days. >> >> Except it's alive and kicking: https://blog.gocept.com/ >> >> :-) >> >> ciao, lele. > > A few people still care, sure. But how alive is a project with 16 > (sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on > GitHub, and 205 issues on GitHub (since 2013)? > I'm not too bothered by number of stars, nor necessarily by the issue count (maybe a lot of their work is discussed by email, not the tracker). Most important, to me, is the number of commits. And that doesn't look too bad; there aren't a huge number of them, but they're fairly consistently being made. So I'd say the project isn't dead, though you could very well argue that it's merely playing catch-up. (I didn't look at the content of the commits in detail or anything.) ChrisA From python.list at tim.thechases.com Sun Oct 22 08:05:07 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 22 Oct 2017 07:05:07 -0500 Subject: choice of web-framework In-Reply-To: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: <20171022070507.78e2cb50@bigbox.christie.dr> On 2017-10-22 12:24, Patrick Vrijlandt wrote: > I would like your recommendation on the choice of a web framework. Might depend on what skills you already bring to the table. If you already know an ORM like SQLAlchemy or a template language like Jinja, you might want to take the "bring the pieces I know/like together" approach. For this, Bottle & Flask are the top contenders last I saw (I did some CherryPy contract work but the docs were pretty horrible at the time). If you are genuinely coming to this greenfield, Django's docs/tutorials make it really easy to get up to speed with all of the parts involved as it has its own ORM and templating language. They can be swapped out if you later need to, but for the average project, they're sufficient and well documented. I happen to be in the Django camp, but based on my experiments with Bottle/Flask, can also recommend them without much hesitation. > The project is completely new, there are no histories to take into > account (current solutions are paper-based). The website involves > questionnaires that will be developed, filled out and stored. Users > are not programmers or developers. They should be authenticated. > Version control is required. I'm not sure what "version control is required" means in this context. Is this version-control of the users' answers? Or version-control of the source code. If it's the source code, the web framework won't help you there, but git, mercurial, or subversion are all good/reasonable choices. If you want to version your user's answers or other aspects of your application, you'll need to design it into your app. There might be plugins/modules to facilitate this on either side of the Django / Flask/Bottle/SQLAlchemy divide. > I'm targeting a single deployment (maybe a second on a > development machine). I usually work on Windows, but Linux can be > considered. While both *can* be deployed on Windows, a Unix-like OS (whether Linux, a BSD, or even a Mac) will likely give you a better deployment experience and better docs. > I'm not afraid to learn a (=one) new framework (that would actually > be fun) but trying out a lot of them is not feasible. My current > goal is a demonstration version of the project as a proof of > concept. I personally find that Django excels at these fast proof-of-concept projects, as you have less concern about integrating disparate pieces together at that stage. > I'm an experienced python programmer but not an experienced web > developer. Both sides offer a "Pythonic" feel to them (some feel less Pythonic) so it's easy to come up to speed on either. > A few years ago I read some books about Zope and Plone, Hah, Zope was my first introduction to Python and I ran screaming. I eventually came back around, but it was a jarring first experience. > The problem seems too complicated for micro frameworks like bottle > of Flask. Django could be the next alternative. Django is the top contender, so if you only have time to investigate one, I'd urge you in that direction. But Flask or Bottle can also certainly handle a project like the one you describe. > Finally, for a new project, I would not like to be confined to > Python 2.7. Flask/Bottle and Django are both Python3 ready. Django, since v2.0 is now 3.0 only. -tkc From mal at europython.eu Sun Oct 22 08:05:43 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Sun, 22 Oct 2017 14:05:43 +0200 Subject: EuroPython 2017: Videos for Thursday available online Message-ID: We are pleased to announce the third batch of cut videos for EuroPython 2017. To see the new videos, please head over to our EuroPython YouTube channel and select the "EuroPython 2017" playlist. The new videos start at entry 96 in the playlist. * EuroPython 2017 Videos * http://europython.tv/ Next week we will release the last batch of videos currently marked as "private". Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/922070616836071425 Thanks. From kwpolska at gmail.com Sun Oct 22 08:24:48 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 22 Oct 2017 14:24:48 +0200 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <87lgk3v2zw.fsf@metapensiero.it> Message-ID: On 22 October 2017 at 13:48, Chris Angelico wrote: > On Sun, Oct 22, 2017 at 10:34 PM, Chris Warrick wrote: >> On 22 October 2017 at 13:25, Lele Gaifax wrote: >>> Chris Warrick writes: >>> >>>> Zope is effectively dead these days. >>> >>> Except it's alive and kicking: https://blog.gocept.com/ >>> >>> :-) >>> >>> ciao, lele. >> >> A few people still care, sure. But how alive is a project with 16 >> (sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on >> GitHub, and 205 issues on GitHub (since 2013)? >> > > I'm not too bothered by number of stars, nor necessarily by the issue > count (maybe a lot of their work is discussed by email, not the > tracker). Most important, to me, is the number of commits. And that > doesn't look too bad; there aren't a huge number of them, but they're > fairly consistently being made. So I'd say the project isn't dead, > though you could very well argue that it's merely playing catch-up. (I > didn't look at the content of the commits in detail or anything.) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list Mailing lists are quiet as well: https://mail.zope.org/pipermail/zope/ https://mail.zope.org/pipermail/zope-dev/ For a web framework, the daily commit count is much less important than the size of the active community. An active community means more people that can offer support, fix bugs, write docs, provide ready-made modules to achieve common tasks. Zope?s community is nonexistent. Django has 1483 contributors and 29k stars on GitHub. Zope has 83 and 85 respectively. https://blog.gocept.com/2016/10/04/zope-resurrection-part-2-defibrillation/ > Zope is not dead. On the sprint there were nearly 20 people who use Zope for their daily work. Oh my, twenty people! That?s a lot! A lot! Yeah, no. Zope is dead. With a few folks going through the ?denial? phase. Or the ?I?ve got legacy code from last decade I can?t be bothered to rewrite? phase. -- Chris Warrick PGP: 5EAAEA16 From nieuws.pv at xs4all.nl Sun Oct 22 09:26:03 2017 From: nieuws.pv at xs4all.nl (Patrick Vrijlandt) Date: Sun, 22 Oct 2017 15:26:03 +0200 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <20171022070507.78e2cb50@bigbox.christie.dr> Message-ID: <59ec9c6c$0$1711$e4fe514c@newszilla.xs4all.nl> Op 22-10-2017 om 14:05 schreef Tim Chase: > I'm not sure what "version control is required" means in this > context. Is this version-control of the users' answers? Or > version-control of the source code. If it's the source code, the web > framework won't help you there, but git, mercurial, or subversion are > all good/reasonable choices. If you want to version your user's > answers or other aspects of your application, you'll need to design > it into your app. There might be plugins/modules to facilitate this > on either side of the Django / Flask/Bottle/SQLAlchemy divide. The version control I was referring to, is indeed users' data. I plan to use Mercurial for the source code. The questionnaires being developed will go through many revisions. The questionnaires being filled in, are enough work to have a provision for mistakes. The idea is much like the "revert" option that MoinMoin and other wikis provide. --Patrick From rosuav at gmail.com Sun Oct 22 10:18:42 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Oct 2017 01:18:42 +1100 Subject: choice of web-framework In-Reply-To: <59ec9c6c$0$1711$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <20171022070507.78e2cb50@bigbox.christie.dr> <59ec9c6c$0$1711$e4fe514c@newszilla.xs4all.nl> Message-ID: On Mon, Oct 23, 2017 at 12:26 AM, Patrick Vrijlandt wrote: > Op 22-10-2017 om 14:05 schreef Tim Chase: > >> I'm not sure what "version control is required" means in this >> context. Is this version-control of the users' answers? Or >> version-control of the source code. If it's the source code, the web >> framework won't help you there, but git, mercurial, or subversion are >> all good/reasonable choices. If you want to version your user's >> answers or other aspects of your application, you'll need to design >> it into your app. There might be plugins/modules to facilitate this >> on either side of the Django / Flask/Bottle/SQLAlchemy divide. > > The version control I was referring to, is indeed users' data. I plan to use > Mercurial for the source code. The questionnaires being developed will go > through many revisions. The questionnaires being filled in, are enough work > to have a provision for mistakes. The idea is much like the "revert" option > that MoinMoin and other wikis provide. Then what I'd do is make sure the questionnaires are saved in a text file (maybe use a Markdown-like notation), and then just call on Mercurial from inside your app. ChrisA From formisc at gmail.com Sun Oct 22 11:21:10 2017 From: formisc at gmail.com (Andrew Z) Date: Sun, 22 Oct 2017 11:21:10 -0400 Subject: Modern website In-Reply-To: References: Message-ID: I realize the following has little todo with python per se. But i hope to get a guidance on how these types of tasks are done nowadays. The task: Ive been asked to create an integration process. That is a few webpages with questioneer with the submission to a "mother" company using its API . Ideally this process will be used by a few 3rd party "child" companies - they would just point a link from their site to mine. Prospective users will fill out the pages and "mother company" will create accounts associated with the referal "child" company. The problem: how do i create a modern , slick website. I also see , that each of "child" would want to extend the look and feel of their site onto this process. So, my process should have at least a few templates i can offer , that would match their site... Im not too excited to use weebly or squarespace - id rather host everything myself. From rosuav at gmail.com Sun Oct 22 11:26:01 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Oct 2017 02:26:01 +1100 Subject: Modern website In-Reply-To: References: Message-ID: On Mon, Oct 23, 2017 at 2:21 AM, Andrew Z wrote: > I realize the following has little todo with python per se. But i hope to > get a guidance on how these types of tasks are done nowadays. > > The task: > Ive been asked to create an integration process. That is a few webpages > with questioneer with the submission to a "mother" company using its API . > Ideally this process will be used by a few 3rd party "child" companies - > they would just point a link from their site to mine. Prospective users > will fill out the pages and "mother company" will create accounts > associated with the referal "child" company. > > The problem: > how do i create a modern , slick website. I also see , that each of > "child" would want to extend the look and feel of their site onto this > process. So, my process should have at least a few templates i can offer , > that would match their site... > Im not too excited to use weebly or squarespace - id rather host everything > myself. Design a single web site, then have the child sites customize the CSS *only*. That way, you guarantee that all functionality is the same, but you can change the look and feel as you need. Modern CSS can do rather a lot in terms of redefining the layout of a page. ChrisA From walters.justin01 at gmail.com Sun Oct 22 14:05:42 2017 From: walters.justin01 at gmail.com (justin walters) Date: Sun, 22 Oct 2017 11:05:42 -0700 Subject: choice of web-framework In-Reply-To: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On Sun, Oct 22, 2017 at 3:24 AM, Patrick Vrijlandt wrote: > Hello list, > > I would like your recommendation on the choice of a web framework. > > The project is completely new, there are no histories to take into account > (current solutions are paper-based). The website involves questionnaires > that will be developed, filled out and stored. Users are not programmers or > developers. They should be authenticated. Version control is required. > Internationalization is not an issue. I expect that the project will add > additional requirements and complexity later on that I can not foresee yet. > I'm targeting a single deployment (maybe a second on a development > machine). I usually work on Windows, but Linux can be considered. > > I'm not afraid to learn a (=one) new framework (that would actually be > fun) but trying out a lot of them is not feasible. My current goal is a > demonstration version of the project as a proof of concept. I may want to > hand it over to a commercial solution at that stage. > > I'm an experienced python programmer but not an experienced web developer. > A few years ago I read some books about Zope and Plone, but never did > serious development with those. I currently maintain an intranet site in > MoinMoin. I assume Zope could still be a potential choice, but it may have > lost the vibrancy of a few years ago. Also, I would not know which version > to choose (Zope 4, BlueBream, or something like Grok). The problem seems > too complicated for micro frameworks like bottle of Flask. Django could be > the next alternative. > > Finally, for a new project, I would not like to be confined to Python 2.7. > > What are your ideas? > > Thanks in advance, > > -- > Patrick > -- > https://mail.python.org/mailman/listinfo/python-list > I think your best choice here would be Django. I've been doing web development with Python for about 5 years now and I have only found thing that Django can't handle: replacing the authentication framework's dependence on SQL. i.e., if you wanted to use a noSQL db like MongoDb for your user model, it's not really possible without a ton of work. Other than that though, Django can pretty much do everything. Since you need to get this prototype done quickly, I think Django is your absolute best choice. The projects motto is "The web framework for perfectionists with deadlines." You get a ton of stuff out of the box: - User authentication - Sessions - Admin interface - Fantastic ORM - Templating (like jinbja2 but with some Django flair) - The best documentation I have ever seen - A huge ecosystem of third party libraries and plugins - A strong and heavilly opinionated MVC architecture - One of ths strongest and best suppported OSS projects in existence - Built in and versioned schema migrations The reasons I would not reccomend Flask/Bottle for your project specifically: - Flask is better for more "customized" solutions or simpler projects - Flask is a great framework, but offers very little out of the box as far as modern web application features - It takes longer to get rolling with Flask(a lot more initial configuration) - When using Flask, many devs end up building their own version of Django anyways - Flask's tutorial is a lot less in-depth than Django's - Although learning Flask in its entirety is much simpler than learning Django in its entirety, it takes more time to get up and running with Flask in my experience. Web2py/zope/other small and old frameworks: - I would stay away from these unless you have a lot of experience with them. - These projects do not have a modern ecosystem of libraries and may not have full Python3 support - You will find less community memebers that are able to help you as there are less people using these frameworks Hug/Sanic/Falcon/ApiStar: - These are designed around the idea of REST apis - They offer less than Flask does out of the box(except for building REST apis) - They are fairly new and have not had the time to build up a supportive ecosystem yet - They rely on new language features like async - Their docs are not up to par with more mature projects Pyramid: Though I haven't ever worked with Pyramid, I have met several people who are very happy with it. I also met with one of the project's core contributors and he spoke about how the architecture is "plugin based". There is the the core library and all of the other layers of the application such as the data layer or authentication, for example, are officially supported plugins. Might be worth looking into. From python.list at tim.thechases.com Sun Oct 22 14:27:23 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Sun, 22 Oct 2017 13:27:23 -0500 Subject: choice of web-framework In-Reply-To: <59ec9c6c$0$1711$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <20171022070507.78e2cb50@bigbox.christie.dr> <59ec9c6c$0$1711$e4fe514c@newszilla.xs4all.nl> Message-ID: <20171022132723.722ea55a@bigbox.christie.dr> On 2017-10-22 15:26, Patrick Vrijlandt wrote: > The version control I was referring to, is indeed users' data. I > plan to use Mercurial for the source code. The questionnaires being > developed will go through many revisions. The questionnaires being > filled in, are enough work to have a provision for mistakes. The > idea is much like the "revert" option that MoinMoin and other wikis > provide. Depends on how much version-control'y'ness you want. Having a "current version with previous version" and if "resurrect version $CURRENT-$N as the most recent version" is sufficient, then it's pretty straight-forward. If you also want to implement diffing, merging diffs between various versions, diffing more than a single text-field blob (a model spread across multiple normalized tables, versioning changes there), etc, you're looking at a whole different game. Additionally, one needs to consider how responses get tied to a questionnaire. If I make a questionnaire that reads "Do you like ice cream?" [ ] Yes [ ] No and you answer "Yes", but then I edit that question so that it reads "Do you like to kick puppies?" you have a problem if it keeps your "Yes" answer and thinks it's linked to the "same" question. All that to say that version-control is often domain-specific and non-trivial. -tkc From Karsten.Hilbert at gmx.net Sun Oct 22 16:15:51 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sun, 22 Oct 2017 22:15:51 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> Message-ID: <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> On Sat, Oct 21, 2017 at 07:10:31PM +0200, M.-A. Lemburg wrote: > > Running a debug build of py27 gave me a first lead: this > > Debian system (Testing, upgraded all the way from various > > releases ago) carries an incompatible mxDateTime which I'll > > take care of. > > > > *** You don't have the (right) mxDateTime binaries installed ! > > Traceback (most recent call last): > > File "./bootstrap_gm_db_system.py", line 87, in > > from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, gmI18N > > File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", line 34, in > > from Gnumed.pycommon import gmDateTime > > File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", line 52, in > > import mx.DateTime as mxDT > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in > > from DateTime import * > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in > > from mxDateTime import * > > File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in > > raise ImportError, why > > ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 > > This error suggests that you have 32- and 64-bit versions of > Python and mxDateTime mixed in your installation. > > Py_InitModule4 is only available in the 32-bit build of > Python. With the 64-bit build, it's called Py_InitModule4_64. > > Since you're getting the same error from faulthandler, > this is where I'd start to investigate. > > "nm" will list all exported and required symbols. As first step, > you should probably check the python binary for its symbols and > see whether it exports Py_InitModule* symbols. Thanks for your input ! The python2.7-dbg build is 32 bits: root at hermes:~# nm /usr/bin/python2.7-dbg | grep Py_InitM 00155b9f T Py_InitModule4TraceRefs python2.7-dbg: Installiert: 2.7.14-2 Installationskandidat: 2.7.14-2 Versionstabelle: *** 2.7.14-2 500 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 2.7.13-2 990 500 http://httpredir.debian.org/debian stretch/main i386 Packages 990 http://httpredir.debian.org/debian buster/main i386 Packages The python2.7 build (no -dbg) does not have symbols. mxDateTime really should be 32 bits, too: python-egenix-mxdatetime: Installiert: 3.2.9-1 Installationskandidat: 3.2.9-1 Versionstabelle: *** 3.2.9-1 990 500 http://httpredir.debian.org/debian stretch/main i386 Packages 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status Let me check the .so file: root at hermes:~# nm /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime_d.so | grep Py_InitM U Py_InitModule4TraceRefs It seems it is - hm ... Thanks, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From Karsten.Hilbert at gmx.net Sun Oct 22 16:24:02 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sun, 22 Oct 2017 22:24:02 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> Message-ID: <20171022202402.sbretkk7q2k5neox@hermes.hilbert.loc> On Sun, Oct 22, 2017 at 10:15:51PM +0200, Karsten Hilbert wrote: > The python2.7-dbg build is 32 bits: ... > The python2.7 build (no -dbg) does not have symbols. More to the point: /usr/bin/python2.7: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=91226399fb434d138f166a5c8eca04385ea2e41f, stripped /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=f09c7cc78b41421d3cee1f418b4d45772a3db701, stripped Regards, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From skybuck2000 at hotmail.com Sun Oct 22 16:24:41 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Sun, 22 Oct 2017 13:24:41 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> Message-ID: <7dd0d6d3-e13e-47b7-b748-a427f558e661@googlegroups.com> Hi, I hope you have the following newsgroup in case you are highly interested in knowing every last detail and every thought I have on this subject matter: https://groups.google.com/forum/#!forum/alt.comp.hardware.pc-homebuilt There I have written some detailed postings. In the other newsgroups I will constrain myself to the most important matter/summary of my activities, findings and productions for your usage. The most important information I want to share with you is the following: 1. I was successfull in running SandSifter software with Linux Mint 18.2 booteable DVD, downloaded ISO from the internet and undocumented instructions have been found for AMD X2 3800+ Dual Core processor. 2. All files are available on my webdrive: www.skybuck.org/SandSifter/ Explore the "unzipped folder" to see what it's all about. 3. I have written two tutorials how you can also run this software on your computer in case you have a DVD drive and DVD disc to burn this software onto. One manual tutorial and one automatic tutorial. The automatic tutorial is the easiest one which I will post here, the automatic tutorial includes a run.sh script which I will also post here, this is to help you run this software on your machine, at the end of this posting I will discuss any possible risks to doing so in case you are worried. Automatic tutorial: Step 1. Download Linux Mint ISO (Successfully tested on Linux Mint 18.2 Sonya) https://www.linuxmint.com/ Step 2. Burn Linux Mint ISO to DVD (Windows 7: Right click on file and choose burn to disc). Step 3. Boot Linux Mint ISO from DVD (Restart computer, if needed go into bios and change boot order, or press F8 to bring up boot menu or something like that) Step 4. Start FireFox Web Browser Step 5. Download SandSifter software and extract to a folder. https://github.com/xoreaxeaxeax/sandsifter (Click "clone or download", then click "download zip", then click "open with archive manager", then click "extract" (top left icon), click "other locations", choose a harddisk or other storage medium which is persistent, click on the storage medium, click create new folder (top right icon), name for folder could be "test", click "extract", click "show the files") Enter the folder "sandsifter-master" by left clicking on it. Step 6. Download Skybuck's Flying run.sh script file Download and save the "run.sh" script file to/inside the "sandsifter-master" folder. http://www.skybuck.org/SandSifter/unzipped/run.sh Step 7. Open terminal window and resize it to make it bigger Right click in the empty space and choose "open in terminal" A window and a prompt/blinking cursor should now come up looking similar to: mint at mint /media/mint/Windows 7 System (New)/test/sandsifter-master $ Make the window bigger so that the summarize script at the end doesn't crash ! Drag and Drop the window at the bottom right corner to make it bigger (Hold the left mouse button to drag and make it bigger then let mouse button go) Step 8. Run Skybuck's Flying Bash Script to install software and run SandSifter type the following command: bash ./run.sh Step 9. Guide the software installation and upgrade process Sometimes it will ask if you want to continue ? Press the Y key. Once it's done installing SandSifter will automatically run and finally a summary will be created. Step 10. Wait for the analysis to complete Once you see instructions scrolling/flying over the screen go take a sleep and wait many hours until it is completely done. Once it is done it will show something like: "May the Force be with you ! Always !" then you know the script is done ! Step 11. Do not open the log files ! The log files (in data folder) may be to big for the Linux Mint 18.2 text and office editors to handle ! This will probably crash/hang the system ! Step 12. Go into the data folder and send the files to the e-mail address: xoreaxeaxeax at gmail.com The run.sh script: echo "Step 1. Install standard C library software" sudo apt-get install libc6-dev echo "Step 2. Install python pip" sudo apt install python-pip echo "Step 3. Update python pip" sudo pip install --upgrade pip echo "Step 4. Install setuptools" sudo pip install setuptools echo "Step 5. Install capstone binaries" sudo apt-get install libcapstone3 echo "Step 6. Install capstone dev source" sudo apt-get install libcapstone-dev echo "Step 7. Install capstone python bindings (this will take a while)" sudo pip install capstone echo "Step 8. Make sandsifter" make echo "Step 9. Run sandsifter" sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t echo "Step 10. Summarize" ./summarize.py data/log echo "" echo "Bash script" echo "Version 0.01 created on 22 october 2017 by Skybuck Flying" echo "To Install, Make, Run, Summarize SandSifter Software and Software Dependencies" echo "Successfully tested on Linux Mint 18.2 Sonya on AMD Dual Core X2 3800+ processor" echo "May the Force be with you ! Always ! =D" echo "Have fun analyzing undocumented instructions !!!!" echo "E-mail results to or contact: xoreaxeaxeax at gmail.com" echo "^^^ !!! Author of SandSifter Software and interested in log files !!! ^^^" echo "" For now I will not discuss the collected data, this will have to be further analyzed, however I will say that the collected data is in this folder: http://www.skybuck.org/SandSifter/unzipped/data/ The log file contains discovered undocumented instruction byte code sequences for further investigation. (Lastly I will try and collect the messages I write on this subject matter in the messages folder so you don't have to scavenge the usenet/web for all info;) a bit tricky but I will try at least :)) Bye, Skybuck. From Karsten.Hilbert at gmx.net Sun Oct 22 17:10:33 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sun, 22 Oct 2017 23:10:33 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <87y3o5gdn9.fsf@handshake.de> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> <87y3o5gdn9.fsf@handshake.de> Message-ID: <20171022211033.xf656nn3ha6v7j2a@hermes.hilbert.loc> On Sat, Oct 21, 2017 at 09:31:54AM +0200, dieter wrote: > > Debug memory block at address p=0x717b7c: API '' > > 0 bytes originally requested > > The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): > > at p-3: 0x03 *** OUCH > > at p-2: 0x4e *** OUCH > > at p-1: 0x00 *** OUCH > > Because memory is corrupted at the start, the count of bytes requested > > may be bogus, and checking the trailing pad bytes may segfault. > > The 4 pad bytes at tail=0x717b7c are not all FORBIDDENBYTE (0xfb): > > at tail+0: 0x00 *** OUCH > > at tail+1: 0x00 *** OUCH > > at tail+2: 0x00 *** OUCH > > at tail+3: 0x00 *** OUCH > > The block was made by call #0 to debug malloc/realloc. > > Fatal Python error: bad ID: Allocated using API '', verified using API 'o' > > ... > > Can anyone give more guidance on what the above python debug > > output might vaguely point to ? > > It points to a memory corruption. > > I would approach the problem by means of debugging: put a write > breakpoint at the corrupted address "0x717b7c" and check what part > of the system accesses it (this assumes you are using a CPU > supporting write breakpoints). > It may be very tedious as the address might be accessed very often > legally before it gets corrupted. > > Another approach may be to use a tool designed for memory debugging, > e.g. "valgrind". Hi Dieter, thanks for your guidance. I fear this approach is out of my class. Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From danceswithnumbers at gmail.com Sun Oct 22 17:22:50 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Sun, 22 Oct 2017 14:22:50 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: Message-ID: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> In Short, I cannot find a single mathematical proof that says you cannot compress random numbers. Pigeon hole and other conjectures are just that. In fact, the biggest fallacy when people start talking about compression is to say that all compression alg rely on redundancies, or repetitive sequences. The million random digits is compressed down to 415,241 kb. I have been only able to compress that down to 352,954 kb. A very small amount. It has taken six years to come up with that small amount. From danceswithnumbers at gmail.com Sun Oct 22 17:25:04 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Sun, 22 Oct 2017 14:25:04 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: Message-ID: <9377d14f-cac8-4755-905b-a35cb3bb0050@googlegroups.com> On Monday, July 11, 2016 at 11:52:27 AM UTC-6, jonas.t... at gmail.com wrote: > What kind of statistic law or mathematical conjecture or is it even a physical law is violated by compression of random binary data? > > I only know that Shanon theorised it could not be done, but were there any proof? > > What is to say that you can not do it if the symbolic representation is richer than the symbolic represenatation of the dataset. > > Isn't it a fact that the set of squareroots actually depict numbers in a shorter way than their actual representation. > > Now the inpretator or program must know the rules. And i have very good rules to make it happen. In Short, I cannot find a single mathematical proof that says you cannot compress random numbers. Pigeon hole and other conjectures are just that. In fact, the biggest fallacy when people start talking about compression is to say that all compression alg rely on redundancies, or repetitive sequences. The million random digits is compressed down to 415,241 kb. I have been only able to compress that down to 352,954 kb. A very small amount. It has taken six years to come up with that small amount. From steve+python at pearwood.info Mon Oct 23 00:21:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 23 Oct 2017 15:21:33 +1100 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> Message-ID: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >>When we have a source of 2 random binary digits, >>there are 2^2 possible outcomes: >>( 0, 0 ), >>( 0, 1 ), >>( 1, 0 ), and >>( 1, 1 ). >>. One cannot aggree upon a code to represent each >>of those four possibilities with just one bit. > > If the probabilities of each outcome is 0.25. When people talk about "random data", they are normally referring to equal probabilities. Just as when they talk about compression in this context, they mean lossless, reversible compression. If the probability of certain codes (either single codes, or sequences of codes) are non-equal, then you can take advantage of that by encoding the common cases into a short representation, and the uncommon and rare cases into a longer representation. As you say: > Otherwise, if ( 0, 0 ) is much more frequent, > we can encode ( 0, 0 ) by "0" and > > ( 0, 1 ) by "101", > ( 1, 0 ) by "110", and > ( 1, 1 ) by "111". > > And we could then use /less/ than two bits on the > average. That's incorrect. On average you use 2.5 bits. (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) Where you can actually get compression is if you limit yourself to only encoding specially selected input data where the code pairs are non-random and you have many more (0,0) than other combinations. But that is highly non-random. For example, if we are compressing a black and white bitmap, say a fax, the most common data is a page which is mostly white with only a bit of black text. That gives us a bitmap with lots and lots of (0,0) pairs and relatively few of the other combinations, which means for the common case of "a page of black text on a white background" the compression works well. (I'm treating 0 as white and 1 as black.) But for purely random input, there will be roughly equal numbers of each pair of bit and on average the encoding will expand the data by 25% (two bits of input maps to 2.5 bits output, on average). > But two bits are (maximally )random if > the probability of every combination /is/ 0.25. In the example you give, we have four possible pairs of bytes: Input: (0,0) (0,1) (1,0) (1,1) # 2 bits on average Output: "0" "101" "110" "111" # 2.5 bits on average So if the incoming data is random (uniformly-distributed), this will expand the size of the data by 25%. Only if the data is non-random and biased heavily towards (0,0) will you see compression savings. Of course in the real world most data we want to compress is non-random, and we can get really impressive compressions. But the cost of that is that you can rarely compress data twice (if you do, you either get very little savings the second time, or it expands) and you cannot compress random (uniformly distributed) data. > (If other combinations than ( 0, 0 ) were extremely > rare, one could even improve the compression more.) What you say is true, but it has nothing to do with Danceswithnumbers' ignorant and ludicrous claim that he can compress random data and that there is no mathematical proof that you cannot. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Mon Oct 23 01:08:36 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 23 Oct 2017 18:08:36 +1300 Subject: Compression of random binary data In-Reply-To: <9377d14f-cac8-4755-905b-a35cb3bb0050@googlegroups.com> References: <9377d14f-cac8-4755-905b-a35cb3bb0050@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > On Monday, July 11, 2016 at 11:52:27 AM UTC-6, jonas.t... at gmail.com wrote: >> What is to say that you can not do it if the symbolic representation is >> richer than the symbolic represenatation of the dataset. The more symbols you have in your alphabet, the more bits are needed to encode each symbol. >> Isn't it a fact that the set of squareroots actually depict numbers in a >> shorter way than their actual representation. No. > In Short, I cannot find a single mathematical proof that says you cannot > compress random numbers. Pigeon hole and other conjectures are just that. No, they are not conjectures, they are proofs. If you don't recognise them as such, then you haven't fully understood them. > In > fact, the biggest fallacy when people start talking about compression is to > say that all compression alg rely on redundancies, or repetitive sequences. I think it's more correct to say they rely on the fact that the input data is encoded *ineffiently* in some way -- it uses more bits than are necessary to represent the information. > The million random digits is compressed down to 415,241 kb. I have been only > able to compress that down to 352,954 kb. A very small amount. It has taken > six years to come up with that small amount. You may well have an algorithm that encodes *some* million digit sequences in less than 415,241 bytes. But I can guarantee that there will be other million-digit sequences that it encodes to *more* than 415,241 bytes. The average over all possible million-digit sequences, assuming they are all equally likely, *cannot* be less than 415,241 bytes. Anything else would be a logical impossibility. Note that "equally likely" is important here. That's what "random data" means in this context -- that all possible input sequences have the same probability. This provides another way of thinking about how compression works: it relies on all input sequences *not* having the same probability. If some are more likely than others, then you can encode the more frequent ones with less bits and the less frequent ones with more bits, and gain overall. -- Greg From steve+python at pearwood.info Mon Oct 23 01:39:35 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 23 Oct 2017 16:39:35 +1100 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> Message-ID: <59ed8099$0$18561$b1db1813$d948b532@news.astraweb.com> On Mon, 23 Oct 2017 08:22 am, danceswithnumbers at gmail.com wrote: > In Short, I cannot find a single mathematical proof that says you cannot > compress random numbers. Three seconds of googling finds this: http://www.faqs.org/faqs/compression-faq/part1/section-8.html which demonstrates a simple, trivial proof. It took me a little longer to find this: https://en.wikipedia.org/wiki/Kolmogorov_complexity#Compression but only because I couldn't remember how to spell Kolmogorov, and that lead me to this: https://en.wikipedia.org/wiki/Incompressible_string for a concrete example. Of course *occasionally* one will, by pure luck, compress a stream of random bits. I run my random number generator and by a fluke it generates a stream of 50 zeroes in a row: 00000000000000000000000000000000000000000000000000 which even a simple run-length encoding scheme can compress: (50*0) But flukes are not what people mean when they talk about compressing random data. They mean some algorithm which can: - transparently (no hiding data in the decompressor, or the file name, or in a secret file hidden on the disk, or anywhere else) - reversibly (there must be a decompressor which when given ONLY the compressed file, and NOTHING else, reconstruct the original) - losslessly (no throwing away data: we must reconstruct the EXACT original, not merely something "close enough") - and *consistently* (flukes don't count: every input must be compressed) compress random data of size N bits to at most N-1 bits. We can prove that this is impossible by simply *counting* how many random strings there are. Let us pick a concrete example, N=8. Then there are 2**8 = 256 possible strings. We need to compress down to at most 7 bits. If we allow the output to vary in length, there are: 2**7 = 128 seven bit strings; 2**6 = 64 six bit strings; 2**5 = 32 five bit strings; 2**4 = 16 four bit strings; 2**3 = 8 three bit strings; 2**2 = 4 two bit strings; 2**1 = 2 one bit strings; 2**0 = 1 zero bit strings which add up to 255 compressed strings in total. But there are 256 possible inputs, and only 255 possible outputs: so at least one input cannot be compressed, or the compression must be lossy (two inputs compress to the same output). By the way: here is a very clever trick for hiding information in the file system: http://www.patrickcraig.co.uk/other/compression.php but as people point out, the information in the file, plus the information in the file system, ends up being *larger* than the original. Well done to Patrick Craig for finding a clever loophole to "win" the challenge, but he did so without actually compressing the original data. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From arj.python at gmail.com Mon Oct 23 01:55:03 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 23 Oct 2017 09:55:03 +0400 Subject: Modern website In-Reply-To: References: Message-ID: do it as django does it .in django you have templates like take the example of a single webpage other pages only modify what they need to. like the footer will remain the same on all pages, so only the top part needs to be specified similarly for your website, you can go along that line ^^, Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 22 Oct 2017 19:21, "Andrew Z" wrote: > I realize the following has little todo with python per se. But i hope to > get a guidance on how these types of tasks are done nowadays. > > The task: > Ive been asked to create an integration process. That is a few webpages > with questioneer with the submission to a "mother" company using its API . > Ideally this process will be used by a few 3rd party "child" companies - > they would just point a link from their site to mine. Prospective users > will fill out the pages and "mother company" will create accounts > associated with the referal "child" company. > > The problem: > how do i create a modern , slick website. I also see , that each of > "child" would want to extend the look and feel of their site onto this > process. So, my process should have at least a few templates i can offer , > that would match their site... > Im not too excited to use weebly or squarespace - id rather host everything > myself. > -- > https://mail.python.org/mailman/listinfo/python-list > From arj.python at gmail.com Mon Oct 23 02:28:29 2017 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 23 Oct 2017 10:28:29 +0400 Subject: choice of web-framework In-Reply-To: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: have you considered Django? i've found found it to be nice ! Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 22 Oct 2017 14:25, "Patrick Vrijlandt" wrote: > Hello list, > > I would like your recommendation on the choice of a web framework. > > The project is completely new, there are no histories to take into account > (current solutions are paper-based). The website involves questionnaires > that will be developed, filled out and stored. Users are not programmers or > developers. They should be authenticated. Version control is required. > Internationalization is not an issue. I expect that the project will add > additional requirements and complexity later on that I can not foresee yet. > I'm targeting a single deployment (maybe a second on a development > machine). I usually work on Windows, but Linux can be considered. > > I'm not afraid to learn a (=one) new framework (that would actually be > fun) but trying out a lot of them is not feasible. My current goal is a > demonstration version of the project as a proof of concept. I may want to > hand it over to a commercial solution at that stage. > > I'm an experienced python programmer but not an experienced web developer. > A few years ago I read some books about Zope and Plone, but never did > serious development with those. I currently maintain an intranet site in > MoinMoin. I assume Zope could still be a potential choice, but it may have > lost the vibrancy of a few years ago. Also, I would not know which version > to choose (Zope 4, BlueBream, or something like Grok). The problem seems > too complicated for micro frameworks like bottle of Flask. Django could be > the next alternative. > > Finally, for a new project, I would not like to be confined to Python 2.7. > > What are your ideas? > > Thanks in advance, > > -- > Patrick > -- > https://mail.python.org/mailman/listinfo/python-list > From rustompmody at gmail.com Mon Oct 23 02:47:02 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 22 Oct 2017 23:47:02 -0700 (PDT) Subject: grapheme cluster library (Posting On Python-List Prohibited) In-Reply-To: References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: On Monday, October 23, 2017 at 8:06:03 AM UTC+5:30, Lawrence D?Oliveiro wrote: > On Saturday, October 21, 2017 at 5:11:13 PM UTC+13, Rustom Mody wrote: > > Is there a recommended library for manipulating grapheme clusters? > > Is this any good? Thanks looks promising. Dunno how much it lives up to the claims [For now the one liner from regex's findall has sufficed: findall(r'\X', ?text?) [Thanks MRAB for the library] > Bear in mind that the logical representation of the text is as code points, graphemes would have more to do with rendering. Heh! Speak of Euro/Anglo-centrism! In a sane world graphemes would be called letters And unicode codepoints would be called something else ? letterlets?? To be fair to the Unicode consortium, they strive hard to call them codepoints But in an anglo-centric world, the conflation of codepoint to letter is inevitable I guess. To hear how a non Roman-centric view of the world would sound: A 'w' is a poorly double-struck 'u' A 't' is a crossed 'l' Reasonable? The lead of https://en.wikipedia.org/wiki/%C3%9C has | ?, or ?, is a character?classified as a separate letter in several extended Latin alphabets | (including Azeri, Estonian, Hungarian and Turkish), but as the letter U with an | umlaut/diaeresis in others such as Catalan, French, Galician, German, Occitan and Spanish. From dieter at handshake.de Mon Oct 23 03:15:38 2017 From: dieter at handshake.de (dieter) Date: Mon, 23 Oct 2017 09:15:38 +0200 Subject: choice of web-framework References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: <8760b62v39.fsf@handshake.de> Patrick Vrijlandt writes: > ... > The project is completely new, there are no histories to take into > account (current solutions are paper-based). The website involves > questionnaires that will be developed, filled out and stored. Users > are not programmers or developers. They should be > authenticated. Version control is required. Internationalization is > not an issue. I expect that the project will add additional > requirements and complexity later on that I can not foresee yet. I'm > targeting a single deployment (maybe a second on a development > machine). I usually work on Windows, but Linux can be considered. I am using Plone (a CMS (= Content Management System) build on top of Zope) for something like this. It has a standard extension "CMFEditions" for version control of its content. The content is managed in the Zope Object Database (= "ZODB"). This is also valid for the revisions. Thus, you do not get "git/mercurial/..."-style version control (based on files) but you see when and by whom a version was created, can revert to a previous version and see differences between versions. There is no merge support, though. The standard Plone content types are likely not sufficient to implement your questionnaires; you will probably define one of more specific for your task. But, this is not too complex. Questions about Plone (and the underlying Zope) can be asked at "https://community.plone.org/". From dieter at handshake.de Mon Oct 23 03:25:22 2017 From: dieter at handshake.de (dieter) Date: Mon, 23 Oct 2017 09:25:22 +0200 Subject: Modern website References: Message-ID: <871slu2un1.fsf@handshake.de> Andrew Z writes: > I realize the following has little todo with python per se. But i hope to > get a guidance on how these types of tasks are done nowadays. > > The task: > Ive been asked to create an integration process. That is a few webpages > with questioneer with the submission to a "mother" company using its API . > Ideally this process will be used by a few 3rd party "child" companies - > they would just point a link from their site to mine. Prospective users > will fill out the pages and "mother company" will create accounts > associated with the referal "child" company. > > The problem: > how do i create a modern , slick website. I also see , that each of > "child" would want to extend the look and feel of their site onto this > process. So, my process should have at least a few templates i can offer , > that would match their site... > Im not too excited to use weebly or squarespace - id rather host everything If the liberty of the "client" sites is of primary concern, then you can use a so called "service oriented architecture". Such an architecture sonsists of components, most of which providing a services via a standard API, destined for programmatic use. You can have services on different levels of abstraction, e.g. a base level using only structural data without any presentation and on top of this some standard presentation (which can be used by clients with more standard presentation requirements). There are several API-frameworks: e.g. frameworks supporting WSDL (= "Web Services Description language", an XML-based technology), JSON-RPC, XML-RPC, ReST, ... From steve+python at pearwood.info Mon Oct 23 03:45:17 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 23 Oct 2017 18:45:17 +1100 Subject: grapheme cluster library References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> Message-ID: <59ed9e0e$0$18578$b1db1813$d948b532@news.astraweb.com> On Mon, 23 Oct 2017 05:47 pm, Rustom Mody wrote: > On Monday, October 23, 2017 at 8:06:03 AM UTC+5:30, Lawrence D?Oliveiro > wrote: [...] >> Bear in mind that the logical representation of the text is as code points, >> graphemes would have more to do with rendering. > > Heh! Speak of Euro/Anglo-centrism! I think that Lawrence may be thinking of glyphs. Glyphs are the display form that are rendered. Graphemes are the smallest unit of written language. > In a sane world graphemes would be called letters Graphemes *aren't* letters. For starters, not all written languages have an alphabet. No alphabet, no letters. Even in languages with an alphabet, not all graphemes are letters. Graphemes include: - logograms (symbols which represent a morpheme, an entire word, or a phrase), e.g. Chinese characters, ampersand &, the ? trademark or ? registered trademark symbols; - syllabic characters such as Japanese kana or Cherokee; - letters of alphabets; - letters with added diacritics; - punctuation marks; - mathematical symbols; - typographical symbols; - word separators; and more. Many linguists also include digraphs (pairs of letters) like the English "th", "sh", "qu", or "gh" as graphemes. https://www.thoughtco.com/what-is-a-grapheme-1690916 https://en.wikipedia.org/wiki/Grapheme > And unicode codepoints would be called something else ? letterlets?? > To be fair to the Unicode consortium, they strive hard to call them > codepoints But in an anglo-centric world, the conflation of codepoint to > letter is inevitable I guess. To hear how a non Roman-centric view of the > world would sound: A 'w' is a poorly double-struck 'u' > A 't' is a crossed 'l' > Reasonable? No, T is not a crossed L -- they are unrelated letters and the visual similarity is a coincidence. They are no more connected than E is just an F with an extra line. But you are more right than you knew regarding W: it *literally was* a doubled-up V (sometimes written U) once upon a time. For a long time W did not appear in the Latin alphabet, even after people used it in written text. It was considered a digraph VV then a ligature and finally, only gradually, a proper letter. As late as the 16th century the German grammatican Valentin Ickelshamer complained that hardly anyone, including school masters, knew what to do with W or what it was called. https://en.wikipedia.org/wiki/W#History > The lead of https://en.wikipedia.org/wiki/%C3%9C has > > | ?, or ?, is a character?classified as a separate letter in several > | extended Latin alphabets > | (including Azeri, Estonian, Hungarian and Turkish), but as the letter U > | with an umlaut/diaeresis in others such as Catalan, French, Galician, > | German, Occitan and Spanish. Indeed: sometimes the same grapheme is considered a letter in one language and a letter-plus-diacritic in another. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From Karsten.Hilbert at gmx.net Mon Oct 23 04:12:21 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Mon, 23 Oct 2017 10:12:21 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171022211033.xf656nn3ha6v7j2a@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> <87y3o5gdn9.fsf@handshake.de> <20171022211033.xf656nn3ha6v7j2a@hermes.hilbert.loc> Message-ID: <20171023081221.xalbvrjxiw7ztv2q@hermes.hilbert.loc> On Sun, Oct 22, 2017 at 11:10:33PM +0200, Karsten Hilbert wrote: >> It points to a memory corruption. > > thanks for your guidance. I fear this approach is out of my class. For what it's worth I have run a successful overnight memory stress test (memtest86+) so it shouldn't be a bad bit in hardware. Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From kevinbongers89 at web.de Mon Oct 23 04:55:48 2017 From: kevinbongers89 at web.de (Kevin van Keeken) Date: Mon, 23 Oct 2017 10:55:48 +0200 Subject: choice of web-framework In-Reply-To: <8760b62v39.fsf@handshake.de> References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> <8760b62v39.fsf@handshake.de> Message-ID: <8d5210ad-561d-5268-3cc0-00ff1754add9@web.de> Greetings, while reading through this topic i would like to know, if cherrypy is a viable web-framework as well? I stumbled upon this project a while ago, but didn't read through it in detail and would like to hear something about it, especially in regards to the project requirements. Kind regards > Patrick Vrijlandt writes: >> ... >> The project is completely new, there are no histories to take into >> account (current solutions are paper-based). The website involves >> questionnaires that will be developed, filled out and stored. Users >> are not programmers or developers. They should be >> authenticated. Version control is required. Internationalization is >> not an issue. I expect that the project will add additional >> requirements and complexity later on that I can not foresee yet. I'm >> targeting a single deployment (maybe a second on a development >> machine). I usually work on Windows, but Linux can be considered. > I am using Plone (a CMS (= Content Management System) build on top of > Zope) for something like this. It has a standard extension > "CMFEditions" for version control of its content. > > The content is managed in the Zope Object Database (= "ZODB"). This is > also valid for the revisions. Thus, you do not get > "git/mercurial/..."-style version control (based on files) but > you see when and by whom a version was created, can revert to a > previous version and see differences between versions. There is no > merge support, though. > > The standard Plone content types are likely not sufficient to > implement your questionnaires; you will probably define one of > more specific for your task. But, this is not too complex. > > Questions about Plone (and the underlying Zope) can be asked at > "https://community.plone.org/". > > From danceswithnumbers at gmail.com Mon Oct 23 05:06:57 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 02:06:57 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> Message-ID: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> Ridiculous? Ludicrous?? Harsh words! First let me clarify before you lump this in with perpetual motion, or cold fusion. It is a mapping solution to compress ANY i repeat ANY random file with numbers of only 0 - 9 such as are in the million rand numbers page. Entirely possible. Since i did it, not by hiding data anywhere then it must be conjecture not law. Jon J Hutton From rosuav at gmail.com Mon Oct 23 05:21:54 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Oct 2017 20:21:54 +1100 Subject: Compression of random binary data In-Reply-To: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> Message-ID: On Mon, Oct 23, 2017 at 8:06 PM, wrote: > Ridiculous? Ludicrous?? > > Harsh words! First let me clarify before you lump this in with perpetual motion, or cold fusion. It is a mapping solution to compress ANY i repeat ANY random file with numbers of only 0 - 9 such as are in the million rand numbers page. Entirely possible. Since i did it, not by hiding data anywhere then it must be conjecture not law. > Where did you get the idea that you start with a representation that uses ten symbols (0-9) and uses an entire byte for each one? Where is that stated? ChrisA From danceswithnumbers at gmail.com Mon Oct 23 05:32:19 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 02:32:19 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> Message-ID: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> According to this website. This is an uncompressable stream. https://en.m.wikipedia.org/wiki/Incompressible_string 1234999988884321 It only takes seven 8 bit bytes to represent this From p.f.moore at gmail.com Mon Oct 23 05:41:55 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 23 Oct 2017 10:41:55 +0100 Subject: Compression of random binary data In-Reply-To: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On 23 October 2017 at 10:32, wrote: > According to this website. This is an uncompressable stream. > > https://en.m.wikipedia.org/wiki/Incompressible_string > > 1234999988884321 > > It only takes seven 8 bit bytes to represent this Would you care to provide the seven 8-bit bytes you propose to use? Paul From rosuav at gmail.com Mon Oct 23 05:46:30 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Oct 2017 20:46:30 +1100 Subject: Compression of random binary data In-Reply-To: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On Mon, Oct 23, 2017 at 8:32 PM, wrote: > According to this website. This is an uncompressable stream. > > https://en.m.wikipedia.org/wiki/Incompressible_string > > 1234999988884321 > > It only takes seven 8 bit bytes to represent this That page says nothing about using a byte to represent each digit. So if you're going to compress the data into bytes, you then have to represent your compressed data in some comparable way. The simplest representation is to treat this as a single number; in hexadecimal, it would be 46339d7a29361, and if you want a byte representation of a number, the most trivial way is to take each pair of hex digits and store the corresponding byte: b"\x4\x63\x39\xd7\xa2\x93\x61" is seven bytes (six and a half, rounded up). This is not data compression; this is the more general question of how you store information at the concrete level. ChrisA From danceswithnumbers at gmail.com Mon Oct 23 05:54:25 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 02:54:25 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <041361c6-15f6-4b3f-be2d-cabb912f72a9@googlegroups.com> I really do not think this has a value besides being a trinket or cute toy. Like i said i can not see how it can be adapted to work as a rand binary compression alg...it only works with 0-9 in any seq. It's taken me six years to solve, but so what. Jon Hutton danceswithnumbers at gmail.com From tjol at tjol.eu Mon Oct 23 05:54:34 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 11:54:34 +0200 Subject: Compression of random binary data In-Reply-To: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <1dcc0480-d458-5b42-5681-7b2fddeece74@tjol.eu> On 2017-10-23 11:32, danceswithnumbers at gmail.com wrote: > According to this website. This is an uncompressable stream. > > https://en.m.wikipedia.org/wiki/Incompressible_string > > 1234999988884321 No, it's not. According to that article, that string is incompressible by a particular algorithm. I can see no more general claims. > > It only takes seven 8 bit bytes to represent this > You amaze me. >>> bin(1234999988884321) '0b100011000110011100111010111101000101001001101100001' >>> len(bin(1234999988884321)[2:]) 51 >>> 7 * 8 56 From marko at pacujo.net Mon Oct 23 06:13:56 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 23 Oct 2017 13:13:56 +0300 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1dcc0480-d458-5b42-5681-7b2fddeece74@tjol.eu> Message-ID: <87o9oy41ej.fsf@elektro.pacujo.net> Thomas Jollans : > On 2017-10-23 11:32, danceswithnumbers at gmail.com wrote: >> According to this website. This is an uncompressable stream. >> >> https://en.m.wikipedia.org/wiki/Incompressible_string >> >> 1234999988884321 > > No, it's not. According to that article, that string is incompressible > by a particular algorithm. I can see no more general claims. Here's a compression algorithm that manages to compress that string into a 0-bit string: * If the original string is 1234999988884321 (whatever that means), return the empty bit string. * Otherwise, prepend a don't-care bit to the original string and return the result of the concatenation. >> It only takes seven 8 bit bytes to represent this > > You amaze me. > >>>> bin(1234999988884321) > '0b100011000110011100111010111101000101001001101100001' >>>> len(bin(1234999988884321)[2:]) > 51 >>>> 7 * 8 > 56 So danceswithnumbers made a correct statement! Anyway, a fun troll. Marko From ben.usenet at bsb.me.uk Mon Oct 23 07:15:18 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 23 Oct 2017 12:15:18 +0100 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> Message-ID: <87tvyqjet5.fsf@bsb.me.uk> danceswithnumbers at gmail.com writes: > ... First let me clarify before you lump this in with > perpetual motion, or cold fusion. It is a mapping solution to compress > ANY i repeat ANY random file with numbers of only 0 - 9 such as are in > the million rand numbers page. Entirely possible. Of course it is. I have half a dozen programs on this machine that do it. I don't need yours. It's possible you want to claim something beyond what you stated, but no one is going to get excited until you actually claim such a thing. -- Ben. From alister.ware at ntlworld.com Mon Oct 23 08:18:57 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 23 Oct 2017 12:18:57 GMT Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On Mon, 23 Oct 2017 10:41:55 +0100, Paul Moore wrote: > On 23 October 2017 at 10:32, wrote: >> According to this website. This is an uncompressable stream. >> >> https://en.m.wikipedia.org/wiki/Incompressible_string >> >> 1234999988884321 >> >> It only takes seven 8 bit bytes to represent this > > Would you care to provide the seven 8-bit bytes you propose to use? > Paul I would suspect he is using BCD & storing 2 values in reach byte that is not what is meant by you cant compress random data. his compression is simply removing redundant space from an inefficient coding Can you compress that sequence on paper when you only have the values 0-9 to work with (forget computer representation & removing un-used bits) -- Old age and treachery will overcome youth and skill. From Karsten.Hilbert at gmx.net Mon Oct 23 08:23:12 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Mon, 23 Oct 2017 14:23:12 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <87y3o5gdn9.fsf@handshake.de> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> <87y3o5gdn9.fsf@handshake.de> Message-ID: <20171023122312.uupsxzhdwat3stt7@hermes.hilbert.loc> On Sat, Oct 21, 2017 at 09:31:54AM +0200, dieter wrote: > It points to a memory corruption. While running valgrind and friends is beyond my capabilitis I have run the problem through gdb to at least obtain a stack trace to see what gives: ... ==> verifying target database schema ... ==> checking migrated data for plausibility ... Done bootstrapping GNUmed database: We very likely succeeded. log: /home/ncq/Projekte/gm-git/gnumed/gnumed/server/bootstrap/bootstrap-latest.log Debug memory block at address p=0x6aab7c: API '' 0 bytes originally requested The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): at p-3: 0x33 *** OUCH at p-2: 0x47 *** OUCH at p-1: 0x00 *** OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 4 pad bytes at tail=0x6aab7c are not all FORBIDDENBYTE (0xfb): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH The block was made by call #0 to debug malloc/realloc. Fatal Python error: bad ID: Allocated using API '', verified using API 'o' Program received signal SIGABRT, Aborted. 0xb7fd9ce9 in __kernel_vsyscall () (gdb) bt #0 0xb7fd9ce9 in __kernel_vsyscall () #1 0xb7d6edd0 in __libc_signal_restore_set (set=0xbfffed40) at ../sysdeps/unix/sysv/linux/nptl-signals.h:79 #2 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48 #3 0xb7d70297 in __GI_abort () at abort.c:89 #4 0x0055fb74 in Py_FatalError (msg=0xbfffefec "bad ID: Allocated using API '\037', verified using API 'o'") at ../Python/pythonrun.c:1700 #5 0x00499adb in _PyObject_DebugCheckAddressApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1640 #6 0x004997a5 in _PyObject_DebugFreeApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1527 #7 0x0049964f in _PyObject_DebugFree (p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1471 #8 0x00471043 in int_dealloc (v=0x6aab7c <_Py_ZeroStruct>) at ../Objects/intobject.c:139 #9 0x00497bee in _Py_Dealloc (op=False) at ../Objects/object.c:2262 #10 0x00489bad in dict_dealloc (mp=0xb7888f34) at ../Objects/dictobject.c:1085 Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: #11 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: #12 0x004b9ab7 in subtype_dealloc (self=) at ../Objects/typeobject.c:1035 Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: #13 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 #14 0x0044dc7a in instancemethod_dealloc (im=0xb78984b4) at ../Objects/classobject.c:2388 #15 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 #16 0x004885d7 in insertdict_by_entry (mp=0xb78880d4, key='_shutdown', hash=598970216, ep=0x88d6d4, value=None) at ../Objects/dictobject.c:519 #17 0x00488857 in insertdict (mp=0xb78880d4, key='_shutdown', hash=598970216, value=None) at ../Objects/dictobject.c:556 #18 0x0048910f in dict_set_item_by_hash_or_entry ( op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': , '__doc__': "Thread modul...(truncated), key='_shutdown', hash=598970216, ep=0x0, value=None) at ../Objects/dictobject.c:795 #19 0x00489285 in PyDict_SetItem ( op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': , '__doc__': "Thread modul...(truncated), key='_shutdown', value=None) at ../Objects/dictobject.c:848 #20 0x00492758 in _PyModule_Clear (m=) at ../Objects/moduleobject.c:125 #21 0x0054a33b in PyImport_Cleanup () at ../Python/import.c:530 #22 0x0055c53c in Py_Finalize () at ../Python/pythonrun.c:458 #23 0x0055fe9c in Py_Exit (sts=0) at ../Python/pythonrun.c:1783 #24 0x0055e0fc in handle_system_exit () at ../Python/pythonrun.c:1151 #25 0x0055e152 in PyErr_PrintEx (set_sys_last_vars=1) at ../Python/pythonrun.c:1161 #26 0x0055dd5b in PyErr_Print () at ../Python/pythonrun.c:1064 #27 0x0055d61f in PyRun_SimpleFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:952 #28 0x0055cc4e in PyRun_AnyFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:752 #29 0x00577cb0 in Py_Main (argc=5, argv=0xbffff684) at ../Modules/main.c:645 #30 0x004259c8 in main (argc=5, argv=0xbffff684) at ../Modules/python.c:20 (gdb) quit A debugging session is active. Inferior 1 [process 32024] will be killed. Quit anyway? (y or n) y Dropping obsoleted staging database gnumed_v2 ... (you may need to provide the password for root) Dropping obsoleted staging database gnumed_v3 ... ... Vague hints from the above might suggest that I go look for whether I do have a second thread and where improperly formed unicode data might come from. Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From obulesu.t at gmail.com Mon Oct 23 08:33:40 2017 From: obulesu.t at gmail.com (T Obulesu) Date: Mon, 23 Oct 2017 05:33:40 -0700 (PDT) Subject: Sockets but calling from different programs Message-ID: <938a5a08-eebe-4a21-a621-3507019c83d2@googlegroups.com> I'm new to python3 and scratching my head to write a program for this logic: classA.py Class A: # class for socket communication basic init method that initializes port, address, connection method send(message): # for sending any message through the given port method receive(): # will be keep on listening for incoming messages classB.py Class B: import the send method from class A send the messages from this class classC.py Class C: import the receive method from the class A receive all the messages from the same socket from here. Note: classA.py, classB.py, ClassC.py are saved in different locations. Can someone help me in writing the code and how can I create a single object and use it in multiple classed? This is how my code looks like: import socket import select class ChatServer: def __init__( self): self.port = 8880; self.srvsock = socket.socket( socket.AF_INET, socket.SOCK_STREAM ) self.srvsock.setsockopt( socket.SOL_SOCKET, socket.SO_REUSEADDR, 1 ) self.srvsock.bind( ("", 8880) ) self.srvsock.listen( 5 ) print("New socket bind") while 1: print("Waiting for new connection") self.sock,(remhost,remport)=self.accept_new_connection() print ('ChatServer started on port %s' % port) def send(self,data): self.sock.send(data) def receieve(self): self.str=sock.recv(1024) print(self.str) I want to run this code first time when my project starts and want to call send() and receive() functions from other two different python files located in different path. From rosuav at gmail.com Mon Oct 23 08:36:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 23 Oct 2017 23:36:05 +1100 Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On Mon, Oct 23, 2017 at 11:18 PM, alister via Python-list wrote: > On Mon, 23 Oct 2017 10:41:55 +0100, Paul Moore wrote: > >> On 23 October 2017 at 10:32, wrote: >>> According to this website. This is an uncompressable stream. >>> >>> https://en.m.wikipedia.org/wiki/Incompressible_string >>> >>> 1234999988884321 >>> >>> It only takes seven 8 bit bytes to represent this >> >> Would you care to provide the seven 8-bit bytes you propose to use? >> Paul > > I would suspect he is using BCD & storing 2 values in reach byte > that is not what is meant by you cant compress random data. > his compression is simply removing redundant space from an inefficient > coding I suspect he is using ASCII and storing one value in each byte. ChrisA From tjol at tjol.eu Mon Oct 23 09:26:18 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 15:26:18 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171023122312.uupsxzhdwat3stt7@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> <87y3o5gdn9.fsf@handshake.de> <20171023122312.uupsxzhdwat3stt7@hermes.hilbert.loc> Message-ID: <28cfa2c6-4716-6b12-b4f0-4a24386c359a@tjol.eu> On 2017-10-23 14:23, Karsten Hilbert wrote: > On Sat, Oct 21, 2017 at 09:31:54AM +0200, dieter wrote: > >> It points to a memory corruption. > > While running valgrind and friends is beyond my capabilitis I > have run the problem through gdb to at least obtain a stack > trace to see what gives: I wouldn't read too much into that. You know that somehow some memory got corrupted. You know at which point Python trips over the error - but that doesn't really tell you anything about how the memory initially got corrupted. > > ... > ==> verifying target database schema ... > ==> checking migrated data for plausibility ... > Done bootstrapping GNUmed database: We very likely succeeded. > log: /home/ncq/Projekte/gm-git/gnumed/gnumed/server/bootstrap/bootstrap-latest.log > Debug memory block at address p=0x6aab7c: API '' > 0 bytes originally requested > The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): > at p-3: 0x33 *** OUCH > at p-2: 0x47 *** OUCH > at p-1: 0x00 *** OUCH > Because memory is corrupted at the start, the count of bytes requested > may be bogus, and checking the trailing pad bytes may segfault. > The 4 pad bytes at tail=0x6aab7c are not all FORBIDDENBYTE (0xfb): > at tail+0: 0x00 *** OUCH > at tail+1: 0x00 *** OUCH > at tail+2: 0x00 *** OUCH > at tail+3: 0x00 *** OUCH > The block was made by call #0 to debug malloc/realloc. > Fatal Python error: bad ID: Allocated using API '', verified using API 'o' > > Program received signal SIGABRT, Aborted. > 0xb7fd9ce9 in __kernel_vsyscall () > (gdb) bt > #0 0xb7fd9ce9 in __kernel_vsyscall () > #1 0xb7d6edd0 in __libc_signal_restore_set (set=0xbfffed40) at ../sysdeps/unix/sysv/linux/nptl-signals.h:79 > #2 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48 > #3 0xb7d70297 in __GI_abort () at abort.c:89 > #4 0x0055fb74 in Py_FatalError (msg=0xbfffefec "bad ID: Allocated using API '\037', verified using API 'o'") at ../Python/pythonrun.c:1700 > #5 0x00499adb in _PyObject_DebugCheckAddressApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1640 > #6 0x004997a5 in _PyObject_DebugFreeApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1527 > #7 0x0049964f in _PyObject_DebugFree (p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1471 > #8 0x00471043 in int_dealloc (v=0x6aab7c <_Py_ZeroStruct>) at ../Objects/intobject.c:139 > #9 0x00497bee in _Py_Dealloc (op=False) at ../Objects/object.c:2262 > #10 0x00489bad in dict_dealloc (mp=0xb7888f34) at ../Objects/dictobject.c:1085 > Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: > #11 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 > Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: > #12 0x004b9ab7 in subtype_dealloc (self=) at ../Objects/typeobject.c:1035 > Python Exception 'utf-8' codec can't decode bytes in position 1-2: unexpected end of data: > #13 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 > #14 0x0044dc7a in instancemethod_dealloc (im=0xb78984b4) at ../Objects/classobject.c:2388 > #15 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 > #16 0x004885d7 in insertdict_by_entry (mp=0xb78880d4, key='_shutdown', hash=598970216, ep=0x88d6d4, value=None) at ../Objects/dictobject.c:519 > #17 0x00488857 in insertdict (mp=0xb78880d4, key='_shutdown', hash=598970216, value=None) at ../Objects/dictobject.c:556 > #18 0x0048910f in dict_set_item_by_hash_or_entry ( > op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': x6fd420>, '__doc__': "Thread modul...(truncated), key='_shutdown', hash=598970216, ep=0x0, value=None) at ../Objects/dictobject.c:795 > #19 0x00489285 in PyDict_SetItem ( > op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': x6fd420>, '__doc__': "Thread modul...(truncated), key='_shutdown', value=None) at ../Objects/dictobject.c:848 > #20 0x00492758 in _PyModule_Clear (m=) at ../Objects/moduleobject.c:125 > #21 0x0054a33b in PyImport_Cleanup () at ../Python/import.c:530 > #22 0x0055c53c in Py_Finalize () at ../Python/pythonrun.c:458 > #23 0x0055fe9c in Py_Exit (sts=0) at ../Python/pythonrun.c:1783 > #24 0x0055e0fc in handle_system_exit () at ../Python/pythonrun.c:1151 > #25 0x0055e152 in PyErr_PrintEx (set_sys_last_vars=1) at ../Python/pythonrun.c:1161 > #26 0x0055dd5b in PyErr_Print () at ../Python/pythonrun.c:1064 > #27 0x0055d61f in PyRun_SimpleFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:952 > #28 0x0055cc4e in PyRun_AnyFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:752 > #29 0x00577cb0 in Py_Main (argc=5, argv=0xbffff684) at ../Modules/main.c:645 > #30 0x004259c8 in main (argc=5, argv=0xbffff684) at ../Modules/python.c:20 > (gdb) quit > A debugging session is active. > > Inferior 1 [process 32024] will be killed. > > Quit anyway? (y or n) y > Dropping obsoleted staging database gnumed_v2 ... > (you may need to provide the password for root) > Dropping obsoleted staging database gnumed_v3 ... > ... > > Vague hints from the above might suggest that I go look for > whether I do have a second thread and where improperly formed > unicode data might come from. > > Karsten > From Karsten.Hilbert at gmx.net Mon Oct 23 09:38:24 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Mon, 23 Oct 2017 15:38:24 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <28cfa2c6-4716-6b12-b4f0-4a24386c359a@tjol.eu> References: <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <20171018114358.fzj4vvsyzrf6rihu@hermes.hilbert.loc> <47a09a35-f929-528f-3110-8d660e6fb46f@tjol.eu> <20171019172745.76i3lltjtck2m5gw@hermes.hilbert.loc> <20171019174907.iqgznoroyieipend@hermes.hilbert.loc> <87y3o5gdn9.fsf@handshake.de> <20171023122312.uupsxzhdwat3stt7@hermes.hilbert.loc> <28cfa2c6-4716-6b12-b4f0-4a24386c359a@tjol.eu> Message-ID: <20171023133824.it7kdeo7rw4puizx@hermes.hilbert.loc> On Mon, Oct 23, 2017 at 03:26:18PM +0200, Thomas Jollans wrote: > >> It points to a memory corruption. > > > > While running valgrind and friends is beyond my capabilitis I > > have run the problem through gdb to at least obtain a stack > > trace to see what gives: > > I wouldn't read too much into that. You know that somehow some memory > got corrupted. You know at which point Python trips over the error - but > that doesn't really tell you anything about how the memory initially got > corrupted. Thanks, I know. However, I am grasping for straws here since I am unable to "properly" debug the python binary as such. Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From neilc at norwich.edu Mon Oct 23 09:40:59 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Mon, 23 Oct 2017 13:40:59 +0000 (UTC) Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On 2017-10-23, Chris Angelico wrote: > On Mon, Oct 23, 2017 at 11:18 PM, alister via Python-list > wrote: >> On Mon, 23 Oct 2017 10:41:55 +0100, Paul Moore wrote: >> >>> On 23 October 2017 at 10:32, >>> wrote: >>>> According to this website. This is an uncompressable stream. >>>> >>>> https://en.m.wikipedia.org/wiki/Incompressible_string >>>> >>>> 1234999988884321 >>>> >>>> It only takes seven 8 bit bytes to represent this >>> >>> Would you care to provide the seven 8-bit bytes you propose >>> to use? >>> Paul >> >> I would suspect he is using BCD & storing 2 values in reach >> byte that is not what is meant by you cant compress random >> data. his compression is simply removing redundant space from >> an inefficient coding > > I suspect he is using ASCII and storing one value in each byte. There's also ZSCII, which stores roughly 3 characters every 2 bytes. Since all the digits are in A2, this sequence would take up 7 bytes in ZSCII as well. http://inform-fiction.org/zmachine/standards/z1point0/sect03.html -- Neil Cerutti From alister.ware at ntlworld.com Mon Oct 23 10:11:28 2017 From: alister.ware at ntlworld.com (alister) Date: Mon, 23 Oct 2017 14:11:28 GMT Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On Mon, 23 Oct 2017 13:40:59 +0000, Neil Cerutti wrote: > On 2017-10-23, Chris Angelico wrote: >> On Mon, Oct 23, 2017 at 11:18 PM, alister via Python-list >> wrote: >>> On Mon, 23 Oct 2017 10:41:55 +0100, Paul Moore wrote: >>> >>>> On 23 October 2017 at 10:32, >>>> wrote: >>>>> According to this website. This is an uncompressable stream. >>>>> >>>>> https://en.m.wikipedia.org/wiki/Incompressible_string >>>>> >>>>> 1234999988884321 >>>>> >>>>> It only takes seven 8 bit bytes to represent this >>>> >>>> Would you care to provide the seven 8-bit bytes you propose to use? >>>> Paul >>> >>> I would suspect he is using BCD & storing 2 values in reach byte that >>> is not what is meant by you cant compress random data. his compression >>> is simply removing redundant space from an inefficient coding >> >> I suspect he is using ASCII and storing one value in each byte. > > There's also ZSCII, which stores roughly 3 characters every 2 bytes. > Since all the digits are in A2, this sequence would take up 7 bytes in > ZSCII as well. > > http://inform-fiction.org/zmachine/standards/z1point0/sect03.html not sure how 16 characters can be represented by either ascii or zscii in only 8 bytes -- I fear explanations explanatory of things explained. From daniel at tangemann-plus.de Mon Oct 23 10:23:04 2017 From: daniel at tangemann-plus.de (Daniel Tangemann) Date: Mon, 23 Oct 2017 16:23:04 +0200 (CEST) Subject: IDLE doesn't recognise installed packages Message-ID: <2096431256.470671.1508768584232@webmail.strato.com> hi, I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message that it's already installed. how do I fix this? cheers From rustompmody at gmail.com Mon Oct 23 10:25:29 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 23 Oct 2017 07:25:29 -0700 (PDT) Subject: grapheme cluster library In-Reply-To: <59ed9e0e$0$18578$b1db1813$d948b532@news.astraweb.com> References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> <59ed9e0e$0$18578$b1db1813$d948b532@news.astraweb.com> Message-ID: On Monday, October 23, 2017 at 1:15:35 PM UTC+5:30, Steve D'Aprano wrote: > On Mon, 23 Oct 2017 05:47 pm, Rustom Mody wrote: > > > On Monday, October 23, 2017 at 8:06:03 AM UTC+5:30, Lawrence D?Oliveiro > > wrote: > [...] > >> Bear in mind that the logical representation of the text is as code points, > >> graphemes would have more to do with rendering. > > > > Heh! Speak of Euro/Anglo-centrism! > > I think that Lawrence may be thinking of glyphs. Glyphs are the display form > that are rendered. Graphemes are the smallest unit of written language. > > > > In a sane world graphemes would be called letters > > Graphemes *aren't* letters. > > For starters, not all written languages have an alphabet. No alphabet, no > letters. Even in languages with an alphabet, not all graphemes are letters. > > Graphemes include: > > - logograms (symbols which represent a morpheme, an entire word, or > a phrase), e.g. Chinese characters, ampersand &, the ? trademark > or ? registered trademark symbols; > > - syllabic characters such as Japanese kana or Cherokee; > > - letters of alphabets; > > - letters with added diacritics; > > - punctuation marks; > > - mathematical symbols; > > - typographical symbols; > > - word separators; > > and more. Many linguists also include digraphs (pairs of letters) like the > English "th", "sh", "qu", or "gh" as graphemes. > > > https://www.thoughtco.com/what-is-a-grapheme-1690916 > > https://en.wikipedia.org/wiki/Grapheme Um? Ok So I am using the wrong word? Your first link says: | For example, the word 'ghost' contains five letters and four graphemes | ('gh,' 'o,' 's,' and 't') Whereas new regex findall does: >>> findall(r'\X', "ghost") ['g', 'h', 'o', 's', 't'] >>> findall(r'\X', "church") ['c', 'h', 'u', 'r', 'c', 'h'] From danceswithnumbers at gmail.com Mon Oct 23 10:29:26 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 07:29:26 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> I'm really not trolling, and even though some are sarcastic i sm learning from your comments. Dec to bin is not bad at removing wasted space, but there is a better way. Here is an example. How would you compress these numbers. If you look for redundancy and then code to a bulky dictionary or change it to binary, one would unflate the other would only get it down to Compress this: 4135124325 Bin to dec...still very large 11110110 01111000 11111101 01100101 New compression method: 11000101 11000111 01001111 A full byte less than bin. I know many are skeptical....thats okay.this has taken 6 years, im not going to insult your intelligence with something so juvenile as dec to bin. I'm really trying to find an application for this since it only deals with digits 0-9 or 0-20 or other strange combinations. Wait did you just give it away in that small example....no, because it also encrypts in a way that has never been done. The compression is better than log(256)?log (10)....wait isn't that impossible, binary is minimalistic. I agree that binary is minimalistic, but the architecture is not, making all arguments conjecture...not laws. From p.f.moore at gmail.com Mon Oct 23 11:28:59 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 23 Oct 2017 16:28:59 +0100 Subject: Compression of random binary data In-Reply-To: <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: On 23 October 2017 at 15:29, wrote: > I'm really not trolling, and even though some are sarcastic i sm learning from your comments. I'm willing to believe that, but if you're trying to claim you have "compressed" data (in a way that satisfies the technical, information-theoretic meaning of the word), you need to be *really* careful not to include hidden information in your assumptions. For example, if I have a string made up of only the numbers 0-7, then I can trivially (octal) store that data in 3 bits per digit. But that's not compression, as there's "hidden information" in the knowledge that you're using a restricted character set. Factor in that information and your string only contains 3 bits of information per digit. Using bytes (characters, if you assume a 1-byte encoding) to hold just the digits 0-9 is inefficient (there's 256 bytes and you're only using 10 of them), and "of course" you can hold that data more efficiently. But that's not "compression", that's simply using a better encoding. In the technical sense, "compression" is about looking at redundancies that go beyond the case of how effectively you pack data into the bytes available. > Dec to bin is not bad at removing wasted space Yep, you're not talking about what people usually refer to as compression, but rather about optimising an encoding. >, but there is a better way. Here is an example. How would you compress these numbers. 10 digits = log2(10) bits of information. So getting down to 4 bits is about encoding. You can go further by using a variable length encoding and "extra knowledge" about which digits come up most commonly to give the common digits shorter representation. That's called Gray coding. You can use the fact that repeated copies of the same digit come up together a lot to replace them by digit + count. That's run-length encoding. There are other more complex approaches. But what *all* of these have in common is that if you provide random input (within the limits of what you support - digit strings here) then you'll *always* get at least one input that encodes to a longer output than your input. > Compress this: > > 4135124325 10 x 10 digits = 10 x log2(10) bits of information = a bit under 34 bits of information > > Bin to dec...still very large > 11110110 > 01111000 > 11111101 > 01100101 4x8 = 32 bits, but there's probably a couple of leading zeros needed if you want to encode all 10-digit numbers. > New compression method: > > 11000101 > 11000111 > 01001111 > > A full byte less than bin. You'll need to explain how to decode this, in a way that can be used to decode the encodings of arbitrary 10-digit numbers, and with any leading zeroes that are needed to cover the whole space of 10-digit numbers, before you can claim you've compressed 10-digit numbers using only 24 bits. And if you do that, you've basically overturned most of information theory, so I'd start by assuming there's a flaw in your argument - sorry about that... ;-) Hope this helps put the subject into context. Compression is a very technical subject, to "do it right". Special cases can be worked out, sure, but the "hidden assumptions" in a method are what make the difference between a "compression algorithm" and a "way of storing my particular data more efficiently". > I know many are skeptical....thats okay.this has taken 6 years, im not going to insult your intelligence with something so juvenile as dec to bin. I'm really trying to find an application for this since it only deals with digits 0-9 or 0-20 or other strange combinations. Wait did you just give it away in that small example....no, because it also encrypts in a way that has never been done. The compression is better than log(256)?log (10)....wait isn't that impossible, binary is minimalistic. I agree that binary is minimalistic, but the architecture is not, making all arguments conjecture...not laws. No-one is going to accept a claim that an algorithm you're not willing to publish is valid. This is about maths/science, not "proprietary algorithms" or anything like that. If you don't publish your methods, people will simply point at information theoretic proofs and say "either you're missing something, or your approach doesn't work in cases that I care about, so thanks but no thanks". Paul From danceswithnumbers at gmail.com Mon Oct 23 11:32:27 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 08:32:27 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: <9c21057a-c5f7-49c6-b978-bf3f27a436b7@googlegroups.com> Just trying to find a practical application for this alg. Not real useful as it stands now. Jon Hutton From danceswithnumbers at gmail.com Mon Oct 23 11:39:41 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 08:39:41 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: Thanks Paul...blunt to the point. My 8 year old can decode this back into base 10, i still have to help him a bit going from base 10 to 8 bit bytes....it's incredibly simple to decode. No dictionary, can easily be done with pencil and paper, does not rely on redundancies. Jon Hutton From tjol at tjol.eu Mon Oct 23 12:12:34 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 18:12:34 +0200 Subject: Compression of random binary data In-Reply-To: <59ed8099$0$18561$b1db1813$d948b532@news.astraweb.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed8099$0$18561$b1db1813$d948b532@news.astraweb.com> Message-ID: <1550059a-f017-08f9-4d31-c8b4e404a66b@tjol.eu> On 2017-10-23 07:39, Steve D'Aprano wrote: > By the way: here is a very clever trick for hiding information in the file > system: > > http://www.patrickcraig.co.uk/other/compression.php > > > but as people point out, the information in the file, plus the information in > the file system, ends up being *larger* than the original. Well done to > Patrick Craig for finding a clever loophole to "win" the challenge, but he > did so without actually compressing the original data. Thanks Steve that was very entertaining. -- Thomas Jollans From immenige2 at gmail.com Mon Oct 23 12:29:19 2017 From: immenige2 at gmail.com (=?UTF-8?B?7J6E7ZiE7KSA?=) Date: Mon, 23 Oct 2017 09:29:19 -0700 (PDT) Subject: I used list, def. why li += [100, 200] , and li = li + [100, 200] is different Message-ID: I am a Korean student, and I am a beginner in English and Python.;( I can't understand about this def If I want to print [1,2,3,4,5] [1,2,3,4,5,100,200] I will make code like this, and I can understand code. def modify(li): li += [100,200] list = [1,2,3,4,5] print(list) modify(list) print(list) BUT, when I make code like this. I will make code like this, and I can understand code. def modify(li): li = li + [100,200] list = [1,2,3,4,5] print(list) modify(list) print(list) python print [1,2,3,4,5] [1,2,3,4,5] why 'li+= [100,200]'and 'li = li + [100,200]' 's print is different please help me From tjol at tjol.eu Mon Oct 23 12:34:18 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 18:34:18 +0200 Subject: Compression of random binary data In-Reply-To: References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: <4a23298c-80ff-a02c-6265-6476ae5fa686@tjol.eu> On 2017-10-23 17:39, danceswithnumbers at gmail.com wrote: > Thanks Paul...blunt to the point. > > My 8 year old can decode this back into base 10, i still have to help him a bit going from base 10 to 8 bit bytes....it's incredibly simple to decode. No dictionary, can easily be done with pencil and paper, does not rely on redundancies. It is, is it? Well, you know the rules: "Code or it didn't happen." From joshmjacobson at gmail.com Mon Oct 23 12:34:52 2017 From: joshmjacobson at gmail.com (Josh Jacobson) Date: Mon, 23 Oct 2017 12:34:52 -0400 Subject: Windows alternative: multiprocessing.connection.wait on Pipe, Tkinter File Handlers Message-ID: The two functions in the subject are not fully implementable on Windows, and so I am looking for an alternative. Relevant SO postings including full code and description, with bounty: Freezes , Input ---- I have two scripts: *Processor_child.py*: Its purpose is to perform a number of data analysis and cleaning operations. This must perform the same operations when run alone (without Tkinter_parent.py) as it does when packaged into a GUI with Tkinter_parent.py. *Tkinter_parent.py*: Its purpose is to provide a GUI for those who can't use Processor_child directly.Within Processor_child, there are for loops that ask the user for input on each iteration. These prompts need to appear in the Tkinter app, accept the input, and send it back to Processor_child. ---- The main issue is having the script not move forward on execution until input has been sent. I know of three ways to theoretically solve this, but *none of these work in production on a Windows machine*: 1. while pipe1.poll() != True: time.sleep(0.1) This causes constant loading and a "freezing" like user experience. 2. multiprocessing.connection.wait >From the documentation: ""Windows: ... Note that pipe handles and socket handles are not waitable handles." 3. Tkinter file handlers >From the documentation: "This feature is not available on Windows." *Given that none of the 3 available options works on Windows, what options are available for Windows machines?* See Tkinter application 'freezes' while continually polling Pipe for contents (multiprocessing) and How can I implement an `input` method in a Tkinter parent script, with the displayed prompt and return value being sent back to a child script? for further description and code samples. From rgaddi at highlandtechnology.invalid Mon Oct 23 12:38:20 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 23 Oct 2017 09:38:20 -0700 Subject: I used list, def. why li += [100,200] , and li = li + [100,200] is different In-Reply-To: References: Message-ID: On 10/23/2017 09:29 AM, ??? wrote: > I am a Korean student, and I am a beginner in English and Python.;( > > I can't understand about this def > > If I want to print > > [1,2,3,4,5] > [1,2,3,4,5,100,200] > > I will make code like this, and I can understand code. > > > def modify(li): > li += [100,200] > > list = [1,2,3,4,5] > print(list) > modify(list) > print(list) > > > BUT, when I make code like this. > > I will make code like this, and I can understand code. > > > def modify(li): > li = li + [100,200] > > list = [1,2,3,4,5] > print(list) > modify(list) > print(list) > > > > python print > > [1,2,3,4,5] > [1,2,3,4,5] > > why 'li+= [100,200]'and 'li = li + [100,200]' 's print is different > please help me > Lists are mutable, they can be changed. Your call to += is equivalent to a call to li.extend([100, 200]), which changes the single existing object pointed to by the "li" reference. The second time, however, you take the existing value that "li" refers to [1,2,3,4,5], create a new object that is ([1,2,3,4,5] + [100,200]), and reassign the local reference "li" to point to that new object. Then your function ends, "li" goes out of scope, nothing points to that newly created object and it gets lost. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From python at mrabarnett.plus.com Mon Oct 23 12:59:07 2017 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 23 Oct 2017 17:59:07 +0100 Subject: I used list, def. why li += [100, 200] , and li = li + [100, 200] is different In-Reply-To: References: Message-ID: On 2017-10-23 17:29, ??? wrote: > I am a Korean student, and I am a beginner in English and Python.;( > > I can't understand about this def > > If I want to print > > [1,2,3,4,5] > [1,2,3,4,5,100,200] > > I will make code like this, and I can understand code. > > > def modify(li): > li += [100,200] > > list = [1,2,3,4,5] > print(list) > modify(list) > print(list) > > > BUT, when I make code like this. > > I will make code like this, and I can understand code. > > > def modify(li): > li = li + [100,200] > > list = [1,2,3,4,5] > print(list) > modify(list) > print(list) > > > > python print > > [1,2,3,4,5] > [1,2,3,4,5] > > why 'li+= [100,200]'and 'li = li + [100,200]' 's print is different > please help me > In both cases, you're binding a list to the local name "li". However, in the first example: li += [100,200] it's doing this: li = li.__iadd__([100,200]) The __iadd__ method is modifying the list li and then returning that modified list. The list that was passed in has been modified. In the second example: li = li + [100,200] it's doing this: li = li.__add__([100,200]) The __add__ method is creating a new list and then returning that new list. The local name "li" now refers to a new list, not the list that was passed in. When the function returns, the local name "li" is forgotten, and the new list that "li" referred to is discarded because there are no other references to it. From danceswithnumbers at gmail.com Mon Oct 23 13:13:16 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 10:13:16 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <4a23298c-80ff-a02c-6265-6476ae5fa686@tjol.eu> Message-ID: <3ef8452f-24e6-4c1c-87c5-3f38734a7e1d@googlegroups.com> Good point.... I hope it has a use, other than a cute toy....i don't see it yet. From BILL_NOSPAM at whoknows.net Mon Oct 23 13:24:28 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Mon, 23 Oct 2017 13:24:28 -0400 Subject: I used list, def. why li += [100,200] , and li = li + [100,200] is different In-Reply-To: References: Message-ID: Rob Gaddi wrote: > On 10/23/2017 09:29 AM, ??? wrote: >> I am a Korean student, and I am a beginner in English and Python.;( >> >> I can't understand about this def >> >> If I want to print >> >> [1,2,3,4,5] >> [1,2,3,4,5,100,200] >> >> I will make code like this, and I can understand code. >> >> >> def modify(li): >> li += [100,200] >> >> list = [1,2,3,4,5] >> print(list) >> modify(list) >> print(list) >> >> >> BUT, when I make code like this. >> >> I will make code like this, and I can understand code. >> >> >> def modify(li): >> li = li + [100,200] >> >> list = [1,2,3,4,5] >> print(list) >> modify(list) >> print(list) >> >> >> >> python print >> >> [1,2,3,4,5] >> [1,2,3,4,5] >> >> why 'li+= [100,200]'and 'li = li + [100,200]' 's print is different >> please help me >> > > Lists are mutable, they can be changed. Your call to += is equivalent > to a call to li.extend([100, 200]), which changes the single existing > object pointed to by the "li" reference. The second time, however, > you take the existing value that "li" refers to [1,2,3,4,5], create a > new object that is ([1,2,3,4,5] + [100,200]), and reassign the local > reference "li" to point to that new object. Then your function ends, > "li" goes out of scope, nothing points to that newly created object > and it gets lost. > The problem and both solutions are great! Thanks for posting! Bill From rhodri at kynesim.co.uk Mon Oct 23 13:27:28 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 23 Oct 2017 18:27:28 +0100 Subject: I used list, def. why li += [100, 200] , and li = li + [100, 200] is different In-Reply-To: References: Message-ID: <74c5429b-d922-293b-1ed2-741f3c61556a@kynesim.co.uk> On 23/10/17 17:29, ??? wrote: > I am a Korean student, and I am a beginner in English and Python.;( > > I can't understand about this def > > If I want to print > > [1,2,3,4,5] > [1,2,3,4,5,100,200] > > I will make code like this, and I can understand code. > > > def modify(li): > li += [100,200] > > list = [1,2,3,4,5] > print(list) > modify(list) > print(list) Here is approximately what Python does when you run this: * Create a list [1,2,3,4,5] * Give the list the name "list". In talking about Python we often say that it *binds* the name "list" to the list [1,2,3,4,5]. (It is usually a bad idea to use names like "list" that also have a meaning in Python, but we will ignore that for now.) list ----\ +-----------+ | 1,2,3,4,5 | +-----------+ * Print the object bound to the name "list", which is the list [1,2,3,4,5]. * Call modify(). This also binds the name "li" to the object that "list" is bound to. list ----\ +-----------+ | 1,2,3,4,5 | +-----------+ li ------/ * Create a new list [100,200] * "li += [100,200]" for a list means "change the list bound to the name 'li' by joining the new list onto the end of it. It is still the same object, it just has some more added to it. list ----\ +-------------------+ | 1,2,3,4,5,100,200 | +-------------------+ li ------/ "list" is still bound to the same object, so you see the changes outside the function modify(). > > BUT, when I make code like this. > > I will make code like this, and I can understand code. > > > def modify(li): > li = li + [100,200] What happens here is a little different. It starts off the same with "li" and "list" bound to the same object: list ----\ +-----------+ | 1,2,3,4,5 | +-----------+ li ------/ and a new object [100,200] that doesn't have a name. What happens next is: * Take the object bound to the name "li" and the unnamed object [100,200], and create a *new* object by joining them together. * Bind the name "li" to this new object list ----\ +-----------+ | 1,2,3,4,5 | +-----------+ li ------\ +-------------------+ | 1,2,3,4,5,100,200 | +-------------------+ Notice that it is only the name "li" that is bound. The name "list" still refers to the original object. When we return from the function, we stop using the name "li" and nothing refers to our new object [1,2,3,4,5,100,200] any more. Python will quietly delete it ("garbage collect") in the background. The key is that "+" on its own creates a new object, while "+=" alters the existing object. -- Rhodri James *-* Kynesim Ltd From neilc at norwich.edu Mon Oct 23 13:41:24 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Mon, 23 Oct 2017 17:41:24 +0000 (UTC) Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: On 2017-10-23, alister via Python-list wrote: >>>>>> 1234999988884321 >>>>>> >>>>>> It only takes seven 8 bit bytes to represent this >>>>> >>>>> Would you care to provide the seven 8-bit bytes you propose to use? >>>>> Paul >>>> >>>> I would suspect he is using BCD & storing 2 values in reach >>>> byte that is not what is meant by you cant compress random >>>> data. his compression is simply removing redundant space >>>> from an inefficient coding >>> >>> I suspect he is using ASCII and storing one value in each byte. >> >> There's also ZSCII, which stores roughly 3 characters every 2 >> bytes. Since all the digits are in A2, this sequence would >> take up 7 bytes in ZSCII as well. >> >> http://inform-fiction.org/zmachine/standards/z1point0/sect03.html > > not sure how 16 characters can be represented by either ascii > or zscii in only 8 bytes Oops! I hastily counted completely wrong. It's 10 bytes in ZSCII version 2, using a shift-lock. ----* 16 bits* ---- 12 -> 0 00101 01001 01010 349 -> 0 01011 01100 10001 999 -> 0 10001 10001 10001 888 -> 0 10000 10000 10000 843 -> 0 10000 01100 01011 21 -> 1 01010 01001 00101 -- Neil Cerutti From gengyangcai at gmail.com Mon Oct 23 15:28:07 2017 From: gengyangcai at gmail.com (Cai Gengyang) Date: Mon, 23 Oct 2017 12:28:07 -0700 (PDT) Subject: Searching For Old Posts On Python In-Reply-To: References: Message-ID: Right ... I am going to continue learning Python then ... On Thursday, October 19, 2017 at 3:39:44 AM UTC+8, Ian wrote: > On Mon, Oct 16, 2017 at 10:42 PM, Cai Gengyang wrote: > > https://duckduckgo.com/html/?q=%22Cai%20Gengyang%22%20python ---- This > > seems to be the only method that works, using DuckDuckGo > > Or any search engine, really. > https://www.google.com/search?q=site%3Apython.org+cai+gengyang From jblack at nopam.com Mon Oct 23 15:37:48 2017 From: jblack at nopam.com (John Black) Date: Mon, 23 Oct 2017 14:37:48 -0500 Subject: choice of web-framework References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: In article , rosuav at gmail.com says... > For the database, I generally use PostgreSQL, unless the job's so > simple it can be done with flat files. Using Flask with SQLAlchemy is > a popular option, but you can also dive into psycopg2 directly (the > "2" doesn't mean "Python 2.7 only", it's fine). The tech stack > Python+Flask+SQLAlchemy+PostgreSQL+Linux is an extremely solid one, or > you can switch out any part fairly easily. > > Disclaimer: I use Flask and SQLAlchemy when I'm teaching my students > how to build web apps in Python, but I don't have much experience with > any other frameworks or ORMs. Other options may very well be viable > and/or superior; all I can say is that the above *is* viable. Chris, thanks for all this detailed information. I am confused though with your database recommendation. You say you teach SQLAlchemy but generally use PostgreSQL yourself. I can maybe guess why there seems to be this contradiction. Perhaps PostgreSQL is better but too advanced for the class you are teaching? Can you clarify on which you think is the better choice? Thanks. John Black From danceswithnumbers at gmail.com Mon Oct 23 15:42:28 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 12:42:28 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> Wow, do programmers actually use zscii. That is huge. So much wated space. From kwpolska at gmail.com Mon Oct 23 15:57:19 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Mon, 23 Oct 2017 21:57:19 +0200 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On 23 October 2017 at 21:37, John Black wrote: > Chris, thanks for all this detailed information. I am confused though > with your database recommendation. You say you teach SQLAlchemy but > generally use PostgreSQL yourself. I can maybe guess why there seems to > be this contradiction. Perhaps PostgreSQL is better but too advanced for > the class you are teaching? Can you clarify on which you think is the > better choice? Thanks. Different Chris, but I?ll answer. Those are two very different things. PostgreSQL is a database server. It talks SQL to clients, stores data, retrieves it when asked. The usual stuff a database server does. Alternatives: SQLite, MySQL, MS SQL, Oracle DB, ? SQLAlchemy is an ORM: an object-relational mapper, and also a database toolkit. SQLAlchemy can abstract multiple database servers/engines (PostgreSQL, SQLite, MySQL, etc.) and work with them from the same codebase. It can also hide SQL from you and instead give you Python classes. If you use an ORM like SQLAlchemy, you get database support without writing a single line of SQL on your own. But you still need a database engine ? PostgreSQL can be one of them. But you can deploy the same code to different DB engines, and it will just work? (assuming you didn?t use any DB-specific features). Alternatives: Django ORM. psycopg2 is an example of a PostgreSQL client library for Python. It implements the Python DB-API and lets you use it to talk to a PostgreSQL server. When using psycopg2, you?re responsible for writing your own SQL statements for the server to execute. In that approach, you?re stuck with PostgreSQL and psycopg2 unless you rewrite your code to be compatible with the other database/library. Alternatives (other DBs): sqlite3, mysqlclient. There are also other PostgreSQL libraries available. -- Chris Warrick PGP: 5EAAEA16 From ian.g.kelly at gmail.com Mon Oct 23 16:26:23 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 23 Oct 2017 14:26:23 -0600 Subject: Compression of random binary data In-Reply-To: <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> Message-ID: On Mon, Oct 23, 2017 at 1:42 PM, wrote: > Wow, do programmers actually use zscii. That is huge. So much wated space. Not really. ZSCII is only relevant if you're writing Z-code or a Z-code interpreter. Those in turn are only relevant if you're writing Infocom games. From tjol at tjol.eu Mon Oct 23 17:26:00 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 23:26:00 +0200 Subject: Windows alternative: multiprocessing.connection.wait on Pipe, Tkinter File Handlers In-Reply-To: References: Message-ID: On 23/10/17 18:34, Josh Jacobson wrote: > The two functions in the subject are not fully implementable on Windows, > and so I am looking for an alternative. > > Relevant SO postings including full code and description, with bounty: > Freezes > , > Input > > > ---- > > I have two scripts: > > *Processor_child.py*: Its purpose is to perform a number of data analysis > and cleaning operations. This must perform the same operations when run > alone (without Tkinter_parent.py) as it does when packaged into a GUI with > Tkinter_parent.py. > > *Tkinter_parent.py*: Its purpose is to provide a GUI for those who can't > use Processor_child directly.Within Processor_child, there are for loops > that ask the user for input on each iteration. These prompts need to appear > in the Tkinter app, accept the input, and send it back to Processor_child. > > ---- > > The main issue is having the script not move forward on execution until > input has been sent. > > I know of three ways to theoretically solve this, but *none of these work > in production on a Windows machine*: > > 1. while pipe1.poll() != True: > > time.sleep(0.1) My intuition tells me that this kind of thing should be in a separate thread rather than buried in the main loop. You might wait in a thread and somehow (no idea what the best way to do this is in tkinter) pass a message to the GUI thread when it's done. Not sure if this will help. > > This causes constant loading and a "freezing" like user experience. > > 2. multiprocessing.connection.wait > > From the documentation: ""Windows: ... Note that pipe handles and socket > handles are not waitable handles." > > 3. Tkinter file handlers > > From the documentation: "This feature is not available on Windows." Both of these are about Windows not having the same select() system call as *nix. You can work around this by using a local (loopback) TCP connection rather than a multiprocessing-supplied pipe (--> socket module). Then you can use select on both Windows and *nix (-->select module). Why do the GUI and the processing have to be in separate scripts? Maybe it would be easier to import the processing bits as a module and then run them "directly" - possibly via multiprocessing or threading. > > > *Given that none of the 3 available options works on Windows, what options > are available for Windows machines?* > > See Tkinter application 'freezes' while continually polling Pipe for > contents (multiprocessing) > > and How can I implement an `input` method in a Tkinter parent script, with > the displayed prompt and return value being sent back to a child script? > > for further description and code samples. > From tjol at tjol.eu Mon Oct 23 17:50:38 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 23 Oct 2017 23:50:38 +0200 Subject: grapheme cluster library In-Reply-To: References: <5650e54a-1083-4c1f-8172-5768fa4594be@googlegroups.com> <59ed9e0e$0$18578$b1db1813$d948b532@news.astraweb.com> Message-ID: On 23/10/17 16:25, Rustom Mody wrote: > On Monday, October 23, 2017 at 1:15:35 PM UTC+5:30, Steve D'Aprano wrote: >> >> and more. Many linguists also include digraphs (pairs of letters) like the >> English "th", "sh", "qu", or "gh" as graphemes. >> >> >> https://www.thoughtco.com/what-is-a-grapheme-1690916 >> >> https://en.wikipedia.org/wiki/Grapheme > > Um? Ok So I am using the wrong word? Your first link says: > | For example, the word 'ghost' contains five letters and four graphemes > | ('gh,' 'o,' 's,' and 't') > > Whereas new regex findall does: > >>>> findall(r'\X', "ghost") > ['g', 'h', 'o', 's', 't'] >>>> findall(r'\X', "church") > ['c', 'h', 'u', 'r', 'c', 'h'] > The definition of a "grapheme" in the Unicode standard does not necessarily line up with linguistic definition of grapheme for any particular language. Even if we assumed that there was a universally agreed definition of the term for every written language (for English there certainly isn't), you'd dictionaries information on which language you're dealing with to pull this trick off. As an example to illustrate why you'd need dictionaries: In Dutch, "ij" (the "long IJ", as opposed to the "greek Y") is generally considered a single letter, or at the very least a single grapheme. There is a unicode codepoint for it (?), but it isn't widely used. So "vrij" (free) has three graphemes (v r ?) and three or four letters. However, in "bijectie" (bijection), "i" and "j" are two separate graphemes, so this word has eight letters and seven or eight graphemes. ("ie" may or may not be one single grapheme...) -- Thomas PS: This may not be obvious to you at first unless you're Dutch. From encore1 at cox.net Mon Oct 23 18:16:29 2017 From: encore1 at cox.net (Dick Holmes) Date: Mon, 23 Oct 2017 15:16:29 -0700 Subject: Installing tkinter on FreeBSD Message-ID: I am trying to use tkinter on a FreeBSD system but the installed versions of Python (2.7 and 3.6) don't have thinter configured. I tried to download the source (no binaries available for FreeBSD) and build a new version of Python but the build reported that it couldn't install _tkinter. Despite this report, Python works for non-tkinter applications. Is there a magical formula for configuring tkinter during or after a build?? TIA! -- Dick Holmes From skybuck2000 at hotmail.com Mon Oct 23 18:50:47 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Mon, 23 Oct 2017 15:50:47 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> Message-ID: <40a57501-fad8-479b-a812-08502f00ded4@googlegroups.com> Question: What do the letters # v l s c Stand for in this source code ? v = not valid/valid l = length s = ? c = ? If somebody with lots of python experience could dive into this code and then tell me I'd be most gratefull ! :) # # ./sifter.py --unk --dis --len --sync --tick -- -P1 -t # ./injector -P1 -t -t -R -0 -s 4293486582 # # insn tested: 129563 # artf found: 0 # runtime: 00:00:04.23 # seed: 4293486582 # arch: 64 # date: 2017-10-22 16:10:51 # # cpu: # processor : 0 # vendor_id : AuthenticAMD # cpu family : 15 # model : 43 # model name : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ # stepping : 1 # microcode : 0x4d # v l s c 0f0d00 1 3 5 2 (0f0d0000000000000000000000000000) 0f0d01 1 3 5 2 (0f0d0100000000000000000000000000) 0f0d02 1 3 5 2 (0f0d0200000000000000000000000000) 0f0d03 1 3 5 2 (0f0d0300000000000000000000000000) 0f0d0400 1 4 5 2 (0f0d0400000000000000000000000000) 0f0d0401 1 4 5 2 (0f0d0401000000000000000000000000) 0f0d0402 1 4 5 2 (0f0d0402000000000000000000000000) 0f0d0403 1 4 5 2 (0f0d0403000000000000000000000000) 0f0d0404 1 4 5 2 (0f0d0404000000000000000000000000) 0f0d040500000000 1 8 5 2 (0f0d0405000000000000000000000000) 0f0d040501000000 1 8 5 2 (0f0d0405010000000000000000000000) 0f0d040502000000 1 8 5 2 (0f0d0405020000000000000000000000) 0f0d040503000000 1 8 5 2 (0f0d0405030000000000000000000000) 0f0d040504000000 1 8 5 2 (0f0d0405040000000000000000000000) Bye, Skybuck. From python at mrabarnett.plus.com Mon Oct 23 19:12:54 2017 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 24 Oct 2017 00:12:54 +0100 Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <40a57501-fad8-479b-a812-08502f00ded4@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> <40a57501-fad8-479b-a812-08502f00ded4@googlegroups.com> Message-ID: On 2017-10-23 23:50, skybuck2000 at hotmail.com wrote: > Question: > > What do the letters > > # v l s c > > Stand for in this source code ? > > v = not valid/valid > l = length > s = ? > c = ? > > If somebody with lots of python experience could dive into this code and then tell me I'd be most gratefull ! :) > > > > # > # ./sifter.py --unk --dis --len --sync --tick -- -P1 -t > # ./injector -P1 -t -t -R -0 -s 4293486582 > # > # insn tested: 129563 > # artf found: 0 > # runtime: 00:00:04.23 > # seed: 4293486582 > # arch: 64 > # date: 2017-10-22 16:10:51 > # > # cpu: > # processor : 0 > # vendor_id : AuthenticAMD > # cpu family : 15 > # model : 43 > # model name : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ > # stepping : 1 > # microcode : 0x4d > # v l s c > 0f0d00 1 3 5 2 (0f0d0000000000000000000000000000) > 0f0d01 1 3 5 2 (0f0d0100000000000000000000000000) > 0f0d02 1 3 5 2 (0f0d0200000000000000000000000000) > 0f0d03 1 3 5 2 (0f0d0300000000000000000000000000) > 0f0d0400 1 4 5 2 (0f0d0400000000000000000000000000) > 0f0d0401 1 4 5 2 (0f0d0401000000000000000000000000) > 0f0d0402 1 4 5 2 (0f0d0402000000000000000000000000) > 0f0d0403 1 4 5 2 (0f0d0403000000000000000000000000) > 0f0d0404 1 4 5 2 (0f0d0404000000000000000000000000) > 0f0d040500000000 1 8 5 2 (0f0d0405000000000000000000000000) > 0f0d040501000000 1 8 5 2 (0f0d0405010000000000000000000000) > 0f0d040502000000 1 8 5 2 (0f0d0405020000000000000000000000) > 0f0d040503000000 1 8 5 2 (0f0d0405030000000000000000000000) > 0f0d040504000000 1 8 5 2 (0f0d0405040000000000000000000000) > > Bye, > Skybuck. > v = valid l = length s = signum c = sicode From skybuck2000 at hotmail.com Mon Oct 23 19:22:15 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Mon, 23 Oct 2017 16:22:15 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> Message-ID: Idea of this software is basically: Generate random bytes and feed them to processor. Observe result of processor if good or bad (error codes). If good check docs. If bad adjust and retry. Somebody wrote a nice short explanation of what SandSifter does to give you an idea (it's a new algorithm to find undocumented instructions fast !): It's guessing possible X86 instructions by exploiting the Instruction Decoder via the (PF) Page Fault result code. Effectively splitting an instruction across two pages and only having one page of it executable. When the decoder fetches the instruction it notices that it's incomplete, attempts to fetch the next part that is on a new non-executable page. The decoder then throws a page fault since it's not executable. So it moves the entire instruction one to the left and tries again with various combinations until it doesn't get a page fault at which point it executes it. And thus it attempts to 'tunnel' through every possible instruction. That's the general very simplified explanation. Bye, Skybuck. From skybuck2000 at hotmail.com Mon Oct 23 19:41:27 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Mon, 23 Oct 2017 16:41:27 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> <40a57501-fad8-479b-a812-08502f00ded4@googlegroups.com> Message-ID: On Tuesday, October 24, 2017 at 1:13:22 AM UTC+2, MRAB wrote: > On 2017-10-23 23:50, skybuck2000 at hotmail.com wrote: > > Question: > > > > What do the letters > > > > # v l s c > > > > Stand for in this source code ? > > > > v = not valid/valid > > l = length > > s = ? > > c = ? > > > > If somebody with lots of python experience could dive into this code and then tell me I'd be most gratefull ! :) > > > > > > > > # > > # ./sifter.py --unk --dis --len --sync --tick -- -P1 -t > > # ./injector -P1 -t -t -R -0 -s 4293486582 > > # > > # insn tested: 129563 > > # artf found: 0 > > # runtime: 00:00:04.23 > > # seed: 4293486582 > > # arch: 64 > > # date: 2017-10-22 16:10:51 > > # > > # cpu: > > # processor : 0 > > # vendor_id : AuthenticAMD > > # cpu family : 15 > > # model : 43 > > # model name : AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ > > # stepping : 1 > > # microcode : 0x4d > > # v l s c > > 0f0d00 1 3 5 2 (0f0d0000000000000000000000000000) > > 0f0d01 1 3 5 2 (0f0d0100000000000000000000000000) > > 0f0d02 1 3 5 2 (0f0d0200000000000000000000000000) > > 0f0d03 1 3 5 2 (0f0d0300000000000000000000000000) > > 0f0d0400 1 4 5 2 (0f0d0400000000000000000000000000) > > 0f0d0401 1 4 5 2 (0f0d0401000000000000000000000000) > > 0f0d0402 1 4 5 2 (0f0d0402000000000000000000000000) > > 0f0d0403 1 4 5 2 (0f0d0403000000000000000000000000) > > 0f0d0404 1 4 5 2 (0f0d0404000000000000000000000000) > > 0f0d040500000000 1 8 5 2 (0f0d0405000000000000000000000000) > > 0f0d040501000000 1 8 5 2 (0f0d0405010000000000000000000000) > > 0f0d040502000000 1 8 5 2 (0f0d0405020000000000000000000000) > > 0f0d040503000000 1 8 5 2 (0f0d0405030000000000000000000000) > > 0f0d040504000000 1 8 5 2 (0f0d0405040000000000000000000000) > > > > Bye, > > Skybuck. > > > > v = valid > l = length > s = signum > c = sicode Thank you, at least signum seems correct, I will investigate sicode later !;) So far found this tabel describing the meaning for these values: https://people.cs.pitt.edu/~alanjawi/cs449/code/shell/UnixSignals.htm Unix Signals SIGHUP 1 Exit Hangup SIGINT 2 Exit Interrupt SIGQUIT 3 Core Quit SIGILL 4 Core Illegal Instruction SIGTRAP 5 Core Trace/Breakpoint Trap SIGABRT 6 Core Abort SIGEMT 7 Core Emulation Trap SIGFPE 8 Core Arithmetic Exception SIGKILL 9 Exit Killed SIGBUS 10 Core Bus Error SIGSEGV 11 Core Segmentation Fault SIGSYS 12 Core Bad System Call SIGPIPE 13 Exit Broken Pipe SIGALRM 14 Exit Alarm Clock SIGTERM 15 Exit Terminated SIGUSR1 16 Exit User Signal 1 SIGUSR2 17 Exit User Signal 2 SIGCHLD 18 Ignore Child Status SIGPWR 19 Ignore Power Fail/Restart SIGWINCH 20 Ignore Window Size Change SIGURG 21 Ignore Urgent Socket Condition SIGPOLL 22 Ignore Socket I/O Possible SIGSTOP 23 Stop Stopped (signal) SIGTSTP 24 Stop Stopped (user) SIGCONT 25 Ignore Continued SIGTTIN 26 Stop Stopped (tty input) SIGTTOU 27 Stop Stopped (tty output) SIGVTALRM 28 Exit Virtual Timer Expired SIGPROF 29 Exit Profiling Timer Expired SIGXCPU 30 Core CPU time limit exceeded SIGXFSZ 31 Core File size limit exceeded SIGWAITING 32 Ignore All LWPs blocked SIGLWP 33 Ignore Virtual Interprocessor Interrupt for Threads Library SIGAIO 34 Ignore Asynch 5 is indeed trap I know that much ! ;) Bye, Skybuck. From tjol at tjol.eu Mon Oct 23 19:56:19 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 24 Oct 2017 01:56:19 +0200 Subject: Installing tkinter on FreeBSD In-Reply-To: References: Message-ID: On 24/10/17 00:16, Dick Holmes wrote: > I am trying to use tkinter on a FreeBSD system but the installed > versions of Python (2.7 and 3.6) don't have thinter configured. I tried > to download the source (no binaries available for FreeBSD) and build a > new version of Python but the build reported that it couldn't install > _tkinter. Despite this report, Python works for non-tkinter > applications. Is there a magical formula for configuring tkinter during > or after a build?? If you have Tcl/Tk installed, the configure script should detect it. Is Tcl/Tk installed? If yes, does the configure script output say anything about tcl, tk or tkinter? -- Thomas From danceswithnumbers at gmail.com Mon Oct 23 22:27:54 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 19:27:54 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> Message-ID: <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> Finally figured out how to turn this into a random binary compression program. Since my transform can compress more than dec to binary. Then i took a random binary stream, changed it to a decimal stream 0-9 tranformed it into a compressed/encrypted binary stream 23.7% smaller. Yes! Decode reverse is easy......sorry so excited i could shout. Jon Hutton From steve+python at pearwood.info Mon Oct 23 23:51:37 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 24 Oct 2017 14:51:37 +1100 Subject: Compression of random binary data References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> Message-ID: <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> On Tue, 24 Oct 2017 01:27 pm, danceswithnumbers at gmail.com wrote: > Finally figured out how to turn this into a random binary compression > program. Since my transform can compress more than dec to binary. Then i > took a random binary stream, changed it to a decimal stream 0-9 tranformed > it into a compressed/encrypted binary stream 23.7% smaller. Smaller than the original binary stream? I don't think so. There's nothing clever about "compressing" a random binary stream if first you expand it, then remove the extra space you added and claim victory because the result is smaller than the expanded version. > Yes! Decode reverse is easy......sorry so excited i could shout. Then this should be easy for you: http://marknelson.us/2012/10/09/the-random-compression-challenge-turns-ten/ All you need to do is compress this file: http://marknelson.us/attachments/million-digit-challenge/AMillionRandomDigits.bin to less than 415241 bytes, and you can win $100. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From danceswithnumbers at gmail.com Tue Oct 24 00:13:00 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Mon, 23 Oct 2017 21:13:00 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> Message-ID: <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> I did that quite a while ago. 352,954 kb. From steve+python at pearwood.info Tue Oct 24 00:21:57 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 24 Oct 2017 15:21:57 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> Message-ID: <59eebfe6$0$18582$b1db1813$d948b532@news.astraweb.com> On Tue, 24 Oct 2017 03:13 pm, danceswithnumbers at gmail.com wrote: > I did that quite a while ago. 352,954 kb. Sure you did. Let's see the code you used. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Oct 24 01:36:04 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Oct 2017 16:36:04 +1100 Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: On Tue, Oct 24, 2017 at 2:28 AM, Paul Moore wrote: > Hope this helps put the subject into context. Compression is a very > technical subject, to "do it right". Special cases can be worked out, > sure, but the "hidden assumptions" in a method are what make the > difference between a "compression algorithm" and a "way of storing my > particular data more efficiently". And to put *this* into context and perspective, both of the above are extremely useful tools/algorithms. Generalized compression algos are used all the time, special-purpose lossy compression algos too, and "way[s] of storing my particular data more efficiently" are utterly crucial to many applications. But they're not universal compression schemes. ChrisA From greg.ewing at canterbury.ac.nz Tue Oct 24 02:18:21 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Oct 2017 19:18:21 +1300 Subject: Compression of random binary data In-Reply-To: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > 1234999988884321 > > It only takes seven 8 bit bytes to represent this This is not surprising. The theoretical minimum size for 16 arbitrary decimal digits is: log2(10) * 16 = 53.15 bits = 6.64 bytes I think you misunderstand what is meant by the phrase "random data cannot be compressed". A string of decimal digits represented in ASCII is *not* completely random. There are 256 possible values for each byte, and you're only using 10 of them. What you *can't* do is compress 16 random decimal digits to less than 6.64 bytes. If you have something that you think can do better than that, we would like to see it. -- Greg From greg.ewing at canterbury.ac.nz Tue Oct 24 02:20:14 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Oct 2017 19:20:14 +1300 Subject: Compression of random binary data In-Reply-To: <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > I did that quite a while ago. 352,954 kb. Are you sure? Does that include the size of all the code, lookup tables, etc. needed to decompress it? But even if you have, you haven't disproved the theorem about compressing random data. All you have is a program that compresses *that particular* sequence of a million digits. To disprove the theorem, you would need to exhibit an algorithm that can compress *any* sequence of a million digits to less than 415,241 bytes. -- Greg From tjreedy at udel.edu Tue Oct 24 02:36:07 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 24 Oct 2017 02:36:07 -0400 Subject: IDLE doesn't recognise installed packages In-Reply-To: <2096431256.470671.1508768584232@webmail.strato.com> References: <2096431256.470671.1508768584232@webmail.strato.com> Message-ID: On 10/23/2017 10:23 AM, Daniel Tangemann wrote: > I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message that > it's already installed. how do I fix this? > cheers Recognition of installed packages is done by the python running IDLE and executing your import statements, by not IDLE. The only effect IDLE could have is any manipulation of sys.path. You can find the executable running IDLE with >>> import sys; sys.executable 'C:\\Programs\\Python37\\pythonw.exe' Find the sys.path being used with >>> sys.path If you run the same binary (minus the 'w' if present), you can find the sys.path used without IDLE. You can also test imports without IDLE in use. It is possible that you have more than one binary around, but I cannot tell from here. To make sure you are running pip with the same binary as IDLE, enter path-to-binary -m pip C:\Programs\Python37\python.exe -m pip list -- Terry Jan Reedy From marko at pacujo.net Tue Oct 24 02:36:19 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 24 Oct 2017 09:36:19 +0300 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <87a80hxdb0.fsf@elektro.pacujo.net> Gregory Ewing : > What you *can't* do is compress 16 random decimal digits to less than > 6.64 bytes. More precisely: Regardless of the compression scheme, the probability of shortening the next bit sequence is less than 0.5 if the bits are distributed evenly, randomly and independently. Often the distribution is not even, or there is interdependence between the bits. Then, a compression scheme can do miracles. Marko From greg.ewing at canterbury.ac.nz Tue Oct 24 03:24:30 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Oct 2017 20:24:30 +1300 Subject: Compression of random binary data In-Reply-To: <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > Compress this: > > 4135124325 > > Bin to dec...still very large > 11110110 > 01111000 > 11111101 > 01100101 Wait right there! You're cheating by dropping off leading 0 bits. The maximum value of a 10 digit decimal number is 9999999999, which in hex is 2540be3ff. That's 34 bits. That's in line with the theoretical number of bits needed: log2(10) * 10 = 33.219 So the binary version of your number above is really 00 11110110 01111000 11111101 01100101 You may think you can get away without storing or transmitting those leading 0 bits, because the decoder can always pad out the data as needed. But to do that, the decoder needs to know *how many* bits to pad out to. That information somehow needs to be part of the encoded data. You need to imagine you're sending the data to someone over a wire. The only thing you can send along the wire are ones and zeroes. You can't convey any information by timing, or turning off the power, or anything like that. How is the receiver going to know when he's got the whole message? There are only two possibilities. Either you decide in advance that all messages will be exactly the same length, which in this case means always sending exactly 34 bits. Or you add some extra bits to the message -- prepend the length in binary, or add an end-of-message code at the end, or something like that. Whatever you do, you'll find that *on average* you will need *at least* 34 bits to be able to represent all possible 10-digit decimal numbers. Some might be shorter, but then others will be longer, and the average won't be less than 34. > New compression method: > > 11000101 > 11000111 > 01001111 > > A full byte less than bin. You need to be *very* careful about what you're claiming here. Are you saying that your algorithm compresses *all possible* sequences of 10 decimal digits to 3 bytes or less? Or can some of them come out longer? -- Greg From danceswithnumbers at gmail.com Tue Oct 24 03:46:47 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Tue, 24 Oct 2017 00:46:47 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: Greg, you're very smart, but you are missing a big key. I'm not padding, you are still thinking inside the box, and will never solve this by doing so. Yes! At least you see my accomplishment, this will compress any random file. From auriocus at gmx.de Tue Oct 24 03:48:36 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 24 Oct 2017 09:48:36 +0200 Subject: Compression of random binary data In-Reply-To: <87o9oy41ej.fsf@elektro.pacujo.net> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1dcc0480-d458-5b42-5681-7b2fddeece74@tjol.eu> <87o9oy41ej.fsf@elektro.pacujo.net> Message-ID: Am 23.10.17 um 12:13 schrieb Marko Rauhamaa: > Thomas Jollans : > >> On 2017-10-23 11:32, danceswithnumbers at gmail.com wrote: >>> According to this website. This is an uncompressable stream. >>> >>> https://en.m.wikipedia.org/wiki/Incompressible_string >>> >>> 1234999988884321 >> >> No, it's not. According to that article, that string is incompressible >> by a particular algorithm. I can see no more general claims. > > Here's a compression algorithm that manages to compress that string into > a 0-bit string: > > * If the original string is 1234999988884321 (whatever that means), > return the empty bit string. > > * Otherwise, prepend a don't-care bit to the original string and return > the result of the concatenation. > ...and that's why there is the "Kolmogorov complexity". You need to append the decompression program to the data to show how much you really saved, which will turn out to be nothing compared to the "trivial decompressor" print "1234999988884321" Christian From danceswithnumbers at gmail.com Tue Oct 24 03:48:47 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Tue, 24 Oct 2017 00:48:47 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: <966c13a5-433d-4c44-9890-64602b3a83d6@googlegroups.com> No leading zeroes are being dropped off....wish this board has an edit button. From miki.tebeka at gmail.com Tue Oct 24 04:13:03 2017 From: miki.tebeka at gmail.com (Miki Tebeka) Date: Tue, 24 Oct 2017 01:13:03 -0700 (PDT) Subject: [ANN] Nuclio: A scalable, open source, real-time processing platform Message-ID: Hi, Just wanted to share a project I'm working on. It a super fast serverless that support Python handlers as well. Check out more at https://www.iguazio.com/nuclio-new-serverless-superhero/ Code at https://github.com/nuclio/nuclio/ Happy hacking, -- Miki From greg.ewing at canterbury.ac.nz Tue Oct 24 04:43:03 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Oct 2017 21:43:03 +1300 Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: Paul Moore wrote: > But that's not "compression", that's simply using a better encoding. > In the technical sense, "compression" is about looking at redundancies > that go beyond the case of how effectively you pack data into the > bytes available. There may be a difference in the way the terms are used, but I don't think there's any fundamental difference. Compression is about finding clever ways to make the encoding better. Either way, the information-theoretic limits on the number of bits needed are the same. -- Greg From greg.ewing at canterbury.ac.nz Tue Oct 24 04:43:07 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 24 Oct 2017 21:43:07 +1300 Subject: Compression of random binary data In-Reply-To: References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > > My 8 year old can decode this back into base 10, Keep in mind that your 8 year old has more information than just the 32 bits you wrote down -- he can also see that there *are* 32 bits and no more. That's hidden information that you're not counting. -- Greg From p.f.moore at gmail.com Tue Oct 24 06:20:37 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Oct 2017 11:20:37 +0100 Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: On 24 October 2017 at 09:43, Gregory Ewing wrote: > Paul Moore wrote: >> >> But that's not "compression", that's simply using a better encoding. >> In the technical sense, "compression" is about looking at redundancies >> that go beyond the case of how effectively you pack data into the >> bytes available. > > > There may be a difference in the way the terms are used, but > I don't think there's any fundamental difference. Compression > is about finding clever ways to make the encoding better. Agreed - I was trying (probably futilely, given the way this thread has gone...) to make a distinction between purely local properties that are typically considered in "how you encode the data" and the detection of more global patterns, which is where what are typically referred to as "compression" algorithms get their power. But sadly, I don't think the OP is actually interested in understanding the background, so the distinction wasn't really worth making :-( > Either way, the information-theoretic limits on the number > of bits needed are the same. Precisely. Paul From ben.usenet at bsb.me.uk Tue Oct 24 06:23:15 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 24 Oct 2017 11:23:15 +0100 Subject: Compression of random binary data References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> Message-ID: <87efpsj14c.fsf@bsb.me.uk> danceswithnumbers at gmail.com writes: > Finally figured out how to turn this into a random binary compression > program. Since my transform can compress more than dec to binary. Then > i took a random binary stream, Forget random data. For one thing it's hard to define, but more importantly no one cares about it. By its very nature, random data is not interesting. What people want is a reversible compression algorithm that works on *arbitrary data* -- i.e. on *any* file at all, no matter how structured and *non-random* it is. For example, run the complete works of Shakespeare through your program. The result is very much not random data, but that's the sort of data people want to compress. If you can compress the output of your compressor you have made a good start. Of course what you really want to be able to do is to compress the output that results from compressing your compressed out. And, of course, you should not stop there. Since you can compress *any* data (not just the boring random stuff) you can keep going -- compressing the compressed output again and again until you end up with a zero-length file. Then you publish in a major journal. Post the link to the journal article when you are done. -- Ben. From steve+python at pearwood.info Tue Oct 24 06:41:56 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 24 Oct 2017 21:41:56 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> Message-ID: <59ef18f5$0$18583$b1db1813$d948b532@news.astraweb.com> On Tue, 24 Oct 2017 05:20 pm, Gregory Ewing wrote: > danceswithnumbers at gmail.com wrote: >> I did that quite a while ago. 352,954 kb. > > Are you sure? Does that include the size of all the > code, lookup tables, etc. needed to decompress it? > > But even if you have, you haven't disproved the theorem about > compressing random data. All you have is a program that > compresses *that particular* sequence of a million digits. > > To disprove the theorem, you would need to exhibit an > algorithm that can compress *any* sequence of a million > digits to less than 415,241 bytes. Indeed -- but let's give Dancerswithnumbers his due. *IF* he is right (a very big "if" indeed) about being able to compress the Rand Corporation "Million Random Digits" in binary form, as given, that alone would be an impressive trick. Compressing the digits in text form is not impressive in the least. As Ben Bacarisse pointed out, most of us will probably already have half a dozen programs that do that. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From p.f.moore at gmail.com Tue Oct 24 06:50:48 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Oct 2017 11:50:48 +0100 Subject: Compression of random binary data In-Reply-To: <87efpsj14c.fsf@bsb.me.uk> References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> Message-ID: On 24 October 2017 at 11:23, Ben Bacarisse wrote: > For example, run the complete works of Shakespeare through your program. > The result is very much not random data, but that's the sort of data > people want to compress. If you can compress the output of your > compressor you have made a good start. Of course what you really want > to be able to do is to compress the output that results from compressing > your compressed out. And, of course, you should not stop there. Since > you can compress *any* data (not just the boring random stuff) you can > keep going -- compressing the compressed output again and again until > you end up with a zero-length file. Oh, and just for fun, if you are able to guarantee compressing arbitrary data, then 1. Take a document you want to compress. 2. Compress it using your magic algorithm. The result is smaller. 3. Compress the compressed data. The result is still smaller. 4. Repeat until you hit 0 bytes. Congratulations - apparently you have a reversible algorithm that compresses every data set to an empty file. (Caveat - there's actually "hidden data" here, as you need to know how many compressions it takes to hit 0 bytes. Because you decrease the size every time, though, that number must be no greater than the size of the original file). Paul From ben.usenet at bsb.me.uk Tue Oct 24 07:04:42 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 24 Oct 2017 12:04:42 +0100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> Message-ID: <871slsiz79.fsf@bsb.me.uk> Paul Moore writes: > On 24 October 2017 at 11:23, Ben Bacarisse wrote: >> For example, run the complete works of Shakespeare through your program. >> The result is very much not random data, but that's the sort of data >> people want to compress. If you can compress the output of your >> compressor you have made a good start. Of course what you really want >> to be able to do is to compress the output that results from compressing >> your compressed out. And, of course, you should not stop there. Since >> you can compress *any* data (not just the boring random stuff) you can >> keep going -- compressing the compressed output again and again until >> you end up with a zero-length file. > > Oh, and just for fun, if you are able to guarantee compressing > arbitrary data, then It's a small point, but you are replying to a post of mine and saying "you". That could make people think that /I/ am claiming to have a perfect compression algorithm. > 1. Take a document you want to compress. > 2. Compress it using your magic algorithm. The result is smaller. > 3. Compress the compressed data. The result is still smaller. > 4. Repeat until you hit 0 bytes. Isn't this just repeating what I said? I must has not written is clearly enough. -- Ben. From rosuav at gmail.com Tue Oct 24 07:14:14 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 24 Oct 2017 22:14:14 +1100 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On Tue, Oct 24, 2017 at 6:57 AM, Chris Warrick wrote: > On 23 October 2017 at 21:37, John Black wrote: >> Chris, thanks for all this detailed information. I am confused though >> with your database recommendation. You say you teach SQLAlchemy but >> generally use PostgreSQL yourself. I can maybe guess why there seems to >> be this contradiction. Perhaps PostgreSQL is better but too advanced for >> the class you are teaching? Can you clarify on which you think is the >> better choice? Thanks. > > Different Chris, but I?ll answer. Those are two very different things. > > PostgreSQL is a database server. It talks SQL to clients, stores data, > retrieves it when asked. The usual stuff a database server does. > Alternatives: SQLite, MySQL, MS SQL, Oracle DB, ? > > SQLAlchemy is an ORM: an object-relational mapper, and also a database > toolkit. SQLAlchemy can abstract multiple database servers/engines > (PostgreSQL, SQLite, MySQL, etc.) and work with them from the same > codebase. It can also hide SQL from you and instead give you Python > classes. If you use an ORM like SQLAlchemy, you get database support > without writing a single line of SQL on your own. But you still need a > database engine ? PostgreSQL can be one of them. But you can deploy > the same code to different DB engines, and it will just work? > (assuming you didn?t use any DB-specific features). Alternatives: > Django ORM. > > psycopg2 is an example of a PostgreSQL client library for Python. It > implements the Python DB-API and lets you use it to talk to a > PostgreSQL server. When using psycopg2, you?re responsible for writing > your own SQL statements for the server to execute. In that approach, > you?re stuck with PostgreSQL and psycopg2 unless you rewrite your code > to be compatible with the other database/library. Alternatives (other > DBs): sqlite3, mysqlclient. There are also other PostgreSQL libraries > available. > Thanks, namesake :) The above is correct and mostly accurate. It IS possible to switch out your back end fairly easily, though, even with psycopg2; there's a standard API that most Python database packages follow. As long as you stick to standard SQL (no PostgreSQL extensions) and the standard API (no psycopg2 extensions), switching databases is as simple as changing your "import psycopg2" into "import cx_oracle" or something. (And, most likely, changing your database credentials.) The point of an ORM is to make your databasing code look and feel like Python code, rather than manually crafting SQL statements everywhere. Here's how a simple database operation looks in SQLAlchemy: def spammify(id): person = session.query(Person).get(id) person.style = "spam" session.commit() Here's the equivalent using psycopg2: def spammify(id): with db, db.cursor() as cur: cur.execute("update people set style='spam' where id=%s", id) With SQLAlchemy, you ask for a particular record, and you get back an object. That object has attributes for all the person's information, and you can both read and write those attributes. Then you commit when you're done. Without SQLAlchemy, you use another language (SQL), embedded within your Python code. The choice is mostly one of style and preference. But if you don't currently have a preference, I would recommend using an ORM. (There are other ORMs than SQLAlchemy, of course; I can't recall the exact syntax for Django's off the top of my head, but it's going to be broadly similar to this.) ChrisA From p.f.moore at gmail.com Tue Oct 24 07:27:03 2017 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Oct 2017 12:27:03 +0100 Subject: Compression of random binary data In-Reply-To: <871slsiz79.fsf@bsb.me.uk> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <871slsiz79.fsf@bsb.me.uk> Message-ID: On 24 October 2017 at 12:04, Ben Bacarisse wrote: > Paul Moore writes: > >> On 24 October 2017 at 11:23, Ben Bacarisse wrote: >>> For example, run the complete works of Shakespeare through your program. >>> The result is very much not random data, but that's the sort of data >>> people want to compress. If you can compress the output of your >>> compressor you have made a good start. Of course what you really want >>> to be able to do is to compress the output that results from compressing >>> your compressed out. And, of course, you should not stop there. Since >>> you can compress *any* data (not just the boring random stuff) you can >>> keep going -- compressing the compressed output again and again until >>> you end up with a zero-length file. >> >> Oh, and just for fun, if you are able to guarantee compressing >> arbitrary data, then > > It's a small point, but you are replying to a post of mine and saying > "you". That could make people think that /I/ am claiming to have a perfect > compression algorithm. Sorry. I intended the meaning "If one is able to..." but I was unclear. My bad. >> 1. Take a document you want to compress. >> 2. Compress it using your magic algorithm. The result is smaller. >> 3. Compress the compressed data. The result is still smaller. >> 4. Repeat until you hit 0 bytes. > > Isn't this just repeating what I said? I must has not written is > clearly enough. More accurately, I didn't read it carefully enough. Again sorry. However, I guess it serves as an example of a compression algorithm - we can trivially compress the content of our two posts into a single post with just as much information content, by deleting my post :-) Paul From steve+python at pearwood.info Tue Oct 24 07:44:57 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 24 Oct 2017 22:44:57 +1100 Subject: Compression of random binary data References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> Message-ID: <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> On Tue, 24 Oct 2017 09:23 pm, Ben Bacarisse wrote: > Forget random data. For one thing it's hard to define, That bit is true. > but more importantly no one cares about it. But that's wrong. For instance: - Encrypted data looks very much like random noise. With more and more data traversing the internet in encrypted form, the ability to compress random noise would be worth billions. - Compressed data looks somewhat like random noise (with a bit of structure). The more it is compressed, the more random it looks. If you could compress random noise, you could take already compressed data, and compress it again, saving even more space. - Many multimedia formats (images, sound, video) are compressed using dedicated encoders. The better the encoder, the more it compresses the data (whether lossy or not) the harder it is to compress it further. If you could compress random noise, you could compress JPGs, MP3s, h265-encoded MKVs, etc, saving even more storage and transmission costs. And most importantly: - Random data is a superset of the arbitrary structured data you mention below. If we could compress random data, then we could compress any data at all, no matter how much or little structure it contained. This is why the ability to compress random data (if it were possible, which it is not) is interesting. Its not because people want to be able to compress last night's lottery numbers, or tables of random digits. > By its very nature, random data is > not interesting. What people want is a reversible compression algorithm > that works on *arbitrary data* -- i.e. on *any* file at all, no matter > how structured and *non-random* it is. In a sense you are right. Compressing randomly generated data would be a parlour trick and not specifically very useful. But if you had such an algorithm, that would change the face of the world. It would be as revolutionary and paradigm breaking as a perpetual motion machine, or discovery of a new continent the size of China in the middle of the Atlantic, or that ? actually does equal 22/7 exactly. And just as impossible. > For example, run the complete works of Shakespeare through your program. > The result is very much not random data, but that's the sort of data > people want to compress. If you can compress the output of your > compressor you have made a good start. Of course what you really want > to be able to do is to compress the output that results from compressing > your compressed out. And, of course, you should not stop there. Since > you can compress *any* data (not just the boring random stuff) you can > keep going -- compressing the compressed output again and again until > you end up with a zero-length file. Indeed. That proof by contradiction is yet another reason we know we can't compress random data -- that is to say, *arbitrary* data. If we had a compression program which could guarantee to ALWAYS shrink ANY file by at least one bit, then we could just apply it over and over again, shrinking the compressed file again and again, until we were left with a zero-byte file: original.dat = 110 MB original.dat.zip.zip.zip.zip.zip.zip.zip = 0 MB And then reverse the process, to turn an empty file back into the original. But given an empty file, how do you distinguish the empty file you get from 'music.mp3' and the identical empty file you get from 'movie.avi'? Obviously you cannot. So we know that the only way to *guarantee* to shrink every possible file is if the compression is lossy. > Then you publish in a major journal. Post the link to the journal > article when you are done. These days there are plenty of predatory journals which will be happy to take Dancerswithnumber's money in return for publishing it in a junk journal. https://en.wikipedia.org/wiki/Predatory_open_access_publishing -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 24 07:48:16 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 24 Oct 2017 22:48:16 +1100 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: <59ef2882$0$14951$b1db1813$d948b532@news.astraweb.com> On Tue, 24 Oct 2017 06:46 pm, danceswithnumbers at gmail.com wrote: > Greg, you're very smart, but you are missing a big key. I'm not padding, > you are still thinking inside the box, and will never solve this by doing > so. Yes! At least you see my accomplishment, this will compress any random > file. Talk is cheap. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From stephanh42 at gmail.com.invalid Tue Oct 24 08:26:50 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 24 Oct 2017 12:26:50 GMT Subject: Installing tkinter on FreeBSD References: Message-ID: <59ef318a$0$1720$e4fe514c@textnews.kpn.nl> Op 2017-10-23, Thomas Jollans schreef : > On 24/10/17 00:16, Dick Holmes wrote: >> I am trying to use tkinter on a FreeBSD system but the installed >> versions of Python (2.7 and 3.6) don't have thinter configured. I tried >> to download the source (no binaries available for FreeBSD). What version of FreeBSD is that? On 11.1 I get: $ pkg search tkinter py27-tkinter-2.7.14_6 Python bindings to the Tk widget set (Python 2.7) py34-tkinter-3.4.7_6 Python bindings to the Tk widget set (Python 3.4) py35-tkinter-3.5.4_6 Python bindings to the Tk widget set (Python 3.5) py36-tkinter-3.6.2_6 Python bindings to the Tk widget set (Python 3.6) pypy-tkinter-5.8.0 PyPy bindings to the Tk widget set and for sure installing py36-tkinter-3.6.2_6 works fine. Stephan From ben.usenet at bsb.me.uk Tue Oct 24 09:29:02 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 24 Oct 2017 14:29:02 +0100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <87wp3khdy9.fsf@bsb.me.uk> Steve D'Aprano writes: > On Tue, 24 Oct 2017 09:23 pm, Ben Bacarisse wrote: > >> Forget random data. For one thing it's hard to define, > > That bit is true. > >> but more importantly no one cares about it. > > But that's wrong. All generalisations are false. I was being hyperbolic. > For instance: > > - Encrypted data looks very much like random noise. With more and more data > traversing the internet in encrypted form, the ability to compress random > noise would be worth billions. > > - Compressed data looks somewhat like random noise (with a bit of structure). > The more it is compressed, the more random it looks. If you could compress > random noise, you could take already compressed data, and compress it again, > saving even more space. > > - Many multimedia formats (images, sound, video) are compressed using > dedicated encoders. The better the encoder, the more it compresses the data > (whether lossy or not) the harder it is to compress it further. If you could > compress random noise, you could compress JPGs, MP3s, h265-encoded MKVs, > etc, saving even more storage and transmission costs. But these are not random data. We care about these because they are are highly structured, non-random data. > And most importantly: > > - Random data is a superset of the arbitrary structured data you mention > below. If we could compress random data, then we could compress any data > at all, no matter how much or little structure it contained. Yes, that's part of my point. Arbitrary data includes random data but it avoids arguments about what random means. > This is why the ability to compress random data (if it were possible, which it > is not) is interesting. Its not because people want to be able to compress > last night's lottery numbers, or tables of random digits. The trouble is a pedagogic one. Saying "you can't compress random data" inevitably leads (though, again, this is just my experience) to endless attempts to define random data. My preferred way out of that is to talk about algorithmic complexity but for your average "I've got a perfect compression algorithm" poster, that is step too far. I think "arbitrary data" (thereby including the results of compression by said algorithm) is the best way to make progress. >> Then you publish in a major journal. Post the link to the journal >> article when you are done. > > These days there are plenty of predatory journals which will be happy to take > Dancerswithnumber's money in return for publishing it in a junk > journal. Sure, but you usually get a huge advantage -- a text to criticise. Your average Usenet crank will keep changing what they say to avoid being pinned down. Plus you get to note the fact that the journal is junk. -- Ben. From ben.usenet at bsb.me.uk Tue Oct 24 09:34:14 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 24 Oct 2017 14:34:14 +0100 Subject: Compression of random binary data References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <59ef2882$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <87she8hdpl.fsf@bsb.me.uk> Steve D'Aprano writes: > On Tue, 24 Oct 2017 06:46 pm, danceswithnumbers at gmail.com wrote: > >> Greg, you're very smart, but you are missing a big key. I'm not padding, >> you are still thinking inside the box, and will never solve this by doing >> so. Yes! At least you see my accomplishment, this will compress any random >> file. > > Talk is cheap. But highly prized. Most Usenet cranks only want to be talked to (they see it as being taken seriously, no matter how rude the respondents are) so for the cost of something cheap (a little babbling) they get an endless stream of highly prized attention. -- Ben. From walters.justin01 at gmail.com Tue Oct 24 11:30:56 2017 From: walters.justin01 at gmail.com (justin walters) Date: Tue, 24 Oct 2017 08:30:56 -0700 Subject: choice of web-framework In-Reply-To: References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: On Tue, Oct 24, 2017 at 4:14 AM, Chris Angelico wrote: > > (There are other ORMs than SQLAlchemy, of course; I can't recall the > exact syntax for Django's off the top of my head, but it's going to be > broadly similar to this.) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > I can help with that: ## Defining a model: class Thing(models.Model): """ This is the "schema" for the `thing` table. The pk field is created automatically and is called `id` by default. This table with have four columns: `id`, `foo`, `baz`, and `score`. """ foo = models.Charfield( max_length=140, blank=False ) baz = models.CharField( max_length=140, blank=True ) score = models.IntegerField() ## Create an object: new_thing = Thing.objects.create(foo="bar", baz="foo") ## Get a list of objects: Thing.objects.all() ## Filter a list of objects: Thing.objects.filter(foo="bar") ## Modify an object: thing = Thing.objects.get(id=1) thing.foo = "baz" thing.save() ## Perform an aggregation: data = Thing.objects.aggregate(avg=Avg("score")) print(data) >>> {"avg": 50} ## Django basic view(called controllers in other frameworks normally) and template: def person_list(request): """ Get a collection of `User` objects from the database. """ people = User.objects.filter(is_active=True).order_by("date_joined") return render( request, "person/list.html", context={"people": people} ) Then, in `templates/person/list.html`: {% extends 'base.html' %} {% block content %}
{% for person in people %}

{{person.first_name}} {{person.last_name}}

{% endfor %}
{% endblock %} Alternatives to Django's ORM and SQLAlchemy include but are not limited to: - Peewee: https://github.com/coleifer/peewee - PonyORM: https://ponyorm.com/ From jblack at nopam.com Tue Oct 24 11:33:56 2017 From: jblack at nopam.com (John Black) Date: Tue, 24 Oct 2017 10:33:56 -0500 Subject: choice of web-framework References: <59ec71e1$0$1750$e4fe514c@newszilla.xs4all.nl> Message-ID: In article , rosuav at gmail.com says... > > On Tue, Oct 24, 2017 at 6:57 AM, Chris Warrick wrote: > > On 23 October 2017 at 21:37, John Black wrote: > >> Chris, thanks for all this detailed information. I am confused though > >> with your database recommendation. You say you teach SQLAlchemy but > >> generally use PostgreSQL yourself. I can maybe guess why there seems to > >> be this contradiction. Perhaps PostgreSQL is better but too advanced for > >> the class you are teaching? Can you clarify on which you think is the > >> better choice? Thanks. > > > > Different Chris, but I?ll answer. Those are two very different things. > > > > PostgreSQL is a database server. It talks SQL to clients, stores data, > > retrieves it when asked. The usual stuff a database server does. > > Alternatives: SQLite, MySQL, MS SQL, Oracle DB, ? > > > > SQLAlchemy is an ORM: an object-relational mapper, and also a database > > toolkit. SQLAlchemy can abstract multiple database servers/engines > > (PostgreSQL, SQLite, MySQL, etc.) and work with them from the same > > codebase. It can also hide SQL from you and instead give you Python > > classes. If you use an ORM like SQLAlchemy, you get database support > > without writing a single line of SQL on your own. But you still need a > > database engine ? PostgreSQL can be one of them. But you can deploy > > the same code to different DB engines, and it will just work? > > (assuming you didn?t use any DB-specific features). Alternatives: > > Django ORM. > > > > psycopg2 is an example of a PostgreSQL client library for Python. It > > implements the Python DB-API and lets you use it to talk to a > > PostgreSQL server. When using psycopg2, you?re responsible for writing > > your own SQL statements for the server to execute. In that approach, > > you?re stuck with PostgreSQL and psycopg2 unless you rewrite your code > > to be compatible with the other database/library. Alternatives (other > > DBs): sqlite3, mysqlclient. There are also other PostgreSQL libraries > > available. > > > > Thanks, namesake :) > > The above is correct and mostly accurate. It IS possible to switch out > your back end fairly easily, though, even with psycopg2; there's a > standard API that most Python database packages follow. As long as you > stick to standard SQL (no PostgreSQL extensions) and the standard API > (no psycopg2 extensions), switching databases is as simple as changing > your "import psycopg2" into "import cx_oracle" or something. (And, > most likely, changing your database credentials.) > > The point of an ORM is to make your databasing code look and feel like > Python code, rather than manually crafting SQL statements everywhere. > Here's how a simple database operation looks in SQLAlchemy: Thank you Chris and Chris! John Black From lele at metapensiero.it Tue Oct 24 11:40:32 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Tue, 24 Oct 2017 17:40:32 +0200 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: <878tg0zh8v.fsf@metapensiero.it> Steve D'Aprano writes: > But given an empty file, how do you distinguish the empty file you get > from 'music.mp3' and the identical empty file you get from 'movie.avi'? That's simple enough: of course one empty file would be "music.mp3.zip.zip.zip", while the other would be "movie.avi.zip.zip.zip.zip.zip"... some sort of https://en.wikipedia.org/wiki/Water_memory applied to file system entries :-) ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From mail at timgolden.me.uk Tue Oct 24 11:46:18 2017 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 24 Oct 2017 16:46:18 +0100 Subject: Compression of random binary data In-Reply-To: <878tg0zh8v.fsf@metapensiero.it> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <878tg0zh8v.fsf@metapensiero.it> Message-ID: On 24/10/2017 16:40, Lele Gaifax wrote: > Steve D'Aprano writes: > >> But given an empty file, how do you distinguish the empty file you get >> from 'music.mp3' and the identical empty file you get from 'movie.avi'? > > That's simple enough: of course one empty file would be > "music.mp3.zip.zip.zip", while the other would be I swear this looks like the lyrics of something or another... "Music MP3 - zip - zip - zip" TJG From tmrsg11 at gmail.com Tue Oct 24 13:58:48 2017 From: tmrsg11 at gmail.com (C W) Date: Tue, 24 Oct 2017 13:58:48 -0400 Subject: h5py.File() gives error message Message-ID: Dear list, The following Python code gives an error message # Python code starts here: import numpy as np import h5py train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") # Python code ends The error message: train_dataset = h5py.File('train_catvnoncat.h5', "r") Traceback (most recent call last): File "", line 1, in File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", line 269, in __init__ fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", line 99, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 78, in h5py.h5f.open OSError: Unable to open file (unable to open file: name = 'train_catvnoncat.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0) My directory is correct, and the dataset folder with file is there. Why error message? Is it h5py.File() or is it my file? Everything seems pretty simple, what's going on? Thank you! From mal at egenix.com Tue Oct 24 14:47:58 2017 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 24 Oct 2017 20:47:58 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> Message-ID: <92875eb4-affb-e550-3d5e-3c880d26a731@egenix.com> On 22.10.2017 22:15, Karsten Hilbert wrote: > On Sat, Oct 21, 2017 at 07:10:31PM +0200, M.-A. Lemburg wrote: > >>> Running a debug build of py27 gave me a first lead: this >>> Debian system (Testing, upgraded all the way from various >>> releases ago) carries an incompatible mxDateTime which I'll >>> take care of. >>> >>> *** You don't have the (right) mxDateTime binaries installed ! >>> Traceback (most recent call last): >>> File "./bootstrap_gm_db_system.py", line 87, in >>> from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, gmI18N >>> File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", line 34, in >>> from Gnumed.pycommon import gmDateTime >>> File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", line 52, in >>> import mx.DateTime as mxDT >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 8, in >>> from DateTime import * >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 9, in >>> from mxDateTime import * >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 13, in >>> raise ImportError, why >>> ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: undefined symbol: Py_InitModule4 >> >> This error suggests that you have 32- and 64-bit versions of >> Python and mxDateTime mixed in your installation. >> >> Py_InitModule4 is only available in the 32-bit build of >> Python. With the 64-bit build, it's called Py_InitModule4_64. >> >> Since you're getting the same error from faulthandler, >> this is where I'd start to investigate. >> >> "nm" will list all exported and required symbols. As first step, >> you should probably check the python binary for its symbols and >> see whether it exports Py_InitModule* symbols. > > Thanks for your input ! > > The python2.7-dbg build is 32 bits: > > root at hermes:~# nm /usr/bin/python2.7-dbg | grep Py_InitM > 00155b9f T Py_InitModule4TraceRefs > > > python2.7-dbg: > Installiert: 2.7.14-2 > Installationskandidat: 2.7.14-2 > Versionstabelle: > *** 2.7.14-2 500 > 500 http://httpredir.debian.org/debian unstable/main i386 Packages > 100 /var/lib/dpkg/status > 2.7.13-2 990 > 500 http://httpredir.debian.org/debian stretch/main i386 Packages > 990 http://httpredir.debian.org/debian buster/main i386 Packages > > The python2.7 build (no -dbg) does not have symbols. > > mxDateTime really should be 32 bits, too: > > python-egenix-mxdatetime: > Installiert: 3.2.9-1 > Installationskandidat: 3.2.9-1 > Versionstabelle: > *** 3.2.9-1 990 > 500 http://httpredir.debian.org/debian stretch/main i386 Packages > 990 http://httpredir.debian.org/debian buster/main i386 Packages > 500 http://httpredir.debian.org/debian unstable/main i386 Packages > 100 /var/lib/dpkg/status > > Let me check the .so file: > > root at hermes:~# nm /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime_d.so | grep Py_InitM > U Py_InitModule4TraceRefs > > It seems it is - hm ... Could you check whether you have similar import errors with other modules that have C extensions ? E.g. lxml. What you're seeing appears to be a compilation problem with Python 2.7.14 on Debian. The executable doesn't appear to export its symbols to the .so files, or only some of them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From hjp-usenet3 at hjp.at Tue Oct 24 16:09:56 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Tue, 24 Oct 2017 22:09:56 +0200 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-23 04:21, Steve D'Aprano wrote: > On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >> > If the probability of certain codes (either single codes, or sequences of > codes) are non-equal, then you can take advantage of that by encoding the > common cases into a short representation, and the uncommon and rare cases > into a longer representation. As you say: > > >> Otherwise, if ( 0, 0 ) is much more frequent, >> we can encode ( 0, 0 ) by "0" and >> >> ( 0, 1 ) by "101", >> ( 1, 0 ) by "110", and >> ( 1, 1 ) by "111". >> >> And we could then use /less/ than two bits on the >> average. > > That's incorrect. On average you use 2.5 bits. > > (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) I disagree. If the distribution is not equal, then the average needs to take the different probabilities into account. Let's assume that (0, 0) has a probability of 90 %, (0, 1) a probability of 10 % and (1, 0) and (1, 1) a probability of 5 % each. Then the average length is 0.9 * 1 bit + 0.1 * 3 bits + 0.05 * 3 bits + 0.05 * 3 bits = 1.5 bits. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From pkpearson at nowhere.invalid Tue Oct 24 17:18:10 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 24 Oct 2017 21:18:10 GMT Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> Message-ID: On Tue, 24 Oct 2017 14:51:37 +1100, Steve D'Aprano wrote: On Tue, 24 Oct 2017 01:27 pm, danceswithnumbers at gmail.com wrote: > Yes! Decode reverse is easy......sorry so excited i could shout. Then this should be easy for you: http://marknelson.us/2012/10/09/the-random-compression-challenge-turns-ten/ All you need to do is compress this file: http://marknelson.us/attachments/million-digit-challenge/AMillionRandomDigits.bin to less than 415241 bytes, and you can win $100. Then, on Mon, 23 Oct 2017 21:13:00 -0700 (PDT), danceswithnumbers wrote: > I did that quite a while ago. But 352,954 kb > 415241 bytes, by several orders of magnitude; so you didn't "do that". (Or are we using the European decimal point?) If you're claiming 352,954 *bytes*, not kb, I invite you to explain why you have not collected Mark Nelson's $100 prize, and untold fame and glory; failing which, your credibility will evaporate. -- To email me, substitute nowhere->runbox, invalid->com. From ian.g.kelly at gmail.com Tue Oct 24 17:24:53 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 24 Oct 2017 15:24:53 -0600 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <59eeb8cb$0$18572$b1db1813$d948b532@news.astraweb.com> <071eb148-3da5-4991-85db-ab0070e8b0ca@googlegroups.com> Message-ID: On Tue, Oct 24, 2017 at 12:20 AM, Gregory Ewing wrote: > danceswithnumbers at gmail.com wrote: >> >> I did that quite a while ago. 352,954 kb. > > > Are you sure? Does that include the size of all the > code, lookup tables, etc. needed to decompress it? My bet is that danceswithnumbers does indeed have a file of that size which is in some way derived from the million random digits, but without any means of losslessly "decompressing" it (thus making it junk data). From steve+python at pearwood.info Tue Oct 24 18:11:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 25 Oct 2017 09:11:01 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <878tg0zh8v.fsf@metapensiero.it> Message-ID: <59efba76$0$18598$b1db1813$d948b532@news.astraweb.com> On Wed, 25 Oct 2017 02:40 am, Lele Gaifax wrote: > Steve D'Aprano writes: > >> But given an empty file, how do you distinguish the empty file you get >> from 'music.mp3' and the identical empty file you get from 'movie.avi'? > > That's simple enough: of course one empty file would be > "music.mp3.zip.zip.zip", while the other would be > "movie.avi.zip.zip.zip.zip.zip"... some sort of > https://en.wikipedia.org/wiki/Water_memory applied to file system entries > :-) Does that mean if I name an empty file serenity2-by-joss-whedon.avi.zip.zip.zip.zip.zip Dancerswithnumbers' magic algorithm will recreate the movie from some alternative universe where it actually exists? Awesome. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 24 18:30:57 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 25 Oct 2017 09:30:57 +1100 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> Message-ID: <59efbf23$0$18562$b1db1813$d948b532@news.astraweb.com> On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: > On 2017-10-23 04:21, Steve D'Aprano wrote: >> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >>> >> If the probability of certain codes (either single codes, or sequences of >> codes) are non-equal, then you can take advantage of that by encoding the >> common cases into a short representation, and the uncommon and rare cases >> into a longer representation. As you say: >> >> >>> Otherwise, if ( 0, 0 ) is much more frequent, >>> we can encode ( 0, 0 ) by "0" and >>> >>> ( 0, 1 ) by "101", >>> ( 1, 0 ) by "110", and >>> ( 1, 1 ) by "111". >>> >>> And we could then use /less/ than two bits on the >>> average. >> >> That's incorrect. On average you use 2.5 bits. >> >> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) > > I disagree. If the distribution is not equal, then the average needs to > take the different probabilities into account. I think I would call that the *weighted* average rather than the average. Regardless of what we call it, of course both you and Stefan are right in how to calculate it, and such a variable-length scheme can be used to compress the data. E.g. given the sequence 00000011 which would take 8 bits in the obvious encoding, we can encode it as "000111" which takes only 6 bits. But the cost of this encoding scheme is that *some* bit sequences expand, e.g. the 8 bit sequence 11111100 is encoded as "1111111110" which requires 10 bits. The end result is that averaged over all possible bit sequences (of a certain size), this encoding scheme requires MORE space than the obvious 0/1 bits. But in practice we don't care much, because the data sequences we care about are usually not "all possible bit sequences", but a heavily restricted subset where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Oct 24 19:09:42 2017 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 25 Oct 2017 10:09:42 +1100 Subject: Compression of random binary data In-Reply-To: <59efba76$0$18598$b1db1813$d948b532@news.astraweb.com> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <878tg0zh8v.fsf@metapensiero.it> <59efba76$0$18598$b1db1813$d948b532@news.astraweb.com> Message-ID: On Wed, Oct 25, 2017 at 9:11 AM, Steve D'Aprano wrote: > On Wed, 25 Oct 2017 02:40 am, Lele Gaifax wrote: > >> Steve D'Aprano writes: >> >>> But given an empty file, how do you distinguish the empty file you get >>> from 'music.mp3' and the identical empty file you get from 'movie.avi'? >> >> That's simple enough: of course one empty file would be >> "music.mp3.zip.zip.zip", while the other would be >> "movie.avi.zip.zip.zip.zip.zip"... some sort of >> https://en.wikipedia.org/wiki/Water_memory applied to file system entries >> :-) > > > Does that mean if I name an empty file > > serenity2-by-joss-whedon.avi.zip.zip.zip.zip.zip > > Dancerswithnumbers' magic algorithm will recreate the movie from some > alternative universe where it actually exists? > > Awesome. Yes, but then you'd get dmca-takedown-request.pdf.zip.zip.zip.zip.zip.zip.zip which would also be empty. ChrisA From Richard at Damon-Family.org Tue Oct 24 19:15:28 2017 From: Richard at Damon-Family.org (Richard Damon) Date: Tue, 24 Oct 2017 19:15:28 -0400 Subject: Compression of random binary data In-Reply-To: <59efbf23$0$18562$b1db1813$d948b532@news.astraweb.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <59efbf23$0$18562$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/24/17 6:30 PM, Steve D'Aprano wrote: > On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: > >> On 2017-10-23 04:21, Steve D'Aprano wrote: >>> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >>>> >>> If the probability of certain codes (either single codes, or sequences of >>> codes) are non-equal, then you can take advantage of that by encoding the >>> common cases into a short representation, and the uncommon and rare cases >>> into a longer representation. As you say: >>> >>> >>>> Otherwise, if ( 0, 0 ) is much more frequent, >>>> we can encode ( 0, 0 ) by "0" and >>>> >>>> ( 0, 1 ) by "101", >>>> ( 1, 0 ) by "110", and >>>> ( 1, 1 ) by "111". >>>> >>>> And we could then use /less/ than two bits on the >>>> average. >>> >>> That's incorrect. On average you use 2.5 bits. >>> >>> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) >> >> I disagree. If the distribution is not equal, then the average needs to >> take the different probabilities into account. > > I think I would call that the *weighted* average rather than the average. > > Regardless of what we call it, of course both you and Stefan are right in how > to calculate it, and such a variable-length scheme can be used to compress > the data. > > E.g. given the sequence 00000011 which would take 8 bits in the obvious > encoding, we can encode it as "000111" which takes only 6 bits. > > But the cost of this encoding scheme is that *some* bit sequences expand, e.g. > the 8 bit sequence 11111100 is encoded as "1111111110" which requires 10 > bits. > > The end result is that averaged over all possible bit sequences (of a certain > size), this encoding scheme requires MORE space than the obvious 0/1 bits. > > But in practice we don't care much, because the data sequences we care about > are usually not "all possible bit sequences", but a heavily restricted subset > where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. > My understanding of the 'Random Data Comprehensibility' challenge is that is requires that the compression take ANY/ALL strings of up to N bits, and generate an output stream no longer than the input stream, and sometime less. It admits that given no-uniformly distributed data, it is possible to compress some patterns, the common ones, and expand others, the uncommon ones, to lower the net average length. What it says can't be done is to have a compression method that compresses EVERY input pattern. That is where the 'Pigeon Hole' principle comes into play which the people who claim to be able to compress random data like to ignore or just attempt to say doesn't apply. From rgaddi at highlandtechnology.invalid Tue Oct 24 21:02:26 2017 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 24 Oct 2017 18:02:26 -0700 Subject: h5py.File() gives error message In-Reply-To: References: Message-ID: On 10/24/2017 10:58 AM, C W wrote: > Dear list, > > The following Python code gives an error message > > # Python code starts here: > import numpy as np > import h5py > train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > > # Python code ends > > The error message: > > train_dataset = h5py.File('train_catvnoncat.h5', "r") > Traceback (most recent call last): > File "", line 1, in > File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", > line 269, in __init__ > fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) > File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", > line 99, in make_fid > fid = h5f.open(name, flags, fapl=fapl) > File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper > File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper > File "h5py/h5f.pyx", line 78, in h5py.h5f.open > OSError: Unable to open file (unable to open file: name = > 'train_catvnoncat.h5', errno = 2, error message = 'No such file or > directory', flags = 0, o_flags = 0) > > My directory is correct, and the dataset folder with file is there. > > Why error message? Is it h5py.File() or is it my file? Everything seems > pretty simple, what's going on? > > Thank you! > Be 100% sure your directory is correct. Try it again with an absolute path to the file. Windows makes it far too easy for the working directory of a program to be other than what you think it is. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From cs at cskk.id.au Wed Oct 25 01:10:02 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 25 Oct 2017 16:10:02 +1100 Subject: Sockets but calling from different programs In-Reply-To: <938a5a08-eebe-4a21-a621-3507019c83d2@googlegroups.com> References: <938a5a08-eebe-4a21-a621-3507019c83d2@googlegroups.com> Message-ID: <20171025051002.GA91760@cskk.homeip.net> On 23Oct2017 05:33, T Obulesu wrote: >I'm new to python3 and scratching my head to write a program for this logic: The tutor list might be a better place for such questions, but since we're here... >classA.py >Class A: > # class for socket communication > basic init method that initializes port, address, connection > method send(message): > # for sending any message through the given port > method receive(): > # will be keep on listening for incoming messages > >classB.py >Class B: > import the send method from class A > send the messages from this class > >classC.py >Class C: > import the receive method from the class A > receive all the messages from the same socket from here. > >Note: > classA.py, classB.py, ClassC.py are saved in different locations. > >Can someone help me in writing the code and how can I create a single object >and use it in multiple classed? That is a pretty normal arrangement. Class A might look like this: class A: def __init__(self, port, address): self.connection = ... make your connection to (address, port) def send(self, msg): send msg using self.connection ... Since classes B and C seem expected to share tha same connection, the natural thing is to set up the connection _separately_ from setting up B and C, and pass the established connection to each. So class B might commence: class B: def __init__(self, conn, ...): self.conn = conn ... whatever other initialisation ... def speak(self, something): self.conn.send(something) You'll notice here that we're _not_ importing anything about class A here. Class B does not need to know class A's name to use it. Because Python uses duck typing, you could pass _any_ object which has a .send() method as "conn" to the class B setup. This allows you to write some other class A2 with those same methods, but using some different type of connection, and pass that in to classes B and C. So a main programme might set things up like this: from classA import A from classB import B from classC import C conn = A(address, port) sender = B(conn, other-stuff...) receiver = C(conn, other-stuff...) B.speak("foo") and so forth. Cheers, Cameron Simpson (formerly cs at zip.com.au) From nomail at com.invalid Wed Oct 25 02:49:00 2017 From: nomail at com.invalid (ast) Date: Wed, 25 Oct 2017 08:49:00 +0200 Subject: Objects with __name__ attribute Message-ID: <59f033de$0$4836$426a74cc@news.free.fr> Hi, I know two Python's objects which have an intrinsic name, classes and functions. def f(): pass >>> f.__name__ 'f' >>> g = f >>> g.__name__ 'f' class Test: pass >>> Test.__name__ 'Test' >>> Test2 = Test >>> Test2.__name__ 'Test' Are there others objects with a __name__ attribute and what is it used for ? Regards From __peter__ at web.de Wed Oct 25 03:21:22 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 25 Oct 2017 09:21:22 +0200 Subject: Objects with __name__ attribute References: <59f033de$0$4836$426a74cc@news.free.fr> Message-ID: ast wrote: > Hi, > > I know two Python's objects which have an intrinsic > name, classes and functions. > > def f(): > pass > >>>> f.__name__ > 'f' >>>> g = f >>>> g.__name__ > 'f' > > class Test: > pass > >>>> Test.__name__ > 'Test' >>>> Test2 = Test >>>> Test2.__name__ > 'Test' > > Are there others objects with a __name__ attribute > and what is it used for ? > > Regards It was used for the object's repr(): $ python Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> def f(): pass ... >>> f.__name__ = "spanish inquisition" >>> f But this has changed: $ python3 Python 3.4.3 (default, Nov 17 2016, 01:08:31) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> def f(): pass ... >>> f.__name__ = "spanish inquisition" >>> f >>> f.__qualname__ = "spanish inquisition" >>> f You should be aware that the module name is used to detect the main module: if __name__ == "__main__": print("this is run as a script") From tmrsg11 at gmail.com Wed Oct 25 03:21:34 2017 From: tmrsg11 at gmail.com (C W) Date: Wed, 25 Oct 2017 03:21:34 -0400 Subject: h5py.File() gives error message In-Reply-To: References: Message-ID: Oh, I was running a debug file, that's why the path is different. The file is here, https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 Is anyone able to get it working? Thank you! On Tue, Oct 24, 2017 at 10:37 PM, Dennis Lee Bieber wrote: > On Tue, 24 Oct 2017 18:02:26 -0700, Rob Gaddi > declaimed the following: > > Whoops, I may have gotten the wrong level of quoting -- my > apologies if > so (I did have agent fetch the original posting, but might not have had > that active when I hit "reply") > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list > From greg.ewing at canterbury.ac.nz Wed Oct 25 03:32:11 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 25 Oct 2017 20:32:11 +1300 Subject: Compression of random binary data In-Reply-To: <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > - Encrypted data looks very much like random noise. There's actually a practical use for that idea. If you can feed the output of an encryption algorithm through a compressor and make it smaller, it means there is a cryptographic weakness in the algorithm that could potentially be exploited. Good encryption algorithms produce output that looks almost completely random to anyone who doesn't know how to decrypt it. -- Greg From greg.ewing at canterbury.ac.nz Wed Oct 25 04:11:25 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 25 Oct 2017 21:11:25 +1300 Subject: Compression of random binary data In-Reply-To: <87wp3khdy9.fsf@bsb.me.uk> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> Message-ID: Ben Bacarisse wrote: > The trouble is a pedagogic one. Saying "you can't compress random data" > inevitably leads (though, again, this is just my experience) to endless > attempts to define random data. It's more about using terms without making sure everyone agrees on the definitions being used. In this context, "random data" really means "uniformly distributed data", i.e. any bit sequence is equally likely to be presented as input. *That's* what information theory says can't be compressed. > I think "arbitrary data" (thereby including the results of compression > by said algorithm) is the best way to make progress. I'm not sure that's much better, because it doesn't home in on the most important thing, which is the probability distribution. -- Greg From greg.ewing at canterbury.ac.nz Wed Oct 25 04:25:07 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 25 Oct 2017 21:25:07 +1300 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <878tg0zh8v.fsf@metapensiero.it> Message-ID: Lele Gaifax wrote: > That's simple enough: of course one empty file would be > "music.mp3.zip.zip.zip", while the other would be > "movie.avi.zip.zip.zip.zip.zip"... some sort of > https://en.wikipedia.org/wiki/Water_memory applied to file system entries :-) If you're allowed to alternate between two compression methods, then the way you decompress music.mp3.zip.zip.tgz.zip...........tgz.zip.tgz is to output 0 each time zip was applied and 1 each time tar/gz was applied. You may be able to take some shortcuts in some cases, e.g. anything beginning with "movie.flv" almost certainly contains a cute kitten video. (Unless it's found inside an encrypted disk partition, in which case it contains porn.) -- Greg From __peter__ at web.de Wed Oct 25 05:20:56 2017 From: __peter__ at web.de (Peter Otten) Date: Wed, 25 Oct 2017 11:20:56 +0200 Subject: h5py.File() gives error message References: Message-ID: C W wrote: > Oh, I was running a debug file, that's why the path is different. > > The file is here, > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 > > Is anyone able to get it working? Thank you! Hm, that file seems to contain HTML and that causes an OSError here, too: $ head -n3 datasets/train_catvnoncat.h5 $ python3 Python 3.4.3 (default, Nov 17 2016, 01:08:31) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import h5py >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in __init__ fid = make_fid(name, mode, userblock_size, fapl) File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in make_fid fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl) File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806) OSError: unable to open file (File accessibilty: Unable to open file) It's not exactly what you see, but that may be due to differing software versions. When I replace the HTML file with its namesake found at https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/code/datasets/train_catvnoncat.h5 I can open it: $ file datasets/train_catvnoncat.h5 datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data $ python3 Python 3.4.3 (default, Nov 17 2016, 01:08:31) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import h5py >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") >>> From Karsten.Hilbert at gmx.net Wed Oct 25 05:51:07 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Wed, 25 Oct 2017 11:51:07 +0200 Subject: right list for SIGABRT python binary question ? In-Reply-To: <92875eb4-affb-e550-3d5e-3c880d26a731@egenix.com> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> <92875eb4-affb-e550-3d5e-3c880d26a731@egenix.com> Message-ID: <20171025095106.vxktihve4bgkdqpg@hermes.hilbert.loc> On Tue, Oct 24, 2017 at 08:47:58PM +0200, M.-A. Lemburg wrote: > >> This error suggests that you have 32- and 64-bit versions of > >> Python and mxDateTime mixed in your installation. > >> > >> Py_InitModule4 is only available in the 32-bit build of > >> Python. With the 64-bit build, it's called Py_InitModule4_64. ... > Could you check whether you have similar import errors with > other modules that have C extensions ? E.g. lxml. > > What you're seeing appears to be a compilation problem > with Python 2.7.14 on Debian. The executable doesn't appear > to export its symbols to the .so files, or only some of them. Let's see: python-lxml: Installiert: 4.0.0-1 Installationskandidat: 4.0.0-1 Versionstabelle: *** 4.0.0-1 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 3.7.1-1 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages python-lxml-dbg: Installiert: (keine) Installationskandidat: 4.0.0-1 Versionstabelle: 4.0.0-1 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 3.7.1-1 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages ncq at hermes:~$ python2.7-dbg Python 2.7.14 (default, Sep 17 2017, 18:50:44) [GCC 7.2.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import lxml [45350 refs] >>> ncq at hermes:~$ python Python 2.7.14 (default, Sep 17 2017, 18:50:44) [GCC 7.2.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import lxml >>> Also, psycogp2 imports just fine. Now that I have python-egenix-mx-base-dbg: Installiert: 3.2.9-1 Installationskandidat: 3.2.9-1 Versionstabelle: *** 3.2.9-1 990 500 http://httpredir.debian.org/debian stretch/main i386 Packages 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status mx.DateTime imports fine as well (but the SIGABRT persists). Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From tjol at tjol.eu Wed Oct 25 09:07:24 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 25 Oct 2017 15:07:24 +0200 Subject: Let's talk about debuggers! Message-ID: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Hi, I just wanted to know what tools everyone used for debugging Python applications - scripts / backend / desktop apps / notebooks / whatever. Apart from the usual dance with log files and strategically inserted print() calls, that is. Of course we all know and mildly dislike pdb. Personally, in practice, I'm most likely to need a debugger when prototyping a function in a Jupyter notebook. There, ipdb, summoned with the %%debug magic incantation, does the trick. Sometimes, though, I miss having a visual debugger. You know, the kind that Visual Basic has had for decades. There's one in Chrome dev tools if you ever have the misfortune of writing JavaScript. What options are there for Python (that work)? What text editors (and IDEs) have a decent integrated debugger or debugging plugin? (Is there anything for Sublime?) Does anyone use them? (How do YOU debug?) I vaguely remember WinPDB, but that hasn't seen a release in more than seven years... -- Thomas Jollans From ian.g.kelly at gmail.com Wed Oct 25 09:27:59 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Wed, 25 Oct 2017 07:27:59 -0600 Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <59efbf23$0$18562$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/24/17, Richard Damon wrote: > My understanding of the 'Random Data Comprehensibility' challenge is > that is requires that the compression take ANY/ALL strings of up to N > bits, and generate an output stream no longer than the input stream, and > sometime less. That's incorrect, at least of the challenge being discussed. Here's the link: http://marknelson.us/2012/10/09/the-random-compression-challenge-turns-ten/ The challenge is just to take a single known file of a million random digits and make it smaller, including the size of the decompresser and without hiding data. So far in 15 years nobody has succeeded even at this, and nobody knows whether it's impossible. For instance it may be the case that the data in the file happens to be the nth prime, in which case it could simply be compressed to the number n with a decompresser that calculates process. > It admits that given no-uniformly distributed data, it is > possible to compress some patterns, the common ones, and expand others, > the uncommon ones, to lower the net average length. What it says can't > be done is to have a compression method that compresses EVERY input > pattern. That is where the 'Pigeon Hole' principle comes into play which > the people who claim to be able to compress random data like to ignore > or just attempt to say doesn't apply. There is a second challenge on that page that is as you state, but the page admits that this second challenge is a bit of a troll since this is provably impossible. From fabien.maussion at gmail.com Wed Oct 25 09:31:21 2017 From: fabien.maussion at gmail.com (Fabien) Date: Wed, 25 Oct 2017 15:31:21 +0200 Subject: Let's talk about debuggers! References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: On 10/25/2017 03:07 PM, Thomas Jollans wrote: > What options are there for Python (that work)? PyCharm's debugger is fine (also available in the community edition) Cheers, Fabien From ned at nedbatchelder.com Wed Oct 25 09:53:10 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 25 Oct 2017 09:53:10 -0400 Subject: Let's talk about debuggers! In-Reply-To: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: On 10/25/17 9:07 AM, Thomas Jollans wrote: > Hi, > > I just wanted to know what tools everyone used for debugging Python > applications - scripts / backend / desktop apps / notebooks / whatever. > Apart from the usual dance with log files and strategically inserted > print() calls, that is. > > Of course we all know and mildly dislike pdb. > > Personally, in practice, I'm most likely to need a debugger when > prototyping a function in a Jupyter notebook. There, ipdb, summoned with > the %%debug magic incantation, does the trick. > > Sometimes, though, I miss having a visual debugger. You know, the kind > that Visual Basic has had for decades. There's one in Chrome dev tools > if you ever have the misfortune of writing JavaScript. > > What options are there for Python (that work)? What text editors (and > IDEs) have a decent integrated debugger or debugging plugin? (Is there > anything for Sublime?) Does anyone use them? (How do YOU debug?) > > I vaguely remember WinPDB, but that hasn't seen a release in more than > seven years... > > pudb is a visual terminal debugger: https://pypi.python.org/pypi/pudb It uses the same commands as pdb, so it's easy to get started, but it gives you a variables pane, with customizable presentation, and so on. One of my favorite features: you can add a set_trace line in your program, and then if during the debugging session you realize you don't want to stop there every time, you can disable that breakpoint even though it's an explicit line of code telling the debugger to stop. --Ned. From rustompmody at gmail.com Wed Oct 25 09:57:04 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 25 Oct 2017 06:57:04 -0700 (PDT) Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> On Wednesday, October 25, 2017 at 6:37:47 PM UTC+5:30, Thomas Jollans wrote: > Hi, > > I just wanted to know what tools everyone used for debugging Python > applications - scripts / backend / desktop apps / notebooks / whatever. > Apart from the usual dance with log files and strategically inserted > print() calls, that is. > > Of course we all know and mildly dislike pdb. > > Personally, in practice, I'm most likely to need a debugger when > prototyping a function in a Jupyter notebook. There, ipdb, summoned with > the %%debug magic incantation, does the trick. > > Sometimes, though, I miss having a visual debugger. You know, the kind > that Visual Basic has had for decades. There's one in Chrome dev tools > if you ever have the misfortune of writing JavaScript. > > What options are there for Python (that work)? What text editors (and > IDEs) have a decent integrated debugger or debugging plugin? (Is there > anything for Sublime?) Does anyone use them? (How do YOU debug?) > > I vaguely remember WinPDB, but that hasn't seen a release in more than > seven years... pdb inside emacs works (to a fashion) And it shows the arrow for current line so its at least quasi-gui I believe idle too is much more usable than a few years earlier From tmrsg11 at gmail.com Wed Oct 25 09:58:20 2017 From: tmrsg11 at gmail.com (C W) Date: Wed, 25 Oct 2017 09:58:20 -0400 Subject: h5py.File() gives error message In-Reply-To: References: Message-ID: wow, thanks so much! I don't know how you figured that it's HTML, but that's awesome! Mike On Wed, Oct 25, 2017 at 5:20 AM, Peter Otten <__peter__ at web.de> wrote: > C W wrote: > > > Oh, I was running a debug file, that's why the path is different. > > > > The file is here, > > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 > > > > Is anyone able to get it working? Thank you! > > Hm, that file seems to contain HTML and that causes an OSError here, too: > > $ head -n3 datasets/train_catvnoncat.h5 > > > > > $ python3 > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import h5py > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in > __init__ > fid = make_fid(name, mode, userblock_size, fapl) > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in > make_fid > fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl) > File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806) > OSError: unable to open file (File accessibilty: Unable to open file) > > It's not exactly what you see, but that may be due to differing software > versions. > When I replace the HTML file with its namesake found at > > https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/cod > e/datasets/train_catvnoncat.h5 > > I can open it: > > $ file datasets/train_catvnoncat.h5 > datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data > > $ python3 > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import h5py > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > >>> > > -- > https://mail.python.org/mailman/listinfo/python-list > From michele.simionato at gmail.com Wed Oct 25 10:08:31 2017 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 25 Oct 2017 07:08:31 -0700 (PDT) Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: pdb plus plus: https://pypi.python.org/pypi/pdbpp From wrw at mac.com Wed Oct 25 10:30:28 2017 From: wrw at mac.com (William Ray Wing) Date: Wed, 25 Oct 2017 10:30:28 -0400 Subject: Let's talk about debuggers! In-Reply-To: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: <823B3F9D-A726-4927-A22D-2D45B1A250BE@mac.com> > On Oct 25, 2017, at 9:07 AM, Thomas Jollans wrote: > > [byte] > What options are there for Python (that work)? What text editors (and > IDEs) have a decent integrated debugger or debugging plugin? I rather like WingIDE (the name is a coincidence). It allows insertion/removal of break points while the code is running. While execution is stopped, it allows direct inspection of the stack (no surprise), but in addition allows execution of python statements or program elements typed into an auxiliary window - including importing things like matplotlib and plotting the current state of data arrays. Its editor is syntax-aware and highlights accidental syntax errors as they are typed. Lots of other features, those just happen to be the ones I use most often. Bill From BILL_NOSPAM at whoknows.net Wed Oct 25 10:46:47 2017 From: BILL_NOSPAM at whoknows.net (Bill) Date: Wed, 25 Oct 2017 10:46:47 -0400 Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: Fabien wrote: > On 10/25/2017 03:07 PM, Thomas Jollans wrote: >> What options are there for Python (that work)? > > PyCharm's debugger is fine (also available in the community edition) +1 > > Cheers, > > Fabien From danceswithnumbers at gmail.com Wed Oct 25 10:49:34 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Wed, 25 Oct 2017 07:49:34 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> Message-ID: <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> Whatever you do, you'll find that *on average* you will need *at least* 34 bits to be able to represent all possible 10-digit decimal numbers. Some might be shorter, but then others will be longer, and the average won't be less than 34. The theoretical limit for arbitrary numbers 0 - 9 must be viewed as an average not as an impossibility to, in some cases be able to compress to or under 34. This is obvious by the decimal to binary function. From formisc at gmail.com Wed Oct 25 11:46:10 2017 From: formisc at gmail.com (Andrew Z) Date: Wed, 25 Oct 2017 11:46:10 -0400 Subject: Ide vs ide In-Reply-To: References: Message-ID: Yeah, lets start the war! // joking! But if i think about it... there are tons articles and flame wars about "a vs b". And yet, what if the question should be different: If you were to create the "ide" for yourself (think lego) , what are the functions that you _use_ and like a lot? From kkgsrkmk at gmail.com Wed Oct 25 11:46:59 2017 From: kkgsrkmk at gmail.com (kkgsrkmk at gmail.com) Date: Wed, 25 Oct 2017 08:46:59 -0700 (PDT) Subject: Test Bank for Campbell Biology 11th Edition by Urry, Cain In-Reply-To: <3b77e8e5-f5e4-4fc6-abc9-73e42a89f429@googlegroups.com> References: <3b77e8e5-f5e4-4fc6-abc9-73e42a89f429@googlegroups.com> Message-ID: Where it didnt come From tjol at tjol.eu Wed Oct 25 12:12:32 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 25 Oct 2017 18:12:32 +0200 Subject: Let's talk about debuggers! In-Reply-To: <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> Message-ID: On 2017-10-25 15:57, Rustom Mody wrote: > > pdb inside emacs works (to a fashion) > And it shows the arrow for current line so its at least quasi-gui > > I believe idle too is much more usable than a few years earlier I haven't used IDLE in years (if ever), partly because Tkinter is so incredibly ugly, and feels so out-of-place on a modern PC (maybe it's tolerable on Windows, I wouldn't know). In any case I just tried it out and good lord that's terrible. You can set breakpoints in the editor (good), it shows locals (excellent), but it doesn't show you what line you're at when stepping through (seriously?). I'll go back to not even touching IDLE with a ten-foot pole now. -- Thomas Jollans From dchilapondwa3 at gmail.com Wed Oct 25 12:19:50 2017 From: dchilapondwa3 at gmail.com (Deught Chilapondwa) Date: Wed, 25 Oct 2017 18:19:50 +0200 Subject: Python-list Digest, Vol 169, Issue 44 In-Reply-To: References: Message-ID: Sir, You have been sending me alot of messages but hardly can I understand. Offcourse I registered with you, but I can't understand the messages reharding what I should do. Can you come back to me with clarification? Impatienly waiting to hear from you. Deught Chilapondwa. On 25 Oct 2017 6:01 pm, wrote: > Send Python-list mailing list submissions to > python-list at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-list > or, via email, send a message with subject or body 'help' to > python-list-request at python.org > > You can reach the person managing the list at > python-list-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-list digest..." > > Today's Topics: > > 1. Re: right list for SIGABRT python binary question ? > (M.-A. Lemburg) > 2. Re: Compression of random binary data (Stefan Ram) > 3. Re: Compression of random binary data (Peter J. Holzer) > 4. Re: Compression of random binary data (Peter Pearson) > 5. Re: Compression of random binary data (Ian Kelly) > 6. Re: Compression of random binary data (Steve D'Aprano) > 7. Re: Compression of random binary data (Steve D'Aprano) > 8. Re: Compression of random binary data (Chris Angelico) > 9. Re: Compression of random binary data (Richard Damon) > 10. h5py.File() gives error message (C W) > 11. Re: h5py.File() gives error message (Rob Gaddi) > 12. Re: h5py.File() gives error message (Dennis Lee Bieber) > 13. Re: h5py.File() gives error message (Dennis Lee Bieber) > 14. Objects with __name__ attribute (ast) > 15. Re: Objects with __name__ attribute (Peter Otten) > 16. Re: h5py.File() gives error message (C W) > 17. Re: Compression of random binary data (Gregory Ewing) > 18. Re: Compression of random binary data (Gregory Ewing) > 19. Re: Compression of random binary data (Gregory Ewing) > 20. Re: Sockets but calling from different programs (Cameron Simpson) > 21. Re: h5py.File() gives error message (Peter Otten) > 22. Re: right list for SIGABRT python binary question ? > (Karsten Hilbert) > 23. Let's talk about debuggers! (Thomas Jollans) > 24. Re: Compression of random binary data (Ian Kelly) > 25. Re: Let's talk about debuggers! (Fabien) > 26. Re: h5py.File() gives error message (C W) > 27. Re: Let's talk about debuggers! (Rustom Mody) > 28. Re: Let's talk about debuggers! (Ned Batchelder) > 29. Re: Let's talk about debuggers! (Michele Simionato) > 30. Re: Let's talk about debuggers! (Bill) > 31. Re: Compression of random binary data > (danceswithnumbers at gmail.com) > 32. Re: Let's talk about debuggers! (William Ray Wing) > 33. Test Bank for Campbell Biology 11th Edition by Urry, Cain > (kkgsrkmk at gmail.com) > > > ---------- Forwarded message ---------- > From: "M.-A. Lemburg" > To: Karsten Hilbert , python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 20:47:58 +0200 > Subject: Re: right list for SIGABRT python binary question ? > > > On 22.10.2017 22:15, Karsten Hilbert wrote: > > On Sat, Oct 21, 2017 at 07:10:31PM +0200, M.-A. Lemburg wrote: > > > >>> Running a debug build of py27 gave me a first lead: this > >>> Debian system (Testing, upgraded all the way from various > >>> releases ago) carries an incompatible mxDateTime which I'll > >>> take care of. > >>> > >>> *** You don't have the (right) mxDateTime binaries installed ! > >>> Traceback (most recent call last): > >>> File "./bootstrap_gm_db_system.py", line 87, in > >>> from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, > gmI18N > >>> File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", > line 34, in > >>> from Gnumed.pycommon import gmDateTime > >>> File "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", > line 52, in > >>> import mx.DateTime as mxDT > >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", > line 8, in > >>> from DateTime import * > >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", > line 9, in > >>> from mxDateTime import * > >>> File "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", > line 13, in > >>> raise ImportError, why > >>> ImportError: /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: > undefined symbol: Py_InitModule4 > >> > >> This error suggests that you have 32- and 64-bit versions of > >> Python and mxDateTime mixed in your installation. > >> > >> Py_InitModule4 is only available in the 32-bit build of > >> Python. With the 64-bit build, it's called Py_InitModule4_64. > >> > >> Since you're getting the same error from faulthandler, > >> this is where I'd start to investigate. > >> > >> "nm" will list all exported and required symbols. As first step, > >> you should probably check the python binary for its symbols and > >> see whether it exports Py_InitModule* symbols. > > > > Thanks for your input ! > > > > The python2.7-dbg build is 32 bits: > > > > root at hermes:~# nm /usr/bin/python2.7-dbg | grep Py_InitM > > 00155b9f T Py_InitModule4TraceRefs > > > > > > python2.7-dbg: > > Installiert: 2.7.14-2 > > Installationskandidat: 2.7.14-2 > > Versionstabelle: > > *** 2.7.14-2 500 > > 500 http://httpredir.debian.org/debian unstable/main i386 > Packages > > 100 /var/lib/dpkg/status > > 2.7.13-2 990 > > 500 http://httpredir.debian.org/debian stretch/main i386 > Packages > > 990 http://httpredir.debian.org/debian buster/main i386 > Packages > > > > The python2.7 build (no -dbg) does not have symbols. > > > > mxDateTime really should be 32 bits, too: > > > > python-egenix-mxdatetime: > > Installiert: 3.2.9-1 > > Installationskandidat: 3.2.9-1 > > Versionstabelle: > > *** 3.2.9-1 990 > > 500 http://httpredir.debian.org/debian stretch/main i386 > Packages > > 990 http://httpredir.debian.org/debian buster/main i386 > Packages > > 500 http://httpredir.debian.org/debian unstable/main i386 > Packages > > 100 /var/lib/dpkg/status > > > > Let me check the .so file: > > > > root at hermes:~# nm /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime_d.so > | grep Py_InitM > > U Py_InitModule4TraceRefs > > > > It seems it is - hm ... > > Could you check whether you have similar import errors with > other modules that have C extensions ? E.g. lxml. > > What you're seeing appears to be a compilation problem > with Python 2.7.14 on Debian. The executable doesn't appear > to export its symbols to the .so files, or only some of them. > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Experts > >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ > >>> Python Database Interfaces ... http://products.egenix.com/ > >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ > ________________________________________________________________________ > > ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > > > > ---------- Forwarded message ---------- > From: Stefan Ram > To: python-list at python.org > Cc: > Bcc: > Date: 24 Oct 2017 18:55:34 GMT > Subject: Re: Compression of random binary data > Ben Bacarisse writes: > >All generalisations are false. I was being hyperbolic. > > I think that I already gave my definition recently: > > A bit source is /random/, when every party has a > probability of no more than 0.5 to correctly predict > the next bit in a black-box test (not knowing the > internals of the source). > > I add today: > > The above definition assumes that the bit source does > not learn about the predictions, so that it cannot > intentionally generate bits depending on the predictions > of any party. > > We cannot experimentally test whether the assertion that > a bit source is random is true (in an absolute sense of > "true"), but at least we can test whether a bit source is > random with /a certain confidence/ (such as 0.99). > > Or, when we know how the generator is built, we can predict > whether it should be random according to its construction > and our theories. > > Using quantum devices we can drop the black-box condition > and even construct white-box random devices. > > Quantum mechanics surely teaches us how to prepare a photon > so that the probability of it passing a certain polarization > filter is 0.5. And we can say that this is a perfect random > generator and even use it to /define/ the meaning of "random": > The bits it gives to us are "random bits". This is the > ideal random bit generator, and an algorithm can provide an > approximation of its behavior. See also: arxiv.org/pdf/1004.1521 > . > > > > > ---------- Forwarded message ---------- > From: "Peter J. Holzer" > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 22:09:56 +0200 > Subject: Re: Compression of random binary data > On 2017-10-23 04:21, Steve D'Aprano wrote: > > On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: > >> > > If the probability of certain codes (either single codes, or sequences of > > codes) are non-equal, then you can take advantage of that by encoding the > > common cases into a short representation, and the uncommon and rare cases > > into a longer representation. As you say: > > > > > >> Otherwise, if ( 0, 0 ) is much more frequent, > >> we can encode ( 0, 0 ) by "0" and > >> > >> ( 0, 1 ) by "101", > >> ( 1, 0 ) by "110", and > >> ( 1, 1 ) by "111". > >> > >> And we could then use /less/ than two bits on the > >> average. > > > > That's incorrect. On average you use 2.5 bits. > > > > (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) > > I disagree. If the distribution is not equal, then the average needs to > take the different probabilities into account. > > Let's assume that (0, 0) has a probability of 90 %, (0, 1) a probability > of 10 % and (1, 0) and (1, 1) a probability of 5 % each. > > Then the average length is > > 0.9 * 1 bit + 0.1 * 3 bits + 0.05 * 3 bits + 0.05 * 3 bits = 1.5 bits. > > hp > > > -- > _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: > |_|_) | | Man feilt solange an seinen Text um, bis > | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr > __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel > > > > ---------- Forwarded message ---------- > From: Peter Pearson > To: python-list at python.org > Cc: > Bcc: > Date: 24 Oct 2017 21:18:10 GMT > Subject: Re: Compression of random binary data > On Tue, 24 Oct 2017 14:51:37 +1100, Steve D'Aprano wrote: > On Tue, 24 Oct 2017 01:27 pm, danceswithnumbers at gmail.com wrote: > > Yes! Decode reverse is easy......sorry so excited i could shout. > > Then this should be easy for you: > > http://marknelson.us/2012/10/09/the-random-compression- > challenge-turns-ten/ > > All you need to do is compress this file: > > http://marknelson.us/attachments/million-digit-challenge/ > AMillionRandomDigits.bin > > to less than 415241 bytes, and you can win $100. > > Then, on Mon, 23 Oct 2017 21:13:00 -0700 (PDT), danceswithnumbers wrote: > > I did that quite a while ago. > > > But 352,954 kb > 415241 bytes, by several orders of magnitude; so > you didn't "do that". (Or are we using the European decimal point?) > > If you're claiming 352,954 *bytes*, not kb, I invite you to explain > why you have not collected Mark Nelson's $100 prize, and untold fame > and glory; failing which, your credibility will evaporate. > > -- > To email me, substitute nowhere->runbox, invalid->com. > > > > ---------- Forwarded message ---------- > From: Ian Kelly > To: > Cc: Python > Bcc: > Date: Tue, 24 Oct 2017 15:24:53 -0600 > Subject: Re: Compression of random binary data > On Tue, Oct 24, 2017 at 12:20 AM, Gregory Ewing > wrote: > > danceswithnumbers at gmail.com wrote: > >> > >> I did that quite a while ago. 352,954 kb. > > > > > > Are you sure? Does that include the size of all the > > code, lookup tables, etc. needed to decompress it? > > My bet is that danceswithnumbers does indeed have a file of that size > which is in some way derived from the million random digits, but > without any means of losslessly "decompressing" it (thus making it > junk data). > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 09:11:01 +1100 > Subject: Re: Compression of random binary data > On Wed, 25 Oct 2017 02:40 am, Lele Gaifax wrote: > > > Steve D'Aprano writes: > > > >> But given an empty file, how do you distinguish the empty file you get > >> from 'music.mp3' and the identical empty file you get from 'movie.avi'? > > > > That's simple enough: of course one empty file would be > > "music.mp3.zip.zip.zip", while the other would be > > "movie.avi.zip.zip.zip.zip.zip"... some sort of > > https://en.wikipedia.org/wiki/Water_memory applied to file system > entries > > :-) > > > Does that mean if I name an empty file > > serenity2-by-joss-whedon.avi.zip.zip.zip.zip.zip > > Dancerswithnumbers' magic algorithm will recreate the movie from some > alternative universe where it actually exists? > > Awesome. > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: "Steve D'Aprano" > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 09:30:57 +1100 > Subject: Re: Compression of random binary data > On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: > > > On 2017-10-23 04:21, Steve D'Aprano wrote: > >> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: > >>> > >> If the probability of certain codes (either single codes, or sequences > of > >> codes) are non-equal, then you can take advantage of that by encoding > the > >> common cases into a short representation, and the uncommon and rare > cases > >> into a longer representation. As you say: > >> > >> > >>> Otherwise, if ( 0, 0 ) is much more frequent, > >>> we can encode ( 0, 0 ) by "0" and > >>> > >>> ( 0, 1 ) by "101", > >>> ( 1, 0 ) by "110", and > >>> ( 1, 1 ) by "111". > >>> > >>> And we could then use /less/ than two bits on the > >>> average. > >> > >> That's incorrect. On average you use 2.5 bits. > >> > >> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) > > > > I disagree. If the distribution is not equal, then the average needs to > > take the different probabilities into account. > > I think I would call that the *weighted* average rather than the average. > > Regardless of what we call it, of course both you and Stefan are right in > how > to calculate it, and such a variable-length scheme can be used to compress > the data. > > E.g. given the sequence 00000011 which would take 8 bits in the obvious > encoding, we can encode it as "000111" which takes only 6 bits. > > But the cost of this encoding scheme is that *some* bit sequences expand, > e.g. > the 8 bit sequence 11111100 is encoded as "1111111110" which requires 10 > bits. > > The end result is that averaged over all possible bit sequences (of a > certain > size), this encoding scheme requires MORE space than the obvious 0/1 bits. > > But in practice we don't care much, because the data sequences we care > about > are usually not "all possible bit sequences", but a heavily restricted > subset > where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > > > > ---------- Forwarded message ---------- > From: Chris Angelico > To: "python-list at python.org" > Cc: > Bcc: > Date: Wed, 25 Oct 2017 10:09:42 +1100 > Subject: Re: Compression of random binary data > On Wed, Oct 25, 2017 at 9:11 AM, Steve D'Aprano > wrote: > > On Wed, 25 Oct 2017 02:40 am, Lele Gaifax wrote: > > > >> Steve D'Aprano writes: > >> > >>> But given an empty file, how do you distinguish the empty file you get > >>> from 'music.mp3' and the identical empty file you get from 'movie.avi'? > >> > >> That's simple enough: of course one empty file would be > >> "music.mp3.zip.zip.zip", while the other would be > >> "movie.avi.zip.zip.zip.zip.zip"... some sort of > >> https://en.wikipedia.org/wiki/Water_memory applied to file system > entries > >> :-) > > > > > > Does that mean if I name an empty file > > > > serenity2-by-joss-whedon.avi.zip.zip.zip.zip.zip > > > > Dancerswithnumbers' magic algorithm will recreate the movie from some > > alternative universe where it actually exists? > > > > Awesome. > > Yes, but then you'd get > dmca-takedown-request.pdf.zip.zip.zip.zip.zip.zip.zip which would also > be empty. > > ChrisA > > > > ---------- Forwarded message ---------- > From: Richard Damon > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 19:15:28 -0400 > Subject: Re: Compression of random binary data > On 10/24/17 6:30 PM, Steve D'Aprano wrote: > >> On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: >> >> On 2017-10-23 04:21, Steve D'Aprano wrote: >>> >>>> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >>>> >>>>> >>>>> If the probability of certain codes (either single codes, or sequences >>>> of >>>> codes) are non-equal, then you can take advantage of that by encoding >>>> the >>>> common cases into a short representation, and the uncommon and rare >>>> cases >>>> into a longer representation. As you say: >>>> >>>> >>>> Otherwise, if ( 0, 0 ) is much more frequent, >>>>> we can encode ( 0, 0 ) by "0" and >>>>> >>>>> ( 0, 1 ) by "101", >>>>> ( 1, 0 ) by "110", and >>>>> ( 1, 1 ) by "111". >>>>> >>>>> And we could then use /less/ than two bits on the >>>>> average. >>>>> >>>> >>>> That's incorrect. On average you use 2.5 bits. >>>> >>>> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) >>>> >>> >>> I disagree. If the distribution is not equal, then the average needs to >>> take the different probabilities into account. >>> >> >> I think I would call that the *weighted* average rather than the average. >> >> Regardless of what we call it, of course both you and Stefan are right in >> how >> to calculate it, and such a variable-length scheme can be used to compress >> the data. >> >> E.g. given the sequence 00000011 which would take 8 bits in the obvious >> encoding, we can encode it as "000111" which takes only 6 bits. >> >> But the cost of this encoding scheme is that *some* bit sequences expand, >> e.g. >> the 8 bit sequence 11111100 is encoded as "1111111110" which requires 10 >> bits. >> >> The end result is that averaged over all possible bit sequences (of a >> certain >> size), this encoding scheme requires MORE space than the obvious 0/1 bits. >> >> But in practice we don't care much, because the data sequences we care >> about >> are usually not "all possible bit sequences", but a heavily restricted >> subset >> where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. >> >> > My understanding of the 'Random Data Comprehensibility' challenge is that > is requires that the compression take ANY/ALL strings of up to N bits, and > generate an output stream no longer than the input stream, and sometime > less. It admits that given no-uniformly distributed data, it is possible to > compress some patterns, the common ones, and expand others, the uncommon > ones, to lower the net average length. What it says can't be done is to > have a compression method that compresses EVERY input pattern. That is > where the 'Pigeon Hole' principle comes into play which the people who > claim to be able to compress random data like to ignore or just attempt to > say doesn't apply. > > > > > ---------- Forwarded message ---------- > From: C W > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 13:58:48 -0400 > Subject: h5py.File() gives error message > Dear list, > > The following Python code gives an error message > > # Python code starts here: > import numpy as np > import h5py > train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > > # Python code ends > > The error message: > > train_dataset = h5py.File('train_catvnoncat.h5', "r") > Traceback (most recent call last): > File "", line 1, in > File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", > line 269, in __init__ > fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) > File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/files.py", > line 99, in make_fid > fid = h5f.open(name, flags, fapl=fapl) > File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper > File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper > File "h5py/h5f.pyx", line 78, in h5py.h5f.open > OSError: Unable to open file (unable to open file: name = > 'train_catvnoncat.h5', errno = 2, error message = 'No such file or > directory', flags = 0, o_flags = 0) > > My directory is correct, and the dataset folder with file is there. > > Why error message? Is it h5py.File() or is it my file? Everything seems > pretty simple, what's going on? > > Thank you! > > > > ---------- Forwarded message ---------- > From: Rob Gaddi > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 18:02:26 -0700 > Subject: Re: h5py.File() gives error message > On 10/24/2017 10:58 AM, C W wrote: > >> Dear list, >> >> The following Python code gives an error message >> >> # Python code starts here: >> import numpy as np >> import h5py >> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") >> >> # Python code ends >> >> The error message: >> >> train_dataset = h5py.File('train_catvnoncat.h5', "r") >> Traceback (most recent call last): >> File "", line 1, in >> File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/ >> files.py", >> line 269, in __init__ >> fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr) >> File "/Users/M/anaconda/lib/python3.6/site-packages/h5py/_hl/ >> files.py", >> line 99, in make_fid >> fid = h5f.open(name, flags, fapl=fapl) >> File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper >> File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper >> File "h5py/h5f.pyx", line 78, in h5py.h5f.open >> OSError: Unable to open file (unable to open file: name = >> 'train_catvnoncat.h5', errno = 2, error message = 'No such file or >> directory', flags = 0, o_flags = 0) >> >> My directory is correct, and the dataset folder with file is there. >> >> Why error message? Is it h5py.File() or is it my file? Everything seems >> pretty simple, what's going on? >> >> Thank you! >> >> > Be 100% sure your directory is correct. Try it again with an absolute > path to the file. Windows makes it far too easy for the working directory > of a program to be other than what you think it is. > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. > > > > ---------- Forwarded message ---------- > From: Dennis Lee Bieber > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 22:35:21 -0400 > Subject: Re: h5py.File() gives error message > On Tue, 24 Oct 2017 13:58:48 -0400, C W declaimed the > following: > > > >train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > > > ># Python code ends > > > >The error message: > > > >train_dataset = h5py.File('train_catvnoncat.h5', "r") > > This is NOT the same statement you show above. > > So either you are typing things from memory, rather than including > (cut&paste) the actual lines, or you have multiple files lying around and > are not executing the version/file you think you are. > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > > > > ---------- Forwarded message ---------- > From: Dennis Lee Bieber > To: python-list at python.org > Cc: > Bcc: > Date: Tue, 24 Oct 2017 22:37:07 -0400 > Subject: Re: h5py.File() gives error message > On Tue, 24 Oct 2017 18:02:26 -0700, Rob Gaddi > declaimed the following: > > Whoops, I may have gotten the wrong level of quoting -- my > apologies if > so (I did have agent fetch the original posting, but might not have had > that active when I hit "reply") > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > > > > ---------- Forwarded message ---------- > From: ast > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 08:49:00 +0200 > Subject: Objects with __name__ attribute > Hi, > > I know two Python's objects which have an intrinsic name, classes and > functions. > > def f(): > pass > > f.__name__ >>>> >>> 'f' > >> g = f >>>> g.__name__ >>>> >>> 'f' > > class Test: > pass > > Test.__name__ >>>> >>> 'Test' > >> Test2 = Test >>>> Test2.__name__ >>>> >>> 'Test' > > Are there others objects with a __name__ attribute > and what is it used for ? > > Regards > > > > > > > ---------- Forwarded message ---------- > From: Peter Otten <__peter__ at web.de> > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 09:21:22 +0200 > Subject: Re: Objects with __name__ attribute > ast wrote: > > > Hi, > > > > I know two Python's objects which have an intrinsic > > name, classes and functions. > > > > def f(): > > pass > > > >>>> f.__name__ > > 'f' > >>>> g = f > >>>> g.__name__ > > 'f' > > > > class Test: > > pass > > > >>>> Test.__name__ > > 'Test' > >>>> Test2 = Test > >>>> Test2.__name__ > > 'Test' > > > > Are there others objects with a __name__ attribute > > and what is it used for ? > > > > Regards > > It was used for the object's repr(): > > $ python > Python 2.7.6 (default, Oct 26 2016, 20:30:19) > [GCC 4.8.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> def f(): pass > ... > >>> f.__name__ = "spanish inquisition" > >>> f > > > But this has changed: > > $ python3 > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> def f(): pass > ... > >>> f.__name__ = "spanish inquisition" > >>> f > > >>> f.__qualname__ = "spanish inquisition" > >>> f > > > You should be aware that the module name is used to detect the main module: > > if __name__ == "__main__": > print("this is run as a script") > > > > > ---------- Forwarded message ---------- > From: C W > To: Dennis Lee Bieber > Cc: python-list at python.org > Bcc: > Date: Wed, 25 Oct 2017 03:21:34 -0400 > Subject: Re: h5py.File() gives error message > Oh, I was running a debug file, that's why the path is different. > > The file is here, > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 > > Is anyone able to get it working? Thank you! > > On Tue, Oct 24, 2017 at 10:37 PM, Dennis Lee Bieber > > wrote: > > > On Tue, 24 Oct 2017 18:02:26 -0700, Rob Gaddi > > declaimed the following: > > > > Whoops, I may have gotten the wrong level of quoting -- my > > apologies if > > so (I did have agent fetch the original posting, but might not have had > > that active when I hit "reply") > > -- > > Wulfraed Dennis Lee Bieber AF6VN > > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > ---------- Forwarded message ---------- > From: Gregory Ewing > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 20:32:11 +1300 > Subject: Re: Compression of random binary data > Steve D'Aprano wrote: > >> - Encrypted data looks very much like random noise. >> > > There's actually a practical use for that idea. If you can feed > the output of an encryption algorithm through a compressor and > make it smaller, it means there is a cryptographic weakness > in the algorithm that could potentially be exploited. Good > encryption algorithms produce output that looks almost completely > random to anyone who doesn't know how to decrypt it. > > -- > Greg > > > > ---------- Forwarded message ---------- > From: Gregory Ewing > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 21:11:25 +1300 > Subject: Re: Compression of random binary data > Ben Bacarisse wrote: > >> The trouble is a pedagogic one. Saying "you can't compress random data" >> inevitably leads (though, again, this is just my experience) to endless >> attempts to define random data. >> > > It's more about using terms without making sure everyone agrees > on the definitions being used. > > In this context, "random data" really means "uniformly distributed > data", i.e. any bit sequence is equally likely to be presented as > input. *That's* what information theory says can't be compressed. > > I think "arbitrary data" (thereby including the results of compression >> by said algorithm) is the best way to make progress. >> > > I'm not sure that's much better, because it doesn't home in > on the most important thing, which is the probability > distribution. > > -- > Greg > > > > ---------- Forwarded message ---------- > From: Gregory Ewing > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 21:25:07 +1300 > Subject: Re: Compression of random binary data > Lele Gaifax wrote: > >> That's simple enough: of course one empty file would be >> "music.mp3.zip.zip.zip", while the other would be >> "movie.avi.zip.zip.zip.zip.zip"... some sort of >> https://en.wikipedia.org/wiki/Water_memory applied to file system >> entries :-) >> > > If you're allowed to alternate between two compression > methods, then the way you decompress > music.mp3.zip.zip.tgz.zip...........tgz.zip.tgz > is to output 0 each time zip was applied and 1 each > time tar/gz was applied. > > You may be able to take some shortcuts in some > cases, e.g. anything beginning with "movie.flv" > almost certainly contains a cute kitten video. > (Unless it's found inside an encrypted disk > partition, in which case it contains porn.) > > -- > Greg > > > > ---------- Forwarded message ---------- > From: Cameron Simpson > To: T Obulesu > Cc: python-list at python.org > Bcc: > Date: Wed, 25 Oct 2017 16:10:02 +1100 > Subject: Re: Sockets but calling from different programs > On 23Oct2017 05:33, T Obulesu wrote: > >> I'm new to python3 and scratching my head to write a program for this >> logic: >> > > The tutor list might be a better place for such questions, but since we're > here... > > classA.py >> Class A: >> # class for socket communication >> basic init method that initializes port, address, connection >> method send(message): >> # for sending any message through the given port >> method receive(): >> # will be keep on listening for incoming messages >> >> classB.py >> Class B: >> import the send method from class A >> send the messages from this class >> >> classC.py >> Class C: >> import the receive method from the class A >> receive all the messages from the same socket from here. >> >> Note: >> classA.py, classB.py, ClassC.py are saved in different locations. >> >> Can someone help me in writing the code and how can I create a single >> object and use it in multiple classed? >> > > That is a pretty normal arrangement. Class A might look like this: > > class A: > def __init__(self, port, address): > self.connection = ... make your connection to (address, port) > def send(self, msg): > send msg using self.connection ... > > Since classes B and C seem expected to share tha same connection, the > natural thing is to set up the connection _separately_ from setting up B > and C, and pass the established connection to each. > > So class B might commence: > > class B: > def __init__(self, conn, ...): > self.conn = conn > ... whatever other initialisation ... > def speak(self, something): > self.conn.send(something) > > You'll notice here that we're _not_ importing anything about class A > here. Class B does not need to know class A's name to use it. > > Because Python uses duck typing, you could pass _any_ object which has a > .send() method as "conn" to the class B setup. This allows you to write > some other class A2 with those same methods, but using some different type > of connection, and pass that in to classes B and C. > > So a main programme might set things up like this: > > from classA import A > from classB import B > from classC import C > > conn = A(address, port) > sender = B(conn, other-stuff...) > receiver = C(conn, other-stuff...) > B.speak("foo") > > and so forth. > > Cheers, > Cameron Simpson (formerly cs at zip.com.au) > > > > ---------- Forwarded message ---------- > From: Peter Otten <__peter__ at web.de> > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 11:20:56 +0200 > Subject: Re: h5py.File() gives error message > C W wrote: > > > Oh, I was running a debug file, that's why the path is different. > > > > The file is here, > > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 > > > > Is anyone able to get it working? Thank you! > > Hm, that file seems to contain HTML and that causes an OSError here, too: > > $ head -n3 datasets/train_catvnoncat.h5 > > > > > $ python3 > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import h5py > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > Traceback (most recent call last): > File "", line 1, in > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in > __init__ > fid = make_fid(name, mode, userblock_size, fapl) > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in > make_fid > fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl) > File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806) > OSError: unable to open file (File accessibilty: Unable to open file) > > It's not exactly what you see, but that may be due to differing software > versions. > When I replace the HTML file with its namesake found at > > https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/ > code/datasets/train_catvnoncat.h5 > > I can open it: > > $ file datasets/train_catvnoncat.h5 > datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data > > $ python3 > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > [GCC 4.8.4] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import h5py > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > >>> > > > > > ---------- Forwarded message ---------- > From: Karsten Hilbert > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 11:51:07 +0200 > Subject: Re: right list for SIGABRT python binary question ? > On Tue, Oct 24, 2017 at 08:47:58PM +0200, M.-A. Lemburg wrote: > > > >> This error suggests that you have 32- and 64-bit versions of > > >> Python and mxDateTime mixed in your installation. > > >> > > >> Py_InitModule4 is only available in the 32-bit build of > > >> Python. With the 64-bit build, it's called Py_InitModule4_64. > ... > > Could you check whether you have similar import errors with > > other modules that have C extensions ? E.g. lxml. > > > > What you're seeing appears to be a compilation problem > > with Python 2.7.14 on Debian. The executable doesn't appear > > to export its symbols to the .so files, or only some of them. > > Let's see: > > python-lxml: > Installiert: 4.0.0-1 > Installationskandidat: 4.0.0-1 > Versionstabelle: > *** 4.0.0-1 990 > 990 http://httpredir.debian.org/debian buster/main i386 > Packages > 500 http://httpredir.debian.org/debian unstable/main i386 > Packages > 100 /var/lib/dpkg/status > 3.7.1-1 500 > 500 http://httpredir.debian.org/debian stretch/main i386 > Packages > python-lxml-dbg: > Installiert: (keine) > Installationskandidat: 4.0.0-1 > Versionstabelle: > 4.0.0-1 990 > 990 http://httpredir.debian.org/debian buster/main i386 > Packages > 500 http://httpredir.debian.org/debian unstable/main i386 > Packages > 3.7.1-1 500 > 500 http://httpredir.debian.org/debian stretch/main i386 > Packages > > > ncq at hermes:~$ python2.7-dbg > Python 2.7.14 (default, Sep 17 2017, 18:50:44) > [GCC 7.2.0] on linux2 > Type "help", "copyright", "credits" or "license" for more > information. > >>> import lxml > [45350 refs] > >>> > > ncq at hermes:~$ python > Python 2.7.14 (default, Sep 17 2017, 18:50:44) > [GCC 7.2.0] on linux2 > Type "help", "copyright", "credits" or "license" for more > information. > >>> import lxml > >>> > > Also, psycogp2 imports just fine. > > Now that I have > > python-egenix-mx-base-dbg: > Installiert: 3.2.9-1 > Installationskandidat: 3.2.9-1 > Versionstabelle: > *** 3.2.9-1 990 > 500 http://httpredir.debian.org/debian stretch/main i386 > Packages > 990 http://httpredir.debian.org/debian buster/main i386 > Packages > 500 http://httpredir.debian.org/debian unstable/main i386 > Packages > 100 /var/lib/dpkg/status > > mx.DateTime imports fine as well (but the SIGABRT persists). > > Karsten > -- > GPG key ID E4071346 @ eu.pool.sks-keyservers.net > E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 > > > > ---------- Forwarded message ---------- > From: Thomas Jollans > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 15:07:24 +0200 > Subject: Let's talk about debuggers! > Hi, > > I just wanted to know what tools everyone used for debugging Python > applications - scripts / backend / desktop apps / notebooks / whatever. > Apart from the usual dance with log files and strategically inserted > print() calls, that is. > > Of course we all know and mildly dislike pdb. > > Personally, in practice, I'm most likely to need a debugger when > prototyping a function in a Jupyter notebook. There, ipdb, summoned with > the %%debug magic incantation, does the trick. > > Sometimes, though, I miss having a visual debugger. You know, the kind > that Visual Basic has had for decades. There's one in Chrome dev tools > if you ever have the misfortune of writing JavaScript. > > What options are there for Python (that work)? What text editors (and > IDEs) have a decent integrated debugger or debugging plugin? (Is there > anything for Sublime?) Does anyone use them? (How do YOU debug?) > > I vaguely remember WinPDB, but that hasn't seen a release in more than > seven years... > > > -- > Thomas Jollans > > > > ---------- Forwarded message ---------- > From: Ian Kelly > To: > Cc: python-list at python.org > Bcc: > Date: Wed, 25 Oct 2017 07:27:59 -0600 > Subject: Re: Compression of random binary data > On 10/24/17, Richard Damon wrote: > > My understanding of the 'Random Data Comprehensibility' challenge is > > that is requires that the compression take ANY/ALL strings of up to N > > bits, and generate an output stream no longer than the input stream, and > > sometime less. > > That's incorrect, at least of the challenge being discussed. Here's the > link: > > http://marknelson.us/2012/10/09/the-random-compression- > challenge-turns-ten/ > > The challenge is just to take a single known file of a million random > digits and make it smaller, including the size of the decompresser and > without hiding data. So far in 15 years nobody has succeeded even at > this, and nobody knows whether it's impossible. For instance it may be > the case that the data in the file happens to be the nth prime, in > which case it could simply be compressed to the number n with a > decompresser that calculates process. > > > It admits that given no-uniformly distributed data, it is > > possible to compress some patterns, the common ones, and expand others, > > the uncommon ones, to lower the net average length. What it says can't > > be done is to have a compression method that compresses EVERY input > > pattern. That is where the 'Pigeon Hole' principle comes into play which > > the people who claim to be able to compress random data like to ignore > > or just attempt to say doesn't apply. > > There is a second challenge on that page that is as you state, but the > page admits that this second challenge is a bit of a troll since this > is provably impossible. > > > > ---------- Forwarded message ---------- > From: Fabien > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 15:31:21 +0200 > Subject: Re: Let's talk about debuggers! > On 10/25/2017 03:07 PM, Thomas Jollans wrote: > >> What options are there for Python (that work)? >> > > PyCharm's debugger is fine (also available in the community edition) > > Cheers, > > Fabien > > > > ---------- Forwarded message ---------- > From: C W > To: Peter Otten <__peter__ at web.de> > Cc: python-list at python.org > Bcc: > Date: Wed, 25 Oct 2017 09:58:20 -0400 > Subject: Re: h5py.File() gives error message > wow, thanks so much! I don't know how you figured that it's HTML, but > that's awesome! > > Mike > > On Wed, Oct 25, 2017 at 5:20 AM, Peter Otten <__peter__ at web.de> wrote: > > > C W wrote: > > > > > Oh, I was running a debug file, that's why the path is different. > > > > > > The file is here, > > > https://www.dropbox.com/s/6jx4rzyg9xwl95m/train_catvnoncat.h5?dl=0 > > > > > > Is anyone able to get it working? Thank you! > > > > Hm, that file seems to contain HTML and that causes an OSError here, too: > > > > $ head -n3 datasets/train_catvnoncat.h5 > > > > > > > > > > $ python3 > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > > [GCC 4.8.4] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > >>> import h5py > > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > > Traceback (most recent call last): > > File "", line 1, in > > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 207, in > > __init__ > > fid = make_fid(name, mode, userblock_size, fapl) > > File "/usr/lib/python3/dist-packages/h5py/_hl/files.py", line 79, in > > make_fid > > fid = h5f.open(name, h5f.ACC_RDONLY, fapl=fapl) > > File "h5f.pyx", line 71, in h5py.h5f.open (h5py/h5f.c:1806) > > OSError: unable to open file (File accessibilty: Unable to open file) > > > > It's not exactly what you see, but that may be due to differing software > > versions. > > When I replace the HTML file with its namesake found at > > > > https://github.com/lalxyy/NEU-MCM-Training-4/blob/master/cod > > e/datasets/train_catvnoncat.h5 > > > > I can open it: > > > > $ file datasets/train_catvnoncat.h5 > > datasets/train_catvnoncat.h5: Hierarchical Data Format (version 5) data > > > > $ python3 > > Python 3.4.3 (default, Nov 17 2016, 01:08:31) > > [GCC 4.8.4] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > >>> import h5py > > >>> train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") > > >>> > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > > ---------- Forwarded message ---------- > From: Rustom Mody > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 06:57:04 -0700 (PDT) > Subject: Re: Let's talk about debuggers! > On Wednesday, October 25, 2017 at 6:37:47 PM UTC+5:30, Thomas Jollans > wrote: > > Hi, > > > > I just wanted to know what tools everyone used for debugging Python > > applications - scripts / backend / desktop apps / notebooks / whatever. > > Apart from the usual dance with log files and strategically inserted > > print() calls, that is. > > > > Of course we all know and mildly dislike pdb. > > > > Personally, in practice, I'm most likely to need a debugger when > > prototyping a function in a Jupyter notebook. There, ipdb, summoned with > > the %%debug magic incantation, does the trick. > > > > Sometimes, though, I miss having a visual debugger. You know, the kind > > that Visual Basic has had for decades. There's one in Chrome dev tools > > if you ever have the misfortune of writing JavaScript. > > > > What options are there for Python (that work)? What text editors (and > > IDEs) have a decent integrated debugger or debugging plugin? (Is there > > anything for Sublime?) Does anyone use them? (How do YOU debug?) > > > > I vaguely remember WinPDB, but that hasn't seen a release in more than > > seven years... > > pdb inside emacs works (to a fashion) > And it shows the arrow for current line so its at least quasi-gui > > I believe idle too is much more usable than a few years earlier > > > > ---------- Forwarded message ---------- > From: Ned Batchelder > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 09:53:10 -0400 > Subject: Re: Let's talk about debuggers! > On 10/25/17 9:07 AM, Thomas Jollans wrote: > >> Hi, >> >> I just wanted to know what tools everyone used for debugging Python >> applications - scripts / backend / desktop apps / notebooks / whatever. >> Apart from the usual dance with log files and strategically inserted >> print() calls, that is. >> >> Of course we all know and mildly dislike pdb. >> >> Personally, in practice, I'm most likely to need a debugger when >> prototyping a function in a Jupyter notebook. There, ipdb, summoned with >> the %%debug magic incantation, does the trick. >> >> Sometimes, though, I miss having a visual debugger. You know, the kind >> that Visual Basic has had for decades. There's one in Chrome dev tools >> if you ever have the misfortune of writing JavaScript. >> >> What options are there for Python (that work)? What text editors (and >> IDEs) have a decent integrated debugger or debugging plugin? (Is there >> anything for Sublime?) Does anyone use them? (How do YOU debug?) >> >> I vaguely remember WinPDB, but that hasn't seen a release in more than >> seven years... >> >> >> > pudb is a visual terminal debugger: https://pypi.python.org/pypi/pudb > > It uses the same commands as pdb, so it's easy to get started, but it > gives you a variables pane, with customizable presentation, and so on. > > One of my favorite features: you can add a set_trace line in your program, > and then if during the debugging session you realize you don't want to stop > there every time, you can disable that breakpoint even though it's an > explicit line of code telling the debugger to stop. > > --Ned. > > > > ---------- Forwarded message ---------- > From: Michele Simionato > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 07:08:31 -0700 (PDT) > Subject: Re: Let's talk about debuggers! > pdb plus plus: https://pypi.python.org/pypi/pdbpp > > > > ---------- Forwarded message ---------- > From: Bill > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 10:46:47 -0400 > Subject: Re: Let's talk about debuggers! > Fabien wrote: > >> On 10/25/2017 03:07 PM, Thomas Jollans wrote: >> >>> What options are there for Python (that work)? >>> >> >> PyCharm's debugger is fine (also available in the community edition) >> > +1 > > >> Cheers, >> >> Fabien >> > > > > > ---------- Forwarded message ---------- > From: danceswithnumbers at gmail.com > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 07:49:34 -0700 (PDT) > Subject: Re: Compression of random binary data > Whatever you do, you'll find that *on average* you > will need *at least* 34 bits to be able to represent > all possible 10-digit decimal numbers. Some might > be shorter, but then others will be longer, and > the average won't be less than 34. > > > The theoretical limit for arbitrary numbers 0 - 9 must be viewed as an > average not as an impossibility to, in some cases be able to compress to or > under 34. This is obvious by the decimal to binary function. > > > > ---------- Forwarded message ---------- > From: William Ray Wing > To: Thomas Jollans > Cc: "William R. Wing" , python-list at python.org > Bcc: > Date: Wed, 25 Oct 2017 10:30:28 -0400 > Subject: Re: Let's talk about debuggers! > > > On Oct 25, 2017, at 9:07 AM, Thomas Jollans wrote: > > > > > > [byte] > > > What options are there for Python (that work)? What text editors (and > > IDEs) have a decent integrated debugger or debugging plugin? > > I rather like WingIDE (the name is a coincidence). It allows > insertion/removal of break points while the code is running. While > execution is stopped, it allows direct inspection of the stack (no > surprise), but in addition allows execution of python statements or program > elements typed into an auxiliary window - including importing things like > matplotlib and plotting the current state of data arrays. Its editor is > syntax-aware and highlights accidental syntax errors as they are typed. > Lots of other features, those just happen to be the ones I use most often. > > Bill > > > ---------- Forwarded message ---------- > From: kkgsrkmk at gmail.com > To: python-list at python.org > Cc: > Bcc: > Date: Wed, 25 Oct 2017 08:46:59 -0700 (PDT) > Subject: Test Bank for Campbell Biology 11th Edition by Urry, Cain > Where it didnt come > > > -- > https://mail.python.org/mailman/listinfo/python-list > > From jtim.arnold at gmail.com Wed Oct 25 12:25:01 2017 From: jtim.arnold at gmail.com (Tim) Date: Wed, 25 Oct 2017 09:25:01 -0700 (PDT) Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: On Wednesday, October 25, 2017 at 9:07:47 AM UTC-4, Thomas Jollans wrote: > Hi, > > I just wanted to know what tools everyone used for debugging Python > applications - scripts / backend / desktop apps / notebooks / whatever. > Apart from the usual dance with log files and strategically inserted > print() calls, that is. > > Thomas Jollans One more vote for pudb. Somehow it fits my brain better than most. From pfdubois at gmail.com Wed Oct 25 12:33:16 2017 From: pfdubois at gmail.com (Paul Dubois) Date: Wed, 25 Oct 2017 16:33:16 +0000 Subject: SciPy 1.0 released! In-Reply-To: References: Message-ID: The DNA of this work goes much further back. After some discussion as mentioned, Jim Hugunin wrote a numerical extension (called Numerical) and after a bit I took over as its coordinator with financial support for my work from Lawrence Livermore National Laboratory. They also supported the Foundation. I do not remember the exact timing but certainly Numerical was running before the 4th International was held at the Lab in the early 90s. Guido put in the complex type and other things to help us. We had endless discussion amongst us as how to spell it: 1i or 1j! One should note that the Lab was breaking new ground in supporting an open source project. SciPy was born to try to improve and extend a product that was already crucial to a lot of people. Indeed, it is a big part of several ongoing projects. Even non-numerical applications are important there. I retired in 2005, and everyone who writes software knows that what they do is ephemeral, but I don?t want history to be lost so soon. Paul Dubois ... moving as we speak to La Mesa, CA On Wed, Oct 25, 2017 at 5:05 AM Ralf Gommers wrote: > Hi all, > > We are extremely pleased to announce the release of SciPy 1.0, 16 years > after > version 0.1 saw the light of day. It has been a long, productive journey > to > get here, and we anticipate many more exciting new features and releases in > the > future. > > > Why 1.0 now? > ------------ > > A version number should reflect the maturity of a project - and SciPy was a > mature and stable library that is heavily used in production settings for a > long time already. From that perspective, the 1.0 version number is long > overdue. > > Some key project goals, both technical (e.g. Windows wheels and continuous > integration) and organisational (a governance structure, code of conduct > and a > roadmap), have been achieved recently. > > Many of us are a bit perfectionist, and therefore are reluctant to call > something "1.0" because it may imply that it's "finished" or "we are 100% > happy > with it". This is normal for many open source projects, however that > doesn't > make it right. We acknowledge to ourselves that it's not perfect, and > there > are some dusty corners left (that will probably always be the case). > Despite > that, SciPy is extremely useful to its users, on average has high quality > code > and documentation, and gives the stability and backwards compatibility > guarantees that a 1.0 label imply. > > > Some history and perspectives > ----------------------------- > > - 2001: the first SciPy release > - 2005: transition to NumPy > - 2007: creation of scikits > - 2008: scipy.spatial module and first Cython code added > - 2010: moving to a 6-monthly release cycle > - 2011: SciPy development moves to GitHub > - 2011: Python 3 support > - 2012: adding a sparse graph module and unified optimization interface > - 2012: removal of scipy.maxentropy > - 2013: continuous integration with TravisCI > - 2015: adding Cython interface for BLAS/LAPACK and a benchmark suite > - 2017: adding a unified C API with scipy.LowLevelCallable; removal of > scipy.weave > - 2017: SciPy 1.0 release > > > **Pauli Virtanen** is SciPy's Benevolent Dictator For Life (BDFL). He > says: > > *Truthfully speaking, we could have released a SciPy 1.0 a long time ago, > so I'm > happy we do it now at long last. The project has a long history, and during > the > years it has matured also as a software project. I believe it has well > proved > its merit to warrant a version number starting with unity.* > > *Since its conception 15+ years ago, SciPy has largely been written by and > for > scientists, to provide a box of basic tools that they need. Over time, the > set > of people active in its development has undergone some rotation, and we > have > evolved towards a somewhat more systematic approach to development. > Regardless, > this underlying drive has stayed the same, and I think it will also > continue > propelling the project forward in future. This is all good, since not long > after 1.0 comes 1.1.* > > **Travis Oliphant** is one of SciPy's creators. He says: > > *I'm honored to write a note of congratulations to the SciPy developers and > the > entire SciPy community for the release of SciPy 1.0. This release > represents > a dream of many that has been patiently pursued by a stalwart group of > pioneers > for nearly 2 decades. Efforts have been broad and consistent over that > time > from many hundreds of people. From initial discussions to efforts coding > and > packaging to documentation efforts to extensive conference and community > building, the SciPy effort has been a global phenomenon that it has been a > privilege to participate in.* > > *The idea of SciPy was already in multiple people?s minds in 1997 when I > first > joined the Python community as a young graduate student who had just fallen > in > love with the expressibility and extensibility of Python. The internet > was > just starting to bringing together like-minded mathematicians and > scientists in > nascent electronically-connected communities. In 1998, there was a > concerted > discussion on the matrix-SIG, python mailing list with people like Paul > Barrett, Joe Harrington, Perry Greenfield, Paul Dubois, Konrad Hinsen, > David > Ascher, and others. This discussion encouraged me in 1998 and 1999 to > procrastinate my PhD and spend a lot of time writing extension modules to > Python that mostly wrapped battle-tested Fortran and C-code making it > available > to the Python user. This work attracted the help of others like Robert > Kern, > Pearu Peterson and Eric Jones who joined their efforts with mine in 2000 so > that by 2001, the first SciPy release was ready. This was long before > Github > simplified collaboration and input from others and the "patch" command and > email was how you helped a project improve.* > > *Since that time, hundreds of people have spent an enormous amount of time > improving the SciPy library and the community surrounding this library has > dramatically grown. I stopped being able to participate actively in > developing > the SciPy library around 2010. Fortunately, at that time, Pauli Virtanen > and > Ralf Gommers picked up the pace of development supported by dozens of other > key > contributors such as David Cournapeau, Evgeni Burovski, Josef Perktold, and > Warren Weckesser. While I have only been able to admire the development > of > SciPy from a distance for the past 7 years, I have never lost my love of > the > project and the concept of community-driven development. I remain driven > even now by a desire to help sustain the development of not only the SciPy > library but many other affiliated and related open-source projects. I am > extremely pleased that SciPy is in the hands of a world-wide community of > talented developers who will ensure that SciPy remains an example of how > grass-roots, community-driven development can succeed.* > > **Fernando Perez** offers a wider community perspective: > > *The existence of a nascent Scipy library, and the incredible --if tiny by > today's standards-- community surrounding it is what drew me into the > scientific Python world while still a physics graduate student in 2001. > Today, > I am awed when I see these tools power everything from high school > education to > the research that led to the 2017 Nobel Prize in physics.* > > *Don't be fooled by the 1.0 number: this project is a mature cornerstone of > the > modern scientific computing ecosystem. I am grateful for the many who have > made it possible, and hope to be able to contribute again to it in the > future. > My sincere congratulations to the whole team!* > > > Highlights of this release > -------------------------- > > Some of the highlights of this release are: > > - Major build improvements. Windows wheels are available on PyPI for the > first time, and continuous integration has been set up on Windows and OS > X > in addition to Linux. > - A set of new ODE solvers and a unified interface to them > (`scipy.integrate.solve_ivp`). > - Two new trust region optimizers and a new linear programming method, with > improved performance compared to what `scipy.optimize` offered > previously. > - Many new BLAS and LAPACK functions were wrapped. The BLAS wrappers are > now > complete. > > > Upgrading and compatibility > --------------------------- > > There have been a number of deprecations and API changes in this release, > which > are documented below. Before upgrading, we recommend that users check that > their own code does not use deprecated SciPy functionality (to do so, run > your > code with ``python -Wd`` and check for ``DeprecationWarning`` s). > > This release requires Python 2.7 or >=3.4 and NumPy 1.8.2 or greater. > > This is also the last release to support LAPACK 3.1.x - 3.3.x. Moving the > lowest supported LAPACK version to >3.2.x was long blocked by Apple > Accelerate > providing the LAPACK 3.2.1 API. We have decided that it's time to either > drop > Accelerate or, if there is enough interest, provide shims for functions > added > in more recent LAPACK versions so it can still be used. > > > New features > ============ > > `scipy.cluster` improvements > ---------------------------- > > `scipy.cluster.hierarchy.optimal_leaf_ordering`, a function to reorder a > linkage matrix to minimize distances between adjacent leaves, was added. > > > `scipy.fftpack` improvements > ---------------------------- > > N-dimensional versions of the discrete sine and cosine transforms and their > inverses were added as ``dctn``, ``idctn``, ``dstn`` and ``idstn``. > > > `scipy.integrate` improvements > ------------------------------ > > A set of new ODE solvers have been added to `scipy.integrate`. The > convenience > function `scipy.integrate.solve_ivp` allows uniform access to all solvers. > The individual solvers (``RK23``, ``RK45``, ``Radau``, ``BDF`` and > ``LSODA``) > can also be used directly. > > > `scipy.linalg` improvements > ---------------------------- > > The BLAS wrappers in `scipy.linalg.blas` have been completed. Added > functions > are ``*gbmv``, ``*hbmv``, ``*hpmv``, ``*hpr``, ``*hpr2``, ``*spmv``, > ``*spr``, > ``*tbmv``, ``*tbsv``, ``*tpmv``, ``*tpsv``, ``*trsm``, ``*trsv``, > ``*sbmv``, > ``*spr2``, > > Wrappers for the LAPACK functions ``*gels``, ``*stev``, ``*sytrd``, > ``*hetrd``, > ``*sytf2``, ``*hetrf``, ``*sytrf``, ``*sycon``, ``*hecon``, ``*gglse``, > ``*stebz``, ``*stemr``, ``*sterf``, and ``*stein`` have been added. > > The function `scipy.linalg.subspace_angles` has been added to compute the > subspace angles between two matrices. > > The function `scipy.linalg.clarkson_woodruff_transform` has been added. > It finds low-rank matrix approximation via the Clarkson-Woodruff Transform. > > The functions `scipy.linalg.eigh_tridiagonal` and > `scipy.linalg.eigvalsh_tridiagonal`, which find the eigenvalues and > eigenvectors of tridiagonal hermitian/symmetric matrices, were added. > > > `scipy.ndimage` improvements > ---------------------------- > > Support for homogeneous coordinate transforms has been added to > `scipy.ndimage.affine_transform`. > > The ``ndimage`` C code underwent a significant refactoring, and is now > a lot easier to understand and maintain. > > > `scipy.optimize` improvements > ----------------------------- > > The methods ``trust-region-exact`` and ``trust-krylov`` have been added to > the > function `scipy.optimize.minimize`. These new trust-region methods solve > the > subproblem with higher accuracy at the cost of more Hessian factorizations > (compared to dogleg) or more matrix vector products (compared to ncg) but > usually require less nonlinear iterations and are able to deal with > indefinite > Hessians. They seem very competitive against the other Newton methods > implemented in scipy. > > `scipy.optimize.linprog` gained an interior point method. Its performance > is > superior (both in accuracy and speed) to the older simplex method. > > > `scipy.signal` improvements > --------------------------- > > An argument ``fs`` (sampling frequency) was added to the following > functions: > ``firwin``, ``firwin2``, ``firls``, and ``remez``. This makes these > functions > consistent with many other functions in `scipy.signal` in which the > sampling > frequency can be specified. > > `scipy.signal.freqz` has been sped up significantly for FIR filters. > > > `scipy.sparse` improvements > --------------------------- > > Iterating over and slicing of CSC and CSR matrices is now faster by up to > ~35%. > > The ``tocsr`` method of COO matrices is now several times faster. > > The ``diagonal`` method of sparse matrices now takes a parameter, > indicating > which diagonal to return. > > > `scipy.sparse.linalg` improvements > ---------------------------------- > > A new iterative solver for large-scale nonsymmetric sparse linear systems, > `scipy.sparse.linalg.gcrotmk`, was added. It implements ``GCROT(m,k)``, a > flexible variant of ``GCROT``. > > `scipy.sparse.linalg.lsmr` now accepts an initial guess, yielding > potentially > faster convergence. > > SuperLU was updated to version 5.2.1. > > > `scipy.spatial` improvements > ---------------------------- > > Many distance metrics in `scipy.spatial.distance` gained support for > weights. > > The signatures of `scipy.spatial.distance.pdist` and > `scipy.spatial.distance.cdist` were changed to ``*args, **kwargs`` in order > to > support a wider range of metrics (e.g. string-based metrics that need extra > keywords). Also, an optional ``out`` parameter was added to ``pdist`` and > ``cdist`` allowing the user to specify where the resulting distance matrix > is > to be stored > > > `scipy.stats` improvements > -------------------------- > > The methods ``cdf`` and ``logcdf`` were added to > `scipy.stats.multivariate_normal`, providing the cumulative distribution > function of the multivariate normal distribution. > > New statistical distance functions were added, namely > `scipy.stats.wasserstein_distance` for the first Wasserstein distance and > `scipy.stats.energy_distance` for the energy distance. > > > Deprecated features > =================== > > The following functions in `scipy.misc` are deprecated: ``bytescale``, > ``fromimage``, ``imfilter``, ``imread``, ``imresize``, ``imrotate``, > ``imsave``, ``imshow`` and ``toimage``. Most of those functions have > unexpected > behavior (like rescaling and type casting image data without the user > asking > for that). Other functions simply have better alternatives. > > ``scipy.interpolate.interpolate_wrapper`` and all functions in that > submodule > are deprecated. This was a never finished set of wrapper functions which > is > not relevant anymore. > > The ``fillvalue`` of `scipy.signal.convolve2d` will be cast directly to the > dtypes of the input arrays in the future and checked that it is a scalar or > an array with a single element. > > ``scipy.spatial.distance.matching`` is deprecated. It is an alias of > `scipy.spatial.distance.hamming`, which should be used instead. > > Implementation of `scipy.spatial.distance.wminkowski` was based on a wrong > interpretation of the metric definition. In scipy 1.0 it has been just > deprecated in the documentation to keep retro-compatibility but is > recommended > to use the new version of `scipy.spatial.distance.minkowski` that > implements > the correct behaviour. > > Positional arguments of `scipy.spatial.distance.pdist` and > `scipy.spatial.distance.cdist` should be replaced with their keyword > version. > > > Backwards incompatible changes > ============================== > > The following deprecated functions have been removed from `scipy.stats`: > ``betai``, ``chisqprob``, ``f_value``, ``histogram``, ``histogram2``, > ``pdf_fromgamma``, ``signaltonoise``, ``square_of_sums``, ``ss`` and > ``threshold``. > > The following deprecated functions have been removed from > `scipy.stats.mstats`: > ``betai``, ``f_value_wilks_lambda``, ``signaltonoise`` and ``threshold``. > > The deprecated ``a`` and ``reta`` keywords have been removed from > `scipy.stats.shapiro`. > > The deprecated functions ``sparse.csgraph.cs_graph_components`` and > ``sparse.linalg.symeig`` have been removed from `scipy.sparse`. > > The following deprecated keywords have been removed in > `scipy.sparse.linalg`: > ``drop_tol`` from ``splu``, and ``xtype`` from ``bicg``, ``bicgstab``, > ``cg``, > ``cgs``, ``gmres``, ``qmr`` and ``minres``. > > The deprecated functions ``expm2`` and ``expm3`` have been removed from > `scipy.linalg`. The deprecated keyword ``q`` was removed from > `scipy.linalg.expm`. And the deprecated submodule ``linalg.calc_lwork`` > was > removed. > > The deprecated functions ``C2K``, ``K2C``, ``F2C``, ``C2F``, ``F2K`` and > ``K2F`` have been removed from `scipy.constants`. > > The deprecated ``ppform`` class was removed from `scipy.interpolate`. > > The deprecated keyword ``iprint`` was removed from > `scipy.optimize.fmin_cobyla`. > > The default value for the ``zero_phase`` keyword of `scipy.signal.decimate` > has been changed to True. > > The ``kmeans`` and ``kmeans2`` functions in `scipy.cluster.vq` changed the > method used for random initialization, so using a fixed random seed will > not necessarily produce the same results as in previous versions. > > `scipy.special.gammaln` does not accept complex arguments anymore. > > The deprecated functions ``sph_jn``, ``sph_yn``, ``sph_jnyn``, ``sph_in``, > ``sph_kn``, and ``sph_inkn`` have been removed. Users should instead use > the functions ``spherical_jn``, ``spherical_yn``, ``spherical_in``, and > ``spherical_kn``. Be aware that the new functions have different > signatures. > > The cross-class properties of `scipy.signal.lti` systems have been removed. > The following properties/setters have been removed: > > Name - (accessing/setting has been removed) - (setting has been removed) > > * StateSpace - (``num``, ``den``, ``gain``) - (``zeros``, ``poles``) > * TransferFunction (``A``, ``B``, ``C``, ``D``, ``gain``) - (``zeros``, > ``poles``) > * ZerosPolesGain (``A``, ``B``, ``C``, ``D``, ``num``, ``den``) - () > > ``signal.freqz(b, a)`` with ``b`` or ``a`` >1-D raises a ``ValueError``. > This > was a corner case for which it was unclear that the behavior was > well-defined. > > The method ``var`` of `scipy.stats.dirichlet` now returns a scalar rather > than > an ndarray when the length of alpha is 1. > > > Other changes > ============= > > SciPy now has a formal governance structure. It consists of a BDFL (Pauli > Virtanen) and a Steering Committee. See `the governance document > < > > https://github.com/scipy/scipy/blob/master/doc/source/dev/governance/governance.rst > >`_ > for details. > > It is now possible to build SciPy on Windows with MSVC + gfortran! > Continuous > integration has been set up for this build configuration on Appveyor, > building > against OpenBLAS. > > Continuous integration for OS X has been set up on TravisCI. > > The SciPy test suite has been migrated from ``nose`` to ``pytest``. > > ``scipy/_distributor_init.py`` was added to allow redistributors of SciPy > to > add custom code that needs to run when importing SciPy (e.g. checks for > hardware, DLL search paths, etc.). > > Support for PEP 518 (specifying build system requirements) was added - see > ``pyproject.toml`` in the root of the SciPy repository. > > In order to have consistent function names, the function > ``scipy.linalg.solve_lyapunov`` is renamed to > `scipy.linalg.solve_continuous_lyapunov`. The old name is kept for > backwards-compatibility. > > > Authors > ======= > > * @arcady + > * @xoviat + > * Anton Akhmerov > * Dominic Antonacci + > * Alessandro Pietro Bardelli > * Ved Basu + > * Michael James Bedford + > * Ray Bell + > * Juan M. Bello-Rivas + > * Sebastian Berg > * Felix Berkenkamp > * Jyotirmoy Bhattacharya + > * Matthew Brett > * Jonathan Bright > * Bruno Jim?nez + > * Evgeni Burovski > * Patrick Callier > * Mark Campanelli + > * CJ Carey > * Robert Cimrman > * Adam Cox + > * Michael Danilov + > * David Haberth?r + > * Andras Deak + > * Philip DeBoer > * Anne-Sylvie Deutsch > * Cathy Douglass + > * Dominic Else + > * Guo Fei + > * Roman Feldbauer + > * Yu Feng > * Jaime Fernandez del Rio > * Orestis Floros + > * David Freese + > * Adam Geitgey + > * James Gerity + > * Dezmond Goff + > * Christoph Gohlke > * Ralf Gommers > * Dirk Gorissen + > * Matt Haberland + > * David Hagen + > * Charles Harris > * Lam Yuen Hei + > * Jean Helie + > * Gaute Hope + > * Guillaume Horel + > * Franziska Horn + > * Yevhenii Hyzyla + > * Vladislav Iakovlev + > * Marvin Kastner + > * Mher Kazandjian > * Thomas Keck > * Adam Kurkiewicz + > * Ronan Lamy + > * J.L. Lanfranchi + > * Eric Larson > * Denis Laxalde > * Gregory R. Lee > * Felix Lenders + > * Evan Limanto > * Julian Lukwata + > * Fran?ois Magimel > * Syrtis Major + > * Charles Masson + > * Nikolay Mayorov > * Tobias Megies > * Markus Meister + > * Roman Mirochnik + > * Jordi Montes + > * Nathan Musoke + > * Andrew Nelson > * M.J. Nichol > * Juan Nunez-Iglesias > * Arno Onken + > * Nick Papior + > * Dima Pasechnik + > * Ashwin Pathak + > * Oleksandr Pavlyk + > * Stefan Peterson > * Ilhan Polat > * Andrey Portnoy + > * Ravi Kumar Prasad + > * Aman Pratik > * Eric Quintero > * Vedant Rathore + > * Tyler Reddy > * Joscha Reimer > * Philipp Rentzsch + > * Antonio Horta Ribeiro > * Ned Richards + > * Kevin Rose + > * Benoit Rostykus + > * Matt Ruffalo + > * Eli Sadoff + > * Pim Schellart > * Nico Schl?mer + > * Klaus Sembritzki + > * Nikolay Shebanov + > * Jonathan Tammo Siebert > * Scott Sievert > * Max Silbiger + > * Mandeep Singh + > * Michael Stewart + > * Jonathan Sutton + > * Deep Tavker + > * Martin Thoma > * James Tocknell + > * Aleksandar Trifunovic + > * Paul van Mulbregt + > * Jacob Vanderplas > * Aditya Vijaykumar > * Pauli Virtanen > * James Webber > * Warren Weckesser > * Eric Wieser + > * Josh Wilson > * Zhiqing Xiao + > * Evgeny Zhurko > * Nikolay Zinov + > * Z? Vin?cius + > > A total of 121 people contributed to this release. > People with a "+" by their names contributed a patch for the first time. > This list of names is automatically generated, and may not be fully > complete. > > > Cheers, > Ralf > -- > https://mail.python.org/mailman/listinfo/python-announce-list > > Support the Python Software Foundation: > http://www.python.org/psf/donations/ > From guru at digitalfantasy.it Wed Oct 25 12:35:34 2017 From: guru at digitalfantasy.it (Daniele Forghieri) Date: Wed, 25 Oct 2017 18:35:34 +0200 Subject: Let's talk about debuggers! In-Reply-To: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: <613f2331-c47f-7727-9634-8e28613e8f8e@digitalfantasy.it> Il 25/10/2017 15:07, Thomas Jollans ha scritto: > Hi, > > I just wanted to know what tools everyone used for debugging Python > applications - scripts / backend / desktop apps / notebooks / whatever. > Apart from the usual dance with log files and strategically inserted > print() calls, that is. > > Of course we all know and mildly dislike pdb. > > Personally, in practice, I'm most likely to need a debugger when > prototyping a function in a Jupyter notebook. There, ipdb, summoned with > the %%debug magic incantation, does the trick. > > Sometimes, though, I miss having a visual debugger. You know, the kind > that Visual Basic has had for decades. There's one in Chrome dev tools > if you ever have the misfortune of writing JavaScript. > > What options are there for Python (that work)? What text editors (and > IDEs) have a decent integrated debugger or debugging plugin? (Is there > anything for Sublime?) Does anyone use them? (How do YOU debug?) > > I vaguely remember WinPDB, but that hasn't seen a release in more than > seven years... > > I use the PyDev extension to Eclipse IDE, with the standard one at work (directly as a plugin to Eclipse) and with LiCplise (a bundle with eclipse, pydev and some other plugins) at home. You can put breakpoint in the code, see all the variables and so forth. ??? Daniele Forghieri From tjreedy at udel.edu Wed Oct 25 16:18:41 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Oct 2017 16:18:41 -0400 Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> Message-ID: <458fd856-8519-12eb-77fc-5a95bc18ad5f@udel.edu> On 10/25/2017 12:12 PM, Thomas Jollans wrote: > On 2017-10-25 15:57, Rustom Mody wrote: >> >> pdb inside emacs works (to a fashion) >> And it shows the arrow for current line so its at least quasi-gui >> >> I believe idle too is much more usable than a few years earlier > > I haven't used IDLE in years (if ever), partly because Tkinter is so > incredibly ugly, and feels so out-of-place on a modern PC (maybe it's > tolerable on Windows, I wouldn't know). On Windows, it uses native widgets when possible, so the look and feel of IDLE's editor is the same as some other apps, such as Notepad++. We are in the process of switching to the ttk widget set, which improve the look on Linux and even more so on Macs. The settings dialog was redone, in part, for 3.6.3 and 3.7. Which version are you using? > In any case I just tried it out and good lord that's terrible. The debugger UI has not gotten its badly needed makeover yet; there is a issue and patch to do so. > You can > set breakpoints in the editor (good), it shows locals (excellent), but > it doesn't show you what line you're at when stepping through (seriously?). Seriously, you either did not look, or you have a broken IDLE installation, or you ran into a severe bug I would like to know about. Rather than never, the next line is shown 2 or 3 times. 1.The module, line number, and code of the next line is shown immediately below the header of buttons and checkboxes. 2. The box below shows the complete stack of Python lines in the process of being run, including (again) the one that will be executed next. For each, it shows the module, line number, and text of the line. Bug: code and line number are reversed on the last line, which is the repeat of the next line. Enhancement: make list more readable with module, number, and code in neat columns. The window will have to be much wider, or each line split as in tracebacks, to do this. 3. If one checks the '[X] source' box at the top, the 'next' line is highlighted in an IDLE editor window. This is currently not checked by default because the debugger would currently open *every* file stepped into. When opening new windows is made optional, source will be checked by default. In summary, I think debugger should rate at least 'good' rather than 'fail' when it comes to showing you the next line. > I'll go back to not even touching IDLE with a ten-foot pole now. Do as you want, but please report accurately. There are bugs around the debugger, but not, that I know of, the one you claim. -- Terry Jan Reedy From mvoicem at gmail.com Wed Oct 25 16:31:16 2017 From: mvoicem at gmail.com (m) Date: Wed, 25 Oct 2017 22:31:16 +0200 Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: <59f0f495$0$5153$65785112@news.neostrada.pl> W dniu 25.10.2017 o?15:53, Ned Batchelder pisze: > On 10/25/17 9:07 AM, Thomas Jollans wrote: >> Hi, >> >> I just wanted to know what tools everyone used for debugging Python >> applications - scripts / backend / desktop apps / notebooks / whatever. >> Apart from the usual dance with log files and strategically inserted >> print() calls, that is. >> >> Of course we all know and mildly dislike pdb. >> >> Personally, in practice, I'm most likely to need a debugger when >> prototyping a function in a Jupyter notebook. There, ipdb, summoned with >> the %%debug magic incantation, does the trick. >> >> Sometimes, though, I miss having a visual debugger. You know, the kind >> that Visual Basic has had for decades. There's one in Chrome dev tools >> if you ever have the misfortune of writing JavaScript. >> >> What options are there for Python (that work)? What text editors (and >> IDEs) have a decent integrated debugger or debugging plugin? (Is there >> anything for Sublime?) Does anyone use them? (How do YOU debug?) >> >> I vaguely remember WinPDB, but that hasn't seen a release in more than >> seven years... >> >> > > pudb is a visual terminal debugger: https://pypi.python.org/pypi/pudb > > It uses the same commands as pdb, so it's easy to get started, but it > gives you a variables pane, with customizable presentation, and so on. > > One of my favorite features: you can add a set_trace line in your > program, and then if during the debugging session you realize you don't > want to stop there every time, you can disable that breakpoint even > though it's an explicit line of code telling the debugger to stop. +1 It's excellent piece of software. I started using it when debugging remote/server things, and ended with using eveywhere. It has two drawbacks: it doesn't remember your commandline, watches and sizes of panes, and has ugly default colour scheme. Few years ago, I used Eclipse+PyDev, but I was tired with recurring strange problems with Eclipse, which eg hanged until I reinstalled eclipse or removed workspace metadata. r. m. From tjol at tjol.eu Wed Oct 25 17:21:46 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 25 Oct 2017 23:21:46 +0200 Subject: Let's talk about debuggers! In-Reply-To: <458fd856-8519-12eb-77fc-5a95bc18ad5f@udel.edu> References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> <458fd856-8519-12eb-77fc-5a95bc18ad5f@udel.edu> Message-ID: <05d926ca-f80d-6b6d-2d38-3d98796bc8f7@tjol.eu> On 25/10/17 22:18, Terry Reedy wrote: > On 10/25/2017 12:12 PM, Thomas Jollans wrote: >> On 2017-10-25 15:57, Rustom Mody wrote: >>> >>> pdb inside emacs works (to a fashion) >>> And it shows the arrow for current line so its at least quasi-gui >>> >>> I believe idle too is much more usable than a few years earlier >> >> I haven't used IDLE in years (if ever), partly because Tkinter is so >> incredibly ugly, and feels so out-of-place on a modern PC (maybe it's >> tolerable on Windows, I wouldn't know). > > On Windows, it uses native widgets when possible, so the look and feel > of IDLE's editor is the same as some other apps, such as Notepad++. We > are in the process of switching to the ttk widget set, which improve the > look on Linux and even more so on Macs. The settings dialog was redone, > in part, for 3.6.3 and 3.7. Which version are you using? This was 3.6.0. The look is "meh", but the file selection dialog is (for me) the real usability disaster. > >> In any case I just tried it out and good lord that's terrible. > > The debugger UI has not gotten its badly needed makeover yet; there is a > issue and patch to do so. Glad to hear it! > >> You can >> set breakpoints in the editor (good), it shows locals (excellent), but >> it doesn't show you what line you're at when stepping through >> (seriously?). > > Seriously, you either did not look, or you have a broken IDLE > installation, or you ran into a severe bug I would like to know about. > Rather than never, the next line is shown 2 or 3 times. Sorry Terry, of course you're right. What I meant to write was that it doesn't *point* to the current line (in the editor) - as apparently emacs does. I did notice it quoting the current line in the debugger window. > > 1.The module, line number, and code of the next line is shown > immediately below the header of buttons and checkboxes. > > 2. The box below shows the complete stack of Python lines in the process > of being run, including (again) the one that will be executed next. For > each, it shows the module, line number, and text of the line. > > Bug: code and line number are reversed on the last line, which is the > repeat of the next line. Enhancement: make list more readable with > module, number, and code in neat columns. The window will have to be > much wider, or each line split as in tracebacks, to do this. This sounds like it'll make a nice improvement. > > 3. If one checks the '[X] source' box at the top, the 'next' line is > highlighted in an IDLE editor window. This is currently not checked by > default because the debugger would currently open *every* file stepped > into. When opening new windows is made optional, source will be checked > by default. Aha! > > In summary, I think debugger should rate at least 'good' rather than > 'fail' when it comes to showing you the next line. > >> I'll go back to not even touching IDLE with a ten-foot pole now. > > Do as you want, but please report accurately. There are bugs around the > debugger, but not, that I know of, the one you claim. Guilty as charged. Thomas From danceswithnumbers at gmail.com Wed Oct 25 17:22:58 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Wed, 25 Oct 2017 14:22:58 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> Message-ID: <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> So if the theoretical min compression limit (log2(n)*(x)) has a 3% margin but your transform has a less than 3% inflate rate at most then there is room for the transform to compress below the theoretical min. With every transform the entropy changes, the potential for greater compression also changes, with each pass you can compress untill the entropy is so random it can no longer be comressed. From mittra at juno.com Wed Oct 25 21:25:39 2017 From: mittra at juno.com (qrious) Date: Wed, 25 Oct 2017 18:25:39 -0700 (PDT) Subject: Determine the container class of an object in Python 3 Message-ID: Class1 is instantiated in Class2 as follows. Class2 also contains another variable, say: class Class2: class1 = Class1() a = 0 I want to create a method myDef() in Class1 that can read or write to a. How do I access a from within myDef() to access a? Calling Class2.a is not an option as Class1 does not have any knowledge about its container class a priori. Also, this will hardcode the path. I want to know of a built-in method (kind of equivalent of super() for inheritance) that will give me the container class reference, Class2 in this case. Thanks. From rosuav at gmail.com Wed Oct 25 21:40:45 2017 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 26 Oct 2017 12:40:45 +1100 Subject: Determine the container class of an object in Python 3 In-Reply-To: References: Message-ID: On Thu, Oct 26, 2017 at 12:25 PM, qrious wrote: > > Class1 is instantiated in Class2 as follows. Class2 also contains another variable, say: > > class Class2: > class1 = Class1() > a = 0 > > I want to create a method myDef() in Class1 that can read or write to a. How do I access a from within myDef() to access a? > > Calling Class2.a is not an option as Class1 does not have any knowledge about its container class a priori. Also, this will hardcode the path. I want to know of a built-in method (kind of equivalent of super() for inheritance) that will give me the container class reference, Class2 in this case. > > Thanks. There isn't any. Relationships like this are one-way; you can go from Class2 to its attribute named "class1", which is an instance of Class1, but you can't go back the other way. There could be multiple ways to get to that same object, so there's no way for it to know which one you want. Now, there MAY be some strange and arcane magic that you can do as you construct that object to figure out what class it's in. But generally, you should avoid that kind of thing. There is (almost certainly) a better way to accomplish whatever it is you're aiming for. ChrisA From steve+python at pearwood.info Wed Oct 25 22:16:42 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 26 Oct 2017 13:16:42 +1100 Subject: Compression of random binary data References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> Message-ID: <59f1458b$0$18634$b1db1813$d948b532@news.astraweb.com> On Thu, 26 Oct 2017 08:22 am, danceswithnumbers at gmail.com wrote: > with each pass you can compress untill the entropy is so random it can no > longer be comressed. Which is another way of saying that you cannot compress random binary data. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Wed Oct 25 22:25:19 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 26 Oct 2017 13:25:19 +1100 Subject: Determine the container class of an object in Python 3 References: Message-ID: <59f14791$0$18584$b1db1813$d948b532@news.astraweb.com> On Thu, 26 Oct 2017 12:25 pm, qrious wrote: > Class1 is instantiated in Class2 as follows. Class2 also contains another > variable, say: > > class Class2: > class1 = Class1() > a = 0 > > I want to create a method myDef() in Class1 that can read or write to a. How > do I access a from within myDef() to access a? The only way is to hard-code the name "Class2" in myDef. But the usual way to fix this problem is: class Class1: def __init__(self, owner): self.owner = owner def myDef(self): return self.owner.a def Class2: def __init__(self): self.class1 = Class1(self) self.a = 0 Technically, it is a circular reference, but don't worry about it, Python's garbage collector can resolve such circular references. If it becomes a problem -- and it shouldn't, but if it does -- you can use a weakref. But seriously, don't worry about it unless you need to. Another solution is to make Class1 a mixin, and then rely on inheritence: class MyMixin: def myDef(self): return self.a class Class2(MyMixin): def __init__(self): self.a = 0 -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From daniel at tangemann-plus.de Thu Oct 26 00:37:54 2017 From: daniel at tangemann-plus.de (Daniel Tangemann) Date: Thu, 26 Oct 2017 06:37:54 +0200 (CEST) Subject: IDLE doesn't recognise installed packages In-Reply-To: References: <2096431256.470671.1508768584232@webmail.strato.com> Message-ID: <419836971.68832.1508992674551@webmail.strato.com> ok, I did that. I noticed that this path: 'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' is missing when I run the python.exe without IDLE. how do I fix this? also I get a syntax error when I try that: "To make sure you are running pip with the same binary as IDLE, enter path-to-binary -m pip C:\Programs\Python37\python.exe -m pip list" I need to have this explained with baby steps, I'm sure I'm just doing something wrong there. thank you for the help! I really appreciate it! > Terry Reedy hat am 24. Oktober 2017 um 08:36 geschrieben: > > > On 10/23/2017 10:23 AM, Daniel Tangemann wrote: > > I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message that > > it's already installed. how do I fix this? > > cheers > > Recognition of installed packages is done by the python running IDLE and > executing your import statements, by not IDLE. The only effect IDLE > could have is any manipulation of sys.path. > > You can find the executable running IDLE with > > >>> import sys; sys.executable > 'C:\\Programs\\Python37\\pythonw.exe' > > Find the sys.path being used with > >>> sys.path > > If you run the same binary (minus the 'w' if present), you can find the > sys.path used without IDLE. You can also test imports without IDLE in use. > > It is possible that you have more than one binary around, but I cannot > tell from here. To make sure you are running pip with the same binary > as IDLE, enter path-to-binary -m pip instance, on windows, given the above > > path> C:\Programs\Python37\python.exe -m pip list > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list From tjol at tjol.eu Thu Oct 26 03:57:54 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 26 Oct 2017 09:57:54 +0200 Subject: Compression of random binary data In-Reply-To: <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> Message-ID: On 2017-10-25 23:22, danceswithnumbers at gmail.com wrote: > With every transform the entropy changes, That's only true if the "transform" loses or adds information. If it loses information, that's lossy compression, which is only useful in very specific (but also extremely common) circumstances. If it adds information, that's poetry, not compression. -- Thomas Jollans From hjp-usenet3 at hjp.at Thu Oct 26 04:22:31 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Thu, 26 Oct 2017 10:22:31 +0200 Subject: Compression of random binary data References: <2462e309-4a21-48fd-b708-c5d98f5159a1@googlegroups.com> <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <59efbf23$0$18562$b1db1813$d948b532@news.astraweb.com> Message-ID: On 2017-10-24 22:30, Steve D'Aprano wrote: > On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: > >> On 2017-10-23 04:21, Steve D'Aprano wrote: >>> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >>>> >>> If the probability of certain codes (either single codes, or sequences of >>> codes) are non-equal, then you can take advantage of that by encoding the >>> common cases into a short representation, and the uncommon and rare cases >>> into a longer representation. As you say: >>> >>> >>>> Otherwise, if ( 0, 0 ) is much more frequent, >>>> we can encode ( 0, 0 ) by "0" and >>>> >>>> ( 0, 1 ) by "101", >>>> ( 1, 0 ) by "110", and >>>> ( 1, 1 ) by "111". >>>> >>>> And we could then use /less/ than two bits on the >>>> average. >>> >>> That's incorrect. On average you use 2.5 bits. >>> >>> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) >> >> I disagree. If the distribution is not equal, then the average needs to >> take the different probabilities into account. > > I think I would call that the *weighted* average rather than the average. If there are 4 guys who are 180 cm tall and one who is 2 metres tall, would you say their average height is 184 cm ((180 * 4 + 200 * 1)/(4 + 1)) or 190 cm ((180 + 200)/2)? For me that's the same situation. > Regardless of what we call it, of course both you and Stefan are right in how > to calculate it, and such a variable-length scheme can be used to compress > the data. > > E.g. given the sequence 00000011 which would take 8 bits in the obvious > encoding, we can encode it as "000111" which takes only 6 bits. > > But the cost of this encoding scheme is that *some* bit sequences expand, e.g. > the 8 bit sequence 11111100 is encoded as "1111111110" which requires 10 > bits. Right. This is most obvious in Huffman encoding, where each symbol is replaced by a sequence of bits which is directly related to the frequency of that symbol. So the letter 'e' might be encoded in 3 or 4 bits (in a corpus of text I happen to have lying around, about 1 in 11 characters is an e and ld(11) = 3.43) while the tilde (about 1 character in 21000) would be encoded in 14 or 15 bits. That's basically how all lossless compression algorithms work: Replacing more frequent bit sequences with shorter sequences and replacing less frequent sequences with longer ones. (Although most operate on variable length byte sequences not individual bytes. I find the LZW algorithm (as used in Unix compress and GIF images) a particularly instructive example). > The end result is that averaged over all possible bit sequences (of a certain > size), this encoding scheme requires MORE space than the obvious 0/1 bits. > > But in practice we don't care much, because the data sequences we care about > are usually not "all possible bit sequences", but a heavily restricted subset > where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. Right. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From stephanh42 at gmail.com.invalid Thu Oct 26 10:24:13 2017 From: stephanh42 at gmail.com.invalid (Stephan Houben) Date: 26 Oct 2017 14:24:13 GMT Subject: Windows alternative: multiprocessing.connection.wait on Pipe, Tkinter File Handlers References: Message-ID: Op 2017-10-23, Thomas Jollans schreef : > You might wait in a thread > and somehow (no idea what the best way to do this is in tkinter) pass a > message to the GUI thread when it's done. AFAIK, this is a real problem in Tkinter on Windows. On Posix you can use the self-pipe trick. But on Windows, Tkinter cannot wait on a pipe. I see two solutions (well, three, really): 1. Use a different toolkit, e.g. PyQt has explicit support for notifying the GUI thread from another thread. 2. Use Cygwin-based Python. If this is an option, the Cygwin people did already all the heavy lifting for providing Posix-like select() semantics. 3. Regular polling. This has been rejected by the OP, but in my experience can produce reasonable results when done "properly", i.e. use the Tkinter "after" method with a reasonable time interval (in the past I have used a strategy of starting with 10 ms and then, on no event, slowly back off to polling every 200ms). It is by far the simplest solution, and AFAIK the only one which will work with the standard Python distribution + Tkinter. Stephan From davide.dalmasso at gmail.com Thu Oct 26 11:04:32 2017 From: davide.dalmasso at gmail.com (Davide Dalmasso) Date: Thu, 26 Oct 2017 08:04:32 -0700 (PDT) Subject: =?UTF-8?B?cmVhZF90YWJsZSDigqwgc3ltYm9s?= Message-ID: Dear all, I'm trying to read a txt file with read_table but in the file there are some string that contain the ? symbol and the procedure returns me an error. I tried with encoding='utf-8' but the problem is still there: pd.read_table('filename.txt', sep=';', encoding='utf-8') Anyone can help me? Many thanks in advance Davide From tjol at tjol.eu Thu Oct 26 11:31:06 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 26 Oct 2017 17:31:06 +0200 Subject: =?UTF-8?Q?Re:_read=5ftable_=e2=82=ac_symbol?= In-Reply-To: References: Message-ID: <63afa9c3-7b07-baf2-7b12-3a203ec19c76@tjol.eu> On 2017-10-26 17:04, Davide Dalmasso wrote: > Dear all, > I'm trying to read a txt file with read_table but in the file there are some string that contain the ? symbol and the procedure returns me an error. > I tried with encoding='utf-8' but the problem is still there: > pd.read_table('filename.txt', sep=';', encoding='utf-8') > Anyone can help me? > Many thanks in advance > > Davide What error? Are you sure the file is UTF-8 encoded? (and not ISO-8859-15, or Windows-1252, or something?) -- Thomas Jollans From torriem at gmail.com Thu Oct 26 12:28:13 2017 From: torriem at gmail.com (Michael Torrie) Date: Thu, 26 Oct 2017 10:28:13 -0600 Subject: =?UTF-8?Q?Re:_read=5ftable_=e2=82=ac_symbol?= In-Reply-To: References: Message-ID: On 10/26/2017 09:04 AM, Davide Dalmasso wrote: > Dear all, > I'm trying to read a txt file with read_table but in the file there are some string that contain the ? symbol and the procedure returns me an error. > I tried with encoding='utf-8' but the problem is still there: > pd.read_table('filename.txt', sep=';', encoding='utf-8') > Anyone can help me? > Many thanks in advance What error? Please post the entire traceback. From salaboud at anderson.edu Thu Oct 26 14:01:14 2017 From: salaboud at anderson.edu (salaboud at anderson.edu) Date: Thu, 26 Oct 2017 11:01:14 -0700 (PDT) Subject: Test Bank for Entrepreneurship The Practice and Mindset 1st Edition by Neck In-Reply-To: References: Message-ID: On Saturday, July 1, 2017 at 1:36:10 PM UTC-4, Test Banks wrote: > Greetings, > > You can get Test Bank for " Entrepreneurship The Practice and Mindset 1st Edition by Neck " at very reasonable price. Our team is available 24/7 and 365 days / year to respond your requests. Send us an email at pro.fast(@)hotmail(dot)com > > Place your order at PRO.FAST(@)HOTMAIL(DOT)COM > > ISBN Numbers for this book (ISBN-10: 1483383520 and ISBN-13: 9781483383521) > > ENTREPRENEURSHIP THE PRACTICE AND MINDSET 1ST EDITION BY NECK TEST BANK > > You can also email for other Entrepreneurship books Solutions and Test Bank at low prices and our team will try to get all resources you need. > > Do not post your reply here. Simply send us an email at PRO.FAST(AT)HOTMAIL(DOT)COM > > CHEERS, i want the test bank when could i get it From tjreedy at udel.edu Thu Oct 26 15:35:07 2017 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Oct 2017 15:35:07 -0400 Subject: IDLE doesn't recognise installed packages In-Reply-To: <419836971.68832.1508992674551@webmail.strato.com> References: <2096431256.470671.1508768584232@webmail.strato.com> <419836971.68832.1508992674551@webmail.strato.com> Message-ID: On 10/26/2017 12:37 AM, Daniel Tangemann wrote: > ok, I did that. I noticed that this path: 'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' is missing when I run the python.exe without IDLE. how do I fix this? Having idlelib on the path or not should not make any difference for anything installed by pip. It is not even needed by IDLE, since IDLE imports its modules via Lib. > also I get a syntax error when I try that: What you try what? Post the entire traceback. > "To make sure you are running pip with the same binary > as IDLE, enter path-to-binary -m pip > Terry Reedy hat am 24. Oktober 2017 um 08:36 geschrieben: >> >> >> On 10/23/2017 10:23 AM, Daniel Tangemann wrote: >>> I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message > that >>> it's already installed. how do I fix this? >>> cheers >> >> Recognition of installed packages is done by the python running IDLE and >> executing your import statements, by not IDLE. The only effect IDLE >> could have is any manipulation of sys.path. >> >> You can find the executable running IDLE with >> >>>>> import sys; sys.executable >> 'C:\\Programs\\Python37\\pythonw.exe' >> >> Find the sys.path being used with >>>>> sys.path >> >> If you run the same binary (minus the 'w' if present), you can find the >> sys.path used without IDLE. You can also test imports without IDLE in use. >> >> It is possible that you have more than one binary around, but I cannot >> tell from here. To make sure you are running pip with the same binary >> as IDLE, enter path-to-binary -m pip > instance, on windows, given the above >> >> path> C:\Programs\Python37\python.exe -m pip list >> >> -- >> Terry Jan Reedy >> >> -- >> https://mail.python.org/mailman/listinfo/python-list -- Terry Jan Reedy From danceswithnumbers at gmail.com Thu Oct 26 16:38:09 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Thu, 26 Oct 2017 13:38:09 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> Message-ID: <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Thomas Jollans On 2017-10-25 23:22,?danceswi... at gmail.com?wrote:? > With every transform the entropy changes,? That's only true if the "transform" loses or adds information.? If it loses information, that's lossy compression, which is only useful? in very specific (but also extremely common) circumstances.? If it adds information, that's poetry, not compression.? Not true! You can transform a stream lossless, and change its entropy without adding to or taking away. These two streams 16 bits are the same but one is reversed. 1111000010101011 This equals 61611 This can be represented using 0-6 log2(7)*5= 14.0367746103 bits 1101010100001111 This equals 54543 This can be represented using 0-5 log2(6)*5= 12.9248125036 bits In reality you can express 54543 with 10 bits. From ben.usenet at bsb.me.uk Thu Oct 26 18:53:30 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 26 Oct 2017 23:53:30 +0100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> Message-ID: <87bmktjzbp.fsf@bsb.me.uk> Gregory Ewing writes: > Ben Bacarisse wrote: >> The trouble is a pedagogic one. Saying "you can't compress random data" >> inevitably leads (though, again, this is just my experience) to endless >> attempts to define random data. > > It's more about using terms without making sure everyone agrees > on the definitions being used. > > In this context, "random data" really means "uniformly distributed > data", i.e. any bit sequence is equally likely to be presented as > input. *That's* what information theory says can't be compressed. But that has to be about the process that gives rise to the data, not the data themselves. No finite collection of bits has the property you describe. If I say: "here is some random data..." you can't tell if it is or is not from a random source. I can, as a parlour trick, compress and recover this "random data" because I chose it. A source of random can be defined but "random data" is much more illusive. >> I think "arbitrary data" (thereby including the results of compression >> by said algorithm) is the best way to make progress. > > I'm not sure that's much better, because it doesn't home in > on the most important thing, which is the probability > distribution. I think the argument that you can't compress arbitrary data is simpler to make. You don't have to define it (except in the very simplest terms) and it's obvious that it includes the results of previous compressions. -- Ben. From toby at tobiah.org Thu Oct 26 19:02:28 2017 From: toby at tobiah.org (Tobiah) Date: Thu, 26 Oct 2017 16:02:28 -0700 Subject: Sequence MIDI events from python. Message-ID: I know that there are a few good MIDI libraries out there. The examples that I've seen for real-time triggering of events rely on a sleep function to realize the timing. This is not accurate or precise enough for musical applications. What options do I have if I want to write a MIDI sequencer in python? I imagine I'd have to sync to an audio device to get the timing right. Thank for any help. Tobiah From toby at tobiah.org Thu Oct 26 19:44:13 2017 From: toby at tobiah.org (Tobiah) Date: Thu, 26 Oct 2017 16:44:13 -0700 Subject: Sequence MIDI events from python. (Posting On Python-List Prohibited) References: <4b706bd6-8a48-4c09-9cd3-85a5451d9531@googlegroups.com> Message-ID: On 10/26/2017 4:30 PM, Lawrence D?Oliveiro wrote: > On Friday, October 27, 2017 at 12:02:40 PM UTC+13, Tobiah wrote: > >> I know that there are a few good MIDI libraries out there. >> The examples that I've seen for real-time triggering >> of events rely on a sleep function to realize the timing. >> This is not accurate or precise enough for musical applications. > > Why not look at the source code of the ?good? ones? > No get. zample plez. From ian.g.kelly at gmail.com Thu Oct 26 21:26:11 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 26 Oct 2017 19:26:11 -0600 Subject: Compression of random binary data In-Reply-To: <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Thu, Oct 26, 2017 at 2:38 PM, wrote: > > Thomas Jollans > > On 2017-10-25 23:22, danceswi... at gmail.com wrote: >> With every transform the entropy changes, > > That's only true if the "transform" loses or adds information. > > If it loses information, that's lossy compression, which is only useful > in very specific (but also extremely common) circumstances. > > If it adds information, that's poetry, not compression. > > > > Not true! You can transform a stream lossless, and change its entropy without adding to or taking away. These two streams 16 bits are the same but one is reversed. > > 1111000010101011 > This equals > 61611 > This can be represented using > 0-6 log2(7)*5= 14.0367746103 bits > > > 1101010100001111 > This equals > 54543 > This can be represented using > 0-5 log2(6)*5= 12.9248125036 bits > > In reality you can express 54543 with 10 bits. I don't know what you're calculating here but it isn't Shannon entropy. Shannon entropy is correctly calculated for a data source, not an individual message, but if we assume the two numbers above to be the result of a Bernoulli process with probabilities matching the frequencies of the bits in the numbers, then the total entropy of 16 events is: py> 16 * (-9/16 * math.log2(9/16) - 7/16 * math.log2(7/16)) 15.81919053261596 Approximately 15.8 bits. This is the same no matter what order the events occur in. From daniel at tangemann-plus.de Thu Oct 26 22:12:34 2017 From: daniel at tangemann-plus.de (Daniel Tangemann) Date: Fri, 27 Oct 2017 04:12:34 +0200 (CEST) Subject: IDLE doesn't recognise installed packages In-Reply-To: References: <2096431256.470671.1508768584232@webmail.strato.com> <419836971.68832.1508992674551@webmail.strato.com> Message-ID: <4226798.121808.1509070354128@webmail.strato.com> hi, I had tried typing: "path-to-binary -m pip Terry Reedy hat am 26. Oktober 2017 um 21:35 geschrieben: > > > On 10/26/2017 12:37 AM, Daniel Tangemann wrote: > > ok, I did that. I noticed that this path: 'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' is missing when I run the python.exe without IDLE. how do I fix this? > > Having idlelib on the path or not should not make any difference for > anything installed by pip. It is not even needed by IDLE, since IDLE > imports its modules via Lib. > > > > also I get a syntax error when I try that: > > What you try what? Post the entire traceback. > > > "To make sure you are running pip with the same binary > > as IDLE, enter path-to-binary -m pip > Your path-to-binary appears to be > > C:\Users\Daniel86\AppData\Local\Programs\Python\Python36\python.exe > > You should be able to replace that with > > py -3.6 > > but try > > py -3.6 -c "import sys; sys.executable" > > to be sure. > > >> Terry Reedy hat am 24. Oktober 2017 um 08:36 geschrieben: > >> > >> > >> On 10/23/2017 10:23 AM, Daniel Tangemann wrote: > >>> I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message > > that > >>> it's already installed. how do I fix this? > >>> cheers > >> > >> Recognition of installed packages is done by the python running IDLE and > >> executing your import statements, by not IDLE. The only effect IDLE > >> could have is any manipulation of sys.path. > >> > >> You can find the executable running IDLE with > >> > >>>>> import sys; sys.executable > >> 'C:\\Programs\\Python37\\pythonw.exe' > >> > >> Find the sys.path being used with > >>>>> sys.path > >> > >> If you run the same binary (minus the 'w' if present), you can find the > >> sys.path used without IDLE. You can also test imports without IDLE in use. > >> > >> It is possible that you have more than one binary around, but I cannot > >> tell from here. To make sure you are running pip with the same binary > >> as IDLE, enter path-to-binary -m pip >> instance, on windows, given the above > >> > >> path> C:\Programs\Python37\python.exe -m pip list > >> > >> -- > >> Terry Jan Reedy > >> > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list From danceswithnumbers at gmail.com Thu Oct 26 22:19:10 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Thu, 26 Oct 2017 19:19:10 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> It looks like that averages my two examples. Hmmmm by the way that equation is really cool....why does it return a high bit count when compared to >>>dec to bin? From randyliu17 at gmail.com Thu Oct 26 22:25:09 2017 From: randyliu17 at gmail.com (randyliu17 at gmail.com) Date: Thu, 26 Oct 2017 19:25:09 -0700 (PDT) Subject: Python noob having a little trouble with strings Message-ID: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> If s1 = "Welcome students", what is the output when you print the following: 1. s4 = 3 * s1 2. s1[3 : 6] 3. 'W' in s1 4. S1[-1] 5. S1[:-1] Any help would be great, thanks! From robertvstepp at gmail.com Thu Oct 26 22:40:51 2017 From: robertvstepp at gmail.com (boB Stepp) Date: Thu, 26 Oct 2017 21:40:51 -0500 Subject: Python noob having a little trouble with strings In-Reply-To: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> Message-ID: On Thu, Oct 26, 2017 at 9:25 PM, wrote: > If s1 = "Welcome students", what is the output when you print the following: > > 1. s4 = 3 * s1 > > 2. s1[3 : 6] > > 3. 'W' in s1 > > 4. S1[-1] > > 5. S1[:-1] > > Any help would be great, thanks! Why not find out for yourself and print these in the Python interpreter? For instance: > py Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. py3: s1 = "Welcome students" py3: s4 = 3 * s1 py3: print(s4) Welcome studentsWelcome studentsWelcome students -- boB From danceswithnumbers at gmail.com Thu Oct 26 22:48:55 2017 From: danceswithnumbers at gmail.com (danceswithnumbers at gmail.com) Date: Thu, 26 Oct 2017 19:48:55 -0700 (PDT) Subject: Compression of random binary data In-Reply-To: <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> Message-ID: <2e19b3bd-ef47-445d-b238-4ceb8834189d@googlegroups.com> Shouldn't that be? py> 16 * (-7/16 * math.log2(7/16) - 6/16 * math.log2(6/16))?= From randyliu17 at gmail.com Thu Oct 26 22:59:10 2017 From: randyliu17 at gmail.com (randyliu17 at gmail.com) Date: Thu, 26 Oct 2017 19:59:10 -0700 (PDT) Subject: Python noob having a little trouble with strings In-Reply-To: References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> Message-ID: <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> On Thursday, October 26, 2017 at 7:41:10 PM UTC-7, boB Stepp wrote: > On Thu, Oct 26, 2017 at 9:25 PM, wrote: > > If s1 = "Welcome students", what is the output when you print the following: > > > > 1. s4 = 3 * s1 > > > > 2. s1[3 : 6] > > > > 3. 'W' in s1 > > > > 4. S1[-1] > > > > 5. S1[:-1] > > > > Any help would be great, thanks! > > Why not find out for yourself and print these in the Python > interpreter? For instance: > > > py > Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > py3: s1 = "Welcome students" > py3: s4 = 3 * s1 > py3: print(s4) > Welcome studentsWelcome studentsWelcome students > > > > -- > boB Hi Bob, thanks for responding. I'm not sure where to do so, my professor had us download Pycharm for mac's which uses python 2.6 From formisc at gmail.com Fri Oct 27 01:18:04 2017 From: formisc at gmail.com (Andrew Z) Date: Fri, 27 Oct 2017 01:18:04 -0400 Subject: How to plot Message-ID: Hello, i'd like to create a graph/plot based a DB table's data, but not sure where to start. I also would like to have the following functionality: a. i'd like to have it in the separate window ( xwindow to be precise). b. and i'd like to have the graph updating with every record added to the table. The workflow: a. main code is running and occasionally adding records to the table b. graph gets updated "automagically" and is shown in a separate from the main terminal window. Main program does not have GUI, nor is Web based, just runs in the terminal on Xwindows. i don't' really care about cross-platform compability and only want to run that on linux xwindows. Thank you for your advise. From marko at pacujo.net Fri Oct 27 01:41:11 2017 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 27 Oct 2017 08:41:11 +0300 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> Message-ID: <87h8ulqhag.fsf@elektro.pacujo.net> Ben Bacarisse : >> In this context, "random data" really means "uniformly distributed >> data", i.e. any bit sequence is equally likely to be presented as >> input. *That's* what information theory says can't be compressed. > > But that has to be about the process that gives rise to the data, not > the data themselves. No finite collection of bits has the property you > describe. Correct. Randomness is meaningful only in reference to future events. Once the events take place, they cease to be random. A finite, randomly generated collection of bits can be tested against a probabilistic hypothesis using statistical methods. Marko From no.email at nospam.invalid Fri Oct 27 01:43:01 2017 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 26 Oct 2017 22:43:01 -0700 Subject: Let's talk about debuggers! References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> <564b3740-85f5-4b3e-9b93-728f91accfc6@googlegroups.com> <458fd856-8519-12eb-77fc-5a95bc18ad5f@udel.edu> Message-ID: <878tfxuowq.fsf@nightsong.com> Terry Reedy writes: > On Windows, [IDLE] uses native widgets when possible... > In summary, I think debugger should rate at least 'good' rather than > fail' when it comes to showing you the next line. I actually like how the Tk widgets look. I've done some semi-industrial applications with tkinter and they have a nice factory-floor vibe. I generally just use pdb for debugging. The feature I miss most is the ability to trap to the debugger if the program throws an unhandled exception. I think some other Python debuggers do support that. I've found it invaluable in other languages. From lutz.horn at posteo.de Fri Oct 27 02:48:01 2017 From: lutz.horn at posteo.de (Lutz Horn) Date: Fri, 27 Oct 2017 08:48:01 +0200 Subject: Python noob having a little trouble with strings In-Reply-To: <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> Message-ID: <20171027064801.GA8014@lutz-pc.ecm4u.intra> On Thu, Oct 26, 2017 at 07:59:10PM -0700, randyliu17 at gmail.com wrote: > Hi Bob, thanks for responding. I'm not sure where to do so, my > professor had us download Pycharm for mac's which uses python 2.6 The code from your question is not specific to Python 2 or 3. Just try it in the Python installation you have available. Lutz From robin at reportlab.com Fri Oct 27 03:57:41 2017 From: robin at reportlab.com (Robin Becker) Date: Fri, 27 Oct 2017 08:57:41 +0100 Subject: Let's talk about debuggers! In-Reply-To: References: <27f3e171-c9ca-2d08-2dd6-bce45d2967fa@tjol.eu> Message-ID: <118b3c5f-0c30-757e-b9ee-0052bab19814@chamonix.reportlab.co.uk> On 25/10/2017 15:08, Michele Simionato wrote: > pdb plus plus: https://pypi.python.org/pypi/pdbpp > I like the idea, but in putty at least changing the terminal size causes pdb++ to detach immediately from the process and mess up the screen. I think this is caused by (5, 'Input/output error') here /home/rptlab/devel/otr/local/lib/python2.7/site-packages/pyrepl/fancy_termios.py in tcsetattr > def copy(self): > return self.__class__(self.as_list()) > def tcgetattr(fd): > return TermState(termios.tcgetattr(fd)) > def tcsetattr(fd, when, attrs): > termios.tcsetattr(fd, when, attrs.as_list()) error is here > class Term(TermState): > TS__init__ = TermState.__init__ > def __init__(self, fd=0): > self.TS__init__(termios.tcgetattr(fd)) > self.fd = fd -- Robin Becker From tjol at tjol.eu Fri Oct 27 04:22:35 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 27 Oct 2017 10:22:35 +0200 Subject: How to plot In-Reply-To: References: Message-ID: <1d550d5d-d92d-0555-1177-bd58f3e238c3@tjol.eu> On 2017-10-27 07:18, Andrew Z wrote: > Hello, > i'd like to create a graph/plot based a DB table's data, but not sure > where to start. I > > also would like to have the following functionality: > a. i'd like to have it in the separate window ( xwindow to be precise). > b. and i'd like to have the graph updating with every record added to the > table. > > The workflow: > a. main code is running and occasionally adding records to the table > b. graph gets updated "automagically" and is shown in a separate from the > main terminal window. > > Main program does not have GUI, nor is Web based, just runs in the terminal > on Xwindows. > > i don't' really care about cross-platform compability and only want to run > that on linux xwindows. > > Thank you for your advise. > Matplotlib is the standard Python plotting library, and it has GUI backends (Tk, Qt, Gtk, ...) that should be perfectly suitable for your purposes. The API maybe takes some getting used to, but there are plenty of examples to be found around the web, and the main documentation is quite good. If you need the plots to update quickly (it doesn't sound like it) then you might need to luck into something else, like pyqtgraph or vispy. -- Thomas Jollans From hjp-usenet3 at hjp.at Fri Oct 27 04:49:54 2017 From: hjp-usenet3 at hjp.at (Peter J. Holzer) Date: Fri, 27 Oct 2017 10:49:54 +0200 Subject: Python noob having a little trouble with strings References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> Message-ID: On 2017-10-27 02:59, randyliu17 at gmail.com wrote: > On Thursday, October 26, 2017 at 7:41:10 PM UTC-7, boB Stepp wrote: >> On Thu, Oct 26, 2017 at 9:25 PM, wrote: [...] >> Why not find out for yourself and print these in the Python >> interpreter? For instance: >> >> > py >> Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 >> bit (AMD64)] on win32 [...] > Hi Bob, thanks for responding. I'm not sure where to do so, my > professor had us download Pycharm for mac's which uses python 2.6 Open "Python Console ..." in the "Tools" menu. Or just open a terminal window and type "python" there. BTW, I find it hard to believe that PyCharm for the Mac "comes with" Python 2.6. Python 2.6 is quite old. The Linux version isn't bundled with a python interpreter and just uses whatever is already installed on the machine. I guess it's the same for the Mac: Your version of MacOS happens to include Python 2.6, so this is what PyCharm uses. Oh, and ask your professor if you are really supposed to use Python 2.6. There are differences between versions (especially between 2.x and 3.x) and you'll be hopelessly confused if you use 2.6 and your professor's examples are written for 3.5. hp -- _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung: |_|_) | | Man feilt solange an seinen Text um, bis | | | hjp at hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpa?t. -- Ralph Babel From ben.usenet at bsb.me.uk Fri Oct 27 05:53:25 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Fri, 27 Oct 2017 10:53:25 +0100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> <87h8ulqhag.fsf@elektro.pacujo.net> Message-ID: <8760b0kjca.fsf@bsb.me.uk> Marko Rauhamaa writes: > Ben Bacarisse : > >>> In this context, "random data" really means "uniformly distributed >>> data", i.e. any bit sequence is equally likely to be presented as >>> input. *That's* what information theory says can't be compressed. >> >> But that has to be about the process that gives rise to the data, not >> the data themselves. No finite collection of bits has the property you >> describe. > > Correct. Randomness is meaningful only in reference to future events. > Once the events take place, they cease to be random. > > A finite, randomly generated collection of bits can be tested against a > probabilistic hypothesis using statistical methods. But beware of parlour tricks. You can't reliably test for random looking data that are, in fact, carefully crafted. If the claim is about compressing arbitrary data, then the claimant won't mind testing inputs chosen by me! The only reason this matters is that such people usually won't reveal the algorithm. -- Ben. From robin at reportlab.com Fri Oct 27 07:15:06 2017 From: robin at reportlab.com (Robin Becker) Date: Fri, 27 Oct 2017 12:15:06 +0100 Subject: headless python app for android/ios Message-ID: <62721d1a-5a7b-1da5-efc9-1ab24ffbb14e@chamonix.reportlab.co.uk> In the past we have developed reportlab applications for use on android/ios devices. We used Kivy for the gui and the kivy setup did allow us to create a working reportlab pdf producer under the kivy gui. It was not exactly easy, but in the end we had a working PDF producer. A possible requirement is for pdf production using reportlab, but with others providing the gui using possible something called electron. I know very little about what is actually possible to provide api's in python under ios/android. Can the front end spawn a python process or must we run some kind of background app with listening sockets etc etc? Can applications call out to the world to download updated templates and so on? Any pointers would be useful. -- Robin Becker From formisc at gmail.com Fri Oct 27 08:53:15 2017 From: formisc at gmail.com (Andrew Z) Date: Fri, 27 Oct 2017 08:53:15 -0400 Subject: How to plot In-Reply-To: <1d550d5d-d92d-0555-1177-bd58f3e238c3@tjol.eu> References: <1d550d5d-d92d-0555-1177-bd58f3e238c3@tjol.eu> Message-ID: Rrhank you Thomas. On Oct 27, 2017 04:23, "Thomas Jollans" wrote: > On 2017-10-27 07:18, Andrew Z wrote: > > Hello, > > i'd like to create a graph/plot based a DB table's data, but not sure > > where to start. I > > > > also would like to have the following functionality: > > a. i'd like to have it in the separate window ( xwindow to be precise). > > b. and i'd like to have the graph updating with every record added to > the > > table. > > > > The workflow: > > a. main code is running and occasionally adding records to the table > > b. graph gets updated "automagically" and is shown in a separate from > the > > main terminal window. > > > > Main program does not have GUI, nor is Web based, just runs in the > terminal > > on Xwindows. > > > > i don't' really care about cross-platform compability and only want to > run > > that on linux xwindows. > > > > Thank you for your advise. > > > > Matplotlib is the standard Python plotting > library, and it has GUI backends (Tk, Qt, Gtk, ...) that should be > perfectly suitable for your purposes. > > The API maybe takes some getting used to, but there are plenty of > examples to be found around the web, and the main documentation is quite > good. > > If you need the plots to update quickly (it doesn't sound like it) then > you might need to luck into something else, like pyqtgraph or vispy. > > > -- > Thomas Jollans > -- > https://mail.python.org/mailman/listinfo/python-list > From davidgab283 at gmail.com Fri Oct 27 09:56:39 2017 From: davidgab283 at gmail.com (David Gabriel) Date: Fri, 27 Oct 2017 15:56:39 +0200 Subject: from packaging import version as pack_version ImportError: No module named packaging Message-ID: Dears, I am running a python code that generates for me this error : from packaging import version as pack_version ImportError: No module named packaging I googled it and I have found so many suggestions regarding updating 'pip' and installing python-setuptools but all of these did not fix this issue. Do you have any idea how can I solve this problem. Thanks in advance. From lutz.horn at posteo.de Fri Oct 27 10:13:43 2017 From: lutz.horn at posteo.de (Lutz Horn) Date: Fri, 27 Oct 2017 16:13:43 +0200 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: References: Message-ID: <20171027140819.GD8014@lutz-pc.ecm4u.intra> On Fri, Oct 27, 2017 at 03:56:39PM +0200, David Gabriel wrote: > from packaging import version as pack_version > ImportError: No module named packaging > > I googled it and I have found so many suggestions regarding updating > 'pip' and installing python-setuptools but all of these did not fix > this issue. So many questions: * What is your Python version? * Do you use virtualenv? * How? * Did you install packaging in this virtualenv? Just one example of making this work: $ mkdir /tmp/pack $ cd /tmp/pack $ virtualenv -p $(which python3.5) . Running virtualenv with interpreter /usr/bin/python3.5 Using base prefix '/usr' New python executable in /tmp/pack/bin/python3.5 Also creating executable in /tmp/pack/bin/python Installing setuptools, pkg_resources, pip, wheel...done. $ source bin/activate $ pip3 install packaging Collecting packaging Using cached packaging-16.8-py2.py3-none-any.whl Collecting six (from packaging) Using cached six-1.11.0-py2.py3-none-any.whl Collecting pyparsing (from packaging) Using cached pyparsing-2.2.0-py2.py3-none-any.whl Installing collected packages: six, pyparsing, packaging Successfully installed packaging-16.8 pyparsing-2.2.0 six-1.11.0 $ python3.5 Python 3.5.2 (default, Sep 14 2017, 22:51:06) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from packaging import version as pack_version >>> From christopher_reimer at icloud.com Fri Oct 27 10:14:31 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Fri, 27 Oct 2017 07:14:31 -0700 Subject: Python noob having a little trouble with strings In-Reply-To: References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> Message-ID: <7D785A87-240F-4FF8-A91F-E01F98241B83@icloud.com> On Oct 27, 2017, at 1:49 AM, Peter J. Holzer wrote: > > BTW, I find it hard to believe that PyCharm for the Mac "comes with" > Python 2.6. Python 2.6 is quite old. The Linux version isn't bundled > with a python interpreter and just uses whatever is already installed on > the machine. I guess it's the same for the Mac: Your version of MacOS > happens to include Python 2.6, so this is what PyCharm uses. I find it hard to believe that a professor would recommend downloading an IDE at the start of an intro class. Students usually start off with a text editor. Chris R. From maheshyadav1771 at gmail.com Fri Oct 27 11:35:20 2017 From: maheshyadav1771 at gmail.com (maheshyadav1771 at gmail.com) Date: Fri, 27 Oct 2017 08:35:20 -0700 (PDT) Subject: psycopg2.ProgrammingError: syntax error at or near "\" Message-ID: <3c257985-9cf2-4b72-8f26-2292bf26614c@googlegroups.com> Hello, I am using 'psycopg2' library to access my PostgreSQL database from a python. I want to pass a variable input in psql query something like this :- psql>>\set my_user table1 psql>>select * from :my_user limit 10; So I am just running these sql commands and getting this error :- >>> cur.execute("""\set my_user table1""") Traceback (most recent call last): File "", line 1, in psycopg2.ProgrammingError: syntax error at or near "\" LINE 1: \set my_user table1 Can anyone please help me out here. Thanks, Vipul From Karsten.Hilbert at gmx.net Fri Oct 27 11:52:44 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Fri, 27 Oct 2017 17:52:44 +0200 Subject: psycopg2.ProgrammingError: syntax error at or near "\" In-Reply-To: <3c257985-9cf2-4b72-8f26-2292bf26614c@googlegroups.com> References: <3c257985-9cf2-4b72-8f26-2292bf26614c@googlegroups.com> Message-ID: <20171027155244.hygiowc6ggy4j6qq@hermes.hilbert.loc> On Fri, Oct 27, 2017 at 08:35:20AM -0700, maheshyadav1771 at gmail.com wrote: > I am using 'psycopg2' library to access my PostgreSQL database from a python. I want to pass a variable input in psql query something like this :- > > psql>>\set my_user table1 > psql>>select * from :my_user limit 10; > > So I am just running these sql commands and getting this error :- > > >>> cur.execute("""\set my_user table1""") > Traceback (most recent call last): > File "", line 1, in > psycopg2.ProgrammingError: syntax error at or near "\" > LINE 1: \set my_user table1 \set is a psql specific pseudo command rather than SQL. Only SQL code can be run through psycopg2. You want to read up on query parameter handling in the psycopg2 docs. Psycopg2 and (/usr/bin/)psql(.exe) are not the same thing. Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From dev at namanbhalla.in Fri Oct 27 12:16:22 2017 From: dev at namanbhalla.in (Naman Bhalla) Date: Fri, 27 Oct 2017 21:46:22 +0530 Subject: Python noob having a little trouble with strings In-Reply-To: <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> Message-ID: <6FE8907E-C7BE-47F2-A1DE-FA5F258D5710@namanbhalla.in> I guess your professor just asked you to download Pycharm. It is just MacOS that happens to have Python 2.6 inbuilt. Had your professor actually wanted you to be using Python 2 (I doubt), that would have been 2.7. Regardless of that I recommend having latest Python 2 or 3 as per your requirements, from python.org. In Pycharm, you can access Python Console from bottom right (Make sure you select correct version of Python in your project settings.) Or in MacOS, just type python and then type commands and check for yourself. BTW, You can know more about what you are going to test here:- https://docs.python.org/3/tutorial/introduction.html#strings Regards. On 27/10/17, 08:33, "python-list-bounces+dev=namanbhalla.in at python.org on behalf of randyliu17 at gmail.com" wrote: On Thursday, October 26, 2017 at 7:41:10 PM UTC-7, boB Stepp wrote: > On Thu, Oct 26, 2017 at 9:25 PM, wrote: > > If s1 = "Welcome students", what is the output when you print the following: > > > > 1. s4 = 3 * s1 > > > > 2. s1[3 : 6] > > > > 3. 'W' in s1 > > > > 4. S1[-1] > > > > 5. S1[:-1] > > > > Any help would be great, thanks! > > Why not find out for yourself and print these in the Python > interpreter? For instance: > > > py > Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > py3: s1 = "Welcome students" > py3: s4 = 3 * s1 > py3: print(s4) > Welcome studentsWelcome studentsWelcome students > > > > -- > boB Hi Bob, thanks for responding. I'm not sure where to do so, my professor had us download Pycharm for mac's which uses python 2.6 -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Fri Oct 27 13:27:22 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Oct 2017 04:27:22 +1100 Subject: Python noob having a little trouble with strings In-Reply-To: <7D785A87-240F-4FF8-A91F-E01F98241B83@icloud.com> References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> <7D785A87-240F-4FF8-A91F-E01F98241B83@icloud.com> Message-ID: On Sat, Oct 28, 2017 at 1:14 AM, Christopher Reimer wrote: > On Oct 27, 2017, at 1:49 AM, Peter J. Holzer wrote: >> >> BTW, I find it hard to believe that PyCharm for the Mac "comes with" >> Python 2.6. Python 2.6 is quite old. The Linux version isn't bundled >> with a python interpreter and just uses whatever is already installed on >> the machine. I guess it's the same for the Mac: Your version of MacOS >> happens to include Python 2.6, so this is what PyCharm uses. > > I find it hard to believe that a professor would recommend downloading an IDE at the start of an intro class. Students usually start off with a text editor. No, I can definitely believe it. Among other advantages, getting everyone onto a cross-platform IDE will tend to paper over a lot of OS differences. You have three sets of instructions for downloading some IDE for Lin/Mac/Win, and then everything else is just "here's how you do it in ". ChrisA From dvl at psu.edu Fri Oct 27 14:05:47 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Fri, 27 Oct 2017 14:05:47 -0400 Subject: Just a quick question about main() In-Reply-To: mailman.41.1509120004.23303.python-list@python.org References: Message-ID: <1509127547l.34210040l.0l@psu.edu> While teaching my introductory course in Python, I occasionally see submissions containing the following two program lines, even before I teach about functions and modules: if __name__ = '__main__': ... main() When I ask about it, I hear things like they got these from other instructors, or from other students who learned it from their instructors, or maybe from some on-line programming tutorial site. I'm all on board with the first of these two lines -- and I teach it myself as soon as I get to modules. My question is more about the second. Do "real" Pythonista's actually define a new function main() instead of putting the unit test right there inside the if? Or am I correct in assuming that this main() is just an artifact from people who have programmed in C, C++, or Java for so long that they cannot imagine a program without a function named "main"? I guess I'm not stuck on that habit, since my programming experiences go way back to the old Fortran days.... Roger Christman Pennsylvania State University From rosuav at gmail.com Fri Oct 27 14:23:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Oct 2017 05:23:15 +1100 Subject: Just a quick question about main() In-Reply-To: <1509127547l.34210040l.0l@psu.edu> References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On Sat, Oct 28, 2017 at 5:05 AM, ROGER GRAYDON CHRISTMAN wrote: > While teaching my introductory course in Python, I occasionally see > submissions containing the following two program lines, even before > I teach about functions and modules: > > if __name__ = '__main__': > ... main() > > When I ask about it, I hear things like they got these from other instructors, > or from other students who learned it from their instructors, or maybe > from some on-line programming tutorial site. > > I'm all on board with the first of these two lines -- and I teach it myself > as soon as I get to modules. > > My question is more about the second. > > Do "real" Pythonista's actually define a new function main() instead > of putting the unit test right there inside the if? > > Or am I correct in assuming that this main() is just an artifact from > people who have programmed in C, C++, or Java for so long that > they cannot imagine a program without a function named "main"? If it's JUST for unit tests, I'd expect no main(), but instead to have it go straight into unittest.main(). IMO, the construct you show there implies three things: 1) This module is intended to be run from the command line 2) This module is intended to be imported by other modules 3) If imported by another module, this can also be invoked as if it were the top-level app. If #1 is not true, you don't need any sort of "if name is main" code, because that's not the point. If #2 is not true, you don't need to guard your main routine, because the condition will always be true. (Note that external unit tests count as From rosuav at gmail.com Fri Oct 27 14:28:06 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 28 Oct 2017 05:28:06 +1100 Subject: Just a quick question about main() In-Reply-To: References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On Sat, Oct 28, 2017 at 5:23 AM, Chris Angelico wrote: > On Sat, Oct 28, 2017 at 5:05 AM, ROGER GRAYDON CHRISTMAN wrote: >> While teaching my introductory course in Python, I occasionally see >> submissions containing the following two program lines, even before >> I teach about functions and modules: >> >> if __name__ = '__main__': >> ... main() >> >> When I ask about it, I hear things like they got these from other instructors, >> or from other students who learned it from their instructors, or maybe >> from some on-line programming tutorial site. >> >> I'm all on board with the first of these two lines -- and I teach it myself >> as soon as I get to modules. >> >> My question is more about the second. >> >> Do "real" Pythonista's actually define a new function main() instead >> of putting the unit test right there inside the if? >> >> Or am I correct in assuming that this main() is just an artifact from >> people who have programmed in C, C++, or Java for so long that >> they cannot imagine a program without a function named "main"? > > If it's JUST for unit tests, I'd expect no main(), but instead to have > it go straight into unittest.main(). IMO, the construct you show there > implies three things: > > 1) This module is intended to be run from the command line > 2) This module is intended to be imported by other modules > 3) If imported by another module, this can also be invoked as if it > were the top-level app. > > If #1 is not true, you don't need any sort of "if name is main" code, > because that's not the point. If #2 is not true, you don't need to > guard your main routine, because the condition will always be true. > (Note that external unit tests count as Whoops, premature send. External unit tests count as importing this from another module, so that satisfies that. But if #3 is not the case, then you might well have an "if name is main" block, but there's no point having "def main()". Unless you would import the module *and then call main*, don't bother having a main(). Just have whatever other code you need, right there in the 'if' block. One thing I'll often do, for instance, is to have a pure function that can be imported, and then do some simple command-line parsing: def spamify(ham, eggs, sausages): """Spamify stuff""" if __name__ == "__main__": import sys _, ham, eggs, sausages, *_ = sys.argv + ["", "", ""] print(spamify(ham, eggs, sausages)) No point having a main() to do that; the real work is in the importable function, and for command-line usage, it simply calls that function and prints the result. ChrisA From tjol at tjol.eu Fri Oct 27 14:30:31 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 27 Oct 2017 20:30:31 +0200 Subject: Just a quick question about main() In-Reply-To: <1509127547l.34210040l.0l@psu.edu> References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On 27/10/17 20:05, ROGER GRAYDON CHRISTMAN wrote: > While teaching my introductory course in Python, I occasionally see > submissions containing the following two program lines, even before > I teach about functions and modules: > > if __name__ = '__main__': > ... main() > > When I ask about it, I hear things like they got these from other instructors, > or from other students who learned it from their instructors, or maybe > from some on-line programming tutorial site. > > I'm all on board with the first of these two lines -- and I teach it myself > as soon as I get to modules. > > My question is more about the second. > > Do "real" Pythonista's actually define a new function main() instead > of putting the unit test right there inside the if? Perhaps not for unit tests, but for scripts, particularly if they come with a larger package, this is actually fairly common: it allows setuptools to generate a wrapper script that runs with the Python version and environment it was installed with. https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html Combined with the way "python -m ..." works, this leads to slightly silly-looking __main__.py modules like this: https://github.com/jupyter/jupyter_core/blob/master/jupyter_core/__main__.py > > Or am I correct in assuming that this main() is just an artifact from > people who have programmed in C, C++, or Java for so long that > they cannot imagine a program without a function named "main"? > > I guess I'm not stuck on that habit, since my programming experiences > go way back to the old Fortran days.... > > > Roger Christman > Pennsylvania State University > From ian.g.kelly at gmail.com Fri Oct 27 14:53:13 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 27 Oct 2017 12:53:13 -0600 Subject: Compression of random binary data In-Reply-To: <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> Message-ID: On Thu, Oct 26, 2017 at 8:19 PM, wrote: > It looks like that averages my two examples. I don't know how you can look at two numbers and then look at a third number that is larger than both of them and conclude it is the average. > Hmmmm by the way that equation is really cool....why does it return a high bit count when compared to >>>dec to bin? I don't understand what you're asking. The binary representation of either of those number is 16 bits, which is larger than my 15.8. From ian.g.kelly at gmail.com Fri Oct 27 14:53:32 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 27 Oct 2017 12:53:32 -0600 Subject: Compression of random binary data In-Reply-To: <2e19b3bd-ef47-445d-b238-4ceb8834189d@googlegroups.com> References: <59ed6e50$0$18636$b1db1813$d948b532@news.astraweb.com> <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> <0d422c20-cf65-494d-a749-dfa6977cc3c8@googlegroups.com> <2e19b3bd-ef47-445d-b238-4ceb8834189d@googlegroups.com> Message-ID: On Thu, Oct 26, 2017 at 8:48 PM, wrote: > Shouldn't that be? > > py> 16 * (-7/16 * math.log2(7/16) - 6/16 * math.log2(6/16)) = No, that's failing to account for 3/16 of the probability space. From grant.b.edwards at gmail.com Fri Oct 27 15:02:26 2017 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 27 Oct 2017 19:02:26 +0000 (UTC) Subject: Just a quick question about main() References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On 2017-10-27, Chris Angelico wrote: > On Sat, Oct 28, 2017 at 5:05 AM, ROGER GRAYDON CHRISTMAN wrote: >> While teaching my introductory course in Python, I occasionally see >> submissions containing the following two program lines,[...] >> if __name__ = '__main__': >> ... main() > If it's JUST for unit tests, I'd expect no main(), but instead to have > it go straight into unittest.main(). IMO, the construct you show there > implies three things: > > 1) This module is intended to be run from the command line > 2) This module is intended to be imported by other modules > 3) If imported by another module, this can also be invoked as if it > were the top-level app. I sometimes create a main function out of habit even if I can't imagine a case #3. A typical situation that I often encounter is that I write a set of functions to perform some task(s) via a serial or network connection using some industrial protocol. [For example, updating firmware in a device.] There are often two use cases: 1) It can be used from the command-line as a stand-alone application with various command line options and arguments that specify the operation[s] to be performed. 2) It can be imported by a GUI application in order to provide to the GUI framework code the functions that can be called to do the individual operations. Even if the real-world end-user use case is purely the GUI one, it's often far easier and faster to also include a main() for deveopment and testing of the functions provided to the GUI. -- Grant Edwards grant.b.edwards Yow! ONE LIFE TO LIVE for at ALL MY CHILDREN in ANOTHER gmail.com WORLD all THE DAYS OF OUR LIVES. From barry at python.org Fri Oct 27 15:36:07 2017 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Oct 2017 15:36:07 -0400 Subject: PEP Post-History Message-ID: <61E959D3-ECEB-4897-B3F6-3D04D8F52E90@python.org> We?ve made a small change to the PEP process which may affect readers of python-list and python-ideas, so I?d like to inform you of it. This change was made to PEP 1 and PEP 12. PEPs must have a Post-History header which records the dates at which the PEP is posted to mailing lists, in order to keep the general Python community in the loop as a PEP moves to final status. Until now, PEPs in development were supposed to be posted at least to python-dev and optionally to python-list[1]. This guideline predated the creation of the python-ideas mailing list. We?ve now changed this guideline so that Post-History will record the dates at which the PEP is posted to python-dev and optionally python-ideas. python-list is dropped from this requirement. python-dev is always the primary mailing list of record for Python development, and PEPs under development should be posted to python-dev as appropriate. python-ideas is the list for discussion of more speculative changes to Python, and it?s often where more complex PEPs, and even proto-PEPs are first raised and their concepts are hashed out. As such, it makes more sense to change the guideline to include python-ideas and/or python-dev. In the effort to keep the forums of record to a manageable number, python-list is dropped. If you have been watching for new PEPs to be posted to python-list, you are invited to follow either python-dev or python-ideas. Cheers, -Barry (on behalf of the Python development community) https://mail.python.org/mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-ideas Both python-dev and python-ideas are available via Gmane. [1] PEPs may have a Discussions-To header which changes the list of forums where the PEP is discussed. In that case, Post-History records the dates that the PEP is posted to those forums. See PEP 1 for details. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From rxjwg98 at gmail.com Fri Oct 27 16:35:33 2017 From: rxjwg98 at gmail.com (Robert) Date: Fri, 27 Oct 2017 13:35:33 -0700 (PDT) Subject: What use is of this 'cast=float ,'? Message-ID: Hi, I read below code snippet on line. I am interested in the second of the last line. cast=float , I've tried it in Python. Even simply with float it has no error, but what use is it? I do see a space before the comma ','. Is it a typo or not? Thanks, self.freqslider=forms.slider( parent=self.GetWin( ), sizer=freqsizer, value=self.freq, callback= self.setfreq, minimum=?samprate/2, maximum=samprate/2, num_steps=100, style=wx.SL_HORIZONTAL, cast=float , proportion=1, ) From ned at nedbatchelder.com Fri Oct 27 17:17:38 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Fri, 27 Oct 2017 17:17:38 -0400 Subject: Just a quick question about main() In-Reply-To: <1509127547l.34210040l.0l@psu.edu> References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On 10/27/17 2:05 PM, ROGER GRAYDON CHRISTMAN wrote: > While teaching my introductory course in Python, I occasionally see > submissions containing the following two program lines, even before > I teach about functions and modules: > > if __name__ = '__main__': > ... main() > > When I ask about it, I hear things like they got these from other instructors, > or from other students who learned it from their instructors, or maybe > from some on-line programming tutorial site. > > I'm all on board with the first of these two lines -- and I teach it myself > as soon as I get to modules. > > My question is more about the second. > > Do "real" Pythonista's actually define a new function main() instead > of putting the unit test right there inside the if? > > Or am I correct in assuming that this main() is just an artifact from > people who have programmed in C, C++, or Java for so long that > they cannot imagine a program without a function named "main"? > > There's no need for "if __name__ == '__main__':" for unit tests. You can let unittest or pytest discover the tests themselves, and run them. I often write this clause: ??? if __name__ == '__main__': ??????? sys.exit(main(sys.argv)) Then I can write tests that call main() to be sure it does what I think it does. Or, I can let setuptools entry_points handle that clause for me: ??? entry_points={ ??????? 'console_scripts': [ ??????????? 'coverage = coverage.cmdline:main', ??????? ], ??? }, --Ned. From edmondo.giovannozzi at gmail.com Fri Oct 27 17:32:09 2017 From: edmondo.giovannozzi at gmail.com (edmondo.giovannozzi at gmail.com) Date: Fri, 27 Oct 2017 14:32:09 -0700 (PDT) Subject: What use is of this 'cast=float ,'? In-Reply-To: References: Message-ID: <248c0e10-80b8-4e07-8648-46b59dda7890@googlegroups.com> Il giorno venerd? 27 ottobre 2017 22:35:45 UTC+2, Robert ha scritto: > Hi, > > I read below code snippet on line. I am interested in the second of the last > line. > > cast=float , > > > I've tried it in Python. Even simply with > > float > > > it has no error, but what use is it? > > > I do see a space before the comma ','. Is it a typo or not? > > > Thanks, > > > > self.freqslider=forms.slider( > parent=self.GetWin( ), > sizer=freqsizer, > value=self.freq, > callback= self.setfreq, > minimum=?samprate/2, > maximum=samprate/2, > num_steps=100, > style=wx.SL_HORIZONTAL, > cast=float , > proportion=1, > ) cast is the name of keyword argument of the function slider called "cast". It likely means that it should return a float. Quite likely inside the function "slider" there will be something like return cast(...) that if you pass float will become equivalent to return float(...) Of course I don't know that function so take it as just a likely possibility. By the way, it can be a method of an object named forms or a function in a module named forms. cheers, From python.list at tim.thechases.com Fri Oct 27 17:43:12 2017 From: python.list at tim.thechases.com (Tim Chase) Date: Fri, 27 Oct 2017 16:43:12 -0500 Subject: What use is of this 'cast=float ,'? In-Reply-To: References: Message-ID: <20171027164312.17994e5f@bigbox.christie.dr> [rearranging for easier responding] On 2017-10-27 13:35, Robert wrote: > self.freqslider=forms.slider( > parent=self.GetWin( ), > sizer=freqsizer, > value=self.freq, > callback= self.setfreq, > minimum=?samprate/2, > maximum=samprate/2, > num_steps=100, > style=wx.SL_HORIZONTAL, > cast=float , > proportion=1, > ) > I am interested in the second of the last line. > > cast=float , The space doesn't do anything. You have a parameter list, so the comma just separates "cast=float" from the next parameter, "proportion=1". The "cast=float" passes the "float()" function as a way of casting the data. In this case, it likely expects a function that takes a number or string, and returns a number that can be used to render the slider's value/position. You could create one that works backwards: def backwards_float(input): return -float(input) # note the "-" inverting the interpreted value forms.slider( ? cast=backwards_float, ? ) > I've tried it in Python. Even simply with > > float This is just the float() function. > it has no error, but what use is it? It's good for passing to something like the above that wants a function to call. The body of the function likely has something like resulting_value = cast(value) which, in this case is the same as resulting_value = float(value) > I do see a space before the comma ','. Is it a typo or not? I think it's unintended. The whole question started off peculiar because outside of a function-invocation thing = other, with the comma creates a one-element tuple and assigns the resulting tuple to "thing" >>> x = 42 >>> x 42 >>> y = 42, >>> y (42,) Usually people will be more explicit because that comma is easy to miss, so they'll write >>> z = (42,) >>> z (42,) so that later people reading the code know that it's intended to be a one-element tuple. -tkc From japy.april at gmail.com Fri Oct 27 18:11:46 2017 From: japy.april at gmail.com (japy.april at gmail.com) Date: Fri, 27 Oct 2017 15:11:46 -0700 (PDT) Subject: Speed Race between old and new version 'working with files' Message-ID: <4223b6ae-e27c-4332-9d94-e0c54628c9e9@googlegroups.com> import time avg = float(0) # copy with WITH function and execute time for i in range(500): start = time.clock() with open('q://my_projects/cricket.mp3', 'rb') as old, open('q://my_projects/new_cricket.mp3', 'wb') as new: for j in old: new.write(j) stop = time.clock() avg += (stop - start) / 500 print('Execute time with WITH version : ', avg) # copy with OLD version OPEN FILE function and execute time for i in range(500): start = time.clock() old = open('q://my_projects/cricket.mp3', 'rb') new = open('q://my_projects/old_cricket.mp3', 'wb') for j in old: new.write(j) old.close() new.close() stop = time.clock() avg += (stop - start) / 500 print('Execute time with OLD version : ', avg) From cs at cskk.id.au Fri Oct 27 18:50:21 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 28 Oct 2017 09:50:21 +1100 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: References: Message-ID: <20171027225021.GA58864@cskk.homeip.net> On 27Oct2017 15:56, David Gabriel wrote: >I am running a python code that generates for me this error : > >from packaging import version as pack_version >ImportError: No module named packaging > >I googled it and I have found so many suggestions regarding updating 'pip' >and installing python-setuptools but all of these did not fix this issue. You need to fetch the "packaging" module. That is what pip is for, eg: pip install --user packaging The --user is to install it in your "personal" python library space. Cheers, Cameron Simpson (formerly cs at zip.com.au) From steve+python at pearwood.info Fri Oct 27 20:13:25 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 28 Oct 2017 11:13:25 +1100 Subject: Speed Race between old and new version 'working with files' References: <4223b6ae-e27c-4332-9d94-e0c54628c9e9@googlegroups.com> Message-ID: <59f3cba8$0$18561$b1db1813$d948b532@news.astraweb.com> On Sat, 28 Oct 2017 09:11 am, japy.april at gmail.com wrote: > import time > > avg = float(0) That should be written as avg = 0.0 or better still not written at all, as it is pointless. > # copy with WITH function and execute time > for i in range(500): > start = time.clock() time.clock() is an old, platform-dependent, low-resolution timer. It has been deprecated in recent versions of Python. It is better to use: from timeit import default_timer as clock and use that, as the timeit module has already chosen the best timer available on your platform. In fact, you probably should be using timeit rather than re-inventing the wheel, unless you have a good reason. > with open('q://my_projects/cricket.mp3', 'rb') as old, > open('q://my_projects/new_cricket.mp3', 'wb') as new: > for j in old: > new.write(j) Reading a *binary file* line-by-line seems rather dubious to me. I wouldn't do it that way, but for now we'll just keep it. The simplest way to do this time comparison would be: oldfile = 'q://my_projects/cricket.mp3' newfile = 'q://my_projects/new_cricket.mp3' from timeit import Timer def test_with(): with open(oldfile, 'rb') as old, \ open(newfile, 'wb') as new: for line in old: new.write(line) def test_without(): old = open(oldfile, 'rb') new = open(newfile, 'wb') for line in old: new.write(line) old.close() new.close() setup = "from __main__ import test_with, test_without" t1 = Timer("test_with()", setup) t2 = Timer("test_without()", setup) print('Time using with statement:') print(min(t1.repeat(number=100, repeat=5))) print('Time not using with statement:') print(min(t2.repeat(number=100, repeat=5))) On my computer, that gives a value of about 5.3 seconds and 5.2 seconds respectively. That figure should be interpreted as: - the best value of five trials (the 'repeat=5' argument); - each trial calls the test_with/test_without function 100 times (the 'number=100' argument) so on my computer, each call to the test function takes around 5/100 seconds, or 50ms. So there's no significant speed difference between the two: using the with statement is a tiny bit slower (less than 2% on my computer). [...] > avg += (stop - start) / 500 Better to skip the pointless initialision of avg and just write: avg = (stop - start)/500 > avg += (stop - start) / 500 > print('Execute time with OLD version : ', avg) That *adds* the time of the second test to the original timing, so your last line should be: print('Execute time with NEW version plus time with OLD version : ', avg) to be accurate. But I don't think that's what you intended. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ian.g.kelly at gmail.com Fri Oct 27 23:38:18 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 27 Oct 2017 21:38:18 -0600 Subject: Just a quick question about main() In-Reply-To: <1509127547l.34210040l.0l@psu.edu> References: <1509127547l.34210040l.0l@psu.edu> Message-ID: In addition to what others have answered, if the code in question has any variables then I'll prefer to put it inside a function and call the function. This ensures that the variables are local and not going. It's a minor code hygiene point, but a good practice in my opinion. From ian.g.kelly at gmail.com Fri Oct 27 23:43:06 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 27 Oct 2017 21:43:06 -0600 Subject: Just a quick question about main() In-Reply-To: References: <1509127547l.34210040l.0l@psu.edu> Message-ID: On Oct 27, 2017 5:38 PM, "Ian Kelly" wrote: In addition to what others have answered, if the code in question has any variables then I'll prefer to put it inside a function and call the function. This ensures that the variables are local and not going. It's a minor code hygiene point, but a good practice in my opinion. s/going/global/ :P From christopher_reimer at icloud.com Sat Oct 28 02:55:28 2017 From: christopher_reimer at icloud.com (Christopher Reimer) Date: Fri, 27 Oct 2017 23:55:28 -0700 Subject: Keep or drop index.html from Django? Message-ID: Greetings, When I set up my static website using Pelican several years ago, many URLs ended with index.html. Now that I'm looking at Django, I got a small set of URLs working with and without index.html to point to the correct pages. I read somewhere that the Django philosophy was to keep the URLs as clean as possible (i.e., no *.html at the end). I can go either way with this. What's the best practice for this? Thank you, Chris R. From rustompmody at gmail.com Sat Oct 28 03:04:51 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 28 Oct 2017 00:04:51 -0700 (PDT) Subject: Ide vs ide In-Reply-To: References: Message-ID: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> On Saturday, October 28, 2017 at 11:59:14 AM UTC+5:30, Andrew Z wrote: > Yeah, lets start the war! > // joking! > > But if i think about it... there are tons articles and flame wars about "a > vs b". > And yet, what if the question should be different: > > If you were to create the "ide" for yourself (think lego) , what are the > functions that you _use_ and like a lot? [Not really an answer to your question?] But in a related direction: I think we need to talk more systematically about - programming-in-the-small: [< 70 lines ? one or so screenfuls; only 1 file] - -in-the-medium : all files in one directory - -in-the-large : multiple directories/languages/OSes etc - -huge : millions of lines; thousands of man-years I think one of the main attractions (to me but also generally to teachers) is that languages like python make programming-in-the-tiny a realistic possibility ie a couple of lines worked out possibly file-less, at the interpreter prompt. [The other day I was writing a program to split alternate lines of a file; Apart from file-handling it was these two lines: for x in lines[0::2]: print(x.strip()) for x in lines[1::2]: print(x.strip()) ] So coming to your question: IDEs are good for medium and (diminishingly) for large programs. Useful python programs are often small; even tiny From tjol at tjol.eu Sat Oct 28 05:02:52 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 28 Oct 2017 11:02:52 +0200 Subject: Keep or drop index.html from Django? In-Reply-To: References: Message-ID: <6d3a0c60-18f7-1ed2-d24f-575d70c78f52@tjol.eu> On 28/10/17 08:55, Christopher Reimer wrote: > Greetings, > > When I set up my static website using Pelican several years ago, many URLs ended with index.html. Now that I'm looking at Django, I got a small set of URLs working with and without index.html to point to the correct pages. > > I read somewhere that the Django philosophy was to keep the URLs as clean as possible (i.e., no *.html at the end). I can go either way with this. What's the best practice for this? Best practice is to keep your URLs working in one way or another. As you said, with Django it's unusual to add artificial file endings like ".html". I'd recommend setting up the URLs in a way that makes sense to you now (probably without index.html and so on) and set up HTTP 301 redirects to get your visitors to the right place. (Probably best in the web server configuration, but you can do this in Django) -- Thomas From chris at withers.org Sat Oct 28 05:04:48 2017 From: chris at withers.org (Chris Withers) Date: Sat, 28 Oct 2017 10:04:48 +0100 Subject: testfixtures 5.3.0 released! Message-ID: Hi All, I'm pleased to announce the release of testfixtures 5.3.0 featuring the following: * Add pytest traceback hiding for|TempDirectory.compare()|. * Add warnings that|log_capture()|,|tempdir()|and|replace()|are not currently compatible with pytest?s fixtures mechanism. * Better support for|stdout|or|stderr|/not/being set to|PIPE|when using|MockPopen|. * Add support to|MockPopen|for using|subprocess.Popen| as a context manager in Python 3. * Add support to|MockPopen|for|stderr=STDOUT|. Thanks to Tim Davies for his work on|MockPopen|. The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris From auriocus at gmx.de Sat Oct 28 07:15:51 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 28 Oct 2017 13:15:51 +0200 Subject: Ide vs ide In-Reply-To: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> References: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> Message-ID: Am 28.10.17 um 09:04 schrieb Rustom Mody: > [The other day I was writing a program to split alternate lines of a file; > Apart from file-handling it was these two lines: > > for x in lines[0::2]: print(x.strip()) > for x in lines[1::2]: print(x.strip()) > ] ...and using the best(TM) tool for it, it is a one-liner: gawk '{ print > "split" NR%2}' input.txt > > So coming to your question: IDEs are good for medium and (diminishingly) for large programs. agreed > Useful python programs are often small; even tiny > Christian From rustompmody at gmail.com Sat Oct 28 07:39:54 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 28 Oct 2017 04:39:54 -0700 (PDT) Subject: Ide vs ide In-Reply-To: References: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> Message-ID: <6b95bedc-a8f8-4be9-a8f7-4d68aec14152@googlegroups.com> On Saturday, October 28, 2017 at 4:46:03 PM UTC+5:30, Christian Gollwitzer wrote: > Am 28.10.17 um 09:04 schrieb Rustom Mody: > > [The other day I was writing a program to split alternate lines of a file; > > Apart from file-handling it was these two lines: > > > > for x in lines[0::2]: print(x.strip()) > > for x in lines[1::2]: print(x.strip()) > > ] > > ...and using the best(TM) tool for it, it is a one-liner: > > gawk '{ print > "split" NR%2}' input.txt Ooooooo!?! From vinay_sajip at yahoo.co.uk Sat Oct 28 08:04:21 2017 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 28 Oct 2017 12:04:21 +0000 (UTC) Subject: ANN: distlib 0.2.6 released on PyPI References: <462443690.8909932.1509192261626.ref@mail.yahoo.com> Message-ID: <462443690.8909932.1509192261626@mail.yahoo.com> I've just released version 0.2.6 of distlib on PyPI [1]. For newcomers,distlib is a library of packaging functionality which is intended to beusable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #99: Updated to handle a case where sys.getfilesystemencoding()? returns None. * Fixed #97: Eliminated a crash in EggInfoDistribution.list_distinfo_files()? which was caused by trying to open a non-existent file. * Fixed #96: SimpleScrapingLocator no longer fails prematurely when scraping? links due to invalid versions. * Improved error messages issued when interpreting markers. * Improved the shebangs written into installed scripts when the interpreter? path is very long or contains spaces (to cater for a limitation in shebang? line parsing on Linux). * Updated launcher binaries. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions forimprovements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.6[2] https://goo.gl/M3kQzR[3] https://bitbucket.org/pypa/distlib/issues/new From davidgab283 at gmail.com Sat Oct 28 08:31:03 2017 From: davidgab283 at gmail.com (David Gabriel) Date: Sat, 28 Oct 2017 14:31:03 +0200 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: <20171027140819.GD8014@lutz-pc.ecm4u.intra> References: <20171027140819.GD8014@lutz-pc.ecm4u.intra> Message-ID: Thanks so Lutz much for your reply. I am using python2.7 and I am running this code in an Openstack instance. I will apply your recommandation and let you know about the result ... Kind regards. 2017-10-27 16:13 GMT+02:00 Lutz Horn : > On Fri, Oct 27, 2017 at 03:56:39PM +0200, David Gabriel wrote: > > from packaging import version as pack_version > > ImportError: No module named packaging > > > > I googled it and I have found so many suggestions regarding updating > > 'pip' and installing python-setuptools but all of these did not fix > > this issue. > > So many questions: > > * What is your Python version? > * Do you use virtualenv? > * How? > * Did you install packaging in this virtualenv? > > Just one example of making this work: > > $ mkdir /tmp/pack > $ cd /tmp/pack > $ virtualenv -p $(which python3.5) . > Running virtualenv with interpreter /usr/bin/python3.5 > Using base prefix '/usr' > New python executable in /tmp/pack/bin/python3.5 > Also creating executable in /tmp/pack/bin/python > Installing setuptools, pkg_resources, pip, wheel...done. > $ source bin/activate > $ pip3 install packaging > Collecting packaging > Using cached packaging-16.8-py2.py3-none-any.whl > Collecting six (from packaging) > Using cached six-1.11.0-py2.py3-none-any.whl > Collecting pyparsing (from packaging) > Using cached pyparsing-2.2.0-py2.py3-none-any.whl > Installing collected packages: six, pyparsing, packaging > Successfully installed packaging-16.8 pyparsing-2.2.0 six-1.11.0 > $ python3.5 > Python 3.5.2 (default, Sep 14 2017, 22:51:06) > [GCC 5.4.0 20160609] on linux > Type "help", "copyright", "credits" or "license" for more > information. > >>> from packaging import version as pack_version > >>> > -- > https://mail.python.org/mailman/listinfo/python-list > From davidgab283 at gmail.com Sat Oct 28 08:33:00 2017 From: davidgab283 at gmail.com (David Gabriel) Date: Sat, 28 Oct 2017 14:33:00 +0200 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: References: <20171027140819.GD8014@lutz-pc.ecm4u.intra> Message-ID: I forget to precise that I am using pycharm. And this issue is reproducible also using command line to run the code. Best regards 2017-10-28 14:31 GMT+02:00 David Gabriel : > Thanks so Lutz much for your reply. > I am using python2.7 and I am running this code in an Openstack instance. > I will apply your recommandation and let you know about the result ... > > Kind regards. > > 2017-10-27 16:13 GMT+02:00 Lutz Horn : > >> On Fri, Oct 27, 2017 at 03:56:39PM +0200, David Gabriel wrote: >> > from packaging import version as pack_version >> > ImportError: No module named packaging >> > >> > I googled it and I have found so many suggestions regarding updating >> > 'pip' and installing python-setuptools but all of these did not fix >> > this issue. >> >> So many questions: >> >> * What is your Python version? >> * Do you use virtualenv? >> * How? >> * Did you install packaging in this virtualenv? >> >> Just one example of making this work: >> >> $ mkdir /tmp/pack >> $ cd /tmp/pack >> $ virtualenv -p $(which python3.5) . >> Running virtualenv with interpreter /usr/bin/python3.5 >> Using base prefix '/usr' >> New python executable in /tmp/pack/bin/python3.5 >> Also creating executable in /tmp/pack/bin/python >> Installing setuptools, pkg_resources, pip, wheel...done. >> $ source bin/activate >> $ pip3 install packaging >> Collecting packaging >> Using cached packaging-16.8-py2.py3-none-any.whl >> Collecting six (from packaging) >> Using cached six-1.11.0-py2.py3-none-any.whl >> Collecting pyparsing (from packaging) >> Using cached pyparsing-2.2.0-py2.py3-none-any.whl >> Installing collected packages: six, pyparsing, packaging >> Successfully installed packaging-16.8 pyparsing-2.2.0 six-1.11.0 >> $ python3.5 >> Python 3.5.2 (default, Sep 14 2017, 22:51:06) >> [GCC 5.4.0 20160609] on linux >> Type "help", "copyright", "credits" or "license" for more >> information. >> >>> from packaging import version as pack_version >> >>> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > From steve+python at pearwood.info Sat Oct 28 10:04:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 29 Oct 2017 01:04:01 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> Message-ID: <59f48e52$0$18634$b1db1813$d948b532@news.astraweb.com> On Fri, 27 Oct 2017 09:53 am, Ben Bacarisse wrote: > A source of random can be defined but "random data" is much more > illusive. Random data = any set of data generated by "a source of random". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From mal at europython.eu Sat Oct 28 10:47:50 2017 From: mal at europython.eu (M.-A. Lemburg) Date: Sat, 28 Oct 2017 16:47:50 +0200 Subject: EuroPython 2017: Videos for Friday available online Message-ID: We are pleased to announce the last batch of cut videos for EuroPython 2017. * All 163 EuroPython 2017 videos are now online * To see the new videos, please head over to our EuroPython YouTube channel and select the "EuroPython 2017" playlist. The new videos start at entry 129 in the playlist. We have also published a few videos from Monday, we had missed in the first batch. * EuroPython 2017 Videos * http://europython.tv/ Enjoy, -- EuroPython 2017 Team http://ep2017.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/924283342291431424 Thanks. From wrw at mac.com Sat Oct 28 11:08:37 2017 From: wrw at mac.com (William Ray Wing) Date: Sat, 28 Oct 2017 11:08:37 -0400 Subject: Python noob having a little trouble with strings In-Reply-To: <20171027064801.GA8014@lutz-pc.ecm4u.intra> References: <878d01db-0ac9-456f-923e-8c13ce695cd8@googlegroups.com> <9f11e2a9-7e20-494a-9e05-4594766ff443@googlegroups.com> <20171027064801.GA8014@lutz-pc.ecm4u.intra> Message-ID: <3106413B-6E43-4BBA-85B5-1B7670ECBF12@mac.com> OSX has been shipping with Python 2.7 for several years. I?m not sure why you are seeing 2.6. Bill > On Oct 27, 2017, at 2:48 AM, Lutz Horn wrote: > > On Thu, Oct 26, 2017 at 07:59:10PM -0700, randyliu17 at gmail.com wrote: >> Hi Bob, thanks for responding. I'm not sure where to do so, my >> professor had us download Pycharm for mac's which uses python 2.6 > > The code from your question is not specific to Python 2 or 3. Just try > it in the Python installation you have available. > > Lutz > -- > https://mail.python.org/mailman/listinfo/python-list From ben.usenet at bsb.me.uk Sat Oct 28 11:18:57 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sat, 28 Oct 2017 16:18:57 +0100 Subject: Compression of random binary data References: <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> <59f48e52$0$18634$b1db1813$d948b532@news.astraweb.com> Message-ID: <87fua3gv1a.fsf@bsb.me.uk> Steve D'Aprano writes: > On Fri, 27 Oct 2017 09:53 am, Ben Bacarisse wrote: > >> A source of random can be defined but "random data" is much more >> illusive. > > Random data = any set of data generated by "a source of random". (I had an editing error there; it should be "a source of random data".) Yes, that's a fine definition, but it has the disadvantage of not being a verifiable property of the thing defined -- you can't know, from the data themselves, if they constitute random data. You would not care about a compression program that worked on some data that looks random, you'd want to present your own data for compression (and then you can use a random source with confidence because the data are yours). That's the big win (for me) of talking about "arbitrary data". -- Ben. From formisc at gmail.com Sat Oct 28 12:11:03 2017 From: formisc at gmail.com (Andrew Z) Date: Sat, 28 Oct 2017 12:11:03 -0400 Subject: Ide vs ide In-Reply-To: <6b95bedc-a8f8-4be9-a8f7-4d68aec14152@googlegroups.com> References: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> <6b95bedc-a8f8-4be9-a8f7-4d68aec14152@googlegroups.com> Message-ID: I like this trajectory of conversation. Can we re define "small tiny" as "scripts"? i can argue, based on my expirience with other languages, that there is no need for an "ide". The most ive ever needed is a text editor and a few plugins with "print". Moving to "average" size projects. What i found useful and what i use constantly: Built in, up to date, correct! help on _installed_classes, functions. The golden standard all these years for me was Delphi (2,3). Drop downs to show list of functions for an object. Also validates your logic as far as you use of correct variable, type etc. Templates- define a language construct, Type a shortcut and get the whole construct inserted. Saves typing. Built-in change control, build. This is the main functionality i use . Needless to say the ide itself has to be professional. And, since im not doing coding 24x7, i prefer "buttons" as opposite to shortcut-to-find-the-shortcut-so-i-can-shortcut to get the action. (Yeah, i found myself far less productive while using atom and company).at the same time, using editors like atom, whenyou work with multiple filetypes is a gods sent . On Oct 28, 2017 07:40, "Rustom Mody" wrote: > On Saturday, October 28, 2017 at 4:46:03 PM UTC+5:30, Christian Gollwitzer > wrote: > > Am 28.10.17 um 09:04 schrieb Rustom Mody: > > > [The other day I was writing a program to split alternate lines of a > file; > > > Apart from file-handling it was these two lines: > > > > > > for x in lines[0::2]: print(x.strip()) > > > for x in lines[1::2]: print(x.strip()) > > > ] > > > > ...and using the best(TM) tool for it, it is a one-liner: > > > > gawk '{ print > "split" NR%2}' input.txt > > Ooooooo!?! > -- > https://mail.python.org/mailman/listinfo/python-list > From elchino at cnn.cn Sat Oct 28 13:24:47 2017 From: elchino at cnn.cn (ElChino) Date: Sat, 28 Oct 2017 19:24:47 +0200 Subject: sys.path[] question Message-ID: From the Python2.7 snippet in [1], Python2.7 reports that my sys.path[] contains: f:\ProgramFiler\Python27\lib\site-packages\pyreadline-2.0-py2.7-win32.egg (a .zip-file) But I have also a 'f:\ProgramFiler\Python27\lib\site-packages\pyreadline' directory. With the complete package AFAICS. So my question is where will Python import a 'import pyreadline' from? E.f. if I do a: python -vv f:\ProgramFiler\Python27\lib\site-packages\pyreadline\test\test_lineeditor.py it shows a: # zipimport: found 89 names in F:\ProgramFiler\Python27\lib\site-packages\pyreadline-2.0-py2.7-win32.egg But no other references to pyreadline-2.0-py2.7-win32.egg. Does that mean that only this .egg is where python imports all pyreadline files from? And if so, can I delete 'f:\ProgramFiler\Python27\lib\site-packages\pyreadline' ? ------ [1]: in a py-path.bat file: f:\ProgramFiler\Python27\python.exe -c "import sys; [sys.stdout.write('%%2d: %%s\n' %% (i,p)) for (i,p) in enumerate(sys.path)]" From rantingrickjohnson at gmail.com Sat Oct 28 13:47:49 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sat, 28 Oct 2017 10:47:49 -0700 (PDT) Subject: What use is of this 'cast=float ,'? In-Reply-To: References: Message-ID: <3d5b09da-6f95-4b42-85b9-2103873181d1@googlegroups.com> On Friday, October 27, 2017 at 3:35:45 PM UTC-5, Robert wrote: > I read below code snippet on line. I am interested in the > second of the last line: `cast=float`. I've tried it in > Python. Even simply with: `float` It has no error, but what > use is it? > > self.freqslider=forms.slider( > parent=self.GetWin( ), > sizer=freqsizer, > value=self.freq, > callback= self.setfreq, > minimum=?samprate/2, > maximum=samprate/2, > num_steps=100, > style=wx.SL_HORIZONTAL, > cast=float , > proportion=1, > ) This is a fine example of the distinctly pungent form of code-smell otherwise known as: "unintuitive naming convention". A more informative keyword argument would have been something like "castFunc" or "castTo" or even "returnType", any of which would indicate that the argument is (depending on the specific implementation of course) expected to be a function object or that some specific type of "casting action" is to be performed on the return value. Whereas, using the ambiguous word "cast" leaves too much to the imagination. And while my crazy-uncle-who-lives-in-the- basement swears that lingerie magazines are better than the nudie mags (because, in the case of the former, more is reserved for the imagination), an API that requires imagination to grokk is a poorly designed API. Here is an interactive example of assigning a builtin to a generic, allbeit very _intuitive_, symbol. >>> value = "12.3" >>> type(value) >>> castFunc = float >>> valueAsFloat = castFunc(value) >>> valueAsFloat 12.3 >>> type(valueAsFloat) >>> value = "12.3" # # And now for integers... # >>> castfunc = int >>> valueAsInteger = castfunc(value) Traceback (most recent call last): File "", line 1, in valueAsInteger = castfunc(value) ValueError: invalid literal for int() with base 10: '12.3' Oops! O:-) Well, don't blame me because Python is so hobbled that it cannot even convert a string-ified version of a float into an integer. *SHRUGS* From stefanossofroniou542 at gmail.com Sat Oct 28 14:42:18 2017 From: stefanossofroniou542 at gmail.com (=?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?=) Date: Sat, 28 Oct 2017 11:42:18 -0700 (PDT) Subject: Coding style in CPython implementation Message-ID: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Greetings everyone. I have noticed that in many if conditions the following syntax is used: a) if (variable == NULL) { ... } b) if (variable == -1) { ... } c) if (variable != NULL) { ... } What I wanted to ask is, is there a particular reason for not choosing a) if (!variable) { ... } in place of if (variable == NULL) { ... }, b) if (-1 == variable) { ... } in place of if (variable == -1) { ... }, and c) if (variable) { ... } in place of if (variable) { ... } ? Especially the (b) syntax is extremely dangerous to assign -1 to variable in case of an accidental mistyping of equals sign; it had happened countless times by now to to many of us that use various C-family languages. Is there a particular reason for using this specific coding style? Regards, Stefanos From bc at freeuk.com Sat Oct 28 14:54:38 2017 From: bc at freeuk.com (bartc) Date: Sat, 28 Oct 2017 19:54:38 +0100 Subject: Coding style in CPython implementation In-Reply-To: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Message-ID: On 28/10/2017 19:42, ???????? ????????? wrote: > Greetings everyone. > > I have noticed that in many if conditions the following syntax is used: > > a) if (variable == NULL) { ... } > b) if (variable == -1) { ... } > c) if (variable != NULL) { ... } > > What I wanted to ask is, is there a particular reason for not choosing > > a) if (!variable) { ... } in place of if (variable == NULL) { ... }, > b) if (-1 == variable) { ... } in place of if (variable == -1) { ... }, and > c) if (variable) { ... } in place of if (variable) { ... } ? (Presumably you meant variable != NULL) > Especially the (b) syntax is extremely dangerous to assign -1 to variable in case of an accidental mistyping of equals sign; it had happened countless times by now to to many of us that use various C-family languages. > > Is there a particular reason for using this specific coding style? Which one do you think is more readable? Which style would you prefer to read? Or do you prefer the more cryptic style in open source code? (The = vs. == issue is of one of numerous design flaws in C, but uglifying source code to get around it isn't the answer.) -- bartc From rosuav at gmail.com Sat Oct 28 15:17:07 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 06:17:07 +1100 Subject: Coding style in CPython implementation In-Reply-To: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 5:42 AM, ???????? ????????? wrote: > Greetings everyone. > > I have noticed that in many if conditions the following syntax is used: > > a) if (variable == NULL) { ... } > b) if (variable == -1) { ... } > c) if (variable != NULL) { ... } > > What I wanted to ask is, is there a particular reason for not choosing > > a) if (!variable) { ... } in place of if (variable == NULL) { ... }, > b) if (-1 == variable) { ... } in place of if (variable == -1) { ... }, and > c) if (variable) { ... } in place of if (variable) { ... } ? > > Especially the (b) syntax is extremely dangerous to assign -1 to variable in case of an accidental mistyping of equals sign; it had happened countless times by now to to many of us that use various C-family languages. > > Is there a particular reason for using this specific coding style? Have you read PEP 7? https://www.python.org/dev/peps/pep-0007/ PEP 7 and PEP 8 are a pair of style guides that govern the CPython source code - PEP 7 for the C code, and PEP 8 for the Python code in the standard library. Unfortunately, many people seem to think that PEP 8 is supposed to govern *their* code, and as such, end up not even realizing that PEP 7 exists to answer all the same sorts of questions. ChrisA From rosuav at gmail.com Sat Oct 28 15:22:32 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 06:22:32 +1100 Subject: Ide vs ide In-Reply-To: References: <3c11c983-573b-4d25-a876-b7f5936a812d@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 6:10 AM, Stefan Ram wrote: > Rustom Mody writes: >>Useful python programs are often small; even tiny > > We must not forget that tiny programs are just large > problems with the size masterfully hidden. > > For example, the ?print? of Python is actually implemented > by a subprogram (in C or Python) which invokes other subprograms. > > If we would add the listing to all subprograms invoked > during the execution of a small Python script such as > > print( 2 ) > > , it would become much larger. > > So this serves to illustrate > > "Fools ignore complexity. > Pragmatists suffer it. > Some can avoid it. > Geniuses remove it." At which point you're just arguing levels of abstraction. If someone comes to me and says, hey look, I can do all this numerical computation and analysis in one line of simple code, I won't argue that it's not really "one line" just because it depends on Pandas - I'll agree with him/her on the basis that Pandas grants you an extremely high level of abstraction for data manipulation. That print function does a lot of work, yes, but logically it does one thing: it prints stuff out. Tiny Python programs are still tiny in terms of elegance and readability. And the CPython interpreter has over four hundred separate files full of unit tests to help ensure that the lower abstraction layers are correctly representing the intent of the higher layers. ChrisA From pkpearson at nowhere.invalid Sat Oct 28 16:03:45 2017 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 28 Oct 2017 20:03:45 GMT Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Thu, 26 Oct 2017 19:26:11 -0600, Ian Kelly wrote: > > . . . Shannon entropy is correctly calculated for a data source, > not an individual message . . . Thank you; I was about to make the same observation. When people talk about the entropy of a particular message, you can bet they're headed for confusion. -- To email me, substitute nowhere->runbox, invalid->com. From ned at nedbatchelder.com Sat Oct 28 16:06:49 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 28 Oct 2017 16:06:49 -0400 Subject: Coding style in CPython implementation In-Reply-To: References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Message-ID: <707c8d54-43de-6295-fd85-dc4f9a533525@nedbatchelder.com> On 10/28/17 3:00 PM, Stefan Ram wrote: > =?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?= writes: >> What I wanted to ask is, is there a particular reason for not choosing > I am not a CPython developer, but here are my 2 cents about > the possibilities: > >> if (variable == NULL) { ... } >> if (!variable) { ... } > ?!variable? is clearer, because it does not depend on the > definition of ?NULL?. ?NULL? is not part of the C language proper. Especially if NULL is not part of the standard, then you need to write "variable == NULL", since "!variable" may not have the same effect after "return NULL;" depending on how NULL is defined. > > (I sometimes like to write ?if( variable ); else { ... }?.) Ick. --Ned. From martin.schoon at gmail.com Sat Oct 28 16:41:34 2017 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 28 Oct 2017 20:41:34 GMT Subject: Repairing Python installation? Message-ID: It seems something is amiss with my Python 2.7 installation. Revisiting Nikola (static web site generator written in Python) for the first time in several years the other day I experience some unexpected problems. I got some help form the Nikola people and the conclusion is something is broken with my Python 2.7. Pip list throws exceptions at me while pip3 list works the way I expect it to do. All this is happening on a Debian machine. Questions: Should I try to pinpoint what is broken (and how is that done) or should I just remove everything Python 2.7 and re-install? Could mixing pip installs with Debian distro installs of Python packages lead to conflicts or other problems? TIA, /Martin From ned at nedbatchelder.com Sat Oct 28 16:48:13 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 28 Oct 2017 16:48:13 -0400 Subject: Coding style in CPython implementation In-Reply-To: References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> <707c8d54-43de-6295-fd85-dc4f9a533525@nedbatchelder.com> Message-ID: <7ddfa7b2-7831-06a1-d59c-9e313f3b791b@nedbatchelder.com> On 10/28/17 4:26 PM, Stefan Ram wrote: > Ned Batchelder writes: >> On 10/28/17 3:00 PM, Stefan Ram wrote: >>> =?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?= writes: >>>> What I wanted to ask is, is there a particular reason for not choosing >>> definition of ?NULL?. ??NULL?? is not part of the C language proper. >> Especially if NULL is not part of the standard, > ?NULL? is defined in the standard, but it is an identifier > from the library and, therefore, needs an ?#include?. > > The language proper is the language core without any > library added. > > The expression ?NULL? AFAIK has only the meaning that > usually is intended if one of the headers needed for > it has been ?#include?d. > > The operator ?!? and the if-statement do not require > any ?#include?. > I certainly wouldn't adjust my coding style to avoid #include'ing the definition of NULL. --Ned. From Karsten.Hilbert at gmx.net Sat Oct 28 16:51:06 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sat, 28 Oct 2017 22:51:06 +0200 Subject: Repairing Python installation? In-Reply-To: References: Message-ID: <20171028205106.md5act7txnqvlfua@hermes.hilbert.loc> On Sat, Oct 28, 2017 at 08:41:34PM +0000, Martin Sch??n wrote: > It seems something is amiss with my Python 2.7 installation. Revisiting > Nikola (static web site generator written in Python) for the first time > in several years the other day I experience some unexpected problems. I > got some help form the Nikola people and the conclusion is something > is broken with my Python 2.7. Pip list throws exceptions at me ... > All this is happening on a Debian machine. Exceptions or crashes of the python interpreter ? Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From piet-l at vanoostrum.org Sat Oct 28 17:56:02 2017 From: piet-l at vanoostrum.org (Piet van Oostrum) Date: Sat, 28 Oct 2017 23:56:02 +0200 Subject: Problem with subprocess.Popen and EINTR Message-ID: I am using Python 2.7.14 on MacOS Sierra. I have a small Python program that calls a shell script in a loop with a time.sleep() in it. The shell script is called with subprocess.Popen(), followed by a subprocess.wait(). No information is exchanged with the shell script. Once in a while I send a SIGINT to the program to cancel the sleep. I don't know if the sleep is active at the time of the SIGINT, but most of the time it will be as it takes much more time than the shell script. I do this by having a handler for the SIGINT which does nothing. It is just there to have the SIGINT silently cancel the sleep. Now after weeks of running correctly it crashed. It appears that the SIGINT came during the Popen. This causes a system call in the Popen to be terminated with EINTR and Popen then retries that system call, but unfortunately it fails. It seems to me that this could be a bug. Here is the traceback: Traceback (most recent call last): File "/Users/piet/bin/smoveinloop.py", line 86, in main() File "/Users/piet/bin/smoveinloop.py", line 65, in main task = subprocess.Popen([TASK], executable=TASK) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 390, in __init__ errread, errwrite) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1001, in _execute_child data = _eintr_retry_call(os.read, errpipe_read, 1048576) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 121, in _eintr_retry_call return func(*args) OSError: [Errno 22] Invalid argument TASK is a string with the name of the shell script. Can anybody shed some light on this? -- Piet van Oostrum WWW: http://piet.vanoostrum.org/ PGP key: [8DAE142BE17999C4] From rosuav at gmail.com Sat Oct 28 18:17:37 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 09:17:37 +1100 Subject: Problem with subprocess.Popen and EINTR In-Reply-To: References: Message-ID: On Sun, Oct 29, 2017 at 8:56 AM, Piet van Oostrum wrote: > I am using Python 2.7.14 on MacOS Sierra. > > I have a small Python program that calls a shell script in a loop with a time.sleep() in it. > The shell script is called with subprocess.Popen(), followed by a subprocess.wait(). > No information is exchanged with the shell script. > > Once in a while I send a SIGINT to the program to cancel the sleep. I don't know if the sleep is active at the time of the SIGINT, but most of the time it will be as it takes much more time than the shell script. > > I do this by having a handler for the SIGINT which does nothing. It is just there to have the SIGINT silently cancel the sleep. > > Now after weeks of running correctly it crashed. It appears that the SIGINT came during the Popen. This causes a system call in the Popen to be terminated with EINTR and Popen then retries that system call, but unfortunately it fails. It seems to me that this could be a bug. That definitely could be a bug. The behaviour of signals and retries of system calls got changed more recently, but Python 2.7 is the oldstable version and didn't get that sort of change. My advice to you is simply live with it; the probability of it recurring is low (if, as you say, the sleep takes the bulk of the time), unless you can migrate to Python 3, to take advantage of the new behaviour. ChrisA From cs at cskk.id.au Sat Oct 28 19:11:04 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 29 Oct 2017 10:11:04 +1100 Subject: Problem with subprocess.Popen and EINTR In-Reply-To: References: Message-ID: <20171028231104.GA32394@cskk.homeip.net> On 28Oct2017 23:56, Piet van Oostrum wrote: >I am using Python 2.7.14 on MacOS Sierra. > >I have a small Python program that calls a shell script in a loop with a time.sleep() in it. >The shell script is called with subprocess.Popen(), followed by a subprocess.wait(). >No information is exchanged with the shell script. > >Once in a while I send a SIGINT to the program to cancel the sleep. I don't know if the sleep is active at the time of the SIGINT, but most of the time it will be as it takes much more time than the shell script. > >I do this by having a handler for the SIGINT which does nothing. It is just there to have the SIGINT silently cancel the sleep. > >Now after weeks of running correctly it crashed. It appears that the SIGINT came during the Popen. This causes a system call in the Popen to be terminated with EINTR and Popen then retries that system call, but unfortunately it fails. It seems to me that this could be a bug. It may be a bug. Or it may be a system call which cannot be meaningfulling retried. But had you considered only activating the handler around the sleep? You still need to copy with SIGINT single I infer that you send this from outside the program. What if you did this: os.signal(SIGINT, SIG_IGN) ... code code code, including the Popen/wait ... old_handler = os.signal(SIGINT, do_nothing_handler) sleep(...) os.signal(SIGINT, old_handler) SIG_IGN is different from a do-nothing handler; it prevents the process seeing the signal at all, and therefore prevents the signal interrupting system calls. So you activate your dummy-quit-sleeping function only around the sleep itself. Cheers, Cameron Simpson (formerly cs at zip.com.au) From cs at cskk.id.au Sat Oct 28 19:16:33 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 29 Oct 2017 10:16:33 +1100 Subject: Problem with subprocess.Popen and EINTR In-Reply-To: <20171028231104.GA32394@cskk.homeip.net> References: <20171028231104.GA32394@cskk.homeip.net> Message-ID: <20171028231633.GA49003@cskk.homeip.net> On 29Oct2017 10:11, Cameron Simpson wrote: >It may be a bug. Or it may be a system call which cannot be meaningfulling >retried. But had you considered only activating the handler around the sleep? >You still need to copy with SIGINT single I infer that you send this from >outside the program. I need more coffee. Or less. "meaningfully". "copy" ==> "cope". "single" => "since". Argh, Cameron Simpson (formerly cs at zip.com.au) From stefanossofroniou542 at gmail.com Sat Oct 28 19:20:54 2017 From: stefanossofroniou542 at gmail.com (=?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?=) Date: Sat, 28 Oct 2017 16:20:54 -0700 (PDT) Subject: Coding style in CPython implementation In-Reply-To: References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Message-ID: <5ecdce2b-f347-4205-9ba8-8a4f12cdfb88@googlegroups.com> On Saturday, October 28, 2017 at 9:54:30 PM UTC+3, bartc wrote: > On 28/10/2017 19:42, ???????? ????????? wrote: > > Greetings everyone. > > > > I have noticed that in many if conditions the following syntax is used: > > > > a) if (variable == NULL) { ... } > > b) if (variable == -1) { ... } > > c) if (variable != NULL) { ... } > > > > What I wanted to ask is, is there a particular reason for not choosing > > > > a) if (!variable) { ... } in place of if (variable == NULL) { ... }, > > b) if (-1 == variable) { ... } in place of if (variable == -1) { ... }, and > > c) if (variable) { ... } in place of if (variable) { ... } ? > > (Presumably you meant variable != NULL) Yep, I forgot to show the != NULL part; good catch! > > > Especially the (b) syntax is extremely dangerous to assign -1 to variable in case of an accidental mistyping of equals sign; it had happened countless times by now to to many of us that use various C-family languages. > > > > Is there a particular reason for using this specific coding style? > > Which one do you think is more readable? > > Which style would you prefer to read? > > Or do you prefer the more cryptic style in open source code? > > > (The = vs. == issue is of one of numerous design flaws in C, but > uglifying source code to get around it isn't the answer.) I understand what you are saying and I don't have a styling preference, at least in CPython's case. I do believe though that if (!d) is a lot clearer than if (d == NULL) as it is safer than falsely assigning NULL in d, by pure mistake. But indeed it's a matter of taste. I just wanted to see whether there is a certain favor around such coding style, that's all. > > > > -- > bartc From pjh at nanoworks.com Sat Oct 28 19:59:55 2017 From: pjh at nanoworks.com (Percival John Hackworth) Date: Sat, 28 Oct 2017 16:59:55 -0700 Subject: Repairing Python installation? References: Message-ID: <0001HW.1FA5517B012AA9B815A74F2CF@news.individual.net> On 28-Oct-2017, Martin Sch??n wrote (in article ): > It seems something is amiss with my Python 2.7 installation. Revisiting > Nikola (static web site generator written in Python) for the first time > in several years the other day I experience some unexpected problems. I > got some help form the Nikola people and the conclusion is something > is broken with my Python 2.7. Pip list throws exceptions at me while > pip3 list works the way I expect it to do. > > All this is happening on a Debian machine. > > Questions: > > Should I try to pinpoint what is broken (and how is that done) or should > I just remove everything Python 2.7 and re-install? > > Could mixing pip installs with Debian distro installs of Python > packages lead to conflicts or other problems? > > TIA, > > /Martin If this site is accessible from the internet, have you looked around other parts of the system? Like date/timestamps for /bin/*. I recently was asked to look at a friends web site and found they were doing several things that I consider a security no-no, like running the tomcat server as root to serve their web site. It had been root-kitted. I helped them setup a new system that ran tomcat under a non-priv user. It's just a SWAG, but perhaps something further is amiss that you don't see. I was really surprised that my friend, a pretty good programmer, was clueless about such things. The only other time I've seen python "damaged" was when some developer decided they wanted a more current version than the default installed on the CentOS system (2.6) which is required for updating the system. They borked it beyond repair and my boss took away root access after we repaired it. From steve+python at pearwood.info Sat Oct 28 20:32:52 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 29 Oct 2017 11:32:52 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: <59f521b7$0$18614$b1db1813$d948b532@news.astraweb.com> On Sun, 29 Oct 2017 07:03 am, Peter Pearson wrote: > On Thu, 26 Oct 2017 19:26:11 -0600, Ian Kelly wrote: >> >> . . . Shannon entropy is correctly calculated for a data source, >> not an individual message . . . > > Thank you; I was about to make the same observation. When > people talk about the entropy of a particular message, you > can bet they're headed for confusion. I don't think that's right. The entropy of a single message is a well-defined quantity, formally called the self-information. The entropy of a data source is the expected value of the self-information of all possible messages coming from that data source. https://en.wikipedia.org/wiki/Self-information We can consider the entropy of a data source as analogous to the population mean, and the entropy of a single message as the sample mean. A single message is like a sample taken from the set of all possible messages. Self-information is sometimes also called "surprisal" as it is a measure of how surprising an event is. If your data source is a coin toss, then actually receiving a Heads has a self-information ("entropy") of 1 bit. That's not very surprising. If your data source is a pair of fair, independent dice, then the self-information of receiving a two and a four is 5.170 bits. Its a logarithmic scale, not linear: if the probability of a message or event is p, the self-information of that event is log_2 (1/p) bits. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From akishorecert at gmail.com Sat Oct 28 21:27:53 2017 From: akishorecert at gmail.com (Kishore Kumar Alajangi) Date: Sun, 29 Oct 2017 06:57:53 +0530 Subject: Listing link urls Message-ID: Hi, I am facing an issue with listing specific urls inside web page, https://economictimes.indiatimes.com/archive.cms Page contains link urls by year and month vise, Ex: /archive/year-2001,month-1.cms I am able to list all required urls using the below code, from bs4 import BeautifulSoup import re, csv import urllib.request import scrapy req = urllib.request.Request('http://economictimes.indiatimes.com/archive.cms', headers={'User-Agent': 'Mozilla/5.0'}) links = [] totalPosts = [] url = "http://economictimes.indiatimes.com" data = urllib.request.urlopen(req).read() page = BeautifulSoup(data,'html.parser') for link in page.findAll('a', href = re.compile('^/archive/')): //retrieving urls starts with "archive" l = link.get('href') links.append(url+l) with open("output.txt", "a") as f: for post in links: post = post + '\n' f.write(post) *sample result in text file:* http://economictimes.indiatimes.com/archive/year-2001,month-1.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-2.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-3.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-4.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-5.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-6.cms List of urls I am storing in a text file, From the month urls I want to retrieve day urls starts with "/archivelist", I am using the below code, but I am not getting any result, If I check with inspect element the urls are available starting with /archivelist, Kindly help me where I am doing wrong. from bs4 import BeautifulSoup import re, csv import urllib.request import scrapy file = open("output.txt", "r") for i in file: urls = urllib.request.Request(i, headers={'User-Agent': 'Mozilla/5.0'}) data1 = urllib.request.urlopen(urls).read() page1 = BeautifulSoup(data1, 'html.parser') for link1 in page1.findAll(href = re.compile('^/archivelist/')): l1 = link1.get('href') print(l1) Thanks, Kishore. From greg.ewing at canterbury.ac.nz Sat Oct 28 22:00:23 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 15:00:23 +1300 Subject: Compression of random binary data In-Reply-To: <87bmktjzbp.fsf@bsb.me.uk> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> Message-ID: Ben Bacarisse wrote: > But that has to be about the process that gives rise to the data, not > the data themselves. > If I say: "here is some random data..." you can't tell if it is or is > not from a random source. I can, as a parlour trick, compress and > recover this "random data" because I chose it. Indeed. Another way to say it is that you can't conclude anything about the source from a sample size of one. If you have a large enough sample, then you can estimate a probability distribution, and calculate an entropy. > I think the argument that you can't compress arbitrary data is simpler > ... it's obvious that it includes the results of previous > compressions. What? I don't see how "results of previous compressions" comes into it. The source has an entropy even if you're not doing compression at all. -- Greg From ian.g.kelly at gmail.com Sat Oct 28 22:04:17 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 28 Oct 2017 16:04:17 -1000 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Oct 28, 2017 10:30 AM, "Stefan Ram" wrote: > Well, then one can ask about the entropy of a data source > thatt only is emitting this message. (If it needs to be endless: > thatt only is emitting this message repeatedly.) If there is only one possible message then the entropy is zero. -1.0 * log2(1.0) == 0 From greg.ewing at canterbury.ac.nz Sat Oct 28 22:17:46 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 15:17:46 +1300 Subject: Compression of random binary data In-Reply-To: <59f48e52$0$18634$b1db1813$d948b532@news.astraweb.com> References: <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> <59f48e52$0$18634$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > Random data = any set of data generated by "a source of random". Any set of data generated by Grant Thompson? https://www.youtube.com/user/01032010814 -- Greg From dan at tombstonezero.net Sat Oct 28 22:18:04 2017 From: dan at tombstonezero.net (Dan Sommers) Date: Sun, 29 Oct 2017 02:18:04 +0000 (UTC) Subject: Coding style in CPython implementation References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> <5ecdce2b-f347-4205-9ba8-8a4f12cdfb88@googlegroups.com> Message-ID: On Sat, 28 Oct 2017 16:20:54 -0700, ???????? ????????? wrote: > I do believe though that if (!d) is a lot clearer than if (d == NULL) > as it is safer than falsely assigning NULL in d, by pure mistake. Having made my living writing C code for a very long time, I always found if (!d) *harder* to read, especially if it happened to be written if(!d) without the space, because it was never a single character name the ! disappeared into the noise. OTOH, = vs. == always stuck out like a sore thumb. Then again, I grew up with monochrome terminals vs. unusably slow syntax highlighting, and grayscale printers (and we had to live in a small shoebox in the middle of the road). YMMV. Obviously. Dan From greg.ewing at canterbury.ac.nz Sat Oct 28 22:18:56 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 15:18:56 +1300 Subject: Compression of random binary data In-Reply-To: <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: danceswithnumbers at gmail.com wrote: > 1111000010101011 > This equals > 61611 > This can be represented using > 0-6 log2(7)*5= 14.0367746103 bits > > > 1101010100001111 > This equals > 54543 > This can be represented using > 0-5 log2(6)*5= 12.9248125036 bits You're missing something fundamental about what entropy is in information theory. It's meaningless to talk about the entropy of a single message. Entropy is a function of the probability distribution of *all* the messages you might want to send. What you calculated in your first example relates to this situation: You want to send someone messages consisting of 5 symbols drawn from the set {0, 1, 2, 3, 4, 5, 6}, where all such messages have equal probability. In that case, you need and average of about 14.03 bits for each message. Note that this has essentially nothing to do with the particular sequence of bits you started with. Your second calculation was for a similar situation, except that the set of symbols is just {0, 1, 2, 3, 4, 5}. There are fewer messages of length 5 that can be constructed from that set, so the number of bits needed is smaller. > In reality you can express 54543 with 10 bits. Again, this statement is meaningless. You can't say *anything* about the number of bits needed to represent that particular number, without knowing what *other* numbers you might want to represent. -- Greg From rosuav at gmail.com Sat Oct 28 22:32:05 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 13:32:05 +1100 Subject: Compression of random binary data In-Reply-To: References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 1:18 PM, Gregory Ewing wrote: > You're missing something fundamental about what > entropy is in information theory. > > It's meaningless to talk about the entropy of a single > message. Entropy is a function of the probability > distribution of *all* the messages you might want to > send. Which is where a lot of "password strength" confusion comes from. How much entropy is there in the password "U3ZINVp3PT0="? Strong password or weak? What about "dark-next-sure-secret"? "with-about-want-really-going"? They were generated by, respectively: double-MIME-encoding four bytes from /dev/random (32 bits of entropy), picking four words from the top 1024 (40 bits), and picking 5 words from the top 64 (30 bits). But just by looking at the passwords themselves, you can't tell that. ChrisA From rosuav at gmail.com Sat Oct 28 22:34:16 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 13:34:16 +1100 Subject: Compression of random binary data In-Reply-To: References: <5b828b84-b392-4942-a579-cce4d0ea4dfb@googlegroups.com> <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 1:32 PM, Chris Angelico wrote: > On Sun, Oct 29, 2017 at 1:18 PM, Gregory Ewing > wrote: >> You're missing something fundamental about what >> entropy is in information theory. >> >> It's meaningless to talk about the entropy of a single >> message. Entropy is a function of the probability >> distribution of *all* the messages you might want to >> send. > > Which is where a lot of "password strength" confusion comes from. How > much entropy is there in the password "U3ZINVp3PT0="? Strong password > or weak? What about "dark-next-sure-secret"? > "with-about-want-really-going"? They were generated by, respectively: > double-MIME-encoding four bytes from /dev/random (32 bits of entropy), > picking four words from the top 1024 (40 bits), and picking 5 words > from the top 64 (30 bits). But just by looking at the passwords > themselves, you can't tell that. To clarify: The "top 1024 words" concept is XKCD 936 style password generation, using my tabletop gaming room's resident parrot. So it's based on the frequency of words used by D&D players. YMMV if you use a different corpus :) ChrisA From greg.ewing at canterbury.ac.nz Sat Oct 28 23:08:20 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 16:08:20 +1300 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: Stefan Ram wrote: > Well, then one can ask about the entropy of a data source > that only is emitting this message. You can, but it's still the *source* that has the entropy, not the message. (And the answer in that case is that the entropy is zero. If there's only one possible message you can ever send, you don't need to send it at all!) -- Greg From greg.ewing at canterbury.ac.nz Sat Oct 28 23:31:52 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 16:31:52 +1300 Subject: Compression of random binary data In-Reply-To: <59f521b7$0$18614$b1db1813$d948b532@news.astraweb.com> References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> <59f521b7$0$18614$b1db1813$d948b532@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > I don't think that's right. The entropy of a single message is a well-defined > quantity, formally called the self-information. > > https://en.wikipedia.org/wiki/Self-information True, but it still depends on knowing (or assuming) the probability of getting that particular message out of the set of all possible messages. This is *not* what danceswithnumbers did when he calculated the "entropy" of his example bit sequences. He didn't define the set they were drawn from or what their probabilities were. -- Greg From rosuav at gmail.com Sat Oct 28 23:50:42 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 14:50:42 +1100 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 2:08 PM, Gregory Ewing wrote: > Stefan Ram wrote: >> >> Well, then one can ask about the entropy of a data source >> that only is emitting this message. > > > You can, but it's still the *source* that has the entropy, > not the message. > > (And the answer in that case is that the entropy is zero. > If there's only one possible message you can ever send, you > don't need to send it at all!) One bit. It might send the message, or it might NOT send the message. And I have known situations in which that is exactly the system used. Back before mobile phones were common, you could sometimes use a payphone to cause someone's phone to ring, but you couldn't actually speak on it. So you had one message you could send: "brrrring brrrrring". A pre-arranged meaning for that message might be "I'm at the railway station, please come and pick me up"... but there's still *some* information in the mere fact of the call. ChrisA From ian.g.kelly at gmail.com Sun Oct 29 03:00:34 2017 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 28 Oct 2017 21:00:34 -1000 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Oct 28, 2017 5:53 PM, "Chris Angelico" wrote: > One bit. It might send the message, or it might NOT send the message. Not sending the message is equivalent to having a second possible message. From rosuav at gmail.com Sun Oct 29 03:03:15 2017 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 29 Oct 2017 18:03:15 +1100 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: On Sun, Oct 29, 2017 at 6:00 PM, Ian Kelly wrote: > On Oct 28, 2017 5:53 PM, "Chris Angelico" wrote: >> One bit. It might send the message, or it might NOT send the message. > > Not sending the message is equivalent to having a second possible message. Okay, now we're getting seriously existential. Is a non-message a message? ChrisA From piet-l at vanoostrum.org Sun Oct 29 03:22:16 2017 From: piet-l at vanoostrum.org (Piet van Oostrum) Date: Sun, 29 Oct 2017 08:22:16 +0100 Subject: Problem with subprocess.Popen and EINTR References: <20171028231104.GA32394@cskk.homeip.net> Message-ID: Cameron Simpson writes: [...] > What if you did this: > > os.signal(SIGINT, SIG_IGN) > ... code code code, including the Popen/wait ... > old_handler = os.signal(SIGINT, do_nothing_handler) > sleep(...) > os.signal(SIGINT, old_handler) > > SIG_IGN is different from a do-nothing handler; it prevents the process > seeing the signal at all, and therefore prevents the signal interrupting > system calls. > > So you activate your dummy-quit-sleeping function only around the sleep itself. Thanks. That is a good idea. I'll try that out. Or I could first try to switch to Python 3, as Chris suggested, to see if that makes the problem disappear. -- Piet van Oostrum WWW: http://piet.vanoostrum.org/ PGP key: [8DAE142BE17999C4] From greg.ewing at canterbury.ac.nz Sun Oct 29 04:39:11 2017 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 29 Oct 2017 21:39:11 +1300 Subject: Compression of random binary data In-Reply-To: References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: Chris Angelico wrote: > One bit. It might send the message, or it might NOT send the message. The entropy formula assumes that you are definitely going to send one of the possible messages. If not sending a message is a possibility, then you need to include an empty message in the set of messages. Another way to think about it: The receiver can be in one of N possible states, and you want to ensure that it's in a particular state. To do that, there must be N possible messages that you can send. If N = 1, the receiver is already in the right state, so you don't need to send it a message at all. Example: If the message is "I love you", and your SO already knows that, you don't need to tell them. (Not according to information theory, at least.) -- Greg From stefanossofroniou542 at gmail.com Sun Oct 29 05:00:49 2017 From: stefanossofroniou542 at gmail.com (=?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?=) Date: Sun, 29 Oct 2017 02:00:49 -0700 (PDT) Subject: Coding style in CPython implementation In-Reply-To: References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> <5ecdce2b-f347-4205-9ba8-8a4f12cdfb88@googlegroups.com> Message-ID: <71e5c81c-4295-4c99-ad6c-cb049da4bc25@googlegroups.com> On Sunday, October 29, 2017 at 4:18:38 AM UTC+2, Dan Sommers wrote: > On Sat, 28 Oct 2017 16:20:54 -0700, ???????? ????????? wrote: > > > I do believe though that if (!d) is a lot clearer than if (d == NULL) > > as it is safer than falsely assigning NULL in d, by pure mistake. > > Having made my living writing C code for a very long time, I always > found if (!d) *harder* to read, especially if it happened to be written > if(!d) without the space, because it was never a single character name > the ! disappeared into the noise. Indeed it gets harder to distinguish it after a lot of time staring at your source code. I guess the following parts from "Zen of Python" apply to this case: - Beautiful is better than ugly. - Explicit is better than implicit. - Simple is better than complex. - Readability counts. > OTOH, = vs. == always stuck out like > a sore thumb. Then again, I grew up with monochrome terminals > vs. unusably slow syntax highlighting, and grayscale printers (and we > had to live in a small shoebox in the middle of the road). > > YMMV. Obviously. > > Dan From steve+python at pearwood.info Sun Oct 29 05:56:38 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 29 Oct 2017 20:56:38 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> Message-ID: <59f5a5d8$0$18569$b1db1813$d948b532@news.astraweb.com> On Sun, 29 Oct 2017 06:03 pm, Chris Angelico wrote: > On Sun, Oct 29, 2017 at 6:00 PM, Ian Kelly wrote: >> On Oct 28, 2017 5:53 PM, "Chris Angelico" wrote: >>> One bit. It might send the message, or it might NOT send the message. >> >> Not sending the message is equivalent to having a second possible message. > > Okay, now we're getting seriously existential. Is a non-message a message? Is zero a number? Is bald a hair colour? Is atheism a religion? Aristotle seriously argued that the smallest whole number was two. Zero was clearly nothing at all, and one wasn't a *number*, it was *unity*. A number is, by definition, a multitude of units. https://philosophy.stackexchange.com/questions/19533/why-does-aristotle-suggest-one-is-not-a-number http://classics.mit.edu/Aristotle/metaphysics.14.xiv.html No message can be a message. British nuclear submarines during the Cold War had orders that if they failed to receive any transmissions from London within a certain time period, they were to crack open the secret orders from the Prime Minister. Nobody knows what is in the orders, as they were destroyed when the PM left office, but they are popularly supposed to have included: - fire your missiles at Russia; - surrender to whoever won the war; - try to flee to Australia or New Zealand and join up with whatever remnants of the British Empire still exist; - do whatever you want. It makes a good, but ultimately futile, exercise to try to guess what the various PMs would have said. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Oct 29 05:57:36 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 29 Oct 2017 20:57:36 +1100 Subject: Compression of random binary data References: <1f61863b-b8f2-41f8-b108-0331a44db43f@googlegroups.com> <3f88da25-54a7-49a2-92db-62cff61da65a@googlegroups.com> <9d347f67-7078-4b8b-8bbc-d8bb5f102726@googlegroups.com> <3a27ba1f-148e-4fa6-9954-9d2837eadbdf@googlegroups.com> <59f521b7$0$18614$b1db1813$d948b532@news.astraweb.com> Message-ID: <59f5a611$0$18569$b1db1813$d948b532@news.astraweb.com> On Sun, 29 Oct 2017 02:31 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: >> I don't think that's right. The entropy of a single message is a >> well-defined quantity, formally called the self-information. >> >> https://en.wikipedia.org/wiki/Self-information > > True, but it still depends on knowing (or assuming) the > probability of getting that particular message out of > the set of all possible messages. Indeed. > This is *not* what danceswithnumbers did when he > calculated the "entropy" of his example bit sequences. > He didn't define the set they were drawn from or > what their probabilities were. I'm not defending or supporting Danceswithnumbers in his confusion. I'm just pointing out that an entropy-like measure of information for individual messages does exist, and people do often call it "entropy". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From martin.schoon at gmail.com Sun Oct 29 06:19:49 2017 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 29 Oct 2017 10:19:49 GMT Subject: Repairing Python installation? References: <20171028205106.md5act7txnqvlfua@hermes.hilbert.loc> Message-ID: Den 2017-10-28 skrev Karsten Hilbert : > On Sat, Oct 28, 2017 at 08:41:34PM +0000, Martin Sch??n wrote: > >> It seems something is amiss with my Python 2.7 installation. Revisiting >> Nikola (static web site generator written in Python) for the first time >> in several years the other day I experience some unexpected problems. I >> got some help form the Nikola people and the conclusion is something >> is broken with my Python 2.7. Pip list throws exceptions at me > ... >> All this is happening on a Debian machine. > > Exceptions or crashes of the python interpreter ? > > Karsten Exceptions. (Disclaimer: I am not a Python expert). Today I tried pip --version and got the following: /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py:1892: UserWarning: /usr/lib/pymodules/python2.7/rpl-1.5.5.egg-info could not be properly decoded in UTF-8 warnings.warn(msg) Reinstalling Python and pip makes no difference. pip3 --version throws no warnings. /Martin From martin.schoon at gmail.com Sun Oct 29 06:25:38 2017 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 29 Oct 2017 10:25:38 GMT Subject: Repairing Python installation? References: <0001HW.1FA5517B012AA9B815A74F2CF@news.individual.net> Message-ID: Den 2017-10-28 skrev Percival John Hackworth : > On 28-Oct-2017, Martin Sch??n wrote > (in article ): > >> It seems something is amiss with my Python 2.7 installation. Revisiting >> Nikola (static web site generator written in Python) for the first time >> in several years the other day I experience some unexpected problems. I >> got some help form the Nikola people and the conclusion is something >> is broken with my Python 2.7. Pip list throws exceptions at me while >> pip3 list works the way I expect it to do. >> > If this site is accessible from the internet, have you looked around other > parts of the system? Like date/timestamps for /bin/*. I recently was asked to > look at a friends web site and found they were doing several things that I > consider a security no-no, like running the tomcat server as root to serve > their web site. It had been root-kitted. I helped them setup a new system > that ran tomcat under a non-priv user. > No web server is involved in this. Programs like Nikola takes pictures, text files (Markdown, ReStructuredText...) + templates as input and spits out html, css and sometimes javascript. Then it is up to you to copy that result to a web server. /Martin From tjol at tjol.eu Sun Oct 29 06:40:45 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 29 Oct 2017 11:40:45 +0100 Subject: sys.path[] question In-Reply-To: References: Message-ID: On 28/10/17 19:24, ElChino wrote: > From the Python2.7 snippet in [1], Python2.7 reports that my > sys.path[] contains: > f:\ProgramFiler\Python27\lib\site-packages\pyreadline-2.0-py2.7-win32.egg > > (a .zip-file) > > But I have also a 'f:\ProgramFiler\Python27\lib\site-packages\pyreadline' > directory. With the complete package AFAICS. > > So my question is where will Python import a 'import pyreadline' from? Paths are searched in the order they're listed in sys.path. You can find out where a module is loaded from by checking its __file__ attribute. Run python (in interactive mode) and execute >>> import pyreadline >>> pyreadline.__file__ > E.f. if I do a: > python -vv > f:\ProgramFiler\Python27\lib\site-packages\pyreadline\test\test_lineeditor.py > > > it shows a: > # zipimport: found 89 names in > F:\ProgramFiler\Python27\lib\site-packages\pyreadline-2.0-py2.7-win32.egg > But no other references to pyreadline-2.0-py2.7-win32.egg. > > Does that mean that only this .egg is where python imports all pyreadline > files from? And if so, can I delete > 'f:\ProgramFiler\Python27\lib\site-packages\pyreadline' ? This sounds like a fairly high risk of breaking something (presumably there is some reason the files are where they are) for absolutely no gain. It's not like it takes up any meaningful amount of space, is it? Why would you want to? -- Thomas From kwpolska at gmail.com Sun Oct 29 06:55:35 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 29 Oct 2017 11:55:35 +0100 Subject: Repairing Python installation? In-Reply-To: References: Message-ID: On 28 October 2017 at 22:41, Martin Sch??n wrote: > It seems something is amiss with my Python 2.7 installation. Revisiting > Nikola (static web site generator written in Python) for the first time > in several years the other day I experience some unexpected problems. I > got some help form the Nikola people and the conclusion is something > is broken with my Python 2.7. Pip list throws exceptions at me while > pip3 list works the way I expect it to do. > > All this is happening on a Debian machine. > > Questions: > > Should I try to pinpoint what is broken (and how is that done) or should > I just remove everything Python 2.7 and re-install? > > Could mixing pip installs with Debian distro installs of Python > packages lead to conflicts or other problems? Yes, it does, you should avoid that at all cost. The best way to do it is by using virtualenv. > Today I tried pip --version and got the following: > /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py:1892: > UserWarning: /usr/lib/pymodules/python2.7/rpl-1.5.5.egg-info could not > be properly decoded in UTF-8 > warnings.warn(msg) That?s a warning, and it happens to be caused by the `rpl` apt package. Remove it from your system and try `pip --version` again: https://github.com/certbot/certbot/issues/3795 Now, onto fixing Nikola issues: As discussed on IRC, Nikola recommends (and will soon require) Python 3. And you?ll be better off with a virtualenv: install `virtualenv` from apt and follow the Getting started guide for Nikola: https://getnikola.com/getting-started.html If you still get unicode issues when compiling posts, make sure they?re saved as UTF-8, and that your locale is configured properly: https://chriswarrick.com/blog/2017/06/18/unix-locales-vs-unicode/ (Nikola?s co-maintainer over here.) -- Chris Warrick PGP: 5EAAEA16 From elchino at cnn.cn Sun Oct 29 07:31:29 2017 From: elchino at cnn.cn (ElChino) Date: Sun, 29 Oct 2017 12:31:29 +0100 Subject: sys.path[] question In-Reply-To: References: Message-ID: Thomas Jollans wrote: > You can find out where a module is loaded from by checking its __file__ > attribute. Run python (in interactive mode) and execute > >>>> import pyreadline >>>> pyreadline.__file__ Thanks for that tip. >> Does that mean that only this .egg is where python imports all pyreadline >> files from? And if so, can I delete >> 'f:\ProgramFiler\Python27\lib\site-packages\pyreadline' ? > > This sounds like a fairly high risk of breaking something (presumably > there is some reason the files are where they are) for absolutely no > gain. It's not like it takes up any meaningful amount of space, is it? > Why would you want to? True, disk space is a few cents per Gigabyte. But maybe to reduce the clutter on my disk and to keep things neat and simple. I feel as a Python newbie, the file-structure / residue from various package-installs clutters the 'Lib/site-packages' a great deal. There is probably a tool to analyse "dead" Python package installs? From steve+python at pearwood.info Sun Oct 29 08:27:01 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 29 Oct 2017 23:27:01 +1100 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> Message-ID: <59f5c917$0$18571$b1db1813$d948b532@news.astraweb.com> On Sun, 29 Oct 2017 01:56 pm, Stefan Ram wrote: > If the entropy of an individual message is not defined, > than it is still available to be defined. I define it > to be log2(1/p), where p is the probability of this > message. I also choose a unit for it, which I call "bit". That is exactly the definition of self-information: https://en.wikipedia.org/wiki/Self-information See also: https://en.wikipedia.org/wiki/Entropy_(information_theory) which lists several forms of related measures of information: - the self-information of an individual message or symbol taken from a given probability distribution; - the entropy of a given probability distribution of messages or symbols; - the entropy rate of a stochastic process. It also suggests a connection between information entropy and thermodynamic entropy, namely that the information entropy of a system is the amount of additional information needed to determine the microstate of a system (the states of all its particles), given the macrostate (identified by the bulk thermodynamic parameters such as temperature, volume, energy). More here: https://physics.stackexchange.com/questions/263197/is-information-entropy-the-same-as-thermodynamic-entropy -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From akishorecert at gmail.com Sun Oct 29 09:16:15 2017 From: akishorecert at gmail.com (Kishore Kumar Alajangi) Date: Sun, 29 Oct 2017 18:46:15 +0530 Subject: Listing link urls In-Reply-To: References: Message-ID: + tutor On Sun, Oct 29, 2017 at 6:57 AM, Kishore Kumar Alajangi < akishorecert at gmail.com> wrote: > Hi, > > I am facing an issue with listing specific urls inside web page, > > https://economictimes.indiatimes.com/archive.cms > > Page contains link urls by year and month vise, > Ex: /archive/year-2001,month-1.cms > > I am able to list all required urls using the below code, > > from bs4 import BeautifulSoup > import re, csv > import urllib.request > import scrapy > req = urllib.request.Request('http://economictimes.indiatimes.com/archive.cms', headers={'User-Agent': 'Mozilla/5.0'}) > > > links = [] > totalPosts = [] > url = "http://economictimes.indiatimes.com" > data = urllib.request.urlopen(req).read() > page = BeautifulSoup(data,'html.parser') > > for link in page.findAll('a', href = re.compile('^/archive/')): //retrieving urls starts with "archive" > l = link.get('href') > links.append(url+l) > > > with open("output.txt", "a") as f: > for post in links: > post = post + '\n' > f.write(post) > > *sample result in text file:* > > http://economictimes.indiatimes.com/archive/year-2001,month-1.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-2.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-3.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-4.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-5.cmshttp://economictimes.indiatimes.com/archive/year-2001,month-6.cms > > > List of urls I am storing in a text file, From the month urls I want to retrieve day urls starts with "/archivelist", I am using > > the below code, but I am not getting any result, If I check with inspect element the urls are available starting with /archivelist, > > > > Kindly help me where I am doing wrong. > > from bs4 import BeautifulSoup > import re, csv > import urllib.request > import scrapy > > file = open("output.txt", "r") > > > for i in file: > > urls = urllib.request.Request(i, headers={'User-Agent': 'Mozilla/5.0'}) > > data1 = urllib.request.urlopen(urls).read() > > page1 = BeautifulSoup(data1, 'html.parser') > > for link1 in page1.findAll(href = re.compile('^/archivelist/')): > > l1 = link1.get('href') > > print(l1) > > > Thanks, > > Kishore. > > > > > > From alb at nospam.chip.org Sun Oct 29 10:18:50 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 10:18:50 -0400 Subject: Invoking return through a function? Message-ID: Hello, I'm wondering if there is a way of writing a function that causes a return from the function that called it. To explain with an example, let's say that I want to exit my function if a dict does not contain a given key. I could write: def testFun(): ... if key not in dict: return ... But if this is a test I need to do a lot of times, I'd like to replace it with something shorter and more explicit: def testFun(): ... checkKey(dict, key) ... and I'd like checkKey to cause a return *from testFun*. In a language like Lisp this would be accomplished by defining checkKey as a macro that expands into the code shown in my first example, so that the return would be inside testFun and not insted checkKey. Is there a way of doing something like this in Python? Another way of phrasing my question is: is there a way to cause a return from a function that is higher up in the call stack, rather than the currently active one, without using try/except? Thanks, Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From ned at nedbatchelder.com Sun Oct 29 10:30:19 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 29 Oct 2017 10:30:19 -0400 Subject: Invoking return through a function? In-Reply-To: References: Message-ID: <891ae8a3-bab3-4ea6-86fd-7fbe0c3b9daf@nedbatchelder.com> On 10/29/17 10:18 AM, Alberto Riva wrote: > Hello, > > I'm wondering if there is a way of writing a function that causes a > return from the function that called it. To explain with an example, > let's say that I want to exit my function if a dict does not contain a > given key. I could write: > > def testFun(): > ? ... > ? if key not in dict: > ??? return > ? ... > > But if this is a test I need to do a lot of times, I'd like to replace > it with something shorter and more explicit: > > def testFun(): > ? ... > ? checkKey(dict, key) > ? ... > > and I'd like checkKey to cause a return *from testFun*. In a language > like Lisp this would be accomplished by defining checkKey as a macro > that expands into the code shown in my first example, so that the > return would be inside testFun and not insted checkKey. Is there a way > of doing something like this in Python? > > Another way of phrasing my question is: is there a way to cause a > return from a function that is higher up in the call stack, rather > than the currently active one, without using try/except? > No, there isn't. --Ned. From steve+python at pearwood.info Sun Oct 29 10:35:33 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 30 Oct 2017 01:35:33 +1100 Subject: Invoking return through a function? References: Message-ID: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 01:18 am, Alberto Riva wrote: > Hello, > > I'm wondering if there is a way of writing a function that causes a > return from the function that called it. To explain with an example, > let's say that I want to exit my function if a dict does not contain a > given key. I could write: > > def testFun(): > ... > if key not in dict: > return > ... > > But if this is a test I need to do a lot of times, I'd like to replace > it with something shorter and more explicit: > > def testFun(): > ... > checkKey(dict, key) > ... You mean *less* explicit. "checkKey" gives absolutely no hint that it causes the current function to return. > and I'd like checkKey to cause a return *from testFun*. In a language > like Lisp this would be accomplished by defining checkKey as a macro > that expands into the code shown in my first example, so that the return > would be inside testFun and not insted checkKey. Is there a way of doing > something like this in Python? Fortunately not. > Another way of phrasing my question is: is there a way to cause a return > from a function that is higher up in the call stack, rather than the > currently active one, without using try/except? No. You really should re-think your strategy. Your suggestion, if possible, would lead to difficult to maintain code where you couldn't easily tell where the exit points of a function where. Imagine reading code like: def foo(x): a = sin(x) b = cos(x) print(a, b) return a + b There's one return, right? No. If Python could do what you are asking for, *every function call* could be a hidden, secret return. What a nightmare that would be. It is bad enough that any function could raise an exception, but at least exceptions halt the normal execution of code (unless explicitly caught). They don't silently continue normal execution. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rantingrickjohnson at gmail.com Sun Oct 29 10:52:38 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 29 Oct 2017 07:52:38 -0700 (PDT) Subject: Coding style in CPython implementation In-Reply-To: <71e5c81c-4295-4c99-ad6c-cb049da4bc25@googlegroups.com> References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> <5ecdce2b-f347-4205-9ba8-8a4f12cdfb88@googlegroups.com> <71e5c81c-4295-4c99-ad6c-cb049da4bc25@googlegroups.com> Message-ID: On Sunday, October 29, 2017 at 4:00:59 AM UTC-5, ???????? ????????? wrote: [...] > I guess the following parts from "Zen of Python" apply to this case: > > - Beautiful is better than ugly. > - Explicit is better than implicit. > - Simple is better than complex. > - Readability counts. Now go acquaint yourself with the syntactical abomination otherwise known as Type-hints, and you will understand that all of the aforementioned "zens" are forgotten parts of GvR's old testament. Design guidelines born of the ideals enshrined in a governmental contract baiting scheme slickly marketed as "cp4e" -- which were originally encoded on punch cards and passed to Tim Peters during secret ceremony atop the roof of CNRI building in Reston, Virgina, on the night of the winter solstice. Now-a-days, Python's design philosophy is less concerned with readability and focuses itself primarily with corporate knob-gobbling. From bc at freeuk.com Sun Oct 29 11:13:22 2017 From: bc at freeuk.com (bartc) Date: Sun, 29 Oct 2017 15:13:22 +0000 Subject: Invoking return through a function? In-Reply-To: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> Message-ID: <1emJB.169114$_O1.43917@fx31.am4> On 29/10/2017 14:35, Steve D'Aprano wrote: > On Mon, 30 Oct 2017 01:18 am, Alberto Riva wrote: >> I'm wondering if there is a way of writing a function that causes a >> return from the function that called it. > You really should re-think your strategy. Your suggestion, if possible, would > lead to difficult to maintain code where you couldn't easily tell where the > exit points of a function where. Imagine reading code like: > > > def foo(x): > a = sin(x) > b = cos(x) > print(a, b) > return a + b > > There's one return, right? No. If Python could do what you are asking for, > *every function call* could be a hidden, secret return. What a nightmare that > would be. > > It is bad enough that any function could raise an exception, Don't exceptions do exactly the same thing as what the OP is proposing? Worse, actually. That is, it could stop executing during any of the first three lines of foo(), and end up continuing execution in the caller, or in the caller's caller, and so on. > but at least > exceptions halt the normal execution of code (unless explicitly caught). They > don't silently continue normal execution. I'm having trouble seeing the difference. If F calls G calls H, then H can arrange an exception to be picked up by F, bypassing normal return paths back to G and back to F. And there's nothing G can do about it. At least, not without a lot of trouble, like wrapping try/except around every statement and sub-statement in G. (What the OP wants was also proposed a few weeks back in comp.lang.c. But that was only within nested functions, so if H is inside G, and G is inside F, then a 'returnall' from H would return directly directly from F. Apparently Lisp allows this...) -- Bartc From ben.usenet at bsb.me.uk Sun Oct 29 11:17:53 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 29 Oct 2017 15:17:53 +0000 Subject: Compression of random binary data References: <708b9680-451b-4c02-8d41-278749be7ff8@googlegroups.com> <671cc9e5-1db5-421d-bfa4-0da5d76bebf2@googlegroups.com> <01704a53-8348-480b-a70b-33f917857041@googlegroups.com> <87efpsj14c.fsf@bsb.me.uk> <59ef27bc$0$14951$b1db1813$d948b532@news.astraweb.com> <87wp3khdy9.fsf@bsb.me.uk> <87bmktjzbp.fsf@bsb.me.uk> Message-ID: <87a80at23i.fsf@bsb.me.uk> Gregory Ewing writes: > Ben Bacarisse wrote: >> But that has to be about the process that gives rise to the data, not >> the data themselves. > >> If I say: "here is some random data..." you can't tell if it is or is >> not from a random source. I can, as a parlour trick, compress and >> recover this "random data" because I chose it. > > Indeed. Another way to say it is that you can't conclude > anything about the source from a sample size of one. > > If you have a large enough sample, then you can estimate > a probability distribution, and calculate an entropy. > >> I think the argument that you can't compress arbitrary data is simpler >> ... it's obvious that it includes the results of previous >> compressions. > > What? I don't see how "results of previous compressions" comes > into it. The source has an entropy even if you're not doing > compression at all. Maybe we are taking at cross purposes. A claim to be able to compress arbitrary data leads immediately to the problem that iterating the compression will yield zero-size results. That, to me, is a simpler argument that talking about data from a random source. -- Ben. From rantingrickjohnson at gmail.com Sun Oct 29 12:21:51 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 29 Oct 2017 09:21:51 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: References: Message-ID: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> On Sunday, October 29, 2017 at 9:19:03 AM UTC-5, Alberto Riva wrote: > Hello, > > I'm wondering if there is a way of writing a function that > causes a return from the function that called it. To > explain with an example, let's say that I want to exit my > function if a dict does not contain a given key. I could > write: > > def testFun(): > ... > if key not in dict: > return > ... > > But if this is a test I need to do a lot of times, I'd like > to replace it with something shorter and more explicit: > > def testFun(): > ... > checkKey(dict, key) > ... > > and I'd like checkKey to cause a return *from testFun*. But under what exact _circumstances_? So, you'd like a call to `checkKey(...)` to cause a return in `testFunc` when: (1) the key exists? (2) the key doesn't exist? (3) the dict is actually a list? (3) The call raises an exception? (4) a seg fault occurs? (5) GvR regains his sanity, self-respect and dignity by 86'ing type-hints? Be more specific please. And _always_ refer to a named function by _name_. Never use random or implicit aliases (aka: "the function", "my function", etc...) in your prose. Both functions *ARE* functions, dontchaknow? > In a language like Lisp Python is nothing like Lisp, and for good reason! Sure, we have a few lispers and functional fanboys who hang around here, and sometimes Rustom can get a little preachy about FP, but mostly, we tolerate the fanboyism -- so long as it's not rabid fanboyism. > this would be accomplished by defining checkKey as a macro > that expands into the code shown in my first example, so > that the return would be inside testFun and not insted > checkKey. Is there a way of doing something like this in > Python? Thankfully, no. But i wouldn't underestimate the rebellious nature of this generation of Py-dev'ers, and the propensity of these folks to entertain feature creep. > Another way of phrasing my question is: is there a way to > cause a return from a function that is higher up in the > call stack, rather than the currently active one, without > using try/except? Possibly. But you've already poo-pooed the easy and obvious solution of using conditional logic. From alb at nospam.chip.org Sun Oct 29 12:35:11 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 12:35:11 -0400 Subject: Invoking return through a function? In-Reply-To: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> Message-ID: On 10/29/2017 10:35 AM, Steve D'Aprano wrote: > On Mon, 30 Oct 2017 01:18 am, Alberto Riva wrote: > >> Hello, >> >> I'm wondering if there is a way of writing a function that causes a >> return from the function that called it. To explain with an example, >> let's say that I want to exit my function if a dict does not contain a >> given key. I could write: >> >> def testFun(): >> ... >> if key not in dict: >> return >> ... >> >> But if this is a test I need to do a lot of times, I'd like to replace >> it with something shorter and more explicit: >> >> def testFun(): >> ... >> checkKey(dict, key) >> ... > > You mean *less* explicit. "checkKey" gives absolutely no hint that it causes > the current function to return. That's just because I used a name that was too generic in my example. I can call it "returnIfKeyMissing", and then it would be clear. >> and I'd like checkKey to cause a return *from testFun*. In a language >> like Lisp this would be accomplished by defining checkKey as a macro >> that expands into the code shown in my first example, so that the return >> would be inside testFun and not insted checkKey. Is there a way of doing >> something like this in Python? > > Fortunately not. > > >> Another way of phrasing my question is: is there a way to cause a return >> from a function that is higher up in the call stack, rather than the >> currently active one, without using try/except? > > No. > > You really should re-think your strategy. Your suggestion, if possible, would > lead to difficult to maintain code where you couldn't easily tell where the > exit points of a function where. But again, it's just a naming problem. Something like returnIfKeyMissing would make it easy to tell where the exit points are. And as Bartc pointed out, we already have this situation with exceptions, so it would be nothing new. Indeed, what I'm asking for could be accomplished by wrapping the body of each function in a try/catch block for an ad-hoc exception type. But again, since the language doesn't have macros, this would have to be done manually in each function. I was just asking for a more convenient way to do it, being a lazy programmer :) > Imagine reading code like: > > def foo(x): > a = sin(x) > b = cos(x) > print(a, b) > return a + b > > There's one return, right? No. If Python could do what you are asking for, > *every function call* could be a hidden, secret return. What a nightmare that > would be. > > It is bad enough that any function could raise an exception, but at least > exceptions halt the normal execution of code (unless explicitly caught). They > don't silently continue normal execution. See above :) Thanks, Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From alb at nospam.chip.org Sun Oct 29 12:45:58 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 12:45:58 -0400 Subject: Invoking return through a function? In-Reply-To: <1emJB.169114$_O1.43917@fx31.am4> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> Message-ID: On 10/29/2017 11:13 AM, bartc wrote: > > (What the OP wants was also proposed a few weeks back in comp.lang.c. > But that was only within nested functions, so if H is inside G, and G is > inside F, then a 'returnall' from H would return directly directly from > F. Apparently Lisp allows this...) Well, not directly, but it can be simulated with the condition system (equivalent to try/except) or with throw/catch, which is similar but doesn't use exceptions. But my point was that in Lisp you don't need to do this, because you can write a macro that expands into a return *in place*, without calling a separate function, and this eliminates the problem entirely. Since Python doesn't have macros I was looking for the next-best solution, but there doesn't seem to be one. Oh, well... Thanks, Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From alb at nospam.chip.org Sun Oct 29 13:02:33 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 13:02:33 -0400 Subject: Invoking return through a function? In-Reply-To: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> Message-ID: On 10/29/2017 12:21 PM, Rick Johnson wrote: > On Sunday, October 29, 2017 at 9:19:03 AM UTC-5, Alberto Riva wrote: >> Hello, >> >> I'm wondering if there is a way of writing a function that >> causes a return from the function that called it. To >> explain with an example, let's say that I want to exit my >> function if a dict does not contain a given key. I could >> write: >> >> def testFun(): >> ... >> if key not in dict: >> return >> ... >> >> But if this is a test I need to do a lot of times, I'd like >> to replace it with something shorter and more explicit: >> >> def testFun(): >> ... >> checkKey(dict, key) >> ... >> >> and I'd like checkKey to cause a return *from testFun*. > > But under what exact _circumstances_? > > So, you'd like a call to `checkKey(...)` to cause a return > in `testFunc` when: > > (1) the key exists? > > (2) the key doesn't exist? > > (3) the dict is actually a list? > > (3) The call raises an exception? > > (4) a seg fault occurs? > > (5) GvR regains his sanity, self-respect and dignity by > 86'ing type-hints? In this specific case, I wanted to invoke a return if the key doesn't exist (as I wrote in my post). But this is not strictly relevant, my question was more general. To phrase it in yet another way: is there *any other way* to invoke return on a function, except by explicitly writing "return" in its body? > Be more specific please. And _always_ refer to a named > function by _name_. Never use random or implicit aliases > (aka: "the function", "my function", etc...) in your prose. > Both functions *ARE* functions, dontchaknow? I do, but apparently everybody else was able to understand what I was asking... :) >> In a language like Lisp > > Python is nothing like Lisp, and for good reason! I would disagree with this. Actually, it's the most Lisp-like language I've encountered in my 25 years of writing software. It has closures, functions are first-class objects, it's interactive, highly reflective, variables are not strongly typed... The only big thing missing is macros, which are a direct result of the code-is-data feature of Lisp. > Sure, we > have a few lispers and functional fanboys who hang around > here, and sometimes Rustom can get a little preachy about > FP, but mostly, we tolerate the fanboyism -- so long as it's > not rabid fanboyism. I don't consider myself a fanboy - I've used Lisp for most of my career, and now I happily use Python for 80% of what I do, even if it means putting up with the occasional limitation, as in this case. >> this would be accomplished by defining checkKey as a macro >> that expands into the code shown in my first example, so >> that the return would be inside testFun and not insted >> checkKey. Is there a way of doing something like this in >> Python? > > Thankfully, no. Why "thankfully"? Having macros in Python would be great. Of course it's not possible because of the syntax, but it would be great nevertheless :) >> Another way of phrasing my question is: is there a way to >> cause a return from a function that is higher up in the >> call stack, rather than the currently active one, without >> using try/except? > > Possibly. But you've already poo-pooed the easy and obvious > solution of using conditional logic. Not really. It's obvious that I'm going to use a conditional, I was just asking for a way to avoid having to type the same conditional over and over. Computers are meant to do work for you, after all, aren't they? :) Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From Karsten.Hilbert at gmx.net Sun Oct 29 13:08:42 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sun, 29 Oct 2017 18:08:42 +0100 Subject: Aw: Re: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> Message-ID: > In this specific case, I wanted to invoke a return if the key doesn't > exist (as I wrote in my post). But this is not strictly relevant, my > question was more general. To phrase it in yet another way: is there > *any other way* to invoke return on a function, except by explicitly > writing "return" in its body? Sure: fall off the end of the function. But that's not what you are after. Karsten From martin.schoon at gmail.com Sun Oct 29 13:11:16 2017 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 29 Oct 2017 17:11:16 GMT Subject: Repairing Python installation? References: Message-ID: Den 2017-10-29 skrev Chris Warrick : > On 28 October 2017 at 22:41, Martin Sch??n wrote: >> >> Could mixing pip installs with Debian distro installs of Python >> packages lead to conflicts or other problems? > > Yes, it does, you should avoid that at all cost. The best way to do it > is by using virtualenv. > OK. Python virtualenv is new to me. > > Now, onto fixing Nikola issues: > > As discussed on IRC, Nikola recommends (and will soon require) Python > 3. And you?ll be better off with a virtualenv: install `virtualenv` > from apt and follow the Getting started guide for Nikola: > https://getnikola.com/getting-started.html > I have installed Python 3 virtualenv and Nikola according to those instructions. I now have a working Nikola but I sure don't know what I am doing :-) How do I get back into that nikola python environment next time? I guess I need to do some reading. > If you still get unicode issues when compiling posts, make sure > they?re saved as UTF-8, and that your locale is configured properly: > https://chriswarrick.com/blog/2017/06/18/unix-locales-vs-unicode/ > The problem with locale went away when I changed DEFAULT_LANG from en to sv in conf.py. pip list still throws exceptions at me but I have a plan. My secondary computer has a very similar Debian installation. There are no Python issues on that machine so I will do some comparative research. Most gratefully, /Martin From kwpolska at gmail.com Sun Oct 29 13:26:53 2017 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 29 Oct 2017 18:26:53 +0100 Subject: Repairing Python installation? In-Reply-To: References: Message-ID: On 29 October 2017 at 18:11, Martin Sch??n wrote: > I have installed Python 3 virtualenv and Nikola according to those > instructions. I now have a working Nikola but I sure don't know what > I am doing :-) How do I get back into that nikola python environment > next time? cd into your virtualenv directory and run `source bin/activate`. -- Chris Warrick PGP: 5EAAEA16 From saxri89 at gmail.com Sun Oct 29 13:47:43 2017 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sun, 29 Oct 2017 10:47:43 -0700 (PDT) Subject: django celery delay function dont exetute Message-ID: <2ce556b6-8edc-4e48-b688-0d451dc2bf2f@googlegroups.com> hello I want to use celery tasks in my Django project and I try to follow this very good tutorial from Mr.Vitor Freitas. but in my case and if try to run it that tutorial project i don't get results back the functions don't execute and in my case and in tutorial(i take message to wait and refresh and nothing after refresh). Any idea Why ? I think so the problem maybe is in RABBITQM server ?some configuration ? Just install Erlang(otp_win64_20.1.exe) and after RabbitMQ(rabbitmq-server-3.6.12.exe) here example code : settings.py CELERY_BROKER_URL = 'amqp://localhost' celery.py from __future__ import absolute_import import os from celery import Celery os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings') app = Celery('mysite') app.config_from_object('django.conf:settings', namespace='CELERY') app.autodiscover_tasks() tasks.py import string from django.contrib.auth.models import User from django.utils.crypto import get_random_string from celery import shared_task @shared_task def create_random_user_accounts(total): for i in range(total): username = 'user_{}'.format(get_random_string(10, string.ascii_letters)) email = '{}@example.com'.format(username) password = get_random_string(50) User.objects.create_user(username=username, email=email, password=password) return '{} random users created with success!'.format(total) _ _init_ _.py from .celery import app as celery_app __all__ = ['celery_app'] views.py from django.contrib.auth.models import User from django.contrib import messages from django.views.generic import TemplateView from django.views.generic.list import ListView from django.views.generic.edit import FormView from django.shortcuts import redirect from .forms import GenerateRandomUserForm from .tasks import create_random_user_accounts class UsersListView(ListView): template_name = 'core/users_list.html' model = User class GenerateRandomUserView(FormView): template_name = 'core/generate_random_users.html' form_class = GenerateRandomUserForm def form_valid(self, form): total = form.cleaned_data.get('total') create_random_user_accounts.delay(total) messages.success(self.request, 'We are generating your random users! Wait a moment and refresh this page.') return redirect('users_list') here details after cd my project path > celery -A mysite worker -l info : C:\Windows\System32>cd C:\Users\username\Desktop\django-celery-example-master C:\Users\username\Desktop\django-celery-example-master>celery -A mysite worker -l info -------------- celery at pc name v4.1.0 (latentcall) ---- **** ----- --- * *** * -- Windows-8-6.2.9200 2017-10-29 18:10:24 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: mysite:0x404e5c0 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 4 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery [tasks] . mysite.core.tasks.create_random_user_accounts [2017-10-29 18:10:24,596: CRITICAL/MainProcess] Unrecoverable error: TypeError('must be integer, not _subprocess_handle',) Traceback (most recent call last): File "C:\Python27\lib\site-packages\celery\worker\worker.py", line 203, in start self.blueprint.start(self) File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 119, in start step.start(parent) File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 370, in start return self.obj.start() File "C:\Python27\lib\site-packages\celery\concurrency\base.py", line 131, in start self.on_start() File "C:\Python27\lib\site-packages\celery\concurrency\prefork.py", line 112, in on_start **self.options) File "C:\Python27\lib\site-packages\billiard\pool.py", line 1007, in __init__ self._create_worker_process(i) File "C:\Python27\lib\site-packages\billiard\pool.py", line 1116, in _create_worker_process w.start() File "C:\Python27\lib\site-packages\billiard\process.py", line 124, in start self._popen = self._Popen(self) File "C:\Python27\lib\site-packages\billiard\context.py", line 383, in _Popen return Popen(process_obj) File "C:\Python27\lib\site-packages\billiard\popen_spawn_win32.py", line 64, in __init__ _winapi.CloseHandle(ht) TypeError: must be integer, not _subprocess_handle Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\billiard\spawn.py", line 159, in spawn_main new_handle = steal_handle(parent_pid, pipe_handle) File "C:\Python27\lib\site-packages\billiard\reduction.py", line 126, in steal_handle _winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE) WindowsError: [Error 6] From rantingrickjohnson at gmail.com Sun Oct 29 14:13:41 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 29 Oct 2017 11:13:41 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> Message-ID: <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> Alberto Riva wrote: > Rick Johnson wrote: > > Alberto Riva wrote: [...] > >> In a language like Lisp > > > > Python is nothing like Lisp, and for good reason! > > I would disagree with this. Actually, it's the most Lisp- > like language I've encountered in my 25 years of writing > software. Really? I'm sorry that you have only encountered a few "academic languages". Do some googling, there are literally thousands of them in the wilds. > It has closures, functions are first-class objects, it's > interactive, highly reflective, variables are not strongly > typed... Congratulations. You've just described Ruby. And an apple- cart full of other languages as well. > I don't consider myself a fanboy - I've used Lisp for most > of my career, and now I happily use Python for 80% of what > I do, even if it means putting up with the occasional > limitation, as in this case. (not that i'm defending Python here...) but how do you know that this so-called missing "feature" is a limitation? How many other languages use this "feature"? And how many programmers swear they could not live without it? We need stats; hyperlinks; and eye witness testimonials. Heck, even a random YouTube vid would be helpful. > Why "thankfully"? Having macros in Python would be great. > Of course it's not possible because of the syntax, but it > would be great nevertheless :) To me, macros would seem to violate Python's "explicit is better than implicit" philosophy. But i admit, i'm something of a zen purist on these matters. O:-) However, if you can find a way to make macros Pythonic, hmm, i might be interested... > > Possibly. But you've already poo-pooed the easy and > > obvious solution of using conditional logic. > > Not really. It's obvious that I'm going to use a > conditional, I was just asking for a way to avoid having to > type the same conditional over and over. Computers are > meant to do work for you, after all, aren't they? :) A reasonable argument, i admit. But again, here in the peaceful fields of "Pythonia", most of us feel that "explicit is better than implicit". And why? Well, because code is read more often than it is written, and as such, explicit code will be of enormous help during the debugging process. Sure, you may save yourself a few key-strokes with features like macros, but at what cost? No action is without consequences, dontchaknow? From rosuav at gmail.com Sun Oct 29 14:24:35 2017 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 30 Oct 2017 05:24:35 +1100 Subject: Coding style in CPython implementation In-Reply-To: References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> <5ecdce2b-f347-4205-9ba8-8a4f12cdfb88@googlegroups.com> <71e5c81c-4295-4c99-ad6c-cb049da4bc25@googlegroups.com> Message-ID: On Mon, Oct 30, 2017 at 12:47 AM, Stefan Ram wrote: > =?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?= writes: >>I guess the following parts from "Zen of Python" apply to this case: > > If we would agree to apply Python rules to C, > then we could also use this excerpt from PEP 8: > > |o Don't compare boolean values to True or False using ==. > | > |Yes: if greeting: > |No: if greeting == True: > |Worse: if greeting is True: Or if we could agree to read PEP 7, there might be an example in there. Granted, it might not be an explicit recommendation, merely a tangential reference in a paragraph about braces, but it could be indicative. ChrisA From alb at nospam.chip.org Sun Oct 29 15:09:37 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 15:09:37 -0400 Subject: Invoking return through a function? In-Reply-To: <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> Message-ID: On 10/29/2017 02:13 PM, Rick Johnson wrote: > Alberto Riva wrote: >> Rick Johnson wrote: >>> Alberto Riva wrote: > > [...] > >>>> In a language like Lisp >>> >>> Python is nothing like Lisp, and for good reason! >> >> I would disagree with this. Actually, it's the most Lisp- >> like language I've encountered in my 25 years of writing >> software. > > Really? I'm sorry that you have only encountered a few > "academic languages". Do some googling, there are literally > thousands of them in the wilds. I'm sorry, I'm not sure I understand what you mean. I've been exposed to quite a few programming languages over the years, and my personal assessment (based on the features I consider important from my own personal point of view) is that Python is the closest to Lisp of the ones that I would consider using. It's definitely closer to Lisp than C, or Java, wouldn't you agree? I don't see where you get the idea that I was only exposed to "academic languages". >> It has closures, functions are first-class objects, it's >> interactive, highly reflective, variables are not strongly >> typed... > > Congratulations. You've just described Ruby. And an apple- > cart full of other languages as well. OK, so? Does that invalidate my statement? And even if it did, that has almost nothing to do with my original question. >> I don't consider myself a fanboy - I've used Lisp for most >> of my career, and now I happily use Python for 80% of what >> I do, even if it means putting up with the occasional >> limitation, as in this case. > > (not that i'm defending Python here...) but how do you know > that this so-called missing "feature" is a limitation? Because (again in my own personal opinion) Lisp macros are wonderful, they increase programmer productivity enormously, and not having them when I'm programming in other languages is a limitation from my point of view, since I enjoy using them so much. > How many other languages use this "feature"? Almost none, unfortunately. So? > And how many > programmers swear they could not live without it? We need > stats; hyperlinks; and eye witness testimonials. Heck, even > a random YouTube vid would be helpful. I don't see why. I'm just talking about something that I personally feel the need for, I'm not trying to convert anyone. >> Why "thankfully"? Having macros in Python would be great. >> Of course it's not possible because of the syntax, but it >> would be great nevertheless :) > > To me, macros would seem to violate Python's "explicit is > better than implicit" philosophy. But that's exactly why I would like to be able to use macros. I think that being able to write "return if this happens" is much more explicit than having to write the full if statement, every time. The idea is that you abstract a pattern giving it a name, so every time you see that name you know immediately what's going to happen, without having to decode the conditional in your mind. >>> Possibly. But you've already poo-pooed the easy and >>> obvious solution of using conditional logic. >> >> Not really. It's obvious that I'm going to use a >> conditional, I was just asking for a way to avoid having to >> type the same conditional over and over. Computers are >> meant to do work for you, after all, aren't they? :) > > A reasonable argument, i admit. But again, here in the > peaceful fields of "Pythonia", most of us feel that > "explicit is better than implicit". And why? Well, because > code is read more often than it is written, and as such, > explicit code will be of enormous help during the debugging > process. Sure, you may save yourself a few key-strokes with > features like macros, but at what cost? No action is without > consequences, dontchaknow? I agree completely. Maybe we're using two different definitions of the word macro (I'm not sure how familiar you are with Lisp macros), but in my view their biggest advantage is that they make the code more readable and more explicit. And the issue of debugging is an orthogonal one, good development environments allow you to work on the original and the macro-expanded code equally easily. Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From ned at nedbatchelder.com Sun Oct 29 15:30:50 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sun, 29 Oct 2017 15:30:50 -0400 Subject: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> Message-ID: <39e45394-7366-43a0-c570-a58699a4a2c8@nedbatchelder.com> On 10/29/17 3:09 PM, Alberto Riva wrote: > On 10/29/2017 02:13 PM, Rick Johnson wrote: >> Alberto Riva wrote: >>> Rick Johnson wrote: >>>> Alberto Riva wrote: >> >> [...] >>>>> In a language like Lisp >>>> >>>> Python is nothing like Lisp, and for good reason! >>> >>> I would disagree with this. Actually, it's the most Lisp- >>> like language I've encountered in my 25 years of writing >>> software. >> >> Really? I'm sorry that you have only encountered a few >> "academic languages". Do some googling, there are literally >> thousands of them in the wilds. > > I'm sorry, I'm not sure I understand what you mean. I've been exposed > to quite a few programming languages over the years, and my personal > assessment (based on the features I consider important from my own > personal point of view) is that Python is the closest to Lisp of the > ones that I would consider using. It's definitely closer to Lisp than > C, or Java, wouldn't you agree? I don't see where you get the idea > that I was only exposed to "academic languages". You don't have to respond to Rick.? He specializes in sarcasm and sneering.? He usually goes by "Ranting Rick."? This has been fairly tame for his postings, but it still doesn't require a response :) --Ned. From bc at freeuk.com Sun Oct 29 16:01:00 2017 From: bc at freeuk.com (bartc) Date: Sun, 29 Oct 2017 20:01:00 +0000 Subject: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> Message-ID: On 29/10/2017 17:02, Alberto Riva wrote: > On 10/29/2017 12:21 PM, Rick Johnson wrote: > In this specific case, I wanted to invoke a return if the key doesn't > exist (as I wrote in my post). But this is not strictly relevant, my > question was more general. To phrase it in yet another way: is there > *any other way* to invoke return on a function, except by explicitly > writing "return" in its body? The proposal seems flawed. Suppose function F() calls G(), and G invokes a double return, not only from G but from F too. That suggests that G knows something about how it's going to be called, and knows that doing so isn't going to cause a problem because F was in the middle of something that needs to be completed before it returns. In general, a function like G() could be invoked from 100 different call-sites all in different circumstances. Furthermore, those call-sites may be in code called from yet 100 other locations. (It may also be necessary to consider what happens when G() isn't called from a function. Or G calls itself recursively. Or G() appears in a 'return' expression. Or deep within a nested expression. Or when F() is itself called from within an expression, and is expected to have a certain return value.) This may be why the other proposal I mentioned used it within nested functions so at least adding some constraints. It is true that Python has exceptions that can do similar things, but the clue there is in the word 'exception'. It does seem that a lot of code uses exceptions which are routinely triggered. -- bartc From rantingrickjohnson at gmail.com Sun Oct 29 16:35:06 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Sun, 29 Oct 2017 13:35:06 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> Message-ID: <6c5fdcfb-c724-4837-8313-b78d55f53a9f@googlegroups.com> Alberto Riva wrote: > Rick Johnson wrote: > > Alberto Riva wrote: > >> Rick Johnson wrote: > >>> Alberto Riva wrote: > > [...] > > > > > > > > > > In a language like Lisp > > > > > > > > Python is nothing like Lisp, and for good reason! > > > > > > I would disagree with this. Actually, it's the most Lisp- > > > like language I've encountered in my 25 years of writing > > > software. > > > > Really? I'm sorry that you have only encountered a few > > "academic languages". Do some googling, there are > > literally thousands of them in the wilds. > > I'm sorry, I'm not sure I understand what you mean. I've > been exposed to quite a few programming languages over the > years, and my personal assessment (based on the features I > consider important from my own personal point of view) is > that Python is the closest to Lisp of the ones that I would > consider using. It's definitely closer to Lisp than C, or > Java, wouldn't you agree? It depends on what metrics you use. Once you remove all the type declarations from C and Java source, all three look very similar, with Python being the "cleanest" syntax of all. OTOH, when i look at Lisp, all i see is a language that was designed for easy machine parsing, not for the human. But that's just a syntactical comparison. Digging deeper, Python and Java are both strongly typed. And with Python's multi-paradigm nature, it is both equally useful in OOP (aka: Java) or imperative (aka: C) style. Aside from static type checking and some low level distractions, Python is very much like both of these languages. In fact, i was familiar with both Java and C before i ever became aware of Python. And the transition felt very natural to me. > I don't see where you get the idea that I was only exposed > to "academic languages". Just a lucky guess, i suppose. > > > It has closures, functions are first-class objects, it's > > > interactive, highly reflective, variables are not > > > strongly typed... > > > > Congratulations. You've just described Ruby. And an apple- > > cart full of other languages as well. > > OK, so? Does that invalidate my statement? And even if it > did, that has almost nothing to do with my original > question. But since your assertion of: "Python, like Lisp, has X,Y and Z features in common" and furthermore that: "I cannot find another language that has exactly these X, Y and Z features", was used by you as a justification for this feature request. That's why i had to attack it. Because it was false. > > > I don't consider myself a fanboy - I've used Lisp for > > > most of my career, and now I happily use Python for 80% > > > of what I do, even if it means putting up with the > > > occasional limitation, as in this case. > > > > (not that i'm defending Python here...) but how do you > > know that this so-called missing "feature" is a > > limitation? > > Because (again in my own personal opinion) Lisp macros are > wonderful, they increase programmer productivity > enormously, and not having them when I'm programming in > other languages is a limitation from my point of view, > since I enjoy using them so much. Well, i can understand your enthusiasm, i myself become highly animated about certain topics (just ask my CO-EDs!). But you also need to understand that i am person of _reasoned_ principles. And i try not to violate my principles just because some random peddler shows-up one day on my front porch with a slick sales pitch for the next generation of vacuum cleaners. My vacuum cleaner works just fine, thank you very much, and until it stops working, or until you convince me that purchasing a new vacuum will bring me a good return on my investment, and, furthermore, will not violate my beloved principles, then i will consider your product. However, so far, your sales pitch and demonstrasion have left much to be desired. And i i'm not mistaken, i believe you machine missed a few Doritos crumbs under that coffee table over there. Hmm... > > How many other languages use this "feature"? > > Almost none, unfortunately. So? Which raises a red flag for me. If this feature is as great as you say, then why has no other language adopted it? Are macros a recent addition to Lisp? As in: "Yesterday"? > > And how many programmers swear they could not live without > > it? We need stats; hyperlinks; and eye witness > > testimonials. Heck, even a random YouTube vid would be > > helpful. > > I don't see why. I'm just talking about something that I > personally feel the need for, I'm not trying to convert > anyone. It would help us decide if the feature is worth adding. If only one person, or a handful of people use a feature, then what is the benefit of spending multiple hours writing, debugging and maintaining it? Adding feature to a language is a very risky process that should never be done hastefully. Only after careful consideration, and majority support from the community, should new feature be added. Would you agree? > > > Why "thankfully"? Having macros in Python would be great. > > > Of course it's not possible because of the syntax, but it > > > would be great nevertheless :) > > > > To me, macros would seem to violate Python's "explicit is > > better than implicit" philosophy. > > But that's exactly why I would like to be able to use > macros. I think that being able to write "return if this > happens" is much more explicit than having to write the > full if statement, every time. The idea is that you > abstract a pattern giving it a name, so every time you see > that name you know immediately what's going to happen, > without having to decode the conditional in your mind. Oh, you mean something like this? # Python2.x (with print function enabled) >>> value = 5 >>> isValueLessThanTenAndGreaterThanfour = (value < 10 and value > 4) >>> myList = range(10, 100, 10) >>> isValueInMyList = myList.count(value)>0 >>> isValueAFloat = isinstance(value, float) >>> if isValueAFloat: ... print('value is a float!') ... elif isValueInMyList: ... print('value is in myList!') ... elif isValueLessThanTenAndGreaterThanfour: ... print('value is less than 10 and greater than four!') ... else: ... print('Where's a Dorito when you need one? o_O') ... value is less than 10 and greater than four! > > > > Possibly. But you've already poo-pooed the easy and > > > > obvious solution of using conditional logic. > > > > > > Not really. It's obvious that I'm going to use a > > > conditional, I was just asking for a way to avoid having > > > to type the same conditional over and over. Computers are > > > meant to do work for you, after all, aren't they? :) > > > > A reasonable argument, i admit. But again, here in the > > peaceful fields of "Pythonia", most of us feel that > > "explicit is better than implicit". And why? Well, because > > code is read more often than it is written, and as such, > > explicit code will be of enormous help during the > > debugging process. Sure, you may save yourself a few key- > > strokes with features like macros, but at what cost? No > > action is without consequences, dontchaknow? > > I agree completely. Maybe we're using two different > definitions of the word macro (I'm not sure how familiar > you are with Lisp macros), but in my view their biggest > advantage is that they make the code more readable and more > explicit. I believe you have totally different definition of "explicit" than i (and possibly most people). You use "explicit" to mean what i understand as being "idiomatic". Or possibly, and more accurately: "idiosyncratic". You have a particular way of doing things. A way that is comfortable for _you_. And there is nothing wrong with that. But one thing you must remember is that Python is not Lisp. And if a feature is to be borrowed from Lisp, then such feature should not be allowed to violate certain principles of Python's core design philosophy. These principles are: >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! And if we cannot find a way for your feature to _not_ violate these fine principles, then it shouldn't even be considered. > And the issue of debugging is an orthogonal one, > good development environments allow you to work on the > original and the macro-expanded code equally easily. I think in many languages, especially Lisp (a language who's syntax is not very human friendly), adding the old "a developement environment cures all ills" to your argument in an attempt to soften the blow created by the destructive feature is reasonable. But again, and i hate to sound like a broken record here, Python is not Lisp. From martin.schoon at gmail.com Sun Oct 29 17:03:37 2017 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 29 Oct 2017 21:03:37 GMT Subject: Repairing Python installation? References: Message-ID: Den 2017-10-29 skrev Chris Warrick : > On 29 October 2017 at 18:11, Martin Sch??n wrote: >> I have installed Python 3 virtualenv and Nikola according to those >> instructions. I now have a working Nikola but I sure don't know what >> I am doing :-) How do I get back into that nikola python environment >> next time? > > cd into your virtualenv directory and run `source bin/activate`. > Thanks /Martin From skybuck2000 at hotmail.com Sun Oct 29 17:29:34 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Sun, 29 Oct 2017 14:29:34 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> Message-ID: <8d38fe87-c322-4e16-9cc8-8b300056d49e@googlegroups.com> I am going to issue a warning about all of this software: SandSifter, Linux Mint 18.2 and install-apt and for windows: git For now I suspect running two instances of SandSifter at same time on Linux Mint 18.2 caused file system corruption as can be seen on these three screenshots also checkdisk log file is included in web folder: http://www.skybuck.org/SandSifter/unzipped/risks/ Possible causes of file system corruptions: 1. Running two instances of SandSifter + Linux Mint 18.2 2. Git on Windows 3. Perhaps a problem was already with file system. 4. BIOS Corruption in recent past. 5. Spark when other person connected laptop to power output... there was a spark. 6. FireFox corruption while browsing or extracting files ! Google detected cooky corruption... so FireFox is also a prime suspect ! 7. Capstone disassembler Be carefull ! I still have to investigate checkdisk log further though ! ;) Bye, Skybuck. From alb at nospam.chip.org Sun Oct 29 17:31:23 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Sun, 29 Oct 2017 17:31:23 -0400 Subject: Invoking return through a function? In-Reply-To: <6c5fdcfb-c724-4837-8313-b78d55f53a9f@googlegroups.com> References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> <6c5fdcfb-c724-4837-8313-b78d55f53a9f@googlegroups.com> Message-ID: As suggested by Ned I'm going to stop replying after pointing out just a couple of things. The most important one is the following: > But since your assertion of: "Python, like Lisp, has X,Y and > Z features in common" and furthermore that: "I cannot find > another language that has exactly these X, Y and Z > features", was used by you as a justification for this > feature request. I was not requesting any feature. I was just asking this group (that I assume includes people who are more experienced in Python than I am) if there was a way of doing something using the tools currently provided by Python. That was all. Then I mentioned the fact that I'm used to a language in which this "something" is not necessary because you can get the same effect in a cleaner and more readable way using macros, but I was absolutely NOT advocating for the addition of Lisp-like macros to Lisp. I know perfectly well that it would be impossible, because they can only work in a language that has Lisp-like syntax (which, incidentally, I find much easier to read than Java syntax, for example; these things are highly subjective). And indeed, the fact that you thought I was asking for this shows that you don't really know what Lisp macros are, otherwise you would know that they're impossible to reproduce in Python. So I apologize for any misunderstandings; I was not trying to sell anything to anyone, least of all vacuum cleaners. I was just asking for information. Thanks, Alberto On 10/29/2017 04:35 PM, Rick Johnson wrote: > Alberto Riva wrote: >> Rick Johnson wrote: >>> Alberto Riva wrote: >>>> Rick Johnson wrote: >>>>> Alberto Riva wrote: >>> [...] >>>>>> >>>>>> In a language like Lisp >>>>> >>>>> Python is nothing like Lisp, and for good reason! >>>> >>>> I would disagree with this. Actually, it's the most Lisp- >>>> like language I've encountered in my 25 years of writing >>>> software. >>> >>> Really? I'm sorry that you have only encountered a few >>> "academic languages". Do some googling, there are >>> literally thousands of them in the wilds. >> >> I'm sorry, I'm not sure I understand what you mean. I've >> been exposed to quite a few programming languages over the >> years, and my personal assessment (based on the features I >> consider important from my own personal point of view) is >> that Python is the closest to Lisp of the ones that I would >> consider using. It's definitely closer to Lisp than C, or >> Java, wouldn't you agree? > > It depends on what metrics you use. Once you remove all the > type declarations from C and Java source, all three look very > similar, with Python being the "cleanest" syntax of all. > OTOH, when i look at Lisp, all i see is a language that was > designed for easy machine parsing, not for the human. But > that's just a syntactical comparison. Digging deeper, Python > and Java are both strongly typed. And with Python's > multi-paradigm nature, it is both equally useful in OOP (aka: Java) > or imperative (aka: C) style. Aside from static type checking and > some low level distractions, Python is very much like both of > these languages. In fact, i was familiar with both Java and > C before i ever became aware of Python. And the transition > felt very natural to me. > >> I don't see where you get the idea that I was only exposed >> to "academic languages". > > Just a lucky guess, i suppose. > >>>> It has closures, functions are first-class objects, it's >>>> interactive, highly reflective, variables are not >>>> strongly typed... >>> >>> Congratulations. You've just described Ruby. And an apple- >>> cart full of other languages as well. >> >> OK, so? Does that invalidate my statement? And even if it >> did, that has almost nothing to do with my original >> question. > > But since your assertion of: "Python, like Lisp, has X,Y and > Z features in common" and furthermore that: "I cannot find > another language that has exactly these X, Y and Z > features", was used by you as a justification for this > feature request. That's why i had to attack it. Because it > was false. > >>>> I don't consider myself a fanboy - I've used Lisp for >>>> most of my career, and now I happily use Python for 80% >>>> of what I do, even if it means putting up with the >>>> occasional limitation, as in this case. >>> >>> (not that i'm defending Python here...) but how do you >>> know that this so-called missing "feature" is a >>> limitation? >> >> Because (again in my own personal opinion) Lisp macros are >> wonderful, they increase programmer productivity >> enormously, and not having them when I'm programming in >> other languages is a limitation from my point of view, >> since I enjoy using them so much. > > Well, i can understand your enthusiasm, i myself become > highly animated about certain topics (just ask my CO-EDs!). > But you also need to understand that i am person of > _reasoned_ principles. And i try not to violate my > principles just because some random peddler shows-up one day > on my front porch with a slick sales pitch for the next > generation of vacuum cleaners. My vacuum cleaner works just > fine, thank you very much, and until it stops working, or > until you convince me that purchasing a new vacuum will > bring me a good return on my investment, and, furthermore, > will not violate my beloved principles, then i will consider > your product. However, so far, your sales pitch and > demonstrasion have left much to be desired. And i i'm not > mistaken, i believe you machine missed a few Doritos crumbs > under that coffee table over there. Hmm... > >>> How many other languages use this "feature"? >> >> Almost none, unfortunately. So? > > Which raises a red flag for me. If this feature is as great > as you say, then why has no other language adopted it? Are > macros a recent addition to Lisp? As in: "Yesterday"? > >>> And how many programmers swear they could not live without >>> it? We need stats; hyperlinks; and eye witness >>> testimonials. Heck, even a random YouTube vid would be >>> helpful. >> >> I don't see why. I'm just talking about something that I >> personally feel the need for, I'm not trying to convert >> anyone. > > It would help us decide if the feature is worth adding. If > only one person, or a handful of people use a feature, then > what is the benefit of spending multiple hours writing, > debugging and maintaining it? Adding feature to a language > is a very risky process that should never be done > hastefully. Only after careful consideration, and majority > support from the community, should new feature be added. > Would you agree? > >>>> Why "thankfully"? Having macros in Python would be great. >>>> Of course it's not possible because of the syntax, but it >>>> would be great nevertheless :) >>> >>> To me, macros would seem to violate Python's "explicit is >>> better than implicit" philosophy. >> >> But that's exactly why I would like to be able to use >> macros. I think that being able to write "return if this >> happens" is much more explicit than having to write the >> full if statement, every time. The idea is that you >> abstract a pattern giving it a name, so every time you see >> that name you know immediately what's going to happen, >> without having to decode the conditional in your mind. > > Oh, you mean something like this? > > # Python2.x (with print function enabled) > >>> value = 5 > >>> isValueLessThanTenAndGreaterThanfour = (value < 10 and value > 4) > >>> myList = range(10, 100, 10) > >>> isValueInMyList = myList.count(value)>0 > >>> isValueAFloat = isinstance(value, float) > >>> if isValueAFloat: > ... print('value is a float!') > ... elif isValueInMyList: > ... print('value is in myList!') > ... elif isValueLessThanTenAndGreaterThanfour: > ... print('value is less than 10 and greater than four!') > ... else: > ... print('Where's a Dorito when you need one? o_O') > ... > value is less than 10 and greater than four! > > >>>>> Possibly. But you've already poo-pooed the easy and >>>>> obvious solution of using conditional logic. >>>> >>>> Not really. It's obvious that I'm going to use a >>>> conditional, I was just asking for a way to avoid having >>>> to type the same conditional over and over. Computers are >>>> meant to do work for you, after all, aren't they? :) >>> >>> A reasonable argument, i admit. But again, here in the >>> peaceful fields of "Pythonia", most of us feel that >>> "explicit is better than implicit". And why? Well, because >>> code is read more often than it is written, and as such, >>> explicit code will be of enormous help during the >>> debugging process. Sure, you may save yourself a few key- >>> strokes with features like macros, but at what cost? No >>> action is without consequences, dontchaknow? >> >> I agree completely. Maybe we're using two different >> definitions of the word macro (I'm not sure how familiar >> you are with Lisp macros), but in my view their biggest >> advantage is that they make the code more readable and more >> explicit. > > I believe you have totally different definition of "explicit" than i > (and possibly most people). You use "explicit" to mean what > i understand as being "idiomatic". Or possibly, and more > accurately: "idiosyncratic". You have a particular way of > doing things. A way that is comfortable for _you_. And there > is nothing wrong with that. But one thing you must remember > is that Python is not Lisp. And if a feature is to be > borrowed from Lisp, then such feature should not be allowed > to violate certain principles of Python's core design > philosophy. These principles are: > > >>> import this > The Zen of Python, by Tim Peters > > Beautiful is better than ugly. > Explicit is better than implicit. > Simple is better than complex. > Complex is better than complicated. > Flat is better than nested. > Sparse is better than dense. > Readability counts. > Special cases aren't special enough to break the rules. > Although practicality beats purity. > Errors should never pass silently. > Unless explicitly silenced. > In the face of ambiguity, refuse the temptation to guess. > There should be one-- and preferably only one --obvious way to do it. > Although that way may not be obvious at first unless you're Dutch. > Now is better than never. > Although never is often better than *right* now. > If the implementation is hard to explain, it's a bad idea. > If the implementation is easy to explain, it may be a good idea. > Namespaces are one honking great idea -- let's do more of those! > > And if we cannot find a way for your feature to _not_ > violate these fine principles, then it shouldn't even be > considered. > >> And the issue of debugging is an orthogonal one, >> good development environments allow you to work on the >> original and the macro-expanded code equally easily. > > I think in many languages, especially Lisp (a language who's > syntax is not very human friendly), adding the old "a > developement environment cures all ills" to your argument in > an attempt to soften the blow created by the destructive > feature is reasonable. But again, and i hate to sound like a > broken record here, Python is not Lisp. > -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From skybuck2000 at hotmail.com Sun Oct 29 18:11:32 2017 From: skybuck2000 at hotmail.com (skybuck2000 at hotmail.com) Date: Sun, 29 Oct 2017 15:11:32 -0700 (PDT) Subject: Sandsifter software finds hidden instructions inside processors. In-Reply-To: <8d38fe87-c322-4e16-9cc8-8b300056d49e@googlegroups.com> References: <54c5080a-f3d2-45d3-8b62-ccadfe7fa16a@googlegroups.com> <8d38fe87-c322-4e16-9cc8-8b300056d49e@googlegroups.com> Message-ID: I do remember a very rare and extreme Windows 7 system hang not so long ago. It was probably caused by having to many FireFox tabs open... and somehow a website/FireFox managed to "hang windows 7". Windows 7 might have been busy trying to write data to the harddisk and somehow during that process it hang. After waiting a bit I had no choice but to press the "reset" button. I think this is what might have caused this file system corruption. A combination of "FireFox memory hogging" and "Windows 7 writing to disk", possibly FireFox cache or cookie related. Bye, Skybuck. From dvl at psu.edu Sun Oct 29 19:28:36 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Sun, 29 Oct 2017 19:28:36 -0400 Subject: Coding style in CPython implementation In-Reply-To: mailman.714.1509286804.1486.python-list@python.org References: Message-ID: <1509319716l.31391954l.0l@psu.edu> NOTE: The case in question was never comparing to True; it was comparing to NULL. There is no "No: if x == None" below, because None is not Boolean. Similarly comparing a pointer to NULL is not the same as comparing it to a Boolean. So I would favor the "Explicit is better than Implicit" in the example cited. Roger Christman Pennsylvania State University On Sun, Oct 29, 2017, Setfan Ram wrote: > =?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?= writes: >>I guess the following parts from "Zen of Python" apply to this case: > > If we would agree to apply Python rules to C, > then we could also use this excerpt from PEP 8: > >|o Don't compare boolean values to True or False using ==. >| >|Yes: if greeting: >|No: if greeting == True: >|Worse: if greeting is True: > > From steve+python at pearwood.info Sun Oct 29 20:42:31 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 30 Oct 2017 11:42:31 +1100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> Message-ID: <59f6757a$0$18560$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 03:35 am, Alberto Riva wrote: > On 10/29/2017 10:35 AM, Steve D'Aprano wrote: [...] >> You mean *less* explicit. "checkKey" gives absolutely no hint that it >> causes the current function to return. > > That's just because I used a name that was too generic in my example. I > can call it "returnIfKeyMissing", and then it would be clear. No it wouldn't. That sounds like it is the "returnIfKeyMissing" function that returns if the key is missing, and presumably pauses forever, never returning, if the key exists. Of course we can *learn* that this is not the case, by reading the docs (if it has any) or the source code (if it is available). But what of every other function, any one of which might have been named like your first example: checkKey We cannot rely on naming conventions, because people won't always follow it. *Any* function could potentially return early from the caller. But honestly, if you're prepared to write: returnIfKeyMissing(dict, key) that's LONGER than: if key not in dict: return Since your motive appears to be the wish to save typing at the expense of readability and maintainability, then I expect that *in practice* you wouldn't use "returnIfKeyMissing" but would prefer a less explicit, shorter name like "checkKey". [...] >> You really should re-think your strategy. Your suggestion, if possible, >> would lead to difficult to maintain code where you couldn't easily tell >> where the exit points of a function where. > > But again, it's just a naming problem. It *really isn't* a naming problem. Being able to force the caller to return is a kind of GOTO, and it violates both the letter and the spirit of the principle that functions should have "one entry, one exit". Of course we already violate the second part of that, by allowing multiple returns. But that doesn't mean that any other violation should be acceptable. Allowing functions to force their caller to exit allows them to violate the caller's post-condition contracts. But even if it were just a naming problem, you under-estimate its magnitude. The word "just" is not justified here: even treated as a naming problem, it is a big problem. Not an end-of-the-world problem, but still big enough. > Something like returnIfKeyMissing > would make it easy to tell where the exit points are. No it wouldn't, because not everyone is going to use such a clumsy, long naming convention, either out of ignorance or for some other reason. Maybe you're using a library where everything is named in Dutch, or Russian. Maybe the function was named "foo()" long ago, and has only subsequently gained the ability to force the caller to return but the name was never changed. > And as Bartc > pointed out, we already have this situation with exceptions, so it would > be nothing new. Pardon me, but it was *me* who pointed out the analogy with exceptions, not Bart. But this is different from an exception. The similarity is quite weak: - any function can raise an exception; - your hoped for feature would allow any function to cause the caller to return; But the differences are profound: - exceptions are an alternative to normal execution flow, they don't silently continue unless explicitly caught and silenced; - if you don't explicitly catch the exception, it is obvious when one occurs, because your program halts and you get an obvious stacktrace; - your suggested force-caller-to-return would silently alter the control flow of the caller, without warning or signal that it had happened. That's a very significant difference. > Indeed, what I'm asking for could be accomplished by > wrapping the body of each function in a try/catch block for an ad-hoc > exception type. A terrible idea. Why not just write a "validate input" function that checks your input and returns True for valid and False for invalid, then: if not valid(args): return That's little harder to type than returnIfNotValid(args) and much more comprehensible. > But again, since the language doesn't have macros And this is the sort of misfeature why Guido is dead-set against Python ever getting them. Yes, we get it that Lisp macros are really, really powerful. But there are two bottle-necks in coding: - writing the code in the first place; - maintaining the code afterwards. Macros are aimed at simplifying the first, at the cost of the second. That's not a strategy Python follows. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Oct 29 20:54:03 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 30 Oct 2017 11:54:03 +1100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> Message-ID: <59f6782e$0$18586$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 02:35 am, Stefan Ram wrote: > So, I guess, we then must accept that sometimes - under > extraordinary circumstances - it should be tolerated to > write a function that is as long as six lines. An entire six lines... you cowboy! *wink* -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Oct 29 20:58:11 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 30 Oct 2017 11:58:11 +1100 Subject: Invoking return through a function? References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <8b098672-364e-44c6-a2c1-fb51c10cbf19@googlegroups.com> Message-ID: <59f67925$0$18586$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 06:09 am, Alberto Riva wrote: > But that's exactly why I would like to be able to use macros. I think > that being able to write "return if this happens" is much more explicit > than having to write the full if statement, every time. There have been proposals in the past for syntax similar to return value if condition as a short-hand for if condition: return value but since the only thing actually saved is a colon (or, at worst, a colon, a newline and an indent) they went nowhere. Sometimes the (hypothetical) gain in readability is outweighed by the increase in unwanted language/syntactic complexity. > The idea is that > you abstract a pattern giving it a name, so every time you see that name > you know immediately what's going to happen, without having to decode > the conditional in your mind. Too much of a good thing... That's fine for when you are writing the code the first time. You see a pattern, you give it a name, and you use that name over and over again. But when you come back to maintain the code in five years time, you have forgotten the name and the pattern. You see the name, not the pattern, because you're only focusing on a small part of the code.[1] What does the mystery name do? Unless you are very lucky and the function is very well named[2], you have to dig back through a chain of functions and macros to work it out. If the pattern was small and simple enough (like the example in this thread, "if key not in dict: return") having to dig back through its chain of functions and macros is more costly and difficult. Having too many named concepts to understand is as big a cognitive burden as having too few. [1] Software maintenance is harder, less rewarding and simply less fun than writing code in the first place. [2] There are only two hard problems in computer science: cache invalidation, and naming things. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jobmattcon at gmail.com Sun Oct 29 21:48:24 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 18:48:24 -0700 (PDT) Subject: why it append one more letter after decode? Message-ID: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> def leftrotate(l, n): return l[n:] + l[:n] def rightrotate(l, n): return l[-n:] + l[:-n] def encode(s, k, kk): l = [ord(i) for i in s] return leftrotate(''.join([chr(i + k) for i in l]), kk) def decode(s, k, kk): l = [ord(i) for i in rightrotate(s, kk)] return ''.join([chr(i - k) for i in l]) yesterday i add above code and run it with batch file it can decode a correct password then i install cx_freeze to produce executable file but today when i run it, i see the source of encrypted password is correct but the decode one, it append one more letter Y at the end of string why? is cx_freeze change the source python 2.7? From jobmattcon at gmail.com Sun Oct 29 21:50:50 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 18:50:50 -0700 (PDT) Subject: why it append one more letter after decode? In-Reply-To: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> Message-ID: if run these function to decode in python interactive console, it can decode correct, but when run with a big project, it append a letter Y On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: > def leftrotate(l, n): > return l[n:] + l[:n] > > def rightrotate(l, n): > return l[-n:] + l[:-n] > > def encode(s, k, kk): > l = [ord(i) for i in s] > return leftrotate(''.join([chr(i + k) for i in l]), kk) > > def decode(s, k, kk): > l = [ord(i) for i in rightrotate(s, kk)] > return ''.join([chr(i - k) for i in l]) > > > yesterday i add above code and run it with batch file > it can decode a correct password > > then i install cx_freeze to produce executable file > > but today when i run it, i see the source of encrypted password is correct > but the decode one, it append one more letter Y at the end of string > > why? > is cx_freeze change the source python 2.7? From jobmattcon at gmail.com Sun Oct 29 22:20:31 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 19:20:31 -0700 (PDT) Subject: ValueError: Error 3 while encrypting in ECB mode Message-ID: >>> from Crypto.Cipher import AES >>> >>> key = 'mysecretpassword' >>> plaintext = 'Secret Message A' >>> encobj = AES.new(key, AES.MODE_ECB) >>> ciphertext = encobj.encrypt("hello") Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\Crypto\Cipher\_mode_ecb.py", line 124, in encrypt raise ValueError("Error %d while encrypting in ECB mode" % result) ValueError: Error 3 while encrypting in ECB mode >>> ciphertext = encobj.encrypt("hellojalsdjflkdsjflds") Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\Crypto\Cipher\_mode_ecb.py", line 124, in encrypt raise ValueError("Error %d while encrypting in ECB mode" % result) ValueError: Error 3 while encrypting in ECB mode From stefanossofroniou542 at gmail.com Sun Oct 29 22:51:34 2017 From: stefanossofroniou542 at gmail.com (=?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?=) Date: Sun, 29 Oct 2017 19:51:34 -0700 (PDT) Subject: Coding style in CPython implementation In-Reply-To: References: <1509319716l.31391954l.0l@psu.edu> Message-ID: On Monday, October 30, 2017 at 2:35:13 AM UTC+2, ROGER GRAYDON CHRISTMAN wrote: > NOTE: The case in question was never comparing to True; it was comparing to > NULL. > > There is no "No: if x == None" below, because None is not Boolean. > Similarly comparing a pointer to NULL is not the same as comparing it to a > Boolean. > > So I would favor the "Explicit is better than Implicit" in the example cited. Thus, my 4 Zen points are accurately interconnected: Being simple and explicit with your code leads to elegant results that are indeed quite pleasant to read. > > Roger Christman > Pennsylvania State University > > On Sun, Oct 29, 2017, Setfan Ram wrote: > > =?UTF-8?B?zqPPhM6tz4bOsc69zr/PgiDOo8+Jz4bPgc6/zr3Or86/z4U=?= > >>I guess the following parts from "Zen of Python" apply to this case: > > > > If we would agree to apply Python rules to C, > > then we could also use this excerpt from PEP 8: > > > >|o Don't compare boolean values to True or False using ==. > >| > >|Yes: if greeting: > >|No: if greeting == True: > >|Worse: if greeting is True: > > > > From jobmattcon at gmail.com Sun Oct 29 23:03:46 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 20:03:46 -0700 (PDT) Subject: why it append one more letter after decode? In-Reply-To: References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> Message-ID: <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> i discover when it decode("\\",1,2) in interactive console is [ but decode in big project it see as decode(r"\\",1,2) [[ it double On Monday, October 30, 2017 at 9:51:01 AM UTC+8, Ho Yeung Lee wrote: > if run these function to decode in python interactive console, > it can decode correct, > > but when run with a big project, it append a letter Y > > > On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: > > def leftrotate(l, n): > > return l[n:] + l[:n] > > > > def rightrotate(l, n): > > return l[-n:] + l[:-n] > > > > def encode(s, k, kk): > > l = [ord(i) for i in s] > > return leftrotate(''.join([chr(i + k) for i in l]), kk) > > > > def decode(s, k, kk): > > l = [ord(i) for i in rightrotate(s, kk)] > > return ''.join([chr(i - k) for i in l]) > > > > > > yesterday i add above code and run it with batch file > > it can decode a correct password > > > > then i install cx_freeze to produce executable file > > > > but today when i run it, i see the source of encrypted password is correct > > but the decode one, it append one more letter Y at the end of string > > > > why? > > is cx_freeze change the source python 2.7? From ikorot01 at gmail.com Sun Oct 29 23:06:12 2017 From: ikorot01 at gmail.com (Igor Korot) Date: Sun, 29 Oct 2017 22:06:12 -0500 Subject: why it append one more letter after decode? In-Reply-To: References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> Message-ID: Hi, On Sun, Oct 29, 2017 at 8:50 PM, Ho Yeung Lee wrote: > > if run these function to decode in python interactive console, > it can decode correct, You probably changed the code somewhere... You just need to debug it and see what is happening... Thank you. > > but when run with a big project, it append a letter Y > > > On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: >> def leftrotate(l, n): >> return l[n:] + l[:n] >> >> def rightrotate(l, n): >> return l[-n:] + l[:-n] >> >> def encode(s, k, kk): >> l = [ord(i) for i in s] >> return leftrotate(''.join([chr(i + k) for i in l]), kk) >> >> def decode(s, k, kk): >> l = [ord(i) for i in rightrotate(s, kk)] >> return ''.join([chr(i - k) for i in l]) >> >> >> yesterday i add above code and run it with batch file >> it can decode a correct password >> >> then i install cx_freeze to produce executable file >> >> but today when i run it, i see the source of encrypted password is correct >> but the decode one, it append one more letter Y at the end of string >> >> why? >> is cx_freeze change the source python 2.7? > > -- > https://mail.python.org/mailman/listinfo/python-list From jobmattcon at gmail.com Sun Oct 29 23:14:20 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 20:14:20 -0700 (PDT) Subject: why it append one more letter after decode? In-Reply-To: <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> Message-ID: it seems that it read csv file from "\\" to "\\\\" On Monday, October 30, 2017 at 11:03:57 AM UTC+8, Ho Yeung Lee wrote: > i discover when > it > decode("\\",1,2) > in interactive console > is [ > > but decode in big project > it see as > decode(r"\\",1,2) > [[ > > it double > > On Monday, October 30, 2017 at 9:51:01 AM UTC+8, Ho Yeung Lee wrote: > > if run these function to decode in python interactive console, > > it can decode correct, > > > > but when run with a big project, it append a letter Y > > > > > > On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: > > > def leftrotate(l, n): > > > return l[n:] + l[:n] > > > > > > def rightrotate(l, n): > > > return l[-n:] + l[:-n] > > > > > > def encode(s, k, kk): > > > l = [ord(i) for i in s] > > > return leftrotate(''.join([chr(i + k) for i in l]), kk) > > > > > > def decode(s, k, kk): > > > l = [ord(i) for i in rightrotate(s, kk)] > > > return ''.join([chr(i - k) for i in l]) > > > > > > > > > yesterday i add above code and run it with batch file > > > it can decode a correct password > > > > > > then i install cx_freeze to produce executable file > > > > > > but today when i run it, i see the source of encrypted password is correct > > > but the decode one, it append one more letter Y at the end of string > > > > > > why? > > > is cx_freeze change the source python 2.7? From ikorot01 at gmail.com Sun Oct 29 23:15:48 2017 From: ikorot01 at gmail.com (Igor Korot) Date: Sun, 29 Oct 2017 22:15:48 -0500 Subject: why it append one more letter after decode? In-Reply-To: <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> Message-ID: Hi, On Sun, Oct 29, 2017 at 10:03 PM, Ho Yeung Lee wrote: > i discover when > it > decode("\\",1,2) > in interactive console > is [ > > but decode in big project > it see as > decode(r"\\",1,2) > [[ > > it double Those 2 lines are different. Thank you. > > On Monday, October 30, 2017 at 9:51:01 AM UTC+8, Ho Yeung Lee wrote: >> if run these function to decode in python interactive console, >> it can decode correct, >> >> but when run with a big project, it append a letter Y >> >> >> On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: >> > def leftrotate(l, n): >> > return l[n:] + l[:n] >> > >> > def rightrotate(l, n): >> > return l[-n:] + l[:-n] >> > >> > def encode(s, k, kk): >> > l = [ord(i) for i in s] >> > return leftrotate(''.join([chr(i + k) for i in l]), kk) >> > >> > def decode(s, k, kk): >> > l = [ord(i) for i in rightrotate(s, kk)] >> > return ''.join([chr(i - k) for i in l]) >> > >> > >> > yesterday i add above code and run it with batch file >> > it can decode a correct password >> > >> > then i install cx_freeze to produce executable file >> > >> > but today when i run it, i see the source of encrypted password is correct >> > but the decode one, it append one more letter Y at the end of string >> > >> > why? >> > is cx_freeze change the source python 2.7? > > -- > https://mail.python.org/mailman/listinfo/python-list From jobmattcon at gmail.com Sun Oct 29 23:33:26 2017 From: jobmattcon at gmail.com (Ho Yeung Lee) Date: Sun, 29 Oct 2017 20:33:26 -0700 (PDT) Subject: why it append one more letter after decode? In-Reply-To: References: <8d9143f5-6209-4321-9fe5-5a5ee7d375d6@googlegroups.com> <3950eeb0-470d-457d-b389-d58791f19354@googlegroups.com> Message-ID: <06bc8a8d-ffb4-4150-9b50-10168bfe9187@googlegroups.com> i fixed by replace("\\\\","\\") On Monday, October 30, 2017 at 11:16:02 AM UTC+8, Igor Korot wrote: > Hi, > > On Sun, Oct 29, 2017 at 10:03 PM, Ho Yeung Lee wrote: > > i discover when > > it > > decode("\\",1,2) > > in interactive console > > is [ > > > > but decode in big project > > it see as > > decode(r"\\",1,2) > > [[ > > > > it double > > Those 2 lines are different. > > Thank you. > > > > > On Monday, October 30, 2017 at 9:51:01 AM UTC+8, Ho Yeung Lee wrote: > >> if run these function to decode in python interactive console, > >> it can decode correct, > >> > >> but when run with a big project, it append a letter Y > >> > >> > >> On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: > >> > def leftrotate(l, n): > >> > return l[n:] + l[:n] > >> > > >> > def rightrotate(l, n): > >> > return l[-n:] + l[:-n] > >> > > >> > def encode(s, k, kk): > >> > l = [ord(i) for i in s] > >> > return leftrotate(''.join([chr(i + k) for i in l]), kk) > >> > > >> > def decode(s, k, kk): > >> > l = [ord(i) for i in rightrotate(s, kk)] > >> > return ''.join([chr(i - k) for i in l]) > >> > > >> > > >> > yesterday i add above code and run it with batch file > >> > it can decode a correct password > >> > > >> > then i install cx_freeze to produce executable file > >> > > >> > but today when i run it, i see the source of encrypted password is correct > >> > but the decode one, it append one more letter Y at the end of string > >> > > >> > why? > >> > is cx_freeze change the source python 2.7? > > > > -- > > https://mail.python.org/mailman/listinfo/python-list From rustompmody at gmail.com Mon Oct 30 00:23:21 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 29 Oct 2017 21:23:21 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> Message-ID: <7f4ed4dc-ad4b-43b6-b647-6004d3e959b4@googlegroups.com> On Sunday, October 29, 2017 at 9:52:01 PM UTC+5:30, Rick Johnson wrote: > On Sunday, October 29, 2017 at 9:19:03 AM UTC-5, Alberto Riva wrote: > > In a language like Lisp > > Python is nothing like Lisp, and for good reason! Sure, we > have a few lispers and functional fanboys who hang around > here, and sometimes Rustom can get a little preachy about > FP, but mostly, we tolerate the fanboyism -- so long as it's > not rabid fanboyism. Rick's personal comments are one of the standard entertainments of this list. Enjoy! The comments on FP are more problematic: - These misconceptions are more widespread than just Rick - They are right enough to be hard to refute - And wrong enough to cause confusion and misdirection - And irrelevant to (threads like) this one My super-short rejoiner to the FP-irrelevancy is: ?If there is one key takeaway from functional programming maybe it should be: "Truth is a function of time" ? In more detail: - Lisp (1960) was the second functional language; its predecessor was something called Formula Translator ? usually shortened to Fortran (c. 1957) - By the late 80s, many FPers had begun regard Lisp as a setback for functional programming. [Dont get me wrong: Ive enjoyed Scheme more than any other programming language. Its just that conflating Lisp and FP is an error (in 2017)] More historical details at http://blog.languager.org/2015/04/cs-history-1.html and sequel At a more conceptual level, people dont get that there are two dimensions - the apply-lambda axis ? usually called functional programming - the eval-quote axis ? which has precious little to do with FP (and is more relevant to your question) These two dimensions of power uniquely coincide in Lisp (Scheme). As you correctly (almost) point out - Python is very much a lisp? its semantic under-belly - However, syntactically its more like C/Java etc in the sense of having a rigid fixed-at-language-design-time syntax More at http://blog.languager.org/2013/08/applying-si-on-sicp.html For a question like macros (syntax-extensions) that sits between the two your question is interesting and I am not sure I know how to do it? I remember seeing something about this recently but my google-foo is failing me at the moment However I would look at - PEP 263 https://www.python.org/dev/peps/pep-0263/ ? Source Code Encodings? - Hooking into the codec module https://docs.python.org/3/library/codecs.html - hooking into ast module https://docs.python.org/3/library/ast.html An old attempt in this direction: http://code.activestate.com/recipes/546539/ You've already got the moral-hi-horse guys giving you the dope of why this is evil. Swallow if you like. My own lightweight suggestion would be that the natural pythonic way of unstructured exit is exceptions. Not as succinct as a tailormade syntax extension. But good enough IMHO PS I personally would be interested if you get this (syntax extension) running From steve+python at pearwood.info Mon Oct 30 00:40:08 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 30 Oct 2017 15:40:08 +1100 Subject: ValueError: Error 3 while encrypting in ECB mode References: Message-ID: <59f6ad2a$0$18614$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 01:20 pm, Ho Yeung Lee wrote: >>>> from Crypto.Cipher import AES >>>> >>>> key = 'mysecretpassword' >>>> plaintext = 'Secret Message A' >>>> encobj = AES.new(key, AES.MODE_ECB) >>>> ciphertext = encobj.encrypt("hello") > Traceback (most recent call last): > File "", line 1, in > File "C:\Python27\lib\site-packages\Crypto\Cipher\_mode_ecb.py", line 124, > in encrypt > raise ValueError("Error %d while encrypting in ECB mode" % result) > ValueError: Error 3 while encrypting in ECB mode What does the documentation for the Crypto.Cipher.AES object say? Did you try googling for the error message? https://duckduckgo.com/?q=Error+3+while+encrypting+in+ECB+mode https://www.google.com.au/search?q=Error+3+while+encrypting+in+ECB+mode which gives me this: https://stackoverflow.com/questions/14179784/python-encrypting-with-pycrypto-aes among other possible answers. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From lele at metapensiero.it Mon Oct 30 02:14:25 2017 From: lele at metapensiero.it (Lele Gaifax) Date: Mon, 30 Oct 2017 07:14:25 +0100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> Message-ID: <87mv49i2m6.fsf@metapensiero.it> ram at zedat.fu-berlin.de (Stefan Ram) writes: > There are many macro processors available, like the C macro > preprocessor, gpp, M4, or funnelweb. You can always use them > for your Python source (which, in this case, isn't exactly Python > source anymore). Even more to the point, MacroPy! See https://github.com/azazel75/macropy for a 3.5+ version. ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From zljubisic at gmail.com Mon Oct 30 03:44:49 2017 From: zljubisic at gmail.com (zljubisic at gmail.com) Date: Mon, 30 Oct 2017 00:44:49 -0700 (PDT) Subject: Pandas to_html cannot apply style Message-ID: <23275d24-8c7f-4e2c-9f5a-d4a9c18995ac@googlegroups.com> Hi, the following code never applies style and I cannot figure out why. Can someone please help? import pandas as pd def function2(row): if row.A == True: color = '#FF0000' else: color = '#00FF00' background_color = 'background-color: {}'.format(color) return [background_color] * len(row.values) idx = pd.date_range('01.01.2017', periods=7, freq='D') A = [False, True, True, False, True, False, True] B = np.random.randn(7) C = np.random.randn(7) data = { 'A' : [False, True, True, False, True, False, True], 'B' : np.random.randn(7), 'C' : np.random.randn(7) } df = pd.DataFrame(data, index=idx) df.style.apply(function2, axis=1) html = df.style.render() if '#FF0000' in html or '#00FF00' in html: print('style applied') else: print('style not applied') Regards. From auriocus at gmx.de Mon Oct 30 03:47:15 2017 From: auriocus at gmx.de (Christian Gollwitzer) Date: Mon, 30 Oct 2017 08:47:15 +0100 Subject: Invoking return through a function? In-Reply-To: <59f6757a$0$18560$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <59f6757a$0$18560$b1db1813$d948b532@news.astraweb.com> Message-ID: Am 30.10.17 um 01:42 schrieb Steve D'Aprano: > On Mon, 30 Oct 2017 03:35 am, Alberto Riva wrote: >> And as Bartc >> pointed out, we already have this situation with exceptions, so it would >> be nothing new. > > Pardon me, but it was *me* who pointed out the analogy with exceptions, not > Bart. But this is different from an exception. > > The similarity is quite weak: > > - any function can raise an exception; > > - your hoped for feature would allow any function to cause the > caller to return; For another data point, I'm regularly using another language, which does have this feature without macros - Tcl. In Tcl, you can return a control code alongside with the value which causes the caller to /return/, throw an /error/ or execute a /break/ or /continue/ statement. It is realised by additional options to "return"; translated into Python syntax it would look like: return ("somevalue", code=return) The caller can still intercept this return code using "catch". This feature can be used to implement one's own control structures, but it is rarely used in other contexts. It is there for completeness; in order to allow functions to be written which mimic exactly break, continue, error (="raise" in Python) and return. In practice, the only situation where it is regularly used is argument checking. Consider: def root(X): if X < 0: return("X must be positive", code=error) root(-1) With "raise" instead of "return", the stack trace points to the line containing "raise", but the actual error is at the call site. With "return code=error" the stack trace ends at the invocation of "root", where the real error is, and thus "root" would behave more like math.sqrt(), where you also don't get a stack trace pointing to the line of C code which throws the error. >> Indeed, what I'm asking for could be accomplished by >> wrapping the body of each function in a try/catch block for an ad-hoc >> exception type. > > A terrible idea. Why not just write a "validate input" function that checks > your input and returns True for valid and False for invalid, then: > > if not valid(args): > return > > That's little harder to type than > > returnIfNotValid(args) > > and much more comprehensible. > Indeed, that is way better for this use case. The OP really wants to abuse that feature. For statistics, I ran a few greps over the entire tcllib: return 16309 return -code error 2271 return -code continue 54 return -code break 7 return -code return 56 Returning an "error" is prevalent, which is mostly used for the case described above. Returning "break", "continue" and "return" is used in the guts of some parsing engines and in tricky control structures, but otherwise very rare. It is also used in Tk code to interrupt a series of bindings, and therefore also in Tkinter code (where the string "break" is returned) > >> But again, since the language doesn't have macros > > And this is the sort of misfeature why Guido is dead-set against Python ever > getting them. > > Yes, we get it that Lisp macros are really, really powerful. But there are two > bottle-necks in coding: > > - writing the code in the first place; > > - maintaining the code afterwards. > > Macros are aimed at simplifying the first, at the cost of the second. That's > not a strategy Python follows. Abusus non tollit usum. Python already has a macro-esque feature, namely decorators. "But everyone can write a decorator which does magic things and hide it under an @blabla...." - yes they can, but on the other hands they allow such wonderful things as asyncio, properties or the numba jit compiler. It is never a good argument to say "there is no feature X, because GvR is such a god that he knew the feature isn't needed, and see? we are doing well without it!". Unfortunately, that logical fallacy is heard quite often in this group. Christian From tjol at tjol.eu Mon Oct 30 04:03:05 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 30 Oct 2017 09:03:05 +0100 Subject: Pandas to_html cannot apply style In-Reply-To: <23275d24-8c7f-4e2c-9f5a-d4a9c18995ac@googlegroups.com> References: <23275d24-8c7f-4e2c-9f5a-d4a9c18995ac@googlegroups.com> Message-ID: <408d0ce7-15e6-e8c0-e5f7-68ab1d64027c@tjol.eu> On 30/10/17 08:44, zljubisic at gmail.com wrote: > Hi, > > the following code never applies style and I cannot figure out why. > Can someone please help? > > import pandas as pd > > def function2(row): > if row.A == True: > color = '#FF0000' > else: > color = '#00FF00' > > background_color = 'background-color: {}'.format(color) > > return [background_color] * len(row.values) > > idx = pd.date_range('01.01.2017', periods=7, freq='D') > A = [False, True, True, False, True, False, True] > B = np.random.randn(7) > C = np.random.randn(7) > > data = { 'A' : [False, True, True, False, True, False, True], > 'B' : np.random.randn(7), > 'C' : np.random.randn(7) > } > > df = pd.DataFrame(data, index=idx) > > df.style.apply(function2, axis=1) This is not an in-place operation: it returns a style which you can then render. style = df.style.apply(function2, axis=1) html = style.render() appears to work. > > html = df.style.render() > > if '#FF0000' in html or '#00FF00' in html: > print('style applied') > else: > print('style not applied') > > Regards. > From zljubisic at gmail.com Mon Oct 30 04:25:14 2017 From: zljubisic at gmail.com (zljubisic at gmail.com) Date: Mon, 30 Oct 2017 01:25:14 -0700 (PDT) Subject: pandas finding field difference between two dataframes Message-ID: Hi, I have to compare two pandas dataframes and find difference between each row. For example, in the code bellow, rows with index 0 and 3 are intentionally different. Row 0 is different in field A, and row 3 is different in field 3. After merging dataframes, I can concentrate to the dfm with dfm['_merge'] != 'both'. How to find which field is different? End result should be: row 0, A row 3, B Regards. import pandas as pd import numpy as np idx = pd.date_range('01.01.2017', periods=7, freq='D') A = [False, True, True, False, True, False, True] B = np.random.randn(7) C = np.random.randn(7) data1 = { 'A' : [False, True, True, False, True, False, True], 'B' : list(range(len(idx))), 'C' : list(range(len(idx), len(idx) + len(idx))) } dfl = pd.DataFrame(data1, index=idx) data2 = data1.copy() data2['A'][0] = True data2['B'][3] = 30 dfr = pd.DataFrame(data2, index=idx) # dfm = dfl.merge(dfr, indicator=True, how='outer', left_on=dfl.index, right_on = dfr.index) dfm = dfl.merge(dfr, indicator=True, how='outer') print(dfl) print(dfr) print(dfm) print(dfl.iloc[[0,3]]) print(dfr.iloc[[0,3]]) Output: A B C 2017-01-01 False 0 7 2017-01-02 True 1 8 2017-01-03 True 2 9 2017-01-04 False 3 10 2017-01-05 True 4 11 2017-01-06 False 5 12 2017-01-07 True 6 13 A B C 2017-01-01 True 0 7 2017-01-02 True 1 8 2017-01-03 True 2 9 2017-01-04 False 30 10 2017-01-05 True 4 11 2017-01-06 False 5 12 2017-01-07 True 6 13 A B C _merge 0 False 0 7 left_only 1 True 1 8 both 2 True 2 9 both 3 False 3 10 left_only 4 True 4 11 both 5 False 5 12 both 6 True 6 13 both 7 True 0 7 right_only 8 False 30 10 right_only Backend TkAgg is interactive backend. Turning interactive mode on. A B C 2017-01-01 False 0 7 2017-01-04 False 3 10 A B C 2017-01-01 True 0 7 2017-01-04 False 30 10 From zljubisic at gmail.com Mon Oct 30 04:43:11 2017 From: zljubisic at gmail.com (zljubisic at gmail.com) Date: Mon, 30 Oct 2017 01:43:11 -0700 (PDT) Subject: Pandas to_html cannot apply style In-Reply-To: References: <23275d24-8c7f-4e2c-9f5a-d4a9c18995ac@googlegroups.com> <408d0ce7-15e6-e8c0-e5f7-68ab1d64027c@tjol.eu> Message-ID: <9eff46a3-661e-4a4a-a199-eb3ff9923dc0@googlegroups.com> > This is not an in-place operation: it returns a style which you can then > render. > > style = df.style.apply(function2, axis=1) > html = style.render() > > appears to work. This was a missing link. Thank you very very much Thomas. Regards and best wishes. From davidgab283 at gmail.com Mon Oct 30 05:48:24 2017 From: davidgab283 at gmail.com (David Gabriel) Date: Mon, 30 Oct 2017 10:48:24 +0100 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: References: <20171027140819.GD8014@lutz-pc.ecm4u.intra> Message-ID: Dears, When I run this command I got this error message: ubuntu at orchestrateur:/tmp/pack$ virtualenv -p $(which python3.5) . Running virtualenv with interpreter /usr/local/sbin/. Traceback (most recent call last): File "/usr/bin/virtualenv", line 3, in virtualenv.main() File "/usr/lib/python2.7/dist-packages/virtualenv.py", line 784, in main popen = subprocess.Popen([interpreter, file] + sys.argv[1:], env=env) File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 13] Permission denied This error is also reproducible using sudo. Please advise how to fix it. Thanks in advance. Best regards 2017-10-28 14:33 GMT+02:00 David Gabriel : > I forget to precise that I am using pycharm. > And this issue is reproducible also using command line to run the code. > > Best regards > > 2017-10-28 14:31 GMT+02:00 David Gabriel : > >> Thanks so Lutz much for your reply. >> I am using python2.7 and I am running this code in an Openstack instance. >> I will apply your recommandation and let you know about the result ... >> >> Kind regards. >> >> 2017-10-27 16:13 GMT+02:00 Lutz Horn : >> >>> On Fri, Oct 27, 2017 at 03:56:39PM +0200, David Gabriel wrote: >>> > from packaging import version as pack_version >>> > ImportError: No module named packaging >>> > >>> > I googled it and I have found so many suggestions regarding updating >>> > 'pip' and installing python-setuptools but all of these did not fix >>> > this issue. >>> >>> So many questions: >>> >>> * What is your Python version? >>> * Do you use virtualenv? >>> * How? >>> * Did you install packaging in this virtualenv? >>> >>> Just one example of making this work: >>> >>> $ mkdir /tmp/pack >>> $ cd /tmp/pack >>> $ virtualenv -p $(which python3.5) . >>> Running virtualenv with interpreter /usr/bin/python3.5 >>> Using base prefix '/usr' >>> New python executable in /tmp/pack/bin/python3.5 >>> Also creating executable in /tmp/pack/bin/python >>> Installing setuptools, pkg_resources, pip, wheel...done. >>> $ source bin/activate >>> $ pip3 install packaging >>> Collecting packaging >>> Using cached packaging-16.8-py2.py3-none-any.whl >>> Collecting six (from packaging) >>> Using cached six-1.11.0-py2.py3-none-any.whl >>> Collecting pyparsing (from packaging) >>> Using cached pyparsing-2.2.0-py2.py3-none-any.whl >>> Installing collected packages: six, pyparsing, packaging >>> Successfully installed packaging-16.8 pyparsing-2.2.0 six-1.11.0 >>> $ python3.5 >>> Python 3.5.2 (default, Sep 14 2017, 22:51:06) >>> [GCC 5.4.0 20160609] on linux >>> Type "help", "copyright", "credits" or "license" for more >>> information. >>> >>> from packaging import version as pack_version >>> >>> >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >> >> > From zljubisic at gmail.com Mon Oct 30 06:03:23 2017 From: zljubisic at gmail.com (zljubisic at gmail.com) Date: Mon, 30 Oct 2017 03:03:23 -0700 (PDT) Subject: Pandas to_html cannot apply style In-Reply-To: References: <23275d24-8c7f-4e2c-9f5a-d4a9c18995ac@googlegroups.com> <408d0ce7-15e6-e8c0-e5f7-68ab1d64027c@tjol.eu> Message-ID: <674c6134-6938-4433-91e0-f8ffb5cbd6e4@googlegroups.com> > This is not an in-place operation: it returns a style which you can then > render. > > style = df.style.apply(function2, axis=1) > html = style.render() > > appears to work. After your suggestion, rows are properly colored, but now I have lost all table lines, font is smaller... Is there an option for keeping table format, and change only rows background color? Regards. From kirillbalunov at gmail.com Mon Oct 30 06:10:51 2017 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Mon, 30 Oct 2017 13:10:51 +0300 Subject: Performance of map vs starmap. Message-ID: Sometime ago I asked this question at SO [1], and among the responses received was paragraph: - `zip` re-uses the returned `tuple` if it has a reference count of 1 when the `__next__` call is made. - `map` build a new `tuple` that is passed to the mapped function every time a `__next__` call is made. Why can not `map` use the same approach as `zip`? Also it turns out that a faster solution looks not reasonable, since it requires additional calculations.. [1] https://stackoverflow.com/questions/46172018/perfomance- of-map-vs-starmap Thanks, - gdg From __peter__ at web.de Mon Oct 30 06:14:34 2017 From: __peter__ at web.de (Peter Otten) Date: Mon, 30 Oct 2017 11:14:34 +0100 Subject: from packaging import version as pack_version ImportError: No module named packaging References: <20171027140819.GD8014@lutz-pc.ecm4u.intra> Message-ID: David Gabriel wrote: > Dears, > > When I run this command I got this error message: > > ubuntu at orchestrateur:/tmp/pack$ virtualenv -p $(which python3.5) . > Running virtualenv with interpreter /usr/local/sbin/. > Traceback (most recent call last): > File "/usr/bin/virtualenv", line 3, in > virtualenv.main() > File "/usr/lib/python2.7/dist-packages/virtualenv.py", line 784, in main > popen = subprocess.Popen([interpreter, file] + sys.argv[1:], env=env) > File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ > errread, errwrite) > File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > OSError: [Errno 13] Permission denied > > > This error is also reproducible using sudo. When `which` does not find python3.5 your virtualenv invocation is effectively $ virtualenv -p . It should be clear that the current directory is usually not a python interpreter ;) > Please advise how to fix it. Provide the path to an actually existing interpreter. If you insist on which-craft test the command first: $ which python3.3 $ which python3.4 /usr/bin/python3.4 So on my system $(which python3.4) might work while $(which python3.3) will not. From voipdev.sourav at gmail.com Mon Oct 30 06:34:28 2017 From: voipdev.sourav at gmail.com (sourav voip) Date: Mon, 30 Oct 2017 16:04:28 +0530 Subject: request.post in If condition Message-ID: Hi All, I'm trying to hit request.post with condition using if-else as below... I;m posting the full script here...as I've tried declaring post url details tested with multiple places. From voipdev.sourav at gmail.com Mon Oct 30 06:42:27 2017 From: voipdev.sourav at gmail.com (sourav voip) Date: Mon, 30 Oct 2017 16:12:27 +0530 Subject: request.post in If condition In-Reply-To: References: Message-ID: Hi All, I'm trying to hit request.post with condition using if-else as below... I;m posting the full script here...as I've tried declaring post url details tested with multiple places.. If condition for disk utiliztion is working perfect ,however request.post is not hitting. Please suggest any hint/clue as I'm a new learner in python. Regards, Sourav ------------------------------------------------------------ ------------------------- #!/usr/bin/python import re,sys,commands import requests # Set the request parameters url = 'https://devxxxx.service-now.com/api/now/table/incident' # Eg. User name="admin", Password="admin" for this code sample. user = 'admin' pwd = 'xxxx' # Set proper headers headers = {"Content-Type":"application/json","Accept":"application/json"} ################# #Set variables command = "df /" critical = 50.0 warning = 40.0 ################# #build regex dfPattern = re.compile('[0-9]+') #get disk utilization diskUtil = commands.getstatusoutput(command) #split out the util % diskUtil = diskUtil[1].split()[11] #look for a match. If no match exit and return an #UNKNOWN (3) state to Nagios matchobj = dfPattern.match(diskUtil) if (matchobj): diskUtil = eval(matchobj.group(0)) else: print "STATE UNKNOWN" sys.exit(3) #Determine state to pass to Nagios #CRITICAL = 2 #WARNING = 1 #OK = 0 if diskUtil >= critical: url = 'https://devxxxx.service-now.com/api/now/table/incident' user = 'admin' pwd = 'xxxx' headers = {"Content-Type":"application/json","Accept":"application/ json"} requests.post(url, auth=(user, pwd), headers=headers ,data="{\"assignment_group\":\Hardware\",\"short_description\":\"Threshold critical\"}") print "FREE SPACE CRITICAL: '/' is %.2f%% full" % (float(diskUtil)) sys.exit(2) elif diskUtil >= warning: requests.post(url, auth=(user, pwd), headers=headers ,data="{\"assignment_group\":\Hardware\",\"short_description\":\"Threshold Warning\"}") print "FREE SPACE WARNING: '/' is %.2f%% full" % (float(diskUtil)) sys.exit(1) else: print "FREE SPACE OK: '/' is %.2f%% full" % (float(diskUtil)) sys.exit(0) On Mon, Oct 30, 2017 at 4:04 PM, sourav voip wrote: > Hi All, > > I'm trying to hit request.post with condition using if-else as below... > I;m posting the full script here...as I've tried declaring post url > details tested with multiple places. > > > > > From dvl at psu.edu Mon Oct 30 07:14:57 2017 From: dvl at psu.edu (ROGER GRAYDON CHRISTMAN) Date: Mon, 30 Oct 2017 07:14:57 -0400 Subject: why it append one more letter after decode? In-Reply-To: mailman.807.1509332780.1486.python-list@python.org References: Message-ID: <1509362097l.26542252l.0l@psu.edu> On Sun, Oct 29, 2017 11:06 PM, Ho Yeung Lee wrote: > if run these function to decode in python interactive console, >it can decode correct, > >but when run with a big project, it append a letter Y > > >On Monday, October 30, 2017 at 9:48:36 AM UTC+8, Ho Yeung Lee wrote: >> def leftrotate(l, n): >> return l[n:] + l[:n] >> >> def rightrotate(l, n): >> return l[-n:] + l[:-n] >> >> def encode(s, k, kk): >> l = [ord(i) for i in s] >> return leftrotate(''.join([chr(i + k) for i in l]), kk) >> >> def decode(s, k, kk): >> l = [ord(i) for i in rightrotate(s, kk)] >> return ''.join([chr(i - k) for i in l]) >> >> >> yesterday i add above code and run it with batch file >> it can decode a correct password >> >> then i install cx_freeze to produce executable file >> >> but today when i run it, i see the source of encrypted password is correct >> but the decode one, it append one more letter Y at the end of string >> >> why? >> is cx_freeze change the source python 2.7? > >i discover when >it >decode("\\",1,2) >in interactive console >is [ > >but decode in big project >it see as >decode(r"\\",1,2) >[[ > >it double > > My first inclination is that the data could have changed. For example, obtaining a string from the keyboard using input() would suppress the carriage return. But obtaining a string from a file might preserve the carriage return at the end of the line. I could imagine then that the extra carriage return contributed an extra character to your data. But then I see you had an example with "\\" in one case, and r"\\" in the other case. These are not equal to each other, and naturally would not give equal results from your function. So that leads to the second possibility that you are not calling the function in the same way. In either case, you cannot blame the function for giving you different results if you give it different data. Roger Christman Pennsylvania State University From mal at egenix.com Mon Oct 30 09:02:25 2017 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 30 Oct 2017 14:02:25 +0100 Subject: right list for SIGABRT python binary question ? In-Reply-To: <20171025095106.vxktihve4bgkdqpg@hermes.hilbert.loc> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> <92875eb4-affb-e550-3d5e-3c880d26a731@egenix.com> <20171025095106.vxktihve4bgkdqpg@hermes.hilbert.loc> Message-ID: <4d7441fd-eba1-b892-dca1-eb673379b101@egenix.com> n 25.10.2017 11:51, Karsten Hilbert wrote: > On Tue, Oct 24, 2017 at 08:47:58PM +0200, M.-A. Lemburg wrote: > >>>> This error suggests that you have 32- and 64-bit versions of >>>> Python and mxDateTime mixed in your installation. >>>> >>>> Py_InitModule4 is only available in the 32-bit build of >>>> Python. With the 64-bit build, it's called Py_InitModule4_64. > ... >> Could you check whether you have similar import errors with >> other modules that have C extensions ? E.g. lxml. >> >> What you're seeing appears to be a compilation problem >> with Python 2.7.14 on Debian. The executable doesn't appear >> to export its symbols to the .so files, or only some of them. > > Let's see: > ... using -dbg packages for everything, the imports work ... Ah, so you were mixing debug packages with non-debug ones. This explains the import errors. > mx.DateTime imports fine as well (but the SIGABRT persists). Do you just see the SIGABRT when running the debug versions or also with the production versions (memory management works differently in Python debug mode) ? BTW: It would help if you'd post the stack trace with symbols. The one you posted in one of your earlier emails only includes addresses. PS: Please CC me on replies as I don't regularly read c.l.p anymore. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Oct 30 2017) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From rhodri at kynesim.co.uk Mon Oct 30 10:27:04 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 30 Oct 2017 14:27:04 +0000 Subject: Invoking return through a function? In-Reply-To: References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> Message-ID: <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> On 29/10/17 16:45, Alberto Riva wrote: > On 10/29/2017 11:13 AM, bartc wrote: >> >> (What the OP wants was also proposed a few weeks back in comp.lang.c. >> But that was only within nested functions, so if H is inside G, and G >> is inside F, then a 'returnall' from H would return directly directly >> from F. Apparently Lisp allows this...) > > Well, not directly, but it can be simulated with the condition system > (equivalent to try/except) or with throw/catch, which is similar but > doesn't use exceptions. > > But my point was that in Lisp you don't need to do this, because you can > write a macro that expands into a return *in place*, without calling a > separate function, and this eliminates the problem entirely. Since > Python doesn't have macros I was looking for the next-best solution, but > there doesn't seem to be one. Oh, well... You can do the same in C. I've had the displeasure of trying to maintain such code. It was near-unreadable, because it constantly broke your expectations of what the code flow *could* be. The fact that something that could return from the middle of your function without the slightest indication was a rich source of bugs. (I didn't have to tell my boss to up our rates for dealing with this code. It took several days to do the work anyway, which was punishment enough at our rates.) -- Rhodri James *-* Kynesim Ltd From rhodri at kynesim.co.uk Mon Oct 30 10:31:00 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 30 Oct 2017 14:31:00 +0000 Subject: Coding style in CPython implementation In-Reply-To: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> References: <6b2df7f3-72a3-4a49-a50c-c169578751c6@googlegroups.com> Message-ID: <56002610-d3bc-dbae-75f8-5513355515f6@kynesim.co.uk> On 28/10/17 19:42, ???????? ????????? wrote: > Greetings everyone. > > I have noticed that in many if conditions the following syntax is used: > > a) if (variable == NULL) { ... } > b) if (variable == -1) { ... } > c) if (variable != NULL) { ... } > > What I wanted to ask is, is there a particular reason for not choosing > > a) if (!variable) { ... } in place of if (variable == NULL) { ... }, "if (variable == NULL)" emphasises that "variable" is a pointer variable, aiding readability. > b) if (-1 == variable) { ... } in place of if (variable == -1) { ... }, and It's just not natural English. It's safer to write it with the constant first, but modern compilers will warn you about assignments in conditions these days, and it's not worth the reduction in readability in my opinion. (Also, if you aren't using the -Werror -Wall flags to the compiler, you deserve everything that will be coming to you.) > c) if (variable) { ... } in place of if (variable) { ... } ? I assume you mean "if (variable != NULL)" here. Again, it emphasises the type; variable will be a pointer. -- Rhodri James *-* Kynesim Ltd From subhendu.panda93 at gmail.com Mon Oct 30 10:43:37 2017 From: subhendu.panda93 at gmail.com (subhendu.panda93 at gmail.com) Date: Mon, 30 Oct 2017 07:43:37 -0700 (PDT) Subject: Getting started with python Message-ID: <7ff675fe-5193-4e1e-8547-e3ac76458e8c@googlegroups.com> Hi, Could you please help me with the below if possible: 1. Best site to go ahead for python. 2. How python is different from other languages and future scope of it. 3. Tasks that are getting done using python in present. 4. Link where I can be able to get python videos, ebooks from basics to expert level free. Thanks Subhendu From BILL_NOSPAM at Noway.net Mon Oct 30 10:55:55 2017 From: BILL_NOSPAM at Noway.net (Bill) Date: Mon, 30 Oct 2017 10:55:55 -0400 Subject: Getting started with python In-Reply-To: <7ff675fe-5193-4e1e-8547-e3ac76458e8c@googlegroups.com> References: <7ff675fe-5193-4e1e-8547-e3ac76458e8c@googlegroups.com> Message-ID: subhendu.panda93 at gmail.com wrote: > Hi, > > Could you please help me with the below if possible: Possible and reasonable are two different things. Why don't you try some web searches and try to answer some of your own questions. I offer this advice as a Python newbe myself. Bill > > 1. Best site to go ahead for python. > 2. How python is different from other languages and future scope of it. > 3. Tasks that are getting done using python in present. > 4. Link where I can be able to get python videos, ebooks from basics to expert level free. > > Thanks > Subhendu From gkpopgr at yahoo.gr Mon Oct 30 12:15:50 2017 From: gkpopgr at yahoo.gr (George Kalamaras) Date: Mon, 30 Oct 2017 18:15:50 +0200 Subject: pythonw.exe error Message-ID: <464264.20920.bm@smtp107.mail.ir2.yahoo.com> When I am running IDLE return to me Missing python36.dll error ???????? ??? ??? ???????????? ??? Windows 10 From ikorot01 at gmail.com Mon Oct 30 12:41:27 2017 From: ikorot01 at gmail.com (Igor Korot) Date: Mon, 30 Oct 2017 12:41:27 -0400 Subject: pythonw.exe error In-Reply-To: <464264.20920.bm@smtp107.mail.ir2.yahoo.com> References: <464264.20920.bm@smtp107.mail.ir2.yahoo.com> Message-ID: Hi, On Oct 30, 2017 11:27 AM, "George Kalamaras via Python-list" < python-list at python.org> wrote: When I am running IDLE return to me Missing python36.dll error ???????? ??? ??? ???????????? ??? Windows 10 Could you please translate this from Greek? Thank you. -- https://mail.python.org/mailman/listinfo/python-list From rustompmody at gmail.com Mon Oct 30 13:15:06 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 30 Oct 2017 10:15:06 -0700 (PDT) Subject: pythonw.exe error In-Reply-To: References: <464264.20920.bm@smtp107.mail.ir2.yahoo.com> Message-ID: On Monday, October 30, 2017 at 10:11:49 PM UTC+5:30, Igor Korot wrote: > Hi, > > > > On Oct 30, 2017 11:27 AM, "George Kalamaras via Python-list" wrote: > > When I am running IDLE return to me Missing python36.dll error > > ???????? ??? ??? ???????????? ??? Windows 10 > > > Could you please translate this from Greek? Google translate tells me this is "Sent from mail for Windows 10" IOW its probably a mail auto-footer of the OP, not the error message From Karsten.Hilbert at gmx.net Mon Oct 30 13:21:18 2017 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Mon, 30 Oct 2017 18:21:18 +0100 Subject: right list for SIGABRT python binary question ? In-Reply-To: <4d7441fd-eba1-b892-dca1-eb673379b101@egenix.com> References: <20171016203215.vk7eacfnsaa3v64m@hermes.hilbert.loc> <59e5570a$0$14952$b1db1813$d948b532@news.astraweb.com> <20171017073043.piszqyu7dsqo55eu@hermes.hilbert.loc> <07fce3dd-afeb-2a1b-5db1-3945081decae@egenix.com> <20171022201551.gnxgqcksuir4aaor@hermes.hilbert.loc> <92875eb4-affb-e550-3d5e-3c880d26a731@egenix.com> <20171025095106.vxktihve4bgkdqpg@hermes.hilbert.loc> <4d7441fd-eba1-b892-dca1-eb673379b101@egenix.com> Message-ID: <20171030172118.difpsh2jo7gk43tx@hermes.hilbert.loc> On Mon, Oct 30, 2017 at 02:02:25PM +0100, M.-A. Lemburg wrote: > PS: Please CC me on replies as I don't regularly read c.l.p anymore. Sure. > >> Could you check whether you have similar import errors with > >> other modules that have C extensions ? E.g. lxml. > >> > >> What you're seeing appears to be a compilation problem > >> with Python 2.7.14 on Debian. The executable doesn't appear > >> to export its symbols to the .so files, or only some of them. > > > > Let's see: > > ... using -dbg packages for everything, the imports work ... > > Ah, so you were mixing debug packages with non-debug ones. This > explains the import errors. I am not sure I do. I installed normal and -dbg packages for modules with c extensions, say psycopg2: python-psycopg2: Installiert: 2.7.3-1 Installationskandidat: 2.7.3-1 Versionstabelle: *** 2.7.3-1 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 2.6.2-1 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages python-psycopg2-dbg: Installiert: 2.7.3-1 Installationskandidat: 2.7.3-1 Versionstabelle: *** 2.7.3-1 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 2.6.2-1 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages Now, when running either python2.7 or python2.7-dbg, which are installed alongside python2.7-dbg: Installiert: 2.7.14-2 Installationskandidat: 2.7.14-2 Versionstabelle: *** 2.7.14-2 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 2.7.13-2 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages python2.7: Installiert: 2.7.14-2 Installationskandidat: 2.7.14-2 Versionstabelle: *** 2.7.14-2 990 990 http://httpredir.debian.org/debian buster/main i386 Packages 500 http://httpredir.debian.org/debian unstable/main i386 Packages 100 /var/lib/dpkg/status 2.7.13-2 500 500 http://httpredir.debian.org/debian stretch/main i386 Packages the Debian version does (should ?) take care to import either the -dbg or the normal version of modules. It seems it so does because when I got both module versions installed I don't get any import errors. So I don't think I am mixing, at least not to my knowledge. (But I still get the SIGABORT on shutdown regardless.) > Do you just see the SIGABRT when running the debug versions or > also with the production versions (memory management works differently > in Python debug mode) ? Either version fails. Why haven't I tried running this under python3 ? Because the whole shebang is part of a wxPython application (GNUmed) and there are no Debian packages for wxPython/wxPhoenix on py3 as far as I know. > BTW: It would help if you'd post the stack trace with symbols. > The one you posted in one of your earlier emails only includes > addresses. I'd gladly comply if I knew how. This is what apt says about python2.7-dbg: Package: python2.7-dbg Version: 2.7.14-2 Description-en: Debug Build of the Python Interpreter (version 2.7) The package holds two things: . - A Python interpreter configured with --pydebug. Dynamically loaded modules are searched as _d.so first. Third party extensions need a separate build to be used by this interpreter. - Debug information for standard python interpreter and extensions. So from that I conferred it does contain symbols. I am getting a fresh run under gdb with python2.7-dbg. Maybe you can point out where you'd have expected to see symbols and help me figure out which -dbg package is missing on this Debian system: [...] ==> verifying target database schema ... ==> checking migrated data for plausibility ... Done bootstrapping GNUmed database: We very likely succeeded. log: /home/ncq/Projekte/gm-git/gnumed/gnumed/server/bootstrap/bootstrap-latest.log Debug memory block at address p=0x6aab7c: API '' 0 bytes originally requested The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb): at p-3: 0x33 *** OUCH at p-2: 0x47 *** OUCH at p-1: 0x00 *** OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 4 pad bytes at tail=0x6aab7c are not all FORBIDDENBYTE (0xfb): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH The block was made by call #0 to debug malloc/realloc. Fatal Python error: bad ID: Allocated using API '', verified using API 'o' Program received signal SIGABRT, Aborted. 0xb7fd9ce9 in __kernel_vsyscall () (gdb) bt #0 0xb7fd9ce9 in __kernel_vsyscall () #1 0xb7d70dd0 in __libc_signal_restore_set (set=0xbfffed40) at ../sysdeps/unix/sysv/linux/nptl-signals.h:79 #2 __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:48 #3 0xb7d72297 in __GI_abort () at abort.c:89 #4 0x0055fb74 in Py_FatalError (msg=0xbfffefec "bad ID: Allocated using API '\037', verified using API 'o'") at ../Python/pythonrun.c:1700 #5 0x00499adb in _PyObject_DebugCheckAddressApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1640 #6 0x004997a5 in _PyObject_DebugFreeApi (api=111 'o', p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1527 #7 0x0049964f in _PyObject_DebugFree (p=0x6aab7c <_Py_ZeroStruct>) at ../Objects/obmalloc.c:1471 #8 0x00471043 in int_dealloc (v=0x6aab7c <_Py_ZeroStruct>) at ../Objects/intobject.c:139 #9 0x00497bee in _Py_Dealloc (op=False) at ../Objects/object.c:2262 #10 0x00489bad in dict_dealloc (mp=0xb79f10d4) at ../Objects/dictobject.c:1085 Python Exception 'utf-8' codec can't decode byte 0xb7 in position 3: invalid start byte: #11 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 Python Exception 'utf-8' codec can't decode byte 0xb7 in position 3: invalid start byte: #12 0x004b9ab7 in subtype_dealloc (self=) at ../Objects/typeobject.c:1035 Python Exception 'utf-8' codec can't decode byte 0xb7 in position 3: invalid start byte: #13 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 #14 0x0044dc7a in instancemethod_dealloc (im=0xb79e9034) at ../Objects/classobject.c:2388 #15 0x00497bee in _Py_Dealloc (op=) at ../Objects/object.c:2262 #16 0x004885d7 in insertdict_by_entry (mp=0xb7891214, key='_shutdown', hash=598970216, ep=0x88e1ec, value=None) at ../Objects/dictobject.c:519 #17 0x00488857 in insertdict (mp=0xb7891214, key='_shutdown', hash=598970216, value=None) at ../Objects/dictobject.c:556 #18 0x0048910f in dict_set_item_by_hash_or_entry ( op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': , '__doc__': "Thread modul...(truncated), key='_shutdown', hash=598970216, ep=0x0, value=None) at ../Objects/dictobject.c:795 #19 0x00489285 in PyDict_SetItem ( op={'current_thread': , '_BoundedSemaphore': None, 'currentThread': , '_Timer': None, '_format_exc': None, 'Semaphore': , '_deque': None, 'activeCount': , '_profile_hook': None, '_sleep': None, '_trace_hook': None, 'ThreadError': , '_enumerate': None, '_start_new_thread': None, 'BoundedSemaphore': , '_shutdown': None, '__all__': ['activeCount', 'active_count', 'Condition', 'currentThread', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'], '_Event': , 'active_count': , '__package__': None, '_Condition': , '_RLock': , '_test': , 'local': , '__doc__': "Thread modul...(truncated), key='_shutdown', value=None) at ../Objects/dictobject.c:848 #20 0x00492758 in _PyModule_Clear (m=) at ../Objects/moduleobject.c:125 #21 0x0054a33b in PyImport_Cleanup () at ../Python/import.c:530 #22 0x0055c53c in Py_Finalize () at ../Python/pythonrun.c:458 #23 0x0055fe9c in Py_Exit (sts=0) at ../Python/pythonrun.c:1783 #24 0x0055e0fc in handle_system_exit () at ../Python/pythonrun.c:1151 #25 0x0055e152 in PyErr_PrintEx (set_sys_last_vars=1) at ../Python/pythonrun.c:1161 #26 0x0055dd5b in PyErr_Print () at ../Python/pythonrun.c:1064 #27 0x0055d61f in PyRun_SimpleFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:952 #28 0x0055cc4e in PyRun_AnyFileExFlags (fp=0x7ee010, filename=0xbffff7e6 "./bootstrap_gm_db_system.py", closeit=1, flags=0xbffff4f4) at ../Python/pythonrun.c:752 #29 0x00577cb0 in Py_Main (argc=5, argv=0xbffff684) at ../Modules/main.c:645 #30 0x004259c8 in main (argc=5, argv=0xbffff684) at ../Modules/python.c:20 (gdb) q A debugging session is active. Inferior 1 [process 12740] will be killed. Quit anyway? (y or n) y Does that help ? Thanks, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From cs at cskk.id.au Mon Oct 30 16:59:35 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 31 Oct 2017 07:59:35 +1100 Subject: from packaging import version as pack_version ImportError: No module named packaging In-Reply-To: References: Message-ID: <20171030205935.GA39060@cskk.homeip.net> On 30Oct2017 10:48, David Gabriel wrote: >When I run this command I got this error message: > >ubuntu at orchestrateur:/tmp/pack$ virtualenv -p $(which python3.5) . [...] >OSError: [Errno 13] Permission denied Peter has explained this failure. Note that in the shell you can show command execution, which is very helpful for figuring out this kind of thing: set -x virtualenv -p $(which python3.5) . set +x which would show you the actual virtualenv command invoked. In particular, it would be apparent (if Peter's suggestion is correct) that the result of "which python3.5" was empty, leading to an invalid "virtualenv" incantation. However, I write about something else: >This error is also reproducible using sudo. I've seen this before. NEVER EVER do this! If something fails, DO NOT reach for "sudo" to "fix" the problem. Usually things fail because of mistaken use (such as your example) or because of some system policy forbidding the action. In both cases the correct process is to understand why the failure occurred, NOT to try to run the action in some special privileged mode to avoid the check. Please try not to reach for "sudo" by reflex. It _will_ get you into trouble one day, either by doing unintended damage to your stuff or by raising the ire of some sysadmin when you try it on some system that is their responsibility. Thank you, Cameron Simpson (formerly cs at zip.com.au) From cs at cskk.id.au Mon Oct 30 19:20:59 2017 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 31 Oct 2017 10:20:59 +1100 Subject: Vim, ctags jump to python standard library source In-Reply-To: References: Message-ID: <20171030232059.GA65914@cskk.homeip.net> On 19Oct2017 12:08, Matt Schepers wrote: >I prefer to use vim and ctags when developing python, but I'm having >trouble getting ctags to index the standard library. Sometimes I would like >to see an object's constructor etc... > >Does anyone know how to do this? Will "ctags -o your-tags-file -a -R /path/to/your/python/lib" not scan the python library? Haven't tried it myself, and the tags file would be HUGE... Hmm... [~]fleet*> ctags -o testtags -a -R /usr/lib/python2.7 [~]fleet*> L testtags -rw-rw-r-- 1 cameron cameron 7208525 31 Oct 10:19 testtags Superficially looks ok... Cheers, Cameron Simpson (formerly cs at zip.com.au) From shelzmike at gmail.com Mon Oct 30 19:48:51 2017 From: shelzmike at gmail.com (Mike Miller) Date: Mon, 30 Oct 2017 23:48:51 +0000 Subject: [Tutor] request.post in If condition In-Reply-To: References: Message-ID: What do you mean when you say it is not hitting? Is there a specific error, or are you saying it simply isn't posting to your site? Mike On Mon, Oct 30, 2017, 8:21 AM sourav voip wrote: > Hi All, > > I'm trying to hit request.post with condition using if-else as below... > I;m posting the full script here...as I've tried declaring post url details > tested with multiple places.. > > If condition for disk utiliztion is working perfect ,however request.post > is not hitting. > Please suggest any hint/clue as I'm a new learner in python. > > Regards, > Sourav > > ------------------------------------------------------------ > ------------------------- > #!/usr/bin/python > import re,sys,commands > import requests > # Set the request parameters > url = 'https://devxxxx.service-now.com/api/now/table/incident' > > # Eg. User name="admin", Password="admin" for this code sample. > user = 'admin' > pwd = 'xxxx' > > # Set proper headers > headers = {"Content-Type":"application/json","Accept":"application/json"} > > > ################# > #Set variables > command = "df /" > critical = 50.0 > warning = 40.0 > ################# > > #build regex > dfPattern = re.compile('[0-9]+') > > #get disk utilization > diskUtil = commands.getstatusoutput(command) > > #split out the util % > diskUtil = diskUtil[1].split()[11] > > #look for a match. If no match exit and return an > #UNKNOWN (3) state to Nagios > > matchobj = dfPattern.match(diskUtil) > if (matchobj): > diskUtil = eval(matchobj.group(0)) > else: > print "STATE UNKNOWN" > sys.exit(3) > > #Determine state to pass to Nagios > #CRITICAL = 2 > #WARNING = 1 > #OK = 0 > if diskUtil >= critical: > url = 'https://devxxxx.service-now.com/api/now/table/incident' > user = 'admin' > pwd = 'xxxx' > headers = {"Content-Type":"application/json","Accept":"application/ > json"} > requests.post(url, auth=(user, pwd), headers=headers > ,data="{\"assignment_group\":\Hardware\",\"short_description\":\"Threshold > critical\"}") > print "FREE SPACE CRITICAL: '/' is %.2f%% full" % (float(diskUtil)) > sys.exit(2) > elif diskUtil >= warning: > requests.post(url, auth=(user, pwd), headers=headers > ,data="{\"assignment_group\":\Hardware\",\"short_description\":\"Threshold > Warning\"}") > print "FREE SPACE WARNING: '/' is %.2f%% full" % (float(diskUtil)) > sys.exit(1) > else: > print "FREE SPACE OK: '/' is %.2f%% full" % (float(diskUtil)) > sys.exit(0) > > > On Mon, Oct 30, 2017 at 4:04 PM, sourav voip > wrote: > > > Hi All, > > > > I'm trying to hit request.post with condition using if-else as below... > > I;m posting the full script here...as I've tried declaring post url > > details tested with multiple places. > > > > > > > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alb at nospam.chip.org Mon Oct 30 22:06:19 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Mon, 30 Oct 2017 22:06:19 -0400 Subject: Invoking return through a function? In-Reply-To: References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> Message-ID: On 10/30/2017 10:27 AM, Rhodri James wrote: > On 29/10/17 16:45, Alberto Riva wrote: >> On 10/29/2017 11:13 AM, bartc wrote: >>> >>> (What the OP wants was also proposed a few weeks back in comp.lang.c. >>> But that was only within nested functions, so if H is inside G, and G >>> is inside F, then a 'returnall' from H would return directly directly >>> from F. Apparently Lisp allows this...) >> >> Well, not directly, but it can be simulated with the condition system >> (equivalent to try/except) or with throw/catch, which is similar but >> doesn't use exceptions. >> >> But my point was that in Lisp you don't need to do this, because you >> can write a macro that expands into a return *in place*, without >> calling a separate function, and this eliminates the problem entirely. >> Since Python doesn't have macros I was looking for the next-best >> solution, but there doesn't seem to be one. Oh, well... > > You can do the same in C.? I've had the displeasure of trying to > maintain such code.? It was near-unreadable, because it constantly broke > your expectations of what the code flow *could* be.? The fact that > something that could return from the middle of your function without the > slightest indication was a rich source of bugs. Just curious: how is this different from calling a function that throws an exception (that may or may not be caught by some other function higher up in the call stack)? That also breaks your expectations of what the code flow would be... To clarify, I agree with your concern. What I mean is that even in normal conditions there's never any guarantee that the code flow is what you expect it to be just by reading the body of a function. The kind of statement I was trying to add would at least have made that explicit: return-if-so-and-so-happens. Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From alb at nospam.chip.org Mon Oct 30 22:15:05 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Mon, 30 Oct 2017 22:15:05 -0400 Subject: Invoking return through a function? In-Reply-To: <7f4ed4dc-ad4b-43b6-b647-6004d3e959b4@googlegroups.com> References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <7f4ed4dc-ad4b-43b6-b647-6004d3e959b4@googlegroups.com> Message-ID: On 10/30/2017 12:23 AM, Rustom Mody wrote: > On Sunday, October 29, 2017 at 9:52:01 PM UTC+5:30, Rick Johnson wrote: >> On Sunday, October 29, 2017 at 9:19:03 AM UTC-5, Alberto Riva wrote: > >>> In a language like Lisp >> >> Python is nothing like Lisp, and for good reason! Sure, we >> have a few lispers and functional fanboys who hang around >> here, and sometimes Rustom can get a little preachy about >> FP, but mostly, we tolerate the fanboyism -- so long as it's >> not rabid fanboyism. > > Rick's personal comments are one of the standard entertainments of this list. Enjoy! > > The comments on FP are more problematic: > > - These misconceptions are more widespread than just Rick > - They are right enough to be hard to refute > - And wrong enough to cause confusion and misdirection > - And irrelevant to (threads like) this one > > My super-short rejoiner to the FP-irrelevancy is: > ?If there is one key takeaway from functional programming maybe it should be: "Truth is a function of time" ? > > In more detail: > - Lisp (1960) was the second functional language; its predecessor was something called Formula Translator ? usually shortened to Fortran (c. 1957) > - By the late 80s, many FPers had begun regard Lisp as a setback for functional programming. [Dont get me wrong: Ive enjoyed Scheme more than any other > programming language. Its just that conflating Lisp and FP is an error (in 2017)] > > More historical details at http://blog.languager.org/2015/04/cs-history-1.html and sequel > > At a more conceptual level, people dont get that there are two dimensions > - the apply-lambda axis ? usually called functional programming > - the eval-quote axis ? which has precious little to do with FP (and is more relevant to your question) > > These two dimensions of power uniquely coincide in Lisp (Scheme). > > As you correctly (almost) point out > - Python is very much a lisp? its semantic under-belly > - However, syntactically its more like C/Java etc in the sense of having a rigid > fixed-at-language-design-time syntax > > More at http://blog.languager.org/2013/08/applying-si-on-sicp.html > > For a question like macros (syntax-extensions) that sits between the two your > question is interesting and I am not sure I know how to do it? > I remember seeing something about this recently but my google-foo is failing me > at the moment > > However I would look at > - PEP 263 https://www.python.org/dev/peps/pep-0263/ ? Source Code Encodings? > - Hooking into the codec module https://docs.python.org/3/library/codecs.html > - hooking into ast module https://docs.python.org/3/library/ast.html > > An old attempt in this direction: http://code.activestate.com/recipes/546539/ > > You've already got the moral-hi-horse guys giving you the dope of why this is evil. Swallow if you like. My own lightweight > suggestion would be that the natural pythonic way of unstructured exit is exceptions. Not as succinct as a tailormade syntax extension. But good enough IMHO > > PS I personally would be interested if you get this (syntax extension) running Thanks for the interesting reply! But once again: I was not proposing to add lisp-style macros to Python; I have no interest in doing that since I believe it's pretty much impossible :) Thanks! Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32) From steve+python at pearwood.info Mon Oct 30 23:00:25 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 31 Oct 2017 14:00:25 +1100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> Message-ID: <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> On Tue, 31 Oct 2017 01:06 pm, Alberto Riva wrote: > On 10/30/2017 10:27 AM, Rhodri James wrote: [...] >> You can do the same in C.? I've had the displeasure of trying to >> maintain such code.? It was near-unreadable, because it constantly broke >> your expectations of what the code flow *could* be.? The fact that >> something that could return from the middle of your function without the >> slightest indication was a rich source of bugs. > > Just curious: how is this different from calling a function that throws > an exception (that may or may not be caught by some other function > higher up in the call stack)? That also breaks your expectations of what > the code flow would be... The mere possibility of an exceptions does not break the normal, unexceptional code flow. You can still reason about the flow "if the data is valid, and there are no exceptions" where execution proceeds from the top of the function down, only exiting if there's a return. That's your base, unexceptional case, and there's no traps there. Any jump (return, break, continue) is explicit and obvious. Then you can think about the exceptional case, where data is invalid and you get an exception, separately. They're independent. In the exceptional case, the control flow is different, but still predictable: an exception is raised, and control exits the current block, to be either caught by the nearest surrounding except block, or if no such block, halting execution. Either way, its all relatively[1] straight-forward, with no surprises. You can't jump into the middle of a function, you can only pop out of the current call-chain into the surrounding function call. But with macros, you can't do that. Anything can be anything. An ordinary looking function call inside a loop might mask a hidden `continue` or `break`. A function call might mask a secret `return`. Python has no GOTO, fortunately, but C has at least two, GOTO and LONGJMP. A C macro could, if I understand correctly, jump into the middle of another function. (Yay for spaghetti code!) I suppose it wouldn't be too awful if macros required dedicated syntax, so at least you could distinguish between "this is a safe, ordinary function" and "this is a macro, it could mean anything". But it would still increase the maintenance burden, proportional to the number of macros used. > To clarify, I agree with your concern. What I mean is that even in > normal conditions there's never any guarantee that the code flow is what > you expect it to be just by reading the body of a function. True -- there's always *two* possible paths: the standard, unexceptional control flow, when no exceptions are raised; and the exceptional control flow. The second is more complex than the first, but in *practice* (if not in theory) we can at least be reasonably sure of what exception is raised and where it is caught. [1] There's no doubt that exception handling, except in the simplest forms, is more complex than code without exception handling. And this is why Rob Pike designed Go without exceptions. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rustompmody at gmail.com Mon Oct 30 23:26:01 2017 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 30 Oct 2017 20:26:01 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <7f4ed4dc-ad4b-43b6-b647-6004d3e959b4@googlegroups.com> Message-ID: On Tuesday, October 31, 2017 at 7:45:18 AM UTC+5:30, Alberto Riva wrote: > On 10/30/2017 12:23 AM, Rustom Mody wrote: > > On Sunday, October 29, 2017 at 9:52:01 PM UTC+5:30, Rick Johnson wrote: > >> On Sunday, October 29, 2017 at 9:19:03 AM UTC-5, Alberto Riva wrote: > > > >>> In a language like Lisp > >> > >> Python is nothing like Lisp, and for good reason! Sure, we > >> have a few lispers and functional fanboys who hang around > >> here, and sometimes Rustom can get a little preachy about > >> FP, but mostly, we tolerate the fanboyism -- so long as it's > >> not rabid fanboyism. > > > > Rick's personal comments are one of the standard entertainments of this list. Enjoy! > > > > The comments on FP are more problematic: > > > > - These misconceptions are more widespread than just Rick > > - They are right enough to be hard to refute > > - And wrong enough to cause confusion and misdirection > > - And irrelevant to (threads like) this one > > > > My super-short rejoiner to the FP-irrelevancy is: > > ?If there is one key takeaway from functional programming maybe it should be: "Truth is a function of time" ? > > > > In more detail: > > - Lisp (1960) was the second functional language; its predecessor was something called Formula Translator ? usually shortened to Fortran (c. 1957) > > - By the late 80s, many FPers had begun regard Lisp as a setback for functional programming. [Dont get me wrong: Ive enjoyed Scheme more than any other > > programming language. Its just that conflating Lisp and FP is an error (in 2017)] > > > > More historical details at http://blog.languager.org/2015/04/cs-history-1.html and sequel > > > > At a more conceptual level, people dont get that there are two dimensions > > - the apply-lambda axis ? usually called functional programming > > - the eval-quote axis ? which has precious little to do with FP (and is more relevant to your question) > > > > These two dimensions of power uniquely coincide in Lisp (Scheme). > > > > As you correctly (almost) point out > > - Python is very much a lisp? its semantic under-belly > > - However, syntactically its more like C/Java etc in the sense of having a rigid > > fixed-at-language-design-time syntax > > > > More at http://blog.languager.org/2013/08/applying-si-on-sicp.html > > > > For a question like macros (syntax-extensions) that sits between the two your > > question is interesting and I am not sure I know how to do it? > > I remember seeing something about this recently but my google-foo is failing me > > at the moment > > > > However I would look at > > - PEP 263 https://www.python.org/dev/peps/pep-0263/ ? Source Code Encodings? > > - Hooking into the codec module https://docs.python.org/3/library/codecs.html > > - hooking into ast module https://docs.python.org/3/library/ast.html > > > > An old attempt in this direction: http://code.activestate.com/recipes/546539/ > > > > You've already got the moral-hi-horse guys giving you the dope of why this is evil. Swallow if you like. My own lightweight > > suggestion would be that the natural pythonic way of unstructured exit is exceptions. Not as succinct as a tailormade syntax extension. But good enough IMHO > > > > PS I personally would be interested if you get this (syntax extension) running > > Thanks for the interesting reply! But once again: I was not proposing to > add lisp-style macros to Python; I have no interest in doing that since > I believe it's pretty much impossible :) https://github.com/pythonql/pythonql is the example that shows how lisp-style extended-syntax (macros) can be done via file-encodings in a modern python. Or if you prefer modern lingo: ?How to embed an internal DSL into python? Note: I am not recommending you take this line; just that its not impossible as you seem to imagine. My own feeling about lisp-macros is conflicted: - They are likely the most unique feature of lisp, putting it at the top of the blub-language tower - They are the single reason Lisp can never succeed like mainstream languages: Any significant Lisp sub-ecosystem will inevitably develop a macro set which succinctly and precisely expresses its needs but is arcane and incomprehensible to someone from another sub-ecosystem. This is Turing-completeness Lisp-specific style: The basic Turing-result is that programs can be arbitrarily complex to the point of being unanalysable. For mainstream languages that complexity sits squarely in the application For lisp there is a choice of simplifying the application by concentrating the complexity in a macro/DSL layer between implementation and application From rosuav at gmail.com Mon Oct 30 23:34:31 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 31 Oct 2017 14:34:31 +1100 Subject: Invoking return through a function? In-Reply-To: <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 31, 2017 at 2:00 PM, Steve D'Aprano wrote: > Python has no GOTO, fortunately, but C has at least two, GOTO and LONGJMP. A C > macro could, if I understand correctly, jump into the middle of another > function. (Yay for spaghetti code!) No, I don't think you do understand them correctly - or at least, I don't know of any way for a C macro to jump into the middle of a function. There are three quite different things mentioned here. 1) The 'goto' statement, which unconditionally jumps you to another location *in the same function* 2) setjmp/longjmp, which is not actually a "goto", but more of a "multi-level return" 3) Preprocessor macros, which expand to some form of C source code. Since macros have to expand to legal C code, they can't do anything that you couldn't do by simply writing out the code - they just hide it behind a token that looks like a function call. To use setjmp/longjmp (which, btw, are actually standard library functions, not language features) to jump into another function, you first have to call the destination; they're much more akin to exception handling than to an unstructured 'goto' (in fact, they're often used in the implementation of exception handling). Basically, longjmp can only rewind you to a previous setjmp, and only if the function that called setjmp has *not returned* - so you can only use it to jump outwards, not inwards. But feel free to point out something I've forgotten, wherein you can actually jump into the middle of a function using a macro. There's enough dark corners in C that it's entirely possible I've missed something obvious. ChrisA From rantingrickjohnson at gmail.com Mon Oct 30 23:51:05 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Mon, 30 Oct 2017 20:51:05 -0700 (PDT) Subject: Invoking return through a function? In-Reply-To: <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> Message-ID: <35a1763b-03d0-4ff3-a761-da40ee9670c8@googlegroups.com> Steve D'Aprano wrote: > > [...] > > I suppose it wouldn't be too awful if macros required > dedicated syntax, so at least you could distinguish between > "this is a safe, ordinary function" and "this is a macro, > it could mean anything". In the same spirit, i've been trying in vain for *YEARS* to convince the PyDev gods to mandate that all clobbered class- methods be marked in some distinguishing fashion so they won't be transparent in the source, but nobody has bothered to listen. I assume the indifference is due to the anti-OOP attitude of the more influential folks in the community. From steve+python at pearwood.info Tue Oct 31 01:32:05 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 31 Oct 2017 16:32:05 +1100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> Message-ID: <59f80ad7$0$18593$b1db1813$d948b532@news.astraweb.com> On Tue, 31 Oct 2017 02:34 pm, Chris Angelico wrote: > On Tue, Oct 31, 2017 at 2:00 PM, Steve D'Aprano > wrote: >> Python has no GOTO, fortunately, but C has at least two, GOTO and LONGJMP. >> A C macro could, if I understand correctly, jump into the middle of another >> function. (Yay for spaghetti code!) > > No, I don't think you do understand them correctly - or at least, I > don't know of any way for a C macro to jump into the middle of a > function. I presume a macro could contain a call to longjmp, yes? Since longjmp can jump into another function (albeit only one which has prepared for it in advance), so can the macro. https://stackoverflow.com/questions/21355110/how-to-goto-into-different-function-in-c And what about assembly? Couldn't you jump into a function from assembly? Of course the stack will be all wrong, but if you're using assembly you have to manage that yourself. > There are three quite different things mentioned here. > > 1) The 'goto' statement, which unconditionally jumps you to another > location *in the same function* Yes. And a large enough function can contain everything. If you wanted to write BASIC-style unstructured code, you could dump everything into one function and use GOTO. > 2) setjmp/longjmp, which is not actually a "goto", but more of a > "multi-level return" Right-oh. So sort of like an exception then. So perhaps C is not quite as unstructured as I had initially thought. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 31 01:35:17 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 31 Oct 2017 16:35:17 +1100 Subject: Invoking return through a function? References: <6b7f9d04-e912-4c58-be55-c5adb05bde22@googlegroups.com> <7f4ed4dc-ad4b-43b6-b647-6004d3e959b4@googlegroups.com> Message-ID: <59f80b97$0$18593$b1db1813$d948b532@news.astraweb.com> On Tue, 31 Oct 2017 02:26 pm, Rustom Mody wrote: > My own feeling about lisp-macros is conflicted: > - They are likely the most unique feature of lisp, putting it at the top of > the blub-language tower > - They are the single reason Lisp can never succeed like mainstream > languages: Any significant Lisp sub-ecosystem will inevitably develop a > macro set which succinctly and precisely expresses its needs but is arcane > and incomprehensible to someone from another sub-ecosystem. Well said. That's one of the disadvantages of Forth as well: since Forth allows you to define your own control-structures, even the structure of the code can be unfamiliar. Another way to put it might be that any sufficiently complex Lisp program doesn't look like Lisp any more. Except perhaps for the myriad parentheses *wink* -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Tue Oct 31 02:59:08 2017 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 31 Oct 2017 17:59:08 +1100 Subject: Invoking return through a function? In-Reply-To: <59f80ad7$0$18593$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> <59f80ad7$0$18593$b1db1813$d948b532@news.astraweb.com> Message-ID: On Tue, Oct 31, 2017 at 4:32 PM, Steve D'Aprano wrote: > On Tue, 31 Oct 2017 02:34 pm, Chris Angelico wrote: > >> On Tue, Oct 31, 2017 at 2:00 PM, Steve D'Aprano >> wrote: >>> Python has no GOTO, fortunately, but C has at least two, GOTO and LONGJMP. >>> A C macro could, if I understand correctly, jump into the middle of another >>> function. (Yay for spaghetti code!) >> >> No, I don't think you do understand them correctly - or at least, I >> don't know of any way for a C macro to jump into the middle of a >> function. > > I presume a macro could contain a call to longjmp, yes? Since longjmp can jump > into another function (albeit only one which has prepared for it in advance), > so can the macro. I'd still consider longjmp to be jumping OUT, not IN - you can't skip the beginning of a function and jump half way into it. For instance: void foo() { printf("Start of function foo\n"); SOME_EVIL_MACRO printf("End of function foo\n"); } void bar() { SOME_OTHER_EVIL_MACRO } int main() { bar(); return 0; } To my knowledge, there's no way to have bar() run just the second half of foo(). The only way to have bar "jump into" foo is to first call foo, which would look like this: jmp_buf buf; void foo() { printf("Start of function foo\n"); if (!setjmp(buf)) { printf("Middle of function foo\n"); bar(); printf("This won't happen"); } else { printf("After longjmp\n"); } printf("End of function foo\n"); } void bar() { printf("Start of function bar\n"); longjmp(buf, 1); printf("This won't happen\n"); } The Python equivalent would be: def foo(): print("Start of function foo") try: print("Middle of function foo") bar() print("This won't happen") except Exception: print("After longjmp/exception") print("End of function foo") def bar(): print("Start of function bar") raise Exception print("This won't happen") So if longjmp can jump into a function, so can Python's raise statement. Yes, it's nonlocal flow control; but it's not an uncontrolled goto that lets you jump into functions in any normal sense of that expression. > https://stackoverflow.com/questions/21355110/how-to-goto-into-different-function-in-c > > And what about assembly? Couldn't you jump into a function from assembly? Of > course the stack will be all wrong, but if you're using assembly you have to > manage that yourself. Oh sure, but that's like saying "I can change the value of a Python string using ctypes, ergo Python strings aren't immutable". When you peel away an abstraction layer, you get to see the underlying code. What you do with that is your responsibility. >> There are three quite different things mentioned here. >> >> 1) The 'goto' statement, which unconditionally jumps you to another >> location *in the same function* > > Yes. And a large enough function can contain everything. If you wanted to > write BASIC-style unstructured code, you could dump everything into one > function and use GOTO. Well sure. C certainly won't stop you from being an idiot :) And Python (or at least CPython) would let you alter the bytecode and actually insert jump opcodes. And, of course, you can hide the abuse behind a decorator, which is as much "action at a distance" as CPP macros. The trouble with trying to make yourself stupider than you really are is that you very often succeed. CS Lewis wasn't talking about software design, but he was pretty much spot on... :D ChrisA From nomail at com.invalid Tue Oct 31 06:48:26 2017 From: nomail at com.invalid (ast) Date: Tue, 31 Oct 2017 11:48:26 +0100 Subject: enum Message-ID: <59f854ff$0$10201$426a74cc@news.free.fr> Hi Below an example of enum which defines an __init__ method. https://docs.python.org/3.5/library/enum.html#planet Documentation says that the value of the enum members will be passed to this method. But in that case __init__ waits for two arguments, mass and radius, while enum member's value is a tuple. It seems that there is a tuple unpacking, but it is not documented, that's not clear class Planet(Enum): ... MERCURY = (3.303e+23, 2.4397e6) ... VENUS = (4.869e+24, 6.0518e6) ... EARTH = (5.976e+24, 6.37814e6) ... MARS = (6.421e+23, 3.3972e6) ... JUPITER = (1.9e+27, 7.1492e7) ... SATURN = (5.688e+26, 6.0268e7) ... URANUS = (8.686e+25, 2.5559e7) ... NEPTUNE = (1.024e+26, 2.4746e7) ... ... def __init__(self, mass, radius): ... self.mass = mass # in kilograms ... self.radius = radius # in meters ... ... @property ... def surface_gravity(self): ... # universal gravitational constant (m3 kg-1 s-2) ... G = 6.67300E-11 ... return G * self.mass / (self.radius * self.radius) From alberto at metapensiero.it Tue Oct 31 07:54:30 2017 From: alberto at metapensiero.it (Alberto Berti) Date: Tue, 31 Oct 2017 12:54:30 +0100 Subject: Invoking return through a function? References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <87mv49i2m6.fsf@metapensiero.it> Message-ID: <87zi87jzwp.fsf@ender.lizardnet> >>>>> "Lele" == Lele Gaifax writes: Lele> ram at zedat.fu-berlin.de (Stefan Ram) writes: Stefan> There are many macro processors available, like the C macro Stefan> preprocessor, gpp, M4, or funnelweb. You can always use them Stefan> for your Python source (which, in this case, isn't exactly Python Stefan> source anymore). Lele> Even more to the point, MacroPy! See https://github.com/azazel75/macropy for a Lele> 3.5+ version. I share the opinion of Stefan and others, it's a bad practice. But just to have some fun I implemented it with MacroPy... This is the module with the test code:: # test.py from test_macros import macros, check_macro def checkKey(k, m): return k in m @check_macro def test(): m = {1: 'a', 2: 'b'} print('before') checkKey(3, m) print('after') here the test function is wrapped by the macro, that is defined as: # test_macros.py from macropy.core.macros import Macros from macropy.core.quotes import macros, q, u, ast_literal from macropy.core.walkers import Walker from macropy.experimental.pattern import macros, switch, _matching, ClassMatcher from ast import Call, Name, Expr macros = Macros() @macros.decorator def check_macro(tree, **kw): @Walker def transform_checkKey(tree, stop, **kw): with switch(tree): if Expr(value=Call(func=Name(id='checkKey'))): with q as result: if not ast_literal[tree.value]: print('exiting!') return stop() else: result = tree return result return transform_checkKey.recurse(tree) The macro is run with all the decorated code of the ``test()`` method passed as the ``tree`` parameter in the form of ast tree. A tree walker is then run to navigate the tree and augment the occurrence of checkKey with the conditional return And finally, a ``main`` module is needed to activate macro parsing and substitution: # test_main.py import macropy.activate import test test.test() cheers, Alberto From bc at freeuk.com Tue Oct 31 08:33:37 2017 From: bc at freeuk.com (bartc) Date: Tue, 31 Oct 2017 12:33:37 +0000 Subject: Invoking return through a function? In-Reply-To: <59f80ad7$0$18593$b1db1813$d948b532@news.astraweb.com> References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> <59f7e74b$0$18561$b1db1813$d948b532@news.astraweb.com> <59f80ad7$0$18593$b1db1813$d948b532@news.astraweb.com> Message-ID: On 31/10/2017 05:32, Steve D'Aprano wrote: > On Tue, 31 Oct 2017 02:34 pm, Chris Angelico wrote: >> No, I don't think you do understand them correctly - or at least, I >> don't know of any way for a C macro to jump into the middle of a >> function. > > I presume a macro could contain a call to longjmp, yes? Since longjmp can jump > into another function (albeit only one which has prepared for it in advance), > so can the macro. > > https://stackoverflow.com/questions/21355110/how-to-goto-into-different-function-in-c > > And what about assembly? Couldn't you jump into a function from assembly? Of > course the stack will be all wrong, but if you're using assembly you have to > manage that yourself. I've used jumping-into-functions quite a bit, mainly to implement threaded code as used to code fast byte-code dispatchers. The actual jump is done in inline assembly (and usually via a function pointer; no actual named label is used), but it needs HLL support to ensure the functions you're jumping into have special entry and exit code (no stack frames or anything). The main problem with a HLL goto into a HLL function is that the label names in the function will not be visible from the goto site. Otherwise it might be possible, with some provisos (eg. any stack frame the function uses may not be properly set up). gcc-C probably has some extensions to do something similar (label pointers for example which can be stored in global data, although you have to execute the function normally first to set them up). None of this would be routine however. Python thinks it's a cut above other languages as it manages without gotos, forgetting that gotos are most likely needed in its implementation, so that someone has done the dirty work! (CPython compiled with gcc almost certainly uses label pointers in its dispatcher, although I believe the gotos are still into the same function.) >> 2) setjmp/longjmp, which is not actually a "goto", but more of a >> "multi-level return" > > Right-oh. So sort of like an exception then. I've implemented setjmp/longjmp, but find them unintuitive to use. When I had to use arbitrary jumps from one function to another in the past (error recovery for example, what people use exceptions for now), I just used inline assembly. It was a lot simpler: assem mov [spvalue],sp # set up recovery data in globals mov [bpvalue],bp mov [recoverylabel],recoverypt end .... do normal work. recoverypt: # here when something has gone wrong When you want to get to the recovery point from anywhere (probably you'd put this in a function): assem mov sp,[spvalue] mov bp,[bpvalue] jmp [recoverylabel] end -- bartc From ganesh1pal at gmail.com Tue Oct 31 10:51:25 2017 From: ganesh1pal at gmail.com (Ganesh Pal) Date: Tue, 31 Oct 2017 20:21:25 +0530 Subject: How to join elements at the beginning and end of the list Message-ID: How to join each elements with a delimiter at (1) beginning and end of the list and (2) connecting all elements of the list Example : >>> value_list = [1, 2, 3, 4, 56, 's'] I want this to be converted in this from '||1||2||3||4||56||s||' Here is my solution >>> values = '||' + '||'.join(map(str, value_list)) + '||' >>> values '||1||2||3||4||56||s||' Iam joining the elements at the beginning and end of the list using '+' operator any other solution, this is not looking neater I am a Linux user using python 2.7 Regards, Ganesh From rhodri at kynesim.co.uk Tue Oct 31 11:01:39 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 31 Oct 2017 15:01:39 +0000 Subject: Invoking return through a function? In-Reply-To: References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> Message-ID: On 31/10/17 02:06, Alberto Riva wrote: Steve D'Aprano gave you a pretty full answer, I just wanted to add: > The kind of > statement I was trying to add would at least have made that explicit: > return-if-so-and-so-happens. That's only obvious in the function that's doing the returning. The function that's doing the calling that gets its expectation of flow control broken has no clue, and that's my problem. -- Rhodri James *-* Kynesim Ltd From neilc at norwich.edu Tue Oct 31 11:29:14 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Tue, 31 Oct 2017 15:29:14 +0000 (UTC) Subject: How to join elements at the beginning and end of the list References: Message-ID: On 2017-10-31, Ganesh Pal wrote: > Here is my solution > >>>> values = '||' + '||'.join(map(str, value_list)) + '||' >>>> values > > '||1||2||3||4||56||s||' > > I am joining the elements at the beginning and end of the list > using '+' operator any other solution, this is not looking > neater > > I am a Linux user using python 2.7 You can use the % operator instead of +, and a generator expression instead of map. It's a pretty small improvement, though. values = '||%s||' % ('||'.join(str(s) for s in value_list)) At least... I THINK you can use that generator expression in 2.7. -- Neil Cerutti From ned at nedbatchelder.com Tue Oct 31 11:49:32 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 31 Oct 2017 11:49:32 -0400 Subject: How to join elements at the beginning and end of the list In-Reply-To: References: Message-ID: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> On 10/31/17 11:29 AM, Neil Cerutti wrote: > On 2017-10-31, Ganesh Pal wrote: >> Here is my solution >> >>>>> values = '||' + '||'.join(map(str, value_list)) + '||' >>>>> values >> '||1||2||3||4||56||s||' >> >> I am joining the elements at the beginning and end of the list >> using '+' operator any other solution, this is not looking >> neater >> >> I am a Linux user using python 2.7 > You can use the % operator instead of +, and a generator > expression instead of map. It's a pretty small improvement, > though. > > values = '||%s||' % ('||'.join(str(s) for s in value_list)) > > At least... I THINK you can use that generator expression in 2.7. > However you solve it, do yourself a favor and write a function to encapsulate it: ??? def wrapped_join(values, sep): ??????? """Join values with sep, and also include sep at the ends.""" ??????? return "{sep}{vals}{sep}".format(sep=sep, vals=sep.join(str(v) for v in values)) --Ned. From neilc at norwich.edu Tue Oct 31 12:12:35 2017 From: neilc at norwich.edu (Neil Cerutti) Date: Tue, 31 Oct 2017 16:12:35 +0000 (UTC) Subject: How to join elements at the beginning and end of the list References: Message-ID: On 2017-10-31, Stefan Ram wrote: > Neil Cerutti writes: >>You can use the % operator instead of +, and a generator >>expression instead of map. It's a pretty small improvement, >>though. > > "Improvement" in what sense? > > C:\>python -m timeit -s "value_list = [1, 2, 3, 4, 56, 's']" "values = '||' + '||'.join(map(str, value_list)) + '||'" > 100000 loops, best of 3: 4.86 usec per loop > > C:\>python -m timeit -s "value_list = [1, 2, 3, 4, 56, 's']" "values = '||%s||' % ('||'.join(str(s) for s in value_list))" > 100000 loops, best of 3: 7.46 usec per loop Hmmm.... minty freshness? -- Neil Cerutti From alberto at metapensiero.it Tue Oct 31 12:28:16 2017 From: alberto at metapensiero.it (Alberto Berti) Date: Tue, 31 Oct 2017 17:28:16 +0100 Subject: Cooperative class tree filtering References: <87376o2ced.fsf@ender.lizardnet> Message-ID: <87r2tjjn8f.fsf@ender.lizardnet> Thanks Ian, >>>>> "Ian" == Ian Kelly writes: Ian> On Thu, Oct 12, 2017 at 5:07 PM, Alberto Berti wrote: Ian> My initial reaction is: is this really worth it? This seems like an Ian> awful lot of code and added complexity in order to do away with two Ian> lines. It's a lot easier to reason about "return Ian> super().filter(element)" and verify that it does the right thing than Ian> for the complicated descriptor above. yes this was also my conclusion, that descriptor is sitting on a branch of its own and I've yet to decide on it. The fact is that i've many cases where I need a coperative superclass-subclass method execution and I was trying to find some other pattern that would fit in ;-) From ned at nedbatchelder.com Tue Oct 31 12:41:42 2017 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 31 Oct 2017 12:41:42 -0400 Subject: How to join elements at the beginning and end of the list In-Reply-To: References: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> Message-ID: <79690d1c-e947-6b6c-afd0-4b5041ccaa46@nedbatchelder.com> On 10/31/17 12:29 PM, Stefan Ram wrote: > Ned Batchelder writes: >> However you solve it, do yourself a favor and write a function to >> encapsulate it: > It is always a good solution to encapsulate a pattern into > a function. So I agree that this is a good suggestion. But > just for the sole sake of information, I'd like to add that > this is also the slowest solution so far (about 10.76 usec). > > This might be a case where macros would be fine. As readable > as a function call, but no runtime-overhead. One can write > > value_list = [1, 2, 3, 4, 56, 's'] > > #define JOIN_WRAPPED(list,string) \ > string + string.join(map(str,list)) + string > > values = JOIN_WRAPPED(value_list,'||') > > print( values ) > > and save it as ?source.c? and execute it using > > gcc -E source.c -o source.py > python source.py > > . This is also not intended to be a recommendation. > I try to avoid micro-optimization.? My guess is that right after calling wrapped_join(), the result will be written to an I/O device of some kind.? If that is so, the time spent in wrapped_join will be irrelevant. --Ned. From mickey695 at gmail.com Tue Oct 31 12:45:14 2017 From: mickey695 at gmail.com (John Smith) Date: Tue, 31 Oct 2017 18:45:14 +0200 Subject: The syntax of replacement fields in format strings Message-ID: If we keep the current implementation as is, perhaps the documentation should at least be altered ? From steve+python at pearwood.info Tue Oct 31 13:03:57 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 01 Nov 2017 04:03:57 +1100 Subject: How to join elements at the beginning and end of the list References: Message-ID: <59f8acff$0$18607$b1db1813$d948b532@news.astraweb.com> On Wed, 1 Nov 2017 02:29 am, Neil Cerutti wrote: > You can use the % operator instead of +, and a generator > expression instead of map. It's a pretty small improvement, > though. > > values = '||%s||' % ('||'.join(str(s) for s in value_list)) > > At least... I THINK you can use that generator expression in 2.7. Generator expressions are slightly slower when you call join. If join knows how many items there are, it can allocate space more efficiently, which is faster. So even though it takes a bit of time to build, and throw away, a temporary list, its actually faster to join a list comp than a generator expression. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 31 13:10:17 2017 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 01 Nov 2017 04:10:17 +1100 Subject: Performance of map vs starmap. References: Message-ID: <59f8ae7c$0$18577$b1db1813$d948b532@news.astraweb.com> On Mon, 30 Oct 2017 09:10 pm, Kirill Balunov wrote: > Sometime ago I asked this question at SO [1], and among the responses > received was paragraph: > > - `zip` re-uses the returned `tuple` if it has a reference count of 1 when > the `__next__` call is made. > - `map` build a new `tuple` that is passed to the mapped function every > time a `__next__` call is made. > > Why can not `map` use the same approach as `zip`? It possibly could, if somebody could be bothered to make that micro-optimization. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From israel at ravnalaska.net Tue Oct 31 13:38:10 2017 From: israel at ravnalaska.net (Israel Brewster) Date: Tue, 31 Oct 2017 09:38:10 -0800 Subject: Thread safety issue (I think) with defaultdict Message-ID: <2236CA4C-C444-44E2-8B8A-4642B2ED3D25@ravnalaska.net> A question that has arisen before (for example, here: https://mail.python.org/pipermail/python-list/2010-January/565497.html ) is the question of "is defaultdict thread safe", with the answer generally being a conditional "yes", with the condition being what is used as the default value: apparently default values of python types, such as list, are thread safe, whereas more complicated constructs, such as lambdas, make it not thread safe. In my situation, I'm using a lambda, specifically: lambda: datetime.min So presumably *not* thread safe. My goal is to have a dictionary of aircraft and when they were last "seen", with datetime.min being effectively "never". When a data point comes in for a given aircraft, the data point will be compared with the value in the defaultdict for that aircraft, and if the timestamp on that data point is newer than what is in the defaultdict, the defaultdict will get updated with the value from the datapoint (not necessarily current timestamp, but rather the value from the datapoint). Note that data points do not necessarily arrive in chronological order (for various reasons not applicable here, it's just the way it is), thus the need for the comparison. When the program first starts up, two things happen: 1) a thread is started that watches for incoming data points and updates the dictionary as per above, and 2) the dictionary should get an initial population (in the main thread) from hard storage. The behavior I'm seeing, however, is that when step 2 happens (which generally happens before the thread gets any updates), the dictionary gets populated with 56 entries, as expected. However, none of those entries are visible when the thread runs. It's as though the thread is getting a separate copy of the dictionary, although debugging says that is not the case - printing the variable from each location shows the same address for the object. So my questions are: 1) Is this what it means to NOT be thread safe? I was thinking of race conditions where individual values may get updated wrong, but this apparently is overwriting the entire dictionary. 2) How can I fix this? Note: I really don't care if the "initial" update happens after the thread receives a data point or two, and therefore overwrites one or two values. I just need the dictionary to be fully populated at some point early in execution. In usage, the dictionary is used to see of an aircraft has been seen "recently", so if the most recent datapoint gets overwritten with a slightly older one from disk storage, that's fine - it's just if it's still showing datetime.min because we haven't gotten in any datapoint since we launched the program, even though we have "recent" data in disk storage thats a problem. So I don't care about the obvious race condition between the two operations, just that the end result is a populated dictionary. Note also that as datapoint come in, they are being written to disk, so the disk storage doesn't lag significantly anyway. The framework of my code is below: File: watcher.py last_points = defaultdict(lambda:datetime.min) # This function is launched as a thread using the threading module when the first client connects def watch(): while true: pointtime= if last_points[] < pointtime: last_points[]=pointtime #DEBUGGING print("At update:", len(last_points)) File: main.py: from .watcher import last_points # This function will be triggered by a web call from a client, so could happen at any time # Client will call this function immediately after connecting, as well as in response to various user actions. def getac(): for record in aclist: last_points[]=record_timestamp #DEBUGGING print("At get AC:", len(last_points)) ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- From HooDunnit at didly42KahZidly.net Tue Oct 31 13:39:38 2017 From: HooDunnit at didly42KahZidly.net (Cousin Stanley) Date: Tue, 31 Oct 2017 10:39:38 -0700 Subject: enum References: <59f854ff$0$10201$426a74cc@news.free.fr> Message-ID: ast wrote: > https://docs.python.org/3.5/library/enum.html#planet > > Documentation says that the value of the enum > members will be passed to this method. > > But in that case __init__ waits for two arguments, mass > and radius, while enum member's value is a tuple. > > It seems that there is a tuple unpacking, but it is > not documented, that's not clear > .... I added the following code to your example to unpack planet.value into mass and radius after first importing Enum .... from enum import Enum def test_01() : print( '\n planet mass radius gravity \n' ) for planet in Planet : mass , radius = planet.value print( ' {:8s} {:9.4E} {:9.4E} {:9.4E} '.format( planet.name , mass , radius , planet.surface_gravity ) ) if __name__ == '__main__' : test_01() A working copy .... http://csphx.net/python/planets_enum.txt -- Stanley C. Kitching Human Being Phoenix, Arizona From jon+usenet at unequivocal.eu Tue Oct 31 13:54:32 2017 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 31 Oct 2017 17:54:32 -0000 (UTC) Subject: Report on non-breaking spaces in posts (was: How to join elements at the beginning and end of the list) References: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> Message-ID: On 2017-10-31, Stefan Ram wrote: > Ned Batchelder writes: >> ?????? def wrapped_join(values, sep): > > Ok, here's a report on me seing non-breaking spaces in > posts in this NG. I have written this report so that you > can see that it's not my newsreader that is converting > something, because there is no newsreader involved. Yes Ned's post definitely had non-breaking spaces in; it looks like Thunderbird is turning strings of spaces into ' \xa0+ ', presumably because things that expect HTML will squash runs of spaces (even though the post is not HTML...) By the way I would commend 'od -tx1z' to you as it is *far* easier to read: 0012460 20 0a 65 6e 63 61 70 73 75 6c 61 74 65 20 69 74 > .encapsulate it< 0012500 3a 0a 0a 20 c2 a0 c2 a0 c2 a0 20 64 65 66 20 77 >:.. ...... def w< 0012520 72 61 70 70 65 64 5f 6a 6f 69 6e 28 76 61 6c 75 >rapped_join(valu< From rhodri at kynesim.co.uk Tue Oct 31 13:55:09 2017 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 31 Oct 2017 17:55:09 +0000 Subject: Report on non-breaking spaces in posts In-Reply-To: References: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> Message-ID: On 31/10/17 17:23, Stefan Ram wrote: > Ned Batchelder writes: >> ?????? def wrapped_join(values, sep): > > Ok, here's a report on me seing non-breaking spaces in > posts in this NG. I have written this report so that you > can see that it's not my newsreader that is converting > something, because there is no newsreader involved. > > Here are some relevant lines from Ned's above post: > > |From: Ned Batchelder > |Newsgroups: comp.lang.python > |Subject: Re: How to join elements at the beginning and end of the list > |Message-ID: Hm. That suggests the mail-to-news gateway has a hand in things. > |Content-Type: text/plain; charset=utf-8; format=flowed > |Content-Transfer-Encoding: 8bit > | ?????? def wrapped_join(values, sep): [snippety snip] > |od -c tmp.txt > |... > |0012620 s u l a t e i t : \n \n ? ? ? > |0012640 ? ? ? d e f w r a p p e d _ > |... > | > |od -x tmp.txt > |... > |0012620 7573 616c 6574 6920 3a74 0a0a c220 c2a0 > |0012640 c2a0 20a0 6564 2066 7277 7061 6570 5f64 > |... > > And you can see, there are two octet pairs ?c220? and > ?c2a0? in the post (directly preceding ?def wrapped?). > (Compare with the Content-Type and Content-Transfer-Encoding > given above.) (Read table with a monospaced font:) > > corresponding > Codepoint UTF-8 ISO-8859-1 interpretation > > U+0020? c2 20 20? SPACE? > U+00A0 c2 a0 a0 NON-BREAKING SPACE > > This makes it clear that there really are codepoints > U+00A0 in what I get from the server, i.e., non-breaking > spaces directly in front of ?def wrapped?. And? Why does that bother you? A non-breaking space is a perfectly valid thing to put into a UTF-8 encoded message. The 0xc2 0x20 byte pair that you misidentify as a space is another matter entirely. 0xc2 0x20 is not a space in UTF-8. It is an invalid code sequence. I don't know how or where it was generated, but it really shouldn't have been. It might have been Ned's MUA, or some obscure bug in the mail-to-news gateway. Does anyone in a position to know have any opinions? -- Rhodri James *-* Kynesim Ltd From ben.usenet at bsb.me.uk Tue Oct 31 14:17:00 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 31 Oct 2017 18:17:00 +0000 Subject: Report on non-breaking spaces in posts References: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> Message-ID: <87bmknfahv.fsf@bsb.me.uk> ram at zedat.fu-berlin.de (Stefan Ram) writes: > ram at zedat.fu-berlin.de (Stefan Ram) writes: >>|od -c tmp.txt >>|... >>|0012620 s u l a t e i t : \n \n ? ? ? >>|0012640 ? ? ? d e f w r a p p e d _ >>|... >>| >>|od -x tmp.txt >>|... >>|0012620 7573 616c 6574 6920 3a74 0a0a c220 c2a0 >>|0012640 c2a0 20a0 6564 2066 7277 7061 6570 5f64 >>|... > > PS: > > Oh, the byte order of ?od -x? did not match my expectations! You might like http://www.bsb.me.uk/software/utf-8-dump/ I use it all the time (but then I would!). -- Ben. From ben.usenet at bsb.me.uk Tue Oct 31 17:37:27 2017 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Tue, 31 Oct 2017 21:37:27 +0000 Subject: Report on non-breaking spaces in posts References: <0b6ba27b-1127-f812-da22-df1e005f75f1@nedbatchelder.com> Message-ID: <87375zf17s.fsf@bsb.me.uk> Rhodri James writes: > On 31/10/17 17:23, Stefan Ram wrote: >> Ned Batchelder writes: >>> ?????? def wrapped_join(values, sep): >> >> Ok, here's a report on me seing non-breaking spaces in >> posts in this NG. I have written this report so that you >> can see that it's not my newsreader that is converting >> something, because there is no newsreader involved. >> >> Here are some relevant lines from Ned's above post: >> >> |From: Ned Batchelder >> |Newsgroups: comp.lang.python >> |Subject: Re: How to join elements at the beginning and end of the list >> |Message-ID: > > Hm. That suggests the mail-to-news gateway has a hand in things. > >> |Content-Type: text/plain; charset=utf-8; format=flowed >> |Content-Transfer-Encoding: 8bit >> | ?????? def wrapped_join(values, sep): > > [snippety snip] > >> |od -c tmp.txt >> |... >> |0012620 s u l a t e i t : \n \n ? ? ? >> |0012640 ? ? ? d e f w r a p p e d _ >> |... >> | >> |od -x tmp.txt >> |... >> |0012620 7573 616c 6574 6920 3a74 0a0a c220 c2a0 >> |0012640 c2a0 20a0 6564 2066 7277 7061 6570 5f64 >> |... >> >> And you can see, there are two octet pairs ?c220? and >> ?c2a0? in the post (directly preceding ?def wrapped?). >> (Compare with the Content-Type and Content-Transfer-Encoding >> given above.) (Read table with a monospaced font:) >> >> corresponding >> Codepoint UTF-8 ISO-8859-1 interpretation >> >> U+0020? c2 20 20? SPACE? >> U+00A0 c2 a0 a0 NON-BREAKING SPACE >> >> This makes it clear that there really are codepoints >> U+00A0 in what I get from the server, i.e., non-breaking >> spaces directly in front of ?def wrapped?. > > And? Why does that bother you? A non-breaking space is a perfectly > valid thing to put into a UTF-8 encoded message. But it's an odd thing to put into Python code (at least there). If the Usenet client is doing it that's surely bad as the code won't run without editing. > The 0xc2 0x20 byte > pair that you misidentify as a space is another matter entirely. > > 0xc2 0x20 is not a space in UTF-8. It is an invalid code sequence. I > don't know how or where it was generated, but it really shouldn't have > been. It wasn't there. It was down to a misreading of the byte-order in the hex dump. -- Ben. From formisc at gmail.com Tue Oct 31 19:40:26 2017 From: formisc at gmail.com (Andrew Z) Date: Tue, 31 Oct 2017 19:40:26 -0400 Subject: matplot plot hangs Message-ID: hello, learning python's plotting by using matplotlib with python35 on fedora 24 x86. Installed matplotlib into user's directory. tk, seemed to work - http://www.tkdocs.com/tutorial/install.html#installlinux - the window shows up just fine. but when trying to run the simple plot ( https://matplotlib.org/examples/pylab_examples/simple_plot.html) the script is hanging on; plt.plot(t, s) attempts to matplotlib.interactive(True) didn't bring anything, I do get an error while trying to switch the backend ( which is fine with me) : plt.switch_backend('Qt4Agg') >>> plt.switch_backend('Qt4Agg') Traceback (most recent call last): File "", line 1, in File "/home/az/.local/lib/python3.5/site-packages/matplotlib/pyplot.py", line 231, in switch_backend _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup() File "/home/az/.local/lib/python3.5/site-packages/matplotlib/backends/__init__.py", line 60, in pylab_setup [backend_name], 0) File "/home/az/.local/lib/python3.5/site-packages/matplotlib/backends/backend_qt4agg.py", line 10, in from .backend_qt4 import ( File "/home/az/.local/lib/python3.5/site-packages/matplotlib/backends/backend_qt4.py", line 18, in from .qt_compat import QtCore, QtWidgets, _getSaveFileName, __version__ File "/home/az/.local/lib/python3.5/site-packages/matplotlib/backends/qt_compat.py", line 150, in from PyQt4 import QtCore, QtGui ImportError: *No module named 'PyQt4'* >>> plt.switch_backend('Agg') >>> plt.switch_backend('TkAgg') summary: why i can't use tkAgg, and how can i troubleshoot the issue? Thank you From rantingrickjohnson at gmail.com Tue Oct 31 20:23:49 2017 From: rantingrickjohnson at gmail.com (Rick Johnson) Date: Tue, 31 Oct 2017 17:23:49 -0700 (PDT) Subject: The syntax of replacement fields in format strings In-Reply-To: References: Message-ID: <37c90dff-39fe-4bac-a2ed-ce475fc1a032@googlegroups.com> On Tuesday, October 31, 2017 at 1:35:33 PM UTC-5, John Smith wrote: > If we keep the current implementation as is, perhaps the > documentation should at least be altered ? You should supply a concise code example that showcases why _you_ feel the docs are not satisfactory. It would help. From alb at nospam.chip.org Tue Oct 31 22:35:32 2017 From: alb at nospam.chip.org (Alberto Riva) Date: Tue, 31 Oct 2017 22:35:32 -0400 Subject: Invoking return through a function? In-Reply-To: References: <59f5e736$0$18583$b1db1813$d948b532@news.astraweb.com> <1emJB.169114$_O1.43917@fx31.am4> <8ba8c8c4-70d5-a7c2-cb5c-1080862b5a70@kynesim.co.uk> Message-ID: On 10/31/2017 11:01 AM, Rhodri James wrote: > On 31/10/17 02:06, Alberto Riva wrote: > > Steve D'Aprano gave you a pretty full answer, I just wanted to add: > >> ?The kind of >> statement I was trying to add would at least have made that explicit: >> return-if-so-and-so-happens. > > That's only obvious in the function that's doing the returning.? The > function that's doing the calling that gets its expectation of flow > control broken has no clue, and that's my problem. Sorry, I wasn't clear: I meant that the function that's doing the returning should be called "return-if-so-and-so-happens", literally ;) So when it appears in the body of another function, it's clear that it *may* invoke a return. Not saying it's a good idea, but just that when you read something like that you can expect evaluation flow to be disrupted. Again: I'm not saying it's a good idea, please don't take this as a serious feature request :) Alberto -- E-mail address: ((lambda (s a b c) (map 'string #'code-char (mapcar (lambda (v) (setq s (+ a (mod (+ s v (- a)) b)))) (map 'list (lambda (v) (- (char-code v) c)) " 1`-THUZ&+Wh1")))) 97 46 73 32)