From rustompmody at gmail.com Sat Oct 1 00:16:54 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 30 Sep 2016 21:16:54 -0700 (PDT) Subject: how to append to list in list comprehension In-Reply-To: References: <534d5506-1810-4a79-ac8f-95a664d17827@googlegroups.com> Message-ID: On Saturday, October 1, 2016 at 9:08:09 AM UTC+5:30, Sayth Renshaw wrote: > I do like [(f + ['0'] if len(f) < 5 else f) for f in fups ] Rustom, if there are better non list comprehension options I would like to know as generally I find then confusing. Two points here ? best taken independently: 1. List comprehensions are confusing 2. When to want/not want them For 1 I suggest you (privately) rewrite them with '|' for 'for' and '?' for 'in' Once you do that they will start looking much more like the origin that inspires them ? set builder notation: https://en.wikipedia.org/wiki/Set-builder_notation >From there I suggest you play with replacing '[]' with '{}' ie actually try out set comprehensions and then others like dict-comprehensions ? very nifty and oft-neglected. And the mother of all ? generator comprehensions. Of course to check it out in python you will need to invert the translation: '|' for 'for' and '?' for 'in' the point of which is to use python as a kind of math assembly language *into* which you *code* but not in which you *think* For 2 its important that you always keep in front of you whether you want to approach a problem declaratively (the buzzword FP!) or imperatively. Python is rather unique in the extent to which it allows both This also makes it uniquely difficult because its all too easy to garble the two styles as John's .append inside a LC illustrates. And the way to ungarble your head is by asking yourself the meta-question: Should I be asking "How to solve this (sub)problem?" or more simply "What is the (sub)problem I wish to solve?" How questions naturally lead to imperative answers; whats to declarative You may be helped with [plug!] my writings on FP: http://blog.languager.org/search/label/FP Particularly the tables in: http://blog.languager.org/2016/01/primacy.html From rustompmody at gmail.com Sat Oct 1 00:32:14 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 30 Sep 2016 21:32:14 -0700 (PDT) Subject: Lawrence D'Oliveiro In-Reply-To: References: <57EE9DC8.2080501@stoneleaf.us> Message-ID: On Friday, September 30, 2016 at 10:45:03 PM UTC+5:30, Ethan Furman wrote: > Lawrence D'Oliveiro is banned from Python List until the new year. > > If his posts show up somewhere else (e.g. Usenet), please ignore them. If you must respond to him please trim out his text so the rest of us don't have to deal with him. > > -- > Python List Moderators Sure Lawrence was/is being rude/difficult/whatever but does it warrant that much severity? If we consider the 4 cases (I know in recent times) that have been banned 1. Mark Lawrence 2. Thomas Lahn 3. jmf 4. and now Lawrence d'Oliveiro that is also the (my subjective) order of rudeness (jmf is never rude but highly troublesome) Of course you/other-mods will have your own metrics But please run your own abuse-metric-eval on this case again And there is found a bug therein consider toning down 2 months to (say) 2 weeks None of the above gainsays the fact that you folks ? list-mods ? do a onerous splendid, mostly invisible job. For which thanks! From rustompmody at gmail.com Sat Oct 1 00:37:47 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 30 Sep 2016 21:37:47 -0700 (PDT) Subject: Lawrence D'Oliveiro In-Reply-To: References: <57EE9DC8.2080501@stoneleaf.us> Message-ID: <92efb39d-34ca-4c2e-9a7f-783c0b0dc052@googlegroups.com> On Saturday, October 1, 2016 at 10:04:01 AM UTC+5:30, Rustom Mody wrote: > On Friday, September 30, 2016 at 10:45:03 PM UTC+5:30, Ethan Furman wrote: > > Lawrence D'Oliveiro is banned from Python List until the new year. > > > > If his posts show up somewhere else (e.g. Usenet), please ignore them. If you must respond to him please trim out his text so the rest of us don't have to deal with him. > > > > -- > > Python List Moderators > > Sure Lawrence was/is being rude/difficult/whatever but does it warrant that much > severity? > > If we consider the 4 cases (I know in recent times) that have been banned > 1. Mark Lawrence > 2. Thomas Lahn > 3. jmf > 4. and now Lawrence d'Oliveiro > > that is also the (my subjective) order of rudeness (jmf is never rude but > highly troublesome) I meant to say ?in decreasing order? From rustompmody at gmail.com Sat Oct 1 00:43:34 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Fri, 30 Sep 2016 21:43:34 -0700 (PDT) Subject: unintuitive for-loop behavior In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <1475292305.1216671.742528841.7309BC44@webmail.messagingengine.com> Message-ID: <26027c4d-0a06-4df0-a323-6dbd211726d4@googlegroups.com> On Saturday, October 1, 2016 at 8:55:19 AM UTC+5:30, Random832 wrote: > On Fri, Sep 30, 2016, at 20:46, Gregory Ewing wrote: > > What *is* necessary and sufficient is to make each iteration > > of the for-loop create a new binding of the loop variable > > (and not any other variable!). > > I don't think that's true. I think this is logic that is excessively > tied to the toy examples that are used to illustrate the problem. > > You don't think it's common [at least, as far as defining a lambda > inside a loop at all is common] to do something like this? > > for a in collection: > b = some_calculation_of(a) > do_something_with(lambda: ... b ...) Common? ? Dunno What I know ? Yuck! [Yeah? someone brought up on Lisp and APL and who finds C++ terrifying!] From flebber.crue at gmail.com Sat Oct 1 01:44:16 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Fri, 30 Sep 2016 22:44:16 -0700 (PDT) Subject: how to append to list in list comprehension In-Reply-To: References: <534d5506-1810-4a79-ac8f-95a664d17827@googlegroups.com> Message-ID: <877d2494-41b7-4677-96cb-ca4712762e96@googlegroups.com> On Saturday, 1 October 2016 14:17:06 UTC+10, Rustom Mody wrote: > On Saturday, October 1, 2016 at 9:08:09 AM UTC+5:30, Sayth Renshaw wrote: > > I do like [(f + ['0'] if len(f) < 5 else f) for f in fups ] Rustom, if there are better non list comprehension options I would like to know as generally I find then confusing. > > Two points here ? best taken independently: > 1. List comprehensions are confusing > 2. When to want/not want them > > > For 1 I suggest you (privately) rewrite them with '|' for 'for' and '?' for 'in' > Once you do that they will start looking much more like the origin that inspires > them ? set builder notation: > https://en.wikipedia.org/wiki/Set-builder_notation > > From there I suggest you play with replacing '[]' with '{}' ie actually try > out set comprehensions and then others like dict-comprehensions ? very nifty > and oft-neglected. And the mother of all ? generator comprehensions. > > Of course to check it out in python you will need to invert the translation: > '|' for 'for' and '?' for 'in' > the point of which is to use python as a kind of math assembly language > *into* which you *code* but not in which you *think* > > For 2 its important that you always keep in front of you whether you want to > approach a problem declaratively (the buzzword FP!) or imperatively. > Python is rather unique in the extent to which it allows both > This also makes it uniquely difficult because its all too easy to garble the > two styles as John's .append inside a LC illustrates. > > And the way to ungarble your head is by asking yourself the meta-question: > Should I be asking "How to solve this (sub)problem?" or more simply > "What is the (sub)problem I wish to solve?" > > How questions naturally lead to imperative answers; whats to declarative > > You may be helped with [plug!] my writings on FP: > http://blog.languager.org/search/label/FP > > Particularly the tables in: > http://blog.languager.org/2016/01/primacy.html Thank You Rustom From flebber.crue at gmail.com Sat Oct 1 02:02:03 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Fri, 30 Sep 2016 23:02:03 -0700 (PDT) Subject: how to append to list in list comprehension In-Reply-To: References: <534d5506-1810-4a79-ac8f-95a664d17827@googlegroups.com> Message-ID: <50c6fed7-0066-458c-973e-facee39852ce@googlegroups.com> On Saturday, 1 October 2016 14:17:06 UTC+10, Rustom Mody wrote: > On Saturday, October 1, 2016 at 9:08:09 AM UTC+5:30, Sayth Renshaw wrote: > > I do like [(f + ['0'] if len(f) < 5 else f) for f in fups ] Rustom, if there are better non list comprehension options I would like to know as generally I find then confusing. > > Two points here ? best taken independently: > 1. List comprehensions are confusing > 2. When to want/not want them > > > For 1 I suggest you (privately) rewrite them with '|' for 'for' and '?' for 'in' > Once you do that they will start looking much more like the origin that inspires > them ? set builder notation: > https://en.wikipedia.org/wiki/Set-builder_notation > > From there I suggest you play with replacing '[]' with '{}' ie actually try > out set comprehensions and then others like dict-comprehensions ? very nifty > and oft-neglected. And the mother of all ? generator comprehensions. > > Of course to check it out in python you will need to invert the translation: > '|' for 'for' and '?' for 'in' > the point of which is to use python as a kind of math assembly language > *into* which you *code* but not in which you *think* > > For 2 its important that you always keep in front of you whether you want to > approach a problem declaratively (the buzzword FP!) or imperatively. > Python is rather unique in the extent to which it allows both > This also makes it uniquely difficult because its all too easy to garble the > two styles as John's .append inside a LC illustrates. > > And the way to ungarble your head is by asking yourself the meta-question: > Should I be asking "How to solve this (sub)problem?" or more simply > "What is the (sub)problem I wish to solve?" > > How questions naturally lead to imperative answers; whats to declarative > > You may be helped with [plug!] my writings on FP: > http://blog.languager.org/search/label/FP > > Particularly the tables in: > http://blog.languager.org/2016/01/primacy.html Your insight has helped. May lack elegance but I have got it working. from lxml import etree import csv import re def clean(attr): p = re.compile('\d+') myList = p.findall(attr) if len(myList) < 5: myList.append('0') return myList[0], myList[1], myList[2], myList[3], myList[4] with open("20161001RAND0.xml", 'rb') as f, open( "output/310916RABD.csv", 'w', newline='') as csvf: tree = etree.parse(f) root = tree.getroot() race_writer = csv.writer(csvf, delimiter=',') for meet in root.iter("meeting"): for race in root.iter("race"): for nom in root.iter("nomination"): meetattr = meet.attrib raceattr = race.attrib nomattr = nom.attrib if nomattr['number'] != '0': print(clean(nomattr['firstup'])) Cheers Sayth From bob.martin at excite.com Sat Oct 1 07:19:48 2016 From: bob.martin at excite.com (Bob Martin) Date: Sat, 01 Oct 2016 07:19:48 BST Subject: Lawrence D'Oliveiro References: <57EE9DC8.2080501@stoneleaf.us> Message-ID: in 765690 20160930 181552 Ethan Furman wrote: >Lawrence D'Oliveiro is banned from Python List until the new year. > >If his posts show up somewhere else (e.g. Usenet), please ignore them. If you must respond to him p >lease trim out his text so the rest of us don't have to deal with him. > >-- >Python List Moderators Why can't you block "PEDOFILO"? From rosuav at gmail.com Sat Oct 1 02:24:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Oct 2016 16:24:43 +1000 Subject: Lawrence D'Oliveiro In-Reply-To: References: <57EE9DC8.2080501@stoneleaf.us> Message-ID: On Sat, Oct 1, 2016 at 3:19 PM, Bob Martin wrote: > in 765690 20160930 181552 Ethan Furman wrote: >>Lawrence D'Oliveiro is banned from Python List until the new year. >> >>If his posts show up somewhere else (e.g. Usenet), please ignore them. If you must respond to him p >>lease trim out his text so the rest of us don't have to deal with him. >> >>-- >>Python List Moderators > > Why can't you block "PEDOFILO"? I've no idea who you're talking about, so my guess is that those posts already aren't making the leap to the mailing list. The newsgroup isn't under the same governance, so if you're reading netnews, you have to manually killfile people. <<< "Excellent. What is a killfile?" "Uh. It's a list of usernames/topics/news items etc that you wish the news- reader to automatically skip so you don't have to wade through rubbish" "Uh No. Remember I said pertaining to Operations. A killfile is in fact a file with a list of names of people you are going to have killed." "Oh. Of course." >>> -- The BSMFH, formerly and subsequently known as the BOFH ChrisA From no.email at nospam.invalid Sat Oct 1 02:29:41 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Fri, 30 Sep 2016 23:29:41 -0700 Subject: Lawrence D'Oliveiro References: <57EE9DC8.2080501@stoneleaf.us> Message-ID: <87lgy81t0q.fsf@nightsong.com> Chris Angelico writes: >> Why can't you block "PEDOFILO"? > I've no idea who you're talking about That's the weird Italian spam that the newsgroup has been getting for a while. I've been wondering for a while if anyone knows what the story is, i.e. why it's on comp.lang.python but not on other newsgroups that I've noticed. From steve+python at pearwood.info Sat Oct 1 03:06:02 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 17:06:02 +1000 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef32ea$0$1612$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> Earlier, I wrote: > On Sat, 1 Oct 2016 10:46 am, Gregory Ewing wrote: [...] >> Whenever there's binding going on, it's necessary to decide >> whether it should be creating a new binding or updating an >> existing one. > > Right. I changed my mind -- I don't think that's correct. I think Greg's suggestion only makes sense for languages where variables are boxes with fixed locations, like C or Pascal. In that case, the difference between creating a new binding and updating a new one is *possibly* meaningful: # create a new binding x: address 1234 ----> [ box contains 999 ] x: address 5678 ----> [ a different box, containing 888 ] What happens to the old x? I have no idea, but the new x is a different box. Maybe the old box remains there, for any existing code that refers to the address of (old) x. Maybe the compiler is smart enough to add address 1234 to the free list of addresses ready to be used for the next variable. Maybe its just lost and unavailable until this function exists. # update an existing binding x: address 1234 ----> [ box contains 999 ] x: address 1234 ----> [ same box, now contains 888 ] That's the normal behaviour of languages like C and Pascal. But its distinct from the previous, hypothetical behaviour. But Python doesn't work that way! Variables aren't modelled by boxes in fixed locations, and there is no difference between "create a new binding" and "update an existing one". They are indistinguishable. In both cases, 'x' is a key in a namespace dict, which is associated with a value. There's no difference between: x = 999 del x x = 888 and x = 999 x = 888 If you consider the namespace dict as a hash table, e.g. something like this (actual implementations may differ): [ UNUSED, UNUSED, (key='y', value=23), (key='a', value=True), UNUSED, (key='x', value=999), UNUSED ] then binding 888 to 'x' must put the key in the same place in the dict, since that's where hash('x') will point. Subject to linear addressing, chaining, re-sizes, or other implementation details of hash tables of course. But all else being equal, you cannot distinguish between the "new binding" and "update existing binding" cases -- in both cases, the same cell in the hash table gets affected, because that's where hash('x') points. If you put it somewhere else, it cannot be found. So I think I was wrong to agree with Greg's statement. I think that for languages like Python where variables are semantically name bindings in a namespace rather than fixed addresses, there is no difference between updating an existing binding and creating a new one. In a language like Python, the only distinction we can make between name bindings is, which namespace is the binding in? In other words, what is the current block of code's scope? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Oct 1 03:41:26 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Oct 2016 17:41:26 +1000 Subject: unintuitive for-loop behavior In-Reply-To: <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef32ea$0$1612$c3e8da3$5496439d@news.astraweb.com> <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 1, 2016 at 5:06 PM, Steve D'Aprano wrote: > Earlier, I wrote: > >> On Sat, 1 Oct 2016 10:46 am, Gregory Ewing wrote: > [...] >>> Whenever there's binding going on, it's necessary to decide >>> whether it should be creating a new binding or updating an >>> existing one. >> >> Right. > > I changed my mind -- I don't think that's correct. > > I think Greg's suggestion only makes sense for languages where variables are > boxes with fixed locations, like C or Pascal. In that case, the difference > between creating a new binding and updating a new one is *possibly* > meaningful: > > # create a new binding > x: address 1234 ----> [ box contains 999 ] > x: address 5678 ----> [ a different box, containing 888 ] > > What happens to the old x? I have no idea, but the new x is a different box. > Maybe the old box remains there, for any existing code that refers to the > address of (old) x. Maybe the compiler is smart enough to add address 1234 > to the free list of addresses ready to be used for the next variable. Maybe > its just lost and unavailable until this function exists. > > # update an existing binding > x: address 1234 ----> [ box contains 999 ] > x: address 1234 ----> [ same box, now contains 888 ] > > > That's the normal behaviour of languages like C and Pascal. But its distinct > from the previous, hypothetical behaviour. > > But Python doesn't work that way! Variables aren't modelled by boxes in > fixed locations, and there is no difference between "create a new binding" > and "update an existing one". They are indistinguishable. In both > cases, 'x' is a key in a namespace dict, which is associated with a value. > ... > In a language like Python, the only distinction we can make between name > bindings is, which namespace is the binding in? In other words, what is the > current block of code's scope? Yes, this is true; however, it's not difficult to create subscopes. I put together a POC patch a while ago to make 'with' blocks able to put their capture names into a separate subscope, transparently. I don't think it's what Python wants, but it would be reasonably coherent semantically ("with expr as name:" puts name, and name alone, into a subscope - any other assignments are not), and wasn't difficult to implement in CPython. ChrisA From steve+python at pearwood.info Sat Oct 1 04:35:15 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 18:35:15 +1000 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57ef7545$0$1588$c3e8da3$5496439d@news.astraweb.com> On Sat, 1 Oct 2016 02:39 am, Chris Angelico wrote: > On Sat, Oct 1, 2016 at 12:36 AM, Grant Edwards > wrote: >> In C99 a for loop has its own namespac: [...] > I believe that's the same semantics as C++ uses, and I agree, it's > very convenient. Among other things, it means that nested loops behave > more like they do in Python, with independent iterators: *scratches head* So let me see if I understand this... You're suggesting that C99, where for-loops have their own namespaces, is MORE like Python (where for-loops DON'T have their own namespace), than C89, which, like Python, DOESN'T give for-loops their own namespace? That's ... curious. I'm not saying you're wrong: after spending three quarters of an hour trying to fix a six line (including one blank line) C program because I accidentally closed a comment with /* instead of */, I will believe anything about C[1]. If you tell me that void causes birth defects and printf is responsible for the police shootings of unarmed black men, I'll believe every word of it. > int main(void) > { > for (int i=0; i<5; ++i) > { > printf("%d:", i); > for (int i=0; i<3; ++i) > printf(" %d", i); > printf("\n"); > } > } > > Now, granted, this is not something I would ever actually recommend > doing, and code review is absolutely justified in rejecting this... So let me see if I understand your argument... for-loop namespaces are good, because they let you write code that you personally wouldn't write and would, in fact, reject in a code review. O-kay. > but it's a lot better than pure function-scope variables, where you'd > get stuck in an infinite loop. That's the risk that you take when you have a C-style for loop and you modify the loop variable. There's nothing special about the inner loop in that regard: #include /* for printf */ int main(void) { for (int i=0; i<5; ++i) { printf("%d:", i); i = 0; } } Solution: don't do that. [1] Apparently useful error messages are one of those things C programmers eschew, like type safety, memory safety, and correctness. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jussi.piitulainen at helsinki.fi Sat Oct 1 04:44:13 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sat, 01 Oct 2016 11:44:13 +0300 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef32ea$0$1612$c3e8da3$5496439d@news.astraweb.com> <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano writes: > In a language like Python, the only distinction we can make between > name bindings is, which namespace is the binding in? In other words, > what is the current block of code's scope? Python already has nested scopes. A local scope for just those variables introduced in for-loops could be carved out as in the following example. The following example function has three parameters and two local variables, one of which is only assigned inside the loop (and it also references one global variable), in addition to the variable introduced in the loop. def eg(wev, a, b): y = 0 for x in wev: # to be given a new semantics f(a, x) f(x, b) y += 1 z = (y > 12) return y, z Replace the loop with a call to a thunk that refers to the variables in the outer scopes, except for the one variable that the loop introduces - that variable is to be local to the thunk. So the example function should be equivalent to something like the following translation. def eg(wev, a, b): y = 0 def g31(): nonlocal y, z # the assigned variables g32 = iter(wev) while 1: # need to catch StopIteration x = next(g32) # now local to g31 f(a, x) f(x, b) y += 1 z = (y > 12) g31() return y, z A different translation would introduce each x in a different function for each step in the iteration. That's just a variation on the overall theme. From flebber.crue at gmail.com Sat Oct 1 04:52:07 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 01:52:07 -0700 (PDT) Subject: generator no iter - how do I call it from another function Message-ID: Evening My file list handler I have created a generator. Before I created it as a generator I was able to use iter on my lxml root objects, now I cannot iter. ? |master U:2 ?:1 ?| ? python3 race.py data/ -e *.xml Traceback (most recent call last): File "race.py", line 83, in dataAttr(rootObs) File "race.py", line 61, in dataAttr for meet in roots.iter("meeting"): AttributeError: 'generator' object has no attribute 'iter' How do I now pull the next iterations through from the other function? from lxml import etree import csv import re import argparse import os parser = argparse.ArgumentParser() parser.add_argument("path", type=str, nargs="+") parser.add_argument( '-e', '--extension', default='', help='File extension to filter by.') # >python race.py XML_examples/ -e .xml args = parser.parse_args() name_pattern = "*" + args.extension my_dir = args.path[0] for dir_path, subdir_list, file_list in os.walk(my_dir): for name_pattern in file_list: full_path = os.path.join(dir_path, name_pattern) def return_files(file_list): """ Take a list of files and return file when called. Calling function to supply attributes """ for filename in sorted(file_list): with open(dir_path + filename) as fd: tree = etree.parse(fd) root = tree.getroot() yield root def clean(attr): """ Split list into lists of 5 elements. if list is less than 5 in length then a 0 is appended and the list returned """ p = re.compile('\d+') myList = p.findall(attr) if len(myList) < 5: myList.append('0') return myList[0], myList[1], myList[2], myList[3], myList[4] def dataAttr(roots): """Get the root object and iter items.""" with open("output/first2.csv", 'w', newline='') as csvf: race_writer = csv.writer(csvf, delimiter=',') for meet in roots.iter("meeting"): print(meet) for race in roots.iter("race"): for nom in roots.iter("nomination"): meetattr = meet.attrib raceattr = race.attrib nomattr = nom.attrib if nomattr['number'] != '0': firsts = clean(nomattr['firstup']) race_writer.writerow( [meetattr['id'], meetattr['date'], meetattr['venue'], raceattr['id'], raceattr['number'], raceattr['distance'], nomattr['id'], nomattr['barrier'], nomattr['weight'], nomattr['rating'], nomattr['description'], nomattr['dob'], nomattr['age'], nomattr['decimalmargin'], nomattr['saddlecloth'], nomattr['sex'], firsts[4]]) rootObs = return_files(file_list) dataAttr(rootObs) From flebber.crue at gmail.com Sat Oct 1 04:55:43 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 01:55:43 -0700 (PDT) Subject: rocket simulation game with just using tkinter In-Reply-To: <57eeee41$0$911$e4fe514c@news.xs4all.nl> References: <57eeee41$0$911$e4fe514c@news.xs4all.nl> Message-ID: On Saturday, 1 October 2016 08:59:28 UTC+10, Irmen de Jong wrote: > Hi, > > I've made a very simple rocket simulation game, inspired by the recent success of SpaceX > where they managed to land the Falcon-9 rocket back on a platform. > > I was curious if you can make a simple graphics animation game with just using Tkinter, > instead of using other game libraries such as PyGame. > As it turns out, that works pretty well and it was quite easy to write. Granted, there's > not much going on on the screen, but still the game runs very smoothly and I think it is > fun for a little while where you try to learn to control the rocket and attempt to > successfully land it on the other launchpad! > > The physics simulation is tied to the game's frame rate boohoo, but the upside is that > you can change the framerate to control the game's difficulty. It's easy at <=20, fun at > 30 and impossible at 60 :) It's running on 30 by default. > > > You can get the code here if you want to give it a try: > https://github.com/irmen/rocketsimulator > > So you just need python 2/3 with tkinter to play this! > > > Have fun > Irmen Well done. An interesting listen that might be up your alley, how-i-built-an-entire-game-and-toolchain-100-in-python on talkpython https://talkpython.fm/episodes/show/78/how-i-built-an-entire-game-and-toolchain-100-in-python Sayth From rosuav at gmail.com Sat Oct 1 05:02:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Oct 2016 19:02:52 +1000 Subject: unintuitive for-loop behavior In-Reply-To: <57ef7545$0$1588$c3e8da3$5496439d@news.astraweb.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef7545$0$1588$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 1, 2016 at 6:35 PM, Steve D'Aprano wrote: > On Sat, 1 Oct 2016 02:39 am, Chris Angelico wrote: > >> On Sat, Oct 1, 2016 at 12:36 AM, Grant Edwards >> wrote: > >>> In C99 a for loop has its own namespac: > [...] > >> I believe that's the same semantics as C++ uses, and I agree, it's >> very convenient. Among other things, it means that nested loops behave >> more like they do in Python, with independent iterators: > > *scratches head* > > So let me see if I understand this... > > You're suggesting that C99, where for-loops have their own namespaces, is > MORE like Python (where for-loops DON'T have their own namespace), than > C89, which, like Python, DOESN'T give for-loops their own namespace? > > That's ... curious. Well, when you put it like that, it does sound very curious!! Here's how I read it: # Python def main(): for i in range(5): print(i, end=": ") for i in range(3): print(i, end=" ") print("") /* Classic C */ int main() { int i; for (i=0; i<5; ++i) { printf("%d:", i); for (i=0; i<3; ++i) printf(" %d", i); printf("\n"); } //C++ or C99 int main() { for (int i=0; i<5; ++i) { printf("%d:", i); for (int i=0; i<3; ++i) printf(" %d", i); printf("\n"); } The first and last have *independent iterators*. When you complete the inner loop, the outer loop picks up where *it* left off, not where you happen to have reassigned the iteration variable. The middle one is an infinite loop. > I'm not saying you're wrong: after spending three quarters of an hour trying > to fix a six line (including one blank line) C program because I > accidentally closed a comment with /* instead of */, I will believe > anything about C[1]. If you tell me that void causes birth defects and > printf is responsible for the police shootings of unarmed black men, I'll > believe every word of it. Did you turn all warnings on? Look for a "pedantic" mode. You'll get a lot more information, assuming you're using a modern decent C compiler. (GCC, for instance, will even warn about situations where it looks like the braces and indentation don't match, thus bringing a Python feature to C, more-or-less.) >> int main(void) >> { >> for (int i=0; i<5; ++i) >> { >> printf("%d:", i); >> for (int i=0; i<3; ++i) >> printf(" %d", i); >> printf("\n"); >> } >> } >> >> Now, granted, this is not something I would ever actually recommend >> doing, and code review is absolutely justified in rejecting this... > > So let me see if I understand your argument... for-loop namespaces are good, > because they let you write code that you personally wouldn't write and > would, in fact, reject in a code review. > > O-kay. Yes. In the first place, I wouldn't necessarily reject this code; it's a minor issue only, thanks to the namespacing and nested scoping. It does cost a little in clarity, and I would point it out to a student, but in less trivial examples than this, it's really not that big a deal. *Without* namespacing to keep them apart, this is a serious problem. Thanks to namespacing, it's only a minor problem (you've shadowed one variable with another). It's comparable to this Python code: def send_message(from, to, subj, body): str = "Mail: From " + from str += "\nRecipient: To " + to str += "\nSubject: " + subj str += "\n\n" + body send_raw_data(str) Would you reject this in a code review? Maybe. I wouldn't yell at you saying "shadowing is fine, you have no right to reject that". But it's also not inherently a problem. The rules of name shadowing are well-defined (in both languages) and designed to *permit*, not reject, constructs like this. >> but it's a lot better than pure function-scope variables, where you'd >> get stuck in an infinite loop. > > That's the risk that you take when you have a C-style for loop and you > modify the loop variable. There's nothing special about the inner loop in > that regard: > > > #include /* for printf */ > int main(void) > { > for (int i=0; i<5; ++i) > { > printf("%d:", i); > i = 0; > } > } > > > Solution: don't do that. Actually, I would call this version a feature. If you deliberately and consciously reassign the iteration variable, it's usually because you wanted to adjust the loop. Classic example: iterating through a series of tokens, and sometimes needing to grab a second token. Python lets you do this by explicitly calling iter() and next(); C lets you do this by incrementing the loop counter (which works only with the less-flexible looping style it has). But when you didn't intend to reassign it, it's usually because you had a local name that happened to collide. Sure, that can happen anywhere, especially with abbreviated variable names (I once had the amusing situation of trying to use "cursor" and "current" in the same namespace, both abbreviated "cur"), but it's usually going to involve some form of inner block construct. It mightn't be a 'for' loop - it might be a 'while' loop, or it might not even be a loop at all, just one branch of an 'if'. Unless, of course, you're customarily writing thousand-line C programs with no control flow structures at all, in which case I have a nice padded cell for you right over here... > [1] Apparently useful error messages are one of those things C programmers > eschew, like type safety, memory safety, and correctness. Totally. Those error messages cost 0.0000001% in run time performance, so they had to go. In all seriousness, though... -Wall makes your C programming life a lot more bearable. ChrisA From michael at felt.demon.nl Sat Oct 1 05:31:42 2016 From: michael at felt.demon.nl (Michael Felt) Date: Sat, 1 Oct 2016 11:31:42 +0200 Subject: Merecurial and Python-2.7.x, Python-3.Y Message-ID: <448d675b-b46a-c7d3-e063-1eed6a4d8ac5@felt.demon.nl> Finally, I got to where I understood what needed to be done to get both Mercurial built - and the the new SSL requirements met. So, running: # hg clone https://hg.python.org/cpython works. What is the next step to getting Python-2.7 AND Python-3.7 so I can submit patches against both versions and/or other versions? Is it going to be a command like in the developer guide (that references Python3.5)? Does this create a new directory, or just undo a lot of things that was just cloned - or was 3.5 the development branch when the guide was last updated? Will I need to clone (as above) several times - one for each version I want to test against - and then run a second command -- OR -- is there an hg clone command to get a specific version (If that is in the guide - my apologies, as I missed it. Thanks for hints and patience! M From jfong at ms4.hinet.net Sat Oct 1 05:34:05 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 1 Oct 2016 02:34:05 -0700 (PDT) Subject: How to make a foreign function run as fast as possible in Windows? In-Reply-To: References: <682e117e-3c50-4e61-9493-bc3167f7a039@googlegroups.com> <6204fe91-c854-44a7-be3a-25d6265845b3@googlegroups.com> Message-ID: Chris Angelico at 2016/10/1 11:25:03AM wrote: > What's it doing? Not every task can saturate the CPU - sometimes they > need the disk or network more. > This function has no I/O or similar activity, just pure data processing, and it takes less than 200 bytes of data area to work with. My CPU is an i3 (4 threads/2 cores). I suppose after this job was assigned to run on a particular core, the OS shouldn't bother it by other system related tasks anymore. If it does, won't be this OS designed stupidly? I was puzzled why the core has no 100% usage? Why always has some percentage of idle? --Jach From flebber.crue at gmail.com Sat Oct 1 06:09:34 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 03:09:34 -0700 (PDT) Subject: generator no iter - how do I call it from another function In-Reply-To: References: Message-ID: My main issue is that usually its just x in ,,, for a generator. But if I change the code for meet in roots.iter("meeting"): to for meet in roots("meeting"): Well its invalid but I need to be able to reference the node, how do I achieve this? Sayth From vlastimil.brom at gmail.com Sat Oct 1 06:28:14 2016 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Sat, 1 Oct 2016 12:28:14 +0200 Subject: generator no iter - how do I call it from another function In-Reply-To: References: Message-ID: 2016-10-01 12:09 GMT+02:00 Sayth Renshaw : > My main issue is that usually its just x in ,,, for a generator. > > But if I change the code > for meet in roots.iter("meeting"): > > to > for meet in roots("meeting"): > > Well its invalid but I need to be able to reference the node, how do I achieve this? > > Sayth > -- Hi, i think, you identified the problem correctly in the previous mail: >> Before I created it as a generator I was able to use iter on my lxml root objects, now I cannot iter. It is not clear, what the previous - working - version was, but there is a difference between using built-in iteration handling (with built-in iter(...) or implicitly via "for elem in some_iterable_object: ...") and with the specialised "iter" method of root in the lxml library (which has specific functionalities). I don't have much experience with that library, but if your change was basically adding a "generator" layer around "roots", you may try to iterate over it to get its elements, which might supposedly support the method, you were using previously. e.g. (completely untested, as there is no xml underlying given in your sample) for root in roots: for meet in root.iter("meeting"): ... hth, vbr From greg.ewing at canterbury.ac.nz Sat Oct 1 06:56:45 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 01 Oct 2016 23:56:45 +1300 Subject: unintuitive for-loop behavior In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <1475292305.1216671.742528841.7309BC44@webmail.messagingengine.com> Message-ID: Random832 wrote: > for a in collection: > b = some_calculation_of(a) > final b: do_something_with(lambda: ... b ...) I would prefer something like for a in collection: let b = some_calculation_of(a): do_something_with(lambda: ... b ...) -- Greg From steve+python at pearwood.info Sat Oct 1 07:15:13 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 21:15:13 +1000 Subject: generator no iter - how do I call it from another function References: Message-ID: <57ef9ac3$0$1590$c3e8da3$5496439d@news.astraweb.com> On Sat, 1 Oct 2016 06:52 pm, Sayth Renshaw wrote: > Evening > > My file list handler I have created a generator. > > Before I created it as a generator I was able to use iter on my lxml root > objects, now I cannot iter. lxml root objects have an iter method. Other objects do not. Thank you for showing your code, but it is MUCH too complicated. Most of the code has nothing to do with your question. You are asking a question about generators, but your code shows argument processing, file processing, regexes, all sorts of things that we have to read and understand before we can answer your question, and none of it is relevant. That's too much work. We are volunteers. If you want to pay us, we'll be happy to spend hours on your code. But if you want free consulting, then you have to do some of the work to make it easy for us. Please cut your code down to the smallest possible amount that shows the problem: http://sscce.org/ If you have a generator, you can iterate over it like this: def generator(): yield 1 yield 2 yield 4 for value in generator(): # NOT generator.iter() print(value) Does that answer your question? If not, then cut your code down to the smallest possible amount that shows the problem: http://sscce.org/ and I'll read it again. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 1 07:21:34 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 21:21:34 +1000 Subject: Copying a compiled Python from one system to another Message-ID: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Long story short: I have no working systems capable of compiling the latest Python 3.6, and no time to upgrade my usual machines to something which will work. However I do have access to another machine (actually a VM) which can compile Python 3.6. It's not practical for me to use it as a my main development machine, but as a temporary measure, I thought I could compile 3.6 on this VM, then copy the python binary to my usual desktop machine. What sort of challenges am I likely to find? Both machines are Linux, but different distros. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 1 07:32:17 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 21:32:17 +1000 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> Message-ID: <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> On Sat, 1 Oct 2016 01:44 pm, Rustom Mody wrote: > Yes one basic problem with comprehensions in python is that they are > defined by assignment not binding to the comprehension variable ?Que Mr Fawlty? I'm sorry, I don't understand you. In Python, all assignments are name bindings. So you seem to be saying: one basic problem with comprehensions in python is that they are defined by binding (to the comprehension variable) not binding to the comprehension variable which confuses me. How do you think that comprehension variables are different from all other variables? And while we're at it, what do you mean by "comprehension variable"? Are you talking about the variable that the comprehension is bound to: comprehension_variable = [x+1 for x in sequence] or are you talking about the loop variable inside the comprehension? foo = [comprehension_variable+1 for comprehension_variable in sequence] It is not clear what you mean. Although I suspect it makes no difference -- either way, the nature of the assignment is identical: its a name binding. > Python copied comprehensions from haskell and copied them wrong > Here are all the things (that I can think of) that are wrong: > 1. Scope leakage from inside to outside the comprehension The core devs agree that this was a mistake. It was rectified in generator expressions, and for comprehensions in Python 3. This is no longer an issue. > 2. Scope leakage from one value to the next I don't know what that means. > 3. The name 'for' misleadingly associates for-loops and comprehensions What is misleading about it? > 4. The for-loop based implementation strategy made into definitional > semantics > 5. The absence of simple binding inside comprehensions: > [f(newvar) for v in l newvar = rhs] I don't know what this means. > 1 was considered sufficiently important to make a breaking change from > python2 to 3 Correct. > 2 is what causes the lambda gotcha I don't think so. > 3 is what makes noobs to take longer than necessary to grok them You haven't shown any evidence that beginners take longer than necessary to grok list comprehensions. Perhaps they take exactly as long as necessary. And you CERTAINLY haven't demonstrated that the average beginner would understand Haskell's list comprehensions more easily than Python's. https://wiki.haskell.org/List_comprehension https://www.haskell.org/onlinereport/exps.html#sect3.11 > 4 is what causes a useless distinction between 1 and 2 ? scope leakage is > scope leakage. What is scope leakage? > The explanatory mechanisms of why/whither/what etc should > at best be secondary 5. is workaroundable with a [... for newvar in [rhs]] > Possible and clunky I don't know what this means. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From greg.ewing at canterbury.ac.nz Sat Oct 1 07:33:30 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 02 Oct 2016 00:33:30 +1300 Subject: unintuitive for-loop behavior In-Reply-To: <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef32ea$0$1612$c3e8da3$5496439d@news.astraweb.com> <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steve D'Aprano wrote: > # create a new binding > x: address 1234 ----> [ box contains 999 ] > x: address 5678 ----> [ a different box, containing 888 ] In the context of CPython and nested functions, replace "box" with "cell". When I said "creating a new binding" I meant that the name x refers to different cells at different times. When I said "updating an existing binding" I meant that the name x still refers to the same cell, but that cell refers to a different object. In a wider context, replace "box" with "slot in a stack frame" or "slot in a namespace dictionary". > But Python doesn't work that way! Variables aren't modelled by boxes in > fixed locations, and there is no difference between "create a new binding" > and "update an existing one". There is very much a distintion. Each time you invoke a function, a new set of bindings is created for all of its parameters and local names. Assigning to those names within the function, on the other hand, updates existing bindings. -- Greg From rosuav at gmail.com Sat Oct 1 08:01:29 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 1 Oct 2016 22:01:29 +1000 Subject: Copying a compiled Python from one system to another In-Reply-To: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 1, 2016 at 9:21 PM, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the latest > Python 3.6, and no time to upgrade my usual machines to something which > will work. > > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could compile > 3.6 on this VM, then copy the python binary to my usual desktop machine. > > What sort of challenges am I likely to find? Both machines are Linux, but > different distros. First off, are they the same CPU architecture? If not, you're going to have a lot of hassles. (This includes x86_64 vs i386. You *can* run a 32-bit Python on a 64-bit Linux, but it'll be a pain.) I'm going to assume that they are. Are both Linuxes of broadly similar vintage? The most likely problems you'll face will be library versions - CPython will happily link against any of several compatible versions of a library, but once it's linked against one, it'll want to find the same version thereafter. Worst case, you could symlink, but that could cause trouble when you upgrade, so I'd advise against it. In fact, your best bet might actually be to *minimize* the number of dev libraries on the build machine. You'll have build failures in optional modules (and thus less modules), but less chances of running into difficulties. Other than that, it should be possible. You'll probably want to export the built Python as a package manager file, eg using checkinstall [1] on Debian-based systems, as it'll be a lot easier to pick up and carry around. But it shouldn't be too hard even working manually. ChrisA [1] https://wiki.debian.org/CheckInstall From rustompmody at gmail.com Sat Oct 1 08:24:38 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 1 Oct 2016 05:24:38 -0700 (PDT) Subject: unintuitive for-loop behavior In-Reply-To: <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Saturday, October 1, 2016 at 5:02:30 PM UTC+5:30, Steve D'Aprano wrote: > On Sat, 1 Oct 2016 01:44 pm, Rustom Mody wrote: > > > Yes one basic problem with comprehensions in python is that they are > > defined by assignment not binding to the comprehension variable > > ?Que Mr Fawlty? > > I'm sorry, I don't understand you. > > In Python, all assignments are name bindings. So you seem to be saying: > > one basic problem with comprehensions in python is that they > are defined by binding (to the comprehension variable) not > binding to the comprehension variable > > which confuses me. How do you think that comprehension variables are > different from all other variables? Sorry... I guess you are right [for once ;-) ] I was making a *generic* distinction between binding-constructs that *create* variables, eg function-defs, lambdas, excepts, withs etc and assignment that *changes* a binding. This distinction is standard in compiled languages like C where a int x; creates the x and x=rhs; modifies it C++ preserves this distinction in user defined types making sure that the constructor called for an assignment and for an initialization are distinct even though they look very similar: x = rhs; vs T x = rhs; In functional languages, both the dynamic ones like lisp as well as the static ones like Haskell are very strict and sacrosanct about this distinction. However in python this does not work because assignment both assigns and creates the variable. Interestingly there is this thread running right now on haskell-cafe: https://groups.google.com/forum/#!topic/haskell-cafe/6tqMdy9nGdc which is inspired by python's ?batteries included? And there there is this comment: ?Python is even more imperative than C++ or Java, it's dynamically typed?? which (I guess) is because of exactly such features Long story short: My saying ?binding? is meaningless in python. I should have said function-binding; ie the mechanism by which functions bind their arguments *anew* And then define comprehensions not as now done in terms of for loops that mutatingly extend the list being built up but as recursive functions that get (re)called for every new value of the comprehension variable passed and therefore fresh-bound as parameter > > And while we're at it, what do you mean by "comprehension variable"? Are you > talking about the variable that the comprehension is bound to: > > comprehension_variable = [x+1 for x in sequence] > > or are you talking about the loop variable inside the comprehension? > > foo = [comprehension_variable+1 for comprehension_variable in sequence] > Yeah by comprehension-variable I mean the one that sits left of the ?in? inside The other obviously needn't exist From steve+python at pearwood.info Sat Oct 1 09:03:39 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 23:03:39 +1000 Subject: Copying a compiled Python from one system to another References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57efb42c$0$1614$c3e8da3$5496439d@news.astraweb.com> On Sat, 1 Oct 2016 10:01 pm, Chris Angelico wrote: >> [...] I thought I could >> compile 3.6 on this VM, then copy the python binary to my usual desktop >> machine. >> >> What sort of challenges am I likely to find? Both machines are Linux, but >> different distros. > > First off, are they the same CPU architecture? I'll say yes. > Are both Linuxes of broadly similar vintage? That depends on what you mean by "broadly similar". As far as I am concerned, a five year difference is not very much, and is broadly similar -- it's not like I'm using Linux from 1991. But the whole point is that I need something with gcc 4.8 (maybe 4.7 will do, not sure) but certainly not 4.4 which is the most recent I can get on my current systems. > The most likely problems > you'll face will be library versions - CPython will happily link > against any of several compatible versions of a library, but once it's > linked against one, it'll want to find the same version thereafter. > Worst case, you could symlink, but that could cause trouble when you > upgrade, so I'd advise against it. Does gcc support static linking? Even if I end up with a much bigger binary, at least I know it will have everything it needs to run and I won't have to deal with DLL hell. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 1 09:15:20 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 01 Oct 2016 23:15:20 +1000 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <57ef32ea$0$1612$c3e8da3$5496439d@news.astraweb.com> <57ef605b$0$1595$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57efb6e9$0$1601$c3e8da3$5496439d@news.astraweb.com> On Sat, 1 Oct 2016 09:33 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: > >> # create a new binding >> x: address 1234 ----> [ box contains 999 ] >> x: address 5678 ----> [ a different box, containing 888 ] > > In the context of CPython and nested functions, replace > "box" with "cell". Cells in Python are an implementation detail. It only applies to CPython (as far as I know), certainly not to IronPython, and the interpreter takes care to ensure that the user-visible behaviour of local variables in cells is identical to the behaviour of name bindings in dicts. The only user-visible semantic differences between variables in a dict and variables in cells are: - writing to locals() won't necessarily affect the actual locals; - in Python 3 only, import * and exec in the local names space may be prohibited. Everything else (that I know of) is indistinguishable. Python takes care to hide the differences, for example even though local variables have a cell pre-allocated when the function is called, trying to access that cell before a value is bound to the local gives a NameError (UnboundLocalError) rather than accessing uninitialised memory, as a naive implementation might have done. In any case, I think this is going off on a rather wide tangent -- how is this specifically relevant to the "unintuitive for-loop behavior" the OP was surprised by? > When I said "creating a new binding" I meant that the > name x refers to different cells at different times. > When I said "updating an existing binding" I meant that > the name x still refers to the same cell, but that cell > refers to a different object. I don't believe that there is any test you can write in Python that will distinguish those two cases. (Excluding introspection of implementation details, such as byte-code, undocumented C-level structures, etc.) As far as ordinary Python operations go, I don't see that there's any difference between the two. When you say: x = 0 x = 1 inside a function, and the interpreter does the name binding twice, there's no way of telling whether it writes to the same cell each time or not. Apart from possible performance and memory use, what difference would it make? It is still the same 'x' name binding, regardless of the implementation details. > In a wider context, replace "box" with "slot in a > stack frame" or "slot in a namespace dictionary". > >> But Python doesn't work that way! Variables aren't modelled by boxes in >> fixed locations, and there is no difference between "create a new >> binding" and "update an existing one". > > There is very much a distintion. Each time you invoke > a function, a new set of bindings is created for all of > its parameters and local names. Assigning to those names > within the function, on the other hand, updates existing > bindings. Certainly when you call a function, the local bindings need to be created. Obviously they didn't exist prior to calling the function! I didn't think that was the difference you were referring to, and I fail to see how it could be relevant to the question of for-loop behaviour. As I understood you, you were referring to assignments to *existing* names. If a binding for x already exists, then and only then do you have a choice between: - update the existing binding for x; - or create a new binding for x. If there is no binding for x yet (such as before the function is called), then you have no choice in the matter: you cannot possibly update what doesn't exist. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jussi.piitulainen at helsinki.fi Sat Oct 1 09:32:46 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sat, 01 Oct 2016 16:32:46 +0300 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: Rustom Mody writes: > And then define comprehensions not as now done in terms of for loops > that mutatingly extend the list being built up but as recursive > functions that get (re)called for every new value of the comprehension > variable passed and therefore fresh-bound as parameter You'd get the magical semantics for comprehensions from the current definition of comprehension semantics in terms of the loop, if the loop semantics was magical. Let me demonstrate. (The b-word is unfortunately not available, so I substitute "magical" instead.) # delayd = [ lambda : c for c in "abracadabra" ] # is roughly/almost equal to the following. delayd = [] for c in "abracadabra": delayd.append(lambda : c) print('Python:', ''.join(f() for f in delayd)) # But that for-loop could have been roughly equal to the following, # giving both the comprehension and the underlying for-loop a # semantics that some people say they would prefer. delayd = [] # reset the list, not part of the loop g1 = iter("abracadabra") try: while True: def g2(c): delayd.append(lambda : c) g2(next(g1)) except StopIteration: pass print('Magick:', ''.join(f() for f in delayd)) # Output from the above: # Python: aaaaaaaaaaa # Magick: abracadabra From nicolasessisbreton at gmail.com Sat Oct 1 09:41:46 2016 From: nicolasessisbreton at gmail.com (nicolasessisbreton at gmail.com) Date: Sat, 1 Oct 2016 06:41:46 -0700 (PDT) Subject: announcing fython Message-ID: <2669c31c-7266-4e14-a521-4758d259a1fd@googlegroups.com> Hi All, I would like to announce a new project call Fython. In short, Fython is Fortran with a Python syntax. Fython permits to write numerical code with the same syntax then Python. Under the hood, the code is transpiled to Fortran and run at top speed. Fython primary goal is to facilitate numerical coding: - Fython code can be easily shared, because a Fython package behaves as a regular Python package. - Fython offers many syntaxtic sugars, such as template and automatic variable completion, increasing programmer productivity. - Fython assists for debugging, and can report the precise line where a segfault happen. Compared to Cython, Fython is dedicated to numerical computing. This dedication produces often clearer and faster code. Compared to Julia, Fython compiles directly to machine instructions. The programmer has total control over the compiled instructions, and this is desirable in some situations. Fython is alpha code, but the code is sufficently stable to be used. I think Fython has the potential to integrate the Python scientific stack. I hope some people will try it and provide feedback on how best to reach this goal. Fython can be find here: fython.readthedocs.io Thanks, Nicolas From auriocus at gmx.de Sat Oct 1 09:44:18 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 1 Oct 2016 15:44:18 +0200 Subject: rocket simulation game with just using tkinter In-Reply-To: <57eeee41$0$911$e4fe514c@news.xs4all.nl> References: <57eeee41$0$911$e4fe514c@news.xs4all.nl> Message-ID: Am 01.10.16 um 00:59 schrieb Irmen de Jong: > Hi, > > I've made a very simple rocket simulation game, inspired by the recent success of SpaceX > You can get the code here if you want to give it a try: > https://github.com/irmen/rocketsimulator Nice! I'll have to rebind the keys before I can successfully play this, though. My [] "keys" are in fact combinations of Alt+5 and Alt+6 on a German Macbook keyboard (on a German PC, it is AltGr+8, AltGr+9). Why not using the arrow keys? These should be pretty universal Christian > So you just need python 2/3 with tkinter to play this! > > > Have fun > Irmen > From grant.b.edwards at gmail.com Sat Oct 1 10:01:26 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 1 Oct 2016 14:01:26 +0000 (UTC) Subject: Copying a compiled Python from one system to another References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-01, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the > latest Python 3.6, and no time to upgrade my usual machines to > something which will work. > > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could compile > 3.6 on this VM, then copy the python binary to my usual desktop machine. You'll probably need to copy more than just the binary. > What sort of challenges am I likely to find? Missing or incompatible libraries. Wrong CPU type. > Both machines are Linux, but different distros. If they were the same distros, you'd have a much better chance. Then, the right way to do it would be to build a binary package (.rpm, .deb, .whatever) on one machine for installation on the other machine using the normal package manager. That way the library situation would get verified (if perhaps not remedied). -- Grant From grant.b.edwards at gmail.com Sat Oct 1 10:02:59 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 1 Oct 2016 14:02:59 +0000 (UTC) Subject: Copying a compiled Python from one system to another References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> <57efb42c$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-01, Steve D'Aprano wrote: > Does gcc support static linking? Yes, but the real question is the CPython makefile includes recipes for a statically-linked target. > Even if I end up with a much bigger binary, at least I know it will > have everything it needs to run and I won't have to deal with DLL > hell. -- Grant From flebber.crue at gmail.com Sat Oct 1 10:09:23 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 07:09:23 -0700 (PDT) Subject: generator no iter - how do I call it from another function In-Reply-To: <57ef9ac3$0$1590$c3e8da3$5496439d@news.astraweb.com> References: <57ef9ac3$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <6b05f07c-dc56-4c24-b924-7a1cc4e76646@googlegroups.com> > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. Loving life. I first started with a simple with open on a file, which allowed me to use code that "iter"'s. for example: for meet in root.iter("meeting"): for race in root.iter("race"): for nom in root.iter("nomination"): meetattr = meet.attrib I then got correct output so wanted to scale up to passing a directory of files. So skipping the code that deals with getting it off the command line I implemented a generator to yield the file root object as needed. This is that code def return_files(file_list): """ Take a list of files and return file when called. Calling function to supply attributes """ for filename in sorted(file_list): with open(dir_path + filename) as fd: tree = etree.parse(fd) root = tree.getroot() yield root My question is though now that I have implemented it this way my I pull in the root via a function first few lines are def dataAttr(roots): """Get the root object and iter items.""" with open("output/first2.csv", 'w', newline='') as csvf: race_writer = csv.writer(csvf, delimiter=',') for meet in roots.iter("meeting"): which I call as rootObs = return_files(file_list) dataAttr(rootObs) So if I use a generator to pass in the root lxml object to a function how do I iter since python provides an error that iters don't exist on python objects? This is said error ? |master U:1 ?:1 ?| ? python3 race.py data/ -e *.xml Traceback (most recent call last): File "race.py", line 77, in dataAttr(rootObs) File "race.py", line 55, in dataAttr for meet in roots.iter("meeting"): AttributeError: 'generator' object has no attribute 'iter' Cheers Sayth From torriem at gmail.com Sat Oct 1 10:21:11 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 1 Oct 2016 08:21:11 -0600 Subject: Copying a compiled Python from one system to another In-Reply-To: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <610fa2ce-2e33-3792-5889-c632060a015f@gmail.com> On 10/01/2016 05:21 AM, Steve D'Aprano wrote: > Long story short: I have no working systems capable of compiling the > latest Python 3.6, and no time to upgrade my usual machines to > something which will work. > > However I do have access to another machine (actually a VM) which > can compile Python 3.6. It's not practical for me to use it as a my > main development machine, but as a temporary measure, I thought I > could compile 3.6 on this VM, then copy the python binary to my usual > desktop machine. > > What sort of challenges am I likely to find? Both machines are Linux, > but different distros. If they use the same major version of glibc, both are either 64-bit or 32-bit, then if you compile to a self-contained location like /opt and then copy that folder over it will run just fine. If you compile to /usr, you'll have to make sure you tar up all the bits that are scattered around the various directories under /usr. What distros are we talking about here? Ideally it would be nice to build install-able packages for the target OS. From jussi.piitulainen at helsinki.fi Sat Oct 1 10:28:11 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sat, 01 Oct 2016 17:28:11 +0300 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: I'm not sure any more to what message this should be a followup, but here is a demonstration of two different semantics of the for-loop variable scope/update, this time with nested loops using the same loop variable name. The first function, tabulate, uses Python semantics ("t" for true, if you like); the second, fabulate, is a translation ("f" for false, if you like) that uses the magical semantics where the loop variable is not only local to the loop but also a different variable on each iteration. The latter property makes no difference in this demonstration, but the former does; there's also a spurious counter that is not local to the nested loops, just to be sure that it works as expected (it does). A summary of sorts: it's possible to demonstrate the scope difference in Python code, with no box in sight; boxes are irrelevant; the relevant issue is what function and when the loop variable is associated with, explicitly or implicitly. def tabulate(m, n): for i in range(m): print(i, end = ': ') c = 0 for i in range(n): print(i, end = ', ' if i + 1 < n else ' : ') c += 1 print(i, c) def fabulate(m, n): c = None # because c belong in this scope g1 = iter(range(m)) try: while True: def g2(i): nonlocal c print(i, end = ': ') c = 0 g3 = iter(range(n)) try: while True: def g4(i): nonlocal c print(i, end = ', ' if i + 1 < n else ' : ') c += 1 g4(next(g3)) except StopIteration: pass print(i, c) g2(next(g1)) except StopIteration: pass print('Python:') tabulate(3, 4) print() print('Magick:') fabulate(3, 4) # Output from the above, each line being # outer i: each inner i : i after inner loop, c # where either c correctly counts inner steps but # Python inner i clobbers outer i, Magick not. # # Python: # 0: 0, 1, 2, 3 : 3 4 # 1: 0, 1, 2, 3 : 3 4 # 2: 0, 1, 2, 3 : 3 4 # # Magick: # 0: 0, 1, 2, 3 : 0 4 # 1: 0, 1, 2, 3 : 1 4 # 2: 0, 1, 2, 3 : 2 4 From zachary.ware+pylist at gmail.com Sat Oct 1 10:30:41 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Sat, 1 Oct 2016 09:30:41 -0500 Subject: Copying a compiled Python from one system to another In-Reply-To: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Oct 1, 2016 06:25, "Steve D'Aprano" wrote: > > Long story short: I have no working systems capable of compiling the latest > Python 3.6, and no time to upgrade my usual machines to something which > will work. Since you're working on a pure-Python module (statistics), I'd recommend updating to the latest changeset that will build, and work from there. I don't know of any changes that would make it impossible for you to update to the last working changeset, build, update to 3.6 tip, and develop and test your statistics changes without rebuilding. You may have some test failures, just run the full test suite after update to determine which failures are not due to your development. Otherwise, I'd suggest compiling your own GCC, but I haven't tried that myself without the aid of Gentoo's package manager. -- Zach (On a phone) From thorsten at thorstenkampe.de Sat Oct 1 10:56:03 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Sat, 1 Oct 2016 16:56:03 +0200 Subject: ConfigParser: use newline in INI file Message-ID: Hi, ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to signal to ConfigParser that there is a line break? Thorsten From rustompmody at gmail.com Sat Oct 1 12:57:03 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 1 Oct 2016 09:57:03 -0700 (PDT) Subject: unintuitive for-loop behavior In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Saturday, October 1, 2016 at 7:02:58 PM UTC+5:30, Jussi Piitulainen wrote: > Rustom Mody writes: > > > And then define comprehensions not as now done in terms of for loops > > that mutatingly extend the list being built up but as recursive > > functions that get (re)called for every new value of the comprehension > > variable passed and therefore fresh-bound as parameter > > You'd get the magical semantics for comprehensions from the current > definition of comprehension semantics in terms of the loop, if the loop > semantics was magical. Let me demonstrate. (The b-word is unfortunately > not available, so I substitute "magical" instead.) > > # delayd = [ lambda : c for c in "abracadabra" ] > # is roughly/almost equal to the following. > > delayd = [] > for c in "abracadabra": > delayd.append(lambda : c) > > print('Python:', ''.join(f() for f in delayd)) > > # But that for-loop could have been roughly equal to the following, > # giving both the comprehension and the underlying for-loop a > # semantics that some people say they would prefer. > > delayd = [] # reset the list, not part of the loop > > g1 = iter("abracadabra") > try: > while True: > def g2(c): > delayd.append(lambda : c) > g2(next(g1)) > except StopIteration: > pass > > print('Magick:', ''.join(f() for f in delayd)) > > # Output from the above: > # Python: aaaaaaaaaaa > # Magick: abracadabra Hoo boy1 Thats some tour de force and makes my head spin Point can be made more simply with map ie if we *define* [exp for cv in l] as map(lambda cv: exp, l) the problem vanishes Demo: First a helper function for demoing: def pam(fl,x): return map(lambda f: f(x), fl) # pam is the complement to map; map runs one fnc on a list of args # pam runs a list of funcs on one arg Trying to make a list of functions that add one, two and three to their arguments fl = [lambda x: x + cv for cv in [1,2,3]] Broken because of python's wrong LC semantics: >>> pam(fl, 3) [6, 6, 6] Transform the LC into a map with the rule above: fl_good = map((lambda cv :lambda x: x+cv), [1,2,3]) Works! >>> pam(fl_good, 3) [4, 5, 6] >>> Which is not very far from the standard workaround for this gotcha: >>> fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]] >>> pam(fl_workaround, 3) [4, 5, 6] >>> Maybe we could say the workaround is the map definition uncurried And then re-comprehension-ified From saxri89 at gmail.com Sat Oct 1 13:50:14 2016 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 1 Oct 2016 10:50:14 -0700 (PDT) Subject: RASTER analysis(slope) Message-ID: <76b02ab7-2399-4360-9b68-654a63b48a1b@googlegroups.com> hello team i want to calculate slope and aspect from some RASTER IMAGE(.grid,tiff,geotiff) who is the better method to can i do this ? with numpy and scipy or with some package? thnx you team From steve+python at pearwood.info Sat Oct 1 14:04:36 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 02 Oct 2016 05:04:36 +1100 Subject: unintuitive for-loop behavior References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57effab6$0$1610$c3e8da3$5496439d@news.astraweb.com> On Sun, 2 Oct 2016 03:57 am, Rustom Mody wrote: > Hoo boy1 > Thats some tour de force and makes my head spin I certainly agree with the second part of your sentence. > Point can be made more simply with map > ie if we *define* > [exp for cv in l] > as > map(lambda cv: exp, l) > > the problem vanishes > > Demo: > > First a helper function for demoing: > > def pam(fl,x): > return map(lambda f: f(x), fl) > # pam is the complement to map; map runs one fnc on a list of args > # pam runs a list of funcs on one arg > > Trying to make a list of functions that add one, two and three to their > arguments > > fl = [lambda x: x + cv for cv in [1,2,3]] > > Broken because of python's wrong LC semantics: >>>> pam(fl, 3) > [6, 6, 6] Its not *broken*, its doing *exactly what you told it to do*. You said, define a function that takes a single argument x, and return x + cv. Then you delayed evaluating it until cv = 3, and passed the argument 3, so of course it returns 6. That's exactly what you told it to calculate. You seem to have the concept that lambda should be magical, and just miraculously know how far back in time to look for the value of cv. And then when it doesn't, you're angry that Python is "broken". But why should it be magical? cv is just an ordinary variable, and like all variables, looking it up returns the value it has at the time you do the look-up, not some time in the past. Let's unroll the loop: fl = [] cv = 1 def f(x): return x + cv fl.append(f) cv = 2 def f(x): return x + cv fl.append(f) cv = 3 def f(x): return x + cv fl.append(f) pam(fl, 3) Are you still surprised that it returns [6, 6, 6]? > Transform the LC into a map with the rule above: > fl_good = map((lambda cv :lambda x: x+cv), [1,2,3]) This is equivalent to something completely different, using a closure over cv, so of course it works: def factory(cv): def inner(x): return x + cv return inner fl_good = [] fl_good.append(factory(1)) fl_good.append(factory(2)) fl_good.append(factory(3)) Each time you call factory(), you get a new scope, with its own independent variable cv. The inner function captures that environment (a closure), which includes that local variable cv. Each invocation of factory leads to an inner function that sees a different local variable which is independent of the others but happens to have the same name. Instead of three functions all looking up a single cv variable, you have three functions looking up three different cv variables. This is essentially why closures exist. > Which is not very far from the standard workaround for this gotcha: >>>> fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]] >>>> pam(fl_workaround, 3) > [4, 5, 6] >>>> > > Maybe we could say the workaround is the map definition uncurried > And then re-comprehension-ified If your students think in terms of map, then fine, but I think it would confuse more people than it would help. Your mileage may vary. There are certainly a number of ways to get the desired behaviour. If you prefer to work with map, go right ahead. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Sat Oct 1 15:44:39 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 1 Oct 2016 15:44:39 -0400 Subject: ConfigParser: use newline in INI file In-Reply-To: References: Message-ID: On 10/1/2016 10:56 AM, Thorsten Kampe wrote: > ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to > signal to ConfigParser that there is a line break? Without an example or two, I don't really understand the question enough to answer. -- Terry Jan Reedy From aivar.annamaa at ut.ee Sat Oct 1 15:48:08 2016 From: aivar.annamaa at ut.ee (Aivar Annamaa) Date: Sat, 01 Oct 2016 22:48:08 +0300 Subject: Thonny 2.0 released (Python IDE for beginners) Message-ID: Hi! Thonny is Python IDE for learning and teaching programming. It is developed in University of Tartu, Estonia. It has an easy to use debugger which shows clearly how Python executes your programs. Unlike most debuggers, it can even show the steps of evaluating an expression, visually explain references, function calls, exceptions etc. For more info and downloads see http://thonny.cs.ut.ee/ [1] best regards, Aivar Annamaa University of Tartu Institute of Computer Science -------------------------------- Thonny 2.0 - Python IDE for beginners (01-Oct-16) Links: ------ [1] http://thonny.cs.ut.ee/ From irmen.NOSPAM at xs4all.nl Sat Oct 1 16:07:13 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Sat, 1 Oct 2016 22:07:13 +0200 Subject: rocket simulation game with just using tkinter In-Reply-To: References: <57eeee41$0$911$e4fe514c@news.xs4all.nl> Message-ID: <57f01771$0$863$e4fe514c@news.xs4all.nl> On 1-10-2016 15:44, Christian Gollwitzer wrote: > Am 01.10.16 um 00:59 schrieb Irmen de Jong: >> Hi, >> >> I've made a very simple rocket simulation game, inspired by the recent success of SpaceX > >> You can get the code here if you want to give it a try: >> https://github.com/irmen/rocketsimulator > > Nice! I'll have to rebind the keys before I can successfully play this, though. My [] > "keys" are in fact combinations of Alt+5 and Alt+6 on a German Macbook keyboard (on a > German PC, it is AltGr+8, AltGr+9). Why not using the arrow keys? These should be pretty > universal Oh, I'm sorry about that, my little knowledge of non-US keyboard layouts shows. Arrow keys could be an option. For my education what are the 2 keys to the right of the P then on your keyboard? Rebinding the keys should be easy though just change them in the keypress and keyrelease methods. Irmen From ben+python at benfinney.id.au Sat Oct 1 16:12:46 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 02 Oct 2016 07:12:46 +1100 Subject: ConfigParser: use newline in INI file References: Message-ID: <85lgy7kev5.fsf@benfinney.id.au> Thorsten Kampe writes: > ConfigParser escapes `\n` in ini values as `\\n`. How do you demonstrate that? Here is an example text of a config file:: >>> import io >>> import textwrap >>> config_text = textwrap.dedent(r""" ... [foo] ... wibble = Lorem\nipsum. ... """) >>> print(config_text) [foo] wibble = Lorem\nipsum. So that text has the characters ?\n? in it. (Note that I had to use the ?r? prefix on the string literal, in order to turn off the special meaning of ?\? and get that character literally.) When I use a ConfigParser it reads ?\n? and reproduces exactly those characters:: >>> import configparser >>> config_file = io.StringIO(config_text) >>> config = configparser.ConfigParser() >>> config.read_file(config_file) >>> print(config['foo']['wibble']) Lorem\nipsum. So you see that the ?\n? characters are preserved exactly by the ConfigParser. > Is there a way to signal to ConfigParser that there is a line break? Yes, you use a line break. See the ?configparser? module documentation: Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser?s mode, blank lines may be treated as parts of multiline values or ignored. Thus:: >>> config_text = textwrap.dedent(r""" ... [foo] ... wibble = Lorem ... ipsum. ... """) >>> print(config_text) [foo] wibble = Lorem ipsum. >>> config_file = io.StringIO(config_text) >>> config = configparser.ConfigParser() >>> config.read_file(config_file) >>> print(config['foo']['wibble']) Lorem ipsum. -- \ ?Only the shallow know themselves.? ?Oscar Wilde, _Phrases and | `\ Philosophies for the Use of the Young_, 1894 | _o__) | Ben Finney From tjreedy at udel.edu Sat Oct 1 16:19:56 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 1 Oct 2016 16:19:56 -0400 Subject: unintuitive for-loop behavior In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/1/2016 8:24 AM, Rustom Mody wrote: > Yeah by comprehension-variable I mean the one that sits left of the > ?in? inside the conprehension. In other words, a 'loop variable within a comprehension'. Keep in mind that there may be multiple targets for the implicit (hidden) assignment, so there may be multiple loop (comprehension) variables even without nested loops. -- Terry Jan Reedy From thorsten at thorstenkampe.de Sat Oct 1 17:58:17 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Sat, 1 Oct 2016 23:58:17 +0200 Subject: ConfigParser: use newline in INI file References: Message-ID: * Terry Reedy (Sat, 1 Oct 2016 15:44:39 -0400) > > On 10/1/2016 10:56 AM, Thorsten Kampe wrote: > > > ConfigParser escapes `\n` in ini values as `\\n`. Is there a way to > > signal to ConfigParser that there is a line break? > > Without an example or two, I don't really understand the question enough > to answer. >>> !cat INI.ini [Asciidoc] _test_stdout =
\n

Hello, World!

\n
>>> import configparser >>> config = configparser.ConfigParser() >>> config.read('INI.ini') ['INI.ini'] >>> config['Asciidoc']['_test_stdout'] '
\\n

Hello, World!

\\n
' ...as you can see, ConfigParser escaped the backslash by doubling it. Which is fine in most cases - except when I want to have something indicating an newline. Thorsten From rosuav at gmail.com Sat Oct 1 18:08:48 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 2 Oct 2016 09:08:48 +1100 Subject: announcing fython In-Reply-To: <2669c31c-7266-4e14-a521-4758d259a1fd@googlegroups.com> References: <2669c31c-7266-4e14-a521-4758d259a1fd@googlegroups.com> Message-ID: On Sat, Oct 1, 2016 at 11:41 PM, wrote: > Fython permits to write numerical code with the same syntax then Python. > Under the hood, the code is transpiled to Fortran and run at top speed. How does this compare to Python+Numpy? How much faster is Fython, and what are the restrictions on the Python code? ChrisA From rosuav at gmail.com Sat Oct 1 18:13:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 2 Oct 2016 09:13:23 +1100 Subject: Copying a compiled Python from one system to another In-Reply-To: <57efb42c$0$1614$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> <57efb42c$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 1, 2016 at 11:03 PM, Steve D'Aprano wrote: >> Are both Linuxes of broadly similar vintage? > > That depends on what you mean by "broadly similar". As far as I am > concerned, a five year difference is not very much, and is broadly > similar -- it's not like I'm using Linux from 1991. But the whole point is > that I need something with gcc 4.8 (maybe 4.7 will do, not sure) but > certainly not 4.4 which is the most recent I can get on my current systems. It's a rubbery term, but basically take the year of release of the distro versions you're using, push 'em forward if they're bleeding-edge distros or back if they're uber-stable, and try to guess the age of the libraries they're running. Or check by looking at some nice over-arching version number like Linux kernel; if they use similar versions of Linux and libc, they probably have roughly similar versions of a lot of other libraries, too. The more similar the library versions, the less issues you'll have. ChrisA From thorsten at thorstenkampe.de Sat Oct 1 18:24:20 2016 From: thorsten at thorstenkampe.de (Thorsten Kampe) Date: Sun, 2 Oct 2016 00:24:20 +0200 Subject: ConfigParser: use newline in INI file References: <85lgy7kev5.fsf@benfinney.id.au> Message-ID: * Ben Finney (Sun, 02 Oct 2016 07:12:46 +1100) > > Thorsten Kampe writes: > > > ConfigParser escapes `\n` in ini values as `\\n`. Indenting solves the problem. I'd rather keep it one line per value but it solves the problem. Thorsten From flebber.crue at gmail.com Sat Oct 1 18:29:51 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 15:29:51 -0700 (PDT) Subject: RASTER analysis(slope) In-Reply-To: <76b02ab7-2399-4360-9b68-654a63b48a1b@googlegroups.com> References: <76b02ab7-2399-4360-9b68-654a63b48a1b@googlegroups.com> Message-ID: On Sunday, 2 October 2016 04:52:13 UTC+11, Xristos Xristoou wrote: > hello team > > i want to calculate slope and aspect from some RASTER IMAGE(.grid,tiff,geotiff) > who is the better method to can i do this ? > with numpy and scipy or with some package? > > > thnx you team I don't know much about the topic however pyDem seems like the appropriate library to use. https://pypi.python.org/pypi/pyDEM/0.1.1 Here is a standford article on it https://pangea.stanford.edu/~samuelj/musings/dems-in-python-pt-3-slope-and-hillshades-.html and a pdf from scipy conference. https://conference.scipy.org/proceedings/scipy2015/pdfs/mattheus_ueckermann.pdf Sayth From no.email at nospam.invalid Sat Oct 1 18:31:32 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 01 Oct 2016 15:31:32 -0700 Subject: Copying a compiled Python from one system to another References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> Message-ID: <878tu71z23.fsf@nightsong.com> Steve D'Aprano writes: > However I do have access to another machine (actually a VM) which can > compile Python 3.6. It's not practical for me to use it as a my main > development machine, but as a temporary measure, I thought I could > compile 3.6 on this VM, then copy the python binary to my usual > desktop machine. How about installing the same OS on the VM that you're running on the development machine? You can get powerful hourly VM's (x86) and dedicated servers (x86 and ARM) really cheap at Scaleway.com. I've used them for a while and they work well. From cfkaran2 at gmail.com Sat Oct 1 18:56:56 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 1 Oct 2016 18:56:56 -0400 Subject: Byte code descriptions somewhere? Message-ID: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> Hi all, I've all of a sudden gotten interested in the CPython interpreter, and started trying to understand how it ingests and runs byte code. I found Include/opcode.h in the python sources, and I found some basic documentation on how to add in new opcodes online, but I haven't found the equivalent of an assembly manual like you might for x86, etc. Is there something similar to a manual dedicated to python byte code? Also, is there a manual for how the interpreter expects the stack, etc. to be setup so that all interactions go as expected (garbage collections works, exceptions work, etc.)? Basically, I want a manual similar to what Intel or AMD might put out for their chips so that all executables behave nicely with one another. Thanks, Cem Karan From ben+python at benfinney.id.au Sat Oct 1 19:02:18 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 02 Oct 2016 10:02:18 +1100 Subject: Byte code descriptions somewhere? References: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> Message-ID: <85eg3zk70l.fsf@benfinney.id.au> Cem Karan writes: > Hi all, I've all of a sudden gotten interested in the CPython > interpreter, and started trying to understand how it ingests and runs > byte code. That sounds like fun! > Is there something similar to a manual dedicated to python byte code? The Python documentation for the ?dis? module shows not only how to use that module for dis-assembly of Python byte code, but also a reference for the byte code. 32.12. dis ? Disassembler for Python bytecode -- \ ?Skepticism is the highest duty and blind faith the one | `\ unpardonable sin.? ?Thomas Henry Huxley, _Essays on | _o__) Controversial Questions_, 1889 | Ben Finney From breamoreboy at gmail.com Sat Oct 1 19:34:30 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 1 Oct 2016 16:34:30 -0700 (PDT) Subject: Byte code descriptions somewhere? In-Reply-To: References: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> Message-ID: <53c4ba26-636d-4eb8-96a4-4d53c2a8008f@googlegroups.com> On Saturday, October 1, 2016 at 11:57:17 PM UTC+1, Cem Karan wrote: > Hi all, I've all of a sudden gotten interested in the CPython interpreter, and started trying to understand how it ingests and runs byte code. I found Include/opcode.h in the python sources, and I found some basic documentation on how to add in new opcodes online, but I haven't found the equivalent of an assembly manual like you might for x86, etc. Is there something similar to a manual dedicated to python byte code? Also, is there a manual for how the interpreter expects the stack, etc. to be setup so that all interactions go as expected (garbage collections works, exceptions work, etc.)? Basically, I want a manual similar to what Intel or AMD might put out for their chips so that all executables behave nicely with one another. > > Thanks, > Cem Karan Further to Ben Finney's answer this https://docs.python.org/devguide/compiler.html should help. Kindest regards. Mark Lawrence. From cfkaran2 at gmail.com Sat Oct 1 19:47:45 2016 From: cfkaran2 at gmail.com (Cem Karan) Date: Sat, 1 Oct 2016 19:47:45 -0400 Subject: Byte code descriptions somewhere? In-Reply-To: <85eg3zk70l.fsf@benfinney.id.au> References: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> <85eg3zk70l.fsf@benfinney.id.au> Message-ID: <017D226F-37E1-4706-8761-071560E211CE@gmail.com> Cool, thank you! Quick experimentation suggests that I don't need to worry about marking anything for garbage collection, correct? The next question is, how do I create a stream of byte codes that can be interpreted by CPython directly? I don't mean 'use the compile module', I mean writing my own byte array with bytes that CPython can directly interpret. Thanks, Cem Karan On Oct 1, 2016, at 7:02 PM, Ben Finney wrote: > Cem Karan writes: > >> Hi all, I've all of a sudden gotten interested in the CPython >> interpreter, and started trying to understand how it ingests and runs >> byte code. > > That sounds like fun! > >> Is there something similar to a manual dedicated to python byte code? > > The Python documentation for the ?dis? module shows not only how to use > that module for dis-assembly of Python byte code, but also a reference > for the byte code. > > 32.12. dis ? Disassembler for Python bytecode > > > > -- > \ ?Skepticism is the highest duty and blind faith the one | > `\ unpardonable sin.? ?Thomas Henry Huxley, _Essays on | > _o__) Controversial Questions_, 1889 | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list From no.email at nospam.invalid Sat Oct 1 19:53:39 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 01 Oct 2016 16:53:39 -0700 Subject: Byte code descriptions somewhere? References: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> <85eg3zk70l.fsf@benfinney.id.au> <017D226F-37E1-4706-8761-071560E211CE@gmail.com> Message-ID: <874m4v1v98.fsf@nightsong.com> Cem Karan writes: > how do I create a stream of byte codes that can be interpreted by > CPython directly? Basically, study the already existing code and do something similar. The CPython bytecode isn't standardized like JVM bytecode. It's designed for the interpreter's convenience, not officially documented, and (somewhat) subject to change between versions. From rosuav at gmail.com Sat Oct 1 19:56:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 2 Oct 2016 10:56:51 +1100 Subject: Byte code descriptions somewhere? In-Reply-To: <017D226F-37E1-4706-8761-071560E211CE@gmail.com> References: <6EBB3CEE-DA1F-4AB8-8ADF-42C25B0C1AE5@gmail.com> <85eg3zk70l.fsf@benfinney.id.au> <017D226F-37E1-4706-8761-071560E211CE@gmail.com> Message-ID: On Sun, Oct 2, 2016 at 10:47 AM, Cem Karan wrote: > Cool, thank you! Quick experimentation suggests that I don't need to worry about marking anything for garbage collection, correct? The next question is, how do I create a stream of byte codes that can be interpreted by CPython directly? I don't mean 'use the compile module', I mean writing my own byte array with bytes that CPython can directly interpret. > "Marking for garbage collection" in CPython is done by refcounts; the bytecode is at a higher level than that. >>> dis.dis("x = y*2") 1 0 LOAD_NAME 0 (y) 3 LOAD_CONST 0 (2) 6 BINARY_MULTIPLY 7 STORE_NAME 1 (x) 10 LOAD_CONST 1 (None) 13 RETURN_VALUE A LOAD operation will increase the refcount (a ref is on the stack), BINARY_MULTIPLY dereferences the multiplicands and adds a ref to the product, STORE will deref whatever previously was stored, etc. To execute your own code, look at types.FunctionType and types.CodeType, particularly the latter's 'codestring' argument (stored as the co_code attribute). Be careful: you can easily crash CPython if you mess this stuff up :) ChrisA From skip.montanaro at gmail.com Sat Oct 1 20:14:11 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sat, 1 Oct 2016 19:14:11 -0500 Subject: Converting PyMember_Get usage to PyMember_GetOne - no docs? Message-ID: I'm trying to convert the python-sybase module C code from oldish Python 2 usage to work in Python 3. The code currently calls PyMember_Get(), which is obsolete. In later versions of Python 2, PyMember_GetOne became the official way to do things, the the former function was still available. In Python 3, PyMember_Get() is gone. This is all well and good, but I can't find documentation for either function, or seemingly examples of code which converted from one to the other. Any pointers? Thx, Skip From raoul.fleckman at gmail.com Sat Oct 1 20:18:04 2016 From: raoul.fleckman at gmail.com (Raoul Fleckman) Date: Sun, 2 Oct 2016 00:18:04 +0000 (UTC) Subject: announcing fython References: <2669c31c-7266-4e14-a521-4758d259a1fd@googlegroups.com> Message-ID: <9om98u.17o.19.1@isomedia.com> On 2016-10-01, Chris Angelico wrote: > On Sat, Oct 1, 2016 at 11:41 PM, wrote: >> Fython permits to write numerical code with the same syntax then Python. >> Under the hood, the code is transpiled to Fortran and run at top speed. > > How does this compare to Python+Numpy? How much faster is Fython, and > what are the restrictions on the Python code? > > ChrisA Interested to hear the answers to those questions, and whether Fython is pass by reference (Fortran) or value (python, unless passing a list, for example); and then there's the 'little' matter of one-based (Fortran) or zero-based (python) arrays? From flebber.crue at gmail.com Sat Oct 1 20:21:24 2016 From: flebber.crue at gmail.com (Sayth Renshaw) Date: Sat, 1 Oct 2016 17:21:24 -0700 (PDT) Subject: inplace text filter - without writing file Message-ID: Hi I have a fileobject which was fine however now I want to delete a line from the file object before yielding. def return_files(file_list): for filename in sorted(file_list): with open(dir_path + filename) as fd: for fileItem in fd: yield fileItem Ned gave an answer over here http://stackoverflow.com/a/6985814/461887 for i, line in enumerate(input_file): if i == 0 or not line.startswith('#'): output.write(line) which I would change because it is the first line and I want to rid 385 para=ParaParser('mutation.in') 386 simu_mutation(para) /home/JPJ/Priya_Ph.D/simple_bool/simplebool/SimpleBool-master/BoolMutation.py in ParaParser(ParaFile) 254 } # define parameters 255 --> 256 for each_line in open(ParaFile).readlines(): 257 para_name = each_line.split('=')[0].strip() 258 para_value = each_line.split('=')[1].strip() IOError: [Errno 2] No such file or directory: 'mutation.in' I can't understand, the problem here. Should I specify anything in the Boolmutation.py file. Any help would be appreciated Thank you Regards Priya From joel.goldstick at gmail.com Wed Oct 5 11:32:58 2016 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 5 Oct 2016 11:32:58 -0400 Subject: Python -Simplebool In-Reply-To: <50d6b0a3-9794-4e0f-9d2d-1b096e67949e@googlegroups.com> References: <50d6b0a3-9794-4e0f-9d2d-1b096e67949e@googlegroups.com> Message-ID: Welcome, Please, format your code. That's a lot of code, and it doesn't run as presented. Use text mode in your email program (or whatever you use to post here), and make sure the indentation is correct. On Wed, Oct 5, 2016 at 11:01 AM, wrote: > Hi, > I am new to Python and I am learning it. Need some help. I downloaded SimpleBool (https://github.com/lujunyan1118/SimpleBool) for simulating Boolean models. SimpleBool works in Python. I downloaded the Canopy and executed Boolmutation.py file. Below is the Boolmutation.py file > > > #!/bin/env python > ''' > SimpleBool is a python package for dynamic simulations of boolean network > Copyright (C) 2013 Junyan Lu > > This program is free software: you can redistribute it and/or modify > it under the terms of the GNU General Public License as published by > the Free Software Foundation, either version 3 of the License, or > (at your option) any later version. > > This program is distributed in the hope that it will be useful, > but WITHOUT ANY WARRANTY; without even the implied warranty of > MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > GNU General Public License for more details. > > You should have received a copy of the GNU General Public License > along with this program. If not, see . > ''' > from __future__ import division > import random > import sys > > class Model: > def __init__(self,INPUT,mut_nodes=[]): > random.seed() > self.KEEP={} > self.INITIAL={} > self.REG_NODES=[] > self.TRUTH_TAB=[] > self.MAPPING={} > self.INPUT={} > self.FINAL={} > def GetNodes(expression): > '''convert one line of expression to a node list''' > nodes = [] > other = ['=', 'and', 'or', 'not'] # remove operator signs > for node in expression.split(): > node=node.strip('*() ') > if node not in other: > nodes.append(node) # remove * ( ) from the node name > return nodes > def IterState(expression, keep): > '''Iterate all the state of input node and output all the inital state and the value of the target node, > used to construct truth table. > Return a list of tuples, the first tuple contain the index of target node and its regulators, > the rest of the tuple contain all the possible state of the target node and its regulators, > the first element in the tuple is the state of target''' > nodes = GetNodes(expression) > record = [] # to store results > all_regulator = nodes[1:] # all regulator of the target > target_node = nodes[0] > record.append(tuple([target_node] + all_regulator)) # record the target node and free regulator > bool_func = expression.split('=')[1].strip() > total_ini = 2 ** len(all_regulator) # n nodes have 2**n combinations > for node in set(all_regulator) & set(keep.keys()): > vars()[node] = keep[node] # set the value of keeped nodes > for index in xrange(total_ini): > state = bin(index)[2:].zfill(len(all_regulator)) # conver a interger to a boolean string with specified length > for i in range(len(all_regulator)): > vars()[all_regulator[i]] = int(state[i]) # set the node variable to logical state, if it is not keeped > if target_node not in keep: > target_val = int(eval(bool_func)) # caculate the target node's value, kept nodes are considered implicitly > else: # if the node value has been keeped by hand, than used that value iregulate of the state of its regulators > target_val = int(keep[target_node]) > record.append(tuple([target_val] + [int(n) for n in state])) > return record > > def ConstructTruthTab(booltext, keep): > '''Construct the truth table that contain all the possibile input state and output state for each node''' > all_result = [] > all_nodes = set([]) # all nodes in boolean rule file > target_nodes = set([]) # nodes have regulators in boolean rule file > RegNodes = [] # a list contain regulator of each node as a tuple. The tuple index in the list is the target node index > TruthTab = [] # a list of dictionary contain the truth table for each node. The sequence is in consist with node sequence in mapping > for line in booltext.split('\n'): > if line != '' and line[0] != '#': > line_nodes = GetNodes(line) > target_nodes = target_nodes | set([line_nodes[0]]) > all_nodes = all_nodes | set(line_nodes) > if line_nodes[0] not in keep.keys(): > try: > all_result.append(IterState(line, keep)) > except: > print "Expressing error of boolean function" > print line > else: #if the node has been kept > all_result.append([(line_nodes[0], line_nodes[0]), (1, 1), (0, 0)]) > unmapped = all_nodes - target_nodes # find the node that do not have regulator, and not specified in the keep list > for unmapped_id in unmapped: > all_result.append([(unmapped_id, unmapped_id), (1, 1), (0, 0)]) # if the node do not have any regulate node, then it regulate by itself > sorted_all = sorted(all_result, key=lambda x:x[0][0]) > mappings = dict(zip([node[0][0] for node in sorted_all], range(len(sorted_all)))) > # generate list of regulators for each node and the truth table, sorted as the mappings > for each_node in sorted_all: > state_dic = {} > regulators = tuple([mappings[node] for node in each_node[0][1:]]) > RegNodes.append(regulators) > for each_state in each_node[1:]: > state_dic[each_state[1:]] = each_state[0] > TruthTab.append(state_dic) > return RegNodes, TruthTab, mappings # > self.INPUT=INPUT > for on_nodes in INPUT['ini_on']: > self.INITIAL[on_nodes] = True > for off_nodes in INPUT['ini_off']: > self.INITIAL[off_nodes] = False > > for on_nodes in INPUT['turn_on']: > self.INITIAL[on_nodes] = True > self.KEEP[on_nodes] = True > for off_nodes in INPUT['turn_off']: > self.KEEP[off_nodes] = False > self.INITIAL[off_nodes] = False > for node,state in mut_nodes: > self.KEEP[node]=state > self.INITIAL[node]=state > self.REG_NODES,self.TRUTH_TAB,self.MAPPING=ConstructTruthTab(open(INPUT['rules']).read(),self.KEEP) > model_verbose={'Sync':'Synchronous','GA':'General Asynchrounous','ROA':'Random Order Asynchrounous'} > print '''Model initialization completed! > Total nodes number: %s > Simulation steps: %s > Simulation rounds: %s > Simulation mode: %s > '''%(len(self.MAPPING.keys()),INPUT['steps'],INPUT['rounds'],model_verbose[INPUT['mode']]) > > def GetNodes(self): > return sorted(self.MAPPING) > def GetFixed(self,file_out='steady.txt'): > all_nodes=self.GetNodes() > on_nodes=[] > off_nodes=[] > output=file(file_out,'w') > for node in all_nodes: > output.writelines('%s\t%s\n'%(node,self.FINAL[node])) > if self.FINAL[node] == 0: > off_nodes.append(node) > elif self.FINAL[node] == 1: > on_nodes.append(node) > print '''%s nodes stabilized on 'ON' state: %s '''%(len(on_nodes),','.join(on_nodes)) > print '''%s nodes stabilized on 'OFF' state: %s '''%(len(off_nodes),','.join(off_nodes)) > output.close() > > > def IterModel(self,missing='random',InitialList=[]): > final_all=[] > traj_all=[] > steps=self.INPUT['steps'] > rounds=self.INPUT['rounds'] > missing=self.INPUT['missing'] > collect=[[0]*len(self.MAPPING)]*(steps+1) > def IterOneSync(InitialState): > '''Iterate model using sychronous method. The most time consuming part, need to be carefully optimized''' > NewState = [str(self.TRUTH_TAB[i][tuple([int(InitialState[j]) for j in self.REG_NODES[i]])]) for i in range(len(InitialState))] > return ''.join(NewState) > def IterOneAsync(InitialState): > '''Iterate model using asynchronous method (General Asynchronous model: update one random node per step)''' > update_index=random.randint(0,len(InitialState)-1) > NewState=list(InitialState) > NewState[update_index] = str(self.TRUTH_TAB[update_index][tuple([int(InitialState[index]) for index in self.REG_NODES[update_index]])]) > return ''.join(NewState) > def IterOneROA(InitialState): > seq=range(len(InitialState)) > random.shuffle(seq) # generate a random sequence of updating list > NewState=list(InitialState) > for i in seq: > NewState[i]= str(self.TRUTH_TAB[i][tuple([int(NewState[index]) for index in self.REG_NODES[i]])]) > #NewState = [str(self.TRUTH_TAB[i][tuple([int(NewState[j]) for j in self.REG_NODES[i]])]) for i in seq] > return ''.join(NewState) > def GenInitial(): > initial_state=[] > for node in sorted(self.MAPPING.keys()): > if node in self.INITIAL: > initial_state.append(str(int(self.INITIAL[node]))) > else: > if missing=='random': > initial_state.append(random.choice(['0','1'])) > else: > initial_state.append(str(int(missing))) > return ''.join(initial_state) > def StringAdd(xlist,ystring): > return [x+int(y) for x,y in zip(xlist,ystring)] > def divide(x): > return x/rounds > for r in range(rounds): > traj=[] > if InitialList==[]: > ini_state=GenInitial() > else: > ini_state=InitialList[r] > traj.append(ini_state) > prev=ini_state > collect[0]=StringAdd(collect[0],prev) > for s in range(steps): > if self.INPUT['mode']=='Sync': > next_state=IterOneSync(prev) > elif self.INPUT['mode']=='GA': > next_state=IterOneAsync(prev) > elif self.INPUT['mode'] == 'ROA': > next_state=IterOneROA(prev) > traj.append(next_state) > collect[s+1]=StringAdd(collect[s+1],next_state) > prev=next_state > traj_all.append(traj) > final_all.append(traj[-1]) > out={} > normalized=[map(divide,each_step) for each_step in collect] > nodes_list=self.GetNodes() > for node_i in range(len(nodes_list)): > out[nodes_list[node_i]]=[state[node_i] for state in normalized] > self.FINAL[nodes_list[node_i]]=out[nodes_list[node_i]][-1] > return out,final_all > > def ParaParser(ParaFile): > '''#parser parameters for simulation and transition matrix building''' > INPUT = {'rules' : 'CAC.txt', > 'ini_on' : 'APC, IKB', > 'ini_off' : 'Proliferation, Apoptosis', > 'turn_on' : 'DC', > 'turn_off' : '', > 'rounds' : '500', > 'steps' : '30', > 'equi_steps': 0, > 'mode' : 'sync', > 'mutation_list' : 'nodes.txt', > 'mutation_mode' : 'single', > 'observe_list' : '', > 'keep_state' : 'False', > 'missing' : 'random' > } # define parameters > > for each_line in open(ParaFile).readlines(): > para_name = each_line.split('=')[0].strip() > para_value = each_line.split('=')[1].strip() > if para_name in INPUT.keys(): > INPUT[para_name] = para_value > else: > print "Error: Unknown Parameters: %s" % para_name > # formalize parameters > > try: > INPUT['rules'] = str(INPUT['rules'].strip()) > INPUT['ini_on'] = [node.strip() for node in INPUT['ini_on'].split(',')] > INPUT['ini_off'] = [node.strip() for node in INPUT['ini_off'].split(',')] > INPUT['turn_on'] = [node.strip() for node in INPUT['turn_on'].split(',')] > INPUT['turn_off'] = [node.strip() for node in INPUT['turn_off'].split(',')] > INPUT['observe_list'] = [node.strip() for node in INPUT['observe_list'].split(',')] > INPUT['mutation_list'] = INPUT['mutation_list'].split(',') > INPUT['keep_state'] = INPUT['keep_state'].split(',') > INPUT['rounds'] = int(INPUT['rounds']) > INPUT['steps'] = int(INPUT['steps']) > INPUT['mode'] = str(INPUT['mode']) > INPUT['equi_steps']=int(INPUT['equi_steps']) > INPUT['mutation_mode']=str(INPUT['mutation_mode']) > for empty_keys in INPUT.keys(): > if INPUT[empty_keys] == ['']: INPUT[empty_keys] = [] > except: > print "Error: Invalid input data types!" > > if INPUT['mode'] not in ['GA', 'Sync','ROA']: print "Wrong simulation method! Using 'GA', 'ROA' or 'Sync'" > return INPUT > > > def ChangeInitial(NodesList,States,MutePairs): > OutStates=[] > for EachState in States: > State=list(EachState) > for MuteNode,MuteState in MutePairs: > State[NodesList.index(MuteNode)]=str(int(MuteState)) > OutStates.append(''.join(State)) > return OutStates > > def simu_mutation(INPUT,missing='random'): > '''Do simulated mutation expriment of the model''' > mut_result={} # to store the mutated result > #generate mutation list, only single mutation now > mut_list=GenerateCombinations(INPUT) > > if INPUT['equi_steps'] == 0: > for mute_pair in mut_list: > mut_result[mute_pair]=[] > print 'Now doing mutation',mute_pair > model=Model(INPUT,mute_pair) > result,final_all=model.IterModel(missing=missing) > for out_node in INPUT['observe_list']: > mut_result[mute_pair].append('%2.2f'%(sum(result[out_node][-50:])/50)) > elif INPUT['equi_steps'] > 0: > print 'Now running pre-mutation simulation to reach steady state\n' > model=Model(INPUT) > AllNodes=model.GetNodes() > result_pre,final_pre=model.IterModel(missing) #pre-equilibration run > print 'Now running mutation scanning\n' > for mute_pair in mut_list: > mut_result[mute_pair]=[] > print 'Now doing mutation',mute_pair > new_model=Model(INPUT,mute_pair) > pre_modified=ChangeInitial(AllNodes,final_pre,mute_pair) > result,final=new_model.IterModel(missing=missing, InitialList=pre_modified) #actual mutation run for each mutation > for out_node in INPUT['observe_list']: > mut_result[mute_pair].append(('%2.2f'%(sum(result_pre[out_node][-50:])/50),'%2.2f'%(sum(result[out_node][-50:])/50))) > > print "Write mutation result to 'mutation_result.csv' " > output=open('mutation_result.csv','w') > if INPUT['equi_steps'] == 0: #direct mutation > output.writelines('MutNodes,MutStates,%s\n'%(','.join(INPUT['observe_list']))) > for mute_pair in mut_list: > nodes=[] > states=[] > for node,state in mute_pair: > nodes.append(node) > states.append(str(state)) > output.writelines('%s,%s,%s\n'%('/'.join(nodes),'/'.join(states),','.join(mut_result[mute_pair]))) > elif INPUT['equi_steps'] > 0: #mutation with pre-equilibration steps > out_list=[] > for node in INPUT['observe_list']: > out_list.append(node) > out_list.append(node+'(mut)') > output.writelines('MutNodes,MutStates,%s\n'%(','.join(out_list))) > for mute_pair in mut_list: > nodes=[] > states=[] > for node,state in mute_pair: > nodes.append(node) > states.append(str(state)) > result_list=[each_result for each_pair in mut_result[mute_pair] for each_result in each_pair] # collapse results > output.writelines('%s,%s,%s\n'%('/'.join(nodes),'/'.join(states),','.join(result_list))) > output.close() > > def GenerateCombinations(INPUT): > StrBool={'True':True,'False':False} > NodeState=[] > Combine=[] > for node_list in INPUT['mutation_list']: > for node in open(node_list): > NodeState.append((node.strip(),StrBool[INPUT['keep_state'][INPUT['mutation_list'].index(node_list)]])) > if INPUT['mutation_mode']=='single': > for mute_state in NodeState: > Combine.append((mute_state,)) > return Combine > elif INPUT['mutation_mode']=='double': > for i in range(len(NodeState)): > for j in range(i+1,len(NodeState)): > if NodeState[i][0] != NodeState[j][0]: > Combine.append((NodeState[i],NodeState[j])) > print 'Total %s combinations.'%(len(Combine)) > return Combine > if __name__ == '__main__': > try: > para=ParaParser(sys.argv[1]) > except: > para=ParaParser('mutation.in') > simu_mutation(para) > > > After running this file, I got the following error:/home/JPJ/Priya_Ph.D/simple_bool/simplebool/SimpleBool-master/BoolMutation.py in () > 383 para=ParaParser(sys.argv[1]) > 384 except: > --> 385 para=ParaParser('mutation.in') > 386 simu_mutation(para) > > /home/JPJ/Priya_Ph.D/simple_bool/simplebool/SimpleBool-master/BoolMutation.py in ParaParser(ParaFile) > 254 } # define parameters > 255 > --> 256 for each_line in open(ParaFile).readlines(): > 257 para_name = each_line.split('=')[0].strip() > 258 para_value = each_line.split('=')[1].strip() > > IOError: [Errno 2] No such file or directory: 'mutation.in' > > > I can't understand, the problem here. Should I specify anything in the Boolmutation.py file. Any help would be appreciated > Thank you > Regards > Priya > -- > https://mail.python.org/mailman/listinfo/python-list -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From mr.marydavid at gmail.com Wed Oct 5 11:38:59 2016 From: mr.marydavid at gmail.com (mr.marydavid at gmail.com) Date: Wed, 5 Oct 2016 08:38:59 -0700 (PDT) Subject: Looking for the best way to make this Message-ID: <7a6fbcbe-f5cc-4b35-90bb-f04dc2d773b2@googlegroups.com> http://www.qualicode.com/EN/images/ScheduleGrid.png looking to add schedule like that to my software trying with Qt but not sure how to . From mr.marydavid at gmail.com Wed Oct 5 12:04:19 2016 From: mr.marydavid at gmail.com (mr.marydavid at gmail.com) Date: Wed, 5 Oct 2016 09:04:19 -0700 (PDT) Subject: Looking for the best way to make this In-Reply-To: <7a6fbcbe-f5cc-4b35-90bb-f04dc2d773b2@googlegroups.com> References: <7a6fbcbe-f5cc-4b35-90bb-f04dc2d773b2@googlegroups.com> Message-ID: Here is another example http://www.codetwo.com/media/images/exchange-sync-screencast-5.jpg From python at mrabarnett.plus.com Wed Oct 5 12:40:51 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 5 Oct 2016 17:40:51 +0100 Subject: Python -Simplebool In-Reply-To: <50d6b0a3-9794-4e0f-9d2d-1b096e67949e@googlegroups.com> References: <50d6b0a3-9794-4e0f-9d2d-1b096e67949e@googlegroups.com> Message-ID: On 2016-10-05 16:01, priya.bmg at gmail.com wrote: > Hi, > I am new to Python and I am learning it. Need some help. I downloaded SimpleBool (https://github.com/lujunyan1118/SimpleBool) for simulating Boolean models. SimpleBool works in Python. I downloaded the Canopy and executed Boolmutation.py file. Below is the Boolmutation.py file > [snip] > > After running this file, I got the following error:/home/JPJ/Priya_Ph.D/simple_bool/simplebool/SimpleBool-master/BoolMutation.py in () > 383 para=ParaParser(sys.argv[1]) > 384 except: > --> 385 para=ParaParser('mutation.in') > 386 simu_mutation(para) > > /home/JPJ/Priya_Ph.D/simple_bool/simplebool/SimpleBool-master/BoolMutation.py in ParaParser(ParaFile) > 254 } # define parameters > 255 > --> 256 for each_line in open(ParaFile).readlines(): > 257 para_name = each_line.split('=')[0].strip() > 258 para_value = each_line.split('=')[1].strip() > > IOError: [Errno 2] No such file or directory: 'mutation.in' > > > I can't understand, the problem here. Should I specify anything in the Boolmutation.py file. Any help would be appreciated > The traceback says what the problem is: """No such file or directory: 'mutation.in'""". You haven't given it the path to the input file on the command line, so it's defaulting to 'mutation.in', which it can't find or doesn't exist. From python at mrabarnett.plus.com Wed Oct 5 12:50:01 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 5 Oct 2016 17:50:01 +0100 Subject: I can't run python on my computer In-Reply-To: <1886398783.261103.1475677140466@mail.yahoo.com> References: <1886398783.261103.1475677140466.ref@mail.yahoo.com> <1886398783.261103.1475677140466@mail.yahoo.com> Message-ID: <04989516-0d91-3558-260e-a80f943ba141@mrabarnett.plus.com> On 2016-10-05 15:19, Camille Benoit via Python-list wrote: > Hi,it has been about a week since the last time I was able to use Python. Most of the time, the interpreter doesn't show up and when it does and I am trying to run a program it displayed the following message: "IDLE's subprocess didn't make connection. Either IDLE can't start a subprocess or personal firewall software is blocking the connection." Can you please help me with this. It'd be a huge help. > > Thank you very much. > There are plenty of answers on StackOverflow, e.g.: Can't run Python via IDLE from Explorer [2013] - IDLE's subprocess didn't make connection http://stackoverflow.com/questions/15888186/cant-run-python-via-idle-from-explorer-2013-idles-subprocess-didnt-make-c Python error - IDLE's subprocess didn't make connection. Either IDLE can't start or personal firewall software is blocking connection http://stackoverflow.com/questions/29567051/python-error-idles-subprocess-didnt-make-connection-either-idle-cant-start It might be caused by trying to run a program whose name is the same as one of Python's standard modules, so that when Python tries to use the module, it gets your program instead. From cloverobert at gmail.com Wed Oct 5 13:35:25 2016 From: cloverobert at gmail.com (Robert Clove) Date: Wed, 5 Oct 2016 23:05:25 +0530 Subject: Has any one automated the vmware-vra setup using python? Message-ID: From noah at neo.co.tz Wed Oct 5 13:46:44 2016 From: noah at neo.co.tz (Noah) Date: Wed, 5 Oct 2016 20:46:44 +0300 Subject: Python and ssh for remote login Message-ID: Hello folk, I would like to use a python script to ssh into a server using a username and password and perhaps ass port. Any ideas on how to script that. Thanks Noah From python at bdurham.com Wed Oct 5 13:56:59 2016 From: python at bdurham.com (Malcolm Greene) Date: Wed, 05 Oct 2016 13:56:59 -0400 Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? Message-ID: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> Looking for a quick way to calculate lines of code/comments in a collection of Python scripts. This isn't a LOC per day per developer type analysis - I'm looking for a metric to quickly judge the complexity of a set of scripts I'm inheriting. Thank you, Malcolm From rosuav at gmail.com Wed Oct 5 14:08:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Oct 2016 05:08:57 +1100 Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? In-Reply-To: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> Message-ID: On Thu, Oct 6, 2016 at 4:56 AM, Malcolm Greene wrote: > Looking for a quick way to calculate lines of code/comments in a > collection of Python scripts. This isn't a LOC per day per developer > type analysis - I'm looking for a metric to quickly judge the complexity > of a set of scripts I'm inheriting. Post the code to GitHub, then click on the file. Up the top, you'll see something like this: https://github.com/Rosuav/LetMeKnow/blob/master/letmeknow.py 231 lines (212 sloc) The first figure is the raw line count. The second is, in theory at least, the number of lines of "actual code". Cheating? Totally. Effective? Yup. ChrisA From rgaddi at highlandtechnology.invalid Wed Oct 5 14:27:53 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 5 Oct 2016 18:27:53 -0000 (UTC) Subject: Python and ssh for remote login References: Message-ID: Noah wrote: > Hello folk, > > I would like to use a python script to ssh into a server using a username > and password and perhaps ass port. > > Any ideas on how to script that. > > Thanks > > Noah paramiko -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From noah at neo.co.tz Wed Oct 5 14:36:32 2016 From: noah at neo.co.tz (Noah) Date: Wed, 5 Oct 2016 21:36:32 +0300 Subject: Python and ssh for remote login In-Reply-To: References: Message-ID: Hi Rob Thank you for your email. I am guessing that is some module. Ok i will pip it and see how it goes. Noah On 5 Oct 2016 21:32, "Rob Gaddi" wrote: > Noah wrote: > > > Hello folk, > > > > I would like to use a python script to ssh into a server using a username > > and password and perhaps ass port. > > > > Any ideas on how to script that. > > > > Thanks > > > > Noah > > paramiko > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. > -- > https://mail.python.org/mailman/listinfo/python-list > From ethan at stoneleaf.us Wed Oct 5 15:01:10 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 05 Oct 2016 12:01:10 -0700 Subject: Python and ssh for remote login In-Reply-To: References: Message-ID: <57F54DF6.40402@stoneleaf.us> On 10/05/2016 10:46 AM, Noah wrote: > I would like to use a python script to ssh into a server using a username > and password [...] I've written a module called scription to help with scripts; it supports giving passwords to programs like ssh. Here's an example from one of my utility scripts: -- 8< --------------------------- #!/usr/local/bin/python from getpass import getpass from antipathy import Path from scription import Command, Execute, Main, OPTION, REQUIRED ... @Command( repo=('repository to pull [default: all]', OPTION), path=('base path to search', OPTION, 'p', Path), ) def pull(repo, *path): ''' retrieve remote change sets ''' password = getpass('[mercurial] password: ') target = repo for repo in workhorse(*path): if target and repo.filename != target: continue history = Execute(('hg', 'pull'), cwd=repo, password=password, pty=True) -- 8< ---------------------------- and in use: ================================== $ hgc --help Available commands/options in hgc incoming displays changesets in remote repo not present locally list displays all repos log-date displays all log entries for matching date outgoing displays changesets in remote repo not present locally parent displays parent of active branch pull retrieve remote change sets push send local changesets to remote repo status display status for each repo update update active files to latest version $ hgc pull [mercurial] password: ... =================================== It's available via pip. Feedback welcome. :) -- ~Ethan~ From noah at neo.co.tz Wed Oct 5 15:06:59 2016 From: noah at neo.co.tz (Noah) Date: Wed, 5 Oct 2016 22:06:59 +0300 Subject: Python and ssh for remote login In-Reply-To: References: <57F54DF6.40402@stoneleaf.us> Message-ID: On 5 Oct 2016 22:02, "Ethan Furman" wrote: > > On 10/05/2016 10:46 AM, Noah wrote: > >> I would like to use a python script to ssh into a server using a username >> and password [...] > > > I've written a module called scription to help with scripts; it supports giving passwords to programs like ssh. > Hi Ethan and for writting this script. > Here's an example from one of my utility scripts: > > -- 8< --------------------------- > #!/usr/local/bin/python > > from getpass import getpass > from antipathy import Path > from scription import Command, Execute, Main, OPTION, REQUIRED > > ... > > @Command( > repo=('repository to pull [default: all]', OPTION), > path=('base path to search', OPTION, 'p', Path), > ) > def pull(repo, *path): > ''' > retrieve remote change sets > ''' > password = getpass('[mercurial] password: ') > target = repo > for repo in workhorse(*path): > if target and repo.filename != target: > continue > history = Execute(('hg', 'pull'), cwd=repo, password=password, pty=True) > > -- 8< ---------------------------- > > and in use: > > ================================== > > $ hgc --help > Available commands/options in hgc > incoming displays changesets in remote repo not present locally > list displays all repos > log-date displays all log entries for matching date > outgoing displays changesets in remote repo not present locally > parent displays parent of active branch > pull retrieve remote change sets > push send local changesets to remote repo > status display status for each repo > update update active files to latest version > > $ hgc pull > [mercurial] password: > > ... > =================================== > > It's available via pip. Feedback welcome. :) > I will pip it too and try it out and give some feedback based on my use case. Thanks a lot. > -- > ~Ethan~ > -- Noah From davros at bellaliant.net Wed Oct 5 15:17:33 2016 From: davros at bellaliant.net (John McKenzie) Date: Wed, 05 Oct 2016 19:17:33 GMT Subject: Passing Variable to Function Message-ID: Hello, all. I have a function that takes three arguments, arguments to express an RGB colour. The function controls an LED light strip (a Blinkytape). Sometimes I might need to be change what colour is sent to the function, so I set a variable with the idea that I can change just the variable later if I need to instead of changing a bunch of different lines. So I have variables along the lines of this: colour ="255, 0, 0" colour2 ="100, 0, 0" My function, written by the Blinkytape people: def changeColor(r, g, b): serialPorts = glob.glob("/dev/ttyACM0*") port = serialPorts[0] if not port: sys.exit("Could not locate a BlinkyTape.") print "BlinkyTape found at: %s" % port bt = BlinkyTape.BlinkyTape(port) bt.displayColor(r, g, b) time.sleep(.1) # Give the serial driver some time to actually send the data bt.close() Later, I have conditional statements like: if latitude > maxNetural and latitude < NorthLine: changeColor(colour) elif latitude > NorthLine: changeColor(colour2) (There is a GPS device connected, there are variables defined based on latitude earlier in the script.) I get an error stating that changeColor takes three arguments and I am just giving it one (either "colour1" or "colour2"). Is there a way to format the "colour" variable so that the changeColor function takes it as the three numbers it is meant to be defined as? Entire script: http://hastebin.com/qaqotusozo.py Thanks. From toytony at sbcglobal.net Wed Oct 5 15:18:43 2016 From: toytony at sbcglobal.net (Mike Adams) Date: Wed, 5 Oct 2016 19:18:43 +0000 (UTC) Subject: Problem with install of Python 3.5.2 on Windows Vista References: <2135313766.5879749.1475695123241.ref@mail.yahoo.com> Message-ID: <2135313766.5879749.1475695123241@mail.yahoo.com> The install seemed to be going well up to near the end when I got the msg 'Python has stopped working', I clicked the button then I got the msg 'Setup was successful'.? I then clicked 'IDLE' and it says it can't find 'api-msi-win-crt-runtime-11-1-0.dll'.? I checked Python Tracker and saw that there was a similar issue with Python 3.5.0 which has been fixed.? So, what do I need to do now to correct this problem? From 007brendan at gmail.com Wed Oct 5 15:22:49 2016 From: 007brendan at gmail.com (Brendan Abel) Date: Wed, 5 Oct 2016 12:22:49 -0700 Subject: Passing Variable to Function In-Reply-To: References: Message-ID: Define your colors as actual number variables instead of a string color = (255,0,0) color2 = (0,0,255) Then use argument expansion to pass them in as separate arguments to the function colorFunction(*color) Brendan On Wed, Oct 5, 2016 at 12:17 PM, John McKenzie wrote: > > Hello, all. > > I have a function that takes three arguments, arguments to express an RGB > colour. The function controls an LED light strip (a Blinkytape). > > Sometimes I might need to be change what colour is sent to the function, > so I set a variable with the idea that I can change just the variable > later if I need to instead of changing a bunch of different lines. > > So I have variables along the lines of this: > > colour ="255, 0, 0" > colour2 ="100, 0, 0" > > > My function, written by the Blinkytape people: > > > def changeColor(r, g, b): > serialPorts = glob.glob("/dev/ttyACM0*") > port = serialPorts[0] > > if not port: > sys.exit("Could not locate a BlinkyTape.") > > print "BlinkyTape found at: %s" % port > > bt = BlinkyTape.BlinkyTape(port) > bt.displayColor(r, g, b) > time.sleep(.1) # Give the serial driver some time to actually send > the data > bt.close() > > > Later, I have conditional statements like: > > > if latitude > maxNetural and latitude < NorthLine: > changeColor(colour) > elif latitude > NorthLine: > changeColor(colour2) > > > > (There is a GPS device connected, there are variables defined based on > latitude earlier in the script.) > > I get an error stating that changeColor takes three arguments and I am > just giving it one (either "colour1" or "colour2"). > > > Is there a way to format the "colour" variable so that the changeColor > function takes it as the three numbers it is meant to be defined as? > > > Entire script: > http://hastebin.com/qaqotusozo.py > > > Thanks. > > > -- > https://mail.python.org/mailman/listinfo/python-list > From alister.ware at ntlworld.com Wed Oct 5 15:23:08 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 05 Oct 2016 19:23:08 GMT Subject: Passing Variable to Function References: Message-ID: On Wed, 05 Oct 2016 19:17:33 +0000, John McKenzie wrote: > Hello, all. > > I have a function that takes three arguments, arguments to express an > RGB > colour. The function controls an LED light strip (a Blinkytape). > > Sometimes I might need to be change what colour is sent to the > function, > so I set a variable with the idea that I can change just the variable > later if I need to instead of changing a bunch of different lines. > > So I have variables along the lines of this: > > colour ="255, 0, 0" > colour2 ="100, 0, 0" > > > My function, written by the Blinkytape people: > > > def changeColor(r, g, b): > serialPorts = glob.glob("/dev/ttyACM0*") > port = serialPorts[0] > > if not port: > sys.exit("Could not locate a BlinkyTape.") > > print "BlinkyTape found at: %s" % port > > bt = BlinkyTape.BlinkyTape(port) > bt.displayColor(r, g, b) > time.sleep(.1) # Give the serial driver some time to actually send > the data > bt.close() > > > Later, I have conditional statements like: > > > if latitude > maxNetural and latitude < NorthLine: > changeColor(colour) > elif latitude > NorthLine: > changeColor(colour2) > > > > (There is a GPS device connected, there are variables defined based on > latitude earlier in the script.) > > I get an error stating that changeColor takes three arguments and I am > just giving it one (either "colour1" or "colour2"). > > > Is there a way to format the "colour" variable so that the changeColor > function takes it as the three numbers it is meant to be defined as? > > > Entire script: > http://hastebin.com/qaqotusozo.py > > > Thanks. use a list & pythons parameter expansion options (I am sure the experts here will be able to provide the correct name) colour1=[100,0,255] def set_colour(r,g,b): set_colour(*colour1) -- Come live with me and be my love, And we will some new pleasures prove Of golden sands and crystal brooks With silken lines, and silver hooks. There's nothing that I wouldn't do If you would be my POSSLQ. You live with me, and I with you, And you will be my POSSLQ. I'll be your friend and so much more; That's what a POSSLQ is for. And everything we will confess; Yes, even to the IRS. Some day on what we both may earn, Perhaps we'll file a joint return. You'll share my pad, my taxes, joint; You'll share my life - up to a point! And that you'll be so glad to do, Because you'll be my POSSLQ. From bc at freeuk.com Wed Oct 5 15:35:55 2016 From: bc at freeuk.com (BartC) Date: Wed, 5 Oct 2016 20:35:55 +0100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: On 05/10/2016 14:43, Rustom Mody wrote: > On Wednesday, October 5, 2016 at 7:05:02 PM UTC+5:30, BartC wrote: >> That's the one it can't find. Actually it can't seem to import any >> standard libraries, so the implementation must be screwed up. >> >> Or is it because I downloaded the Minimal** rather Full version? The >> minimal version is already 1.7GB! Can it really be that big and yet not >> include such basic stuff? If so, what on earth is taking up all the >> space! (The full version was 10-15% bigger.) >> >> (** >> https://downloads.haskell.org/~platform/8.0.1/HaskellPlatform-8.0.1-minimal-x86_64-setup-a.exe) > > Minimal means only compiler: NO libraries except what the compiler needs to compile. > Almost certainly wrong for a beginner?s testing the waters. > > You should probably download haskell-platform (full) (I've downloaded the full version. Now I have 25,000 files in 2200MB. And it still doesn't work. This is exactly why I persist with my own language implementations. My current one is only 1/5000th the size but standard libraries are included!) -- Bartc From irmen.NOSPAM at xs4all.nl Wed Oct 5 15:40:23 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Wed, 5 Oct 2016 21:40:23 +0200 Subject: rocket simulation game with just using tkinter In-Reply-To: References: <57eeee41$0$911$e4fe514c@news.xs4all.nl> <57f01771$0$863$e4fe514c@news.xs4all.nl> <57f101bb$0$940$e4fe514c@news.xs4all.nl> <57f3f83d$0$27768$e4fe514c@news.xs4all.nl> <70j8vb9itp3f23vau8a5eo5qsojjvtpvo6@4ax.com> Message-ID: <57f55728$0$16772$e4fe514c@news.xs4all.nl> On 5-10-2016 2:43, Dennis Lee Bieber wrote: > Or do what I once did with the Lunar Lander game on my college > computer... Target accuracy: Excellent... Landing gear? somewhere on the > other side of the moon. My vertical velocity was sub-inches per second -- > the horizontal velocity was in multi-feet per second. My lander game takes the magnitude of the velocity into account and not the direction, so you will still crash when going down a pixel per second but strafing like mad :) > I also used to crash with more fuel than I started with... Game didn't > do a sign check on "lbs of fuel to burn", so -10lbs @ 180deg had the same > effect as 10lbs @ 0deg, but gained fuel. I forgot to add a fuel gauge! My rocket can fly for all eternity if you manage to keep it in the air :P The first rocket landing game I ever came across must have been a text only version running on my commodore 64 where it was only printing your current altitude and velocity and fuel level, and then stopped to ask for input. I think it asked for values for how long to burn the engine and for how many seconds. Then it recomputed everything and stopped again for input until you landed safely or crashed... It must have been a short BASIC listing I copied over from a magazine or something. Irmen From ben+python at benfinney.id.au Wed Oct 5 15:54:20 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 06 Oct 2016 06:54:20 +1100 Subject: Code complexity analysis of Python code (was: Quick way to calculate lines of code/comments in a collection of Python scripts?) References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> Message-ID: <8560p6k1w3.fsf@benfinney.id.au> Malcolm Greene writes: > Looking for a quick way to calculate lines of code/comments in a > collection of Python scripts. This isn't a LOC per day per developer > type analysis - I'm looking for a metric to quickly judge the complexity > of a set of scripts I'm inheriting. Prospector is a static analysis tool for Python code. Its documentation promises to do what you're asking, please let us know how it works for you. -- \ ?[Entrenched media corporations will] maintain the status quo, | `\ or die trying. Either is better than actually WORKING for a | _o__) living.? ?ringsnake.livejournal.com, 2007-11-12 | Ben Finney From michael at felt.demon.nl Wed Oct 5 16:06:50 2016 From: michael at felt.demon.nl (Michael Felt) Date: Wed, 5 Oct 2016 22:06:50 +0200 Subject: working with ctypes and complex data structures In-Reply-To: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> Message-ID: On 02-Oct-16 19:50, Michael Felt wrote: > I am trying to understand the documentation re: ctypes and interfacing > with existing libraries. > > I am reading the documentation, and also other sites that have largely > just copied the documentation - as well as the "free chapter" at > O'Reilly (Python Cookbook). > Using O'Reilly and the standard documentation I now have these two excerpts. The second def works, the first does not. Someone helping me understand what I am not reading properly in this bit of documentation - is much appreciated. >>>from ctypes import c_int, WINFUNCTYPE, windll >>>from ctypes.wintypes import HWND, LPCSTR, UINT >>>prototype = WINFUNCTYPE(c_int, HWND, LPCSTR, LPCSTR, UINT) >>>paramflags = (1, "hwnd", 0), (1, "text", "Hi"), (1, "caption", None), (1, "flags", 0) >>>MessageBox = prototype(("MessageBoxA", windll.user32), paramflags) >>> The MessageBox foreign function can now be called in these ways: >>> >>>MessageBox() >>>MessageBox(text="Spam, spam, spam") >>>MessageBox(flags=2, text="foo bar") >>> Note that MessageBox() may give two arguments. My code: note both def init() are meant to do the same thing, just different call syntax. +76 class cpu_total: +77 def __init__(self): +78 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") +79 prototype = CFUNCTYPE(c_int, c_void_p, c_void_p, c_int, c_int) +80 args = (1, "name", None), (2, "buff", None), (1, "size", 0), (1, "count", 1) +81 cpu_total = prototype(("perfstat_cpu_total", __perfstat__), args) +82 +83 buff = perfstat_cpu_total_t() +84 cpu_total(buff=buff, size=sizeof(buff)) +85 +86 class perfstat_cpu_total: +87 def __init__(self): +88 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") +89 cpu_total = __perfstat__.perfstat_cpu_total +90 +91 _perfstat_cpu_total = perfstat_cpu_total_t() +92 ret = cpu_total(None, +93 byref(_perfstat_cpu_total), +94 sizeof(perfstat_cpu_total_t), 1) +95 self.boot = _perfstat_cpu_total.lbolt.v / 100 +96 self.ncpus = _perfstat_cpu_total.ncpus +97 self._v = _perfstat_cpu_total When I call the second I can print values and even call functions, reload the _v, etc.. The first class fails with this message: a = cpu_total() Traceback (most recent call last): File "", line 135, in File "", line 84, in __init__ TypeError: call takes exactly 1 arguments (2 given) Thanks for your attention. From saxri89 at gmail.com Wed Oct 5 16:14:18 2016 From: saxri89 at gmail.com (Xristos Xristoou) Date: Wed, 5 Oct 2016 13:14:18 -0700 (PDT) Subject: python and dos files question Message-ID: hello some programs allow to export tools to dos .bat files for running from command line. my question is how can i do to running .bat files and how can i do to define some variables from the .bat file before the running. for example i want to define 4 variable and run this .bat file with the python variables can be strings paths,images and text. .bat file like this : @ECHO OFF REM SET program1=C:\program1\Modules REM SET PATH=%PATH%;C:\program1 REM Tool: Tool1 program1_cmd tool1 0 -variable1=NULL -variable2=NULL -variable3=NULL -variable4=NULL - PAUSE From eryksun at gmail.com Wed Oct 5 16:20:38 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 5 Oct 2016 20:20:38 +0000 Subject: Problem with install of Python 3.5.2 on Windows Vista In-Reply-To: <2135313766.5879749.1475695123241@mail.yahoo.com> References: <2135313766.5879749.1475695123241.ref@mail.yahoo.com> <2135313766.5879749.1475695123241@mail.yahoo.com> Message-ID: On Wed, Oct 5, 2016 at 7:18 PM, Mike Adams wrote: > The install seemed to be going well up to near the end when I got the msg 'Python > has stopped working', I clicked the button then I got the msg 'Setup was successful'. > I then clicked 'IDLE' and it says it can't find 'api-msi-win-crt-runtime-11-1-0.dll'. I > checked Python Tracker and saw that there was a similar issue with Python 3.5.0 > which has been fixed. So, what do I need to do now to correct this problem? Make sure your OS is up to date via Windows Update. Microsoft is supposed to be distributing the Universal CRT as an OS component for Vista SP2. If you're up to date and the system still can't find CRT DLLs, then try manually installing the CRT update. It's available for download here: https://support.microsoft.com/en-us/kb/3118401 From emile at fenx.com Wed Oct 5 16:29:38 2016 From: emile at fenx.com (Emile van Sebille) Date: Wed, 5 Oct 2016 13:29:38 -0700 Subject: working with ctypes and complex data structures In-Reply-To: References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> Message-ID: On 10/05/2016 01:06 PM, Michael Felt wrote: > > > On 02-Oct-16 19:50, Michael Felt wrote: >> I am trying to understand the documentation re: ctypes and interfacing >> with existing libraries. >> >> I am reading the documentation, and also other sites that have largely >> just copied the documentation - as well as the "free chapter" at >> O'Reilly (Python Cookbook). >> > Using O'Reilly and the standard documentation I now have these two > excerpts. The second def works, the first does not. > > Someone helping me understand what I am not reading properly in this bit > of documentation - is much appreciated. > >>>> from ctypes import c_int, WINFUNCTYPE, windll >>>> from ctypes.wintypes import HWND, LPCSTR, UINT >>>> prototype = WINFUNCTYPE(c_int, HWND, LPCSTR, LPCSTR, UINT) >>>> paramflags = (1, "hwnd", 0), (1, "text", "Hi"), (1, >>>> "caption", None), (1, "flags", 0) >>>> MessageBox = prototype(("MessageBoxA", windll.user32), paramflags) >>>> > > The MessageBox foreign function can now be called in these ways: > >>>> > >>>> MessageBox() >>>> MessageBox(text="Spam, spam, spam") >>>> MessageBox(flags=2, text="foo bar") >>>> > > Note that MessageBox() may give two arguments. > > My code: note both def init() are meant to do the same thing, just > different call syntax. > > +76 class cpu_total: > +77 def __init__(self): > +78 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") > +79 prototype = CFUNCTYPE(c_int, c_void_p, c_void_p, c_int, > c_int) > +80 args = (1, "name", None), (2, "buff", None), (1, "size", > 0), (1, "count", 1) > +81 cpu_total = prototype(("perfstat_cpu_total", > __perfstat__), args) > +82 > +83 buff = perfstat_cpu_total_t() > +84 cpu_total(buff=buff, size=sizeof(buff)) > +85 > +86 class perfstat_cpu_total: > +87 def __init__(self): > +88 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") > +89 cpu_total = __perfstat__.perfstat_cpu_total > +90 > +91 _perfstat_cpu_total = perfstat_cpu_total_t() > +92 ret = cpu_total(None, > +93 byref(_perfstat_cpu_total), > +94 sizeof(perfstat_cpu_total_t), 1) > +95 self.boot = _perfstat_cpu_total.lbolt.v / 100 > +96 self.ncpus = _perfstat_cpu_total.ncpus > +97 self._v = _perfstat_cpu_total > > > When I call the second I can print values and even call functions, > reload the _v, etc.. > > The first class fails with this message: > > a = cpu_total() > Traceback (most recent call last): > File "", line 135, in > File "", line 84, in __init__ > TypeError: call takes exactly 1 arguments (2 given) > > Thanks for your attention. the line it's objecting to is: > +84 cpu_total(buff=buff, size=sizeof(buff)) because the expectation is defined as: > +77 def __init__(self): HTH, Emile From michael at felt.demon.nl Wed Oct 5 17:03:01 2016 From: michael at felt.demon.nl (Michael Felt) Date: Wed, 5 Oct 2016 23:03:01 +0200 Subject: working with ctypes and complex data structures In-Reply-To: References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> Message-ID: <8ee86d1e-df95-6aad-f9cd-119b8bbafba9@felt.demon.nl> On 05-Oct-16 22:29, Emile van Sebille wrote: Thanks for the reply! After a shirt coffeebreak - back into the fray - and I found the following: > +76 class cpu_total: > +77 def __init__(self): > +78 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") > +79 prototype = CFUNCTYPE(c_int, c_void_p, c_void_p, c_int, > c_int) error #2 - see below > +80 args = (1, "name", None), (2, "buff", None), (1, "size", > 0), (1, "count", 1) error #1. paramater type 2 (the buffer might be where data is being put, but for the call, the pointer is INPUT) > +81 cpu_total = prototype(("perfstat_cpu_total", > __perfstat__), args) > +82 > +83 buff = perfstat_cpu_total_t() > +84 cpu_total(buff=buff, size=sizeof(buff)) > +85 The error #2 is #2, because only after I corrected the inputs did I get Type Exceptions. I had hoped that c_void_p was a "typeless" type, mainly the right size, but the ctypes interface is more particular when you use the CFUNCTYPE() - so now I have my boilerplate for what I hope can be a tutorial for interfacing - by hand - with complex functions, i.e., using one class as the data aka typedef defintion, and a second class to work on it. In template form: class perfstat_xxx_t: _fields_ = ( ... ) class perfstat_xxx: def init(self): _perflib = ctypes.CDLL("libperfstat.a(shr_64.o)") # AIX member in an archive _fn_xxx = _perflib.xxx # ALL AIX perfstat routines have 4 arguments: (name, buff, sizeof_buff, count) # the buff is a pointer to where the information will be stored _fn = CFUNCTYPE(c_int, c_void_p, POINTER(xxx_t), c_int, c_int) _args = (1, "name", None), (1, "buff", None), (1, "size", 0), (1, "count", 1) _xxx = _fn(("xxx", _fn_xxx), _args) _buff = perfstat_cpu_total_t() _xxx(buff=_buff, size=sizeof(_buff)) self._v = _buff So, the next question/test will be to move the _fields_ into the "second" class definition. I just need to find what "self" attribute that is. From michael at felt.demon.nl Wed Oct 5 17:11:49 2016 From: michael at felt.demon.nl (Michael Felt) Date: Wed, 5 Oct 2016 23:11:49 +0200 Subject: working with ctypes and complex data structures In-Reply-To: References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> Message-ID: <9802a3bd-94cb-e58b-d73c-173bbcfe576b@felt.demon.nl> Never said thank you - so, thanks! What I need to do was add the .v at the end so I was accessing the value of the structure. Unlilke Linux, AIX - for reasons unknown to all, they have the time_t definition that is specific to the ABI size, at least for these performance libraries that probably originated before 64-bit was a concern. So my guess is that the dual size in the struct, depending on the application size (not the kernel) is to maintain compatibility with 32-bit applications that had been built against/on a 32-bit kernel. So, lucky for me it did not work intiallly - because it made me pause and understand better what I had written. And now the real thankyou for the detail you shared! M On 03-Oct-16 17:53, eryk sun wrote: > On Mon, Oct 3, 2016 at 2:35 PM, Michael Felt wrote: >> On 02-Oct-16 23:44, eryk sun wrote: >>> On Sun, Oct 2, 2016 at 5:50 PM, Michael Felt >>> wrote: >>> >>>> b) what I am not understanding - as the basic documentation shows >>>> FOO.value as the way to set/get the value of a _field_ >>> You may be surprised when accessing simple-type fields such as c_int >>> and c_char_p. These simple types have getter and setter functions that >>> get called automatically whenever the type is used as a function >>> result, array index, or struct field. For example: >> OK - so lucky me - it "does not work" like earlier examples because I am >> referencing, generally, c_ulonglong - and these do not have a automatic >> getter/setter function? If not, how do I go about making one (preferably >> without needing to right a "method" for each and every _field_ in a class. > No, c_ulonglong is a simple (fundamental) type, which behaves just > like other simple types such as c_int or c_char_p. > > On platform's with a 64-bit long, c_ulonglong is an alias for c_ulong > (i.e. type "L"). On the other hand, on 64-bit Windows, c_ulonglong is > an unsigned quad word (i.e. type "Q"). > > All simple types subclass ctypes._SimpleCData and define a `_type_` > code such as "c" for c_char. On 64-bit Linux the simple types are > defined as follows: > > ?: c_bool > c: c_char > z: c_char_p > u: c_wchar > Z: c_wchar_p > P: c_void_p > > b: c_int8, c_byte > B: c_uint8, c_ubyte > h: c_int16, c_short > H: c_uint16, c_ushort > i: c_int32, c_int > I: c_uint32, c_uint > l: c_int64, c_long, c_longlong, c_ssize_t > L: c_uint64, c_ulong, c_ulonglong, c_size_t > > f: c_float > d: c_double > g: c_longdouble > > For convenience, simple types are automatically converted when > accessed as a function result, struct field, or array index. As I > mentioned previously, the only way around this behavior is to use a > subclass. A subclass doesn't get automatically converted because it > might define custom methods and attributes that need to be preserved. > >>> I'd alias the type instead of defining a struct, e.g. `time_t = >>> c_long`. This preserves automatic conversion of the simple type. >> The reason for the not using alias is because a) I was trying to be more >> inline with the text of the include file. I will have to check the sizeof >> c_long (i.e., sizeof(long) in both 32 and 64-bit modes > I don't follow. Currently you wrap a c_int or c_long in a struct when > you could just use those types directly. You have to check the pointer > size, but then it's up to you what assumptions to make about the > target platform's integer sizes. Currently on a 64-bit system you're > assuming a Unix-style LP64 data model [1], in which a long is 64-bit. > That should be fine if you're writing Unix-specific code that doesn't > care about LLP64 Windows systems. > > Wrapping the type in a struct provides more type safety, but if I > wanted that I'd create my own simple subclass. For example, assuming > time_t should be a signed integer that's the same size as a pointer: > > class time_t(ctypes._SimpleCData): > if ctypes.sizeof(ctypes.c_void_p) == ctypes.sizeof(ctypes.c_longlong): > _type_ = ctypes.c_longlong._type_ > elif ctypes.sizeof(ctypes.c_void_p) == ctypes.sizeof(ctypes.c_long): > _type_ = ctypes.c_long._type_ > elif ctypes.sizeof(ctypes.c_void_p) == ctypes.sizeof(ctypes.c_int): > _type_ = ctypes.c_int._type_ > # else raise AttributeError for missing _type_ > > [1]: https://en.wikipedia.org/wiki/64-bit_computing#64-bit_data_models From bev at bevhoward.com Wed Oct 5 17:19:38 2016 From: bev at bevhoward.com (Beverly Howard) Date: Wed, 5 Oct 2016 14:19:38 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: Thanks for the responses... appreciated. >> print("value value data data data", end="\r") << That makes sense, but it also seems to suggest that there is no other way to position the cursor prior to printing. For example, if that line is halfway down the screen, is there any way to position the cursor two lines above it? fwiw, I am open to creating functions if there are positioning options, both to meet the need, but, more importantly, to learn. Beverly Howard From rosuav at gmail.com Wed Oct 5 17:33:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Oct 2016 08:33:27 +1100 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: On Thu, Oct 6, 2016 at 8:19 AM, Beverly Howard wrote: > Thanks for the responses... appreciated. > >>> print("value value data data data", end="\r") << > > That makes sense, but it also seems to suggest that there is no other way to position the cursor prior to printing. > > For example, if that line is halfway down the screen, is there any way to position the cursor two lines above it? > > fwiw, I am open to creating functions if there are positioning options, both to meet the need, but, more importantly, to learn. What I showed you was the very simplest way of doing things. If you want to move the cursor around, I would recommend the 'curses' library: https://docs.python.org/3/library/curses.html https://docs.python.org/3/howto/curses.html There are other ways, too; with more info on what you're trying to accomplish, we could better advise. It might be that a GUI will serve you well, particularly if you have several pieces of information that you want to update. ChrisA From bev at bevhoward.com Wed Oct 5 17:33:43 2016 From: bev at bevhoward.com (Beverly Howard) Date: Wed, 5 Oct 2016 14:33:43 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: >> if it is a pi controlling the system I would tend towards controlling it from a web page via the network. to keep it updating would require AJAX style programming of the web page. << Thanks. I am interested in eventually doing that, but it seems that learning how to do it on a local console first would be to my advantage... especially during initial testing stages. fwiw, this project is to transfer (actually re-create) a basic program that I wrote for a Tandy Model 100 portable back in the early 80's to control ceramic kilns which get to over 2,000 degrees F. Worked perfectly until a few years ago when there were no longer any Tandy Model 100s available at any cost ;-) (In case anyone is interested in fossilized projects, see http://bevhoward.com/kiln/KilnCtrl.htm) Thanks again for the response and pointers, Beverly Howard From rosuav at gmail.com Wed Oct 5 17:34:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Oct 2016 08:34:22 +1100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: On Thu, Oct 6, 2016 at 6:35 AM, BartC wrote: > This is exactly why I persist with my own language implementations. My > current one is only 1/5000th the size but standard libraries are included!) Yes, it has all the standard library that *you* expect. Does it have what some random person on the internet will expect? Here are a few examples from the standard library of Python: * Unicode character names and categories * Date/time manipulation * Arbitrary-precision rationals * Statistical functions * zlib (zip/gzip) compression and decompression * sqlite3 databasing * BSD sockets * Various internet protocols (HTTP, FTP, SMTP, IMAP) And that's not even counting the stuff that, if it were proposed today, would probably be pushed to PyPI. Standard libraries tend to be fairly large because they're catering to a lot of people's needs. Mind you, two gig is still a lot, I'm not denying that; but you're saying that yours is half a meg, and I doubt that a full-featured language of today can exist inside a floppy disk. (Also, does your implementation compile directly to machine code, or does it depend on something else? It's easy to make something small if it depends on another compiler/interpreter.) ChrisA From bev at bevhoward.com Wed Oct 5 17:36:04 2016 From: bev at bevhoward.com (Beverly Howard) Date: Wed, 5 Oct 2016 14:36:04 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: >> I would recommend the 'curses' library: << Great! Thanks! I'll be back... I'm sure. Beverly Howard From bev at bevhoward.com Wed Oct 5 17:42:21 2016 From: bev at bevhoward.com (Beverly Howard) Date: Wed, 5 Oct 2016 14:42:21 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: <1be50977-5425-4092-a341-e7510b60891f@googlegroups.com> I _think_ I see what I need... window.move(new_y, new_x) Move cursor to (new_y, new_x) ...even if not, I now know it's there somewhere. Thanks for the specific links... they are very valuable. Beverly Howard From 007brendan at gmail.com Wed Oct 5 17:48:54 2016 From: 007brendan at gmail.com (Brendan Abel) Date: Wed, 5 Oct 2016 14:48:54 -0700 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: You should look into using PyQt or PySide. They are python bindings for the very popular Qt GUI framework. On Wed, Oct 5, 2016 at 2:33 PM, Beverly Howard wrote: > >> if it is a pi controlling the system I would tend towards controlling it > from a web page via the network. to keep it updating would require AJAX > style programming of the web page. << > > Thanks. I am interested in eventually doing that, but it seems that > learning how to do it on a local console first would be to my advantage... > especially during initial testing stages. > > fwiw, this project is to transfer (actually re-create) a basic program > that I wrote for a Tandy Model 100 portable back in the early 80's to > control ceramic kilns which get to over 2,000 degrees F. > > Worked perfectly until a few years ago when there were no longer any Tandy > Model 100s available at any cost ;-) (In case anyone is interested in > fossilized projects, see http://bevhoward.com/kiln/KilnCtrl.htm) > > Thanks again for the response and pointers, > Beverly Howard > > -- > https://mail.python.org/mailman/listinfo/python-list > From greg.ewing at canterbury.ac.nz Wed Oct 5 17:58:06 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 10:58:06 +1300 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: BartC wrote: > This is exactly why I persist with my own language implementations. My > current one is only 1/5000th the size but standard libraries are included!) You might like to try a different Haskell implementation, such as Hugs: https://wiki.haskell.org/Hugs According to the web page, it comes with most of the same libraries as ghc. It's based on an older version of the Haskell standard, though, so I'm not sure if it's up to date with modern practice regarding monads, etc. Hugs is what I used last time I messed around with functional programming, and it seemed to work very nicely. I don't remember exactly how big it was, but it certainly wasn't measured in gigabytes! That's just ridiculous. -- Greg From 4kir4.1i at gmail.com Wed Oct 5 18:12:50 2016 From: 4kir4.1i at gmail.com (Akira Li) Date: Thu, 06 Oct 2016 01:12:50 +0300 Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: <87twcqtpgd.fsf@gmail.com> Beverly Howard writes: >...snip... > A primary question would be, "What are options for building a display > that would update displayed values without scrolling?" To rewrite only the last character, you could use '\b': import os import itertools import time for c in map(str.encode, itertools.cycle('\-/|')): os.write(1, b'\b'+c) time.sleep(.3) To rewrite only the last line, you could use '\r': for i in range(1, 101): time.sleep(.1) print('\r{:4.0%}'.format(i/100), flush=True, end='') To control the full screen in the terminal e.g., to change the position, you could use *blessings* module [1]: from blessings import Terminal # $ pip install blessings line = 'Line in the middle of the terminal.' term = Terminal() with term.hidden_cursor(), term.fullscreen(): x = (term.width - len(line)) // 2 y = (term.height - 1) // 2 with term.location(x, y): print(term.bold_white_on_black(line)) with term.location(0, term.height - 1): input('press to exit..') For interactive command-line applications from a simple prompt to full screen terminal applications, you could use *prompt_toolkit* module [2]. > My first goal is to build a program that would interface with a > Raspberry Pi to control a heating process. > For flexibility, you could split your program into a server and a client that controls it (text, GUI, web). To control the Raspberry Pi from your mobile device, you could try Kivy [3] or just create something quick in Pythonista 3 on iOS [4]. To create a GUI on desktop, you could try Tkinter, Gtk, Qt frameworks [5,6]. For the web part on the server, you could try *flask*, *bottle* modules [7], to return json data to a client using http protocol: from bottle import route # $ pip install bottle @route('/data') def data(): return dict(data=[1,2,3]) On the (text/GUI) client in Python: import requests # $ pip install requests data = requests.get('https://your.server.example.com/data').json() Or in the browser using jQuery [8]: $.getJSON('https://your.server.example.com/data', function(data) { //use data here }); To make the interaction more responsive, you could use WebSocket protocol e.g., via Flask-SocketIO on the server [9]. [1]: https://pypi.python.org/pypi/blessings/ [2]: https://python-prompt-toolkit.readthedocs.io/ [3]: https://kivy.org [4]: http://omz-software.com/pythonista/ [5]: http://www.tkdocs.com/tutorial/onepage.html [6]: http://doc.qt.io/qt-4.8/gallery-gtk.html [7]: http://bottlepy.org/docs/stable/ [8]: http://api.jquery.com/jquery.getjson/ [9]: https://flask-socketio.readthedocs.io From eryksun at gmail.com Wed Oct 5 18:15:35 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 5 Oct 2016 22:15:35 +0000 Subject: working with ctypes and complex data structures In-Reply-To: <8ee86d1e-df95-6aad-f9cd-119b8bbafba9@felt.demon.nl> References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> <8ee86d1e-df95-6aad-f9cd-119b8bbafba9@felt.demon.nl> Message-ID: On Wed, Oct 5, 2016 at 9:03 PM, Michael Felt wrote: > >> +80 args = (1, "name", None), (2, "buff", None), (1, "size", >> 0), (1, "count", 1) > > error #1. paramater type 2 (the buffer might be where data is being put, but > for the call, the pointer is INPUT) An output parameter (type 2) has ctypes implicitly create the buffer and insert it in the argument list. It gets returned as the call's result. Here's a contrived example: class Buf(ctypes.Structure): _fields_ = (('value', ctypes.c_char * 100),) LP_Buf = ctypes.POINTER(Buf) proto = ctypes.CFUNCTYPE(ctypes.c_int, LP_Buf, ctypes.c_char_p, ctypes.c_int) flags = ((2, 'buf'), (1, 'fmt'), (1, 'arg', 42)) sprintf = proto(('sprintf', ctypes.CDLL(None)), flags) >>> r = sprintf(b'The answer is %d.') >>> r.value b'The answer is 42.' This works best when combined with an errcheck function. This lets you see the actual arguments that were passed in the call and the original result: def errcheck(result, func, args): print('result:', result) print('args:', args) return args sprintf.errcheck = errcheck >>> r = sprintf(b'The answer is %d.') result: 17 args: (<__main__.Buf object at 0x7fede840bb70>, b'The answer is %d.', 42) When the errcheck function returns "args", this tells ctypes to continue with its normal post-processing, so instead of the actual 17 result, it returns the "buf" output parameter: >>> r.value b'The answer is 42.' Notice that I declared the output parameter's type as a pointer type, but ctypes is smart enough to create a Buf instance for me instead of just a pointer. Also, because the argument type is a pointer type, its from_param method implicitly passes the Buf instance by reference. > class perfstat_xxx: > def init(self): > # AIX member in an archive > _perflib = ctypes.CDLL("libperfstat.a(shr_64.o)") > _fn_xxx = _perflib.xxx > # ALL AIX perfstat routines have 4 arguments: > # (name, buff, sizeof_buff, count) > # the buff is a pointer to where the information will be stored > _fn = CFUNCTYPE(c_int, c_void_p, POINTER(xxx_t), c_int, c_int) > _args = ((1, "name", None), (1, "buff", None), (1, "size", 0), > (1, "count", 1)) > _xxx = _fn(("xxx", _fn_xxx), _args) You should really do this work at the class or module level. Every time this init() method is called, you're needlessly incrementing the reference count on the "libperfstat.a(shr_64.o)" shared library and needlessly creating and instantiating the function pointer type. From bc at freeuk.com Wed Oct 5 18:45:24 2016 From: bc at freeuk.com (BartC) Date: Wed, 5 Oct 2016 23:45:24 +0100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: On 05/10/2016 22:34, Chris Angelico wrote: > On Thu, Oct 6, 2016 at 6:35 AM, BartC wrote: >> This is exactly why I persist with my own language implementations. My >> current one is only 1/5000th the size but standard libraries are included!) > > Yes, it has all the standard library that *you* expect. Does it have > what some random person on the internet will expect? Here are a few > examples from the standard library of Python: > > * Unicode character names and categories > * Date/time manipulation > * Arbitrary-precision rationals > * Statistical functions > * zlib (zip/gzip) compression and decompression > * sqlite3 databasing > * BSD sockets > * Various internet protocols (HTTP, FTP, SMTP, IMAP) > > And that's not even counting the stuff that, if it were proposed > today, would probably be pushed to PyPI. Standard libraries tend to be > fairly large because they're catering to a lot of people's needs. Mind > you, two gig is still a lot, I'm not denying that; but you're saying > that yours is half a meg, and I doubt that a full-featured language of > today can exist inside a floppy disk. No, it doesn't have all that. But believe me, even with all those things, and even if it worked, a 2GB language implementation is *MASSIVE*, not just a lot. Even Python, not exactly small, is only 2.5% the size (for 3.4 and presumably including all the libraries in your list). As to what it consists of, well it doesn't include any How-To videos which would account for much of it. But it does seem to have MSYS (ie. half a Unix system, 200MB); Mingw (a gcc C compiler, 350MB); docs (180MB); and a library (1300MB, but looking at it much of it seems to be useless junk). > (Also, does your implementation compile directly to machine code, or > does it depend on something else? It's easy to make something small if > it depends on another compiler/interpreter.) The language in question is interpreted. It depends on a compiler/interpreter which is currently a 300KB executable but could end up a bit bigger. The most basic libraries are another 60KB (which will probably end up inside the executable). The only dependencies are what is already present in an OS (msvcrt.dll for example). Small languages are perfectly viable: the JIT version of Lua, for example, is only about 225KB, and is very fast. If I wanted to send you program.lua, and you didn't have Lua, I only need to add luajit.exe and lus51.dll (for Windows). With my system, it would be program.q and r.exe, even for multi-module apps. What would I need for program.hs? (I already know of some solutions for program.py; that's not so straightforward either. But Python installations are more common.) -- Bartc From rosuav at gmail.com Wed Oct 5 19:27:38 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Oct 2016 10:27:38 +1100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: On Thu, Oct 6, 2016 at 9:45 AM, BartC wrote: > On 05/10/2016 22:34, Chris Angelico wrote: >> >> On Thu, Oct 6, 2016 at 6:35 AM, BartC wrote: >>> >>> This is exactly why I persist with my own language implementations. My >>> current one is only 1/5000th the size but standard libraries are >>> included!) >> >> >> Yes, it has all the standard library that *you* expect. Does it have >> what some random person on the internet will expect? Here are a few >> examples from the standard library of Python: >> >> * Unicode character names and categories >> * Date/time manipulation >> * Arbitrary-precision rationals >> * Statistical functions >> * zlib (zip/gzip) compression and decompression >> * sqlite3 databasing >> * BSD sockets >> * Various internet protocols (HTTP, FTP, SMTP, IMAP) >> >> And that's not even counting the stuff that, if it were proposed >> today, would probably be pushed to PyPI. Standard libraries tend to be >> fairly large because they're catering to a lot of people's needs. Mind >> you, two gig is still a lot, I'm not denying that; but you're saying >> that yours is half a meg, and I doubt that a full-featured language of >> today can exist inside a floppy disk. > > > No, it doesn't have all that. But believe me, even with all those things, > and even if it worked, a 2GB language implementation is *MASSIVE*, not just > a lot. Yeah - I agreed with you on that. 2GB+ is huge. But if you try to keep your language ubertiny, you're going to have a _lot_ of people complaining that your stdlib doesn't have . > Even Python, not exactly small, is only 2.5% the size (for 3.4 and > presumably including all the libraries in your list). Right. I didn't calculate the percentages, but my cpython/Lib directory is something like 40MB. You could shrink that down a ton by eliminating the tests and zipping the rest up, without losing primary functionality, but it'd still be a good solid bit - about 5MB. > As to what it consists of, well it doesn't include any How-To videos which > would account for much of it. But it does seem to have MSYS (ie. half a Unix > system, 200MB); Mingw (a gcc C compiler, 350MB); docs (180MB); and a library > (1300MB, but looking at it much of it seems to be useless junk). > >> (Also, does your implementation compile directly to machine code, or >> does it depend on something else? It's easy to make something small if >> it depends on another compiler/interpreter.) > > The language in question is interpreted. It depends on a > compiler/interpreter which is currently a 300KB executable but could end up > a bit bigger. The most basic libraries are another 60KB (which will probably > end up inside the executable). The only dependencies are what is already > present in an OS (msvcrt.dll for example). Ignoring OS-provided deps is perfectly fine. But if, for instance, your program emits C code, then it depends on a C compiler, which adds to the effective download. > Small languages are perfectly viable: the JIT version of Lua, for example, > is only about 225KB, and is very fast. > > If I wanted to send you program.lua, and you didn't have Lua, I only need to > add luajit.exe and lus51.dll (for Windows). With my system, it would be > program.q and r.exe, even for multi-module apps. See above about how restricted the stdib is. Lua, on its own, is not a full-featured language for writing general-purpose applications. It's designed to be embedded in something else, and it's great at that, but it doesn't have all the features of a modern apps language. You could write an Ook interpreter in a handful of bytes of code, but it's not what you would write an app in. Since you're talking about Windows, I grabbed file sizes for Windows installers; for Pike, the .msi file is a 24MB download, and Python clocks in variously at 18MB (32-bit 2.7.12) up to 28MB (64-bit 3.5.2 full installation). Those kinds of figures are pretty reasonable for full-featured language interpreters. You get enough for it to actually be usable as-is, but you're not getting a full C dev environment. ChrisA From bev at bevhoward.com Wed Oct 5 20:29:15 2016 From: bev at bevhoward.com (Beverly Howard) Date: Wed, 5 Oct 2016 17:29:15 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: Thanks guys... I'm getting overwhelmed with good info, so, I am going to disappear for a while and will comeback with questions when I get far enough to ask them. Again, thanks! Beverly Howard From 380162267qq at gmail.com Wed Oct 5 20:30:22 2016 From: 380162267qq at gmail.com (380162267qq at gmail.com) Date: Wed, 5 Oct 2016 17:30:22 -0700 (PDT) Subject: I am comfused about the name behavior of Python in recursion Message-ID: <63ae68fe-a429-4cbb-86e3-cdfc477b12ca@googlegroups.com> Google told me Python name is a label attaching to the object. But in this recursive function,the name 'a' will point to different number object. def rec(a): a+=1 if a<10: rec(a) print(a) rec(0) gives me 10....1 normally.Why it works? Because of the stack memory management? Thank you From torriem at gmail.com Wed Oct 5 21:54:11 2016 From: torriem at gmail.com (Michael Torrie) Date: Wed, 5 Oct 2016 19:54:11 -0600 Subject: Python and ssh for remote login In-Reply-To: References: Message-ID: On 10/05/2016 11:46 AM, Noah wrote: > Hello folk, > > I would like to use a python script to ssh into a server using a username > and password and perhaps ass port. > > Any ideas on how to script that. If paramiko doesn't fit your needs, traditionally this sort of work was done with the pexpect module for drying a TTY. There is a submodule of pexpect called pxssh for automating things. http://pexpect.readthedocs.io/en/stable/api/pxssh.html Note that pexpect uses your normal ssh binary. Paramiko is a complete implementation of the ssh protocol in python. Both modules are useful and fill certain needs. From ben+python at benfinney.id.au Wed Oct 5 22:48:09 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 06 Oct 2016 13:48:09 +1100 Subject: I am comfused about the name behavior of Python in recursion References: <63ae68fe-a429-4cbb-86e3-cdfc477b12ca@googlegroups.com> Message-ID: <85r37ui45y.fsf@benfinney.id.au> 380162267qq at gmail.com writes: > Google told me Python name is a label attaching to the object. Well, ?Google told me? has no necessary bearing on whether it's true :-) so that's not a good citation to give. If you need to know what Python terminology means, the Python documentation is better. so, I don't think ?name? has a technical meaning for Python. Rather, Python has an assignment operation, which binds a name (or some other reference) to an object. > But in this recursive function,the name 'a' will point to different number object. > > def rec(a): > a+=1 > if a<10: > rec(a) > print(a) > > rec(0) gives me 10....1 normally. Yes. What would you expect instead, and why? Maybe you think the *name* is passed around. That doesn't happen; in Python, names are not tractable in that way. By referencing the name ?a?, you immediately have the object referred to by that name. You don't have the name itself, and when you pass that object, the name you happened to use does not come with it. -- \ ?Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.? ?Richard P. Feynman, 1964 | Ben Finney From rustompmody at gmail.com Wed Oct 5 22:55:45 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 5 Oct 2016 19:55:45 -0700 (PDT) Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: <0b6a3a14-6683-477c-8bbc-7f14f7e27120@googlegroups.com> On Thursday, October 6, 2016 at 1:06:07 AM UTC+5:30, BartC wrote: > On 05/10/2016 14:43, Rustom Mody wrote: > > On Wednesday, October 5, 2016 at 7:05:02 PM UTC+5:30, BartC wrote: > > >> That's the one it can't find. Actually it can't seem to import any > >> standard libraries, so the implementation must be screwed up. > >> > >> Or is it because I downloaded the Minimal** rather Full version? The > >> minimal version is already 1.7GB! Can it really be that big and yet not > >> include such basic stuff? If so, what on earth is taking up all the > >> space! (The full version was 10-15% bigger.) > >> > >> (** > >> https://downloads.haskell.org/~platform/8.0.1/HaskellPlatform-8.0.1-minimal-x86_64-setup-a.exe) > > > > Minimal means only compiler: NO libraries except what the compiler needs to compile. > > Almost certainly wrong for a beginner?s testing the waters. > > > > You should probably download haskell-platform (full) > > (I've downloaded the full version. Now I have 25,000 files in 2200MB. > And it still doesn't work. We are now crossing the fuzzy OT boundary! Wont persist too much on list about this. Just a few comments: Many noobs (to programming/Python/computers/life!) appear out here with similar questions ? it doesn?t work!!! I would guess most of them feel similar to you. I would like to believe that most of them get out of their difficulty with the help of more experienced responders out here. Likewise Haskell. Go to haskell-cafe or haskell-beginners mailing-lists and you should cross this roadblock easily enough https://mail.haskell.org/mailman/listinfo/haskell-cafe https://mail.haskell.org/mailman/listinfo/beginners You can write to me also ? off-list! Just that I dont consider myself a haskell expert. Still less a haskell-on-windows expert For now I will only reiterate haskell-platform without System.* does not seem reasonable. You took a wrong turning somewhere. Is haskell large? Yes! Should it be? OT discussion, relevance of which asymptotically vanishes thanks to Moore?s law From greg.ewing at canterbury.ac.nz Thu Oct 6 00:57:15 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 17:57:15 +1300 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> Message-ID: Chris Angelico wrote: > Hence my > query about how variadic functions and automatic currying work - how > does it know whether to curry or run? Calling it "automatic" was probably a bad choice of words. I don't mean to imply that Haskell goes around currying things behind your back when you don't want it to. Rather, Haskell provides a very concise syntax for defining and using curried functions -- so concise that it's easier to use it than not, so it tends to get used by default even when currying isn't strictly necessary. The Haskell definition f x y z = x + y + z is equivalent to the Python definition f = lambda x: lambda y: lambda z: x + y + z To call it, you write f 1 2 3 Function application associates to the left, so that's equivalent to ((f 1) 2) 3) or in Python, f(1)(2)(3) If you don't want currying, there are various options. For example, you could define it to take a tuple: f (x, y, z) = x + y + z which is equivalent to the Python def f(args): x, y, z = args return x + y + z Then you would call it as f (1, 2, 3) i.e. construct a tuple of values and then pass it as the single argument. For a variable number of arguments, you could use a list instead of a tuple, but again you would need to explicitly construct the list when calling. I hope you can see from this that there's no confusion over "whether to curry or run". It all depends on how you define the function. -- Greg From greg.ewing at canterbury.ac.nz Thu Oct 6 01:03:37 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 18:03:37 +1300 Subject: I am comfused about the name behavior of Python in recursion In-Reply-To: <63ae68fe-a429-4cbb-86e3-cdfc477b12ca@googlegroups.com> References: <63ae68fe-a429-4cbb-86e3-cdfc477b12ca@googlegroups.com> Message-ID: 380162267qq at gmail.com wrote: > def rec(a): > a+=1 > if a<10: > rec(a) > print(a) > > rec(0) gives me 10....1 normally.Why it works? Because of the stack memory management? Yes. There isn't just one 'a' here, there's a different one each time rec is called. > > Thank you From gandalf at shopzeus.com Thu Oct 6 01:23:01 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 6 Oct 2016 07:23:01 +0200 Subject: Python 3.5 amd64 and win32service In-Reply-To: References: <9964f2f3-9e8e-4928-db88-bf350b5e102e@shopzeus.com> <81bcab97-3b9f-c8a2-f3da-3c06bb965831@shopzeus.com> <856e35f6-0545-602d-5fe8-c6a94843e37e@shopzeus.com> <6412c001-fb36-5ded-919c-58f7fe87d26f@shopzeus.com> Message-ID: > On Wed, Oct 5, 2016 at 7:43 AM, Nagy L?szl? Zsolt wrote: >> It did not help. I still get the same error message (service did not >> respond in time). > Can you run the service executable directly? From the command line > check `sc qc TestService`. Run the BINARY_PATH_NAME executable, e.g. > > for /f "tokens=1* delims=: " %i in ( > 'sc qc TestService ^| findstr BINARY_PATH_NAME') do @%j > > output: > > C - Python Service Manager > Options: > -register - register the EXE - this should generally not be necessary. > -debug servicename [parms] - debug the Python service. > > NOTE: You do not start the service using this program - start the > service using Control Panel, or 'net start service_name' All of the commands below were executed in cmd.exe running in elevated mode (as administrator): c:\Users\Laci\Python\Projects\gateway>service_test.py install Installing service TestService Service installed c:\Users\Laci\Python\Projects\gateway>sc qc TestService [SC] QueryServiceConfig SUCCESS SERVICE_NAME: TestService TYPE : 10 WIN32_OWN_PROCESS START_TYPE : 3 DEMAND_START ERROR_CONTROL : 1 NORMAL BINARY_PATH_NAME : "C:\Users\Laci\AppData\Local\Programs\Python\Python35\lib\site-packages\win32\PythonService.exe" LOAD_ORDER_GROUP : TAG : 0 DISPLAY_NAME : Test Service DEPENDENCIES : SERVICE_START_NAME : LocalSystem It seems to be the correct executable: c:\Users\Laci\Python\Projects\gateway>py -3 Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.executable 'C:\\Users\\Laci\\AppData\\Local\\Programs\\Python\\Python35\\python.exe' The service starts to fail in any way: * service_test.py start * net start testservice * from the services.msc with the start button The error message suggest that the service does not respont to the start request. The MWE I provided is so simple. By now, It should be obvious what is missing from it. :-( From gandalf at shopzeus.com Thu Oct 6 01:34:02 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Thu, 6 Oct 2016 07:34:02 +0200 Subject: Python 3.5 amd64 and win32service In-Reply-To: References: <9964f2f3-9e8e-4928-db88-bf350b5e102e@shopzeus.com> <81bcab97-3b9f-c8a2-f3da-3c06bb965831@shopzeus.com> <856e35f6-0545-602d-5fe8-c6a94843e37e@shopzeus.com> <6412c001-fb36-5ded-919c-58f7fe87d26f@shopzeus.com> Message-ID: <8275c3c2-5ba7-f4b7-969d-ac82c9d6ac2b@shopzeus.com> > > The MWE I provided is so simple. By now, It should be obvious what is > missing from it. :-( > > The problem is NOT with my code. I just installed a new virtual machine, and installed python3.5 amd64 + pypiwin32 on it. On that machine, the test service works perfectly! So it is with my machine. My machine has Python2.7 32bit, Python 3.5 32bit and Python 3.5 64bit installed. Can this be the problem? But how? What to do next? From rosuav at gmail.com Thu Oct 6 02:00:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 6 Oct 2016 17:00:57 +1100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 6, 2016 at 3:57 PM, Gregory Ewing wrote: > Chris Angelico wrote: >> >> Hence my >> query about how variadic functions and automatic currying work - how >> does it know whether to curry or run? > > > Calling it "automatic" was probably a bad choice of words. > I don't mean to imply that Haskell goes around currying > things behind your back when you don't want it to. No no, I never meant it like that. Automatic currying can be a huge boon, but also a stringent limitation. > The Haskell definition > > f x y z = x + y + z > > is equivalent to the Python definition > > f = lambda x: lambda y: lambda z: x + y + z > > To call it, you write > > f 1 2 3 Right, and that's fine as long as the function takes a fixed number of arguments. > For a variable number of arguments, you could use a list > instead of a tuple, but again you would need to explicitly > construct the list when calling. In other words, it still has to take a fixed number of arguments. Effectively, variadic functions are _impossible_, so instead you use argument packaging. That's not necessarily a bad thing. Not all language features need to be everywhere. Python has named arguments, and that's great; other high level languages generally do the same sort of thing with an options mapping. For example, here's how Pike lets you create a subprocess: // Process.Process(command_line, options); Process.Process("ffmpeg -i file.blk file.mkv", ([ "cwd": "/path/whatever", "uid": 1000, "gid": 1000, "timeout": 15, ])); command_line is either a string (which will be split according to shell rules) or an array of strings (already split); options is a mapping. Here's Python's version of that: # subprocess.run(args, *, **kwargs) subprocess.run("ffmpeg -i file.blk file.mkv", cwd="/path/whatever", # uid and gid not supported, but you get the idea timeout=15, ) args is either a string or a list of strings, and options are provided via keyword arguments. (Interestingly, though Python and Pike offer a lot of options as you create a subprocess, there are actually very few that are common. Weird.) This is not a problem. And if variadic functions are a problem to Haskell, so be it. They don't exist. That does at least answer my question about "how does Haskell handle variadic functions and currying" - and, importantly, it answers the question that comes up periodically of "why can't Python do automatic currying". > I hope you can see from this that there's no confusion over > "whether to curry or run". It all depends on how you define > the function. Yeah. No confusion because it does fit into the simple structure that I described as also working for Python. There's one other consideration. With Python functions, you often want to run a function for its side effects and ignore its return value. With automatic currying, you'd get no error doing that with insufficient arguments, you'd just be dropping the curried egg - erm, I mean, function - on the ground. It'd be a subtlety like this: print("Heading") print("=======") print print("First line of data") print("Second line of data") In Py2, this prints a blank line underneath the heading. In Py3, it merely evaluates the print function, then drops it. This can currently happen only with the zero argument case, but if you accept insufficient arguments to mean curry, it could happen any time. That's somewhat unideal. ChrisA From rustompmody at gmail.com Thu Oct 6 02:34:07 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 5 Oct 2016 23:34:07 -0700 (PDT) Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> Message-ID: <044fac6a-ba03-4442-9309-175b1faa38db@googlegroups.com> On Thursday, October 6, 2016 at 11:31:13 AM UTC+5:30, Chris Angelico wrote: > On Thu, Oct 6, 2016 at 3:57 PM, Gregory Ewing wrote: > > Chris Angelico wrote: > >> > >> Hence my > >> query about how variadic functions and automatic currying work - how > >> does it know whether to curry or run? > > > > > > Calling it "automatic" was probably a bad choice of words. > > I don't mean to imply that Haskell goes around currying > > things behind your back when you don't want it to. > > No no, I never meant it like that. Automatic currying can be a huge > boon, but also a stringent limitation. > > > The Haskell definition > > > > f x y z = x + y + z > > > > is equivalent to the Python definition > > > > f = lambda x: lambda y: lambda z: x + y + z > > > > To call it, you write > > > > f 1 2 3 > > Right, and that's fine as long as the function takes a fixed number of > arguments. > > > For a variable number of arguments, you could use a list > > instead of a tuple, but again you would need to explicitly > > construct the list when calling. > > In other words, it still has to take a fixed number of arguments. > Effectively, variadic functions are _impossible_, so instead you use > argument packaging. > > That's not necessarily a bad thing. Not all language features need to > be everywhere. Python has named arguments, and that's great; other > high level languages generally do the same sort of thing with an > options mapping. For example, here's how Pike lets you create a > subprocess: > > // Process.Process(command_line, options); > Process.Process("ffmpeg -i file.blk file.mkv", ([ > "cwd": "/path/whatever", > "uid": 1000, "gid": 1000, > "timeout": 15, > ])); > > command_line is either a string (which will be split according to > shell rules) or an array of strings (already split); options is a > mapping. Here's Python's version of that: > > # subprocess.run(args, *, **kwargs) > subprocess.run("ffmpeg -i file.blk file.mkv", > cwd="/path/whatever", > # uid and gid not supported, but you get the idea > timeout=15, > ) > > args is either a string or a list of strings, and options are provided > via keyword arguments. (Interestingly, though Python and Pike offer a > lot of options as you create a subprocess, there are actually very few > that are common. Weird.) This is not a problem. And if variadic > functions are a problem to Haskell, so be it. They don't exist. That > does at least answer my question about "how does Haskell handle > variadic functions and currying" - and, importantly, it answers the > question that comes up periodically of "why can't Python do automatic > currying". > > > I hope you can see from this that there's no confusion over > > "whether to curry or run". It all depends on how you define > > the function. > > Yeah. No confusion because it does fit into the simple structure that > I described as also working for Python. > > There's one other consideration. With Python functions, you often want > to run a function for its side effects and ignore its return value. > With automatic currying, you'd get no error doing that with > insufficient arguments, you'd just be dropping the curried egg - erm, > I mean, function - on the ground. It'd be a subtlety like this: > > print("Heading") > print("=======") > print > print("First line of data") > print("Second line of data") > > In Py2, this prints a blank line underneath the heading. In Py3, it > merely evaluates the print function, then drops it. This can currently > happen only with the zero argument case, but if you accept > insufficient arguments to mean curry, it could happen any time. That's > somewhat unideal. > > ChrisA It is fair to say that Haskell's variadic function support is poorer than C's [Collecting into a list fails for different types Collecting into a tuple fails because (t1,t2) and (t1,t2,t3) are incompatible types. There's some heavy artillery called type families round this whose messiness puts haskell squarely into C++ camp] Python's variadicity is better than any other language (I know) (Common Lisp??) [Except for mutable-default gotcha] From nomail at com.invalid Thu Oct 6 02:53:53 2016 From: nomail at com.invalid (ast) Date: Thu, 6 Oct 2016 08:53:53 +0200 Subject: static, class and instance methods Message-ID: <57f5f503$0$24766$426a74cc@news.free.fr> Hello, In a class there are three possible types of methods, the static methods, the class methods and the instance methods * Class methods are decorated, eg: @classmethod def func(cls, a, b): ... I read that the decorator tranforms 'func' as a descriptor, and when this descriptor is read, it provided a new function 'func' where the 1st parameter cls is already filled with the class. That's ok for me. * For instance methods, there is no decorator: def funct2(self, a, b): ... self is automatically filled with the instance when we call funct2 from an instance and not filled if funct2 is called from a class. But there is no decorator, why ? Is python doing the conversion of funct2 to a descriptor itself, behind the scene ? * static methods are decorated too @staticmethod def funct3(a, b): ... The 1st argument is not supposed to be automatically filled So what the decorator used for ? Just to distinguish funct3 from an instance method ? regards From dieter at handshake.de Thu Oct 6 03:05:22 2016 From: dieter at handshake.de (dieter) Date: Thu, 06 Oct 2016 09:05:22 +0200 Subject: 'str' object has no attribute 'intersection' and unhashable set References: <8e5182a1-1530-4757-a4fc-a2ff3c77e315@googlegroups.com> <87r37vmegk.fsf@handshake.de> Message-ID: <87lgy2ot3h.fsf@handshake.de> meInvent bbird writes: > ... > is there any algorithm tutorials or books or external library that > can have a better result for finding repeated lines as group in grouping > application I do not yet understand your problem (and still am not ready to look at your code). When I need grouping for elements based on some feature ("F(elem)") I usually use: groups = {} for elem in elements: feature = F(elem) fg = groups.get(feature) if fg is None: groups[feature] = fg = [] fg.append(elem) After this look, the dict "groups" maps features to the list of elements with the corresponding feature. For this to work, the features must be hashable. If they are not naturally, I derive a consistent (same feature leads to same representation) hashable representation for them and use that for the grouping. From greg.ewing at canterbury.ac.nz Thu Oct 6 04:15:17 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 21:15:17 +1300 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> Message-ID: Chris Angelico wrote: > There's one other consideration. With Python functions, you often want > to run a function for its side effects and ignore its return value. You can't just ignore a return value in Haskell. The return value is always going *somewhere*. If you leave off an argument, the result is going to be a function instead of whatever type the function was meant to return. Since Haskell is statically-typed, this will give you a type error at some point. > print("Heading") > print("=======") > print You're talking about I/O here, which complicates things somewhat. Essentially, the equivalent of 'print' is going to be a function that takes the value to be printed and returns another function of a specific type. If it's not the right type, it won't fit into the machinery that executes I/O operations. So leaving the argument off the print would lead to a type error, because you'd be trying to plug the print function itself into the I/O machine, and it's not the right type. -- Greg From greg.ewing at canterbury.ac.nz Thu Oct 6 04:23:53 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 21:23:53 +1300 Subject: Assignment versus binding In-Reply-To: <044fac6a-ba03-4442-9309-175b1faa38db@googlegroups.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <044fac6a-ba03-4442-9309-175b1faa38db@googlegroups.com> Message-ID: Rustom Mody wrote: > It is fair to say that Haskell's variadic function support is poorer than C's > [Collecting into a list fails for different types If you want an arbitrary number of args of unrelated types, you would need to define a wrapper type that can encapsulate all the ones you're interested in. -- Greg From greg.ewing at canterbury.ac.nz Thu Oct 6 04:33:11 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 21:33:11 +1300 Subject: static, class and instance methods (Reposting On Python-List Prohibited) In-Reply-To: References: <57f5f503$0$24766$426a74cc@news.free.fr> Message-ID: Lawrence D?Oliveiro wrote: > > Every function is already a descriptor. Which you can see with a simple experiment: >>> def f(self): ... print("self =", self) ... >>> g = f.__get__(17, None) >>> g >>> g() self = 17 -- Greg From greg.ewing at canterbury.ac.nz Thu Oct 6 04:46:51 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 21:46:51 +1300 Subject: Assignment versus binding In-Reply-To: <87h98rxnt5.fsf@nightsong.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <87h98rxnt5.fsf@nightsong.com> Message-ID: Paul Rubin wrote: > It's useful to write some Python things in monadic style, but monads > make the most sense as type operators, which don't map onto Python that > well. There probably isn't much point in using the monadic style in Python, since the main reason for it is to express stateful processes in a functional framework, and Python already has more direct ways of expressing stateful processes. Also, if you translate monadic code directly from Haskell to Python, you get something which is... not all that pleasant to use. You'll see that when I've posted my Python-based monads essay... -- Greg From nomail at com.invalid Thu Oct 6 05:03:15 2016 From: nomail at com.invalid (ast) Date: Thu, 6 Oct 2016 11:03:15 +0200 Subject: static, class and instance methods (Reposting On Python-List Prohibited) In-Reply-To: References: <57f5f503$0$24766$426a74cc@news.free.fr> Message-ID: <57f61356$0$19739$426a74cc@news.free.fr> "Lawrence D?Oliveiro" a ?crit dans le message de news:f5314bdd-a98f-4a16-b546-bd8efe4dd568 at googlegroups.com... > On Thursday, October 6, 2016 at 7:54:08 PM UTC+13, ast wrote: >> But there is no decorator, why ? Is python doing the conversion >> of funct2 to a descriptor itself, behind the scene ? > > Every function is already a descriptor. So you get the instance-method behaviour automatically. > >> * static methods are decorated too > > This is to turn off the instance-method behaviour. I do not understand your answer. Consider this function: def add(a, b): return a+b You say that a function is always stored as a descriptor object, so when I execute sum = f(4, 6) from which class it is supposed to come from ? From greg.ewing at canterbury.ac.nz Thu Oct 6 05:10:02 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 06 Oct 2016 22:10:02 +1300 Subject: Assignment versus binding In-Reply-To: <871szv15a7.fsf@bsb.me.uk> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ee5bb4$0$1617$c3e8da3$5496439d@news.astraweb.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: Ben Bacarisse wrote: > import Control.Concurrent > > spam io = do x <- io; > print x; > print (x+1) > > main = spam (do threadDelay (2*10^6); return 1) > > It matches the Python in that the delay happens once. To get the > behaviour being hinted at (two delays) you need to re-bind the IO > action: > > spam io = do x <- io; > print x; > x <- io; > print (x+1) Yep. What's actually happening here is that spam is being passed a function, and that function takes an extra implicit argument representing "the state of the world". The 'x <- io' part is syntactic sugar for something that calls io with the current state of the world, and then evaluates all the stuff after the semicolon with the new state of the world. So the first version only evaluates io once, and the second one evaluates it twice with different arguments (the states of the world at two different times). (At least *conceptually* that's what happens. The Haskell interpreter probably doesn't actually carry the state of the whole world around inside it. :-) -- Greg From fiedzia at gmail.com Thu Oct 6 05:21:54 2016 From: fiedzia at gmail.com (Maciej Dziardziel) Date: Thu, 6 Oct 2016 02:21:54 -0700 (PDT) Subject: any json data validation library recommendation? (Reposting on Python-Lists Prohitibited) In-Reply-To: <736cd5ae-02f5-40e3-9f47-0fb838251787@googlegroups.com> References: <736cd5ae-02f5-40e3-9f47-0fb838251787@googlegroups.com> Message-ID: <4ed12081-93f8-4dda-a826-253c5c18bf0f@googlegroups.com> On Thursday, October 6, 2016 at 7:20:58 AM UTC+1, Lawrence D?Oliveiro wrote: > What action are users supposed to take on such errors, other than include them in a bug report? By users I mean API users (developers). Most common action would be to add a missing field, correct typo or change value type. Failing validation is usually user error, and rarely leads to reporting a bug. From __peter__ at web.de Thu Oct 6 05:35:31 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 06 Oct 2016 11:35:31 +0200 Subject: static, class and instance methods (Reposting On Python-List Prohibited) References: <57f5f503$0$24766$426a74cc@news.free.fr> <57f61356$0$19739$426a74cc@news.free.fr> Message-ID: ast wrote: > > "Lawrence D?Oliveiro" a ?crit dans le message de > news:f5314bdd-a98f-4a16-b546-bd8efe4dd568 at googlegroups.com... >> On Thursday, October 6, 2016 at 7:54:08 PM UTC+13, ast wrote: >>> But there is no decorator, why ? Is python doing the conversion >>> of funct2 to a descriptor itself, behind the scene ? >> >> Every function is already a descriptor. So you get the instance-method >> behaviour automatically. >> >>> * static methods are decorated too >> >> This is to turn off the instance-method behaviour. > > > I do not understand your answer. > Consider this function: > > def add(a, b): > return a+b > > You say that a function is always stored as > a descriptor object, so when I execute > > sum = f(4, 6) > > from which class it is supposed to come from ? It doesn't matter. You can invoke it in the usual way >>> def add(a, b): return a + b ... >>> add(2, 3) 5 and even put it into an instance: >>> class A: pass ... >>> a = A() >>> a.foo = add >>> a.foo(3, 4) 7 Only when you put it into a class the first argument will be the instance >>> A.bar = add >>> a.bar(5, 6) Traceback (most recent call last): File "", line 1, in TypeError: add() takes 2 positional arguments but 3 were given because under the hood add.__get__() is invoked. Doing it manually: >>> add.__get__(a)(5, 6) Traceback (most recent call last): File "", line 1, in TypeError: add() takes 2 positional arguments but 3 were given Unlike Python functions those implemented in C don't support the descriptor protocol out of the box, so >>> A.baz = sum >>> a.baz([1,2,3]) 6 does not pass the instance a as an argument. From michael at felt.demon.nl Thu Oct 6 06:15:07 2016 From: michael at felt.demon.nl (Michael Felt) Date: Thu, 6 Oct 2016 12:15:07 +0200 Subject: working with ctypes and complex data structures In-Reply-To: References: <78e0b87d-830a-ee34-2c45-9f7d8662e0ca@felt.demon.nl> <0b0c6443-87fb-44cc-8fae-203d38bd5f58@felt.demon.nl> Message-ID: On 04-Oct-16 04:48, eryk sun wrote: > On Mon, Oct 3, 2016 at 9:27 PM, Michael Felt wrote: >> int perfstat_subsystem_total( >> perfstat_id_t *name, >> perfstat_subsystem_total_t *userbuff, >> int sizeof_struct, >> int desired_number); >> ... >> +79 class cpu_total: >> +80 def __init__(self): >> +81 __perfstat__ = CDLL("libperfstat.a(shr_64.o)") Thanks for the model below - a lot to think about. Some was expected, some is a gift! Python is "a lot to learn". This helps a great deal. > Move loading the library and defining function prototypes to either > the module or class body. Also, don't use dunder names. The convention > for a private attribute is a single underscore. For type safety, > define each function's argtypes (and restype where required, the > default is c_int). For a more Pythonic interface, define a ctypes > errcheck function that encapsulates raising an appropriate exception > when a C call fails. For example: > > import ctypes > > # If the following types are used generally, define them at > # the module level, else define them in the class namespace > # that uses them. > > perfstat = ctypes.CDLL("libperfstat.a(shr_64.o)") > > class perfstat_id_t(ctypes.Structure): > pass > > IDENTIFIER_LENGTH = 64 > > class time64_t(ctypes._SimpleCData): > _type_ = ctypes.c_int64._type_ > > time_t = time64_t > > class perfstat_cpu_total_t(ctypes.Structure): > _fields_ = (("ncpus", ctypes.c_int), > ("ncpus_cfg", ctypes.c_int), > ("description", ctypes.c_char * IDENTIFIER_LENGTH), > ("buffer1", ctypes.c_ulonglong * 15), > ("lbolt", time_t), > ("loadavg", ctypes.c_ulonglong * 3), > ("buffer2", ctypes.c_ulonglong * 29), > ("ncpus_high", ctypes.c_int), > ("puser", ctypes.c_ulonglong), > ("psys", ctypes.c_ulonglong), > ("pidle", ctypes.c_ulonglong), > ("pwait", ctypes.c_ulonglong), > ("buffer3", ctypes.c_ulonglong * 12)) > > def _perfstat_errcheck(result, func, args): > if result == -1: > # get error info and > raise SomeException() > return args > > class CPUTotal(object): > # These may be defined here or just referenced here. > _perfstat = perfstat > _perfstat_id_t = perfstat_id_t > _perfstat_cpu_total_t = perfstat_cpu_total_t > > _cpu_total = _perfstat.perfstat_cpu_total > _cpu_total.errcheck = _perfstat_errcheck > _cpu_total.argtypes = ( > ctypes.POINTER(_perfstat_id_t), > ctypes.POINTER(_perfstat_cpu_total_t), > ctypes.c_int, ctypes.c_int) > > def __init__(self): > self._tcpu = self._perfstat_cpu_total_t() > self._cpu_total(None, > ctypes.byref(self._tcpu), > ctypes.sizeof(self._tcpu), 1) From alister.ware at ntlworld.com Thu Oct 6 06:31:50 2016 From: alister.ware at ntlworld.com (alister) Date: Thu, 06 Oct 2016 10:31:50 GMT Subject: I am comfused about the name behavior of Python in recursion References: <63ae68fe-a429-4cbb-86e3-cdfc477b12ca@googlegroups.com> Message-ID: On Wed, 05 Oct 2016 17:30:22 -0700, 380162267qq wrote: > Google told me Python name is a label attaching to the object. > But in this recursive function,the name 'a' will point to different > number object. > > def rec(a): > a+=1 if a<10: > rec(a) > print(a) > > rec(0) gives me 10....1 normally.Why it works? Because of the stack > memory management? > > Thank you google local & global variables (this relates to many languages not just python) -- Excerpts From The First Annual Nerd Bowl (#1) JOHN SPLADDEN: Hi, and welcome to the first annual Nerd Bowl in sunny Silicon Valley. BRYANT DUMBELL: We're coming to you live from the Transmeta Dome to watch the battle between the North Carolina Mad Hatters and the Michigan Portalbacks as they compete for the coveted Linus Torvalds Trophy. SPLADDEN: This is shaping up to be one hell of a match. The Mad Hatters -- sponsored by Linux distributor Red Hat -- have been on fire the past month. But the Andover.Net sponsored Michigan Portalbacks are on a tear as well, thanks in part to the stellar performance of Rob "Taco Boy" Malda. DUMBELL: Taco Boy is quite a star, John. Last week at the Kernelbowl he blew away the Transmeta Secret Agents when he scored 51 points singlehandedly in the Flying CompactDiscus round. SPLADDEN: But then Mad Hatter's Alan Cox was voted this season's Most Valuable Hacker in the Eastern Division. So, this game is going to be quite a show. From lanuradha at gmail.com Thu Oct 6 06:33:51 2016 From: lanuradha at gmail.com (Anuradha Laxminarayan) Date: Thu, 6 Oct 2016 03:33:51 -0700 (PDT) Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <87h98rxnt5.fsf@nightsong.com> Message-ID: <4166579c-e83d-43c6-9911-1cb29e11f79c@googlegroups.com> On Thursday, 6 October 2016 14:17:06 UTC+5:30, Gregory Ewing wrote: > Paul Rubin wrote: > > It's useful to write some Python things in monadic style, but monads > > make the most sense as type operators, which don't map onto Python that > > well. > > There probably isn't much point in using the monadic style > in Python, since the main reason for it is to express > stateful processes in a functional framework, and Python > already has more direct ways of expressing stateful processes. Agreed > > Also, if you translate monadic code directly from Haskell > to Python, you get something which is... not all that > pleasant to use. Of course ;-) ?Pleasant to use? is not my primary need here. > You'll see that when I've posted my Python-based monads essay... The reason I would like this essay is that I want to teach the notion of first-classing of computation, if possible without the learning curve of Haskell syntax, type-system etc There is a well-known joke in the Haskell space that ?Haskell is an excellent imperative language? I guess this is because monads can first-class arbitrary imperative (computational) notions. Since in python the two worlds of purely functional and imperative meet being able to see this in python would be neat! - Anuradha From alister.ware at ntlworld.com Thu Oct 6 06:36:14 2016 From: alister.ware at ntlworld.com (alister) Date: Thu, 06 Oct 2016 10:36:14 GMT Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: On Wed, 05 Oct 2016 14:33:43 -0700, Beverly Howard wrote: >>> if it is a pi controlling the system I would tend towards controlling >>> it > from a web page via the network. to keep it updating would require AJAX > style programming of the web page. << > > Thanks. I am interested in eventually doing that, but it seems that > learning how to do it on a local console first would be to my > advantage... especially during initial testing stages. > > fwiw, this project is to transfer (actually re-create) a basic program > that I wrote for a Tandy Model 100 portable back in the early 80's to > control ceramic kilns which get to over 2,000 degrees F. > > Worked perfectly until a few years ago when there were no longer any > Tandy Model 100s available at any cost ;-) (In case anyone is > interested in fossilized projects, see > http://bevhoward.com/kiln/KilnCtrl.htm) > > Thanks again for the response and pointers, > Beverly Howard you may also want to look at the pi newsgroup as well comp.sys.raspberry- pi for any non python queries (& I am sure they would be interested in the python ones as well) -- Two brothers, Mort and Bill, like to sail. While Bill has a great deal of experience, he certainly isn't the rigger Mort is. From eryksun at gmail.com Thu Oct 6 06:48:22 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 6 Oct 2016 10:48:22 +0000 Subject: Python 3.5 amd64 and win32service In-Reply-To: References: <9964f2f3-9e8e-4928-db88-bf350b5e102e@shopzeus.com> <81bcab97-3b9f-c8a2-f3da-3c06bb965831@shopzeus.com> <856e35f6-0545-602d-5fe8-c6a94843e37e@shopzeus.com> <6412c001-fb36-5ded-919c-58f7fe87d26f@shopzeus.com> Message-ID: On Thu, Oct 6, 2016 at 5:23 AM, Nagy L?szl? Zsolt wrote: > "C:\Users\Laci\AppData\Local\Programs\Python\Python35\lib\site-packages\win32\PythonService.exe" I wanted you to run the above executable, not python.exe. If it fails you'll get more information about why it's failing when run directly then when the service controller runs it. Since you're using a per-user installation of Python 3.5, possibly it can't find python35.dll. From bc at freeuk.com Thu Oct 6 07:03:37 2016 From: bc at freeuk.com (BartC) Date: Thu, 6 Oct 2016 12:03:37 +0100 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: On 05/10/2016 23:12, Akira Li wrote: > Beverly Howard writes: > >> ...snip... >> A primary question would be, "What are options for building a display >> that would update displayed values without scrolling?" > > To rewrite only the last character, you could use '\b': > > import os > import itertools > import time > for c in map(str.encode, itertools.cycle('\-/|')): > os.write(1, b'\b'+c) > time.sleep(.3) > > To rewrite only the last line, you could use '\r': > > for i in range(1, 101): > time.sleep(.1) > print('\r{:4.0%}'.format(i/100), flush=True, end='') > > To control the full screen in the terminal e.g., to change the position, > you could use *blessings* module [1]: > > from blessings import Terminal # $ pip install blessings > > line = 'Line in the middle of the terminal.' > term = Terminal() > with term.hidden_cursor(), term.fullscreen(): > x = (term.width - len(line)) // 2 > y = (term.height - 1) // 2 > with term.location(x, y): > print(term.bold_white_on_black(line)) > > with term.location(0, term.height - 1): > input('press to exit..') > > For interactive command-line applications from a simple prompt to full > screen terminal applications, you could use *prompt_toolkit* module [2]. > >> My first goal is to build a program that would interface with a >> Raspberry Pi to control a heating process. >> > > For flexibility, you could split your program into a server and a client > that controls it (text, GUI, web). All this advice seems to be getting out of hand, with suggestions of 'curses' and 'blessings' and using GUI. I've tried 'ncurses' elsewhere and it was over the top for what I wanted to do. The OP wants to runs on Pi which I think runs Linux. So all they are asking is, is there a way of randomly positioning the cursor within the terminal window so that the next output is at that position. Something like an escape sequence. Terminal screens have been around a long time, you'd think someone would have had such a requirement before! I'd quite like to know too. However I've just tried a test sequence ("[P1d" to move the cursor to row 1) and it didn't work. If there's reason why something so basic won't work (hence the need for curses etc) then that would be useful to know too. (And how does curses manage it?!) -- Bartc From steve+python at pearwood.info Thu Oct 6 07:31:23 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 06 Oct 2016 22:31:23 +1100 Subject: static, class and instance methods References: <57f5f503$0$24766$426a74cc@news.free.fr> Message-ID: <57f6360d$0$1609$c3e8da3$5496439d@news.astraweb.com> On Thu, 6 Oct 2016 05:53 pm, ast wrote: [...] > * For instance methods, there is no decorator: > > def funct2(self, a, b): > ... > > self is automatically filled with the instance when we call > funct2 from an instance and not filled if funct2 is called from > a class. > But there is no decorator, why ? Is python doing the conversion > of funct2 to a descriptor itself, behind the scene ? Functions don't need to be decorated because they are already descriptors. Descriptors have __get__, __set__ or __delete__ methods. In the case of methods, it is __get__ which does the work. Normally this happens automatically, but we can do it by hand: py> class X(object): ... def func(self): ... pass ... py> x = X() py> vars(X)['func'] py> vars(X)['func'].__get__(x, X) > py> x.func > > * static methods are decorated too > > @staticmethod > def funct3(a, b): > ... > > The 1st argument is not supposed to be automatically filled > So what the decorator used for ? > Just to distinguish funct3 from an instance method ? Correct. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Thu Oct 6 07:32:54 2016 From: bc at freeuk.com (BartC) Date: Thu, 6 Oct 2016 12:32:54 +0100 Subject: Assignment versus binding In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <871szv15a7.fsf@bsb.me.uk> Message-ID: On 06/10/2016 00:27, Chris Angelico wrote: > On Thu, Oct 6, 2016 at 9:45 AM, BartC wrote: >> Small languages are perfectly viable: the JIT version of Lua, for example, >> is only about 225KB, and is very fast. >> >> If I wanted to send you program.lua, and you didn't have Lua, I only need to >> add luajit.exe and lus51.dll (for Windows). With my system, it would be >> program.q and r.exe, even for multi-module apps. > > See above about how restricted the stdib is. Lua, on its own, is not a > full-featured language for writing general-purpose applications. It's > designed to be embedded in something else, and it's great at that, but > it doesn't have all the features of a modern apps language. Some languages don't come with their own libraries but are expected to hook into third party ones. OpenGL for example, which on Windows existed as a language-neutral set of DLL files (iirc). A language might provide 'bindings' as a convenience, or perhaps an easy-to-use set of wrappers. It's just a lower-level way of doing things compared to what you might be used to. You could > write an Ook interpreter in a handful of bytes of code, but it's not > what you would write an app in. Since you're talking about Windows, I > grabbed file sizes for Windows installers; for Pike, the .msi file is > a 24MB download, and Python clocks in variously at 18MB (32-bit > 2.7.12) up to 28MB (64-bit 3.5.2 full installation). Those kinds of > figures are pretty reasonable for full-featured language interpreters. > You get enough for it to actually be usable as-is, but you're not > getting a full C dev environment. And those are compressed download sizes. I started creating interpreters when they had to run within a tiny corner of 640KB main memory, and they were a tremendously useful way of helping implement applications by using scripts. (Scripts also acted as overlays resident on floppies and loaded as needed.) It also meant users could instantly create and run their own scripts (without using huge, sprawling and, in 1980s, incredibly slow compilers). You can't just dismiss something as useless just because it's relatively small. (I've been using my interpreters to run compilers. What specialist libraries does a compiler need other than file i/o? None!) -- Bartc From frank at chagford.com Thu Oct 6 08:09:56 2016 From: frank at chagford.com (Frank Millman) Date: Thu, 6 Oct 2016 14:09:56 +0200 Subject: Is it possible to use 'groupby' asynchronously? Message-ID: Hi all I have used itertools.groupby before, and I love it. I used it to process a csv file and 'break' on change of a particular field. It worked very well. Now I want to use it to process a database table. I can select the rows in the desired sequence with no problem. However, I am using asyncio, so I am reading the rows asynchronously. My 'reader' class has __aiter__() and __anext__() defined. If I pass the reader to groupby, I get the error message 'object is not iterable'. Before I spend hours trying to figure it out, can anyone confirm if this is doable at all, or is groupby not designed for this. Thanks Frank Millman From __peter__ at web.de Thu Oct 6 08:38:24 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 06 Oct 2016 14:38:24 +0200 Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: BartC wrote: > On 05/10/2016 23:12, Akira Li wrote: >> Beverly Howard writes: >> >>> ...snip... >>> A primary question would be, "What are options for building a display >>> that would update displayed values without scrolling?" >> >> To rewrite only the last character, you could use '\b': >> >> import os >> import itertools >> import time >> for c in map(str.encode, itertools.cycle('\-/|')): >> os.write(1, b'\b'+c) >> time.sleep(.3) >> >> To rewrite only the last line, you could use '\r': >> >> for i in range(1, 101): >> time.sleep(.1) >> print('\r{:4.0%}'.format(i/100), flush=True, end='') >> >> To control the full screen in the terminal e.g., to change the position, >> you could use *blessings* module [1]: >> >> from blessings import Terminal # $ pip install blessings >> >> line = 'Line in the middle of the terminal.' >> term = Terminal() >> with term.hidden_cursor(), term.fullscreen(): >> x = (term.width - len(line)) // 2 >> y = (term.height - 1) // 2 >> with term.location(x, y): >> print(term.bold_white_on_black(line)) >> >> with term.location(0, term.height - 1): >> input('press to exit..') >> >> For interactive command-line applications from a simple prompt to full >> screen terminal applications, you could use *prompt_toolkit* module [2]. >> >>> My first goal is to build a program that would interface with a >>> Raspberry Pi to control a heating process. >>> >> >> For flexibility, you could split your program into a server and a client >> that controls it (text, GUI, web). > > All this advice seems to be getting out of hand, with suggestions of > 'curses' and 'blessings' and using GUI. I've tried 'ncurses' elsewhere > and it was over the top for what I wanted to do. > > The OP wants to runs on Pi which I think runs Linux. > > So all they are asking is, is there a way of randomly positioning the > cursor within the terminal window so that the next output is at that > position. Something like an escape sequence. Terminal screens have been > around a long time, you'd think someone would have had such a > requirement before! > > I'd quite like to know too. However I've just tried a test sequence > ("[P1d" to move the cursor to row 1) and it didn't work. If there's > reason why something so basic won't work (hence the need for curses etc) > then that would be useful to know too. (And how does curses manage it?!) Perhaps you picked the wrong escape sequence? I tried ;H taken from , and it seems to work (the test system is not a Pi though): $ python3 Python 3.4.3 (default, Sep 14 2016, 12:36:27) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> for i in range(20, 10, -1): ... print("\033[{0};5HHello from line {0}".format(i), end="") ... Hello from line 11>>> Hello from line 12 Hello from line 13 Hello from line 14 Hello from line 15 Hello from line 16 Hello from line 17 Hello from line 18 Hello from line 19 Hello from line 20 From mail at timgolden.me.uk Thu Oct 6 08:45:41 2016 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 6 Oct 2016 13:45:41 +0100 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: <3c263d7d-5b79-190d-9d1e-363b9a347e0c@timgolden.me.uk> On 06/10/2016 13:38, Peter Otten wrote: > BartC wrote: >> All this advice seems to be getting out of hand, with suggestions of >> 'curses' and 'blessings' and using GUI. I've tried 'ncurses' elsewhere >> and it was over the top for what I wanted to do. >> >> The OP wants to runs on Pi which I think runs Linux. >> >> So all they are asking is, is there a way of randomly positioning the >> cursor within the terminal window so that the next output is at that >> position. Something like an escape sequence. Terminal screens have been >> around a long time, you'd think someone would have had such a >> requirement before! >> >> I'd quite like to know too. However I've just tried a test sequence >> ("[P1d" to move the cursor to row 1) and it didn't work. If there's >> reason why something so basic won't work (hence the need for curses etc) >> then that would be useful to know too. (And how does curses manage it?!) > > Perhaps you picked the wrong escape sequence? I tried ;H > taken from , and it > seems to work (the test system is not a Pi though): > For lightweight stuff like this, colorama is useful (basically wraps the ANSI sequences): https://pypi.python.org/pypi/colorama TJG From jeanmichel at sequans.com Thu Oct 6 08:53:25 2016 From: jeanmichel at sequans.com (jmp) Date: Thu, 06 Oct 2016 14:53:25 +0200 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: On 10/05/2016 11:33 PM, Chris Angelico wrote: > On Thu, Oct 6, 2016 at 8:19 AM, Beverly Howard wrote: >> Thanks for the responses... appreciated. >> >>>> print("value value data data data", end="\r") << >> >> That makes sense, but it also seems to suggest that there is no other way to position the cursor prior to printing. >> >> For example, if that line is halfway down the screen, is there any way to position the cursor two lines above it? >> >> fwiw, I am open to creating functions if there are positioning options, both to meet the need, but, more importantly, to learn. > > What I showed you was the very simplest way of doing things. If you > want to move the cursor around, I would recommend the 'curses' > library: > > https://docs.python.org/3/library/curses.html > https://docs.python.org/3/howto/curses.html > > There are other ways, too; with more info on what you're trying to > accomplish, we could better advise. It might be that a GUI will serve > you well, particularly if you have several pieces of information that > you want to update. > > ChrisA Since someone mentioned curses, I'll add that I've used npyscreen (built on top of ncurses), successfully, creating small guis with very few code lines. If you like oldschool gui, it's a must. http://npyscreen.readthedocs.io/introduction.html jm From bc at freeuk.com Thu Oct 6 09:13:50 2016 From: bc at freeuk.com (BartC) Date: Thu, 6 Oct 2016 14:13:50 +0100 Subject: User Interface Suggestions? (newbie) In-Reply-To: References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: On 06/10/2016 13:38, Peter Otten wrote: > BartC wrote: >> All this advice seems to be getting out of hand, with suggestions of >> 'curses' and 'blessings' and using GUI. I've tried 'ncurses' elsewhere >> and it was over the top for what I wanted to do. >> >> The OP wants to runs on Pi which I think runs Linux. >> >> So all they are asking is, is there a way of randomly positioning the >> cursor within the terminal window so that the next output is at that >> position. Something like an escape sequence. Terminal screens have been >> around a long time, you'd think someone would have had such a >> requirement before! >> >> I'd quite like to know too. However I've just tried a test sequence >> ("[P1d" to move the cursor to row 1) and it didn't work. If there's >> reason why something so basic won't work (hence the need for curses etc) >> then that would be useful to know too. (And how does curses manage it?!) > > Perhaps you picked the wrong escape sequence? I tried ;H > taken from , and it > seems to work (the test system is not a Pi though): Hey, that worked! (Maybe I'll finally be able to port my editor and IDE to Linux after all.) Maybe that can help the OP too. These sequences can trivially be wrapped with easy-to-use functions such as setpos(row,column). (Ideally there needs to be a way to get terminal size in rows and columns too.) -- Bartc From nad at python.org Thu Oct 6 10:36:59 2016 From: nad at python.org (Ned Deily) Date: Thu, 6 Oct 2016 10:36:59 -0400 Subject: Copying a compiled Python from one system to another In-Reply-To: <57f08c57$0$1609$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> <878tu71z23.fsf@nightsong.com> <57f07625$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f08c57$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-02 00:25, Steve D'Aprano wrote: > On Sun, 2 Oct 2016 01:58 pm, Chris Angelico wrote: >> Hmm, I've possibly missed something here, which may indicate a >> problem. Why can't your existing machines build? Is it because they >> have too-old versions of tools, and if so, which? > Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most > recent any of my systems support is 4.4. I'm using a gcc 4.2 to build 3.6. From steve+python at pearwood.info Thu Oct 6 11:01:13 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 02:01:13 +1100 Subject: static, class and instance methods (Reposting On Python-List Prohibited) References: <57f5f503$0$24766$426a74cc@news.free.fr> <57f61356$0$19739$426a74cc@news.free.fr> Message-ID: <57f6673a$0$1598$c3e8da3$5496439d@news.astraweb.com> On Thu, 6 Oct 2016 08:03 pm, ast wrote: > Consider this function: > > def add(a, b): > return a+b > > You say that a function is always stored as > a descriptor object, so when I execute > > sum = f(4, 6) > > from which class it is supposed to come from ? It doesn't. The descriptor protocol only gets called by classes, so when you call a function directly, the special __get__ method isn't used. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Oct 6 11:04:35 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 02:04:35 +1100 Subject: Copying a compiled Python from one system to another References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> <878tu71z23.fsf@nightsong.com> <57f07625$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f08c57$0$1609$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57f66804$0$1598$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 01:36 am, Ned Deily wrote: > On 2016-10-02 00:25, Steve D'Aprano wrote: >> On Sun, 2 Oct 2016 01:58 pm, Chris Angelico wrote: >>> Hmm, I've possibly missed something here, which may indicate a >>> problem. Why can't your existing machines build? Is it because they >>> have too-old versions of tools, and if so, which? >> Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most >> recent any of my systems support is 4.4. > > I'm using a gcc 4.2 to build 3.6. How on earth do you do that? What magic incantations are you using? I understand that Python 3.6 is now using C99 features that aren't available before gcc 4.8. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Thu Oct 6 11:08:02 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 02:08:02 +1100 Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: <57f668d3$0$1620$c3e8da3$5496439d@news.astraweb.com> On Thu, 6 Oct 2016 10:03 pm, BartC wrote: > I'd quite like to know too. However I've just tried a test sequence > ("[P1d" to move the cursor to row 1) and it didn't work. If there's > reason why something so basic won't work (hence the need for curses etc) > then that would be useful to know too. (And how does curses manage it?!) I believe that curses uses a large database of terminal types and associated escape sequences. So when you start it up, it determines what terminal type you have (that's black magic itself) and then works out what escape sequences you need. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nomail at com.invalid Thu Oct 6 11:13:11 2016 From: nomail at com.invalid (ast) Date: Thu, 6 Oct 2016 17:13:11 +0200 Subject: static, class and instance methods (Reposting On Python-List Prohibited) In-Reply-To: <57f6673a$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <57f5f503$0$24766$426a74cc@news.free.fr> <57f61356$0$19739$426a74cc@news.free.fr> <57f6673a$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <57f66a34$0$3325$426a74cc@news.free.fr> "Steve D'Aprano" a ?crit dans le message de news:57f6673a$0$1598$c3e8da3$5496439d at news.astraweb.com... > On Thu, 6 Oct 2016 08:03 pm, ast wrote: > >> Consider this function: >> >> def add(a, b): >> return a+b >> >> You say that a function is always stored as >> a descriptor object, so when I execute >> >> sum = f(4, 6) >> >> from which class it is supposed to come from ? > > > It doesn't. The descriptor protocol only gets called by classes, so when you > call a function directly, the special __get__ method isn't used. > > yes, it is clear, thanks to all (I didn't know that functions were descriptors with a __get__ method) From desolate.soul.me at gmail.com Thu Oct 6 11:22:05 2016 From: desolate.soul.me at gmail.com (desolate.soul.me at gmail.com) Date: Thu, 6 Oct 2016 08:22:05 -0700 (PDT) Subject: BeautifulSoup help !! Message-ID: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> So I've just started up with python and an assignment was given to me by a company as an recruitment task. I need to web scrap the coupons of all the websites available on http://www.couponraja.in and export it to csv format. The details which I need to be present in the csv are the coupon title , vendor , validity , description/detail , url to the vendor , image url of the coupon. I have gone through many tutorials on beautifulsoup and have a beginners understanding of using it. Wrote a code as well , but the problem Im facing here is when i collect info from the divs which contains all those info , Im getting it in with all the html tags and the info is clustered. Code m using : import requests from bs4 import BeautifulSoup url = "https://www.couponraja.in/amazon" r = requests.get(url) soup = BeautifulSoup(r.content) g_data = soup.find_all("div", {"class": "nw-offrtxt"}) for item in g_data: print item.contents also will need help on how to export the info to csv format , I just know I need to import csv then write the information to a csv file. But not getting through on how to achieve that. Any help will be appreciated. From alister.ware at ntlworld.com Thu Oct 6 11:30:07 2016 From: alister.ware at ntlworld.com (alister) Date: Thu, 06 Oct 2016 15:30:07 GMT Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> Message-ID: <36uJz.57117$gT.12654@fx29.am4> On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > So I've just started up with python and an assignment was given to me by > a company as an recruitment task. > so by your own admission you have just started with python yet you consider your self suitable for employment? -- "Unibus timeout fatal trap program lost sorry" - An error message printed by DEC's RSTS operating system for the PDP-11 From desolate.soul.me at gmail.com Thu Oct 6 11:50:25 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Thu, 6 Oct 2016 08:50:25 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <36uJz.57117$gT.12654@fx29.am4> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> Message-ID: <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> On Thursday, October 6, 2016 at 9:00:21 PM UTC+5:30, alister wrote: > On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > > > So I've just started up with python and an assignment was given to me by > > a company as an recruitment task. > > > so by your own admission you have just started with python yet you > consider your self suitable for employment? > > > -- > "Unibus timeout fatal trap program lost sorry" > - An error message printed by DEC's RSTS operating system for the PDP-11 yup ... training will be provided further , all they want to confirm is atleast I have basic knowledge of how the language works and they wont have to tell me how to install python on the system or any further extensions to it. Im not quite as to why they provided me with this assignment when my cv clearly states that im good with .net and not python. I hope you could help here and not just trolling around. From rosuav at gmail.com Thu Oct 6 11:57:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 02:57:54 +1100 Subject: Is it possible to use 'groupby' asynchronously? In-Reply-To: References: Message-ID: On Thu, Oct 6, 2016 at 11:09 PM, Frank Millman wrote: > Hi all > > I have used itertools.groupby before, and I love it. I used it to process a > csv file and 'break' on change of a particular field. It worked very well. > > Now I want to use it to process a database table. I can select the rows in > the desired sequence with no problem. However, I am using asyncio, so I am > reading the rows asynchronously. > > Before I spend hours trying to figure it out, can anyone confirm if this is > doable at all, or is groupby not designed for this. Most of itertools is going to assume synchronous iterables. However, the docs give pure-Python equivalents for quite a few: https://docs.python.org/3/library/itertools.html#itertools.groupby You may be able to tweak that into an actual "itertools.agroupby" or "aitertools.groupby". ChrisA From rosuav at gmail.com Thu Oct 6 12:00:48 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 03:00:48 +1100 Subject: BeautifulSoup help !! In-Reply-To: <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> Message-ID: On Fri, Oct 7, 2016 at 2:50 AM, Navneet Siddhant wrote: > On Thursday, October 6, 2016 at 9:00:21 PM UTC+5:30, alister wrote: >> On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: >> >> > So I've just started up with python and an assignment was given to me by >> > a company as an recruitment task. >> > >> so by your own admission you have just started with python yet you >> consider your self suitable for employment? > > yup ... training will be provided further , all they want to confirm is atleast I have basic knowledge of how the language works and they wont have to tell me how to install python on the system or any further extensions to it. Im not quite as to why they provided me with this assignment when my cv clearly states that im good with .net and not python. > > I hope you could help here and not just trolling around. It's not trolling around, it's a very serious question. You are asking for assistance with something that was assigned to you *as a recruitment task*. Were you told that asking for help was a legitimate solution? If you need to ask for help with this task, will you also need to ask for help with everything you'll be doing once employed? Maybe the right thing to do is dedicate a few solid hours (an evening or afternoon or something) to just messing around with Python and bs4. Learn the stuff you're trying to get hired for. ChrisA From alister.ware at ntlworld.com Thu Oct 6 12:04:08 2016 From: alister.ware at ntlworld.com (alister) Date: Thu, 06 Oct 2016 16:04:08 GMT Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> Message-ID: On Thu, 06 Oct 2016 08:50:25 -0700, Navneet Siddhant wrote: > On Thursday, October 6, 2016 at 9:00:21 PM UTC+5:30, alister wrote: >> On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: >> >> > So I've just started up with python and an assignment was given to me >> > by a company as an recruitment task. >> > >> so by your own admission you have just started with python yet you >> consider your self suitable for employment? >> >> >> -- >> "Unibus timeout fatal trap program lost sorry" >> - An error message printed by DEC's RSTS operating system for the >> PDP-11 > > > yup ... training will be provided further , all they want to confirm is > atleast I have basic knowledge of how the language works and they wont > have to tell me how to install python on the system or any further > extensions to it. Im not quite as to why they provided me with this > assignment when my cv clearly states that im good with .net and not > python. > > I hope you could help here and not just trolling around. well if this is just a start of training it is a bit different, we dont like writing code for homework assignments (even less for comercial assignments) but can give some general pointers. you code currently gets the div from the soup object of the page & as you say it contains all of the HTML (it is also a soup object) what you need to do is use another find_all to extract the elements that contain the information you require. depending on the page layout you may need to drill down multiple levels & combine multiple different elements to get what you require performing the tasks manually with the page html source before automating is the usual approach. -- It turned out that the worm exploited three or four different holes in the system. From this, and the fact that we were able to capture and examine some of the source code, we realized that we were dealing with someone very sharp, probably not someone here on campus. -- Dr. Richard LeBlanc, associate professor of ICS, in Georgia Tech's campus newspaper after the Internet worm. From nad at python.org Thu Oct 6 12:08:54 2016 From: nad at python.org (Ned Deily) Date: Thu, 6 Oct 2016 12:08:54 -0400 Subject: Copying a compiled Python from one system to another In-Reply-To: <57f66804$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <57ef9c40$0$1611$c3e8da3$5496439d@news.astraweb.com> <878tu71z23.fsf@nightsong.com> <57f07625$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f08c57$0$1609$c3e8da3$5496439d@news.astraweb.com> <57f66804$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-06 11:04, Steve D'Aprano wrote: > On Fri, 7 Oct 2016 01:36 am, Ned Deily wrote: >> On 2016-10-02 00:25, Steve D'Aprano wrote: >>> On Sun, 2 Oct 2016 01:58 pm, Chris Angelico wrote: >>>> Hmm, I've possibly missed something here, which may indicate a >>>> problem. Why can't your existing machines build? Is it because they >>>> have too-old versions of tools, and if so, which? >>> Yes, this. You need gcc 4.8 or better to build CPython 3.6, and the most >>> recent any of my systems support is 4.4. >> >> I'm using a gcc 4.2 to build 3.6. > > How on earth do you do that? What magic incantations are you using? > > I understand that Python 3.6 is now using C99 features that aren't available > before gcc 4.8. 3.6 is only using a subset of the C99 features that were proposed in the earlier python-dev discussion. In this case, I am using the Apple version of GNU gcc-4.2 that shipped with Xcode 3 on OS X 10.5 and 10.6. I haven't tried it with other versions of gcc-4.2 but it's worth trying. From steve+python at pearwood.info Thu Oct 6 12:12:32 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 03:12:32 +1100 Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> Message-ID: <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 02:30 am, alister wrote: > On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > >> So I've just started up with python and an assignment was given to me by >> a company as an recruitment task. >> > so by your own admission you have just started with python yet you > consider your self suitable for employment? What's your problem Alister? Do you think that junior devs aren't allowed to ask for help? Desolate.Soul.Me has either applied for a job, and their interview test is "do this task using Python", or he's been accepted in a new job, and the same applies. Whether it's a learning exercise, a test of skill + initiative, or actual work given to a junior developer, Desolate.Soul.Me is perfectly entitled to ask for help. This isn't some artificially constrained academic homework, with stupidly strict and hypocritical rules about so-called "plagiarism". This is the real world where you take all the help you can get and you shouldn't feel ashamed for asking for help. ESPECIALLY in the open source world, including Python, where one of the community values is to share expertise. My own employer has hired plenty of junior developers and given them relatively minor tasks to do as a learning exercise. We're not going to trust a junior developer with a critical piece of code, but we might say: "Scrape this website. Use Python. Here's the Python For Beginners book. Here's the Python documentation, and a few more forums where you can ask for help. If you get stuck, and aren't getting useful answers from the forums, you can ask Lisa. But not today, as she's busy doing a critical release and can't be disturbed." P.S. Desolate.Soul.Me, you might be taken a bit more seriously if you give a name, or at least a moniker or nick-name which is easier for others to refer to you by. It doesn't have to be your birthname, or legal name. What do your friends and workmates call you? I don't know Beautiful Soup, so I'm afraid I can't help. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From desolate.soul.me at gmail.com Thu Oct 6 12:27:32 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Thu, 6 Oct 2016 09:27:32 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <71b8dbc9-cd37-42d2-8c81-e18a83b306d6@googlegroups.com> On Thursday, October 6, 2016 at 9:42:47 PM UTC+5:30, Steve D'Aprano wrote: > On Fri, 7 Oct 2016 02:30 am, alister wrote: > > > On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > > > >> So I've just started up with python and an assignment was given to me by > >> a company as an recruitment task. > >> > > so by your own admission you have just started with python yet you > > consider your self suitable for employment? > > What's your problem Alister? Do you think that junior devs aren't allowed to > ask for help? > > Desolate.Soul.Me has either applied for a job, and their interview test > is "do this task using Python", or he's been accepted in a new job, and the > same applies. > > Whether it's a learning exercise, a test of skill + initiative, or actual > work given to a junior developer, Desolate.Soul.Me is perfectly entitled to > ask for help. > > This isn't some artificially constrained academic homework, with stupidly > strict and hypocritical rules about so-called "plagiarism". This is the > real world where you take all the help you can get and you shouldn't feel > ashamed for asking for help. ESPECIALLY in the open source world, including > Python, where one of the community values is to share expertise. > > My own employer has hired plenty of junior developers and given them > relatively minor tasks to do as a learning exercise. We're not going to > trust a junior developer with a critical piece of code, but we might say: > > "Scrape this website. Use Python. Here's the Python For Beginners > book. Here's the Python documentation, and a few more forums where > you can ask for help. If you get stuck, and aren't getting useful > answers from the forums, you can ask Lisa. But not today, as she's > busy doing a critical release and can't be disturbed." > > > P.S. Desolate.Soul.Me, you might be taken a bit more seriously if you give a > name, or at least a moniker or nick-name which is easier for others to > refer to you by. It doesn't have to be your birthname, or legal name. What > do your friends and workmates call you? > > > I don't know Beautiful Soup, so I'm afraid I can't help. > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. Yes ... Im allowed to take help as this group and python forums and other links were suggested to me by them. Have been trying since morning not getting the desired results , posted here as last resort .. and Im ok if I dont find what m seeking. Though m getting close , lets see what I could achieve. From noah at neo.co.tz Thu Oct 6 12:29:46 2016 From: noah at neo.co.tz (Noah) Date: Thu, 6 Oct 2016 19:29:46 +0300 Subject: BeautifulSoup help !! In-Reply-To: <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: +1 at Steve.... On 6 Oct 2016 19:17, "Steve D'Aprano" wrote: > On Fri, 7 Oct 2016 02:30 am, alister wrote: > > > On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > > > >> So I've just started up with python and an assignment was given to me by > >> a company as an recruitment task. > >> > > so by your own admission you have just started with python yet you > > consider your self suitable for employment? > > What's your problem Alister? Do you think that junior devs aren't allowed > to > ask for help? > > Desolate.Soul.Me has either applied for a job, and their interview test > is "do this task using Python", or he's been accepted in a new job, and the > same applies. > > Whether it's a learning exercise, a test of skill + initiative, or actual > work given to a junior developer, Desolate.Soul.Me is perfectly entitled > to > ask for help. > > This isn't some artificially constrained academic homework, with stupidly > strict and hypocritical rules about so-called "plagiarism". This is the > real world where you take all the help you can get and you shouldn't feel > ashamed for asking for help. ESPECIALLY in the open source world, including > Python, where one of the community values is to share expertise. > > My own employer has hired plenty of junior developers and given them > relatively minor tasks to do as a learning exercise. We're not going to > trust a junior developer with a critical piece of code, but we might say: > > "Scrape this website. Use Python. Here's the Python For Beginners > book. Here's the Python documentation, and a few more forums where > you can ask for help. If you get stuck, and aren't getting useful > answers from the forums, you can ask Lisa. But not today, as she's > busy doing a critical release and can't be disturbed." > > > P.S. Desolate.Soul.Me, you might be taken a bit more seriously if you > give a > name, or at least a moniker or nick-name which is easier for others to > refer to you by. It doesn't have to be your birthname, or legal name. What > do your friends and workmates call you? > > > I don't know Beautiful Soup, so I'm afraid I can't help. > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. > > -- > https://mail.python.org/mailman/listinfo/python-list > From desolate.soul.me at gmail.com Thu Oct 6 12:30:18 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Thu, 6 Oct 2016 09:30:18 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <71b8dbc9-cd37-42d2-8c81-e18a83b306d6@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> <71b8dbc9-cd37-42d2-8c81-e18a83b306d6@googlegroups.com> Message-ID: On Thursday, October 6, 2016 at 9:57:46 PM UTC+5:30, Navneet Siddhant wrote: > On Thursday, October 6, 2016 at 9:42:47 PM UTC+5:30, Steve D'Aprano wrote: > > On Fri, 7 Oct 2016 02:30 am, alister wrote: > > > > > On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: > > > > > >> So I've just started up with python and an assignment was given to me by > > >> a company as an recruitment task. > > >> > > > so by your own admission you have just started with python yet you > > > consider your self suitable for employment? > > > > What's your problem Alister? Do you think that junior devs aren't allowed to > > ask for help? > > > > Desolate.Soul.Me has either applied for a job, and their interview test > > is "do this task using Python", or he's been accepted in a new job, and the > > same applies. > > > > Whether it's a learning exercise, a test of skill + initiative, or actual > > work given to a junior developer, Desolate.Soul.Me is perfectly entitled to > > ask for help. > > > > This isn't some artificially constrained academic homework, with stupidly > > strict and hypocritical rules about so-called "plagiarism". This is the > > real world where you take all the help you can get and you shouldn't feel > > ashamed for asking for help. ESPECIALLY in the open source world, including > > Python, where one of the community values is to share expertise. > > > > My own employer has hired plenty of junior developers and given them > > relatively minor tasks to do as a learning exercise. We're not going to > > trust a junior developer with a critical piece of code, but we might say: > > > > "Scrape this website. Use Python. Here's the Python For Beginners > > book. Here's the Python documentation, and a few more forums where > > you can ask for help. If you get stuck, and aren't getting useful > > answers from the forums, you can ask Lisa. But not today, as she's > > busy doing a critical release and can't be disturbed." > > > > > > P.S. Desolate.Soul.Me, you might be taken a bit more seriously if you give a > > name, or at least a moniker or nick-name which is easier for others to > > refer to you by. It doesn't have to be your birthname, or legal name. What > > do your friends and workmates call you? > > > > > > I don't know Beautiful Soup, so I'm afraid I can't help. > > > > > > > > -- > > Steve > > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > > enough, things got worse. > > Yes ... Im allowed to take help as this group and python forums and other links were suggested to me by them. Have been trying since morning not getting the desired results , posted here as last resort .. and Im ok if I dont find what m seeking. Though m getting close , lets see what I could achieve. Btw my institute has referred me for this job in the company who has further emailed me with the assignment to complete after which I will have face to face interviews. Im still wondering as why I was told to do a python assignment when nowhere in my CV i have mentioned about python nor it was taught to us in the curriculum From noah at neo.co.tz Thu Oct 6 12:32:20 2016 From: noah at neo.co.tz (Noah) Date: Thu, 6 Oct 2016 19:32:20 +0300 Subject: Python and ssh for remote login In-Reply-To: References: Message-ID: On 6 Oct 2016 04:56, "Michael Torrie" wrote: > > On 10/05/2016 11:46 AM, Noah wrote: > > Hello folk, > > > > I would like to use a python script to ssh into a server using a username > > and password and perhaps ass port. > > > > Any ideas on how to script that. > > If paramiko doesn't fit your needs, traditionally this sort of work was > done with the pexpect module for drying a TTY. There is a submodule of > pexpect called pxssh for automating things. > Hi Micheal Thank youn for your suggestion. I played around with paramiko today and the results are owesome. > http://pexpect.readthedocs.io/en/stable/api/pxssh.html > > Note that pexpect uses your normal ssh binary. Paramiko is a complete > implementation of the ssh protocol in python. Both modules are useful > and fill certain needs. So i am going to also try pexpect and everything pexpect and i will let you know. Thank you so much Noah From ian.g.kelly at gmail.com Thu Oct 6 12:36:07 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 6 Oct 2016 10:36:07 -0600 Subject: Is it possible to use 'groupby' asynchronously? In-Reply-To: References: Message-ID: On Thu, Oct 6, 2016 at 9:57 AM, Chris Angelico wrote: > On Thu, Oct 6, 2016 at 11:09 PM, Frank Millman wrote: >> Hi all >> >> I have used itertools.groupby before, and I love it. I used it to process a >> csv file and 'break' on change of a particular field. It worked very well. >> >> Now I want to use it to process a database table. I can select the rows in >> the desired sequence with no problem. However, I am using asyncio, so I am >> reading the rows asynchronously. >> >> Before I spend hours trying to figure it out, can anyone confirm if this is >> doable at all, or is groupby not designed for this. > > Most of itertools is going to assume synchronous iterables. However, > the docs give pure-Python equivalents for quite a few: > > https://docs.python.org/3/library/itertools.html#itertools.groupby > > You may be able to tweak that into an actual "itertools.agroupby" or > "aitertools.groupby". There is already a third-party "aitertools" library, so there should be no need to write one from scratch. https://pypi.python.org/pypi/aitertools/0.1.0 From steve+python at pearwood.info Thu Oct 6 12:38:38 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 03:38:38 +1100 Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> Message-ID: <57f67e10$0$22141$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 03:00 am, Chris Angelico wrote: > You are asking > for assistance with something that was assigned to you *as a > recruitment task*. Were you told that asking for help was a legitimate > solution? Why should he need to be told that? Asking for help *is* a legitimate solution, just as legitimate as Reading The Fine Manual, searching on Google, asking your work peers for mentoring, or doing a training course. What's next? Are we going to start dumping on people for reading the manual? "If you can't intuit the right method calls, you're cheating"? "What, you had to *study* to learn the language? With a teacher and everything? Cheater!" Asking questions and sharing expertise is what we're here for, and it is an absolutely critical part of open-source communities. The idea that people cannot ask for help because they are working, or being interviewed for work, should be anathema to us all. Schooling is slightly different in that (1) whether we like it or not, many schools enforce ridiculously stupid and strict rules against plagiarism, so bizarrely the best way we can help students is to refuse to help them[1]; and (2) when people are learning, making their own mistakes and wrong turns is often a good learning exercise. For students, it is the learning process itself which is more important than the solution -- and the less experienced the student, the more that is the case. But heaven forbid that students copy recipes from the documentation, from the ActiveState Cookbook, or Stackoverflow without attribution like professionals do. The current anti-plagiarism (so-called plagiarism) climate goes directly against the principles of free and open code and information. Requiring every trivial thought and phrase to be wholly original[2] or else paid for (whether paid for in money or in credit/ reputation points) is a direct assault against the intellectual Commons. [1] The negative effects of an accusation of plagiarism for few lines of allegedly copied code or text may be far, far worse than if the student actually learns nothing. If you learn nothing, there's at least a chance that your teacher will grade on a curve and you'll squeeze in a passing grade, but allegedly plagiarise, even *your own work*[3] and you may lose all your academic future. [2] An impossibility. [3] Quite possibly the stupidest thing that has come out of academia since post-modernism. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jtim.arnold at gmail.com Thu Oct 6 12:46:45 2016 From: jtim.arnold at gmail.com (Tim) Date: Thu, 6 Oct 2016 09:46:45 -0700 (PDT) Subject: segfault using shutil.make_archive Message-ID: I need to zip up a directory that's about 400mb. I'm using shutil.make_archive and I'm getting this response: Segmentation fault: 11 (core dumped) The code is straightforward (and works on other, smaller dirs): shutil.make_archive(os.path.join(zip_dir, zname), 'zip', tgt_dir) I guess I could drop the convenience of make_archive and use zipfile but that seems to be exactly what make_archive does. I'm on FreeBSD, python2.7. Anyone seen this before? thanks, --tim From desolate.soul.me at gmail.com Thu Oct 6 13:00:38 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Thu, 6 Oct 2016 10:00:38 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> Message-ID: On Thursday, October 6, 2016 at 8:52:18 PM UTC+5:30, Navneet Siddhant wrote: > So I've just started up with python and an assignment was given to me by a company as an recruitment task. > > I need to web scrap the coupons of all the websites available on http://www.couponraja.in and export it to csv format. > The details which I need to be present in the csv are the coupon title , vendor , validity , description/detail , url to the vendor , image url of the coupon. > > I have gone through many tutorials on beautifulsoup and have a beginners understanding of using it. Wrote a code as well , but the problem Im facing here is when i collect info from the divs which contains all those info , Im getting it in with all the html tags and the info is clustered. > > Code m using : > > import requests > from bs4 import BeautifulSoup > > url = "https://www.couponraja.in/amazon" > r = requests.get(url) > soup = BeautifulSoup(r.content) > g_data = soup.find_all("div", {"class": "nw-offrtxt"}) > for item in g_data: > print item.contents > > also will need help on how to export the info to csv format , I just know I need to import csv then write the information to a csv file. > But not getting through on how to achieve that. > > Any help will be appreciated. Thanx for the support Steve . N yes you can call me Sid .. :) . . .. I guess I shouldnt have mentioned as this was a recruitment task. If needed I can post a screenshot of the mail I got which says I can take help from anywhere possible as long as the assignment is done. Wont be simply copying pasting the code as question related to the same will be asked in the interview. I just need a proper understanding as to what I need to do to get the results. Also how to export the result to csv format. From mr.puneet.goyal at gmail.com Thu Oct 6 13:06:29 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Thu, 6 Oct 2016 10:06:29 -0700 (PDT) Subject: A newbie doubt on methods/functions calling Message-ID: Hi I just started learning python. Is there any way to call functions in different way ? Rather calling obj.function(arg1, arg2) I would like to call like below "obj function arg1 arg2" this function is part of a class. class myClass: def function(arg1, arg2): # do something Is it possible to do in python ? May be not directly but using some other methods. Regards, Puneet From grant.b.edwards at gmail.com Thu Oct 6 13:07:40 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 6 Oct 2016 17:07:40 +0000 (UTC) Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> Message-ID: On 2016-10-06, BartC wrote: > All this advice seems to be getting out of hand, with suggestions of > 'curses' and 'blessings' and using GUI. I've tried 'ncurses' elsewhere > and it was over the top for what I wanted to do. > > The OP wants to runs on Pi which I think runs Linux. It can run Linux. Whether the OP's is or not, I don't know. > So all they are asking is, is there a way of randomly positioning > the cursor within the terminal window so that the next output is at > that position. Yes. > Something like an escape sequence. Terminal screens have been around > a long time, you'd think someone would have had such a requirement > before! Yes, they did. They wrote the curses library. > I'd quite like to know too. However I've just tried a test sequence > ("[P1d" to move the cursor to row 1) and it didn't work. That doesn't look like an ANSI terminal escape sequence to me -- but I have no idea what terminal type you're using, so I can't tell you whether it's right or not. The ANSI seqeunce to move the cursor to row 1, column 1, is '[1;1m'. Both positions default to 1, so '[m' does the same thing. > If there's reason why something so basic won't work (hence the need > for curses etc) then that would be useful to know too. (And how does > curses manage it?!) Curses knows how to deal with various different terminal types and it also knows about the intracacies of the Unix tty API. If you want to, you can just assume your terminal uses ANSI escape sequences: https://en.wikipedia.org/wiki/ANSI_escape_code That will mostly work on most terminals you run into these days. If you want to go one step further, you can use the terminfo library to deal with different terminal types, but I have no idea how to use it without ncurses. If people care about their programs working with different terminal types, they use ncurses. -- Grant Edwards grant.b.edwards Yow! ! Up ahead! It's a at DONUT HUT!! gmail.com From grant.b.edwards at gmail.com Thu Oct 6 13:09:30 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 6 Oct 2016 17:09:30 +0000 (UTC) Subject: User Interface Suggestions? (newbie) References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> <87twcqtpgd.fsf@gmail.com> <57f668d3$0$1620$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-06, Steve D'Aprano wrote: > On Thu, 6 Oct 2016 10:03 pm, BartC wrote: > >> I'd quite like to know too. However I've just tried a test sequence >> ("[P1d" to move the cursor to row 1) and it didn't work. If there's >> reason why something so basic won't work (hence the need for curses etc) >> then that would be useful to know too. (And how does curses manage it?!) > > I believe that curses uses a large database of terminal types and > associated escape sequences. So when you start it up, it determines > what terminal type you have (that's black magic itself) and then > works out what escape sequences you need. Nothing magic, it just looks at the 'TERM' enviroment varible and looks up that value in the terminfo database. -- Grant Edwards grant.b.edwards Yow! I wonder if there's at anything GOOD on tonight? gmail.com From rosuav at gmail.com Thu Oct 6 13:12:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 04:12:27 +1100 Subject: BeautifulSoup help !! In-Reply-To: <57f67e10$0$22141$c3e8da3$5496439d@news.astraweb.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <85b23c7f-a475-49d1-812f-78579df492dc@googlegroups.com> <57f67e10$0$22141$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 7, 2016 at 3:38 AM, Steve D'Aprano wrote: > On Fri, 7 Oct 2016 03:00 am, Chris Angelico wrote: > >> You are asking >> for assistance with something that was assigned to you *as a >> recruitment task*. Were you told that asking for help was a legitimate >> solution? > > Why should he need to be told that? Asking for help *is* a legitimate > solution, just as legitimate as Reading The Fine Manual, searching on > Google, asking your work peers for mentoring, or doing a training course. Let me clarify a bit. Is "asking someone else to write your code" acceptable? I would say no, in the same way that getting someone else to do your job interview for you is inappropriate. You're right, though, "asking for help" was too broad a description for the situation. Apologies to the OP; of course you can ask questions and get help. You just need to figure out where the boundaries are between "stuff you know and are being hired for", "stuff you don't know and are being hired for, and are learning on the job", and "stuff you don't know and don't need to know, and can get someone else to do for you". The boundary between the first two is not overly important, but if there's not much in the first category, you're going to be very much in over your head. It's the boundary between the latter two that's more important, and which determines the kind of help you're asking for. But please don't accuse us of "trolling around" when you're asked this kind of thing. ChrisA From rosuav at gmail.com Thu Oct 6 13:23:45 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 04:23:45 +1100 Subject: BeautifulSoup help !! In-Reply-To: References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> Message-ID: On Fri, Oct 7, 2016 at 4:00 AM, Navneet Siddhant wrote: > I guess I shouldnt have mentioned as this was a recruitment task. If needed I can post a screenshot of the mail I got which says I can take help from anywhere possible as long as the assignment is done. Wont be simply copying pasting the code as question related to the same will be asked in the interview. > I just need a proper understanding as to what I need to do to get the results. > Also how to export the result to csv format. A screenshot isn't necessary - we trust you to not flat-out lie to us. (And if you did, well, this is a public list, so your deception would burn you pretty thoroughly once someone finds out.) Stating that you can take help from anywhere would have been a good clarification. Anyway. One of the annoying facts of the real world is that web scraping is *hard*. We have awesome tools like Beautiful Soup that save us a lot of hassle, but ultimately, you have to look at the results and figure out which parts are "interesting" (that is, the parts that have the data you want, or tell you about its structure, or something like that). I strongly recommend messing with bs4 at the interactive prompt; basically, just play around with everything you get hold of. Eventually, you want to be building up a series of rows, where each row is a list of column values; you write a row to the CSV file, and your job's done. Most likely, you're going to have some sort of primary loop - outside of that loop you have bs4 navigation to get you the main block of info, and inside it, you parse through some wad of stuff to find the truly interesting info, and at the bottom of the loop, you write something to the CSV file. Hope that's of some help! ChrisA From __peter__ at web.de Thu Oct 6 13:30:04 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 06 Oct 2016 19:30:04 +0200 Subject: A newbie doubt on methods/functions calling References: Message-ID: mr.puneet.goyal at gmail.com wrote: > Hi > > I just started learning python. Is there any way to call functions in > different way ? > > Rather calling obj.function(arg1, arg2) I would like to call like below > > "obj function arg1 arg2" How would the machine reading the above know that you didn't mean obj(function, arg1, arg2) or obj.function.arg1(arg2) ? > this function is part of a class. > > class myClass: > def function(arg1, arg2): > # do something > > Is it possible to do in python ? May be not directly but using some other > methods. No, that would be a different language, and if someone were to implement it this would make you a newbie in two languages -- hardly an improvement ;) From steve+python at pearwood.info Thu Oct 6 13:31:06 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 04:31:06 +1100 Subject: A newbie doubt on methods/functions calling References: Message-ID: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 04:06 am, mr.puneet.goyal at gmail.com wrote: > Hi > > I just started learning python. Is there any way to call functions in > different way ? > > Rather calling obj.function(arg1, arg2) I would like to call like below > > "obj function arg1 arg2" No. This will be a syntax error. > this function is part of a class. > > class myClass: > def function(arg1, arg2): > # do something > > Is it possible to do in python ? May be not directly but using some other > methods. The only way to do this will be to write your own pre-processor, which will parse your source code, and translate it from your language to valid Python. That's a lot of work for very little value -- I recommend you just learn the Python syntax rather than trying to force it to be something it is not. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From desolate.soul.me at gmail.com Thu Oct 6 13:34:02 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Thu, 6 Oct 2016 10:34:02 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> Message-ID: <5a6575d2-1477-4b2b-bebe-8b73a0336f9d@googlegroups.com> I guess I will have to extract data from multiple divs as only extracting data from the parent div which has other divs in it with the different data is coming up all messed up. Will play around and see if I could get through it. Let me clarify once again I dont need complete code , a resource where I could find more info about using Beautifulsoup will be appreciated. Also do I need some kind of plugin etc to extract data to csv ? or it is built in python and I could simply import csv and write other commands needed ?? From random832 at fastmail.com Thu Oct 6 13:45:17 2016 From: random832 at fastmail.com (Random832) Date: Thu, 06 Oct 2016 13:45:17 -0400 Subject: segfault using shutil.make_archive In-Reply-To: References: Message-ID: <1475775917.3857757.748069537.24765A88@webmail.messagingengine.com> On Thu, Oct 6, 2016, at 12:46, Tim wrote: > I need to zip up a directory that's about 400mb. > I'm using shutil.make_archive and I'm getting this response: > > Segmentation fault: 11 (core dumped) > > The code is straightforward (and works on other, smaller dirs): Are you able to make a test case that reproduces it reliably without any specifics about what data you are using (e.g. can you generate a directory full of blank [or /dev/urandom if the compressed size is a factor] files that causes this)? Is it a large number of files or a large total size of files? From random832 at fastmail.com Thu Oct 6 14:04:01 2016 From: random832 at fastmail.com (Random832) Date: Thu, 06 Oct 2016 14:04:01 -0400 Subject: segfault using shutil.make_archive In-Reply-To: <1475775917.3857757.748069537.24765A88@webmail.messagingengine.com> References: <1475775917.3857757.748069537.24765A88@webmail.messagingengine.com> Message-ID: <1475777041.3863313.748088345.3E4FA8D3@webmail.messagingengine.com> On Thu, Oct 6, 2016, at 13:45, Random832 wrote: > On Thu, Oct 6, 2016, at 12:46, Tim wrote: > > I need to zip up a directory that's about 400mb. > > I'm using shutil.make_archive and I'm getting this response: > > > > Segmentation fault: 11 (core dumped) > > > > The code is straightforward (and works on other, smaller dirs): > > Are you able to make a test case that reproduces it reliably without any > specifics about what data you are using (e.g. can you generate a > directory full of blank [or /dev/urandom if the compressed size is a > factor] files that causes this)? Is it a large number of files or a > large total size of files? Also consider passing a logging object to make_archive and see if it prints out anything interesting before the segfault. import shutil import logging import sys logger = logging.getLogger() logger.setLevel(logging.DEBUG) logger.addHandler(logging.StreamHandler(sys.stderr)) shutil.make_archive(..., logger=logger) From torriem at gmail.com Thu Oct 6 14:20:50 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 6 Oct 2016 12:20:50 -0600 Subject: BeautifulSoup help !! In-Reply-To: <5a6575d2-1477-4b2b-bebe-8b73a0336f9d@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <5a6575d2-1477-4b2b-bebe-8b73a0336f9d@googlegroups.com> Message-ID: <4354bc98-c4b3-a5ac-4696-cdf726ceb99d@gmail.com> On 10/06/2016 11:34 AM, Navneet Siddhant wrote: > I guess I will have to extract data from multiple divs as only > extracting data from the parent div which has other divs in it with > the different data is coming up all messed up. Will play around and > see if I could get through it. Let me clarify once again I dont need > complete code , a resource where I could find more info about using > Beautifulsoup will be appreciated. Also do I need some kind of > plugin etc to extract data to csv ? or it is built in python and I > could simply import csv and write other commands needed ?? Writing CSV files from Python is relatively easy. Though you may need to understand basic Python data types first, such as lists and dicts. The module itself is mostly documented here: https://docs.python.org/3/library/csv.html And there are numerous examples of its use: https://pymotw.com/2/csv/ https://www.getdatajoy.com/examples/python-data-analysis/read-and-write-a-csv-file-with-the-csv-module To name but two of the first of many google search results. Sounds to me like you need to spend some time learning basic Python data types and how to iterate through them (lists and dicts mainly). BS4 uses both lists and dicts for nearly everything. An hour or two should be enough to get a handle on this. The nice thing about Python is you can build things and run them in an incremental, and mostly interactive way. Regularly print out things so you can see what structure the data has. BS4 is very good about string representations of all its structures you can print them out without knowing anything about them. For example, if you were searching for a tag: results = soup.find('a', attrs = {'data-search-key': "name" }) you can just do: print (results) And easily see how things are nested. Then you can use that to drill down using list indexing to get just the part you need. I suspect if they hire you and you work more on Python you'll grow to really like it. I suppose that Alister expressed consternation because much of what you ask can be solved with some good old fashioned searching of google and is no different from learning any language, including C# which you already know. Python is not C# of course, but the basic principles behind programming are nearly universally-applied to nearly all programming languages. For a programmer, it shouldn't be too hard to move from language to language. Except for some of the more idiomatic things about a language, many aspects are syntactically equivalent. From mbg1708 at planetmail.com Thu Oct 6 14:31:15 2016 From: mbg1708 at planetmail.com (mbg1708 at planetmail.com) Date: Thu, 6 Oct 2016 11:31:15 -0700 (PDT) Subject: User Interface Suggestions? (newbie) In-Reply-To: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> References: <137f8094-f9ab-405d-8727-846ecac022d2@googlegroups.com> Message-ID: <732bb537-c165-40cf-bb51-78cb2a19722a@googlegroups.com> On Wednesday, 5 October 2016 14:10:21 UTC+1, Beverly Howard wrote: > I'm new to Python, but have three decades of experience with FoxPro and VFP plus I started programming in Basic and still comfortable with that. > > I have spent some time with Python and am now fairly familiar with the syntax and functions, but building a user interface has me stopped. > > A primary question would be, "What are options for building a display that would update displayed values without scrolling?" > > My first goal is to build a program that would interface with a Raspberry Pi to control a heating process. > > Thanks in advance, > Beverly Howard I've had great results using glade and python3. Glade provides a graphical tool for designing GUI windows. Once the GUI is designed, glade writes out an XML file which can be read by either C or Python programs. Reading the XML from python3 requires that you import gi.repository. The python code after that is straightforward. (See http://python-gtk-3-tutorial.readthedocs.io/en/latest/builder.html) From jtim.arnold at gmail.com Thu Oct 6 14:43:18 2016 From: jtim.arnold at gmail.com (Tim) Date: Thu, 6 Oct 2016 11:43:18 -0700 (PDT) Subject: segfault using shutil.make_archive In-Reply-To: References: <1475775917.3857757.748069537.24765A88@webmail.messagingengine.com> <1475777041.3863313.748088345.3E4FA8D3@webmail.messagingengine.com> Message-ID: <502d910f-202b-45aa-9537-411c611a0ade@googlegroups.com> On Thursday, October 6, 2016 at 2:04:20 PM UTC-4, Random832 wrote: > On Thu, Oct 6, 2016, at 13:45, Random832 wrote: > > On Thu, Oct 6, 2016, at 12:46, Tim wrote: > > > I need to zip up a directory that's about 400mb. > > > I'm using shutil.make_archive and I'm getting this response: > > > > > > Segmentation fault: 11 (core dumped) > > > > > > The code is straightforward (and works on other, smaller dirs): > > > > Are you able to make a test case that reproduces it reliably without any > > specifics about what data you are using (e.g. can you generate a > > directory full of blank [or /dev/urandom if the compressed size is a > > factor] files that causes this)? Is it a large number of files or a > > large total size of files? > > Also consider passing a logging object to make_archive and see if it > prints out anything interesting before the segfault. > > import shutil > import logging > import sys > logger = logging.getLogger() > logger.setLevel(logging.DEBUG) > logger.addHandler(logging.StreamHandler(sys.stderr)) > shutil.make_archive(..., logger=logger) Interesting. I tried the above in an interactive python session and never saw the problem at all. So then I added the logger to my program code and watched it go. Using the program, I still get the segfault. It happens on the same file; I removed it and then it happens on the next file in the list. It looks like it (python process) has no more space, but the machine has plenty of disk space and available memory. I didn't think about the number of files, I'll look into that. For now, I'm just using subprocess and the zip command, which is working. thanks, --Tim From myspamacct at earthlink.net Thu Oct 6 16:59:17 2016 From: myspamacct at earthlink.net (Jolly Good Spam) Date: Thu, 6 Oct 2016 13:59:17 -0700 Subject: Question on multiple Python users in one application Message-ID: <7293921E664445E582856EB2983E7F6B@watson1> Hello. Please pardon a newbie question. I have a Windows multi-user program that can be used by an arbitrary number of users at once. They can work independently, or they can share data at the file or even variable level if they want. I want to give them the ability to write Python programs within this environment. So I intend to embed CPython access in the program. The basic embedding of CPython seems straight forward. But since I have multiple users, each needs their own Python sandbox, so if they all compile programs with variable 'spam', it doesn't collide. Of course they can all have different programs running at the same time too. I think I need to do some behind-the-scenes thread management and possibly namespace management to allow this to work correctly. But this is where I'm confused, and not sure where to start. I know how to tell my users apart. The program is multi-threaded, but there is not a direct correspondence between a user and a thread; a single user can switch threads, but I know when this happens. Can someone please suggest what I should be looking at and doing to be able to effectively run multiple independent Pythons in a single program? Thank you for your help! Loren From no.email at nospam.invalid Thu Oct 6 17:11:09 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 06 Oct 2016 14:11:09 -0700 Subject: Question on multiple Python users in one application References: <7293921E664445E582856EB2983E7F6B@watson1> Message-ID: <878tu1xjwy.fsf@nightsong.com> "Jolly Good Spam" writes: > Can someone please suggest what I should be looking at and doing to be > able to effectively run multiple independent Pythons in a single > program? Put each Python in a separate process and communicate by IPC. From rosuav at gmail.com Thu Oct 6 17:34:42 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 08:34:42 +1100 Subject: Question on multiple Python users in one application In-Reply-To: <7293921E664445E582856EB2983E7F6B@watson1> References: <7293921E664445E582856EB2983E7F6B@watson1> Message-ID: On Fri, Oct 7, 2016 at 7:59 AM, Jolly Good Spam wrote: > I have a Windows multi-user program that can be used by an arbitrary number > of users at once. They can work independently, or they can share data at the > file or even variable level if they want. I want to give them the ability to > write Python programs within this environment. So I intend to embed CPython > access in the program. Okay. Before you go one micron further, answer this critical question: *Do you trust your users?* Would you permit your users to have complete access to the computer that this program is running on? If they're all people in the same company, running something on the company's own server, you're fine. But if there's even the slightest chance that a malicious user will be on this system, you MUST NOT permit arbitrary code. CPython is *not* a secured environment. So, on to the specifics. > The basic embedding of CPython seems straight forward. But since I have > multiple users, each needs their own Python sandbox, so if they all compile > programs with variable 'spam', it doesn't collide. Of course they can all have > different programs running at the same time too. You want them to be able to share data, even at the level of a single variable. That strongly suggests using the same CPython embed for all your users. You can avoid name collisions simply by giving each user a module; one person's "spam" doesn't collide with another user's "spam" any more than math.log collides with logging.log. However, this is *not* protecting one user from another - it just protects against accidents. (I could easily reach into someone else's module by typing "fred.spam = 123".) So the question really becomes: How independent should the Pythons be? Sharing data is far easier if they're less isolated, but then it's harder to multithread. And above all, the security question. ChrisA From python at lucidity.plus.com Thu Oct 6 17:34:55 2016 From: python at lucidity.plus.com (Erik) Date: Thu, 6 Oct 2016 22:34:55 +0100 Subject: Question on multiple Python users in one application In-Reply-To: <878tu1xjwy.fsf@nightsong.com> References: <7293921E664445E582856EB2983E7F6B@watson1> <878tu1xjwy.fsf@nightsong.com> Message-ID: <84a5535b-79ec-acd9-2bf7-cd8591ed795e@lucidity.plus.com> On 06/10/16 22:11, Paul Rubin wrote: > "Jolly Good Spam" writes: >> Can someone please suggest what I should be looking at and doing to be >> able to effectively run multiple independent Pythons in a single >> program? > > Put each Python in a separate process and communicate by IPC. Loren says that this is to be CPython embedded into an existing application (in which each user can be identified somehow). We need to understand first what the process/threading/per-user model of the existing application is. E. From lwilton at earthlink.net Thu Oct 6 17:40:12 2016 From: lwilton at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 14:40:12 -0700 Subject: Question on multiple Python users in one application Message-ID: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> (Apologies for the broken threading on this reply, I'm just getting the list access set up.) > Put each Python in a separate process and communicate by IPC. I thought of that, but the multi-user program is a virtual machine implementation, and the programs running on the machine have resources like variables, arrays, files, and databases that the users are likely to want to access from Python as native Python objects. It's unlikely or impossible that they would or could pass all objects as command line parameters to a Python program. While it is certianly possible to marshall every variable access through IPC, it isn't real efficient. I would much perfer to avoid this if I possibly can. Thanks, Loren From no.email at nospam.invalid Thu Oct 6 17:46:15 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 06 Oct 2016 14:46:15 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> Message-ID: <874m4pxiag.fsf@nightsong.com> "Loren Wilton" writes: > While it is certianly possible to marshall every variable access > through IPC, it isn't real efficient. I would much perfer to avoid > this if I possibly can. Maybe you could use Python proxy objects accessing a shared memory segment. Though as Chris Angelico mentions, you may have a security issue depending on what the users are up to. Can you say what the application is for? Does the user language have to be Python? What you're describing might be easier with Erlang. From python at lucidity.plus.com Thu Oct 6 17:55:51 2016 From: python at lucidity.plus.com (Erik) Date: Thu, 6 Oct 2016 22:55:51 +0100 Subject: Question on multiple Python users in one application In-Reply-To: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> Message-ID: <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> On 06/10/16 22:40, Loren Wilton wrote: > the multi-user program is a virtual machine > implementation That's not relevant (unless you mean each user is running in their own VM, in which case you _really_ need to describe your execution environment). BTW, you _really_ need to describe your execution environment ;) > and the programs running on the machine have resources > like variables, arrays, files, and databases Python variables? Python arrays (lists? tuples?)? Or some other sort of "variables" and "arrays" that exist outside of Python? You _really_ need to describe your execution environment ;) E ;) From myspamacct at earthlink.net Thu Oct 6 18:09:37 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 15:09:37 -0700 Subject: Question on multiple Python users in one application References: <7293921E664445E582856EB2983E7F6B@watson1> Message-ID: <3EE4AF9FACFA4A088BCFC0CE10C18321@watson1> > Okay. Before you go one micron further, answer this critical question: > > *Do you trust your users?* The basic answer is yes. This program amounts to a virtual machine implementation of a multi-processor and multi-user mainframe computer. It runs on a Windows box of its own. It has an IO subsystem of its own with a pretty hefty firewall machine in front of it before it gets to internet access. The Windows OS doesn't have network access at all. That may sound like a joke to you, but some of the biggest companies in the world have this machine running in their back rooms. > Would you permit your users to have complete access to the computer > that this program is running on? If they're all people in the same > company, running something on the company's own server, you're fine. > But if there's even the slightest chance that a malicious user will be > on this system, you MUST NOT permit arbitrary code. CPython is *not* a > secured environment. This is understodd and accepted. I should say at the moment this is at the feasibility and experimental stage. It would be possible to firewall this into a separate application to protect the VM data, but the machine users would still have access to the PC side thru CPython. It would be possible to go a step farther and put Python in a PC of its own connected thru Infiniband. We've done this with Java, but the results are painful. The goal at the moment is to see if I can make it appear that I have Python running more or less natively on the virtual machine. This means that Python will have direct access to user variables, files, and direct database access for the mainframe databases. Since the mainframe data format doesn't match that of the commmon Python objects, I'll either have to subclass the Python objects or make my own objects similar to the equivalent Python objects for things like numbers and strings. That is a project for tomorrow, not today. It seems straight-forward, just a lot of work. > You want them to be able to share data, even at the level of a single > variable. Yes. > That strongly suggests using the same CPython embed for all > your users. That was the conclusion I'd initially come to, yes. > You can avoid name collisions simply by giving each user a > module; one person's "spam" doesn't collide with another user's "spam" > any more than math.log collides with logging.log. However, this is > *not* protecting one user from another - it just protects against > accidents. (I could easily reach into someone else's module by typing > "fred.spam = 123".) This particular mainframe architecture is odd, by today's machine standards. It is a stack machine. The stack is not a simple linear stack as far as addressing is concerned, but more of a tree structure. All user programs have stacks that are rooted at D0 in the operating system, at D1 in their own code file, and at higher levels in various nested program procedures. A new process can be created by forking at any level, and the new process shares all data with the original program from that level outward. It is possible for one stack to export procedure entry points that can be called by other stacks. Variables, arrays, files, etc, can be passed fron one stack (process) to another stack (process). > So the question really becomes: How independent should the Pythons be? > Sharing data is far easier if they're less isolated, but then it's > harder to multithread. The desirable goal is to make it look like Python is a native language running on this machine, but also giving access to existing Python libraries running on the PC that contains this virtual machine. Effectively this is extending the boundaries of the virtual machine to include processor chips of a different architecture within the scope of the VM. This has security implications, but at the moment they are (at least partially) understood and accepted. > And above all, the security question. See last sentence above. Thanks! Loren From myspamacct at earthlink.net Thu Oct 6 18:21:34 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 15:21:34 -0700 Subject: Question on multiple Python users in one application References: <7293921E664445E582856EB2983E7F6B@watson1> <878tu1xjwy.fsf@nightsong.com> <84a5535b-79ec-acd9-2bf7-cd8591ed795e@lucidity.plus.com> Message-ID: <0419AFC88AB54EC29DD68AF83E9C6C28@watson1> > We need to understand first what the process/threading/per-user model of > the existing application is. The program is a virtual machine for an old mainframe architecture. You could think of a VM running a Linux implementaiton as a kind of crude mental reference model. The emulated mainframe is a multi-processor machine. We use separate threads to represent each physical processor, and those threads are assigned to separate real Intel processors underneath the program, in Windows. A given user program can have multiple threads. Each thread can be running or waiting for something. If the user thread is running it will be on one of the virtual processors, so will be on some one of the processor threads in the application. The user thread can be interrupted by an IO completion interrupt or timer interrupt or give up control in some way, and be kicked off the virtual processor. When next run it might be on some other virtual processor, and thus on some other thread in the VM application. Again, this is just like a normal user program running in say Linux. So the VM application has many "processor" threads, but a user does not correspond (except transiently) to any given processor thread. The VM also has a number of other threads for various purposes. It would be possible to make a dedicated thread for each user that started a Python program. This could get messy to manage, and there is always the possibility of runnnig out of threads, but it might be a simple way to keep things separated. I think that might allpw Python to use thread local storage to some advantage, if I've understood the docs thatI've read correctly. Loren From myspamacct at earthlink.net Thu Oct 6 18:47:33 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 15:47:33 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: >> the multi-user program is a virtual machine >> implementation >> and the programs running on the machine have resources >> like variables, arrays, files, and databases > > Python variables? Python arrays (lists? tuples?)? Or some other sort of > "variables" and "arrays" that exist outside of Python? > > You _really_ need to describe your execution environment ;) http://bitsavers.informatik.uni-stuttgart.de/pdf/burroughs/B6500_6700/1035441_B6500_B7500_Stack_Mechanism_1968.pdf The linked paper By Erv Hauck and Ben Dent describes what our ancestor machine looked like in 1968, and is about the most recent documentation commonly available on the web. Its been a few years since then, and the machine has changed a little over time, but the basics of the word format and stack mechanism and program structure are basically unchanged. Since the machine is no longer represented by hardware, but instead by an emulator running on Windows, we have some additional capabilities. For instance, instead of writing a Python interpreter that will run inside the virtual machine structure (effectively being a normal B6500 program, in terms of the referenced document) I can use a Python interpreter (CPython) that is an existing DLL running in the WIndows environment, and do a procedure call from my B6500 environment into the Python environment. Effectively this is a call from one stack-based procedure using one bytecode to another stack-based procedure using a different bytecode. The interpreter for the B6500 bytecode will realize what it is calling, fabricate an appropriate call into CPython (possibly switching to a different thread if that is necessary) and then let the Python code do its thing. Eventually the procedure call will (probably) return, and the B6500 code will resume execution. Basically there is a fence where we swich between interpreters on a procedure call or return boundary. The Python code is running as (I hope) a native Windows DLL, so should be able to access any existing Python libraries that exist on the WIndows machine. Obviously this Python code will be using Windows-shaped data objects like integers, floats, and strings. The B6500 code also has integers, floats, doubles, and strings, but they all look different from what Windows data looks like. Since Python accesses things as objects and objects have accessor methods, I am reasonably sure I can either subclass existing Python objects to access B6500 data, or simply make new objects to access that data. Thus the Python code, even though it is running in its own interpreter, will be able to access resources within the virtual machine environment. Does that help? I can expound more if needed, but I don't want to bloat the newsgroup with irrelevent stuff. I don't think my main concern here is being able to call the CPython interpreter routines, but instead it is to be able to provide separate sandboxes for the various programs ("stacks", in B6500 terminology) that might have their own Python sessions or programs. One possible scenario is a B6500 user, sitting at a terminal, and typing "run python". That should get him a copyright notice and a >>> prompt, and he should be able to carry on just as though he was sitting at a Windows command line. The guy at the terminal in the next office should be able to be doing the same thing at the same time. Loren From no.email at nospam.invalid Thu Oct 6 18:49:45 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 06 Oct 2016 15:49:45 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: <87zimhw0s6.fsf@nightsong.com> "Loren Wilton" writes: > I don't think my main concern here is being able to call the CPython > interpreter routines, but instead it is to be able to provide separate > sandboxes for the various programs ("stacks", in B6500 terminology) > that might have their own Python sessions or programs. This is much easier to do with Lua than CPython. I haven't looked at Micropython much yet. From myspamacct at earthlink.net Thu Oct 6 18:56:56 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 15:56:56 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <87zimhw0s6.fsf@nightsong.com> Message-ID: <4B80C22F905040319A1ACC56EA07CF31@watson1> > "Loren Wilton" writes: >> I don't think my main concern here is being able to call the CPython >> interpreter routines, but instead it is to be able to provide separate >> sandboxes for the various programs ("stacks", in B6500 terminology) >> that might have their own Python sessions or programs. > > This is much easier to do with Lua than CPython. I haven't looked at > Micropython much yet. Unfortunately the requirement is to be able to run Python because of its popularity and the size of the existing code base of available libraries. Other languages would be interesting at some point, but only if there is some real business use for the language. Unless banks and clearing houses are commonly using Lua, it might be a hard sell. They are using Python. Loren From rosuav at gmail.com Thu Oct 6 19:06:56 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 10:06:56 +1100 Subject: Question on multiple Python users in one application In-Reply-To: <3EE4AF9FACFA4A088BCFC0CE10C18321@watson1> References: <7293921E664445E582856EB2983E7F6B@watson1> <3EE4AF9FACFA4A088BCFC0CE10C18321@watson1> Message-ID: On Fri, Oct 7, 2016 at 9:09 AM, Loren Wilton wrote: >> Okay. Before you go one micron further, answer this critical question: >> >> *Do you trust your users?* > > > The basic answer is yes. This program amounts to a virtual machine > implementation of a multi-processor and multi-user mainframe computer. It > runs on a Windows box of its own. It has an IO subsystem of its own with a > pretty hefty firewall machine in front of it before it gets to internet > access. The Windows OS doesn't have network access at all. That may sound > like a joke to you, but some of the biggest companies in the world have this > machine running in their back rooms. Good. And no, that's most definitely not a joke; having a VM with no network access is a very smart way to manage security. So you don't have to worry about someone deliberately trying to mess you around. This saves a *TON* of trouble. > The goal at the moment is to see if I can make it appear that I have Python > running more or less natively on the virtual machine. This means that Python > will have direct access to user variables, files, and direct database access > for the mainframe databases. Since the mainframe data format doesn't match > that of the commmon Python objects, I'll either have to subclass the Python > objects or make my own objects similar to the equivalent Python objects for > things like numbers and strings. That is a project for tomorrow, not today. > It seems straight-forward, just a lot of work. That's the inherent work of trying to make one thing talk to another. Most likely, you can ignore most of the differences, and just make sure your script writers are aware, for instance, that integer wrap-around doesn't exist in Python. Unless someone's been deliberately exploiting those kinds of features, that's unlikely to cause problems. >> You can avoid name collisions simply by giving each user a >> module; one person's "spam" doesn't collide with another user's "spam" >> any more than math.log collides with logging.log. However, this is >> *not* protecting one user from another - it just protects against >> accidents. (I could easily reach into someone else's module by typing >> "fred.spam = 123".) > > > This particular mainframe architecture is odd, by today's machine standards. > It is a stack machine. The stack is not a simple linear stack as far as > addressing is concerned, but more of a tree structure. All user programs > have stacks that are rooted at D0 in the operating system, at D1 in their > own code file, and at higher levels in various nested program procedures. A > new process can be created by forking at any level, and the new process > shares all data with the original program from that level outward. It is > possible for one stack to export procedure entry points that can be called > by other stacks. Variables, arrays, files, etc, can be passed fron one stack > (process) to another stack (process). > >> So the question really becomes: How independent should the Pythons be? >> Sharing data is far easier if they're less isolated, but then it's >> harder to multithread. > > > The desirable goal is to make it look like Python is a native language > running on this machine, but also giving access to existing Python libraries > running on the PC that contains this virtual machine. Effectively this is > extending the boundaries of the virtual machine to include processor chips > of a different architecture within the scope of the VM. This has security > implications, but at the moment they are (at least partially) understood and > accepted. Hmmmmmm. Okay, so you have some pretty fundamental architectural differences to deal with. I think what you're doing is going to be possible, but either (a) this is a large and ongoing job, or (b) Python won't feel truly native - it'll be a bridging system, and people will need to understand both ends. Sounds like it'd be a good fun project, though. ChrisA From grant.b.edwards at gmail.com Thu Oct 6 19:09:20 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 6 Oct 2016 23:09:20 +0000 (UTC) Subject: A newbie doubt on methods/functions calling References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-06, Steve D'Aprano wrote: > The only way to do this will be to write your own pre-processor, which will > parse your source code, and translate it from your language to valid > Python. That's a lot of work for very little value -- I recommend you just > learn the Python syntax rather than trying to force it to be something it > is not. [Cue the decades-old story about the elaborate set of C macros that I once saw somebody using so he could write a C program that looked like some flavor of structured BASIC.] -- Grant Edwards grant.b.edwards Yow! Don't SANFORIZE me!! at gmail.com From torriem at gmail.com Thu Oct 6 19:09:40 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 6 Oct 2016 17:09:40 -0600 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> On 10/06/2016 04:47 PM, Loren Wilton wrote: > The Python code is running as (I hope) a native Windows DLL, so should be > able to access any existing Python libraries that exist on the WIndows > machine. Obviously this Python code will be using Windows-shaped data > objects like integers, floats, and strings. > > The B6500 code also has integers, floats, doubles, and strings, but they all > look different from what Windows data looks like. Since Python accesses > things as objects and objects have accessor methods, I am reasonably sure I > can either subclass existing Python objects to access B6500 data, or simply > make new objects to access that data. Thus the Python code, even though it > is running in its own interpreter, will be able to access resources within > the virtual machine environment. So I take it that currently users access the software running in the virtual mainframe over telnet or some form of serial link and that they interact with it in a text terminal? This point is fairly important, because if it's true, then you really don't have any in-band way of clients talking to some Python process(es) running on the machine hosting the virtual mainframe. > > Does that help? I can expound more if needed, but I don't want to bloat the > newsgroup with irrelevent stuff. It's a start. This is fairly interesting. I love the welding of old and new technologies in innovative ways. Please continue. > I don't think my main concern here is being able to call the CPython > interpreter routines, but instead it is to be able to provide separate > sandboxes for the various programs ("stacks", in B6500 terminology) that > might have their own Python sessions or programs. You'll just have to implement Python support for these frames of reference. Python is not going to be running on the virtual hardware; it's running on Windows. All you need to do is provide a way for the remote end users to talk to the Python server running on the windows machine would define sessions for the users and manage the interaction with the virtual mainframe. No matter how you cut it, Python is separate from the users and separate from the mainframe since you can't run Python on the mainframe itself. The only thing that makes sense is to develop some kind of Python-based server architecture that clients can talk to to do things via Python on behalf of the user. At least that's the way I see it pending an answer to the question of how users interact with this mainframe and its programs. From rosuav at gmail.com Thu Oct 6 19:14:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 10:14:30 +1100 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: On Fri, Oct 7, 2016 at 9:47 AM, Loren Wilton wrote: > I don't think my main concern here is being able to call the CPython > interpreter routines, but instead it is to be able to provide separate > sandboxes for the various programs ("stacks", in B6500 terminology) that > might have their own Python sessions or programs. > > One possible scenario is a B6500 user, sitting at a terminal, and typing > "run python". That should get him a copyright notice and a >>> prompt, and > he should be able to carry on just as though he was sitting at a Windows > command line. The guy at the terminal in the next office should be able to > be doing the same thing at the same time. Ah, that probably means you want separate interpreters, then. My previous idea of just having separate modules wouldn't isolate well enough to feel properly comfortable (for instance, the interactive interpreter uses "_" in the builtins to store your last result, and if you have multiple users sharing the builtins, they'd be trampling over each other's underscores). But if you accept that this is a shared environment, with the consequences thereof, you could have easy and convenient sharing - and might even be able to have a tiny hack around the interactive interpreter to make it use a different module name, instead of __main__. So you could do something like this: #### User "Fred" #### run python Python 3.6, yada yada Type "help" yada yada >>> spam = 1234 #### User "Joe" #### run python Python 3.6, yada yada Type "help" yada yada >>> spam = 4321 >>> import fred >>> print(fred.spam) 1234 >>> print(spam) 4321 >>> fred.ham = 2 #### User "Fred" #### >>> print(spam) 1234 >>> print(ham) 2 So they would be *visibly* sharing state. This might be acceptable to you and your users. It would be pretty easy to code up (with the possible exception of the interactive mode). ChrisA From myspamacct at earthlink.net Thu Oct 6 19:27:19 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 16:27:19 -0700 Subject: A newbie doubt on methods/functions calling References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <231E1E51F31B492191BFEA2F66C9D62D@watson1> > [Cue the decades-old story about the elaborate set of C macros that I > once saw somebody using so he could write a C program that looked like > some flavor of structured BASIC.] I once wrote a set pf C defines so that I could compile Pascal with a C compiler without having to change the Pascal source code. Fortunately that was about 40 years ago, and I've long since lost the source for those macros. The goal here isn't to make one thing look like another that it isn't. The goal is to "write a Python environment for a mainframe", ideally without having to do all of the writing to do it from scratch. Lots of people seem to have written Python interpeters in various languages. I guess I could sit down and write one in Algol or NEWP and have a native Python environment. But what good would it do me? This machine isn't something at a university where the first-year hackers write their 5-line programs with no IO. Everything I read says the strength of Python is that there are many existing 3rd party libraries that do lots of useful things. Since a lot of them are distributed as binaries, they would not work in this mainframe environment. So I don't want to WRITE a Python interpreter for the actual mainframe environment. I want to use an interpreter for an existing environment (Windows) where there are already a lot of existing libraries. But since a lot of the data to be analyzed is on the mainframe environment, and the results would be wanted there too, I need to extend the Python data access to the mainframe environment. Loren From bc at freeuk.com Thu Oct 6 19:48:03 2016 From: bc at freeuk.com (BartC) Date: Fri, 7 Oct 2016 00:48:03 +0100 Subject: A newbie doubt on methods/functions calling In-Reply-To: References: Message-ID: On 06/10/2016 18:06, mr.puneet.goyal at gmail.com wrote: > Hi > > I just started learning python. Is there any way to call functions in different way ? > > Rather calling obj.function(arg1, arg2) I would like to call like below > > "obj function arg1 arg2" As has been pointed out, it's difficult to tell whether a space should mean a "." or "(". Or, as is also possible, "=" or "+" or any other kind of operator. But even sticking only with "." and "(", then something like: a b c d e f g could be interpreted in a myriad different ways. Python requires that this is determined at 'compile' time, but information about what each name is (class, instance, function, attribute etc), which might help resolve the ambiguity, isn't available until run-time. > this function is part of a class. > > class myClass: > def function(arg1, arg2): > # do something > > Is it possible to do in python ? May be not directly but using some other methods. It would be hard without extra clues, or extra restrictions on what is possible. But even if a translator could figure out what is what, a human reader will have trouble. What's the problem with typing "." and "(" anyway; keyboard problems? (If so then keep away from languages such as C and C++ because they have a /lot/ more punctuation!) Or used to a language that doesn't require (,) around function arguments? -- Bartc From myspamacct at earthlink.net Thu Oct 6 20:03:30 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 17:03:30 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> Message-ID: <50935B8C00204F18BA2CC2EF9B78D806@watson1> > So I take it that currently users access the software running in the > virtual mainframe over telnet or some form of serial link and that they > interact with it in a text terminal? This point is fairly important, > because if it's true, then you really don't have any in-band way of > clients talking to some Python process(es) running on the machine > hosting the virtual mainframe. Um... Yes and no. Consider a Linux web server environment that happens to have Python installed. A programmer in the organization that ownns the server can probably log on with an SSH session and type Python and then have an interactive Python session where does simple computations or database accesses, or where he runs existing Python scripts. But that isn't how the machine will normally run. Normally the web pages might have queries or other items that run bits of Python code, or Python scripts, some of which might access or even update a database. Normally there might be nightly cron jobs, some of which might be Pyton scripts. > then you really don't have any in-band way of > clients talking to some Python process(es) running on the machine > hosting the virtual mainframe. I'm not clear on what you mean by "in band way". If you mean that there is something on the Windows side of the machine that might wake up and decide it wants to talk to Python, then no, that can't happen. But mainframes these days are generally not interactive in the sense of a Telent terminal. The programming machines have something like that, but the production machines usualy don't. There may be terminals, but they are probably automatic teller machines, or airline reservation terminals. These don't interact with a shell in the Unix sense, they interact with a series of predefined transaction commands that control program access, or maybe are accessed by a single dedicated program. Even web pages, if well written, can only access fixed assets. >> I don't think my main concern here is being able to call the CPython >> interpreter routines, but instead it is to be able to provide separate >> sandboxes for the various programs ("stacks", in B6500 terminology) that >> might have their own Python sessions or programs. > > You'll just have to implement Python support for these frames of > reference. Python is not going to be running on the virtual hardware; > it's running on Windows. All you need to do is provide a way for the > remote end users to talk to the Python server running on the windows > machine would define sessions for the users and manage the interaction > with the virtual mainframe. No matter how you cut it, Python is > separate from the users and separate from the mainframe since you can't > run Python on the mainframe itself. The only thing that makes sense is > to develop some kind of Python-based server architecture that clients > can talk to to do things via Python on behalf of the user. If I need to write these sort of hooks, I'm new enough to Python to not have much of a clue where to start, and would appreciate all the help I can get. I've read that Python supports 'threads', and I'd assumed (maybe incorrectly) that these were somewhat separate environments that could be operating concurrently (modulo the GC lock). I assume that data can be shared between the threads, and probably will share automatically if the name of the item (or the reference) matches. But I'd also assumed that I might be able to do something like start a thread and tell it that it is working in "module user_42" or some such, so that by default it's data would not be global. Assuming that I'm not completely off-base about using threads, I don't really understand how to create a thread or destroy it. I'm also not completely clear on how much useful data there is that is thread local, and how much in the way of Python objects goes in the gloabl heap. Would using threads help me do what I want? Or am I completely off base? Or is there some other approach that I've missed that might be better? Loren From torriem at gmail.com Thu Oct 6 20:21:51 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 6 Oct 2016 18:21:51 -0600 Subject: Question on multiple Python users in one application In-Reply-To: <50935B8C00204F18BA2CC2EF9B78D806@watson1> References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> Message-ID: On 10/06/2016 06:03 PM, Loren Wilton wrote: >> So I take it that currently users access the software running in the >> virtual mainframe over telnet or some form of serial link and that they >> interact with it in a text terminal? This point is fairly important, >> because if it's true, then you really don't have any in-band way of >> clients talking to some Python process(es) running on the machine >> hosting the virtual mainframe. > > Um... Yes and no. > > Consider a Linux web server environment that happens to have Python > installed. > > A programmer in the organization that ownns the server can probably log on > with an SSH session and type Python and then have an interactive Python > session where does simple computations or database accesses, or where he > runs existing Python scripts. > > But that isn't how the machine will normally run. Please let us know how the machine normally runs and how your clients interact with it. We're still very much working in the dark here since we can't even determine how users or clients interact with the mainframe and the kinds of general tasks they do. You mention a web interface. How do you envision your clients invoking Python? > Normally the web pages might have queries or other items that run bits of > Python code, or Python scripts, some of which might access or even update a > database. Normally there might be nightly cron jobs, some of which might be > Pyton scripts. Well that last bit is comparatively easier and doesn't require any threading or multi-user capability in python itself. The script runs, talks to the mainframe over some protocol you've apparently established, retrieves some data and performs some sort of command. > >> then you really don't have any in-band way of >> clients talking to some Python process(es) running on the machine >> hosting the virtual mainframe. > > I'm not clear on what you mean by "in band way". If you mean that there is > something on the Windows side of the machine that might wake up and decide > it wants to talk to Python, then no, that can't happen. I mean that if you client is talking to the mainframe over some kind of protocol already, then you can't really add Python to the mix directly since it does not run on the mainframe. But like I said, we don't know how your clients are using the mainframe and its program, so I can't say what the "band" actually is precisely. But in other terms of reference, if you have a web server that lets a client do something with the mainframe, then using the browser would be considered "in band" since command and control functions through the same interface. If you wanted to add some sort of python console, you'd have to either add something to the protocol to allow this interaction, or do it via some other out-of-band way, such as ssh-ing and running a script. > But mainframes these days are generally not interactive in the sense of a > Telent terminal. The programming machines have something like that, but the > production machines usualy don't. There may be terminals, but they are > probably automatic teller machines, or airline reservation terminals. These > don't interact with a shell in the Unix sense, they interact with a series > of predefined transaction commands that control program access, or maybe are > accessed by a single dedicated program. Even web pages, if well written, can > only access fixed assets. How do the users interact with these predefined transaction commands? There must be some protocol you've implemented to allow this over a remote connection. >> You'll just have to implement Python support for these frames of >> reference. Python is not going to be running on the virtual hardware; >> it's running on Windows. All you need to do is provide a way for the >> remote end users to talk to the Python server running on the windows >> machine would define sessions for the users and manage the interaction >> with the virtual mainframe. No matter how you cut it, Python is >> separate from the users and separate from the mainframe since you can't >> run Python on the mainframe itself. The only thing that makes sense is >> to develop some kind of Python-based server architecture that clients >> can talk to to do things via Python on behalf of the user. > > If I need to write these sort of hooks, I'm new enough to Python to not have > much of a clue where to start, and would appreciate all the help I can get. Well this is an advanced Python topic for sure. But it's one you'll have to delve into eventually it sounds like. Google for information on embedding Python. > I've read that Python supports 'threads', and I'd assumed (maybe > incorrectly) that these were somewhat separate environments that could be > operating concurrently (modulo the GC lock). I assume that data can be > shared between the threads, and probably will share automatically if the > name of the item (or the reference) matches. But I'd also assumed that I > might be able to do something like start a thread and tell it that it is > working in "module user_42" or some such, so that by default it's data would > not be global. > > Assuming that I'm not completely off-base about using threads, I don't > really understand how to create a thread or destroy it. I'm also not > completely clear on how much useful data there is that is thread local, and > how much in the way of Python objects goes in the gloabl heap. > > Would using threads help me do what I want? Or am I completely off base? Or > is there some other approach that I've missed that might be better? No idea as it's still not clear what you want to accomplish exactly. You are providing a lot of details, just not concrete ones that show how things are currently done and how you want to change that by adding Python to the mix. From myspamacct at earthlink.net Thu Oct 6 20:22:48 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 17:22:48 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: > Ah, that probably means you want separate interpreters, then. I hope not, unless I can get separate simultaneous interpreters out of one CPython.dll in one Windows process space. I'm guessing that I can't, but since I really don't know what I'm doing with Python yet, maybe I'm wrong there. > (for instance, the interactive > interpreter uses "_" in the builtins to store your last result, and if > you have multiple users sharing the builtins, they'd be trampling over > each other's underscores). I'm very new to Python and I suspect I don't completely understand what you mean by 'builtins' in this instance. Does Python have a non-module-scope namespace that contains things like _ ? Could you give me a pointer to any docs on this? If it does, do you have an opinion on how hard it might be to wrap it in some form of module (or "super module"?) scope? > But if you accept that this is a shared > environment, with the consequences thereof, you could have easy and > convenient sharing - and might even be able to have a tiny hack around > the interactive interpreter to make it use a different module name, > instead of __main__. So you could do something like this: > > #### User "Fred" #### > etc. > > So they would be *visibly* sharing state. This might be acceptable to > you and your users. It would be pretty easy to code up (with the > possible exception of the interactive mode). What you show in example is just about exactly what I want. Each user by default will have his own auto-generated (or maybe overridable by the user) namespace, so normally can work independently. But given that he can get the namespace handle for some other user (which might not be an interactive user, it might be a background process) then they can share data. The worry I have is any implicitly global variables, like underscore, and whatever else may exist in this global namespace. I'd really want some way to wrap those and make them unique for each user. I'm guessing that may mean hacking the code for CPython. I'd rather not do that, unless people would like the code back so I don't have my own unique branch that would break on each new release. But if that is what is needed, it is certainly something I could do. One concern I have is if two users both "import widget". Are they now sharing the widget namespace? I suspect they are, and that is probably undesirable. Or does Python have a hierarchical namespace concept, so that we would have fred.widget.ham and joe.widget.ham after fred and joe both import widget? Thanks again! Loren From myspamacct at earthlink.net Thu Oct 6 20:24:51 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 17:24:51 -0700 Subject: A newbie doubt on methods/functions calling References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> <231E1E51F31B492191BFEA2F66C9D62D@watson1> Message-ID: <2A2EA71E8F0D4545AADECF09E75DF1E9@watson1> Oops, apologies for replying to the wrong thread! Loren From myspamacct at earthlink.net Thu Oct 6 21:21:56 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 18:21:56 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> Message-ID: <681005F8873847FC9D289ADE66747798@watson1> > No idea as it's still not clear what you want to accomplish exactly. > You are providing a lot of details, just not concrete ones that show how > things are currently done and how you want to change that by adding > Python to the mix. Hum, this could take a lot of pages of text to describe how the existing environment works in detail, and some of it is considered proprietary so I couldn't say anyway. But I can try to give you some general concepts. Then once I have a description of the existing environment, I can say how I would like to extend it to handle Python. Also see my reply a few moments ago to Chris, where he siggested how to users might interact if they were sitting at terminals. Viruses are common today on commodity machines and operating systems. Mainframes in general avoid this by obscurity. Most people don't believe that mainframes still exist (this is what they have been taught in school) so they don't think of hacking them. Also, most people have probably never heard of a mainframe other than an IBM 370 with AIX, so have no idea what the environment looks like, so they can't hack it. Just to make life harder for the hackers, mainframes these days usually don't have anything resembling interactive shell access that you can physically reach without a badge to open a locked door or two. They typically don't access the internet directly. They are a backend database machine with a frontend web server and a firewall between it and the internet. The general way that banking and airline networks work is that they are private networks. They may be phone lines carrying RS-232 or similar traffic. They may be some sort of packet network that may or may not be IP based, and quite possibly does not use public network wires at all. When it comes to how terminals access the mainframes, there is a routing program on the mainframe that knows the name and address of every possible terminal that can access the system. A transaction is usually something like a 6 or 8 character uppercase word followed by positional character data. The routing program gets this message, looks up the terminal name, makes sure it is allwed to do this trnasaction, looks up th eprogram that will handle the transaction, and sends the termianl name and input data to that program. The program does whatever it needs to do, and sends the terminal name and a screenload of data back to the routing program. The routing program forwards the response data to the terminal. Web sites like Orbitz have web forms that make search or transaction requests. The web server takes the information from the web page, transforms it into one of these six character (or so) transaction requests, and pretends it is a terminal, and sends the 'screen data' to the mainframe. The mainframe processes the request from the "terminal" and sends back a screenload of data. The reply may be more than one screen, so there may be more transactions back and forth to get all the data. Then the CGI program running on the web server formats this into the web page response. Some mainframes may also be web servers. In this case the web server program on the mainframe will interact using the same text commands with the backend programs that handle the terminals, in many cases. Mainframes can have interactive terminals, possibly thousands of them. This is rare these days except for the programming machines. But the transaction termainls could be used as interactive terminals, by running a transaction that connects them to a PYTHON application. So there could be many users (thousands of them) all doing independent things in Python at once. There could be web page transaction servers that are talking to back-end programs that invoke Python commands or scripts. In fact the web page handler could appear to be a Python interpreter running a different script for each web page request. There are also nightly, weekly, monthly, year-end, and ad-hoc reports that are run on mainframes, typically to gether from the databases. Any or all of these reports could be Python scripts, or could be programs that somehow invoked snippits of Python code. Ok, that is a user-level operational view of a typical mainframe. Now for some lower level details of the particular mainframe of interest. Our mainframe is emulated in a single Windows program. Any given instance can have from 1 to 32 processors, and up to about 100 GW of RAM, and tens of thousands of disk drives online. It has an operating system of its own that can run tens of thousands of independent processoes ("stacks") at the same time. The OS does all of the sort of scheduling, IO, and communications things that an OS normally does. There are hundreds of system libraries available that programs can use to perform common functions. User programs don't have separate address spaces in the sense that you understand. Memory isn't paged. Effectively memory consists of objects that have handles and offsets within that handle. There are descriptors that have a handle and an offset and a character size, and operators that take descriptors and possibly lengths, and do things like add or multiply or more or compare data. Users create data by requesting handles from the OS, which allocates the memory and returns a handle. The handle is tracked with the owning stack, and will be deleted when the stack exits, if not depeted before then for some reason. Users can share data by passing descriptors between them. This is very similar to passing references in Python. If I implemented Python netively in this system, it would read source from a file (a terminal is a file) and compile bytecode, and put it in a code file, marked as PythonCode, as opposed to say CobolCode. The compiled code might then be executed automatically, or the code file might be made permanent on disk, and can be run some time later. When it is run, the byte code would be read into memory by the OS, and the OS would call the initial instruciton in the program. Of course the OS and all the other programs are being interpreted by the VM. When the VM sees the PythonCode, it will switch to the Python interpreter, and user execution will continue on the same stack, but now in Python. When the Pyton code exits back to the OS, we will switch to the OS code file, and also switch abck to the Newp interpreter. Since we exited the stack that had the Python local data, it will be deallocated. Since allocated arrays will have their "mom descriptors" in that same deallocated stack, they will also be deallocated, unless they are shared by some other stack that still exists. But the drawback with that, other than having to write a Python interpreter an dget it working and support it, is that I cna't call any existing libraries, since none of them are compiled for my mainframe architecture. And likely few of them would ever be. So that isn't a good approach. But I can use parts of the idea. The VM is a program running on Windows. The mainframe object code is just byte code. The VM has routines to handle every operator and each kind of Descriptor that they might validly use. One form of Descriptor is only used by the CALL instruction. It allows an "internal call" to a procedure that is part of the VM. Assuming that the VM linked to CPython, I could use one form of this to pass a text line to PyRun_SimpleString. Then I cna write a trivial program that will read from a terminal file and pass any received text to SimpleString, and somehow route any response back to the terminal file. If I knew I was only ever going to have one Python user at a time, this would be sufficient. But if I assume that I'm going to have two independent users calling SimpleSting (or one of the other invocation interfaces, like InteractiveLoop), then I need to start thinking about sandboxing, so that they don't accidentally trash each other's data. I assume that it is possible to compile Python down to bytecode and save that in a "code file" so that it can be executed later without having to be re-translated. Possibly if the user had a Python script on the mainframe that he wanted to compile, I could pass the text to CPython, and capture the generated bytecode and store it as a mainframe code file, rather than on the PC side. Then this could be executed just like any other mainframe program. When the VM saw a call into a PythonCode code file, it would in some way pass the byte code as a string to CPython and have it interpret it. Again, may PythonCode files might be running at once, and unless they deliberately wanted to share data, data sharing would have to be avoided. Well, that was very long winded, but hopefuly it gives you some slightidea of the environment and desired results. Loren From rosuav at gmail.com Thu Oct 6 21:22:10 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 12:22:10 +1100 Subject: Question on multiple Python users in one application In-Reply-To: <50935B8C00204F18BA2CC2EF9B78D806@watson1> References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> Message-ID: On Fri, Oct 7, 2016 at 11:03 AM, Loren Wilton wrote: > I've read that Python supports 'threads', and I'd assumed (maybe > incorrectly) that these were somewhat separate environments that could be > operating concurrently (modulo the GC lock). I assume that data can be > shared between the threads, and probably will share automatically if the > name of the item (or the reference) matches. But I'd also assumed that I > might be able to do something like start a thread and tell it that it is > working in "module user_42" or some such, so that by default it's data would > not be global. > > Assuming that I'm not completely off-base about using threads, I don't > really understand how to create a thread or destroy it. I'm also not > completely clear on how much useful data there is that is thread local, and > how much in the way of Python objects goes in the gloabl heap. > Threads don't really have much to do with namespacing and data sharing. The two are broadly orthogonal. Basically, threads in Python exist so that one function call can be running code while another one is blocked (eg on I/O). Other than that, they're just two "current stack location" markers, but they use the same namespaces, the same globals, etc. ChrisA From rosuav at gmail.com Thu Oct 6 21:41:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 12:41:12 +1100 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: On Fri, Oct 7, 2016 at 11:22 AM, Loren Wilton wrote: >> Ah, that probably means you want separate interpreters, then. > > > I hope not, unless I can get separate simultaneous interpreters out of one > CPython.dll in one Windows process space. I'm guessing that I can't, but > since I really don't know what I'm doing with Python yet, maybe I'm wrong > there. It is possible to do that, but based on the info you give in this post, I withdraw the recommendation for separate interpreters. >> (for instance, the interactive >> interpreter uses "_" in the builtins to store your last result, and if >> you have multiple users sharing the builtins, they'd be trampling over >> each other's underscores). > > > I'm very new to Python and I suspect I don't completely understand what you > mean by 'builtins' in this instance. Does Python have a non-module-scope > namespace that contains things like _ ? Could you give me a pointer to any > docs on this? In Python, there are three [1] namespaces: function locals, module globals, and built-ins. Function locals are exactly what you'd expect: one invocation of a function has one set of locals, and another invocation (incl recursion) has a completely separate set. Module-level names are called "globals", but they're not process-wide like in C; code in a different module has separate globals. Built-ins are for "standard" names, like range(), str(), len(), and so on. They are shared among all modules in a process. [1] There's also the class namespace, used while you're building a class. Amongst our namespaces are such diverse elements as....... > If it does, do you have an opinion on how hard it might be to wrap it in > some form of module (or "super module"?) scope? I wouldn't. It's generally immutable. What I'd advise is having some way to terminate Python and restart it (which will take care of any serious messes), and for the rest, just leave it shared. >> But if you accept that this is a shared >> environment, with the consequences thereof, you could have easy and >> convenient sharing - and might even be able to have a tiny hack around >> the interactive interpreter to make it use a different module name, >> instead of __main__. So you could do something like this: >> >> #### User "Fred" #### >> etc. >> >> So they would be *visibly* sharing state. This might be acceptable to >> you and your users. It would be pretty easy to code up (with the >> possible exception of the interactive mode). > > > What you show in example is just about exactly what I want. Each user by > default will have his own auto-generated (or maybe overridable by the user) > namespace, so normally can work independently. But given that he can get the > namespace handle for some other user (which might not be an interactive > user, it might be a background process) then they can share data. That would align very nicely with modules in Python. > The worry I have is any implicitly global variables, like underscore, and > whatever else may exist in this global namespace. I'd really want some way > to wrap those and make them unique for each user. I'm guessing that may mean > hacking the code for CPython. I'd rather not do that, unless people would > like the code back so I don't have my own unique branch that would break on > each new release. But if that is what is needed, it is certainly something I > could do. It's probably only the underscore, and that's only for interactive work. Seems to me it's not going to be too big a deal. > One concern I have is if two users both "import widget". Are they now > sharing the widget namespace? I suspect they are, and that is probably > undesirable. Or does Python have a hierarchical namespace concept, so that > we would have fred.widget.ham and joe.widget.ham after fred and joe both > import widget? Yes, they would. An import would be used in two ways: 1) Standard library modules, or third-party extension modules. It's 99.999% likely that sharing these won't be a problem. 2) Accessing another user's namespace in order to share data. Sharing these modules is the very mechanic of data sharing. Your users would have to be aware that they're sharing state. Off the top of my head, I can think of very few consequences of this, most of them having workarounds (for instance, they'd share PRNG state, but you can explicitly construct an RNG and have your own dedicated state; also, the decimal module has shared contexts, so you'd have to be careful if you ever want to change its parameters). The only real requirement would be a Python that doesn't do the usual thing with __main__, but instead has a special module name. This would be easy to do for script running, but might require a little more work for interactive mode. Worst case, you roll your own REPL using exec(). It wouldn't be too hard. I think this is doable. ChrisA From rosuav at gmail.com Thu Oct 6 21:48:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 12:48:43 +1100 Subject: Question on multiple Python users in one application In-Reply-To: <681005F8873847FC9D289ADE66747798@watson1> References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <681005F8873847FC9D289ADE66747798@watson1> Message-ID: On Fri, Oct 7, 2016 at 12:21 PM, Loren Wilton wrote: > The VM is a program running on Windows. The mainframe object code is just > byte code. The VM has routines to handle every operator and each kind of > Descriptor that they might validly use. One form of Descriptor is only used > by the CALL instruction. It allows an "internal call" to a procedure that is > part of the VM. Assuming that the VM linked to CPython, I could use one > form of this to pass a text line to PyRun_SimpleString. Then I cna write a > trivial program that will read from a terminal file and pass any received > text to SimpleString, and somehow route any response back to the terminal > file. The source code for SimpleString looks like this: int PyRun_SimpleStringFlags(const char *command, PyCompilerFlags *flags) { PyObject *m, *d, *v; m = PyImport_AddModule("__main__"); if (m == NULL) return -1; d = PyModule_GetDict(m); v = PyRun_StringFlags(command, Py_file_input, d, d, flags); if (v == NULL) { PyErr_Print(); return -1; } Py_DECREF(v); return 0; } If that had been your original plan, it's dead simple to enhance it to use per-user module names. Just do this same work, but substitute a different module name right at the beginning! Other extremely-high-level interface functions are similar. You should have no trouble making this embed work; and then it's just a matter of figuring out the bridge code between that and the rest of the VM. (Okay, so it's probably not going to be as easy as that made it sound, but still. Definitely plausible.) ChrisA From no.email at nospam.invalid Thu Oct 6 21:52:45 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 06 Oct 2016 18:52:45 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> Message-ID: <87vax5vsb6.fsf@nightsong.com> "Loren Wilton" writes: > I've read that Python supports 'threads', and I'd assumed (maybe > incorrectly) that these were somewhat separate environments that could > be operating concurrently (modulo the GC lock). I assume that data can > be shared between the threads, Threads all run in the same address space. Data "can be" shared is an understatement--it IS shared, so threads can interfere with each other very easily. You have to be quite careful to prevent that, and thread programming has a reputation for being difficult for that reason. > Would using threads help me do what I want? I don't think Python threads are the answer. You want a separate interpreter per user, which is annoying to do with CPython. Do you have concrete performance expectations about the sharing of data between interpreter sessions? Those old mainframes were very slow compared to today's machines, even counting Python's interpretation overhead. From no.email at nospam.invalid Thu Oct 6 21:58:39 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Thu, 06 Oct 2016 18:58:39 -0700 Subject: A newbie doubt on methods/functions calling References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> <231E1E51F31B492191BFEA2F66C9D62D@watson1> Message-ID: <87oa2wx6ls.fsf@nightsong.com> "Loren Wilton" writes: > strength of Python is that there are many existing 3rd party libraries > that do lots of useful things. Since a lot of them are distributed as > binaries, they would not work in this mainframe environment. Python libraries are usually available as source, either in Python or in C. From rosuav at gmail.com Thu Oct 6 22:03:55 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 13:03:55 +1100 Subject: Question on multiple Python users in one application In-Reply-To: <87vax5vsb6.fsf@nightsong.com> References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <87vax5vsb6.fsf@nightsong.com> Message-ID: On Fri, Oct 7, 2016 at 12:52 PM, Paul Rubin wrote: > "Loren Wilton" writes: >> I've read that Python supports 'threads', and I'd assumed (maybe >> incorrectly) that these were somewhat separate environments that could >> be operating concurrently (modulo the GC lock). I assume that data can >> be shared between the threads, > > Threads all run in the same address space. Data "can be" shared is an > understatement--it IS shared, so threads can interfere with each other > very easily. You have to be quite careful to prevent that, and thread > programming has a reputation for being difficult for that reason. To be fair, the sharing of data between threads is no different from other concerns about global state. The only thing threads change is that you can have multiple functions doing stuff at once. state = [1,2,3] def func(): state[1] += 5 # do work state[1] -= 5 This function would probably work in most single-threaded environments. (Proper use of try/finally would make it safer.) This "patch, operate, unpatch" model becomes dangerous in a threaded system... but it also becomes dangerous in any case of recursion. Or just calling a function from inside func() that cares about state. Or anything like that. Mutable globals must be strictly managed, Alice; unproductive ones should be *eliminated*. ChrisA From torriem at gmail.com Thu Oct 6 23:07:53 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 6 Oct 2016 21:07:53 -0600 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <681005F8873847FC9D289ADE66747798@watson1> Message-ID: <6e5b7c3e-9657-0be3-483d-bcd3f933df0f@gmail.com> On 10/06/2016 07:48 PM, Chris Angelico wrote: > If that had been your original plan, it's dead simple to enhance it to > use per-user module names. Just do this same work, but substitute a > different module name right at the beginning! Other > extremely-high-level interface functions are similar. You should have > no trouble making this embed work; and then it's just a matter of > figuring out the bridge code between that and the rest of the VM. I still don't understand why this has to be in the same process space as the VM. Wouldn't it be a lot simpler to create a simple RPC layer (all localhost of course) to interface between the VM and a Python server that spins up multiple processes or sessions? Kind of like how Python for a web server would work. If a request comes in-band from the client to the VM that necessitates handing off to Python, the RPC layer would do this. Just trying to think architecturally here. From mr.puneet.goyal at gmail.com Thu Oct 6 23:23:37 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Thu, 6 Oct 2016 20:23:37 -0700 (PDT) Subject: A newbie doubt on methods/functions calling In-Reply-To: References: Message-ID: <55db2057-d619-4b78-ad9b-a6a6daddeee8@googlegroups.com> Well I jump from TCL to Python. And found that it was very convenient to use Procs there. So I was looking for that luxury in Python. I am not trying to reinvent the wheel. I was just curious to know if there is any possibility to create a caller function in my way (TCL) where I can call python function/proc inside myProc. From myspamacct at earthlink.net Thu Oct 6 23:46:10 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 20:46:10 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <87vax5vsb6.fsf@nightsong.com> Message-ID: > From: "Paul Rubin" > I don't think Python threads are the answer. You want a separate > interpreter per user, which is annoying to do with CPython. Do you have > concrete performance expectations about the sharing of data between > interpreter sessions? Those old mainframes were very slow compared to > today's machines, even counting Python's interpretation overhead. Without separate context it sounds like Python threads are moderately useless except in special circumstances, and this certainly isn't one of those circumstances. As for performance, especially with shared data, it is an interesting question, but one not easily answered at the moment. The VM does a lot of interesting JIT type stuff, so gets performance that easily exceeds the most recent hardware implementation of the machine, which was only made a few years ago. Yes, a raw PC can do things faster (obviously). But people like the mainframe environment for the same reason people like Python over C: it does what they want, easier. They are willing to pay some performance for this -- but only some. They still have network response times to meet and need to get the monthend batch processing done before midnight. In our case, shared data is no slower to access than process-unique data. Well, maybe a little slower if you have to acquire and release a lock, because that takes a few nanoseconds or so. So I'd prefer that Python shared data access was not a lot slower than non-shared access. My impression is that all data is in a common heap, so a lock is required to access anything. I'd think that makes all data access equally slow or equally fast. I haven't found much data on multiple separate interpreters within a single process space. What exactly do you get that is unique to each interpreter? Separate heaps? Seperate GC locks? Separate process threads? I would guess maybe separate builtins. Does multiple interpreters make it harder to share data between the interpreters? Loren From random832 at fastmail.com Thu Oct 6 23:50:56 2016 From: random832 at fastmail.com (Random832) Date: Thu, 06 Oct 2016 23:50:56 -0400 Subject: A newbie doubt on methods/functions calling In-Reply-To: <231E1E51F31B492191BFEA2F66C9D62D@watson1> References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> <231E1E51F31B492191BFEA2F66C9D62D@watson1> Message-ID: <1475812256.2781845.748506905.7B32970B@webmail.messagingengine.com> On Thu, Oct 6, 2016, at 19:27, Loren Wilton wrote: > So I don't want to WRITE a Python interpreter for the actual mainframe > environment. I want to use an interpreter for an existing environment > (Windows) where there are already a lot of existing libraries. But > since a lot of the data to be analyzed is on the mainframe > environment, and the results would be wanted there too, I need to > extend the Python data access to the mainframe environment. Honestly, the best implementation strategy I can think of is to first implement a Python interpreter for the actual mainframe environment. Then invent an RPC layer that can semi-transparently bridge the two for when you want to call a module that only exists in the Windows environment (or call _from_ such a module back to an object/module that only exists in the mainframe environment), whether it's because it requires a Windows library or because you want the Windows python interpreter to do the heavy lifting because it's faster. Do you have C in the mainframe environment? From myspamacct at earthlink.net Fri Oct 7 00:06:19 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 21:06:19 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <87vax5vsb6.fsf@nightsong.com> Message-ID: > To be fair, the sharing of data between threads is no different from > other concerns about global state. The only thing threads change is > that you can have multiple functions doing stuff at once. > > state = [1,2,3] > > def func(): > state[1] += 5 > # do work > state[1] -= 5 It occurs to me that threads must have some state of their own. For instance, if I have two thread and they are both in that same function above, one thread needs to know that it is currently on the bytecode for the first statement, and the other needs to know that it is in the bytecode for some of the # do work. In other words, each thread has a current instruction pointer. If the thread has a current instruction pointer, then it must also know what function it is currently in, and therfore have a pointer (or some such) to the current function locals: def func1(): ham = 0.0; ham += 1; // thread 1 here def func2(): ham = 0.0; ham -= 1; // thread 2 here In my C++ understanding of threads, those manipulations on ham won't interact. I would hope this is also true in Python? If I'm right that a thread knows what function it is in, and therefore has function-level state access to the current function, then I'd think that it also has to have module level state. After all, functions occur inside modules, or at least they can: spam.py: ham = 0.0; def func(): ham += 1; // thread 1 here eggs.py: ham = 0.0; def func(): ham -= 1; // thread 2 here Here I would again hope that the simultaneous threads would not cause collisions on ham, since (I think?) there should be spam.ham and eggs.ham, which would be two separate things. The worry I can see is if the CPython interpreter has taken some shortcuts and has global variables for the pointers to the module globals and the funciton locals. In that case, if two threads aren't in the same function I can see all hell breaking loose pretty easily. Loren From mr.puneet.goyal at gmail.com Fri Oct 7 00:13:25 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Thu, 6 Oct 2016 21:13:25 -0700 (PDT) Subject: A newbie doubt on methods/functions calling In-Reply-To: References: Message-ID: <89c37162-1cca-4861-974e-9afbb478314b@googlegroups.com> Let me rephrase my question in other way. class myClass: def __init__(self, var): self.var = var myObj = myClass(abc) # I am calling instance with function name and arguments myObj func1 arg1 arg2 Can i associate any function like __init__ with instance ? Means if I just use instance anywhere as above, it calls that function in the class and take everything as argument which I mentioned after instance? Something like below class myClass: def __call__(self, args): # Then I parse the args here and call function internally self.func1(arg1, arg2) From myspamacct at earthlink.net Fri Oct 7 00:14:16 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 21:14:16 -0700 Subject: Question on multiple Python users in one application References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <681005F8873847FC9D289ADE66747798@watson1> <6e5b7c3e-9657-0be3-483d-bcd3f933df0f@gmail.com> Message-ID: > I still don't understand why this has to be in the same process space as > the VM. Wouldn't it be a lot simpler to create a simple RPC layer (all > localhost of course) to interface between the VM and a Python server > that spins up multiple processes or sessions? Kind of like how Python > for a web server would work. If a request comes in-band from the client > to the VM that necessitates handing off to Python, the RPC layer would > do this. Assuming that all Python usages are single procedure calls, then all of the parameters can be marshalled and passed to another process. I suppose it may or may not be possible to modify the internals of structured parameters (arrays, lists, dictionaries, etc) and return the modified values, depending on how marshalling works. And it may or may not be fast. If the Python code wants to make reverse calls back into the VM environment (and it may well want to do this, for instance for database access) then it is still possible but starts to get trickier, and again possibly slower. Depending on use cases, any marshalling slowness might not matter at all. Or it might matter a great deal. Unfortunately I don't yet have any clearly defined use cases from the people that should be thinking about that. So at the moment I'm trying to follow the original concept spec as closely as possible, which is "integrate Python in the VM environment." Loren From myspamacct at earthlink.net Fri Oct 7 00:20:49 2016 From: myspamacct at earthlink.net (Loren Wilton) Date: Thu, 6 Oct 2016 21:20:49 -0700 Subject: A newbie doubt on methods/functions calling References: <57f68a5b$0$1606$c3e8da3$5496439d@news.astraweb.com> <231E1E51F31B492191BFEA2F66C9D62D@watson1> <1475812256.2781845.748506905.7B32970B@webmail.messagingengine.com> Message-ID: <5A0DCC300461444884D9CA64FAC673E3@watson1> > Honestly, the best implementation strategy I can think of is to first > implement a Python interpreter for the actual mainframe environment. > Then invent an RPC layer that can semi-transparently bridge the two for > when you want to call a module that only exists in the Windows > environment (or call _from_ such a module back to an object/module that > only exists in the mainframe environment), whether it's because it > requires a Windows library or because you want the Windows python > interpreter to do the heavy lifting because it's faster. > > Do you have C in the mainframe environment? Essentially the answer is "no". There is a thing that calls itself a C compiler, but it can only compile truely trivial programs, and then only after a great deal of manual hacking on the source code to change the syntax to what this compiler likes. There is no equivalent of "make". I'd be better off starting with a Python interpreter in JaveScript or the like, if I wanted to do a transliteration that would actually work. I'm trying to avoid this, it would be months of work. Loren From nomail at com.invalid Fri Oct 7 01:01:04 2016 From: nomail at com.invalid (ast) Date: Fri, 7 Oct 2016 07:01:04 +0200 Subject: static, class and instance methods (Reposting On Python-List Prohibited) In-Reply-To: References: <57f5f503$0$24766$426a74cc@news.free.fr> Message-ID: <57f72c13$0$7975$426a74cc@news.free.fr> "Gregory Ewing" a ?crit dans le message de news:e5mgi9Fp1bsU1 at mid.individual.net... > Lawrence D?Oliveiro wrote: >> >> Every function is already a descriptor. > > Which you can see with a simple experiment: > > >>> def f(self): > ... print("self =", self) > ... I thought yesterday that every thing was clear. But I have a doubt now with the following line: > >>> g = f.__get__(17, None) The signature of __get__ method is __get__ (self, inst, owner) so once again the first parameter is filled automatically. Is method __get__ itself a descriptor with an attribute __get__ to perform the operation ? Hum, it would be endless ... vars(f.__get__) TypeError: vars() argument must have __dict__ attribute no, so how does it work here ? > >>> g > > >>> g() > self = 17 > > -- > Greg From rosuav at gmail.com Fri Oct 7 01:10:59 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 16:10:59 +1100 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <87vax5vsb6.fsf@nightsong.com> Message-ID: On Fri, Oct 7, 2016 at 3:06 PM, Loren Wilton wrote: > If the thread has a current instruction pointer, then it must also know what > function it is currently in, and therfore have a pointer (or some such) to > the current function locals: > > def func1(): > ham = 0.0; > ham += 1; // thread 1 here > > def func2(): > ham = 0.0; > ham -= 1; // thread 2 here > > In my C++ understanding of threads, those manipulations on ham won't > interact. I would hope this is also true in Python? > > If I'm right that a thread knows what function it is in, and therefore has > function-level state access to the current function, then I'd think that it > also has to have module level state. After all, functions occur inside > modules, or at least they can: > > spam.py: > ham = 0.0; > def func(): > ham += 1; // thread 1 here > > eggs.py: > ham = 0.0; > def func(): > ham -= 1; // thread 2 here > > Here I would again hope that the simultaneous threads would not cause > collisions on ham, since (I think?) there should be spam.ham and eggs.ham, > which would be two separate things. Correct on both counts. Threads executing different functions cannot trample on each other's locals. Even if they're executing different invocations of the same function, they won't trample on locals. With your module example (assuming you add "global ham" to each function), they would be fine too; however, if two threads are in two invocations of the same function, it's possible for them to overwrite each other's change. (It'd require pinpoint timing, but it could happen.) So if you're going to use threads, you have to not use mutable global state. That's honestly not a difficult rule to follow; most programmers should have a healthy caution around mutable global state anyway, regardless of threads. ChrisA From lists at ozindfw.net Fri Oct 7 01:30:10 2016 From: lists at ozindfw.net (Oz-in-DFW) Date: Fri, 7 Oct 2016 00:30:10 -0500 Subject: Doubled backslashes in Windows paths Message-ID: I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (Intel)] on Windows 7 I'm trying to write some file processing that looks at file size, extensions, and several other things and I'm having trouble getting a reliably usable path to files. The problem *seems* to be doubled backslashes in the path, but I've read elsewhere that this is just an artifact of the way the interpreter displays the strings. I'm getting an error message on an os.path.getsize call; Path: - "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" - Traceback (most recent call last): File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in if os.path.getsize(path)>10000: File "C:\Python32\lib\genericpath.py", line 49, in getsize return os.stat(filename).st_size WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: '"C:\\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b845386.jpg"' >From (snippet) path = '"'+dirpath+name+'"' path = os.path.normpath(path) print("Path: -",path,"-") if os.path.getsize(path)>10000: print("Path: ",path," Size: ",os.path.getsize(dirpath+name)) but if I manually use a console window and cut and paste the path I print, it works; C:\>dir "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" Volume in drive C is Windows7_OS Directory of C:\Users\Rich\Desktop\2B_Proc 10/03/2016 08:35 AM 59,200 2307e60da6451986dd8d23635b845386.jpg 1 File(s) 59,200 bytes 0 Dir(s) 115,857,260,544 bytes free So the file is there and the path is correct. I'm adding quotes to the path to deal with directories and filenames that have spaces in them. If I drop the quotes, everything works as I expect *until* I encounter the first file/path with spaces. I'll happily RTFM, but I need some hints as to which FM to R -- mailto:oz at ozindfw.net Oz POB 93167 Southlake, TX 76092 (Near DFW Airport) From dieter at handshake.de Fri Oct 7 02:24:45 2016 From: dieter at handshake.de (dieter) Date: Fri, 07 Oct 2016 08:24:45 +0200 Subject: segfault using shutil.make_archive References: Message-ID: <87wphkk76a.fsf@handshake.de> Tim writes: > I need to zip up a directory that's about 400mb. > I'm using shutil.make_archive and I'm getting this response: > > Segmentation fault: 11 (core dumped) > > The code is straightforward (and works on other, smaller dirs): > > shutil.make_archive(os.path.join(zip_dir, zname), 'zip', tgt_dir) > > I guess I could drop the convenience of make_archive and use zipfile but that seems to be exactly what make_archive does. >From your description (above and in other messages of this thread), I guess, that the cause is a (running out of) memory problem. Memory allocations are frequent (in many places) and they fail rarely. Therefore, there is quite a high probability that some of those allocations fail to check that the allocation has been successful. If this happens (and the allocation fails), then a "SIGSEGV" is almost certain. I would try to move the task on a system with a larger address space (more RAM, more swap space, maybe a 64 bit system). It may succeed there. From gandalf at shopzeus.com Fri Oct 7 02:37:33 2016 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Fri, 7 Oct 2016 08:37:33 +0200 Subject: Python 3.5 amd64 and win32service In-Reply-To: References: <9964f2f3-9e8e-4928-db88-bf350b5e102e@shopzeus.com> <81bcab97-3b9f-c8a2-f3da-3c06bb965831@shopzeus.com> <856e35f6-0545-602d-5fe8-c6a94843e37e@shopzeus.com> <6412c001-fb36-5ded-919c-58f7fe87d26f@shopzeus.com> Message-ID: <93a31325-dc2c-8fa2-34f9-4d93409d69aa@shopzeus.com> >> "C:\Users\Laci\AppData\Local\Programs\Python\Python35\lib\site-packages\win32\PythonService.exe" > I wanted you to run the above executable, not python.exe. If it fails > you'll get more information about why it's failing when run directly > then when the service controller runs it. Since you're using a > per-user installation of Python 3.5, possibly it can't find > python35.dll. This is something I cannot understand. This is what happens when I start it from the command line, as a normal user: C:\>"C:\Users\Laci\AppData\Local\Programs\Python\Python35\lib\site-packages\win32\PythonService.exe" -debug TestService Debugging service TestService - press Ctrl+C to stop. Info 0x40001002 - The TestService service has started. *Meanwhile, the service manager shows that the service is not started*. Then I press Ctrl+C several times and I see this: C:\>"C:\Users\Laci\AppData\Local\Programs\Python\Python35\lib\site-packages\win32\PythonService.exe" -debug TestService Debugging service TestService - press Ctrl+C to stop. Info 0x40001002 - The TestService service has started. Stopping debug service. Stopping debug service. Stopping debug service. Stopping debug service. Stopping debug service. But the process is not closed. I had to terminate pythonservice.exe from task manager. Next thing I tried is to start it as an administrator: there is no difference. On the console it says that it has been started, but in service manager it appears to be stopped. I got no messages about missing DLL files at all. The very same MWE works perfectly on a new virtual machine with Python 3.5.2 amd64 + pypiwin32. Any idea what could be wrong? From greg.ewing at canterbury.ac.nz Fri Oct 7 03:35:52 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 07 Oct 2016 20:35:52 +1300 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> Message-ID: Loren Wilton wrote: > I've read that Python supports 'threads', and I'd assumed (maybe > incorrectly) that these were somewhat separate environments that could > be operating concurrently (modulo the GC lock). Not really. Python threads are just a thin wrapper around OS threads, and don't provide any more isolation than the OS's native threads do. All the threads in a Python process share one set of imported modules, and therefore one set of global data. For example, there will only be one sys module, and thus things like sys.stdin and sys.stdout will be shared between threads. If one user pointed his sys.stdin somewhere else, it would affect all the others. There is *some* support in CPython for multiple interpeters within a process, but I don't know how much isolation that provides, or whether it has even kept up enough with all the changes over the years to still be usable. I know that subinterpreters are not fully isolated, e.g. built-in constants and type objects are shared. There is also the GIL to consider, which will prevent more than one thread from running Python code at the same time. Does Python really need access to data in the VM's memory, or just to data in its disk files, etc? If the latter, you might not need to run Python in the same process as the VM as long as you can arrange some kind of I/O bridge. Does the OS running on the VM have any notion of a remote file access protocol a la NFS that an external Python process could use to access data? -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 7 04:20:39 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 07 Oct 2016 21:20:39 +1300 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> Message-ID: Loren Wilton wrote: > One concern I have is if two users both "import widget". Are they now sharing > the widget namespace? I suspect they are, and that is probably undesirable. Yes, they will be, unless you use sub-interpreters. I've just been looking at the current docs for sub-interpreters, and assuming they're not lying too wildly, sub-interpreters might be what you're after. https://docs.python.org/3/c-api/init.html An excerpt: PyThreadState* Py_NewInterpreter() Create a new sub-interpreter. This is an (almost) totally separate environment for the execution of Python code. In particular, the new interpreter has separate, independent versions of all imported modules, including the fundamental modules builtins, __main__ and sys. The table of loaded modules (sys.modules) and the module search path (sys.path) are also separate. The new environment has no sys.argv variable. It has new standard I/O stream file objects sys.stdin, sys.stdout and sys.stderr (however these refer to the same underlying file descriptors). -- Greg From steve+python at pearwood.info Fri Oct 7 04:50:56 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 19:50:56 +1100 Subject: static, class and instance methods References: <57f5f503$0$24766$426a74cc@news.free.fr> <57f72c13$0$7975$426a74cc@news.free.fr> Message-ID: <57f761f1$0$1596$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 04:01 pm, ast wrote: > > "Gregory Ewing" a ?crit dans le message de > news:e5mgi9Fp1bsU1 at mid.individual.net... >> Lawrence D?Oliveiro wrote: >>> >>> Every function is already a descriptor. >> >> Which you can see with a simple experiment: >> >> >>> def f(self): >> ... print("self =", self) >> ... > > I thought yesterday that every thing was clear. But I have > a doubt now with the following line: > >> >>> g = f.__get__(17, None) > > The signature of __get__ method is __get__ (self, inst, owner) so > once again the first parameter is filled automatically. > Is method __get__ itself a descriptor with an attribute __get__ to perform > the operation ? Hum, it would be endless ... No it is not: py> def f(): pass ... py> hasattr(f.__get__, '__get__') False So what kind of thing is f.__get__? f itself is a function, which looks like this: py> f but f.__get__ is something different: py> f.__get__ "Method-wrapper" is an undocumented internal implementation detail. It isn't part of the Python language, just part of the runtime implementation. If we use Jython instead, we see something different: steve at orac:~$ jython Jython 2.5.1+ (Release_2_5_1, Aug 4 2010, 07:18:19) [OpenJDK Server VM (Sun Microsystems Inc.)] on java1.6.0_31 Type "help", "copyright", "credits" or "license" for more information. >>> def f(): pass ... >>> f >>> f.__get__ So the Python language doesn't make any promises about what sort of thing __get__ will be, only that it exists and is callable. Anything I say here is an implementation detail: it may change in the future, it may have already changed, and different Python implementations may do things differently. Back to CPython: f.__get__ is a method-wrapper. That gives us a hint that although __get__ itself isn't a descriptor, it is a thing returned by something which is a descriptor. Remember that all functions are instances of a built-in class, FunctionType: py> from types import FunctionType py> assert type(f) is FunctionType py> vars(FunctionType)['__get__'] It is *that* "slot wrapper" thing which is itself a descriptor: py> vars(FunctionType)['__get__'].__get__ and it returns f.__get__: py> vars(FunctionType)['__get__'].__get__(f) which returns a method: py> class X(object): pass ... py> x = X() py> vars(FunctionType)['__get__'].__get__(f)(x, X) > -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Oct 7 05:17:54 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 7 Oct 2016 20:17:54 +1100 Subject: segfault using shutil.make_archive In-Reply-To: <87wphkk76a.fsf@handshake.de> References: <87wphkk76a.fsf@handshake.de> Message-ID: On Fri, Oct 7, 2016 at 5:24 PM, dieter wrote: > Memory allocations are frequent (in many places) and they fail rarely. > Therefore, there is quite a high probability that some of those > allocations fail to check that the allocation has been successful. > If this happens (and the allocation fails), then a "SIGSEGV" is > almost certain. Which, BTW, doesn't stop them from being bugs. If this can be pinned down, it definitely warrants a bugs.python.org issue. ChrisA From stephen_tucker at sil.org Fri Oct 7 05:26:55 2016 From: stephen_tucker at sil.org (Stephen Tucker) Date: Fri, 7 Oct 2016 10:26:55 +0100 Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: Hi Oz, This might only be tangential to your actual issue, but, there again, it might be the tiny clue that you actually need. In Python, I use raw strings and single backslashes in folder hierarchy strings to save the problem of the backslash in ordinary strings. Even with this policy, however, there is a slight "gotcha": Although it is claimed that r" ... " suspends the escape interpretation of the backslash in the string, a raw string cannot end with a backslash: myraw = "\a\b\" provokes the error message: SyntaxError: EOL while scanning string literal To see how well this approach deals with folder hierarchies with spaces in their names, I created the following file: c:\Python27\ArcGIS10.4\Lib\idlelib\sjt\sp in\test.txt Note the space in the folder name sp in . In IDLE, I then issued the following statement: infile= open (r"c:\Python27\ArcGIS10.4\Lib\idlelib\sjt\sp in\test.txt", "r") Note that I didn't need to get enclosing quotes into my folder hierarchy string. It begins with c and ends with t . The statement worked as you might expect, and granted me access to the lines in the file. It seems that it is only necessary to enclose a folder hierarchy string in quotes when defining the string in a situation in which spaces would otherwise be interpreted as terminators. The classis case of this is in a command with parameters in a batch file. If the string has been defined before it is presented to the Windows Command interpreter, the spaces are accepted as part of it without the need then of enclosing quotes. Hope this helps. Yours, Stephen Tucker. On Fri, Oct 7, 2016 at 6:30 AM, Oz-in-DFW wrote: > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > v.1900 32 bit (Intel)] on Windows 7 > > I'm trying to write some file processing that looks at file size, > extensions, and several other things and I'm having trouble getting a > reliably usable path to files. > > The problem *seems* to be doubled backslashes in the path, but I've read > elsewhere that this is just an artifact of the way the interpreter > displays the strings. > > I'm getting an error message on an os.path.getsize call; > > Path: - > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" - > Traceback (most recent call last): > File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in > if os.path.getsize(path)>10000: > File "C:\Python32\lib\genericpath.py", line 49, in getsize > return os.stat(filename).st_size > WindowsError: [Error 123] The filename, directory name, or volume > label syntax is incorrect: > '"C:\\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b8453 > 86.jpg"' > > From (snippet) > > path = '"'+dirpath+name+'"' > path = os.path.normpath(path) > print("Path: -",path,"-") > if os.path.getsize(path)>10000: > print("Path: ",path," Size: > ",os.path.getsize(dirpath+name)) > > but if I manually use a console window and cut and paste the path I > print, it works; > > C:\>dir > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" > Volume in drive C is Windows7_OS > > Directory of C:\Users\Rich\Desktop\2B_Proc > > 10/03/2016 08:35 AM 59,200 > 2307e60da6451986dd8d23635b845386.jpg > 1 File(s) 59,200 bytes > 0 Dir(s) 115,857,260,544 bytes free > > So the file is there and the path is correct. I'm adding quotes to the > path to deal with directories and filenames that have spaces in them. > If I drop the quotes, everything works as I expect *until* I encounter > the first file/path with spaces. > > I'll happily RTFM, but I need some hints as to which FM to R > > -- > mailto:oz at ozindfw.net > Oz > POB 93167 > Southlake, TX 76092 (Near DFW Airport) > > > > -- > https://mail.python.org/mailman/listinfo/python-list > From __peter__ at web.de Fri Oct 7 05:27:00 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 07 Oct 2016 11:27 +0200 Subject: Doubled backslashes in Windows paths References: Message-ID: Oz-in-DFW wrote: > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > v.1900 32 bit (Intel)] on Windows 7 > > I'm trying to write some file processing that looks at file size, > extensions, and several other things and I'm having trouble getting a > reliably usable path to files. > > The problem *seems* to be doubled backslashes in the path, but I've read > elsewhere that this is just an artifact of the way the interpreter > displays the strings. > > I'm getting an error message on an os.path.getsize call; > > Path: - > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" - > Traceback (most recent call last): > File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in > if os.path.getsize(path)>10000: > File "C:\Python32\lib\genericpath.py", line 49, in getsize > return os.stat(filename).st_size > WindowsError: [Error 123] The filename, directory name, or volume > label syntax is incorrect: > '"C: \\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b845386.jpg"' > > From (snippet) > > path = '"'+dirpath+name+'"' > path = os.path.normpath(path) > print("Path: -",path,"-") > if os.path.getsize(path)>10000: > print("Path: ",path," Size: > ",os.path.getsize(dirpath+name)) > > but if I manually use a console window and cut and paste the path I > print, it works; > > C:\>dir > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" > Volume in drive C is Windows7_OS > > Directory of C:\Users\Rich\Desktop\2B_Proc > > 10/03/2016 08:35 AM 59,200 > 2307e60da6451986dd8d23635b845386.jpg > 1 File(s) 59,200 bytes > 0 Dir(s) 115,857,260,544 bytes free > > So the file is there and the path is correct. I'm adding quotes to the > path to deal with directories and filenames that have spaces in them. > If I drop the quotes, everything works as I expect *until* I encounter > the first file/path with spaces. You have to omit the extra quotes. That the non-working path has spaces in it is probably not the cause of the problem. If a string *literal* is used, e. g. "C:\Path\test file.txt" there may be combinations of the backslash and the character that follows that are interpreted specially -- in the example \P is just \ followed by a P whereas \t is a TAB (chr(9)): >>> print("C:\Path\test file.txt") C:\Path est file.txt To fix the problem either use forward slashes (which are understood by Windows, too) or or duplicate all backslashes, >>> print("C:\\Path\\test file.txt") C:\Path\test file.txt or try the r (for "raw string") prefix: >>> print(r"C:\Path\test file.txt") C:\Path\test file.txt This applies only to string literals in Python source code; if you read a filename from a file or get it as user input there is no need to process the string: >>> input("Enter a file name: ") Enter a file name: C:\Path\test file.txt 'C:\\Path\\test file.txt' > I'll happily RTFM, but I need some hints as to which FM to R > https://docs.python.org/3.5/tutorial/introduction.html#strings https://docs.python.org/3.5/reference/lexical_analysis.html#string-and-bytes-literals PS: Unrelated, but a portable way to combine paths is os.path.join() which will also insert the platform specific directory separator if necessary: >>> print(os.path.join("foo", "bar\\", "baz")) foo\bar\baz From steve+python at pearwood.info Fri Oct 7 06:46:03 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 07 Oct 2016 21:46:03 +1100 Subject: Doubled backslashes in Windows paths References: Message-ID: <57f77ced$0$1617$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 04:30 pm, Oz-in-DFW wrote: > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > v.1900 32 bit (Intel)] on Windows 7 > > I'm trying to write some file processing that looks at file size, > extensions, and several other things and I'm having trouble getting a > reliably usable path to files. > > The problem *seems* to be doubled backslashes in the path, but I've read > elsewhere that this is just an artifact of the way the interpreter > displays the strings. Indeed. Why don't you show us the actual path you use? You show us this snippet: > path = '"'+dirpath+name+'"' > path = os.path.normpath(path) > print("Path: -",path,"-") > if os.path.getsize(path)>10000: > print("Path: ",path," Size: > ",os.path.getsize(dirpath+name)) but apparently dirpath and name are secrets, because you don't show us what they are *wink* However, using my awesome powers of observation *grin* I see that your filename (whatever it is) begins and ends with quotation marks. So instead of having a file name like: C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg you actually have a file name like: "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" In this example, the quotation marks " at the beginning and end are *NOT* the string delimiters, they are the first and last characters in the string. Which might explain why Windows complains about the label: > WindowsError: [Error 123] The filename, directory name, or volume > label syntax is incorrect: That's because "C: is an illegal volume label (disk name? I'm not really a Windows user, and I'm not quite sure what the correct terminology here would be). So I suggest you change your code from this: > path = '"'+dirpath+name+'"' > path = os.path.normpath(path) to this instead: path = dirpath + name path = os.path.normpath(path) > So the file is there and the path is correct. I'm adding quotes to the > path to deal with directories and filenames that have spaces in them. The os.path and os functions do not need you to escape the file name with quotes to handle spaces. It is very likely that if you call out to another Windows program using the os.system() call you will need to worry about escaping the spaces, but otherwise, you don't need them. > If I drop the quotes, everything works as I expect Indeed. > *until* I encounter the first file/path with spaces. And what happens then? My guess is that you get a different error, probably in a different place. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From daiyueweng at gmail.com Fri Oct 7 07:38:03 2016 From: daiyueweng at gmail.com (Daiyue Weng) Date: Fri, 7 Oct 2016 12:38:03 +0100 Subject: default argument value is mutable Message-ID: Hi, I declare two parameters for a function with default values [], def one_function(arg, arg1=[], arg2=[]): PyCharm warns me: Default argument value is mutable, what does it mean? and how to fix it? cheers From jeanmichel at sequans.com Fri Oct 7 07:55:30 2016 From: jeanmichel at sequans.com (jmp) Date: Fri, 07 Oct 2016 13:55:30 +0200 Subject: default argument value is mutable In-Reply-To: References: Message-ID: On 10/07/2016 01:38 PM, Daiyue Weng wrote: > Hi, I declare two parameters for a function with default values [], > > def one_function(arg, arg1=[], arg2=[]): > > PyCharm warns me: > > Default argument value is mutable, > > what does it mean? and how to fix it? > > cheers > You'll run into this bug def foo(a=[]): a.append('item') print a foo() foo() ['item'] ['item', 'item'] #what ?? 2 'item' ?? default argument are evaluated when defining the function, meaning the list you bound to the default argument is shared by all the function calls. Easily fixed: def foo(a=None): a = [] if a is None else a print a foo() foo() ['item'] ['item'] So the rule of thumb for default argument value is "No mutable" Cheers, Jm note : python 2.7 code From nomail at com.invalid Fri Oct 7 07:59:37 2016 From: nomail at com.invalid (ast) Date: Fri, 7 Oct 2016 13:59:37 +0200 Subject: default argument value is mutable In-Reply-To: References: Message-ID: <57f78e2e$0$3324$426a74cc@news.free.fr> "Daiyue Weng" a ?crit dans le message de news:mailman.208.1475840291.30834.python-list at python.org... > Hi, I declare two parameters for a function with default values [], > > def one_function(arg, arg1=[], arg2=[]): > > PyCharm warns me: > > Default argument value is mutable, > > what does it mean? and how to fix it? > > cheers You could ignore this warning if you know what you are doing, imho. The default value being mutable it is possible to change it inside the function. eg: def test(x=[0]): ... print(x) ... x[0] += 1 >>> test() [0] >>> test() [1] From nomail at com.invalid Fri Oct 7 08:07:10 2016 From: nomail at com.invalid (ast) Date: Fri, 7 Oct 2016 14:07:10 +0200 Subject: default argument value is mutable In-Reply-To: References: Message-ID: <57f78ff2$0$19773$426a74cc@news.free.fr> "jmp" a ?crit dans le message de news:mailman.209.1475841371.30834.python-list at python.org... > On 10/07/2016 01:38 PM, Daiyue Weng wrote: > > So the rule of thumb for default argument value is "No mutable" > > Cheers, It can be used to store some variables from one call of a function to an other one. def test( _store={'x':0}): x = _store['x'] ..... do some stuff _store['x'] = x From bc at freeuk.com Fri Oct 7 08:39:39 2016 From: bc at freeuk.com (BartC) Date: Fri, 7 Oct 2016 13:39:39 +0100 Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: On 07/10/2016 06:30, Oz-in-DFW wrote: > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > v.1900 32 bit (Intel)] on Windows 7 > > I'm trying to write some file processing that looks at file size, > extensions, and several other things and I'm having trouble getting a > reliably usable path to files. > > The problem *seems* to be doubled backslashes in the path, but I've read > elsewhere that this is just an artifact of the way the interpreter > displays the strings. > > I'm getting an error message on an os.path.getsize call; > > Path: - > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" - > Traceback (most recent call last): > File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in > if os.path.getsize(path)>10000: > File "C:\Python32\lib\genericpath.py", line 49, in getsize > return os.stat(filename).st_size > WindowsError: [Error 123] The filename, directory name, or volume > label syntax is incorrect: > '"C:\\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b845386.jpg"' I tried to recreate this error and it seems the getsize function doesn't like quotes in the path. Whether \ is correctly written as \\ in a string literal, or a raw string is used with the r prefix and a single \, or a \ has been put into path by any other means, then these will still be displayed as \\ in the error message, which is strange. The error handler is expanding \ characters to \\. But the main error appears to be due to the presence of quotes, whether at each end, or inside the path, enclosing an element with spaces for example. Try using len(path)>10000 instead; it might be near enough (the 10000 sounds arbitrary anyway). -- Bartc From jeanmichel at sequans.com Fri Oct 7 08:48:00 2016 From: jeanmichel at sequans.com (jmp) Date: Fri, 07 Oct 2016 14:48:00 +0200 Subject: default argument value is mutable In-Reply-To: <57f78ff2$0$19773$426a74cc@news.free.fr> References: <57f78ff2$0$19773$426a74cc@news.free.fr> Message-ID: On 10/07/2016 02:07 PM, ast wrote: > > "jmp" a ?crit dans le message de > news:mailman.209.1475841371.30834.python-list at python.org... >> On 10/07/2016 01:38 PM, Daiyue Weng wrote: > >> >> So the rule of thumb for default argument value is "No mutable" >> >> Cheers, > > > It can be used to store some variables from one call of > a function to an other one. > > def test( _store={'x':0}): > > x = _store['x'] > ..... do some stuff > _store['x'] = x For personal dirty scripts, possibly, for all other situations, never. Especially since there's nothing in the code above that cannot be solved using standard idioms . That is if you care about anyone reading your code ;) jm From eryksun at gmail.com Fri Oct 7 09:00:38 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 7 Oct 2016 13:00:38 +0000 Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: On Fri, Oct 7, 2016 at 9:27 AM, Peter Otten <__peter__ at web.de> wrote: > To fix the problem either use forward slashes (which are understood by > Windows, too) Using forward slash in place of backslash is generally fine, but you need to be aware of common exceptions, such as the following: (1) Paths with forward slashes aren't generally accepted in command lines. (2) Forward slash is just another name character in the kernel object namespace. Thus "Global/Spam" is a local name with a slash in it, while r"Global\Spam" is a "Spam" object created in the global r"\BaseNamedObjects" directory because "Global" is a local object symbolic link to the global r"\BaseNamedObjects". (3) Extended paths (i.e. paths prefixed by \\?\) bypass the user-mode path normalization that replaces slash with backslash. Such paths cannot use forward slashes to delimit filesystem path components. Given (2), forward slash potentially could be parsed as part of a filename. Fortunately, slash is a reserved character in all Microsoft filesystems. Thus "dir1/dir2" is an invalid 9-character name. You may ask why Windows doesn't hard-code replacing slash with backslash, even for extended paths. Some filesystems may allow slash in names, or for some other purpose (no that I know of any). Also, device names may contain slashes, just like any other Windows object name. For example, let's define a DOS pseudo-device (really an object symbolic link) named "Eggs/Spam" to reference Python's installation directory: import os, sys, ctypes kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) device_name = 'Eggs/Spam' target_path = os.path.dirname(sys.executable) kernel32.DefineDosDeviceW(0, device_name, target_path) >>> os.listdir(r'\\?\Eggs/Spam\Doc') ['python352.chm'] Note that the final component, "Doc", must be delimited by a backslash. If it instead used slash, Windows would look for a device named "Eggs/Spam/Doc". For example: >>> os.listdir(r'\\?\Eggs/Spam/Doc') Traceback (most recent call last): File "", line 1, in FileNotFoundError: [WinError 3] The system cannot find the path specified: '\\\\?\\Eggs/Spam/Doc' For the case of r"\\?\Eggs/Spam\Doc/python352.chm", parsing the path succeeds up to the last component, "Doc/python352.chm", which is an invalid name: >>> os.stat(r'\\?\Eggs/Spam\Doc/python352.chm') Traceback (most recent call last): File "", line 1, in OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\Eggs/Spam\\Doc/python352.chm' Windows reparses the object symbolic link "Eggs/Spam" to its target path. In my case that's r"\??\C:\Program Files\Python35". The "C:" drive in my case is an object symbolic link to r"\Device\HarddiskVolume2" (typical for Windows 10), so the path gets reparsed as r"\Device\HarddiskVolume2\Program Files\Python35\Doc/python352.chm". The volume device "HarddiskVolume2" contains an NTFS filesystem, so the NTFS driver parses the remaining path, r"\Program Files\Python35\Doc/python352.chm", and rejects "Doc/python352.chm" as an invalid name. From steve+python at pearwood.info Fri Oct 7 09:16:35 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 08 Oct 2016 00:16:35 +1100 Subject: default argument value is mutable References: Message-ID: <57f7a0d0$0$1619$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 10:38 pm, Daiyue Weng wrote: > Hi, I declare two parameters for a function with default values [], > > def one_function(arg, arg1=[], arg2=[]): > > PyCharm warns me: > > Default argument value is mutable, > > what does it mean? and how to fix it? The usual way to avoid that is: def one_function(arg, arg1=None, arg2=None): if arg1 is None: arg1 = [] # gets a new list each call if arg2 is None: arg2 = [] # gets a new list each call Otherwise, arg1 and arg2 will get the same list each time, not a fresh empty list. Here is a simple example: def example(arg=[]): print(id(arg), arg) arg.append(1) py> def example(arg=[]): ... print(id(arg), arg) ... arg.append(1) ... py> x = [] py> example(x) 3081877196 [] But with the default argument, you get the same list each time: py> example() 3081877100 [] py> example() 3081877100 [1] py> example() 3081877100 [1, 1] py> example() 3081877100 [1, 1, 1] -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 7 09:27:29 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 08 Oct 2016 00:27:29 +1100 Subject: default argument value is mutable References: <57f78ff2$0$19773$426a74cc@news.free.fr> Message-ID: <57f7a2c1$0$22141$c3e8da3$5496439d@news.astraweb.com> On Fri, 7 Oct 2016 11:48 pm, jmp wrote: > On 10/07/2016 02:07 PM, ast wrote: >> It can be used to store some variables from one call of >> a function to an other one. >> >> def test( _store={'x':0}): >> >> x = _store['x'] >> ..... do some stuff >> _store['x'] = x > > For personal dirty scripts, possibly, for all other situations, never. > Especially since there's nothing in the code above that cannot be solved > using standard idioms . Using a default list as static storage *is* a standard idiom. > That is if you care about anyone reading your code ;) Here's another example of a mutable default argument: https://www.python.org/doc/essays/graphs/ Although it isn't actually being mutated. Nevertheless, if it is good enough for Guido, then it should be good enough for anyone. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nomail at com.invalid Fri Oct 7 09:45:35 2016 From: nomail at com.invalid (ast) Date: Fri, 7 Oct 2016 15:45:35 +0200 Subject: default argument value is mutable In-Reply-To: References: <57f78ff2$0$19773$426a74cc@news.free.fr> Message-ID: <57f7a705$0$7114$426a74cc@news.free.fr> "jmp" a ?crit dans le message de news:mailman.210.1475844513.30834.python-list at python.org... > On 10/07/2016 02:07 PM, ast wrote: >> >> "jmp" a ?crit dans le message de >> news:mailman.209.1475841371.30834.python-list at python.org... >>> On 10/07/2016 01:38 PM, Daiyue Weng wrote: >> >>> >>> So the rule of thumb for default argument value is "No mutable" >>> >>> Cheers, >> >> >> It can be used to store some variables from one call of >> a function to an other one. >> >> def test( _store={'x':0}): >> >> x = _store['x'] >> ..... do some stuff >> _store['x'] = x > > For personal dirty scripts, possibly, for all other situations, never. not so dirty in my opinion > Especially since there's nothing in the code above that cannot be solved using standard idioms . Yes, putting _store dictionnary outside _store={'x':0} def test( ): x = _store['x'] ..... do some stuff _store['x'] = x or using a global variable, but you pollute in both case the global scope unnecessary. > That is if you care about anyone reading your code ;) > > jm > > > > > > From ned at nedbatchelder.com Fri Oct 7 09:55:33 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Fri, 7 Oct 2016 06:55:33 -0700 (PDT) Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: <4f32dd63-d86f-40c7-931e-bc1564e84c88@googlegroups.com> On Friday, October 7, 2016 at 8:39:55 AM UTC-4, BartC wrote: > On 07/10/2016 06:30, Oz-in-DFW wrote: > > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > > v.1900 32 bit (Intel)] on Windows 7 > > > > I'm trying to write some file processing that looks at file size, > > extensions, and several other things and I'm having trouble getting a > > reliably usable path to files. > > > > The problem *seems* to be doubled backslashes in the path, but I've read > > elsewhere that this is just an artifact of the way the interpreter > > displays the strings. > > > > I'm getting an error message on an os.path.getsize call; > > > > Path: - > > "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" - > > Traceback (most recent call last): > > File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in > > if os.path.getsize(path)>10000: > > File "C:\Python32\lib\genericpath.py", line 49, in getsize > > return os.stat(filename).st_size > > WindowsError: [Error 123] The filename, directory name, or volume > > label syntax is incorrect: > > '"C:\\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b845386.jpg"' > > I tried to recreate this error and it seems the getsize function doesn't > like quotes in the path. > > Whether \ is correctly written as \\ in a string literal, or a raw > string is used with the r prefix and a single \, or a \ has been put > into path by any other means, then these will still be displayed as \\ > in the error message, which is strange. The error handler is expanding \ > characters to \\. For error messages destined for developers, it is good practice to use %r or {!r} to get the repr() of a string. This will show quotes around the string, and use backslash escapes to present the contents unambiguously. That's what's expanding \ to \\, and adding the single quotes you see in the message. --Ned. From eryksun at gmail.com Fri Oct 7 10:02:25 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 7 Oct 2016 14:02:25 +0000 Subject: Doubled backslashes in Windows paths In-Reply-To: <57f77ced$0$1617$c3e8da3$5496439d@news.astraweb.com> References: <57f77ced$0$1617$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 7, 2016 at 10:46 AM, Steve D'Aprano wrote: > That's because > > "C: > > is an illegal volume label (disk name? I'm not really a Windows user, and > I'm not quite sure what the correct terminology here would be). It's not an illegal device name, per se. A DOS device can be defined with the name '"C:'. For example: >>> kernel32.DefineDosDeviceW(0, '"C:', 'C:') 1 >>> os.path.getsize(r'\\.\"C:\Windows\py.exe') 889504 However, without the DOS device prefix (either \\.\ or \\?\), Windows has to normalize the path as a classic DOS path before passing it to the kernel. Let's see how Windows 10 normalizes this path by setting a breakpoint on the NtCreateFile system call: >>> os.path.getsize(r'"C:\Windows\py.exe"') Breakpoint 0 hit ntdll!NtCreateFile: 00007ffb`a6c858e0 4c8bd1 mov r10,rcx A kernel path is stored in an OBJECT_ATTRIBUTES structure, which has the path, a handle (for opening relative to another object), and flags such as whether or not the path is case insensitive. The debugger's !obja extension command shows the contents of this structure: 0:000> !obja @r8 Obja +000000a6c8bef038 at 000000a6c8bef038: Name is "C:\Windows\py.exe" OBJ_CASE_INSENSITIVE We see that the user-mode path normalization code doesn't know what to make of a path starting with '"', so it just punts the path to the kernel object manager. In turn the object manager rejects this path because it's not rooted in the object namespace (i.e. it's not of the form "\??\..." or "\Device\...", etc): 0:000> pt; r rax rax=00000000c0000033 The kernel status code 0xC0000033 is STATUS_OBJECT_NAME_INVALID. Note that a path ending in '"' is still illegal even if we explicitly use the r'\\.\"C:' DOS device. For example: >>> os.path.getsize(r'\\.\"C:\Windows\py.exe"') Breakpoint 0 hit ntdll!NtCreateFile: 00007ffb`a6c858e0 4c8bd1 mov r10,rcx 0:000> !obja @r8 Obja +000000a6c8bef038 at 000000a6c8bef038: Name is \??\"C:\Windows\py.exe" OBJ_CASE_INSENSITIVE 0:000> pt; r rax rax=00000000c0000033 In this case it fails because the final '"' in the name (after .exe) is reserved by the I/O manager, along with '<' and '>', respectively as DOS_DOT, DOS_STAR, and DOS_QM. These characters aren't allowed in filesystem names. They get used to implement the semantics of DOS wildcards in NT system calls (the regular '*' and '?' wildcards are also reserved). See FsRtlIsNameInExpression [1], which, for example, a filesystem may use in its implementation of the NtQueryDirectoryFile [2] system call to handle the optional FileName parameter with wildcard matching. [1]: https://msdn.microsoft.com/en-us/library/ff546850 [2]: https://msdn.microsoft.com/en-us/library/ff567047 From jeanmichel at sequans.com Fri Oct 7 10:09:19 2016 From: jeanmichel at sequans.com (jmp) Date: Fri, 07 Oct 2016 16:09:19 +0200 Subject: default argument value is mutable In-Reply-To: <57f7a705$0$7114$426a74cc@news.free.fr> References: <57f78ff2$0$19773$426a74cc@news.free.fr> <57f7a705$0$7114$426a74cc@news.free.fr> Message-ID: On 10/07/2016 03:45 PM, ast wrote: > > "jmp" a ?crit dans le message de > news:mailman.210.1475844513.30834.python-list at python.org... >> On 10/07/2016 02:07 PM, ast wrote: >>> >>> "jmp" a ?crit dans le message de >>> news:mailman.209.1475841371.30834.python-list at python.org... >>>> On 10/07/2016 01:38 PM, Daiyue Weng wrote: >>> >>>> >>>> So the rule of thumb for default argument value is "No mutable" >>>> >>>> Cheers, >>> >>> >>> It can be used to store some variables from one call of >>> a function to an other one. >>> >>> def test( _store={'x':0}): >>> >>> x = _store['x'] >>> ..... do some stuff >>> _store['x'] = x >> >> For personal dirty scripts, possibly, for all other situations, never. > > not so dirty in my opinion You made a point, it's not that dirty, it's an "advanced technique" that is often actually an error when you don't know what you're doing. See the OP's code. I'm in an environment where people use python as a tool more than an effective powerful language to write complex applications. That's probably why I'm bias on this issue and prefer the cautious approach. >> Especially since there's nothing in the code above that cannot be >> solved using standard idioms . > > Yes, putting _store dictionnary outside > > _store={'x':0} > > def test( ): > > x = _store['x'] > ..... do some stuff > _store['x'] = x > > > or using a global variable, but you pollute in both case > the global scope unnecessary. What about def test(): if not hasattr(test, '_store'): test._store={'x':0} test._store['x'] += 1 Neither you pollute the global namespace, nor you pollute the function signature with implementation specifics. jm From darcy at Vex.Net Fri Oct 7 10:16:01 2016 From: darcy at Vex.Net (D'Arcy J.M. Cain) Date: Fri, 7 Oct 2016 10:16:01 -0400 Subject: default argument value is mutable In-Reply-To: References: <57f78ff2$0$19773$426a74cc@news.free.fr> <57f7a705$0$7114$426a74cc@news.free.fr> Message-ID: <20161007101601.626370ba@imp> On Fri, 07 Oct 2016 16:09:19 +0200 jmp wrote: > What about > > def test(): > if not hasattr(test, '_store'): test._store={'x':0} > test._store['x'] += 1 Why is everyone working so hard to avoid creating a class? -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From nomail at com.invalid Fri Oct 7 11:55:40 2016 From: nomail at com.invalid (ast) Date: Fri, 7 Oct 2016 17:55:40 +0200 Subject: default argument value is mutable In-Reply-To: References: <57f78ff2$0$19773$426a74cc@news.free.fr> <57f7a705$0$7114$426a74cc@news.free.fr> Message-ID: <57f7c581$0$3304$426a74cc@news.free.fr> "jmp" a ?crit dans le message de news:mailman.213.1475849391.30834.python-list at python.org... > On 10/07/2016 03:45 PM, ast wrote: >> >> "jmp" a ?crit dans le message de >> news:mailman.210.1475844513.30834.python-list at python.org... >>> On 10/07/2016 02:07 PM, ast wrote: >>>> >>>> "jmp" a ?crit dans le message de >>>> news:mailman.209.1475841371.30834.python-list at python.org... >>>>> On 10/07/2016 01:38 PM, Daiyue Weng wrote: >>>> >>>>> >>>>> So the rule of thumb for default argument value is "No mutable" >>>>> >>>>> Cheers, >>>> >>>> >>>> It can be used to store some variables from one call of >>>> a function to an other one. >>>> >>>> def test( _store={'x':0}): >>>> >>>> x = _store['x'] >>>> ..... do some stuff >>>> _store['x'] = x >>> >>> For personal dirty scripts, possibly, for all other situations, never. >> >> not so dirty in my opinion > > You made a point, it's not that dirty, it's an "advanced technique" that is often actually an > error when you don't know what you're doing. See the OP's code. > > I'm in an environment where people use python as a tool more than an effective powerful language > to write complex applications. That's probably why I'm bias on this issue and prefer the cautious > approach. > >>> Especially since there's nothing in the code above that cannot be >>> solved using standard idioms . >> >> Yes, putting _store dictionnary outside >> >> _store={'x':0} >> >> def test( ): >> >> x = _store['x'] >> ..... do some stuff >> _store['x'] = x >> >> >> or using a global variable, but you pollute in both case >> the global scope unnecessary. > > What about > > def test(): > if not hasattr(test, '_store'): test._store={'x':0} > test._store['x'] += 1 > > Neither you pollute the global namespace, nor you pollute the function signature with > implementation specifics. > > jm > OK There is this solution too we often see: def test(x): .... x = some stuff .... return(x) x=0 x = test(x) From jtim.arnold at gmail.com Fri Oct 7 12:45:35 2016 From: jtim.arnold at gmail.com (Tim) Date: Fri, 7 Oct 2016 09:45:35 -0700 (PDT) Subject: segfault using shutil.make_archive In-Reply-To: References: <87wphkk76a.fsf@handshake.de> Message-ID: <13434458-fea2-4ca9-ba3a-7baa13016d7b@googlegroups.com> On Friday, October 7, 2016 at 5:18:11 AM UTC-4, Chris Angelico wrote: > On Fri, Oct 7, 2016 at 5:24 PM, dieter wrote: > > Memory allocations are frequent (in many places) and they fail rarely. > > Therefore, there is quite a high probability that some of those > > allocations fail to check that the allocation has been successful. > > If this happens (and the allocation fails), then a "SIGSEGV" is > > almost certain. > > Which, BTW, doesn't stop them from being bugs. If this can be pinned > down, it definitely warrants a bugs.python.org issue. > > ChrisA Thanks. I can make a test case. It doesn't look like it has been reported before; I'll add an issue once I can reproduce. --tim From bc at freeuk.com Fri Oct 7 12:50:05 2016 From: bc at freeuk.com (BartC) Date: Fri, 7 Oct 2016 17:50:05 +0100 Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: On 07/10/2016 13:39, BartC wrote: > On 07/10/2016 06:30, Oz-in-DFW wrote: >> I'm getting an error message on an os.path.getsize call; > But the main error appears to be due to the presence of quotes, whether > at each end, or inside the path, enclosing an element with spaces for > example. Try using len(path)>10000 instead; it might be near enough (the > 10000 sounds arbitrary anyway). Forget that. Apparently .getsize returns the size of the file not the length of the path! (That os.path. bit misled me.) In that case just leave out the quotes. I don't think you need them even if the file-path contains embedded spaces. That would be an issue on a command-line (as spaces separate parameters), not inside a string. But if quotes are present in user-input that represents a file-name, then they might need to be removed. -- Bartc From pdorange at pas-de-pub-merci.mac.com Fri Oct 7 12:59:06 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Fri, 7 Oct 2016 18:59:06 +0200 Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <5a6575d2-1477-4b2b-bebe-8b73a0336f9d@googlegroups.com> Message-ID: <1murere.1x0brds15iakyfN%pdorange@pas-de-pub-merci.mac.com> Navneet Siddhant wrote: > I guess I will have to extract data from multiple divs as only extracting > data from the parent div which has other divs in it with the different > data is coming up all messed up. Will play around and see if I could get > through it. Let me clarify once again I dont need complete code , a > resource where I could find more info about using Beautifulsoup will be > appreciated. Also do I need some kind of plugin etc to extract data to > csv ? or it is built in python and I could simply import csv and write > other commands needed ?? BeautifulSoup was a good tool to do the task. But it would require a bit more code to accomplish this. I don't have enough time to look at the (coupon) data but i work on similar task and often data are clustered throught many dov and html tag. You need to understand (reverse ingeneering) the data structure and extract piece of data (Beautifulsoup and lot of tools for that) then aggregate the data in an internal structure (you design this according to your needs : a class), the you sipmply export this aggregated data to CSV using the csv module. The main task is to extract data with BeautifulSoup. BS provide tool to extract from div, or any html tag, just play with it a little, and read docs. Don't know if it could help but here a sample code i used sometime as an example and also in real life to extract data (river level in real time) from a french web site (vigiecrue) : retrieve the page, extract data, extract last river level and mean the 24h last levels. Note : it was probably not beautiful python code, but it works for the purpose it was written. -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From torriem at gmail.com Fri Oct 7 13:05:24 2016 From: torriem at gmail.com (Michael Torrie) Date: Fri, 7 Oct 2016 11:05:24 -0600 Subject: segfault using shutil.make_archive In-Reply-To: References: Message-ID: <6efda754-3f3d-ec64-9e00-46671c33a9c1@gmail.com> On 10/06/2016 10:46 AM, Tim wrote: > I need to zip up a directory that's about 400mb. > I'm using shutil.make_archive and I'm getting this response: > > Segmentation fault: 11 (core dumped) > > The code is straightforward (and works on other, smaller dirs): > > shutil.make_archive(os.path.join(zip_dir, zname), 'zip', tgt_dir) > > I guess I could drop the convenience of make_archive and use zipfile but that seems to be exactly what make_archive does. > > I'm on FreeBSD, python2.7. > > Anyone seen this before? Does normal the normal zip utility crash also when trying to zip up this large directory? I'm not familiar with how shutils.make_archive works, but since it's part of shutils, I suspect it's calling the zip command as a subprocess, rather than using the Python zip module. Given the size of your directory I doubt the zip module would work anyway. If the zip command works, you could just execute it using popen from within Python. From h2m2 at mailworks.org Fri Oct 7 13:34:40 2016 From: h2m2 at mailworks.org (h2m2) Date: Fri, 07 Oct 2016 19:34:40 +0200 Subject: bugreport os.path.dirname and os.path.basename Message-ID: <1475861680.3077230.749137969.1A6DD68A@webmail.messagingengine.com> in the meantime, I am subscribed and confirmed - the bug below as requested ----- Original message ----- From: h2m2 To: python-list at python.org Subject: bugreport os.path.dirname and os.path.basename Date: Wed, 05 Oct 2016 16:49:17 +0200 bug: os.path.dirname('G:\Programmierung\Python\Portugal Training\Python-for-Algorithms--Data-Structures--and--Life\Algorithm Analysis and Big O\testFile.ipynb') result and bug is: 'G:\\Programmierung\\Python\\Portugal Training\\Python-for-Algorithms--Data-Structures--and--Life' if I am not mistaken: correct result would be 'G:\\Programmierung\\Python\\Portugal Training\\Python-for-Algorithms--Data-Structures--and--Life\\Algorithm Analysis and Big O' analog with basename 'Algorithm Analysis and Big O\testFile.ipynb' again, I was expecting 'testFile.ipynb' only With less complex constructs , the function works well. Only that is doubtfull when the names start getting more kind of theses days .... Cheers ! From rosuav at gmail.com Fri Oct 7 13:38:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Oct 2016 04:38:50 +1100 Subject: bugreport os.path.dirname and os.path.basename In-Reply-To: <1475861680.3077230.749137969.1A6DD68A@webmail.messagingengine.com> References: <1475861680.3077230.749137969.1A6DD68A@webmail.messagingengine.com> Message-ID: On Sat, Oct 8, 2016 at 4:34 AM, h2m2 wrote: > os.path.dirname('G:\Programmierung\Python\Portugal > Training\Python-for-Algorithms--Data-Structures--and--Life\Algorithm > Analysis and Big O\testFile.ipynb') > > result and bug is: > 'G:\\Programmierung\\Python\\Portugal > Training\\Python-for-Algorithms--Data-Structures--and--Life' > > if I am not mistaken: correct result would be > 'G:\\Programmierung\\Python\\Portugal > Training\\Python-for-Algorithms--Data-Structures--and--Life\\Algorithm > Analysis and Big O' Not a bug. You have something that you perhaps don't understand: backslash escapes. When you have "Big O\testfile", that string does *not* have a backslash in it. Use raw string literals, forward slashes, or doubled backslashes, to avoid this. ChrisA From lists at ozindfw.net Fri Oct 7 13:41:51 2016 From: lists at ozindfw.net (Oz-in-DFW) Date: Fri, 7 Oct 2016 12:41:51 -0500 Subject: Doubled backslashes in Windows paths [Resolved] In-Reply-To: References: Message-ID: On 10/7/2016 12:30 AM, Oz-in-DFW wrote: > I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC > v.1900 32 bit (Intel)] on Windows 7 > > I'm trying to write some file processing that looks at file size, > extensions, and several other things and I'm having trouble getting a > reliably usable path to files. Thanks to all who have replied. It looks like I have this part fixed. I had it fixed sooner than I realized because of cranial-anal-retention - the os.path.getsize() call was not using the variable into which I was placing the corrected path. D'oh! The root problem was that the path did not have a trailing slash when concatenated with the filename. This was handled properly by an os.path.join() call as recommended by Dennis Lee Bieber. So what now works is path = os.path.join(dirpath,name) if os.path.getsize(path)>10000: print("Path: ",path," Size: ",os.path.getsize(path)) Raw strings didn't apply because the content of /dirpath /and /name /are the result of an os.walk() call. As Dennis Lee Bieber also observed, the call to path = os.path.normpath(path) wasn't needed. It does no harm as it changes the slash directions to keep the Windows command line happy, but the Python functions are just as happy without the changes. Thanks to all who replied. -- mailto:oz at ozindfw.net Oz POB 93167 Southlake, TX 76092 (Near DFW Airport) From marko at pacujo.net Fri Oct 7 14:02:26 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 07 Oct 2016 21:02:26 +0300 Subject: default argument value is mutable References: <57f78ff2$0$19773$426a74cc@news.free.fr> <57f7a705$0$7114$426a74cc@news.free.fr> <20161007101601.626370ba@imp> Message-ID: <87vax49gwd.fsf@elektro.pacujo.net> "D'Arcy J.M. Cain" : > On Fri, 07 Oct 2016 16:09:19 +0200 > jmp wrote: >> What about >> >> def test(): >> if not hasattr(test, '_store'): test._store={'x':0} >> test._store['x'] += 1 > > Why is everyone working so hard to avoid creating a class? Hear, hear! Marko From davros at bellaliant.net Fri Oct 7 15:06:54 2016 From: davros at bellaliant.net (John McKenzie) Date: Fri, 07 Oct 2016 19:06:54 GMT Subject: Passing Variable to Function References: Message-ID: Brendan and Alister, thank you both for responding. I am very inexperienced with python, but not new to computers so on my own I realized the strings vs number mistake. (I seem to work with and learn about Python for a few weeks at a time with 6 to 12 months in between. Need to change that and focus on it for awhile.) After my first error I removed the quotation marks from my variables knowing that strings vs numbers was involved. I copied and pasted from my original script when I showed it to you. Opps. Anyway, I had no idea how to expand tuples nor that it was not done automatically. Thank you you both for sharing that info. After making variables numbers by removing the quotes and putting an asterisk in front of my colour1, colour2, etc variables whenever I passed them to a function in order to expand them, the error messages disappeared. It appears to work just fine now. Thanks. From dkoleary at olearycomputers.com Fri Oct 7 15:35:09 2016 From: dkoleary at olearycomputers.com (Doug OLeary) Date: Fri, 7 Oct 2016 12:35:09 -0700 (PDT) Subject: xml parsing with lxml Message-ID: <622ea3b0-88b4-420b-89e0-9e7c6e866715@googlegroups.com> Hey; I'm trying to gather information from a number of weblogic configuration xml files using lxml. I've found any number of tutorials on the web but they all seem to assume a knowledge that I apparently don't have... that, or I'm just being rock stupid today - that's distinct possibility too. The xml looks like: Domain1 10.3.5.0 [[snipp]] [[realm children snipped] myrealm [[snip]] [[snip]] [[snip]] [[snip]] [[snip]] byTime 14 02:00 Info [[snip]] 40024 true snip]] [[children snipped]] ${hostname} ${hostname} 40022 javac [[children snipped] [[rest snipped] The tutorials all start out well enough with: $ python Python 3.5.2 (default, Aug 22 2016, 09:04:07) [GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import etree >>> doc = etree.parse('config.xml') Now what? For instance, how do I list the top level children of .*?? In that partial list, it'd be name, domain-version, security-configuration, log, and server. For some reason, I'm not able to make the conceptual leap to get to the first step of those tutorials. The end goal of this exercise is to programatically identify weblogic clusters and their hosts. thanks Doug O'Leary From desolate.soul.me at gmail.com Fri Oct 7 16:12:48 2016 From: desolate.soul.me at gmail.com (Navneet Siddhant) Date: Fri, 7 Oct 2016 13:12:48 -0700 (PDT) Subject: BeautifulSoup help !! In-Reply-To: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> Message-ID: i was able to almost get the results i require but only half of the info is showing up. i.e suppose there are 20 coupons showing on the page but when i print them using the container they are in only 5 are printed on the screen. Also figured out how to write to csv file , but even here only 1 row is being written , how do I iterate through all the info which is printed on the screen and export them to a csv ? im using writerows([object which contains the data]) on the screen only 5 rows are printed and in the csv only 1 row is exported. From gordon at panix.com Fri Oct 7 16:21:30 2016 From: gordon at panix.com (John Gordon) Date: Fri, 7 Oct 2016 20:21:30 +0000 (UTC) Subject: xml parsing with lxml References: <622ea3b0-88b4-420b-89e0-9e7c6e866715@googlegroups.com> Message-ID: In <622ea3b0-88b4-420b-89e0-9e7c6e866715 at googlegroups.com> Doug OLeary writes: > >>> from lxml import etree > >>> doc =3D etree.parse('config.xml') > Now what? For instance, how do I list the top level children of > ? root = doc.getroot() for child in root: print(child.tag) -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From dkoleary at olearycomputers.com Fri Oct 7 16:27:47 2016 From: dkoleary at olearycomputers.com (Doug OLeary) Date: Fri, 7 Oct 2016 13:27:47 -0700 (PDT) Subject: xml parsing with lxml In-Reply-To: References: <622ea3b0-88b4-420b-89e0-9e7c6e866715@googlegroups.com> Message-ID: On Friday, October 7, 2016 at 3:21:43 PM UTC-5, John Gordon wrote: > root = doc.getroot() > for child in root: > print(child.tag) > Excellent! thank, you sir! that'll get me started. Appreciate the reply. Doug O'Leary From saxri89 at gmail.com Fri Oct 7 17:25:15 2016 From: saxri89 at gmail.com (Xristos Xristoou) Date: Fri, 7 Oct 2016 14:25:15 -0700 (PDT) Subject: define variables in the txt file using python Message-ID: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> hello i have one .txt file and i want to create python script with sys.argv or argparse or other package to define some variables in the txt file and i take some result. txt file maybe like this : input number 1= %var1% input number 2= %var2% result = %vresult(var1-var2)% how can i write a python script to update dynamic variables in the txt file ? i dont want with replace i thing so arguments is the better. From greg.ewing at canterbury.ac.nz Fri Oct 7 19:33:47 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 08 Oct 2016 12:33:47 +1300 Subject: Question on multiple Python users in one application In-Reply-To: References: <7293921E664445E582856EB2983E7F6B@watson1> <3EE4AF9FACFA4A088BCFC0CE10C18321@watson1> <1oefvb5v7jgp3lv20ueaoj9fjaflndk113@4ax.com> Message-ID: Dennis Lee Bieber wrote: > What is it... A Burroughs mainframe running a version of FORTH? The Burroughs architecture is a stack architecture, so the machine code looks like a version of Forth in some ways. -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 7 19:38:24 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sat, 08 Oct 2016 12:38:24 +1300 Subject: Question on multiple Python users in one application In-Reply-To: References: <2C6C94C3291A44ACA3EC8647F4BF1B5C@watson1> <7cca3717-6b98-2ae2-1a76-0ef2f1673535@lucidity.plus.com> <691ffbe3-6718-389f-9aa5-39d1a6f1352d@gmail.com> <50935B8C00204F18BA2CC2EF9B78D806@watson1> <681005F8873847FC9D289ADE66747798@watson1> Message-ID: Dennis Lee Bieber wrote: > normally the main file of a Python program is still the plain text, It doesn't have to be, though -- you can do 'python somefile.pyc' and it will happily run it. > and imported > modules are retrieved from the file system as (if found) PYC pre-compiled, > otherwise the text PY file is loaded. They don't exist as a part of a > monolithic executable program. I don't think that would be a problem for Loren, but if it is, the .pyc files can be wrapped up in a zip file, and Python can import them directly from there. -- Greg From bc at freeuk.com Fri Oct 7 20:02:58 2016 From: bc at freeuk.com (BartC) Date: Sat, 8 Oct 2016 01:02:58 +0100 Subject: Doubled backslashes in Windows paths [Resolved] In-Reply-To: References: Message-ID: On 07/10/2016 18:41, Oz-in-DFW wrote: > On 10/7/2016 12:30 AM, Oz-in-DFW wrote: >> I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC >> v.1900 32 bit (Intel)] on Windows 7 >> >> I'm trying to write some file processing that looks at file size, >> extensions, and several other things and I'm having trouble getting a >> reliably usable path to files. > Thanks to all who have replied. It looks like I have this part fixed. > > I had it fixed sooner than I realized because of cranial-anal-retention > - the os.path.getsize() call was not using the variable into which I was > placing the corrected path. D'oh! > > The root problem was that the path did not have a trailing slash when > concatenated with the filename. This was handled properly by an > os.path.join() call as recommended by Dennis Lee Bieber. But the error message in your OP showed what looked like a correctly formed path (apart from the quotes). Without a / or \ between dirpath and filename, the error message would have been different (probably about not being able to find the file). > So what now works is > > path = os.path.join(dirpath,name) > if os.path.getsize(path)>10000: > print("Path: ",path," Size: ",os.path.getsize(path)) > > Raw strings didn't apply because the content of /dirpath /and /name /are > the result of an os.walk() call. When I was creating such path-handling routines, anything returning the name of a path /always/ ended with \ or / for that reason: so that you could just stick a file at the end without needing to check whether the path had the right ending or not. -- Bartc From gengyangcai at gmail.com Sat Oct 8 02:02:01 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Fri, 7 Oct 2016 23:02:01 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement Message-ID: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> Any idea how to correct this error ? Looks fine to me .... >>> rect_x = 50 # -------- Main Program Loop ----------- while not done: for event in pygame.event.get(): # User did something if event.type == pygame.QUIT: # If user clicked close done = True # Flag that we are done so we exit this loop SyntaxError: multiple statements found while compiling a single statement Thanks ... From steve+python at pearwood.info Sat Oct 8 02:16:09 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 08 Oct 2016 17:16:09 +1100 Subject: SyntaxError: multiple statements found while compiling a single statement References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> Message-ID: <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> On Sat, 8 Oct 2016 05:02 pm, Cai Gengyang wrote: > Any idea how to correct this error ? Looks fine to me .... As usual, the first and most important rule of debugging is to use ALL the information the compiler gives you. Where is the rest of the traceback? At the very least, even if nothing else, the compiler will print its best guess at the failed line with a caret ^ pointing at the place it realised there was an error. Asking us to debug your problem without giving us ALL the information is like that old joke: A man makes an appointment at the doctor because he isn't feeling well. He goes in to see the doctor, who says "So, Mr Smith, what seems to be the problem?" The man says "You're the medical expert, you tell me." I do not think it is possible to get the error message you claim from running the code you show. I think there is more code that you haven't shown us, and that the error is there. The only way I can reproduce the error message you give is by using the compile() function: py> compile("a=1\nb=2", "", "single") Traceback (most recent call last): File "", line 1, in File "", line 1 a=1 ^ SyntaxError: multiple statements found while compiling a single statement Notice how the compiler shows the offending line and puts a caret ^ at the end? Unless you show us what code actually causes this error message, I don't think it is possible to help you. We're not mind-readers. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gengyangcai at gmail.com Sat Oct 8 02:29:13 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Fri, 7 Oct 2016 23:29:13 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> Unfortunately, in this case, it is 100% of the information I am giving you. You can try it yourself. The code is in the first section (8.1) of http://programarcadegames.com/index.php?chapter=introduction_to_animation&lang=en#section_8 Just copy and paste it into your Python IDLE and let me know what you get ... On Saturday, October 8, 2016 at 2:19:26 PM UTC+8, Steve D'Aprano wrote: > On Sat, 8 Oct 2016 05:02 pm, Cai Gengyang wrote: > > > Any idea how to correct this error ? Looks fine to me .... > > As usual, the first and most important rule of debugging is to use ALL the > information the compiler gives you. Where is the rest of the traceback? At > the very least, even if nothing else, the compiler will print its best > guess at the failed line with a caret ^ pointing at the place it realised > there was an error. > > Asking us to debug your problem without giving us ALL the information is > like that old joke: > > > A man makes an appointment at the doctor because he isn't feeling > well. He goes in to see the doctor, who says "So, Mr Smith, what > seems to be the problem?" > > The man says "You're the medical expert, you tell me." > > > I do not think it is possible to get the error message you claim from > running the code you show. I think there is more code that you haven't > shown us, and that the error is there. > > The only way I can reproduce the error message you give is by using the > compile() function: > > py> compile("a=1\nb=2", "", "single") > Traceback (most recent call last): > File "", line 1, in > File "", line 1 > a=1 > ^ > SyntaxError: multiple statements found while compiling a single statement > > > Notice how the compiler shows the offending line and puts a caret ^ at the > end? > > Unless you show us what code actually causes this error message, I don't > think it is possible to help you. We're not mind-readers. > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. From dieter at handshake.de Sat Oct 8 02:40:04 2016 From: dieter at handshake.de (dieter) Date: Sat, 08 Oct 2016 08:40:04 +0200 Subject: define variables in the txt file using python References: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> Message-ID: <87zimf4a4b.fsf@handshake.de> Xristos Xristoou writes: > i have one .txt file and i want to create python script with sys.argv or argparse or other package to define some variables in the txt file and i take some result. > > txt file maybe like this : > > input number 1= %var1% > input number 2= %var2% > result = %vresult(var1-var2)% > > how can i write a python script to update dynamic variables in the txt file ? > i dont want with replace i thing so arguments is the better. You read the file in; you look for things in the resulting string with features you find in the "re" (= "Regular Expression") module and maybe replace them; you write the result out. Things like "%vresult(var1-var2)%" may be a bit more difficult. You might need to use a parser library to get the expressions parsed and then evaluate it with your own Python code. Python by itself does not include a general parser library (only one for Python source code), but there are a few available for Pytin in third party packages. From steve+python at pearwood.info Sat Oct 8 02:56:08 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 08 Oct 2016 17:56:08 +1100 Subject: SyntaxError: multiple statements found while compiling a single statement References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> Message-ID: <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> On Sat, 8 Oct 2016 05:29 pm, Cai Gengyang wrote: > Unfortunately, in this case, it is 100% of the information I am giving > you. Exactly. That's the problem. You need to give us more information, like how you are trying to run this code. Are you using an IDE? Which IDE? > You can try it yourself. The code is in the first section (8.1) of > > http://programarcadegames.com/index.php?chapter=introduction_to_animation&lang=en#section_8 > > Just copy and paste it into your Python IDLE and let me know what you get I don't normally use IDLE, and you shouldn't assume that everyone does. *That* is the extra information we need to solve the problem: The IDLE interactive interpreter in Python 3 does not seem to allow you to paste multiple lines at once. If you do, it highlights the first line of code with a red background and prints the error: SyntaxError: multiple statements found while compiling a single statement For example: Python 3.3.0rc3 (default, Sep 27 2012, 18:44:58) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux Type "copyright", "credits" or "license()" for more information. >>> # Define some colors BLACK = (0, 0, 0) WHITE = (255, 255, 255) GREEN = (0, 255, 0) RED = (255, 0, 0) SyntaxError: multiple statements found while compiling a single statement >>> I can't show the colour highlighting, but the line BLACK = ... is highlighted in red. You can either copy and paste one line at a time, or you can use the File > Open menu command to open a complete script. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gengyangcai at gmail.com Sat Oct 8 03:47:59 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 8 Oct 2016 00:47:59 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: This is the result when I copy and paste it one line at a time : >>> rect_x = 50 >>> rect_y = 50 >>> while not done: for event in pygame.event.get(): if event.type == pygame.QUIT: done = True SyntaxError: invalid syntax On Saturday, October 8, 2016 at 3:02:09 PM UTC+8, Steve D'Aprano wrote: > On Sat, 8 Oct 2016 05:29 pm, Cai Gengyang wrote: > > > Unfortunately, in this case, it is 100% of the information I am giving > > you. > > Exactly. That's the problem. You need to give us more information, like how > you are trying to run this code. Are you using an IDE? Which IDE? > > > > You can try it yourself. The code is in the first section (8.1) of > > > > > http://programarcadegames.com/index.php?chapter=introduction_to_animation&lang=en#section_8 > > > > Just copy and paste it into your Python IDLE and let me know what you get > > > I don't normally use IDLE, and you shouldn't assume that everyone does. > *That* is the extra information we need to solve the problem: > > The IDLE interactive interpreter in Python 3 does not seem to allow you to > paste multiple lines at once. If you do, it highlights the first line of > code with a red background and prints the error: > > SyntaxError: multiple statements found while compiling a single statement > > > > For example: > > > Python 3.3.0rc3 (default, Sep 27 2012, 18:44:58) > [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux > Type "copyright", "credits" or "license()" for more information. > >>> # Define some colors > BLACK = (0, 0, 0) > WHITE = (255, 255, 255) > GREEN = (0, 255, 0) > RED = (255, 0, 0) > SyntaxError: multiple statements found while compiling a single statement > >>> > > > > I can't show the colour highlighting, but the line BLACK = ... is > highlighted in red. > > > You can either copy and paste one line at a time, or you can use the File > > Open menu command to open a complete script. > > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. From __peter__ at web.de Sat Oct 8 05:13:48 2016 From: __peter__ at web.de (Peter Otten) Date: Sat, 08 Oct 2016 11:13:48 +0200 Subject: define variables in the txt file using python References: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> Message-ID: Xristos Xristoou wrote: > hello > > > i have one .txt file and i want to create python script with sys.argv or > argparse or other package to define some variables in the txt file and i > take some result. > > > > txt file maybe like this : > > input number 1= %var1% > input number 2= %var2% > result = %vresult(var1-var2)% > > how can i write a python script to update dynamic variables in the txt > file ? i dont want with replace i thing so arguments is the better. $ cat xristos.txt input number 1= %var1% input number 2= %var2% result = %vresult(var1-var2)% $ cat xristos2.py import argparse import sys parser = argparse.ArgumentParser() parser.add_argument("infile") parser.add_argument("vars", nargs="*") parser.add_argument("--outfile", type=argparse.FileType("w")) args = parser.parse_args() ns = {} exec("\n".join(args.vars), ns) with open(args.infile) as f: text = f.read() parts = text.split("%") for i in range(1, len(parts), 2): parts[i] = str(eval(parts[i], ns)) print("".join(parts), file=args.outfile, end="") $ python3 xristos2.py xristos.txt var1=42 var2=33 'def vresult(a): return a*a' --outfile - input number 1= 42 input number 2= 33 result = 81 Be warned that the text file is now basically a Python script and can do everything a script can do -- including wiping your hard drive. From jobmattcon at gmail.com Sat Oct 8 05:58:15 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Sat, 8 Oct 2016 02:58:15 -0700 (PDT) Subject: how to refactor nested for loop into smaller for loop assume each of them independent? Message-ID: how to refactor nested for loop into smaller for loop assume each of them independent? because memory is not enough for ii in range(1,2000): for jj in range(1,2000): for kk in range(1,2000): print run(ii,jj,kk) From rosuav at gmail.com Sat Oct 8 06:03:43 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Oct 2016 21:03:43 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On Sat, Oct 8, 2016 at 8:58 PM, meInvent bbird wrote: > how to refactor nested for loop into smaller for loop assume each of them independent? > > because memory is not enough > > for ii in range(1,2000): > for jj in range(1,2000): > for kk in range(1,2000): > print run(ii,jj,kk) Since you're using Python 2, you could try xrange instead of range. But it won't make a lot of difference. You're trying to call your run function 8,000,000,000 times... that is a *lot* of function calls. Consider a faster algorithm instead! ChrisA From bc at freeuk.com Sat Oct 8 06:12:56 2016 From: bc at freeuk.com (BartC) Date: Sat, 8 Oct 2016 11:12:56 +0100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On 08/10/2016 11:03, Chris Angelico wrote: > On Sat, Oct 8, 2016 at 8:58 PM, meInvent bbird wrote: >> how to refactor nested for loop into smaller for loop assume each of them independent? >> >> because memory is not enough >> >> for ii in range(1,2000): >> for jj in range(1,2000): >> for kk in range(1,2000): >> print run(ii,jj,kk) > > Since you're using Python 2, you could try xrange instead of range. > But it won't make a lot of difference. You're trying to call your run > function 8,000,000,000 times... that is a *lot* of function calls. > Consider a faster algorithm instead! Printing even one character 8 billion times is likely to take quite a few hours too. The OP's code however is a good demonstration of how crazy Python's original for-range loop was: you need to construct a list of N elements just to be able to count to N. How many years was it until xrange was introduced? -- bartc From rosuav at gmail.com Sat Oct 8 06:54:01 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 8 Oct 2016 21:54:01 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On Sat, Oct 8, 2016 at 9:12 PM, BartC wrote: > On 08/10/2016 11:03, Chris Angelico wrote: >> >> On Sat, Oct 8, 2016 at 8:58 PM, meInvent bbird >> wrote: >>> >>> how to refactor nested for loop into smaller for loop assume each of them >>> independent? >>> >>> because memory is not enough >>> >>> for ii in range(1,2000): >>> for jj in range(1,2000): >>> for kk in range(1,2000): >>> print run(ii,jj,kk) >> >> >> Since you're using Python 2, you could try xrange instead of range. >> But it won't make a lot of difference. You're trying to call your run >> function 8,000,000,000 times... that is a *lot* of function calls. >> Consider a faster algorithm instead! > > > Printing even one character 8 billion times is likely to take quite a few > hours too. > > The OP's code however is a good demonstration of how crazy Python's original > for-range loop was: you need to construct a list of N elements just to be > able to count to N. How many years was it until xrange was introduced? Hardly matters in this case. The memory usage of a list of 2000 ints is: >>> sys.getsizeof(range(2000)) 16072 >>> 2000 * sys.getsizeof(1999) 48000 About 64KB. Not overly significant. (A bit less than the naive figure of 64,072 bytes, because the low integers are cached.) Even if run() is defined as merely returning an empty string, it's going to take a long time to print eight billion newlines... ChrisA From rustompmody at gmail.com Sat Oct 8 07:05:53 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 8 Oct 2016 04:05:53 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> On Saturday, October 8, 2016 at 1:21:50 PM UTC+5:30, Cai Gengyang wrote: > This is the result when I copy and paste it one line at a time : > > >>> rect_x = 50 > >>> rect_y = 50 > >>> while not done: > for event in ... There's sure something strange about your formatting you want something like this I think: rect_x = 50 rect_y = 50 while not done: for event in pygame.event.get(): if event.type == pygame.QUIT: done = True On a related note in addition to Steven's 2 suggestions - paste 1 line at a time - Write the code in a py file you could also try it out without Idle ie start python at your shell (or terminal or cmd.exe or whatever) And try it there From bc at freeuk.com Sat Oct 8 07:16:57 2016 From: bc at freeuk.com (BartC) Date: Sat, 8 Oct 2016 12:16:57 +0100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On 08/10/2016 11:54, Chris Angelico wrote: > On Sat, Oct 8, 2016 at 9:12 PM, BartC wrote: >> On 08/10/2016 11:03, Chris Angelico wrote: >>> >>> On Sat, Oct 8, 2016 at 8:58 PM, meInvent bbird >>> wrote: >>>> >>>> how to refactor nested for loop into smaller for loop assume each of them >>>> independent? >>>> >>>> because memory is not enough >>>> >>>> for ii in range(1,2000): >>>> for jj in range(1,2000): >>>> for kk in range(1,2000): >>>> print run(ii,jj,kk) >>> >>> >>> Since you're using Python 2, you could try xrange instead of range. >>> But it won't make a lot of difference. You're trying to call your run >>> function 8,000,000,000 times... that is a *lot* of function calls. >>> Consider a faster algorithm instead! >> >> >> Printing even one character 8 billion times is likely to take quite a few >> hours too. >> >> The OP's code however is a good demonstration of how crazy Python's original >> for-range loop was: you need to construct a list of N elements just to be >> able to count to N. How many years was it until xrange was introduced? > > Hardly matters in this case. The memory usage of a list of 2000 ints is: > >>>> sys.getsizeof(range(2000)) > 16072 >>>> 2000 * sys.getsizeof(1999) > 48000 > About 64KB. Not overly significant. Well, there'll be three lots. But it's true there won't be 8 billion elements which is what I immediately thought when the OP said they'd run out of memory. So perhaps something is going on inside run() to account for it. -- Bartc From steve+python at pearwood.info Sat Oct 8 07:29:04 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 08 Oct 2016 22:29:04 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: Message-ID: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> On Sat, 8 Oct 2016 09:12 pm, BartC wrote: > The OP's code however is a good demonstration of how crazy Python's > original for-range loop was: you need to construct a list of N elements > just to be able to count to N. How many years was it until xrange was > introduced? Python 1.4 (that's 1996) had xrange, but I don't know when it was added. https://docs.python.org/release/1.4/lib/node26.html#SECTION00330000000000000000 The oldest version I have access to is the *extremely* primitive 0.9. Not surprisingly, it doesn't have xrange -- but it lacks a lot of things, including globals(), map(), named exceptions, "" strings ('' is okay), exponentiation, and more. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From breamoreboy at gmail.com Sat Oct 8 07:46:43 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 8 Oct 2016 04:46:43 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> Message-ID: On Saturday, October 8, 2016 at 12:07:47 PM UTC+1, Rustom Mody wrote: > On Saturday, October 8, 2016 at 1:21:50 PM UTC+5:30, Cai Gengyang wrote: > > This is the result when I copy and paste it one line at a time : > > > > >>> rect_x = 50 > > >>> rect_y = 50 > > >>> while not done: > > for event in ... > > > There's sure something strange about your formatting > > you want something like this I think: > > rect_x = 50 > rect_y = 50 > while not done: > for event in pygame.event.get(): > if event.type == pygame.QUIT: > done = True > > On a related note in addition to Steven's 2 suggestions > - paste 1 line at a time > - Write the code in a py file > > you could also try it out without Idle > ie start python at your shell (or terminal or cmd.exe or whatever) > And try it there The easiest way is to use the ipython %paste magic command. Kindest regards. Mark Lawrence. From alain at universite-de-strasbourg.fr.invalid Sat Oct 8 07:47:26 2016 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Sat, 08 Oct 2016 13:47:26 +0200 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: Message-ID: <87r37rxdtd.fsf@universite-de-strasbourg.fr.invalid> meInvent bbird writes: > how to refactor nested for loop into smaller for loop assume each of them independent? > > because memory is not enough > > for ii in range(1,2000): > for jj in range(1,2000): > for kk in range(1,2000): > print run(ii,jj,kk) n = 0 while n < 2000*2000*2000: ii = n // (2000*2000) jj = ( n % (2000*2000) ) // 2000 kk = n % 2000 run(ii,jj,kk) n += 1 or: for n in xrange(2000*2000*2000): ii,rr = divmod(n,2000*2000) jj,kk = divmod(rr,2000) run(ii,jj,kk) (I think CPython wll fold the constants, but I haven't checked). Are you sure this is your (real) problem? -- Alain. From saxri89 at gmail.com Sat Oct 8 10:03:55 2016 From: saxri89 at gmail.com (Xristos Xristoou) Date: Sat, 8 Oct 2016 07:03:55 -0700 (PDT) Subject: define variables in the txt file using python In-Reply-To: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> References: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> Message-ID: <75e82f69-6e5d-4b4b-9b12-a0c8f0b3dc26@googlegroups.com> ?? ???????, 8 ????????? 2016 - 12:25:28 ?.?. UTC+3, ? ??????? Xristos Xristoou ??????: > hello > > > i have one .txt file and i want to create python script with sys.argv or argparse or other package to define some variables in the txt file and i take some result. > > > > txt file maybe like this : > > input number 1= %var1% > input number 2= %var2% > result = %vresult(var1-var2)% > > how can i write a python script to update dynamic variables in the txt file ? > i dont want with replace i thing so arguments is the better. thanks Mr.Peter your code look nice,i have one question,must be running from cmd line ?i cant to run your code in the python script ? From gengyangcai at gmail.com Sat Oct 8 10:44:44 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 8 Oct 2016 07:44:44 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> Message-ID: Somehow it still doesnt work --- it keeps giving the syntaxerror, inconsistent use of tabs and indentation message EVEN though i use only the enter and space buttons and never touched the tab button a single time. Im pretty sure there is something wrong with the program itself. Gonna download pycharm and start testing the programs in it --- heard that it gives a better user experience with indentations On Saturday, October 8, 2016 at 7:46:57 PM UTC+8, bream... at gmail.com wrote: > On Saturday, October 8, 2016 at 12:07:47 PM UTC+1, Rustom Mody wrote: > > On Saturday, October 8, 2016 at 1:21:50 PM UTC+5:30, Cai Gengyang wrote: > > > This is the result when I copy and paste it one line at a time : > > > > > > >>> rect_x = 50 > > > >>> rect_y = 50 > > > >>> while not done: > > > for event in ... > > > > > > There's sure something strange about your formatting > > > > you want something like this I think: > > > > rect_x = 50 > > rect_y = 50 > > while not done: > > for event in pygame.event.get(): > > if event.type == pygame.QUIT: > > done = True > > > > On a related note in addition to Steven's 2 suggestions > > - paste 1 line at a time > > - Write the code in a py file > > > > you could also try it out without Idle > > ie start python at your shell (or terminal or cmd.exe or whatever) > > And try it there > > The easiest way is to use the ipython %paste magic command. > > Kindest regards. > > Mark Lawrence. From jonas at wielicki.name Sat Oct 8 11:01:47 2016 From: jonas at wielicki.name (Jonas Wielicki) Date: Sat, 08 Oct 2016 17:01:47 +0200 Subject: [ANN] aioxmpp 0.7.1 (sic!) released Message-ID: <1530940.nugAWNs8XF@sinistra> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Dear subscribers, I am pleased to announce the release of aioxmpp 0.7.1 (footnote 1). The current release can be obtained from GitHub [1] (check out the v0.6.0 tag or the master branch) or PyPI [2]. The HTML documentation can be found at [3]. Examples can be found in the GitHub repository, in the examples subdirectory. aioxmpp is a Python library based on asyncio. It implements the client side of the XMPP protocol (RFC 6120 and others). For a more detailed description of the package, please review the README of the package on GitHub [1] or on PyPI [2]. For more information on XMPP, please see [9]. It is licensed under the terms of the GNU Lesser General Public License Version 3.0 or later. The most important change in this release is the change of the license from GPLv3+ to LGPLv3+! (the same holds for aiosasl, which just had its v0.2.1 release) The functional highlights of this release are base support for Data Forms [XEP-0004] and a first application of those in the Multi User Chats [XEP-0045] implementation. Aside from that, the examples have been refactored and rewritten. A Quickstart section was added to the documentation [4], hopefully helping people to get started with aioxmpp easier. Bugs, feature requests, patches and questions can be directed to either the aioxmpp mailing list [5], the GitHub issue tracker [6] or the XMPP Multi-User chat [7], whatever floats your boat. Please direct security-relevant issue reports directly to me (jonas at wielicki.name), preferrably encrypted using my GPG public key [8]. best regards and happy-asyncio-ing, Jonas Wielicki footnote 1: 0.7.0 and 0.7.1 are functionally identical. I messed up the 0.7.0 release by forgetting to change the license in the setup.py to LGPLv3+, which made PyPI show the incorrect GPLv3 license. [1]: https://github.com/horazont/aioxmpp [2]: https://pypi.python.org/pypi/aioxmpp [3]: https://docs.zombofant.net/aioxmpp/0.7/ [4]: https://docs.zombofant.net/aioxmpp/0.7/user-guide/quickstart.html [5]: https://lists.zombofant.net/mailman/listinfo/aioxmpp-devel [6]: https://github.com/horazont/aioxmpp/issues [7]: aioxmpp at conference.zombofant.net [8]: https://sks-keyservers.net/pks/lookup?op=get&search=0xE5EDE5AC679E300F AA5A 78FF 508D 8CF4 F355 F682 E5ED E5AC 679E 300F [9]: https://xmpp.org/ [XEP-0004]: https://xmpp.org/extensions/xep-0004.html [XEP-0045]: https://xmpp.org/extensions/xep-0045.html -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJX+QpbAAoJEMBiAyWXYliKcscQALcWdCreg8kz5xxwAPq3JiiD 4K8kVL605h2JVvSFDJ83XqrR23A56gYattyg+D3jJ5/ek5ckTJwvmc39gnJOlQj+ N0KN4QqkikC5ivqSSvJItOKvBowRvCJ3qs5hdKJ93COTVPX+jbRo4swgGhLf8txj zIQApIeKU09YiFqWZjI4WT84ovG/jmVuqAE5QWhl8Hxc6+taKNpgAgwwT/2uSF0q wJkcbHekEeygET4xk6W858k0zNW88ybhBLS31boxsRRqcvHWyyb2DWprb4mWtI9K aJVUJGaq+kPdyWwCctdzpEc8Pvm3yb+s34SWx0eYhNVEOmFEPc5y+D++pzIyiue1 njDl7KURTHRHyCuKxd75ekWjzB7bkdXyr1mAAn6NlUMuIYOU0Nmf6Df8BWrCO/st vP4rPzWOOnCN30E6Q+hl1CZql68rDqoon2Le2zJe+2osljc4lm98hBwsxEXXm9ay u4SkvTuh3GYQmjtcW4AIU5aA8s9PpusJYKLkBZOZV1vx5hSwe8ZoSEHXPOdbhdkm 25b4Vk7yLM6M4JtVn7Iq0KK9wb1pisvEFlkhpVmAMqPNhYEpjvT4nGkV98NfKKla PpCqXiLh7Q+vD7yE7tW3Ss190QlqklMwIg2wZ/OlsJFcsTOlp0WHCgRyzKdhLcAW 1LbrAlB27J4X5NBZTK3a =srny -----END PGP SIGNATURE----- From __peter__ at web.de Sat Oct 8 12:58:24 2016 From: __peter__ at web.de (Peter Otten) Date: Sat, 08 Oct 2016 18:58:24 +0200 Subject: define variables in the txt file using python References: <248e1a72-2a72-4c0d-a7c4-d1da51dc326d@googlegroups.com> <75e82f69-6e5d-4b4b-9b12-a0c8f0b3dc26@googlegroups.com> Message-ID: Xristos Xristoou wrote: > ?? ???????, 8 ????????? 2016 - 12:25:28 ?.?. UTC+3, ? ??????? Xristos > Xristoou ??????: >> hello >> >> >> i have one .txt file and i want to create python script with sys.argv or >> argparse or other package to define some variables in the txt file and i >> take some result. >> >> >> >> txt file maybe like this : >> >> input number 1= %var1% >> input number 2= %var2% >> result = %vresult(var1-var2)% >> >> how can i write a python script to update dynamic variables in the txt >> file ? i dont want with replace i thing so arguments is the better. > > thanks Mr.Peter your code look nice,i have one question,must be running > from cmd line ?i cant to run your code in the python script ? You said > i have one .txt file and i want to create python script with sys.argv or > argparse or other package to define some variables in the txt file and i > take some result. in your original post, so I assumed you wanted to take the arguments from the command line. Generally speaking: expect hints and code samples on this list rather than ready-to-use programs. If you make an effort to adapt the example script we will gladly help you with the parts you cannot solve yourself. To get you started here's the gist of my previous post cast into a function: >>> def fill_template(template, ns): ... parts = template.split("%") ... for i in range(1, len(parts), 2): ... parts[i] = str(eval(parts[i], ns)) ... return "".join(parts) ... >>> print(fill_template("x = %var1%\ny = %var1**2%", dict(var1=22))) x = 22 y = 484 From random832 at fastmail.com Sat Oct 8 14:31:50 2016 From: random832 at fastmail.com (Random832) Date: Sat, 08 Oct 2016 14:31:50 -0400 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: <1475951510.35866.749873569.625EFC13@webmail.messagingengine.com> On Sat, Oct 8, 2016, at 06:12, BartC wrote: > The OP's code however is a good demonstration of how crazy Python's > original for-range loop was: you need to construct a list of N elements > just to be able to count to N. How many years was it until xrange was > introduced? Python 1.4 had it, and that's the earliest version of the documentation that exists on the website. Python 1.0.1 does include rangeobject, with a timestamp on rangeobject.h of 1994-01-01. Python 0.9.1 does not. So it was added some time between the original alt.sources posting (February 1991) and version 1.0 (January 1994). And one could argue that 1.0 was the first "real" release of the language. And of course, you were always free to write: i = 0 while i < 999999: i = i + 1 From random832 at fastmail.com Sat Oct 8 14:45:23 2016 From: random832 at fastmail.com (Random832) Date: Sat, 08 Oct 2016 14:45:23 -0400 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> References: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> On Sat, Oct 8, 2016, at 07:29, Steve D'Aprano wrote: > The oldest version I have access to is the *extremely* primitive 0.9. Not > surprisingly, it doesn't have xrange -- but it lacks a lot of things, > including globals(), map(), named exceptions, "" strings ('' is okay), > exponentiation, and more. Really? I'm not sure exactly what you mean by "named exceptions", but 0.9.1 has RuntimeError, EOFError, TypeError, MemoryError, NameError, SystemError, and KeyboardInterrupt... but exceptions aren't objects, they're strings. They're caught by identity, though - "except 'type error'" fails to catch TypeError, and vice versa. The most interesting thing that I remember noticing about python 0.9.1 is that == and != don't exist - the equality comparison operators were <> and a context-sensitive =. From tjreedy at udel.edu Sat Oct 8 17:58:17 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 8 Oct 2016 17:58:17 -0400 Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/8/2016 2:56 AM, Steve D'Aprano wrote: >> Just copy and paste it into your Python IDLE and let me know what you get > > > I don't normally use IDLE, and you shouldn't assume that everyone does. > *That* is the extra information we need to solve the problem: > > The IDLE interactive interpreter in Python 3 does not seem to allow you to > paste multiple lines at once. Actually, it does. I pasted the following multiline statement. >>> while not done: for event in pygame.event.get(): # User did something if event.type == pygame.QUIT: # If user clicked close done = True # Flag that we are done so we exit this loop Traceback (most recent call last): File "", line 1, in while not done: NameError: name 'done' is not defined ie, the single statement was compiled. >>>> # Define some colors > BLACK = (0, 0, 0) > WHITE = (255, 255, 255) > SyntaxError: multiple statements found while compiling a single statement Here you pasted multiple *statements* (on separate lines). This is a known difference between console and IDLE Shell. It is partly related to being line versus statement oriented, partly due to window paste being handled by system console versus tk => tkinter => IDLE run by python. Note that in a console, each line is preceded by the primary prompt ('>>> '). Ditto if one types the lines into IDLE. There has been some discussion of changing IDLE, but it would not be trivial. The console splits pasted text into lines and feeds a line at a time to Python. IDLE would have to intercept the pasted text before tk displays it, parse (split) the paste into statements (rather than lines), and display and run one statement at a time. I would rather spend the effort on making it possible to run code in a new editor window without explicitly saving it. -- Terry Jan Reedy From chrischris201444 at gmail.com Sat Oct 8 18:10:29 2016 From: chrischris201444 at gmail.com (chrischris201444 at gmail.com) Date: Sat, 8 Oct 2016 15:10:29 -0700 (PDT) Subject: delete pixel from the raster image with specific range value Message-ID: <763f4140-b739-44db-ab06-49206e11a503@googlegroups.com> any idea how to delete pixel from the raster image with specific range value using numpy/scipy or gdal? for example i have a raster image with the 5 class : 1. 0-100 2. 100-200 3. 200-300 4. 300-500 5. 500-1000 and i want to delete class 1 range value or maybe class 1,2,4,5 From greg.ewing at canterbury.ac.nz Sat Oct 8 20:14:52 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 09 Oct 2016 13:14:52 +1300 Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> Message-ID: Cai Gengyang wrote: > Somehow it still doesnt work --- it keeps giving the syntaxerror, > inconsistent use of tabs and indentation message EVEN though i use only the > enter and space buttons and never touched the tab button a single time. There was another thread about this a short time ago. It turns out that when you press enter in the IDLE REPL, it auto-indents the next line -- but it does it using *tabs*, not spaces. So if you do your manual indentation with spaces, or paste something in that uses spaces, you can easily end up with invalid tab/space mixtures. The solution is to always use the tab key for indentation when working interactively in IDLE. I think this should be reported as a bug in IDLE, because it's very unintuitive behaviour, and a beginner has little chance of figuring out what's going on by themselves. -- Greg From steve+python at pearwood.info Sat Oct 8 21:52:40 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 09 Oct 2016 12:52:40 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> Message-ID: <57f9a2e9$0$1616$c3e8da3$5496439d@news.astraweb.com> On Sun, 9 Oct 2016 05:45 am, Random832 wrote: > On Sat, Oct 8, 2016, at 07:29, Steve D'Aprano wrote: >> The oldest version I have access to is the *extremely* primitive 0.9. Not >> surprisingly, it doesn't have xrange -- but it lacks a lot of things, >> including globals(), map(), named exceptions, "" strings ('' is okay), >> exponentiation, and more. > > Really? I'm not sure exactly what you mean by "named exceptions", but > 0.9.1 has RuntimeError, EOFError, TypeError, MemoryError, NameError, > SystemError, and KeyboardInterrupt... but exceptions aren't objects, > they're strings. Yes, that's what I mean. I may have worded it poorly, but if you look at a modern Python, and compare it to 0.9.1: py> globls Traceback (most recent call last): File "", line 1, in NameError: name 'globls' is not defined >>> globls Unhandled exception: undefined name: globls Stack backtrace (innermost last): File "", line 1 the exception handling is different. As you say, exceptions are just strings, and there's no exception hierarchy. > The most interesting thing that I remember noticing about python 0.9.1 > is that == and != don't exist - the equality comparison operators were > <> and a context-sensitive =. And very limited choice for string delimiters: no raw strings, triple-quote strings, or "". -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Oct 8 21:53:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 9 Oct 2016 12:53:08 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> References: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> Message-ID: On Sun, Oct 9, 2016 at 5:45 AM, Random832 wrote: > On Sat, Oct 8, 2016, at 07:29, Steve D'Aprano wrote: >> The oldest version I have access to is the *extremely* primitive 0.9. Not >> surprisingly, it doesn't have xrange -- but it lacks a lot of things, >> including globals(), map(), named exceptions, "" strings ('' is okay), >> exponentiation, and more. > > Really? I'm not sure exactly what you mean by "named exceptions", but > 0.9.1 has RuntimeError, EOFError, TypeError, MemoryError, NameError, > SystemError, and KeyboardInterrupt... but exceptions aren't objects, > they're strings. > > They're caught by identity, though - "except 'type error'" fails to > catch TypeError, and vice versa. Fascinating! What about: except sys.intern('type error') ? Or does interning of strings not exist yet :) ChrisA From steve+python at pearwood.info Sat Oct 8 22:19:38 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 09 Oct 2016 13:19:38 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> Message-ID: <57f9a963$0$1596$c3e8da3$5496439d@news.astraweb.com> On Sun, 9 Oct 2016 12:53 pm, Chris Angelico wrote: >> They're caught by identity, though - "except 'type error'" fails to >> catch TypeError, and vice versa. > > Fascinating! What about: except sys.intern('type error') ? Or does > interning of strings not exist yet :) >>> intern Unhandled exception: undefined name: intern Stack backtrace (innermost last): File "", line 1 >>> import sys >>> sys.intern Unhandled exception: undefined name: intern Stack backtrace (innermost last): File "", line 1 There's no `is` operator, or id() function, so it's impossible to tell whether small ints and strings are interned/cached except by reading the source. No longs (let alone unified int/long). The weirdest thing seems to be that {} exists (and is called a dictionary, not dict) but there's no dict builder syntax: >>> type({}) >>> d = {'a': 1} Parsing error: file , line 1: d = {'a': 1} ^ Unhandled exception: run-time error: syntax error and keys have to be strings: >>> d = {} >>> d[1] = 2 Unhandled exception: type error: illegal argument type for built-in operation Stack backtrace (innermost last): File "", line 1 >>> d['1'] = 2 >>> print d {'1': 2} There are functions (def) but not lambda, but no `class` statement. Primitive days indeed. And yet already usable. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gengyangcai at gmail.com Sat Oct 8 22:51:52 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 8 Oct 2016 19:51:52 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> Message-ID: <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> This is my latest result : I copy and pasted one line at a time into the IDLE and used ONLY the "enter-return" button to move on to the next line and this time I didnt get an indentation error but instead a traceback error >>> rect_x = 50 >>> rect_y = 50 >>> while not done: for event in pygame.event.get(): if event.type == pygame.QUIT: done = True Traceback (most recent call last): File "", line 1, in while not done: NameError: name 'done' is not defined On Sunday, October 9, 2016 at 8:15:07 AM UTC+8, Gregory Ewing wrote: > Cai Gengyang wrote: > > Somehow it still doesnt work --- it keeps giving the syntaxerror, > > inconsistent use of tabs and indentation message EVEN though i use only the > > enter and space buttons and never touched the tab button a single time. > > There was another thread about this a short time ago. > It turns out that when you press enter in the IDLE REPL, > it auto-indents the next line -- but it does it using > *tabs*, not spaces. So if you do your manual indentation > with spaces, or paste something in that uses spaces, you > can easily end up with invalid tab/space mixtures. > > The solution is to always use the tab key for indentation > when working interactively in IDLE. > > I think this should be reported as a bug in IDLE, because > it's very unintuitive behaviour, and a beginner has little > chance of figuring out what's going on by themselves. > > -- > Greg From steve+python at pearwood.info Sat Oct 8 23:42:19 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 09 Oct 2016 14:42:19 +1100 Subject: SyntaxError: multiple statements found while compiling a single statement References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> Message-ID: <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> On Sun, 9 Oct 2016 01:51 pm, Cai Gengyang wrote: > This is my latest result : I copy and pasted one line at a time into the > IDLE and used ONLY the "enter-return" button to move on to the next line > and this time I didnt get an indentation error but instead a traceback > error: > Traceback (most recent call last): > File "", line 1, in > while not done: > NameError: name 'done' is not defined Right. That's because 'done' is not defined. Why don't you try my suggestion of saving the code into a .py file, then using the File > Open command to open it? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From gengyangcai at gmail.com Sat Oct 8 23:51:49 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sat, 8 Oct 2016 20:51:49 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> Message-ID: <1735f189-4d61-4497-9b57-53d8e432f805@googlegroups.com> I defined both done and pygame in this piece of code, but now i get a new error that i have never seen before, an AttributeError >>> rect_x = 50 >>> rect_y = 50 >>> done = False >>> pygame = True >>> while not done: for event in pygame.event.get(): if event.type == pygame.QUIT: done = True Traceback (most recent call last): File "", line 2, in for event in pygame.event.get(): AttributeError: 'bool' object has no attribute 'event' On Sunday, October 9, 2016 at 11:42:33 AM UTC+8, Steve D'Aprano wrote: > On Sun, 9 Oct 2016 01:51 pm, Cai Gengyang wrote: > > > This is my latest result : I copy and pasted one line at a time into the > > IDLE and used ONLY the "enter-return" button to move on to the next line > > and this time I didnt get an indentation error but instead a traceback > > error: > > > Traceback (most recent call last): > > File "", line 1, in > > while not done: > > NameError: name 'done' is not defined > > Right. > > That's because 'done' is not defined. > > Why don't you try my suggestion of saving the code into a .py file, then > using the File > Open command to open it? > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. From john_ladasky at sbcglobal.net Sun Oct 9 01:20:42 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Sat, 8 Oct 2016 22:20:42 -0700 (PDT) Subject: PyQt5, OpenGL, where to start, minimal example code? In-Reply-To: References: <264bb3df-012d-4072-ba48-408f7aabf01b@googlegroups.com> <05047887-1031-4A3A-811D-1C6EF2DC2B90@riverbankcomputing.com> <08AA3E2B-F8A0-4AA6-9BFD-DAC93B0AFDD7@riverbankcomputing.com> Message-ID: <6ca4159a-b722-4021-a03e-783080473fe2@googlegroups.com> Well, I've made some progress. My program doesn't draw any 3D objects yet, but it creates an OpenGL rendering window, binds the OpenGL functions, and generates no errors. Here's the corrected initializeGL method: def initializeGL(self): c = self.context() f = QSurfaceFormat() # The default p = QOpenGLVersionProfile(f) self.GL = c.versionFunctions(p) super().initializeGL() On Tuesday, October 4, 2016 at 12:56:53 AM UTC-7, Phil Thompson wrote: > On 4 Oct 2016, at 5:57 am, John Ladasky wrote: > > > > On Monday, October 3, 2016 at 1:30:29 AM UTC-7, Phil Thompson wrote: > >> On 3 Oct 2016, at 4:29 am, John Ladasky wrote: > > If I ever understand a GUI like PyQt5 well enough, I'd like to contribute to its documentation. Sigh. > > If you are an OpenGL expert Ha! While I have experience with quite a few scientific computing Python tools (numpy, scipy, matplotlib, pandas, scikit-learn), I know less about OpenGL than I do PyQt5. There's always more to learn. >then you could help a lot by answering my questions about how individual functions should be bound. In all the examples I've encountered on the Net (most of which are written in C++ -- or if they are PyQt examples, they're binding older versions of OpenGL), the preferred approach seems to be the one that I've taken above. If you want to call an OpenGL function, you have to look it up in the self.GL namespace. From tjreedy at udel.edu Sun Oct 9 01:36:39 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 9 Oct 2016 01:36:39 -0400 Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <1735f189-4d61-4497-9b57-53d8e432f805@googlegroups.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> <1735f189-4d61-4497-9b57-53d8e432f805@googlegroups.com> Message-ID: On 10/8/2016 11:51 PM, Cai Gengyang wrote: >>>> pygame = True >>>> while not done: > for event in pygame.event.get(): > Traceback (most recent call last): > File "", line 2, in > for event in pygame.event.get(): > AttributeError: 'bool' object has no attribute 'event' pygame == True and True has not attribute 'event'. Believe the error messages until you have real reason to question them. They are perhaps 99.9% accurate by now/ -- Terry Jan Reedy From steve+python at pearwood.info Sun Oct 9 02:41:43 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 09 Oct 2016 17:41:43 +1100 Subject: SyntaxError: multiple statements found while compiling a single statement References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> <1735f189-4d61-4497-9b57-53d8e432f805@googlegroups.com> Message-ID: <57f9e6a9$0$1606$c3e8da3$5496439d@news.astraweb.com> On Sun, 9 Oct 2016 02:51 pm, Cai Gengyang wrote: > I defined both done and pygame in this piece of code, but now i get a new > error that i have never seen before, an AttributeError AttributeError usually means you have the wrong kind of object: py> mylist = {} # oops, a dict not a list py> mylist.append(1) Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'append' or sometimes you have the right object but misspelled the attribute or method: py> mylist = [] py> mylist.apend(1) # oops, spelling error Traceback (most recent call last): File "", line 1, in AttributeError: 'list' object has no attribute 'apend' py> mylist.append(1) py> print(mylist) [1] >>>> rect_x = 50 >>>> rect_y = 50 >>>> done = False >>>> pygame = True Why have you defined pygame = True? Is that what the code on the website does? My guess is that you are supposed to say: import pygame Why don't you try my suggestion of saving the code into a .py file, then using the File > Open command to open it? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rustompmody at gmail.com Sun Oct 9 02:51:14 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 8 Oct 2016 23:51:14 -0700 (PDT) Subject: SyntaxError: multiple statements found while compiling a single statement In-Reply-To: <57f9e6a9$0$1606$c3e8da3$5496439d@news.astraweb.com> References: <72bab892-e8ec-4ed8-8ccb-614ef976b69d@googlegroups.com> <57f88f2b$0$1603$c3e8da3$5496439d@news.astraweb.com> <56e8ac96-d0fb-4b71-befb-ce9ce17fded3@googlegroups.com> <57f8988b$0$1614$c3e8da3$5496439d@news.astraweb.com> <93664a22-7625-425d-8869-78d562cbdb4b@googlegroups.com> <8d147aec-123b-40a3-b8fd-28f9d0bcdcda@googlegroups.com> <57f9bc9c$0$1622$c3e8da3$5496439d@news.astraweb.com> <1735f189-4d61-4497-9b57-53d8e432f805@googlegroups.com> <57f9e6a9$0$1606$c3e8da3$5496439d@news.astraweb.com> Message-ID: <70b50075-7955-459c-82a1-b0ff5bcb790c@googlegroups.com> On Sunday, October 9, 2016 at 12:12:06 PM UTC+5:30, Steve D'Aprano wrote: > On Sun, 9 Oct 2016 02:51 pm, Cai Gengyang wrote: > > > I defined both done and pygame in this piece of code, but now i get a new > > error that i have never seen before, an AttributeError > > AttributeError usually means you have the wrong kind of object: > > py> mylist = {} # oops, a dict not a list > py> mylist.append(1) > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'dict' object has no attribute 'append' > > > or sometimes you have the right object but misspelled the attribute or > method: > > py> mylist = [] > py> mylist.apend(1) # oops, spelling error > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'list' object has no attribute 'apend' > py> mylist.append(1) > py> print(mylist) > [1] > > > > >>>> rect_x = 50 > >>>> rect_y = 50 > >>>> done = False > >>>> pygame = True > > Why have you defined pygame = True? Is that what the code on the website > does? > > My guess is that you are supposed to say: > > import pygame > > > > Why don't you try my suggestion of saving the code into a .py file, then > using the File > Open command to open it? One meta-suggestion: It may be that with python you can go further in less time than with older languages like C++, Java etc. But less time doesn?t mean no time ? You still need to learn, to walk the learning curve In short: Dont skimp on the tutorial: https://docs.python.org/3/tutorial/ even (especially) if you find the examples uninteresting/irrelevant From greg.ewing at canterbury.ac.nz Sun Oct 9 03:48:13 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 09 Oct 2016 20:48:13 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> Message-ID: <57F9F63D.8080300@canterbury.ac.nz> Here's the first part of the essay I said I'd write about monads: http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html Hope it's useful, Greg From no.email at nospam.invalid Sun Oct 9 03:54:06 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 09 Oct 2016 00:54:06 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: <87bmyuufdt.fsf@nightsong.com> Gregory Ewing writes: > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html https://byorgey.wordpress.com/2009/01/12/abstraction-intuition-and-the-monad-tutorial-fallacy/ From greg.ewing at canterbury.ac.nz Sun Oct 9 03:57:36 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 09 Oct 2016 20:57:36 +1300 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: <57f8d883$0$22142$c3e8da3$5496439d@news.astraweb.com> <1475952323.37549.749880945.424F0409@webmail.messagingengine.com> Message-ID: Chris Angelico wrote: > Fascinating! What about: except sys.intern('type error') ? Or does > interning of strings not exist yet :) Even if it was, I don't think there was any guarantee that the "official" strings representing those exceptions would be interned. You were supposed to use the provided names to refer to built-in exceptions, to make sure you were referring to the same object. Doing so also had the nice benefit that when exceptions became classes, most existing exception-handling code continued to work unchanged. It's almost like Guido had a time machine! -- Greg From no.email at nospam.invalid Sun Oct 9 04:14:40 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 09 Oct 2016 01:14:40 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: <877f9iuefj.fsf@nightsong.com> Gregory Ewing writes: > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html Erratum in Haskell section: Lists in Haskell are linked lists, and [h|t] represents a list whose first element is h and the rest of the list is t. [h|t] should say h:t . From greg.ewing at canterbury.ac.nz Sun Oct 9 06:27:08 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 09 Oct 2016 23:27:08 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <877f9iuefj.fsf@nightsong.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <877f9iuefj.fsf@nightsong.com> Message-ID: Paul Rubin wrote: > [h|t] should say h:t . Thanks, corrected. (You can probably tell I'm not a regular Haskell user. Not sure where I got [h|t] from -- maybe I was thinking of Prolog?) -- Greg From greg.ewing at canterbury.ac.nz Sun Oct 9 06:35:50 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Sun, 09 Oct 2016 23:35:50 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <87bmyuufdt.fsf@nightsong.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> Message-ID: Paul Rubin wrote: > Gregory Ewing writes: > >>http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html > > https://byorgey.wordpress.com/2009/01/12/abstraction-intuition-and-the-monad-tutorial-fallacy/ Well, at least I refrained from saying that monads are like burritos! I don't claim that everyone will suddenly understand everything there is to know about monads after reading my essay. But maybe it will help some people understand a little bit. It certainly couldn't help anyone if I didn't write it. -- Greg From bc at freeuk.com Sun Oct 9 07:17:18 2016 From: bc at freeuk.com (BartC) Date: Sun, 9 Oct 2016 12:17:18 +0100 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> Message-ID: On 09/10/2016 11:35, Gregory Ewing wrote: > Paul Rubin wrote: >> Gregory Ewing writes: >> >>> http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html >>> >> >> https://byorgey.wordpress.com/2009/01/12/abstraction-intuition-and-the-monad-tutorial-fallacy/ >> > > Well, at least I refrained from saying that monads are > like burritos! > > I don't claim that everyone will suddenly understand > everything there is to know about monads after reading > my essay. But maybe it will help some people understand > a little bit. It certainly couldn't help anyone if I > didn't write it. > I think the easiest way to deal with monads is pretend they don't exist. (I've no idea what they are, but from glancing at your essay, they look complicated.) Even if someone did struggle for a week to figure out what they mean and how to use them, how is anyone else supposed to understand their code? How is anyone going to have confidence that any such code will work? And Python (suddenly remembering which group this is) is supposed to be more accessible than other languages. It's not Haskell. -- Bartc From gengyangcai at gmail.com Sun Oct 9 08:01:52 2016 From: gengyangcai at gmail.com (Cai Gengyang) Date: Sun, 9 Oct 2016 05:01:52 -0700 (PDT) Subject: Function to take the minimum of 3 numbers Message-ID: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> I'm moving on to chapter 9 (http://programarcadegames.com/index.php?lang=en&chapter=lab_functions) of programarcadegames for the time being and going back to chapter 8 later (its just fucking frustrating and doesn't seem to work for the time being and I have a very bad temper). At least for chapter 9, I got the first part correct at first try --- define a function that takes and prints the smallest of 3 numbers. I pasted it here for reference and discussion. The code can also be further modified to include a clause that says that if two numbers tie for smallest, choose any of the two numbers. (I will attempt to do it and then post it here again for future discussion with users here >>> def min3(a, b, c): min3 = a if b < min3: min3 = b if c < min3: min3 = c if b < c: min3 = b return min3 >>> print(min3(4, 7, 5)) 4 From ben.usenet at bsb.me.uk Sun Oct 9 09:09:04 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Sun, 09 Oct 2016 14:09:04 +0100 Subject: Function to take the minimum of 3 numbers References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: <8737k5vfdb.fsf@bsb.me.uk> Cai Gengyang writes: > I'm moving on to chapter 9 > (http://programarcadegames.com/index.php?lang=en&chapter=lab_functions) > of programarcadegames for the time being and going back to chapter 8 > later (its just fucking frustrating and doesn't seem to work for the > time being and I have a very bad temper). > > At least for chapter 9, I got the first part correct at first try --- > define a function that takes and prints the smallest of 3 numbers. I > pasted it here for reference and discussion. Presumably you must do this without Python's min function? > The code can also be > further modified to include a clause that says that if two numbers tie > for smallest, choose any of the two numbers. What would be the point? Your code already does that. >>>> def min3(a, b, c): > min3 = a > if b < min3: > min3 = b > if c < min3: > min3 = c > if b < c: > min3 = b > return min3 The last if is not needed. Forced to pretend that there is no min function already, I'd be tempted to write def min3(a, b, c): def min2(x, y): return x if x < y else y; return min2(a, min2(b, c)) -- Ben. From bc at freeuk.com Sun Oct 9 09:41:29 2016 From: bc at freeuk.com (BartC) Date: Sun, 9 Oct 2016 14:41:29 +0100 Subject: Function to take the minimum of 3 numbers In-Reply-To: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On 09/10/2016 13:01, Cai Gengyang wrote: > I'm moving on to chapter 9 (http://programarcadegames.com/index.php?lang=en&chapter=lab_functions) of programarcadegames for the time being and going back to chapter 8 later (its just fucking frustrating and doesn't seem to work for the time being and I have a very bad temper). > > At least for chapter 9, I got the first part correct at first try --- define a function that takes and prints the smallest of 3 numbers. I pasted it here for reference and discussion. The code can also be further modified to include a clause that says that if two numbers tie for smallest, choose any of the two numbers. (I will attempt to do it and then post it here again for future discussion with users here > > >>>> def min3(a, b, c): > min3 = a > if b < min3: > min3 = b > if c < min3: > min3 = c > if b < c: > min3 = b > return min3 > >>>> print(min3(4, 7, 5)) > 4 > The exercise says you must use an if/elif chain. The first thing that comes to mind is: def min3(a,b,c): if a<=b and a<=c: return a elif b<=a and b<=c: return b else: return c The bit about numbers tying for smallest is not meaningful; the caller can't tell if a minimum value of 42 came from a, b or c. -- Bartc From no.email at nospam.invalid Sun Oct 9 11:10:34 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 09 Oct 2016 08:10:34 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <877f9iuefj.fsf@nightsong.com> Message-ID: <8737k5v9qt.fsf@nightsong.com> Gregory Ewing writes: > Not sure where I got [h|t] from -- maybe I was thinking of Prolog?) I've never used Prolog. Erlang is said to have Prolog-like syntax and it uses [h|t], so maybe Prolog uses it too. (Erlang was originally written in Prolog). From rosuav at gmail.com Sun Oct 9 11:24:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 02:24:16 +1100 Subject: Function to take the minimum of 3 numbers In-Reply-To: References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On Mon, Oct 10, 2016 at 12:41 AM, BartC wrote: > The exercise says you must use an if/elif chain. The first thing that comes > to mind is: > > def min3(a,b,c): > if a<=b and a<=c: > return a > elif b<=a and b<=c: > return b > else: > return c > > The bit about numbers tying for smallest is not meaningful; the caller can't > tell if a minimum value of 42 came from a, b or c. You assume that all equal numbers are identical. (And I say "numbers" only because the problem definition did. This code does not require numbers necessarily.) >>> min3(1, True, 1.0) 1 >>> min3(1.0, True, 1) 1.0 >>> min3(True, 1.0, 1) True >>> 1 == 1.0 == True True >>> x = 1.0 >>> y = 1.0 >>> z = 1.0 >>> min3(x, y, z) is x True >>> min3(x, y, z) is y False >>> min3(x, y, z) is z False Your function provably returns the first number if given equal numbers. That stipulation is far from meaningless; it explicitly permits any means of selection, as long as one of the input values is returned. Thus, for instance, this must always be valid: >>> assert min3(x, y, z) in (x, y, z) So this solution would NOT be valid: def min2(a, b): return (a+b)/2 - abs(a-b)/2 def min3(a, b, c): return min2(a, min2(b, c)) It's mathematically perfect but inappropriate to this challenge, as it violates the given rule. ChrisA From rosuav at gmail.com Sun Oct 9 11:26:09 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 02:26:09 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On Mon, Oct 10, 2016 at 12:26 AM, Dennis Lee Bieber wrote: > {This response is delayed as I'm waiting for the program to complete so I > can get the run time} > {Well... it's been near 24 hours and still merrily scrolling sums on my > console -- so I'm going to kill the running program} Eight BILLION of them. I wouldn't bother waiting for that. ChrisA From breamoreboy at gmail.com Sun Oct 9 12:04:25 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 9 Oct 2016 09:04:25 -0700 (PDT) Subject: Function to take the minimum of 3 numbers In-Reply-To: References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On Sunday, October 9, 2016 at 2:41:41 PM UTC+1, BartC wrote: > On 09/10/2016 13:01, Cai Gengyang wrote: > > I'm moving on to chapter 9 (http://programarcadegames.com/index.php?lang=en&chapter=lab_functions) of programarcadegames for the time being and going back to chapter 8 later (its just fucking frustrating and doesn't seem to work for the time being and I have a very bad temper). > > > > At least for chapter 9, I got the first part correct at first try --- define a function that takes and prints the smallest of 3 numbers. I pasted it here for reference and discussion. The code can also be further modified to include a clause that says that if two numbers tie for smallest, choose any of the two numbers. (I will attempt to do it and then post it here again for future discussion with users here > > > > > >>>> def min3(a, b, c): > > min3 = a > > if b < min3: > > min3 = b > > if c < min3: > > min3 = c > > if b < c: > > min3 = b > > return min3 > > > >>>> print(min3(4, 7, 5)) > > 4 > > > > The exercise says you must use an if/elif chain. The first thing that > comes to mind is: > > def min3(a,b,c): > if a<=b and a<=c: > return a > elif b<=a and b<=c: > return b > else: > return c > > The bit about numbers tying for smallest is not meaningful; the caller > can't tell if a minimum value of 42 came from a, b or c. > > -- > Bartc The Pythonic way if b >= a <= c: ... Kindest regards. Mark Lawrence. From jussi.piitulainen at helsinki.fi Sun Oct 9 13:33:43 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Sun, 09 Oct 2016 20:33:43 +0300 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: Message-ID: Chris Angelico writes: > On Mon, Oct 10, 2016 at 12:26 AM, Dennis Lee Bieber wrote: >> {This response is delayed as I'm waiting for the program to complete so I >> can get the run time} >> {Well... it's been near 24 hours and still merrily scrolling sums on my >> console -- so I'm going to kill the running program} > > Eight BILLION of them. I wouldn't bother waiting for that. Eight billion is a big number. I know from experience that computers can compute a big number of computations in a short time. Therefore, I would expect the computer to compute eight billion computations in a short time. Maybe. Apparently some big numbers are more big than others. From bc at freeuk.com Sun Oct 9 13:57:39 2016 From: bc at freeuk.com (BartC) Date: Sun, 9 Oct 2016 18:57:39 +0100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On 09/10/2016 18:33, Jussi Piitulainen wrote: > Chris Angelico writes: > >> On Mon, Oct 10, 2016 at 12:26 AM, Dennis Lee Bieber wrote: >>> {This response is delayed as I'm waiting for the program to complete so I >>> can get the run time} >>> {Well... it's been near 24 hours and still merrily scrolling sums on my >>> console -- so I'm going to kill the running program} >> >> Eight BILLION of them. I wouldn't bother waiting for that. > > Eight billion is a big number. I know from experience that computers can > compute a big number of computations in a short time. Therefore, I would > expect the computer to compute eight billion computations in a short > time. Maybe. Compute, maybe. But the OP is printing something out at each step. Printing to a console is very, very slow, compared with even with running CPython. On my machine, CPython might run through an empty loop at 10 to 20M iterations per second, but is some 4000 times slower if it has to print one "A" per line at each step. The OP might try redirecting the output to a file (assuming he doesn't want to read every single line of output as it's produced). That will be much faster, although it depends on the workload in the run() function being executed. Or only print every 1000th line or something if progress has to be monitored. -- Bartc From radarrisat at gmail.com Sun Oct 9 14:56:43 2016 From: radarrisat at gmail.com (Risat Haque) Date: Sun, 9 Oct 2016 11:56:43 -0700 (PDT) Subject: Python Data base help Message-ID: <2516a370-483f-4b51-b369-77404c9b98da@googlegroups.com> Hey, i have a data base filled with numbers from a recent drone flight. IT contains, alt, long, lat, and time. In python, i want to ask the user to put in a time : askTime = (input("Choose a time (HourMinSec):")) With this, I need it to search through the entire data base to find that number. ("UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h") EX: Choose a time (HourMinSec): 8:58:24 Data: UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h" How can I do this? From alister.ware at ntlworld.com Sun Oct 9 15:24:09 2016 From: alister.ware at ntlworld.com (alister) Date: Sun, 09 Oct 2016 19:24:09 GMT Subject: BeautifulSoup help !! References: <7f31537b-8fd4-4a68-81e5-206010714bb7@googlegroups.com> <36uJz.57117$gT.12654@fx29.am4> <57f677f2$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, 07 Oct 2016 03:12:32 +1100, Steve D'Aprano wrote: > On Fri, 7 Oct 2016 02:30 am, alister wrote: > >> On Thu, 06 Oct 2016 08:22:05 -0700, desolate.soul.me wrote: >> >>> So I've just started up with python and an assignment was given to me >>> by a company as an recruitment task. >>> >> so by your own admission you have just started with python yet you >> consider your self suitable for employment? > > What's your problem Alister? Do you think that junior devs aren't > allowed to ask for help? > I read the original post as I have just started learning python & have applied for a job as a python programmer (which struck me as being a bit ambitious) you will see that I replied quite promptly & (I think) constructively to the OP when he clarified the situation. -- I fill MY industrial waste containers with old copies of the "WATCHTOWER" and then add HAWAIIAN PUNCH to the top ... They look NICE in the yard ... From irmen.NOSPAM at xs4all.nl Sun Oct 9 15:27:11 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Sun, 9 Oct 2016 21:27:11 +0200 Subject: Python Data base help In-Reply-To: <2516a370-483f-4b51-b369-77404c9b98da@googlegroups.com> References: <2516a370-483f-4b51-b369-77404c9b98da@googlegroups.com> Message-ID: <57fa9a11$0$880$e4fe514c@news.xs4all.nl> On 9-10-2016 20:56, Risat Haque wrote: > Hey, i have a data base filled with numbers from a recent drone flight. IT contains, alt, long, lat, and time. > In python, i want to ask the user to put in a time : > askTime = (input("Choose a time (HourMinSec):")) > > With this, I need it to search through the entire data base to find that number. > > ("UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h") > > EX: > > Choose a time (HourMinSec): 8:58:24 > Data: UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h" > > How can I do this? > What is your 'database'? >From the little information you provided it seems that it is just a text file where every drone measurement is on a line. So simply read every line and check if the time entered is in that line, then print it. Irmen From alister.ware at ntlworld.com Sun Oct 9 15:29:34 2016 From: alister.ware at ntlworld.com (alister) Date: Sun, 09 Oct 2016 19:29:34 GMT Subject: Passing Variable to Function References: Message-ID: On Fri, 07 Oct 2016 19:06:54 +0000, John McKenzie wrote: > Brendan and Alister, thank you both for responding. > > I am very inexperienced with python, but not new to computers so on my > own I realized the strings vs number mistake. (I seem to work with and > learn about Python for a few weeks at a time with 6 to 12 months in > between. I spend most of my time simply lurking here because I am very much an amateur (although in the distant pas I was a microcontroler firmware programmer in 6800 assembler). Primarily program in python for personal pleasure. this ws on occasion where i felt I was able to offer as good an explanation as anyone on the list so was glad to give something back -- Why use Windows, since there is a door? From breamoreboy at gmail.com Sun Oct 9 15:42:38 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 9 Oct 2016 12:42:38 -0700 (PDT) Subject: Python Data base help In-Reply-To: <2516a370-483f-4b51-b369-77404c9b98da@googlegroups.com> References: <2516a370-483f-4b51-b369-77404c9b98da@googlegroups.com> Message-ID: <9ed2b137-24e2-471b-ac15-1926930acca2@googlegroups.com> On Sunday, October 9, 2016 at 7:56:57 PM UTC+1, Risat Haque wrote: > Hey, i have a data base filled with numbers from a recent drone flight. IT contains, alt, long, lat, and time. > In python, i want to ask the user to put in a time : > askTime = (input("Choose a time (HourMinSec):")) > > With this, I need it to search through the entire data base to find that number. > > ("UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h") > > EX: > > Choose a time (HourMinSec): 8:58:24 > Data: UTC: 8:58:24 lat: 50.97 long: -114.05 Alt: 1047.40m SOG: 1.04 km/h" > > How can I do this? I'd use strptime[1] to check the input then search your database, whatever that may be, based on that. [1] https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime Kindest regards. Mark Lawrence. From orgnut at yahoo.com Sun Oct 9 16:46:10 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Sun, 9 Oct 2016 13:46:10 -0700 Subject: Function to take the minimum of 3 numbers In-Reply-To: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On 10/09/2016 05:01 AM, Cai Gengyang wrote: >>>> def min3(a, b, c): > min3 = a > if b < min3: > min3 = b > if c < min3: > min3 = c > if b < c: > min3 = b > return min3 > >>>> print(min3(4, 7, 5)) > 4 > This is NOT a recommendation here, just a different way of looking at the problem (and is probably cheating anyway)... def min3(a, b, c): return sorted([a, b, c])[0] # Create a list, sort it and return the 1st element It can be extended to take any number of parameters: def minn(*args): return sorted(args)[0] The * syntax takes all the parameters and puts them into a single tuple. This is _almost_ the same as the built-in min() function ? the built-in requires a single (sequence) parameter, my minn() function requires multiple individual parameters. (Both versions fail with zero parameters.) -- -=- Larry -=- From chrischris201444 at gmail.com Sun Oct 9 16:55:41 2016 From: chrischris201444 at gmail.com (chrischris201444 at gmail.com) Date: Sun, 9 Oct 2016 13:55:41 -0700 (PDT) Subject: calculator between raster Message-ID: any idea how to calculator rasters images ?like ((raster1+raster2)*(raster4*9)} for example. maybe is easy that question but i dont work again with raster and python i have gdal,numpy,scipy,pygeoprocesing and more From davidbenny2000 at gmail.com Sun Oct 9 17:00:14 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sun, 9 Oct 2016 14:00:14 -0700 (PDT) Subject: How to read maple .m file into Python? Message-ID: <88402c49-c69b-491d-a53b-b7b2db7019ca@googlegroups.com> i saved a list of matrix of algebra into .m file in maple How to read and import into Python for sympy to use? From davidbenny2000 at gmail.com Sun Oct 9 17:08:22 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sun, 9 Oct 2016 14:08:22 -0700 (PDT) Subject: How to split a large nested for loop and distribute to hundreds of Amazon instance to run this kind of Python code? Message-ID: https://social.msdn.microsoft.com/Forums/vstudio/en-US/5f0a9a51-a256-4671-a5fc-e213949e7204/how-to-refactor-3-nested-for-loop-into-smaller-for-loop-assume-each-of-them-independent?forum=csharpgeneral I wrote a algorithm to split for loop to generate maplesoft code for limited memory Assume I convert this algorithm into Python script and use sympy to do dsolve How to distribute to hundreds of Amazon instance to run each smaller for loop From auriocus at gmx.de Sun Oct 9 17:08:48 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 9 Oct 2016 23:08:48 +0200 Subject: calculator between raster In-Reply-To: References: Message-ID: Am 09.10.16 um 22:55 schrieb chrischris201444 at gmail.com: > any idea how to calculator rasters images ?like ((raster1+raster2)*(raster4*9)} > for example. > maybe is easy that question but i dont work again with raster and python > i have gdal,numpy,scipy,pygeoprocesing and more With PIL you can read an image file into numpy array, do your calculations the way you wrote, and transform it back to an image (file, or display). Try: from PIL import Image import numpy arr = numpy.asarray(Image.open('somefile.png')) Christian From chrischris201444 at gmail.com Sun Oct 9 17:13:28 2016 From: chrischris201444 at gmail.com (chrischris201444 at gmail.com) Date: Sun, 9 Oct 2016 14:13:28 -0700 (PDT) Subject: calculator between raster In-Reply-To: References: Message-ID: <36896ad4-0681-4558-87bd-0f03fccf69c0@googlegroups.com> ?? ???????, 9 ????????? 2016 - 11:55:54 ?.?. UTC+3, ? ??????? chrischr... at gmail.com ??????: > any idea how to calculator rasters images ?like ((raster1+raster2)*(raster4*9)} > for example. > maybe is easy that question but i dont work again with raster and python > i have gdal,numpy,scipy,pygeoprocesing and more is easy to show me an example?i am very begin From breamoreboy at gmail.com Sun Oct 9 17:18:34 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 9 Oct 2016 14:18:34 -0700 (PDT) Subject: How to read maple .m file into Python? In-Reply-To: <88402c49-c69b-491d-a53b-b7b2db7019ca@googlegroups.com> References: <88402c49-c69b-491d-a53b-b7b2db7019ca@googlegroups.com> Message-ID: On Sunday, October 9, 2016 at 10:00:26 PM UTC+1, Ho Yeung Lee wrote: > i saved a list of matrix of algebra into .m file in maple > How to read and import into Python for sympy to use? I'd start here http://www.maplesoft.com/support/help/Maple/view.aspx?path=UserManual/Chapter11 which was the first hit I got on google for "read .m maple file with python". Kindest regards. Mark Lawrence. From rosuav at gmail.com Sun Oct 9 17:55:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 08:55:50 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: Message-ID: On Mon, Oct 10, 2016 at 4:57 AM, BartC wrote: > On 09/10/2016 18:33, Jussi Piitulainen wrote: >> >> Chris Angelico writes: >> >>> On Mon, Oct 10, 2016 at 12:26 AM, Dennis Lee Bieber wrote: >>>> >>>> {This response is delayed as I'm waiting for the program to complete so >>>> I >>>> can get the run time} >>>> {Well... it's been near 24 hours and still merrily scrolling sums on my >>>> console -- so I'm going to kill the running program} >>> >>> >>> Eight BILLION of them. I wouldn't bother waiting for that. >> >> >> Eight billion is a big number. I know from experience that computers can >> compute a big number of computations in a short time. Therefore, I would >> expect the computer to compute eight billion computations in a short >> time. Maybe. > > > Compute, maybe. But the OP is printing something out at each step. Printing > to a console is very, very slow, compared with even with running CPython. > > On my machine, CPython might run through an empty loop at 10 to 20M > iterations per second, but is some 4000 times slower if it has to print one > "A" per line at each step. Yeah. Quickest way to slow down a calculation is to naively add a progress indicator to it. (Actually, that's not quite fair. The quickest way is to add a time-of-day call every iteration, in a misguided attempt to reduce console output to "one every 0.1 seconds". That's crazy slow.) > The OP might try redirecting the output to a file (assuming he doesn't want > to read every single line of output as it's produced). That will be much > faster, although it depends on the workload in the run() function being > executed. > > Or only print every 1000th line or something if progress has to be > monitored. Yeah, if it's just for progress status. Of course, that does assume that the run() function doesn't have anything particularly costly in it. If it does, well, dis gonna take a while.... ChrisA From marko at pacujo.net Sun Oct 9 18:40:45 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 10 Oct 2016 01:40:45 +0300 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? References: Message-ID: <87eg3pp2mq.fsf@elektro.pacujo.net> Chris Angelico : > Yeah, if it's just for progress status. Of course, that does assume > that the run() function doesn't have anything particularly costly in > it. If it does, well, dis gonna take a while.... Dealing with multigigabyte data streams is not over the top nowadays. Marko From rosuav at gmail.com Sun Oct 9 18:48:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 09:48:57 +1100 Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: <87eg3pp2mq.fsf@elektro.pacujo.net> References: <87eg3pp2mq.fsf@elektro.pacujo.net> Message-ID: On Mon, Oct 10, 2016 at 9:40 AM, Marko Rauhamaa wrote: > Chris Angelico : >> Yeah, if it's just for progress status. Of course, that does assume >> that the run() function doesn't have anything particularly costly in >> it. If it does, well, dis gonna take a while.... > > Dealing with multigigabyte data streams is not over the top nowadays. This is true, but ultimately, it all comes down to the run() function. With no further posts since the drive-by at the beginning, we have nothing solid to discuss this on. ChrisA From drsalists at gmail.com Sun Oct 9 19:52:05 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 9 Oct 2016 16:52:05 -0700 Subject: Signals and Threads in Python 3.5 or so Message-ID: I have a program http://stromberg.dnsalias.org/~dstromberg/looper/ that I use and maintain. It's like GNU parallel or similar - yet another "run n processes, m at a time" implementation. Interestingly, I've only used/tested it on Linux, but it's under a Microsoft copyright because Microsoft acquired a Linux company I was working for. Anyway, it seems to work well, except one annoying bug. That bug is: if you control-C the top-level process, all the subprocesses are left running. I've been thinking about making it catch SIGINT, SIGTERM and SIGHUP, and having it SIGKILL its active subprocesses upon receiving one of these signals. However, it's multithreaded, and I've heard that in CPython, threads and signals don't mix well. Is this still an issue in CPython 3.5? If yes, how can I work around it? Thanks! From ian.g.kelly at gmail.com Sun Oct 9 19:56:20 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 9 Oct 2016 18:56:20 -0500 Subject: Function to take the minimum of 3 numbers In-Reply-To: References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On Oct 9, 2016 2:57 PM, wrote: On Sunday, October 9, 2016 at 2:41:41 PM UTC+1, BartC wrote: > def min3(a,b,c): > if a<=b and a<=c: > return a > elif b<=a and b<=c: > return b > else: > return c The Pythonic way if b >= a <= c: ... Better: if a <= b <= c: ... Using consistent operators is not required but is easier to read and less confusing. From ben+python at benfinney.id.au Sun Oct 9 20:11:38 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 10 Oct 2016 11:11:38 +1100 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: <858ttxnjut.fsf@benfinney.id.au> Gregory Ewing writes: > Here's the first part of the essay I said I'd write about > monads: Thank you for tackling this. > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html You appreciate an important feature of this: Often it is presented in a very abstract and mathematical way, which tends to have an eye-glazing effect on people who aren't hard-core mathematicians. This is an attempt to convey the basic ideas behind monads to someone who is a programmer rather than a mathematician. One important lack that I often see in essays written by mathematicians about programming, is the tendency to use opaque single-character symbols., such as might be easy to write in chalk on a blackboard. In a programming language, though, we do not have the constraint of writing them with chalk on a blackboard. Symbols can be given *descriptive names*, that communicate the semantic meaning of the symbol. Can you re-write the code examples, for example:: def add(x, f, s): s.add(x) return f(s) so that they are PEP 8 conformant? That means (in addition to 4-column indentation at each level) that you should *choose descriptive names* and use them consistently, to help the reader understand the code. -- \ ?Telling pious lies to trusting children is a form of abuse, | `\ plain and simple.? ?Daniel Dennett, 2010-01-12 | _o__) | Ben Finney From funkyhat at gmail.com Sun Oct 9 20:38:58 2016 From: funkyhat at gmail.com (Matt Wheeler) Date: Mon, 10 Oct 2016 00:38:58 +0000 Subject: Function to take the minimum of 3 numbers In-Reply-To: References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On Mon, 10 Oct 2016, 00:56 Ian Kelly, wrote: > On Oct 9, 2016 2:57 PM, wrote: > The Pythonic way > > if b >= a <= c: > ... > > > Better: > > if a <= b <= c: > ... > That's not equivalent. Consider `a, b, c = 1, 7, 4` > Using consistent operators is not required but is easier to read and less > confusing. > Unfortunately in this case it's also less correct > -- -- Matt Wheeler http://funkyh.at From rosuav at gmail.com Sun Oct 9 20:45:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 11:45:23 +1100 Subject: Signals and Threads in Python 3.5 or so In-Reply-To: References: Message-ID: On Mon, Oct 10, 2016 at 10:52 AM, Dan Stromberg wrote: > That bug is: if you control-C the top-level process, all the > subprocesses are left running. > > I've been thinking about making it catch SIGINT, SIGTERM and SIGHUP, > and having it SIGKILL its active subprocesses upon receiving one of > these signals. > > However, it's multithreaded, and I've heard that in CPython, threads > and signals don't mix well. Generally, expect SIGINT to be handled by the main thread, and code accordingly. But I've never used the low-level thread and _thread modules, so I'd be inclined to follow through with the TODO and make it use threading instead. This simple example appears to work fine on my system (Debian Stretch, Linux 4.6, amd64), but I can't make any hard-and-fast guarantees. import threading import signal import random import time halt = False def thread(): global halt try: print(threading.current_thread(), "-- start") for i in range(random.randrange(10, 20)): time.sleep(1) if halt: break print(threading.current_thread(), "--", i) except KeyboardInterrupt: print(threading.current_thread(), "-- SIGINT") halt = True print(threading.current_thread(), "-- end") for i in range(5): threading.Thread(target=thread).start() thread() Hit Ctrl-C and all threads will quickly terminate. ChrisA From no.email at nospam.invalid Sun Oct 9 20:56:55 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 09 Oct 2016 17:56:55 -0700 Subject: Signals and Threads in Python 3.5 or so References: Message-ID: <87oa2tt414.fsf@nightsong.com> Dan Stromberg writes: > That bug is: if you control-C the top-level process, all the > subprocesses are left running. Are you setting the daemon flag? From no.email at nospam.invalid Sun Oct 9 21:00:07 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sun, 09 Oct 2016 18:00:07 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> Message-ID: <87k2dht3vs.fsf@nightsong.com> > Well, at least I refrained from saying that monads are like burritos! But they are! See: http://blog.plover.com/prog/burritos.html From rosuav at gmail.com Sun Oct 9 21:36:16 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 10 Oct 2016 12:36:16 +1100 Subject: Function to take the minimum of 3 numbers In-Reply-To: References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: On Mon, Oct 10, 2016 at 11:38 AM, Matt Wheeler wrote: > On Mon, 10 Oct 2016, 00:56 Ian Kelly, wrote: > >> On Oct 9, 2016 2:57 PM, wrote: >> The Pythonic way >> >> if b >= a <= c: >> ... >> >> >> Better: >> >> if a <= b <= c: >> ... >> > > That's not equivalent. Consider `a, b, c = 1, 7, 4` > > >> Using consistent operators is not required but is easier to read and less >> confusing. >> > > Unfortunately in this case it's also less correct Proof that it's confusing: Someone tried to tidy up the code, and unwittingly changed its semantics. ChrisA From mahanmarwat at gmail.com Mon Oct 10 00:53:19 2016 From: mahanmarwat at gmail.com (Mahan Marwat) Date: Sun, 9 Oct 2016 21:53:19 -0700 (PDT) Subject: Solid Approach For Creating A Desktop Application Message-ID: <5764ec44-556d-4228-8e9e-934f4b66dd95@googlegroups.com> I want to know what will be your approach creating a solid/reliable application in Python? i.e 1. Which GUI framework will you use i.e PyQT or what? (will you make it work removing the window default border) 2. What will you use for it's style/appearance i.e CSS or what? 3. Which tool will you use to make an executable of your application i.e Py2exe or what? 4. Which tool will you use to make an installable of your application i.e Nsis or what? My approach: Python --> PyGTK --> Cx_Freeze --> Nsis And last and important question, does your application will work smoothly and will not crash? If I am missing something like testing your application etc... Please mention those things too. I want to know a full method of creating a solid desktop application. From jobmattcon at gmail.com Mon Oct 10 01:01:23 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Sun, 9 Oct 2016 22:01:23 -0700 (PDT) Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: <87eg3pp2mq.fsf@elektro.pacujo.net> Message-ID: https://social.msdn.microsoft.com/Forums/vstudio/en-US/5f0a9a51-a256-4671-a5fc-e213949e7204/how-to-refactor-3-nested-for-loop-into-smaller-for-loop-assume-each-of-them-independent?forum=csharpgeneral since when i dsolve a differential ideal used near 5GB memory for one ideal , i feel that i need to use amazon instance, i do not know whether sympy use same amount of memory i prepared to split a for loop which for combination if use i < j and j < k so 84 Billion /2 = 42 Billions combinations each script for doing 100 combinations range, it generate around 4XX files if read the final file it generated, is for ii from 1925 to 2025 do for jj from 2040 to 2040 do for kk from 2040 to 2040 do it seems missed 2026 to 2040 this range in the final file i feel that my crafted algorithm is not full combinations I wrote a algorithm to split for loop to generate maplesoft code for limited memory Assume I convert this algorithm into Python script and use sympy to do dsolve How to distribute to hundreds of Amazon instance to run each smaller for loop i am not familiar with using Amazon, such as how to create hundreds of EC2 instances i guess use python script to ssh all remote instance run a python script with bash command which in python script On Monday, October 10, 2016 at 6:49:12 AM UTC+8, Chris Angelico wrote: > On Mon, Oct 10, 2016 at 9:40 AM, Marko Rauhamaa wrote: > > Chris Angelico : > >> Yeah, if it's just for progress status. Of course, that does assume > >> that the run() function doesn't have anything particularly costly in > >> it. If it does, well, dis gonna take a while.... > > > > Dealing with multigigabyte data streams is not over the top nowadays. > > This is true, but ultimately, it all comes down to the run() function. > With no further posts since the drive-by at the beginning, we have > nothing solid to discuss this on. > > ChrisA From rustompmody at gmail.com Mon Oct 10 01:19:17 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 9 Oct 2016 22:19:17 -0700 (PDT) Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <57F9F63D.8080300@canterbury.ac.nz> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: On Sunday, October 9, 2016 at 1:18:32 PM UTC+5:30, Gregory Ewing wrote: > Here's the first part of the essay I said I'd write about > monads: > > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html > > Hope it's useful, > Greg Thanks Greg... looks v useful... the continuation-oriented approach. Will say more after a more careful study From davidbenny2000 at gmail.com Mon Oct 10 02:13:53 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Sun, 9 Oct 2016 23:13:53 -0700 (PDT) Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: References: <87eg3pp2mq.fsf@elektro.pacujo.net> Message-ID: <564b5349-d2f8-4d03-9da0-72bacd854e39@googlegroups.com> Sorry my calculation is wrong, it should have around 14 billions of combinations after using program to count. From marko at pacujo.net Mon Oct 10 02:27:36 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Mon, 10 Oct 2016 09:27:36 +0300 Subject: Signals and Threads in Python 3.5 or so References: Message-ID: <87zimcoh0n.fsf@elektro.pacujo.net> Dan Stromberg : > That bug is: if you control-C the top-level process, all the > subprocesses are left running. Sorry, don't have a solution for your particular Python situation. > I've been thinking about making it catch SIGINT, SIGTERM and SIGHUP, > and having it SIGKILL its active subprocesses upon receiving one of > these signals. SIGKILL is probably a bad idea. It doesn't give the subprocess any opportunity for a graceful exit. For example, Python's try/finally blocks will fail to work as advertised. Better use a catchable signal. > However, it's multithreaded, and I've heard that in CPython, threads > and signals don't mix well. Python does confuse matters, but both threads and signals are problematic entities under Linux. You need to be very well versed in the semantics of both operating system concepts (man 7 pthreads, man 7 signal). Marko From greg.ewing at canterbury.ac.nz Mon Oct 10 02:50:40 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 10 Oct 2016 19:50:40 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <87k2dht3vs.fsf@nightsong.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> <87k2dht3vs.fsf@nightsong.com> Message-ID: Paul Rubin wrote: >>Well, at least I refrained from saying that monads are like burritos! > > But they are! See: http://blog.plover.com/prog/burritos.html Oh, undoubtedly. I just don't think it helps understand how burritos are used in prog... er, that is, how monads... well, you know what I mean. -- Greg From dieter at handshake.de Mon Oct 10 03:05:19 2016 From: dieter at handshake.de (dieter) Date: Mon, 10 Oct 2016 09:05:19 +0200 Subject: Signals and Threads in Python 3.5 or so References: Message-ID: <87h98kvg40.fsf@handshake.de> Dan Stromberg writes: > I have a program http://stromberg.dnsalias.org/~dstromberg/looper/ > that I use and maintain. > > It's like GNU parallel or similar - yet another "run n processes, m at > a time" implementation. Interestingly, I've only used/tested it on > Linux, but it's under a Microsoft copyright because Microsoft acquired > a Linux company I was working for. > > Anyway, it seems to work well, except one annoying bug. > > That bug is: if you control-C the top-level process, all the > subprocesses are left running. > > I've been thinking about making it catch SIGINT, SIGTERM and SIGHUP, > and having it SIGKILL its active subprocesses upon receiving one of > these signals. > > However, it's multithreaded, and I've heard that in CPython, threads > and signals don't mix well. I would not state it this way. Signals are delivered to processes (not process threads). That leads to the question which thread (in a multi thread application) will handle a signal when it arrives. In Python, the design decision has been not to use the currently running thread (which may not be a Python thread at all or may have executed Python code but at the moment runs C code outside of Python) but let the signal be handled deterministically by the main thread (once it starts again to execute Python code). This looks like a sane decision to me. Your problem description indicates that the problem is not with signal handling by threads but with signal delivery by the operating system. In Linux, SIGINT signals in response to a "CTRL-C" are delivered to the process group of the "control terminal". Thus, with appropriate Linux calls, you can control whether or not such a SIGINT is delivered to a subprocess. Other operating systems may not provide such a control or it may need different (OS dependent) calls. From list at qtrac.plus.com Mon Oct 10 03:08:25 2016 From: list at qtrac.plus.com (Mark Summerfield) Date: Mon, 10 Oct 2016 00:08:25 -0700 (PDT) Subject: Solid Approach For Creating A Desktop Application In-Reply-To: <5764ec44-556d-4228-8e9e-934f4b66dd95@googlegroups.com> References: <5764ec44-556d-4228-8e9e-934f4b66dd95@googlegroups.com> Message-ID: On Monday, October 10, 2016 at 5:53:37 AM UTC+1, Mahan Marwat wrote: > I want to know what will be your approach creating a solid/reliable application in Python? > i.e > 1. Which GUI framework will you use i.e PyQT or what? (will you make it work removing the window default border) > 2. What will you use for it's style/appearance i.e CSS or what? > 3. Which tool will you use to make an executable of your application i.e Py2exe or what? > 4. Which tool will you use to make an installable of your application i.e Nsis or what? > My approach: Python --> PyGTK --> Cx_Freeze --> Nsis > And last and important question, does your application will work smoothly and will not crash? > If I am missing something like testing your application etc... Please mention those things too. > I want to know a full method of creating a solid desktop application. 1. The choice of framework will depend on the target platform(s) and your personal preference. In my case I use Python 3.4 + PySide 1.2 (which uses Qt 4.8). 2. For desktop applications it is better not to do any styling yourself but to respect the user's preferences. Qt will automatically do this and I guess other toolkits might do this too. 3./4. I use cx_Freeze to produce a .msi file for Windows but which tool you use will depend on your target platform(s). cx_Freeze puts a .exe inside the .msi, so it does everything needed. For testing I use Python's own unittest framework. I have found this toolchain to be very reliable. From no.email at nospam.invalid Mon Oct 10 03:22:27 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 10 Oct 2016 00:22:27 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> <87k2dht3vs.fsf@nightsong.com> Message-ID: <87fuo4u0r0.fsf@nightsong.com> Gregory Ewing writes: > Oh, undoubtedly. I just don't think it helps understand how burritos > are used in prog... er, that is, how monads... well, you know what I > mean. How about in math? https://www.cs.cmu.edu/~edmo/silliness/burrito_monads.pdf ;-) From no.email at nospam.invalid Mon Oct 10 03:24:01 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 10 Oct 2016 00:24:01 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <87bmyuufdt.fsf@nightsong.com> <87k2dht3vs.fsf@nightsong.com> <87fuo4u0r0.fsf@nightsong.com> Message-ID: <87a8ecu0oe.fsf@nightsong.com> Paul Rubin writes: > https://www.cs.cmu.edu/~edmo/silliness/burrito_monads.pdf Whoops, url changed: http://emorehouse.web.wesleyan.edu/silliness/burrito_monads.pdf From davidbenny2000 at gmail.com Mon Oct 10 06:43:35 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Mon, 10 Oct 2016 03:43:35 -0700 (PDT) Subject: how to refactor nested for loop into smaller for loop assume each of them independent? In-Reply-To: <564b5349-d2f8-4d03-9da0-72bacd854e39@googlegroups.com> References: <87eg3pp2mq.fsf@elektro.pacujo.net> <564b5349-d2f8-4d03-9da0-72bacd854e39@googlegroups.com> Message-ID: I updated the code in msdn forum, I calculated from the end of file Discover file 4550 takes a long time to run, Assume it runs a whole day a file, 4550 days I guess need 12 years to finish full combination if only run at home. Hope Python sympy can be faster than cmaple in Amazon instance From davidbenny2000 at gmail.com Mon Oct 10 07:01:49 2016 From: davidbenny2000 at gmail.com (Ho Yeung Lee) Date: Mon, 10 Oct 2016 04:01:49 -0700 (PDT) Subject: Is there a free graph library to show program flow by tranverse AST tree Message-ID: <83c74e19-8faa-41ac-ab46-d2536725645f@googlegroups.com> can this graph library handle recursive function call this symbol If it recursive call a function 3 times then in the inner loop call another function , can this graph library Show this relationship I find video that viv can show this program flow. From jtim.arnold at gmail.com Mon Oct 10 09:01:17 2016 From: jtim.arnold at gmail.com (Tim) Date: Mon, 10 Oct 2016 06:01:17 -0700 (PDT) Subject: segfault using shutil.make_archive In-Reply-To: References: <6efda754-3f3d-ec64-9e00-46671c33a9c1@gmail.com> Message-ID: On Friday, October 7, 2016 at 1:05:43 PM UTC-4, Michael Torrie wrote: > On 10/06/2016 10:46 AM, Tim wrote: > > I need to zip up a directory that's about 400mb. > > I'm using shutil.make_archive and I'm getting this response: > > > > Segmentation fault: 11 (core dumped) > > > > The code is straightforward (and works on other, smaller dirs): > > > > shutil.make_archive(os.path.join(zip_dir, zname), 'zip', tgt_dir) > > > > I guess I could drop the convenience of make_archive and use zipfile but that seems to be exactly what make_archive does. > > > > I'm on FreeBSD, python2.7. > > > > Anyone seen this before? > > Does normal the normal zip utility crash also when trying to zip up this > large directory? I'm not familiar with how shutils.make_archive works, > but since it's part of shutils, I suspect it's calling the zip command > as a subprocess, rather than using the Python zip module. Given the > size of your directory I doubt the zip module would work anyway. > > If the zip command works, you could just execute it using popen from > within Python. I'm now using subprocess and the zip command from FreeBSD, and it's working. So far I can't reproduce a small example of the problem with make_archive. --Tim From amornsak.nak at gmail.com Mon Oct 10 09:22:52 2016 From: amornsak.nak at gmail.com (Nune9) Date: Mon, 10 Oct 2016 06:22:52 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest possible. Message-ID: <9c051de8-5a7e-4554-9833-99f353c851cb@googlegroups.com> Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest possible. The first I'm sorry for my English language. I have a list is land = [10,20,30,40,110,50,18,32,5] and I want to find each values of summation (don't sorted values in list) as It has highest possible example. I want to dividing N=3 part from list as above and divieded each part has highest values that as possible. *** (don't sorted values in list) first part has values is = 10, 20, 30, 40 summation = 10+20+30+40 = 100 second part has values is = 110 summation = 110 and third part has values is = 50, 18, 32, 5 summation = 105 that 100, 110 and 105 is highest values as possible in list by dividing N=3 part. and then If I want to divide N(1,2,3,...,N) part from any numeric list??? Thank you for python code (: From vek.m1234 at gmail.com Mon Oct 10 09:32:58 2016 From: vek.m1234 at gmail.com (Veek M) Date: Mon, 10 Oct 2016 19:02:58 +0530 Subject: PyQT - Signals and Slots? Message-ID: I'm reading Rapid GUI Programming - Mark Summerfield with Python and QT pg 131. Basically the mechanism is an event table which maps a 'signal' to a 'function/slot' -correct? self.connect(dial, SIGNAL("valueChanged(int)"), spinbox.setValue) Here, dial.valueChanged -> spinbox.setValue s.connect(w, SIGNAL("signalSignature"), functionName) s.connect(w, SIGNAL("signalSignature"), instance.methodName) s.connect(w, SIGNAL("signalSignature"), instance, SLOT("slotSignature")) Here, w.signalSignature -> functionName -> instance.methodName -> instance.slotSignature If signalSignature is a C++ implemented thingy then we got to pass type info as part of the mapping so "signalSignature(int, float, const char *). PyQT signals are any type and any number of args.. If the Slot-function is implemented in C++ it's better to use the SLOT() mechanism: self.connect(dial, SIGNAL("valueChanged(int)"), spinbox, SLOT("setValue(int)")) Here, we are mapping dial.valueChanged(int) --> spinbox.setValue(int) ---- The part i found tricky was this: class ZeroSpinBox(QSpinBox): zeros = 0 def __init__(self, parent=None): super(ZeroSpinBox, self).__init__(parent) self.connect(self, SIGNAL("valueChanged(int)"), self.checkzero) def checkzero(self): if self.value() == 0: self.zeros += 1 self.emit(SIGNAL("atzero"), self.zeros) ZeroSpinBox.valueChanged -> ZeroSpinBox.checkzero? Why is he mapping back to himself? Shouldn't it be widget-to-widget? Then he raises a signal 'atzero' with one arg - the lack of a atzero() implies it's a 'short-circuit' signal so self.zeroes is a python data type. And in the book he says: ####################### Here is how we connect to the signal in the form?s __init__() method: zerospinbox = ZeroSpinBox() ... self.connect(zerospinbox, SIGNAL("atzero"), self.announce) Again, we must not use parentheses because it is a short-circuit signal. And for completeness, here is the slot it connects to in the form: def announce(self, zeros): print "ZeroSpinBox has been at zero %d times" % zeros ########################### Whaaa...t?? Could someone explain what exactly is his grand design besides being awfully circuitous? So he has some other Form thingy.. and in that he sets up another mapping from ZeroSpinBox.atzero --> ZeroSpinBox.announce ???? where's announce and atzero defined? From amornsak.nak at gmail.com Mon Oct 10 09:38:10 2016 From: amornsak.nak at gmail.com (amornsak.nak at gmail.com) Date: Mon, 10 Oct 2016 06:38:10 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. Message-ID: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. The first I'm sorry for my English language. I have a list is land = [10,20,30,40,110,50,18,32,5] and I want to find each values of summation (don't sorted values in list) as It has highest as possible example. I want to dividing N=3 part from list as above and divieded each part has highest values that as possible. *** (don't sorted values in list) first part has values is = 10, 20, 30, 40 summation = 10+20+30+40 = 100 second part has values is = 110 summation = 110 and third part has values is = 50, 18, 32, 5 summation = 105 that 100, 110 and 105 is highest values as possible in list by dividing N=3 part. and then If I want to divide N(1,2,3,...,N) part from any more numeric list??? Thank you for python code (: From amornsak.nak at gmail.com Mon Oct 10 09:46:11 2016 From: amornsak.nak at gmail.com (Nuen9) Date: Mon, 10 Oct 2016 06:46:11 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. Message-ID: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. The first I'm sorry for my English language. I have a list is land = [10,20,30,40,110,50,18,32,5] and I want to find each values of summation (don't sorted values in list) as It has highest as possible example. I want to dividing N=3 part from list as above and divieded each part has highest values that as possible. *** (don't sorted values in list) first part has values is = 10, 20, 30, 40 summation = 10+20+30+40 = 100 second part has values is = 110 summation = 110 and third part has values is = 50, 18, 32, 5 summation = 105 that 100, 110 and 105 is highest values as possible in list by dividing N=3 part. and then If I want to divide N(1,2,3,...,N) part from any more numeric list??? Thank you for python code (: From ben.usenet at bsb.me.uk Mon Oct 10 09:52:22 2016 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Mon, 10 Oct 2016 14:52:22 +0100 Subject: Deviding N(1, 2, 3, .., N) part from numeric list as summation of each values(don't sorted) has highest possible. References: <9c051de8-5a7e-4554-9833-99f353c851cb@googlegroups.com> Message-ID: <87eg3ocnvt.fsf@bsb.me.uk> Nune9 writes: > I have a list is > > land = [10,20,30,40,110,50,18,32,5] > > and I want to find each values of summation (don't sorted values in > list) as It has highest possible > > example. > > I want to dividing N=3 part from list as above and divieded each part > has highest values that as possible. > *** (don't sorted values in list) > > first part has values is = 10, 20, 30, 40 > summation = 10+20+30+40 = 100 > > second part has values is = 110 > summation = 110 > > and third part has values is = 50, 18, 32, 5 > summation = 105 > > that 100, 110 and 105 is highest values as possible in list by > dividing N=3 part. So, given a sequence of numbers, you want to partition it into N consecutive sub-sequences such that the sums of these sub sequences is, somehow, maximal. I say "somehow" because there are various possible meanings for "highest" when applies to set of numbers. Have I got that part right? It would help if you clarified a couple of things. Are all numbers in the sequence always positive, and what makes one set of numbers "higher" than any other? I imagine you want the most equal partition -- the one that minimises the variance in the set of sums. > and then If I want to divide N(1,2,3,...,N) part from any numeric > list??? I don't understand this last bit at all. The repeated use of N is not helping me. -- Ben. From steve+python at pearwood.info Mon Oct 10 10:11:42 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 11 Oct 2016 01:11:42 +1100 Subject: Function to take the minimum of 3 numbers References: <545212c5-1c67-450e-b9a1-b0e68a4963cf@googlegroups.com> Message-ID: <57fba19f$0$1597$c3e8da3$5496439d@news.astraweb.com> On Mon, 10 Oct 2016 10:56 am, Ian Kelly wrote: > On Oct 9, 2016 2:57 PM, alleged: > The Pythonic way > > if b >= a <= c: > ... I doubt that would be considered Pythonic by many people. Chained comparisons are Pythonic, but not legal but weird combinations like `a == b < c is not d >= e <= f`. > Better: > > if a <= b <= c: > ... That's more like it. Unfortunately it doesn't mean the same as Mark's version: b >= a <= c means b >= a and a <= c which is True for a = 1, b = 3 and c = 2; a <= b <= c means a <= b and b <= c which is False for a = 1, b = 3 and c = 2. > Using consistent operators is not required but is easier to read and less > confusing. Indeed. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From mr.puneet.goyal at gmail.com Mon Oct 10 10:13:06 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Mon, 10 Oct 2016 07:13:06 -0700 (PDT) Subject: How to process syntax errors Message-ID: Hi Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. For example, inside example.py I just mention below line Obj = myClass() Obj xyz Obj is instance of a class. But there is syntax error on xyz. So I want to grab that error and process. Regards, Puneet From steve+python at pearwood.info Mon Oct 10 10:31:07 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 11 Oct 2016 01:31:07 +1100 Subject: Deviding N(1, 2, 3, .., N) part from numeric list as summation of each values(don't sorted) has highest as possible. References: Message-ID: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> On Tue, 11 Oct 2016 12:38 am, amornsak.nak at gmail.com wrote: > I have a list is > > land = [10,20,30,40,110,50,18,32,5] > > and I want to find each values of summation (don't sorted values in list) > as It has highest as possible > > example. > > I want to dividing N=3 part from list as above and divieded each part has > highest values that as possible. *** (don't sorted values in list) > > first part has values is = 10, 20, 30, 40 > summation = 10+20+30+40 = 100 Why do you add four numbers? Why not add all the numbers? 10+20+30+40+110+50+18+32+5 = 315 > second part has values is = 110 > summation = 110 Why do you only take one number? > and third part has values is = 50, 18, 32, 5 > summation = 105 Why do you take four numbers? > that 100, 110 and 105 is highest values as possible in list by dividing > N=3 part. I do not understand, what do you mean "dividing N=3 part"? Could you do this? [10,20,30,40,110,50,18,32,5] split into three lists: [10,20,30,40,110,50,18], [32], [5] Or these three lists: [10], [20,30,40,110,50,18], [32,5] Or these three lists: [10,20,30], [40,110,50], [18,32,5] How do you split the list into three? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From list at qtrac.plus.com Mon Oct 10 10:49:35 2016 From: list at qtrac.plus.com (Mark Summerfield) Date: Mon, 10 Oct 2016 07:49:35 -0700 (PDT) Subject: PyQT - Signals and Slots? In-Reply-To: References: Message-ID: The ZeroSpinBox is a tiny example designed to show how the signal/slot mechanism works. It is just a QSpinBox with the addition of remembering how many times (all the) ZeroSpinBox(es) have had a 0 value. Nowadays the connections would be made with a new improved syntax: self.connect(self, SIGNAL("valueChanged(int)"), self.checkzero) # OLD self.valueChanged.connect(self.checkzero) # NEW # or self.valueChanged[int].connect(self.checkzero) # NEW Similarly the emit: self.emit(SIGNAL("atzero"), self.zeros) # OLD self.atzero.emit(self.zeros) # NEW What's happening inside ZeroSpinBox? Whenever its value is set to 0 it calls its own checkzero() method, and this in turn emits an atzero signal with the number of zeros so far. Why does it do this? Just to show the mechanism. It doesn't matter whether you connect a widget to itself (unusual but done here), or to another widget (the norm). See the book's website https://www.qtrac.eu/pyqtbook.html for all the source code including some updated with the new syntax. (But the book is old, so even the Python 3.1 examples aren't as Pythonic as they would be written in Python 3.4+.) From infosecflag at gmail.com Mon Oct 10 10:58:44 2016 From: infosecflag at gmail.com (infosecflag at gmail.com) Date: Mon, 10 Oct 2016 07:58:44 -0700 (PDT) Subject: Newbie Need Help On Regex! Message-ID: <9c05d361-6a24-49d9-8d37-d647fbe13679@googlegroups.com> Hey guys! I am new to learning regex in python and I'm wondering how do I use regex in python to store the integers(positive and negative) i want into a list! For e.g. This is the data in a list. [u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=-5,B=5)', u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=5,Y=5)', u'\x1b[0m[\x1b[1m\x1b[10m\xbb\x1b[0m\x1b[36m]\x1b[0m : '] How do I extract the values of A and B and store them in a variable I want using regex? Thank you and appreciate it :) From infosecflag at gmail.com Mon Oct 10 10:59:59 2016 From: infosecflag at gmail.com (infosecflag at gmail.com) Date: Mon, 10 Oct 2016 07:59:59 -0700 (PDT) Subject: Newbie Need Help On Regex! Message-ID: Hey guys! I am new to learning regex in python and I'm wondering how do I use regex in python to store the integers(positive and negative) i want into a list! For e.g. This is the data in a list. [u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=-5,B=5)', u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=5,Y=5)', u'\x1b[0m[\x1b[1m\x1b[10m\xbb\x1b[0m\x1b[36m]\x1b[0m : '] How do I extract the values of A and B and store them in a variable I want using regex? Thank you and appreciate it :) From amornsak.nak at gmail.com Mon Oct 10 11:20:39 2016 From: amornsak.nak at gmail.com (Nuen9) Date: Mon, 10 Oct 2016 08:20:39 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> Message-ID: <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> ????? ???????????? 10 ?????? ?.?. 2016 21 ?????? 31 ???? 25 ?????? UTC+7, Steve D'Aprano ????????: > On Tue, 11 Oct 2016 12:38 am, amornsak.nak at gmail.com wrote: > > > I have a list is > > > > land = [10,20,30,40,110,50,18,32,5] > > > > and I want to find each values of summation (don't sorted values in list) > > as It has highest as possible > > > > example. > > > > I want to dividing N=3 part from list as above and divieded each part has > > highest values that as possible. *** (don't sorted values in list) > > > > first part has values is = 10, 20, 30, 40 > > summation = 10+20+30+40 = 100 > > Why do you add four numbers? Why not add all the numbers? > > 10+20+30+40+110+50+18+32+5 = 315 > > > second part has values is = 110 > > summation = 110 > > Why do you only take one number? > > > > and third part has values is = 50, 18, 32, 5 > > summation = 105 > > Why do you take four numbers? > > > > that 100, 110 and 105 is highest values as possible in list by dividing > > N=3 part. > > I do not understand, what do you mean "dividing N=3 part"? > > Could you do this? > > [10,20,30,40,110,50,18,32,5] > > split into three lists: > > [10,20,30,40,110,50,18], [32], [5] > > Or these three lists: > > [10], [20,30,40,110,50,18], [32,5] > > Or these three lists: > > [10,20,30], [40,110,50], [18,32,5] > > How do you split the list into three? > > > > > > > -- > Steve > ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure > enough, things got worse. I'm sorry for my English language. OK, I have list = [10,20,30,40,110,50,18,32,5] and want to find three values of summation from my list by split any way list but three values of summation has closely example. [10,20,30], [40,110,50], [18,32,5] = 60, 200, 55 -> it is not closely *** 60 = 10+20+30, 200 = 40+110+50, 18+32+5 = 55 and [10], [20,30,40,110,50,18], [32,5] = 10, 268, 37 -> and it is not closely and [10,20,30,40],[110],[50,18,32,5] = 100,110,105 -> it's closely The Really answers that I want is [10,20,30,40],[110],[50,18,32,5] = 100,110,105 from list[10,20,30,40,110,50,18,32,5] And I don't know what I have to explain for my questions in English because My English so bad T_T Thank you for reading my text (; From rosuav at gmail.com Mon Oct 10 11:24:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 11 Oct 2016 02:24:37 +1100 Subject: How to process syntax errors In-Reply-To: References: Message-ID: On Tue, Oct 11, 2016 at 1:13 AM, wrote: > Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. > > For example, > > inside example.py > > I just mention below line > > > Obj = myClass() > Obj xyz > > Obj is instance of a class. But there is syntax error on xyz. So I want to grab that error and process. Yes and no. Syntax errors are detected when the script is compiled, so you can't do something like this: try: Obj xyz except SyntaxError: print("Try Obj.xyz instead!") However, you can catch this at some form of outer level. If you're using exec/eval to run the code, you can guard that: code = """ obj = MyClass() # using PEP 8 names obj xyz """ try: exec(code) except SyntaxError: print("Blah blah blah") Alternatively, you can put the code into a file and import it: # otherfile.py obj = MyClass() obj xyz # mainfile.py try: import otherfile except SyntaxError: do_stuff() This is probably the easiest solution, if your code's already in a separate file. ChrisA From maillists at pp.inet.fi Mon Oct 10 11:42:46 2016 From: maillists at pp.inet.fi (K. Elo) Date: Mon, 10 Oct 2016 18:42:46 +0300 Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> Message-ID: <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Hi! Could it be, "Nuen9", that you would like to find a split where the split sums are close to each other? In other words, you define the number of splits (in your example: 3) and the algortihm should test all possible combinations and select the split where the sum differences are smallest. Best, Kimmo 10.10.2016, 18:20, Nuen9 wrote: > ????? ???????????? 10 ?????? ?.?. 2016 21 ?????? 31 ???? 25 ?????? UTC+7, Steve D'Aprano ????????: >> On Tue, 11 Oct 2016 12:38 am, amornsak.nak at gmail.com wrote: > I'm sorry for my English language. > > OK, I have list = [10,20,30,40,110,50,18,32,5] and want to find three values of summation from my list by split any way list but three values of summation has closely > example. > [10,20,30], [40,110,50], [18,32,5] = 60, 200, 55 -> it is not closely > *** 60 = 10+20+30, 200 = 40+110+50, 18+32+5 = 55 > and [10], [20,30,40,110,50,18], [32,5] = 10, 268, 37 -> and it is not closely > and [10,20,30,40],[110],[50,18,32,5] = 100,110,105 -> it's closely > > The Really answers that I want is [10,20,30,40],[110],[50,18,32,5] = 100,110,105 from list[10,20,30,40,110,50,18,32,5] > > And I don't know what I have to explain for my questions in English because My English so bad T_T > > Thank you for reading my text > > (; > From pdorange at pas-de-pub-merci.mac.com Mon Oct 10 11:44:38 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Mon, 10 Oct 2016 17:44:38 +0200 Subject: How to process syntax errors References: Message-ID: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> Chris Angelico wrote: > Yes and no. Syntax errors are detected when the script is compiled, so > you can't do something like this: You're right, except that Python is never compiled, it was just checked for syntax error before interpreting code. > > However, you can catch this at some form of outer level. If you're > using exec/eval to run the code, you can guard that: Your solution are OK, but that's not very "pythonic". Python was not design to be used in such a way (intercepting syntax error). There was hack to made somthing like requested but that only a hack and should not be used in real world. -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From vek.m1234 at gmail.com Mon Oct 10 11:48:13 2016 From: vek.m1234 at gmail.com (Veek M) Date: Mon, 10 Oct 2016 21:18:13 +0530 Subject: PyQT - Signals and Slots? References: Message-ID: Mark Summerfield wrote: > > The ZeroSpinBox is a tiny example designed to show how the signal/slot > mechanism works. It is just a QSpinBox with the addition of > remembering how many times (all the) ZeroSpinBox(es) have had a 0 > value. > > Nowadays the connections would be made with a new improved syntax: > > self.connect(self, SIGNAL("valueChanged(int)"), self.checkzero) # OLD > self.valueChanged.connect(self.checkzero) # NEW > # or > self.valueChanged[int].connect(self.checkzero) # NEW > > Similarly the emit: > self.emit(SIGNAL("atzero"), self.zeros) # OLD > self.atzero.emit(self.zeros) # NEW > > What's happening inside ZeroSpinBox? Whenever its value is set to 0 it > calls its own checkzero() method, and this in turn emits an atzero > signal with the number of zeros so far. Why does it do this? Just to > show the mechanism. It doesn't matter whether you connect a widget to > itself (unusual but done here), or to another widget (the norm). > > See the book's website https://www.qtrac.eu/pyqtbook.html > for all the source code including some updated with the new syntax. > (But the book is old, so even the Python 3.1 examples aren't as > Pythonic as they would be written in Python 3.4+.) ah - okay, so i was on the right track - thanks :) cool that you hang out here :) From rosuav at gmail.com Mon Oct 10 12:13:39 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 11 Oct 2016 03:13:39 +1100 Subject: How to process syntax errors In-Reply-To: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> Message-ID: On Tue, Oct 11, 2016 at 2:44 AM, Pierre-Alain Dorange wrote: > Chris Angelico wrote: > >> Yes and no. Syntax errors are detected when the script is compiled, so >> you can't do something like this: > > You're right, except that Python is never compiled, it was just checked > for syntax error before interpreting code. https://docs.python.org/3/library/functions.html#compile It's compiled. >> However, you can catch this at some form of outer level. If you're >> using exec/eval to run the code, you can guard that: > > Your solution are OK, but that's not very "pythonic". > > Python was not design to be used in such a way (intercepting syntax > error). There was hack to made somthing like requested but that only a > hack and should not be used in real world. The error is raised as an exception, which means it is most definitely meant to be able to be caught. If it were utterly and completely fatal, it would simply be handled as "print message to standard error, terminate process" as some forms of fatal error are. (But even in some of those cases - SystemError and MemoryError - they can still be raised as exceptions.) There's nothing unpythonic about catching an exception if you know how to handle it. My examples did have somewhat unpythonic behaviour (catching an exception only to print a message and continue), but that's because they're placeholders. ChrisA From amornsak.nak at gmail.com Mon Oct 10 12:23:04 2016 From: amornsak.nak at gmail.com (Nuen9) Date: Mon, 10 Oct 2016 09:23:04 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Message-ID: <6ecd4d03-0337-4e73-8967-58696b3191be@googlegroups.com> ????? ???????????? 10 ?????? ?.?. 2016 22 ?????? 46 ???? 33 ?????? UTC+7, K. Elo ????????: > Hi! > > Could it be, "Nuen9", that you would like to find a split where the > split sums are close to each other? In other words, you define the > number of splits (in your example: 3) and the algortihm should test all > possible combinations and select the split where the sum differences are > smallest. > > Best, > Kimmo > > 10.10.2016, 18:20, Nuen9 wrote: > > ????? ???????????? 10 ?????? ?.?. 2016 21 ?????? 31 ???? 25 ?????? UTC+7, Steve D'Aprano ????????: > >> On Tue, 11 Oct 2016 12:38 am, amornsak.nak at gmail.com wrote: > > I'm sorry for my English language. > > > > OK, I have list = [10,20,30,40,110,50,18,32,5] and want to find three values of summation from my list by split any way list but three values of summation has closely > > example. > > [10,20,30], [40,110,50], [18,32,5] = 60, 200, 55 -> it is not closely > > *** 60 = 10+20+30, 200 = 40+110+50, 18+32+5 = 55 > > and [10], [20,30,40,110,50,18], [32,5] = 10, 268, 37 -> and it is not closely > > and [10,20,30,40],[110],[50,18,32,5] = 100,110,105 -> it's closely > > > > The Really answers that I want is [10,20,30,40],[110],[50,18,32,5] = 100,110,105 from list[10,20,30,40,110,50,18,32,5] > > > > And I don't know what I have to explain for my questions in English because My English so bad T_T > > > > Thank you for reading my text > > > > (; > > Yes it is, I want example python code for finding my answers but I don't code my answers. please help me. From amornsak.nak at gmail.com Mon Oct 10 12:25:28 2016 From: amornsak.nak at gmail.com (Nuen9) Date: Mon, 10 Oct 2016 09:25:28 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Message-ID: > Hi! > > Could it be, "Nuen9", that you would like to find a split where the > split sums are close to each other? In other words, you define the > number of splits (in your example: 3) and the algortihm should test all > possible combinations and select the split where the sum differences are > smallest. > > Best, > Kimmo Yes it is, I want example python code for finding my answers but I don't code my answers. please help me. From torriem at gmail.com Mon Oct 10 12:28:29 2016 From: torriem at gmail.com (Michael Torrie) Date: Mon, 10 Oct 2016 10:28:29 -0600 Subject: PyQT - Signals and Slots? In-Reply-To: References: Message-ID: <7a69dbf5-f3f5-0451-181d-f7d22e3d360d@gmail.com> On 10/10/2016 07:32 AM, Veek M wrote: > Whaaa...t?? Could someone explain what exactly is his grand design > besides being awfully circuitous? So he has some other Form thingy.. and > in that he sets up another mapping from ZeroSpinBox.atzero --> > ZeroSpinBox.announce ???? where's announce and atzero defined? In your example code, "atzero" is implicitly defined as a signal in the connect() call. Under the hood in Qt, signals are all dispatched by strings when you emit() the signal. PyQt does allow you to define signals in a more pythonic way using class variables: class Foo(QWidget): atzero = SIGNAL( signature ) The signature can be a list of python types, or strings describing the C++ types. This is where the C++ nature of Qt leaks into PyQt. In C++ since the moc compiler takes C++ code and C++ type names and generates a meta-class that defines signals in terms of strings (moc does basic compile-time type checking), in Python we also have to provide signatures in strings also if we need something more than core python types (int, bool, etc). This signature helps PyQt (well Qt really) marshal the types properly when the signal's callback is called. But yes the example is convoluted and circuitous. I think it's just to demonstrate how signals and slots work, though. That's what the comments make clear. This isn't how you'd implement a real widget with customized behavior. In general I don't think widgets normally want to catch their own signals. There are other mechanisms for that. Usually you subclass the widget and then override the appropriate *Event() method. For example, to do something special on value changes I believe you'd subclass QSpinBox and override the changeEvent() method. Inside that you would get your value, do your logic, and potentially emit a custom signal. And of course calling the parent changeEvent() to propagate the event through to the parent class. From emile at fenx.com Mon Oct 10 12:48:15 2016 From: emile at fenx.com (Emile van Sebille) Date: Mon, 10 Oct 2016 09:48:15 -0700 Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Message-ID: On 10/10/2016 09:25 AM, Nuen9 wrote: >> Hi! >> >> Could it be, "Nuen9", that you would like to find a split where the >> split sums are close to each other? In other words, you define the >> number of splits (in your example: 3) and the algortihm should test all >> possible combinations and select the split where the sum differences are >> smallest. >> >> Best, >> Kimmo > > Yes it is, I want example python code for finding my answers but I don't code my answers. please help me. > Here's one way: n=3 land = [10,20,30,40,110,50,18,32,5] lsum = sum(land) lavg = lsum/n close = lavg/n idx = 0 lnsums = [0]*n ln=0 while idx < len(land) and ln lavg-close: ln+=1 print lnsums -- Emile From maillists at pp.inet.fi Mon Oct 10 13:29:32 2016 From: maillists at pp.inet.fi (K. Elo) Date: Mon, 10 Oct 2016 20:29:32 +0300 Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Message-ID: Hi! Here one possible solution: --- snip --- land = [10,20,30,40,110,50,18,32,5] landlength=len(land) winnersplit=[] for i in range(landlength-2): for j in range(landlength-1-i): splitsums=[sum(land[0:(i+1)]), sum(land[(i+1):(i+j+2)]), sum(land[(i+j+2):landlength])] differences=abs(splitsums[1]-splitsums[0])+abs(splitsums[2]-splitsums[1]) if (len(winnersplit)==0 or differences<(abs(sum(winnersplit[1])-sum(winnersplit[0]))+abs(sum(winnersplit[2])-sum(winnersplit[1])))): winnersplit=[land[0:(i+1)], land[(i+1):(i+j+2)], land[(i+j+2):landlength]] print(splitsums," <<< WE HAVE A NEW LEADER! >>>") else: print(splitsums, " <<< split differences too large :( >>>") print("And the winner is ... ", winnersplit) --- snip --- HTH, Kimmo 10.10.2016, 19:25, Nuen9 kirjoitti: >> Hi! >> >> Could it be, "Nuen9", that you would like to find a split where the >> split sums are close to each other? In other words, you define the >> number of splits (in your example: 3) and the algortihm should test all >> possible combinations and select the split where the sum differences are >> smallest. >> >> Best, >> Kimmo > > Yes it is, I want example python code for finding my answers but I don't code my answers. please help me. > From bc at freeuk.com Mon Oct 10 13:48:35 2016 From: bc at freeuk.com (BartC) Date: Mon, 10 Oct 2016 18:48:35 +0100 Subject: How to process syntax errors In-Reply-To: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> Message-ID: On 10/10/2016 16:44, Pierre-Alain Dorange wrote: > Chris Angelico wrote: > >> Yes and no. Syntax errors are detected when the script is compiled, so >> you can't do something like this: > > You're right, except that Python is never compiled, it was just checked > for syntax error before interpreting code. In the case of CPython, source code is compiled into byte-code before it's interpreted. Part of that process is detecting syntax errors. AFAIK, this is usually done a module at a time (convert an entire module to byte-code, then start interpreting). It might be a little different with functions such as eval and exec. -- Bartc From breamoreboy at gmail.com Mon Oct 10 15:44:13 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 10 Oct 2016 12:44:13 -0700 (PDT) Subject: Newbie Need Help On Regex! In-Reply-To: <9c05d361-6a24-49d9-8d37-d647fbe13679@googlegroups.com> References: <9c05d361-6a24-49d9-8d37-d647fbe13679@googlegroups.com> Message-ID: <9aae0de5-b479-4499-913f-c0fc4ee164b3@googlegroups.com> On Monday, October 10, 2016 at 3:58:56 PM UTC+1, infos... at gmail.com wrote: > Hey guys! > > I am new to learning regex in python and I'm wondering how do I use regex in python to store the integers(positive and negative) i want into a list! > > For e.g. > > This is the data in a list. > > [u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=-5,B=5)', u'\x1b[0m[\x1b[1m\x1b[0m\xbb\x1b[0m\x1b[36m]\x1b[0m (A=5,Y=5)', u'\x1b[0m[\x1b[1m\x1b[10m\xbb\x1b[0m\x1b[36m]\x1b[0m : '] > > How do I extract the values of A and B and store them in a variable I want using regex? > > Thank you and appreciate it :) What makes you think you need a regex? Why don't you write some code and when you have problems, post it here. Then we'll give you some answers. Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Oct 10 15:52:48 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 10 Oct 2016 12:52:48 -0700 (PDT) Subject: How to process syntax errors In-Reply-To: References: Message-ID: On Monday, October 10, 2016 at 3:15:40 PM UTC+1, mr.pune... at gmail.com wrote: > Hi > > Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. > > For example, > > inside example.py > > I just mention below line > > > Obj = myClass() > Obj xyz > > Obj is instance of a class. But there is syntax error on xyz. So I want to grab that error and process. > > Regards, Puneet Simple, catch it in your IDE, no need to try and get clever. Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Oct 10 16:00:21 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 10 Oct 2016 13:00:21 -0700 (PDT) Subject: Deviding N(1,2,3,..,N) part from numeric list as summation of each values(don't sorted) has highest as possible. In-Reply-To: References: <57fba62c$0$1590$c3e8da3$5496439d@news.astraweb.com> <43a41625-2d0c-4a14-b84c-9adea35d08f8@googlegroups.com> <9ab8f055-af1c-6757-4615-80c88f1f398d@pp.inet.fi> Message-ID: On Monday, October 10, 2016 at 5:25:46 PM UTC+1, Nuen9 wrote: > > Hi! > > > > Could it be, "Nuen9", that you would like to find a split where the > > split sums are close to each other? In other words, you define the > > number of splits (in your example: 3) and the algortihm should test all > > possible combinations and select the split where the sum differences are > > smallest. > > > > Best, > > Kimmo > > Yes it is, I want example python code for finding my answers but I don't code my answers. please help me. Awfully sorry but we don't write your code for you. I suggest that you read any Python tutorial and try running code at an interactive prompt, then when you run into problems ask a specific question about a specific problem. Kindest regards. Mark Lawrence. From martin.schoon at gmail.com Mon Oct 10 17:03:31 2016 From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=) Date: 10 Oct 2016 21:03:31 GMT Subject: Free Python e-books from O'Reilly Message-ID: http://www.oreilly.com/programming/free/ Any of these good enough to bother getting? Yes, I know, they are for no pay but if not worth reading still a waste of time and memory space. /Martin From saxri89 at gmail.com Mon Oct 10 17:50:20 2016 From: saxri89 at gmail.com (Xristos Xristoou) Date: Mon, 10 Oct 2016 14:50:20 -0700 (PDT) Subject: Raster top and lowe x,y how ca i get Message-ID: How can i get left lower X,Y left top X,Y right lower X,Y and right Top X,Y from a raster image like dem(digital elevation model). I am nembie sorry if i have stupid quest From uri at speedy.net Mon Oct 10 18:08:02 2016 From: uri at speedy.net (Uri Even-Chen) Date: Tue, 11 Oct 2016 01:08:02 +0300 Subject: Free Python e-books from O'Reilly In-Reply-To: References: Message-ID: I think it's good that so many free books are about Python, more than any other specific programming language. But I haven't read any of them yet. *Uri Even-Chen* [image: photo] Phone: +972-54-3995700 Email: uri at speedy.net Website: http://www.speedysoftware.com/uri/en/ On Tue, Oct 11, 2016 at 12:03 AM, Martin Sch??n wrote: > http://www.oreilly.com/programming/free/ > > Any of these good enough to bother getting? Yes, I know, they > are for no pay but if not worth reading still a waste of time > and memory space. > > /Martin > -- > https://mail.python.org/mailman/listinfo/python-list > From nad at python.org Mon Oct 10 21:12:17 2016 From: nad at python.org (Ned Deily) Date: Mon, 10 Oct 2016 21:12:17 -0400 Subject: [RELEASE] Python 3.6.0b2 is now available Message-ID: <4E66D65C-0EFF-41B9-9B5E-1B3FF60151B4@python.org> On behalf of the Python development community and the Python 3.6 release team, I'm happy to announce the availability of Python 3.6.0b2. 3.6.0b2 is the second of four planned beta releases of Python 3.6, the next major release of Python, and marks the end of the feature development phase for 3.6. Among the new major new features in Python 3.6 are: * PEP 468 - Preserving the order of **kwargs in a function * PEP 487 - Simpler customization of class creation * PEP 495 - Local Time Disambiguation * PEP 498 - Literal String Formatting * PEP 506 - Adding A Secrets Module To The Standard Library * PEP 509 - Add a private version to dict * PEP 515 - Underscores in Numeric Literals * PEP 519 - Adding a file system path protocol * PEP 520 - Preserving Class Attribute Definition Order * PEP 523 - Adding a frame evaluation API to CPython * PEP 524 - Make os.urandom() blocking on Linux (during system startup) * PEP 525 - Asynchronous Generators (provisional) * PEP 526 - Syntax for Variable Annotations (provisional) * PEP 528 - Change Windows console encoding to UTF-8 (provisional) * PEP 529 - Change Windows filesystem encoding to UTF-8 (provisional) * PEP 530 - Asynchronous Comprehensions Please see "What?s New In Python 3.6" for more information: https://docs.python.org/3.6/whatsnew/3.6.html You can find Python 3.6.0b2 here: https://www.python.org/downloads/release/python-360b2/ Beta releases are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. We strongly encourage maintainers of third-party Python projects to test with 3.6 during the beta phase and report issues found to bugs.python.org as soon as possible. While the release is feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2016-12-05). Our goal is have no changes after rc1. To achieve that, it will be extremely important to get as much exposure for 3.6 as possible during the beta phase. Please keep in mind that this is a preview release and its use is not recommended for production environments The next planned release of Python 3.6 will be 3.6.0b3, currently scheduled for 2016-10-31. More information about the release schedule can be found here: https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad at python.org -- [] From gordon at panix.com Tue Oct 11 00:32:48 2016 From: gordon at panix.com (John Gordon) Date: Tue, 11 Oct 2016 04:32:48 +0000 (UTC) Subject: Free Python e-books from O'Reilly References: Message-ID: In Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?= writes: > http://www.oreilly.com/programming/free/ > Any of these good enough to bother getting? Yes, I know, they > are for no pay but if not worth reading still a waste of time > and memory space. I downloaded several of them and noticed that they were all fairly short, none more than 80 pages or so. I suspect these books are somewhat lighter fare than the typical O'Reilly tome. Not necessarily a bad thing, but worth mentioning. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From tjreedy at udel.edu Tue Oct 11 00:58:03 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Oct 2016 00:58:03 -0400 Subject: How to process syntax errors In-Reply-To: References: Message-ID: On 10/10/2016 11:24 AM, Chris Angelico wrote: > On Tue, Oct 11, 2016 at 1:13 AM, wrote: >> Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. > However, you can catch this at some form of outer level. If you're > using exec/eval to run the code, you can guard that: > > code = """ > obj = MyClass() # using PEP 8 names > obj xyz > """ > try: > exec(code) > except SyntaxError: > print("Blah blah blah") Far from being 'un-pythonic' (Puneet's response), this is essentially what the interpreter does, either without or with a separate explicit compile step. IDLE splits the exec into two pieces in two processes. Except for the two process part, this either uses or copies code.InteractiveInterpreter in the stdlib. The fact that the latter separates compile and exec is what allows IDLE to subclass it and run the code object elsewhere, by over-riding the runcode method. # in IDLE process: code = try: code_ob = compile(code, ....) except (SyntaxError, ) as e: # replace ^ with red background highlight else: # in user process exec(code, fake_main) # stdout, stderr connect to IDLE process -- Terry Jan Reedy From pdorange at pas-de-pub-merci.mac.com Tue Oct 11 04:02:39 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Tue, 11 Oct 2016 10:02:39 +0200 Subject: How to process syntax errors References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> Message-ID: <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Chris Angelico wrote: > >> Yes and no. Syntax errors are detected when the script is compiled, so > >> you can't do something like this: > > > > You're right, except that Python is never compiled, it was just checked > > for syntax error before interpreting code. > > https://docs.python.org/3/library/functions.html#compile > > It's compiled. Using this function, the code is "compiled". I do not think this function is often used and most python project simply use the interpreter (which do a small translation into byte-code to be faster and check syntax error before running interpretation). > > >> However, you can catch this at some form of outer level. If you're > >> using exec/eval to run the code, you can guard that: > > > > Your solution are OK, but that's not very "pythonic". > > > > Python was not design to be used in such a way (intercepting syntax > > error). There was hack to made somthing like requested but that only a > > hack and should not be used in real world. > > The error is raised as an exception, which means it is most definitely > meant to be able to be caught. Using compile() function yes. So yes there is a way to check "syntax error" before executing code (using compile function and exceptions) but it was not standard, nor widely used... It was still a hack for me, but perhaps i misunderstood or misinterpret. -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From rosuav at gmail.com Tue Oct 11 04:19:48 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 11 Oct 2016 19:19:48 +1100 Subject: How to process syntax errors In-Reply-To: <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Message-ID: On Tue, Oct 11, 2016 at 7:02 PM, Pierre-Alain Dorange wrote: > Chris Angelico wrote: > >> >> Yes and no. Syntax errors are detected when the script is compiled, so >> >> you can't do something like this: >> > >> > You're right, except that Python is never compiled, it was just checked >> > for syntax error before interpreting code. >> >> https://docs.python.org/3/library/functions.html#compile >> >> It's compiled. > > Using this function, the code is "compiled". > I do not think this function is often used and most python project > simply use the interpreter (which do a small translation into byte-code > to be faster and check syntax error before running interpretation). When you import, the same form of compilation is done. It always happens. Python does not execute your raw source code - it'd be too inefficient. ChrisA From ben+python at benfinney.id.au Tue Oct 11 04:28:14 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 11 Oct 2016 19:28:14 +1100 Subject: Python code is compiled before execution (was: How to process syntax errors) References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Message-ID: <85wphfmgrl.fsf_-_@benfinney.id.au> pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) writes: > Chris Angelico wrote: > > > https://docs.python.org/3/library/functions.html#compile > > > > [Python code is] compiled. > > Using this function, the code is "compiled". You have it backward: Python code is compiled. That's what the Python runtime requires. It often happens automatically, but it happens for all Python code that gets executed. What that function does is allow you to *separately* compile the Python code. > I do not think this function is often used and most python project > simply use the interpreter (which do a small translation into byte-code > to be faster and check syntax error before running interpretation). The ?small translation into byte-code? *is* compilation. Don't make the mistake that the only product of ?compile? is some CPU code; that is a foolishly narrow definition. (Good sigmonster, have a cookie.) -- \ ?[I]t is impossible for anyone to begin to learn that which he | `\ thinks he already knows.? ?Epictetus, _Discourses_ | _o__) | Ben Finney From marko at pacujo.net Tue Oct 11 04:38:03 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 11 Oct 2016 11:38:03 +0300 Subject: How to process syntax errors References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Message-ID: <87zimb702c.fsf@elektro.pacujo.net> Chris Angelico : >>> > You're right, except that Python is never compiled, it was just >>> > checked for syntax error before interpreting code. >>> >>> https://docs.python.org/3/library/functions.html#compile >>> >>> It's compiled. >> >> Using this function, the code is "compiled". >> I do not think this function is often used and most python project >> simply use the interpreter (which do a small translation into >> byte-code to be faster and check syntax error before running >> interpretation). > > When you import, the same form of compilation is done. It always > happens. Python does not execute your raw source code - it'd be too > inefficient. There's no clear line between compilation and interpretation. You have the source code and the execution semantics. All kinds of opportune transformations can take place before the execution, after the execution, during the execution etc as long as the correct semantics are obeyed. Marko From jfong at ms4.hinet.net Tue Oct 11 06:34:03 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Tue, 11 Oct 2016 03:34:03 -0700 (PDT) Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? Message-ID: I have two files in the Q:\lib directory: Q:\lib>dir 2007/03/11 AM 08:02 5,260 lib_MARK.so 2007/03/11 AM 08:02 4,584 lib_mark.so Under Python 3.4.4 I got: >>> f = open('lib_MARK.so', 'br') >>> data = f.read() >>> f.close() >>> len(data) 4584 I know Windows won't, but can Python make it true? --Jach From laborhm at gmail.com Tue Oct 11 07:33:32 2016 From: laborhm at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCb0L7Qs9GD0L3QvtCy?=) Date: Tue, 11 Oct 2016 04:33:32 -0700 (PDT) Subject: Python Shell in Russian exists/possible?? Message-ID: <61c6eb6b-59fb-4f09-a08f-ef0476fa0ee1@googlegroups.com> I need the Python Shell for use in education (turtle graphics, etc.), but the UI must be localized in the Russian language. The question is if it's at all possible to feed the strings in or rebuild it or... From pdorange at pas-de-pub-merci.mac.com Tue Oct 11 08:35:58 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Tue, 11 Oct 2016 14:35:58 +0200 Subject: Python code is compiled before execution References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> Message-ID: <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> Ben Finney wrote: > The "small translation into byte-code" *is* compilation. > > Don't make the mistake that the only product of "compile" is some CPU > code; that is a foolishly narrow definition. OK right. For my part, i differenciate a strict compilation (ie. C) from a translation into byte-code (ie. Python). From design there was differences as Python was an interpreted language, not a compiled (AOT). But as there was only one word, let do with it ;-) Of course in real language world there was many situation and all kind of combinaison : strict interpretation, byte-code compilation, JIT compilation, AOT compilation... So yes Python compile (bytecode). -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From eryksun at gmail.com Tue Oct 11 08:48:08 2016 From: eryksun at gmail.com (eryk sun) Date: Tue, 11 Oct 2016 12:48:08 +0000 Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: References: Message-ID: On Tue, Oct 11, 2016 at 10:34 AM, wrote: > I have two files in the Q:\lib directory: > > Q:\lib>dir > 2007/03/11 AM 08:02 5,260 lib_MARK.so > 2007/03/11 AM 08:02 4,584 lib_mark.so > > Under Python 3.4.4 I got: > >>>> f = open('lib_MARK.so', 'br') >>>> data = f.read() >>>> f.close() >>>> len(data) > 4584 > > I know Windows won't, but can Python make it true? Windows can do case-sensitive file operations, but it's not easy in Python. You have to use the Windows API directly via _winapi, ctypes, or PyWin32, depending on your needs. First, disable the registry setting that forces the kernel's object manager to be case insensitive for operations on its own object types (i.e. object directories and symlinks) and I/O object types (e.g. devices, files). Here's a .reg file to do this: Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\kernel] "obcaseinsensitive"=dword:00000000 You have to reboot after making this change. This alone doesn't make the Windows API case sensitive, but it does enable individual CreateFile calls to be case sensitive, via the POSIX semantics flag. Once you have a handle for the target file or directory, you can accomplish most file operations via GetFileInformationByHandleEx [1] and SetFileInformationByHandle [2]. [1]: https://msdn.microsoft.com/en-us/library/aa364953 [2]: https://msdn.microsoft.com/en-us/library/aa365539 Here's a basic example, based on your example with "lib_mark.so" and "lib_MARK.so": import os import _winapi GENERIC_READ = 0x80000000 GENERIC_WRITE = 0x40000000 OPEN_EXISTING = 3 FILE_SHARE_READ = 1 FILE_SHARE_WRITE = 2 FILE_SHARE_DELETE = 4 FILE_FLAG_POSIX_SEMANTICS = 0x001000000 def create_file(path, access=GENERIC_READ | GENERIC_WRITE, sharing=FILE_SHARE_READ | FILE_SHARE_WRITE, disposition=OPEN_EXISTING, flags=0): sharing |= FILE_SHARE_DELETE flags |= FILE_FLAG_POSIX_SEMANTICS return _winapi.CreateFile(path, access, sharing, 0, disposition, flags, 0) >>> os.listdir('.') ['lib_MARK.so', 'lib_mark.so'] >>> h1 = create_file('lib_MARK.so') >>> len(_winapi.ReadFile(h1, 10000)[0]) 5260 >>> h2 = create_file('lib_mark.so') >>> len(_winapi.ReadFile(h2, 10000)[0]) 4584 From bc at freeuk.com Tue Oct 11 09:19:08 2016 From: bc at freeuk.com (BartC) Date: Tue, 11 Oct 2016 14:19:08 +0100 Subject: Python code is compiled before execution In-Reply-To: <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> Message-ID: On 11/10/2016 13:35, Pierre-Alain Dorange wrote: > Ben Finney wrote: > >> The "small translation into byte-code" *is* compilation. >> >> Don't make the mistake that the only product of "compile" is some CPU >> code; that is a foolishly narrow definition. > > OK right. > For my part, i differenciate a strict compilation (ie. C) from a > translation into byte-code (ie. Python). From design there was > differences as Python was an interpreted language, not a compiled (AOT). > But as there was only one word, let do with it ;-) > > Of course in real language world there was many situation and all kind > of combinaison : strict interpretation, byte-code compilation, JIT > compilation, AOT compilation... > > So yes Python compile (bytecode). I use 'byte-code compiler' and 'native-code compiler' when the distinction is important. Python is not really suited for AOT native-code compilation. Here I might just say CPython if I'm assuming a normal byte-code compiler and byte-code interpreter model. -- bartc From bc at freeuk.com Tue Oct 11 09:22:34 2016 From: bc at freeuk.com (BartC) Date: Tue, 11 Oct 2016 14:22:34 +0100 Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: References: <4fkpvbtb9uauoc4chl2uqjfo8i6cciccef@4ax.com> Message-ID: On 11/10/2016 13:49, Dennis Lee Bieber wrote: > Unfortunately, Windows is one of the poorest documented OS I've > encountered The problem is more that there is too much documentation! -- Bartc From michael at felt.demon.nl Tue Oct 11 10:29:40 2016 From: michael at felt.demon.nl (Michael Felt) Date: Tue, 11 Oct 2016 16:29:40 +0200 Subject: Conventions and requirements for a python module Message-ID: <5d2b6740-9295-063e-bda2-c182305b23ee@felt.demon.nl> From reading the python source, and other projects I am looking to patch I see that there is often a file __init__.py, sometimes empty (only comments), sometimes not. I have tried looking in what I hope are the "regular" places such as: https://docs.python.org, readthedocs (it took 454 seconds to build something - what did it build, and where do I get it? I was assuming it was "latest documentation" and I even tried the old, no-longer maintained, wiki. A search for __init__.py, except for on https://packaging.python.org/ that talks about a setup command to extract the version from __init__.py - I have not been able to find anything on "standards" for packages that are more than a single .py file. Probably, I am not looking correctly - however, I do hope someone also notices that finding this is not straight forward for a novice in python. Had I lacked curiosity I would have given up, moved on. Instead - this email. In short, I would rather try to "read the manual" and ask questions when I get stuck than ask "stupid" questions. That only makes me look bad ;) (e.g., in my other thread the idea of using subclasses was made frequently. That was something I was considering, but what they are, how they are specified, what they achieve rather than guessing that it means - intent some related class xxx: lines. re: project setup: when to use _filename.py - when to use filename.py; what is the difference/significance of a sub-directory when "masking" complexity, better implementation details, is the goal (i.e., there is the API and there are the nitty-gritty implementation details). The key thing I have learned is that I must not assume that python is anything like what I already know/knew. Assume python "does it differently" and mistakes maybe fewer. Thanks for your time and pointers (to links, if not direct advice) Michael From torriem at gmail.com Tue Oct 11 11:30:57 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 11 Oct 2016 09:30:57 -0600 Subject: Conventions and requirements for a python module In-Reply-To: <5d2b6740-9295-063e-bda2-c182305b23ee@felt.demon.nl> References: <5d2b6740-9295-063e-bda2-c182305b23ee@felt.demon.nl> Message-ID: On 10/11/2016 08:29 AM, Michael Felt wrote: > From reading the python source, and other projects I am looking to > patch I see that there is often a file __init__.py, sometimes empty > (only comments), sometimes not. > > I have tried looking in what I hope are the "regular" places such as: > https://docs.python.org, readthedocs (it took 454 seconds to build > something - what did it build, and where do I get it? I was assuming it > was "latest documentation" and I even tried the old, no-longer > maintained, wiki. > > A search for __init__.py, except for on https://packaging.python.org/ > that talks about a setup command to extract the version from __init__.py > - I have not been able to find anything on "standards" for packages that > are more than a single .py file. > > Probably, I am not looking correctly - however, I do hope someone also > notices that finding this is not straight forward for a novice in > python. Had I lacked curiosity I would have given up, moved on. Instead > - this email. Yes all this is in the docs: https://docs.python.org/3/tutorial/modules.html#packages Also there are links that show up when searching google for "python packages": http://www.learnpython.org/en/Modules_and_Packages http://docs.python-guide.org/en/latest/writing/structure/#packages Packages are convenient for organizing code. Typically I put code into __init__.py to organize the package's symbols. In other words whatever symbols get imported when the package is imported are defined here. Sometimes I'll import things from sub-modules from within __init__.py and name and organize them. Sometimes submodules are left on their own to be imported separately. Though the notation "from foo import *" is discouraged, if a programmer is wont to do that, the package can limit the damage somewhat by defining the __all__ variable. From daiyueweng at gmail.com Tue Oct 11 11:50:07 2016 From: daiyueweng at gmail.com (Daiyue Weng) Date: Tue, 11 Oct 2016 16:50:07 +0100 Subject: shadows name from outer scope Message-ID: Hi, I have the following code structure, def one_func(param1, param2, param3): process_outcome = {'dict': None} outcome = another_func(param1, process_outcome, db_host=param2, funcs_to_run=param3) PyCharm warns me Shadows name from outer scope on every variable here, param1,2,3, process_outcome. I am wondering how to resolve the issues, and is it vital to have variables not shadowed? cheers From lanuradha at gmail.com Tue Oct 11 12:19:32 2016 From: lanuradha at gmail.com (Anuradha Laxminarayan) Date: Tue, 11 Oct 2016 09:19:32 -0700 (PDT) Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <57F9F63D.8080300@canterbury.ac.nz> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> On Sunday, 9 October 2016 13:18:32 UTC+5:30, Gregory Ewing wrote: > Here's the first part of the essay I said I'd write about > monads: > > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html > > Hope it's useful, > Greg Thanks, that made a very interesting read. In Haskell, the type M a -> M b-> M b tells a good deal of the story of seq In a Pythonic context, continuations seem to be an effective way to carry that story. Small minor haskell suggestions from a pedagodic pov [related to Ben Finney's suggestion?] seq f g h = f (\s1 -> g h s1) better be written as seq f g x = f (\s1 -> g x s1) because naming conventions imply that h is function. - Anuradha From grant.b.edwards at gmail.com Tue Oct 11 12:31:42 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 11 Oct 2016 16:31:42 +0000 (UTC) Subject: Python code is compiled before execution References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> Message-ID: On 2016-10-11, Pierre-Alain Dorange wrote: > For my part, i differenciate a strict compilation (ie. C) from a > translation into byte-code (ie. Python). FWIW I've seen C compilers that produced byte-code. They allowed for a highly interactive developemnt environment. -- Grant Edwards grant.b.edwards Yow! One FISHWICH coming at up!! gmail.com From pdorange at pas-de-pub-merci.mac.com Tue Oct 11 12:40:16 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Tue, 11 Oct 2016 18:40:16 +0200 Subject: Python code is compiled before execution References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> Message-ID: <1muyx0j.1vzq67osqwjzgN%pdorange@pas-de-pub-merci.mac.com> Grant Edwards wrote: > > For my part, i differenciate a strict compilation (ie. C) from a > > translation into byte-code (ie. Python). > > FWIW I've seen C compilers that produced byte-code. They allowed for > a highly interactive developemnt environment. I do not want to fight every word and every context. As said before, in real world, all is possible and all situation is possible : from pure interpreter to pure compiler (low level machine language I just want, at the beginning to point that, by principe Python was an interpreted language, and i found strange the initial request. And sure, i overinterpret or simplify thing (compiling)... But it seems that all this talk do not interested the initial requester (Mr Puneet) : at that time he has not answer to the thread. -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From steve+python at pearwood.info Tue Oct 11 12:53:41 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 12 Oct 2016 03:53:41 +1100 Subject: Python Shell in Russian exists/possible?? References: <61c6eb6b-59fb-4f09-a08f-ef0476fa0ee1@googlegroups.com> Message-ID: <57fd1917$0$22141$c3e8da3$5496439d@news.astraweb.com> On Tue, 11 Oct 2016 10:33 pm, ?????? ??????? wrote: > I need the Python Shell for use in education (turtle graphics, etc.), but > the UI must be localized in the Russian language. The question is if it's > at all possible to feed the strings in or rebuild it or... Yes it is possible, but it is probably a lot of work. There are at least two projects for localising Python into non-English. One is a "toy" project, just to show that it can be done: http://www.fiber-space.de/EasyExtend/doc/teuton/teuton.htm but I understand that the second is quite serious: http://www.chinesepython.org/english/english.html or at least was very serious a few years ago. Unfortunately I don't know if it is still maintained. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 11 13:12:05 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 12 Oct 2016 04:12:05 +1100 Subject: How to process syntax errors References: Message-ID: <57fd1d67$0$1585$c3e8da3$5496439d@news.astraweb.com> On Tue, 11 Oct 2016 03:58 pm, Terry Reedy wrote: > On 10/10/2016 11:24 AM, Chris Angelico wrote: >> On Tue, Oct 11, 2016 at 1:13 AM, wrote: >>> Is there any way to capture syntax errors and process them ? I want to >>> write a function which calls every time whenever there is syntax error >>> in the program. > >> However, you can catch this at some form of outer level. If you're >> using exec/eval to run the code, you can guard that: >> >> code = """ >> obj = MyClass() # using PEP 8 names >> obj xyz >> """ >> try: >> exec(code) >> except SyntaxError: >> print("Blah blah blah") > > Far from being 'un-pythonic' (Puneet's response), this is essentially > what the interpreter does, either without or with a separate explicit > compile step. There is a *huge* difference between writing an interpreter which takes source code written in idiomatic Python: # source code looks like this x = [1, 2] y = len(x) print(y + 1) reading that source code into a string (either all at once, or a line at a time), then using exec() to execute the string: try: exec(source_code) except SyntaxError: # emit error and what I believe the OP wishes to do, which is to put the entire source code inside a call to exec: # source code looks like this: try: text = """\ x = [1, 2] y = len(x) print(y + 1) """ exec(text) except SyntaxError: # recover from syntax error somehow, and keep processing What the OP appears to want is not to write Python code at all, but to write something where function calls are delimited with spaces: # source code looks like this: try: text = """\ x = [1, 2] y = len x print y + 1 """ exec(text) except SyntaxError: # automatically re-format the syntax errors into legal # Python code and execute again See the OP's thread with subject "A newbie doubt on methods/functions calling" from last week. I believe that he hasn't given up on the hope to avoid writing parentheses and just use spaces, even though we've already explained that is ambiguous: func a b c # is that func(a, b, c) or func(a(b, c)) or func(a, b(c)) or ... ? I don't believe that mr.puneet.goyal is planning to write an interpreter for Python. I believe that he's hoping to find some trick or hack so he can avoid parentheses for function calls, and he hopes that by catching SyntaxError he can do this. But I could be wrong... -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 11 13:14:40 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 12 Oct 2016 04:14:40 +1100 Subject: Python code is compiled before execution References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> Message-ID: <57fd1e02$0$1585$c3e8da3$5496439d@news.astraweb.com> On Wed, 12 Oct 2016 12:19 am, BartC wrote: > Python is not really suited for AOT native-code compilation. You might be right, but the author of Nuitka disagrees. http://nuitka.net/ Nice to see that there's a new release, only a week or so ago! -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Tue Oct 11 13:17:44 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 11 Oct 2016 10:17:44 -0700 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> Message-ID: <87k2dest3b.fsf@nightsong.com> Anuradha Laxminarayan writes: > seq f g x = f (\s1 -> g x s1) > because naming conventions imply that h is function. Also if this operation is what it looks like, it's usually called "bind". seq is something else entirely. From rustompmody at gmail.com Tue Oct 11 13:25:38 2016 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 11 Oct 2016 10:25:38 -0700 (PDT) Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> Message-ID: On Tuesday, October 11, 2016 at 9:53:25 PM UTC+5:30, Anuradha Laxminarayan wrote: > On Sunday, 9 October 2016 13:18:32 UTC+5:30, Gregory Ewing wrote: > > Here's the first part of the essay I said I'd write about > > monads: > > > > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html > > > > Hope it's useful, > > Greg > > Thanks, that made a very interesting read. Yeah agreed! Just got it running in ghc. Some small modifications needed: import Data.Set add x f s = f (insert x s) listify f s = f (toList s) s seqq f g x = f (\s1 -> g x s1) dedup (x:t) = seqq (add x) (dedup t) dedup [] = listify run_smf f a = f a (\r s -> r) empty > seq f g h = f (\s1 -> g h s1) > better be written as > seq f g x = f (\s1 -> g x s1) > because naming conventions imply that h is function. I think it would be good to make state and adhere to a name-convention eg c for continuation f,g for general function s for state etc Otherwise excellent work! V useful From dhawanpawan32 at gmail.com Tue Oct 11 13:57:24 2016 From: dhawanpawan32 at gmail.com (dhawanpawan32 at gmail.com) Date: Tue, 11 Oct 2016 10:57:24 -0700 (PDT) Subject: Not getting output Message-ID: def abc(a,b): l=[] for i in range(a,b+1): if i%2!=0: l.append(i) print l return 1 def bg(): y=abc(11,31) print y I am not getting the output, but if i call method abc outside the function then it's coming From dhawanpawan32 at gmail.com Tue Oct 11 13:58:31 2016 From: dhawanpawan32 at gmail.com (dhawanpawan32 at gmail.com) Date: Tue, 11 Oct 2016 10:58:31 -0700 (PDT) Subject: while loop Message-ID: n=6 x=1 while x<=n: print "*"*x x+=1 while n>=x: n=n-1 print "*"* n Only first loop is executing not the second one? From larry.martell at gmail.com Tue Oct 11 14:10:11 2016 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 11 Oct 2016 14:10:11 -0400 Subject: while loop In-Reply-To: References: Message-ID: On Tue, Oct 11, 2016 at 1:58 PM, wrote: > > n=6 > x=1 > while x<=n: > print "*"*x > x+=1 > while n>=x: > n=n-1 > print "*"* n > > > Only first loop is executing not the second one? Because after the first loop n < x From jussi.piitulainen at helsinki.fi Tue Oct 11 14:25:04 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 11 Oct 2016 21:25:04 +0300 Subject: Not getting output References: Message-ID: dhawanpawan32 at gmail.com writes: > def abc(a,b): > l=[] > for i in range(a,b+1): > if i%2!=0: > > l.append(i) > print l > return 1 > > def bg(): > y=abc(11,31) > print y > > I am not getting the output, but if i call method abc outside the > function then it's coming Do you perhaps expect abc to return the list that it builds and prints? If so, use some other name than l. One that doesn't look so much like the number 1. From torriem at gmail.com Tue Oct 11 14:28:41 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 11 Oct 2016 12:28:41 -0600 Subject: Python Shell in Russian exists/possible?? In-Reply-To: <61c6eb6b-59fb-4f09-a08f-ef0476fa0ee1@googlegroups.com> References: <61c6eb6b-59fb-4f09-a08f-ef0476fa0ee1@googlegroups.com> Message-ID: On 10/11/2016 05:33 AM, ?????? ??????? wrote: > I need the Python Shell for use in education (turtle graphics, etc.), > but the UI must be localized in the Russian language. The question is > if it's at all possible to feed the strings in or rebuild it or... For educational purposes, you might find the Jupyter Notebook to be worthwhile. It's the interactive Python interpreter but in a format you can use from the browser and it acts like a notebook too, allowing editing and actually building of code from interactive snippets. There is some effort underway to build localization of language into it. Unfortunately this effort is in its early stages. There's also a module someone wrote for it to let you do the turtle graphics module inside the notebook. Anyway, if they had localization all finished, I'd say Jupyter Notebook would be the way to go for your purposes. From jussi.piitulainen at helsinki.fi Tue Oct 11 14:30:06 2016 From: jussi.piitulainen at helsinki.fi (Jussi Piitulainen) Date: Tue, 11 Oct 2016 21:30:06 +0300 Subject: while loop References: Message-ID: dhawanpawan32 at gmail.com writes: > n=6 > x=1 > while x<=n: > print "*"*x > x+=1 > while n>=x: > n=n-1 > print "*"* n > > > Only first loop is executing not the second one? It's a basic fact about while loops that after the loop the condition is false. The two conditions x <= n and n >= x are equivalent. From torriem at gmail.com Tue Oct 11 14:31:05 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 11 Oct 2016 12:31:05 -0600 Subject: while loop In-Reply-To: References: Message-ID: <7baed72b-4361-a90b-b8bb-14ab6e497462@gmail.com> On 10/11/2016 11:58 AM, dhawanpawan32 at gmail.com wrote: > > n=6 > x=1 > while x<=n: > print "*"*x > x+=1 > while n>=x: > n=n-1 > print "*"* n > > > Only first loop is executing not the second one? Did you try printing out the loop variable to see what it does and what it is after the loop is finished? From torriem at gmail.com Tue Oct 11 14:33:07 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 11 Oct 2016 12:33:07 -0600 Subject: Not getting output In-Reply-To: References: Message-ID: <51c0bba9-b875-6147-4f2c-5bce31b90af4@gmail.com> On 10/11/2016 11:57 AM, dhawanpawan32 at gmail.com wrote: > def abc(a,b): > l=[] > for i in range(a,b+1): > if i%2!=0: > > l.append(i) > print l > return 1 > > def bg(): > y=abc(11,31) > print y > I am not getting the output, but if i call method abc outside the function then it's coming Is this your entire program? Because all I see are two function definitions. You have no code that actually calls either of these two functions from the main level of your code. Remember that Python executes scripts in a procedural manner. From bc at freeuk.com Tue Oct 11 14:45:05 2016 From: bc at freeuk.com (BartC) Date: Tue, 11 Oct 2016 19:45:05 +0100 Subject: Not getting output In-Reply-To: References: Message-ID: On 11/10/2016 18:57, dhawanpawan32 at gmail.com wrote: > def abc(a,b): > l=[] > for i in range(a,b+1): > if i%2!=0: > > l.append(i) > print l > return 1 > > def bg(): > y=abc(11,31) > print y > I am not getting the output, but if i call method abc outside the function then it's coming > Try adding: bg() at the end in order to call bg. -- Bartc From bc at freeuk.com Tue Oct 11 15:03:05 2016 From: bc at freeuk.com (BartC) Date: Tue, 11 Oct 2016 20:03:05 +0100 Subject: Python code is compiled before execution In-Reply-To: <57fd1e02$0$1585$c3e8da3$5496439d@news.astraweb.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <85wphfmgrl.fsf_-_@benfinney.id.au> <1muykzr.158y9g21vqrm9sN%pdorange@pas-de-pub-merci.mac.com> <57fd1e02$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 11/10/2016 18:14, Steve D'Aprano wrote: > On Wed, 12 Oct 2016 12:19 am, BartC wrote: > >> Python is not really suited for AOT native-code compilation. > > You might be right, but the author of Nuitka disagrees. > > http://nuitka.net/ I tried the same thing with a dynamic language: translating byte-code into a series of function calls in a compiled language. So it wasn't really compiling to proper native code; the native code still had to deal with the byte-code rather than do the work directly. It didn't really work: I saved the minor overheads of byte-code dispatching, but it ended up being slower than byte-code. (Perhaps to do with worse cache performance and fewer inlining opportunities.) From looking at the overview in that link, it might be taking a similar approach. But maybe CPython dispatch overheads are higher, or maybe it just does a better job! -- Bartc From michael at felt.demon.nl Tue Oct 11 16:02:33 2016 From: michael at felt.demon.nl (Michael Felt) Date: Tue, 11 Oct 2016 22:02:33 +0200 Subject: Conventions and requirements for a python module In-Reply-To: References: <5d2b6740-9295-063e-bda2-c182305b23ee@felt.demon.nl> Message-ID: <5af89277-d56e-927a-5ed6-b76608e28f33@felt.demon.nl> On 11/10/2016 17:30, Michael Torrie wrote: > On 10/11/2016 08:29 AM, Michael Felt wrote: >> From reading the python source, and other projects I am looking to >> patch I see that there is often a file __init__.py, sometimes empty >> (only comments), sometimes not. >> >> I have tried looking in what I hope are the "regular" places such as: >> https://docs.python.org, readthedocs (it took 454 seconds to build >> something - what did it build, and where do I get it? I was assuming it >> was "latest documentation" and I even tried the old, no-longer >> maintained, wiki. >> >> A search for __init__.py, except for on https://packaging.python.org/ >> that talks about a setup command to extract the version from __init__.py >> - I have not been able to find anything on "standards" for packages that >> are more than a single .py file. >> >> Probably, I am not looking correctly - however, I do hope someone also >> notices that finding this is not straight forward for a novice in >> python. Had I lacked curiosity I would have given up, moved on. Instead >> - this email. > Yes all this is in the docs: > https://docs.python.org/3/tutorial/modules.html#packages Thanks! > > Also there are links that show up when searching google for "python > packages": > http://www.learnpython.org/en/Modules_and_Packages > http://docs.python-guide.org/en/latest/writing/structure/#packages And again! > Packages are convenient for organizing code. Typically I put code into > __init__.py to organize the package's symbols. In other words whatever > symbols get imported when the package is imported are defined here. > Sometimes I'll import things from sub-modules from within __init__.py > and name and organize them. Sometimes submodules are left on their own > to be imported separately. Though the notation "from foo import *" is > discouraged, if a programmer is wont to do that, the package can limit > the damage somewhat by defining the __all__ variable. > > As I said, probably not looking correctly - looking under the wrong baskets. Tutorials is not where I usually look first - I like them better after I have done some basic reading to confirm and/or correct my expectations. From laborhm at gmail.com Tue Oct 11 17:25:18 2016 From: laborhm at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCb0L7Qs9GD0L3QvtCy?=) Date: Tue, 11 Oct 2016 14:25:18 -0700 (PDT) Subject: Python Shell in Russian exists/possible?? In-Reply-To: References: <61c6eb6b-59fb-4f09-a08f-ef0476fa0ee1@googlegroups.com> Message-ID: <45cc350f-c35a-44b5-9247-4bc48160d9ad@googlegroups.com> ?????, 12 ??????? 2016 ?., 4:29:02 UTC+10 ???????????? Michael Torrie ???????: > On 10/11/2016 05:33 AM, ?????? ??????? wrote: > > I need the Python Shell for use in education (turtle graphics, etc.), > > but the UI must be localized in the Russian language. The question is > > if it's at all possible to feed the strings in or rebuild it or... > > For educational purposes, you might find the Jupyter Notebook to be > worthwhile. It's the interactive Python interpreter but in a format you > can use from the browser and it acts like a notebook too, allowing > editing and actually building of code from interactive snippets. There > is some effort underway to build localization of language into it. > Unfortunately this effort is in its early stages. > > There's also a module someone wrote for it to let you do the turtle > graphics module inside the notebook. > > Anyway, if they had localization all finished, I'd say Jupyter Notebook > would be the way to go for your purposes. Thanks for the leads, B.R. From bc at freeuk.com Tue Oct 11 18:23:33 2016 From: bc at freeuk.com (BartC) Date: Tue, 11 Oct 2016 23:23:33 +0100 Subject: while loop (Reposting On Python-List Prohibited) In-Reply-To: <29beec92-3097-4f43-9c41-bb356302aa6a@googlegroups.com> References: <29beec92-3097-4f43-9c41-bb356302aa6a@googlegroups.com> Message-ID: On 11/10/2016 22:26, Lawrence D?Oliveiro wrote: > On Wednesday, October 12, 2016 at 6:58:46 AM UTC+13, dhawan... at gmail.com wrote: >> Only first loop is executing not the second one? > > n=6 > x=1 > while x<=n: > print("*"*x) > x+=1 > print('n=', n) > print('x=', x) > while n>=x: > n=n-1 > print("*"* n) > > * > ** > *** > **** > ***** > ****** > n= 6 > x= 7 > > Moral: always use debug print statements to figure out what is going on. > 'else' can be used here: n=6 x=1 while x<=n: print "*"*x x+=1 while n>=x: n=n-1 print "*"* n else: print ("2nd loop exit n=",n,"x=",x) -- Bartc From tjreedy at udel.edu Tue Oct 11 19:13:52 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Oct 2016 19:13:52 -0400 Subject: shadows name from outer scope In-Reply-To: References: Message-ID: On 10/11/2016 11:50 AM, Daiyue Weng wrote: > Hi, I have the following code structure, > > def one_func(param1, param2, param3): > > process_outcome = {'dict': None} > > > outcome = another_func(param1, process_outcome, db_host=param2, > funcs_to_run=param3) > > PyCharm warns me > > Shadows name from outer scope > > on every variable here, param1,2,3, process_outcome. Have you checked that this is true? Code checkers sometimes have bugs. In any case, we cannot comment on this without code. > I am wondering how to resolve the issues, Ignore the warning or turn it off. > and is it vital to have variables not shadowed? No, sometimes it is the right thing to do. Some warnings dependably indicate something wrong. Some do not and are just a hint that maybe you should think about what you did. Was it accidental (a typo) or intentional? -- Terry Jan Reedy From tjreedy at udel.edu Tue Oct 11 19:48:30 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Oct 2016 19:48:30 -0400 Subject: How to process syntax errors In-Reply-To: <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Message-ID: On 10/11/2016 4:02 AM, Pierre-Alain Dorange wrote: > Using this function, the code is "compiled". > I do not think this function is often used and most python project > simply use the interpreter (which do a small translation into byte-code > to be faster and check syntax error before running interpretation You seem to be confusing CPython with, for instance, simple BASIC interpreters that tokenized the code, translated keywords to function numbers, and did other 'small translations' before execution. The CPython compiler lexes (tokenizes), ll(1) parses to a syntax tree, does some analysis and transformation of the tree, and translates it to the bytecode for an stack machine. All done using standard compiler theory. One can even ask compile() to stop with the syntax tree and return that instead of a code object. One can then do one's own tree manipulation and even code generation. > So yes there is a way to check "syntax error" before executing code > (using compile function and exceptions) but it was not standard, > nor widely used... In CPython, the built-in Python-visible compile function *is* the CPython compile function used to compile *all* python code. Leaving out interaction with the OS and initialization, one could loosely define CPython as exec(compile(open('x.py').read())). -- Terry Jan Reedy From steve+python at pearwood.info Tue Oct 11 20:07:42 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 12 Oct 2016 11:07:42 +1100 Subject: Pip error on installing Python 3.5 Message-ID: <57fd7ed0$0$1619$c3e8da3$5496439d@news.astraweb.com> I've just installed Python 3.5 from the source tarball and received an unexpected error related to pip. On Linux, as a regular user (except for the last line): wget https://www.python.org/ftp/python/3.5.2/Python-3.5.2.tgz tar xvf Python-3.5.2.tgz cd Python-3.5.2 ./configure make sudo make altinstall There were no obvious errors until the end: The directory '/home/steve/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Ignoring indexes: https://pypi.python.org/simple The directory '/home/steve/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Can anyone explain what is going on, why I should care, and how I should fix it? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From skip.montanaro at gmail.com Tue Oct 11 20:59:59 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 11 Oct 2016 19:59:59 -0500 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: Message-ID: I'm trying to port some code from Python 2 to 3, and beyond the usual mechanical stuff, I've encountered a difference between the str() of floats. Here's an example. In Python 3 I get: >>> print(repr(27.04 - 0.01)) 27.029999999999998 >>> print(str(27.04 - 0.01)) 27.029999999999998 but in Python 2 str() behaves differently: >>> print repr(27.04 - 0.01) 27.029999999999998 >>> print str(27.04 - 0.01) 27.03 My test case writes through a csv writer, which writes the str() of each element to the output. My test assertions are looking at the output, not the numeric input, so the option to test if the number is "close enough" isn't readily available. I suppose I should adjust my test case, as in this case 27.04-0.01 is different than 27.03. Is there documentation of this particular change? My searching turned up documentation of plenty of other changes, but not this particular one. Thx, Skip From ethan at stoneleaf.us Tue Oct 11 21:11:51 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 11 Oct 2016 18:11:51 -0700 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: Message-ID: <57FD8DD7.80003@stoneleaf.us> On 10/11/2016 05:59 PM, Skip Montanaro wrote: > Is there documentation of this particular change? My searching turned up > documentation of plenty of other changes, but not this particular one. 3.1 What's new: https://docs.python.org/3/whatsnew/3.1.html It's the third hit when searching for 'float'. Assuming I understand what it's saying. ;) -- ~Ethan~ From ben+python at benfinney.id.au Tue Oct 11 21:17:04 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 12 Oct 2016 12:17:04 +1100 Subject: repr/str diff between Python 2 and 3 References: Message-ID: <857f9emkmn.fsf@benfinney.id.au> Skip Montanaro writes: > >>> print repr(27.04 - 0.01) > 27.029999999999998 > >>> print str(27.04 - 0.01) > 27.03 > > My test case writes through a csv writer, which writes the str() of each > element to the output. For Python 2, that's a mistake: str(object='') Return a string containing a nicely printable representation of an object. [?] The difference with repr(object) is that str(object) does not always attempt to return a string that is acceptable to eval(); its goal is to return a printable string. [?] So, for preserving the value, you don't want ?str(obj)?. That's what ?repr(obj)? is for: repr(object) Return a string containing a printable representation of an object. [?] For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval() [?] > Is there documentation of this particular change? My searching turned > up documentation of plenty of other changes, but not this particular > one. Only that one should not rely on ?str? preserving the value accurately, as documented in Python 2. -- \ ?[Entrenched media corporations will] maintain the status quo, | `\ or die trying. Either is better than actually WORKING for a | _o__) living.? ?ringsnake.livejournal.com, 2007-11-12 | Ben Finney From skip.montanaro at gmail.com Tue Oct 11 21:30:43 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 11 Oct 2016 20:30:43 -0500 Subject: repr/str diff between Python 2 and 3 In-Reply-To: <57FD8DD7.80003@stoneleaf.us> References: <57FD8DD7.80003@stoneleaf.us> Message-ID: > https://docs.python.org/3/whatsnew/3.1.html > > It's the third hit when searching for 'float'. Assuming I understand what it's saying. ;) Thanks. Is that the "David Gay's algorithm"? That seems to apply only to repr(), while the change I observed was in str(). Skip From skip.montanaro at gmail.com Tue Oct 11 21:46:30 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 11 Oct 2016 20:46:30 -0500 Subject: repr/str diff between Python 2 and 3 In-Reply-To: <857f9emkmn.fsf@benfinney.id.au> References: <857f9emkmn.fsf@benfinney.id.au> Message-ID: > Only that one should not rely on ?str? preserving the value accurately, > as documented in Python 2. Sure, but this choice is out of my hands. It's csv.writerow that calls str(), not me. I could probably subclass csv.writer and csv.DictWriter, and override the writerow method, but I would prefer not to if I can avoid it. I could also propose a change to the writerow method, but - even if accepted - it wouldn't change earlier 3.x behavior. Skip > -- > \ ?[Entrenched media corporations will] maintain the status quo, | > `\ or die trying. Either is better than actually WORKING for a | > _o__) living.? ?ringsnake.livejournal.com, 2007-11-12 | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list From tjreedy at udel.edu Tue Oct 11 21:46:40 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Oct 2016 21:46:40 -0400 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: Message-ID: On 10/11/2016 8:59 PM, Skip Montanaro wrote: > I'm trying to port some code from Python 2 to 3, and beyond the usual > mechanical stuff, I've encountered a difference between the str() of > floats. Here's an example. In Python 3 I get: > >>>> print(repr(27.04 - 0.01)) > 27.029999999999998 >>>> print(str(27.04 - 0.01)) > 27.029999999999998 > > but in Python 2 str() behaves differently: > >>>> print repr(27.04 - 0.01) > 27.029999999999998 >>>> print str(27.04 - 0.01) > 27.03 > > My test case writes through a csv writer, which writes the str() of each > element to the output. My test assertions are looking at the output, not > the numeric input, so the option to test if the number is "close enough" > isn't readily available. I suppose I should adjust my test case, as in this > case 27.04-0.01 is different than 27.03. Or you can make the text version-specific. Basing tests on strings generated by Python is fragile because they are subject to change. For instance, 3.1 has a new algorithm for repr(float), discussed in What's New in 3.1. The repr for other objects can and does change with new versions, as to exception messages. (They are only changed in bugfix releases if there is a real and significant but.) > Is there documentation of this particular change? My searching turned up > documentation of plenty of other changes, but not this particular one. There was discussed on the development lists. I believe this change was in 3.0, in which case there would be no mention in the 3.x docs. I did not see mention in What New in 3.0, though. The change would certainly be in the NEWS file that lists all commits. This is now reformatted into an html changelog linked from What's New, but this started after in 3.0. -- Terry Jan Reedy From ben+python at benfinney.id.au Tue Oct 11 22:01:48 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 12 Oct 2016 13:01:48 +1100 Subject: repr/str diff between Python 2 and 3 References: <857f9emkmn.fsf@benfinney.id.au> Message-ID: <8537k2mik3.fsf@benfinney.id.au> Skip Montanaro writes: > > Only that one should not rely on ?str? preserving the value accurately, > > as documented in Python 2. > > Sure, but this choice is out of my hands. It's csv.writerow that calls > str(), not me. Ah, good old ?csv?. If the implementation is leaking an abstraction and you can't change the implementation, then yes, making your test cases aware of the difference is an option you'll need to consider. -- \ ?Do I believe in God? ? without clarification of a kind I have | `\ never seen, I don?t know whether I believe or don?t believe in | _o__) whatever a questioner has in mind.? ?Noam Chomsky | Ben Finney From jfong at ms4.hinet.net Tue Oct 11 22:36:41 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Tue, 11 Oct 2016 19:36:41 -0700 (PDT) Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: References: Message-ID: <93b381d3-25e4-4f30-819d-faa0ca17e17a@googlegroups.com> Hi, eryk, thanks for your solution. I had try to find the document of the _winapi module, but can't find any in my installed Python directory. Can you give me a link to look for? > This alone doesn't make the Windows API case sensitive, but it does > enable individual CreateFile calls to be case sensitive, via the POSIX > semantics flag. Does it means that an application program which uses the POSIX semantics flag in its API call will perform case-sensitive automatically after the "obcaseinsensitive" flag was modified by the user? --Jach From jfong at ms4.hinet.net Tue Oct 11 22:39:14 2016 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Tue, 11 Oct 2016 19:39:14 -0700 (PDT) Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: <3fe59d08-2e98-4af4-9a86-32a345bf27d7@googlegroups.com> References: <3fe59d08-2e98-4af4-9a86-32a345bf27d7@googlegroups.com> Message-ID: <3383991f-8502-44fe-a3da-1f281482f25e@googlegroups.com> wxjm... at gmail.com at 2016/10/11 9:40:21PM wrote: > If you are about to modify your registry, do not > forget to switch your Windows in a *utf-8 mode*. Have no idea how to "switch" Windows in a "utf-8 mode"? What will happens if not? Can you give a simple example? Thanks ahead. --Jach From ethan at stoneleaf.us Tue Oct 11 23:08:08 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 11 Oct 2016 20:08:08 -0700 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: <57FD8DD7.80003@stoneleaf.us> Message-ID: <57FDA918.1050604@stoneleaf.us> On 10/11/2016 06:30 PM, Skip Montanaro wrote: >>https://docs.python.org/3/whatsnew/3.1.html >> >> It's the third hit when searching for 'float'. Assuming I understand what it's saying. ;) > > Thanks. Is that the "David Gay's algorithm"? That seems to apply only to repr(), while the change I observed was in str(). But if __str__ is not defined, __repr__ is used. And I believe that is the case with float. Too lazy to check, though. ;) -- ~Ethan~ From tjreedy at udel.edu Tue Oct 11 23:15:15 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 11 Oct 2016 23:15:15 -0400 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: <57FD8DD7.80003@stoneleaf.us> Message-ID: On 10/11/2016 9:30 PM, Skip Montanaro wrote: >> https://docs.python.org/3/whatsnew/3.1.html >> >> It's the third hit when searching for 'float'. Assuming I understand > what it's saying. ;) > > Thanks. Is that the "David Gay's algorithm"? That seems to apply only to > repr(), while the change I observed was in str(). Since str(float) is now, I believe, the same as repr(float), it means that in general, you have two possible changes to str ;-). -- Terry Jan Reedy From steve+comp.lang.python at pearwood.info Tue Oct 11 23:19:57 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 12 Oct 2016 14:19:57 +1100 Subject: OFF TOPIC mov is Turing complete Message-ID: <57fdabde$0$11091$c3e8da3@news.astraweb.com> Completely off-topic, but too awesome not to share: The x86 assembly language "mov" instruction is Turing complete: https://www.cl.cam.ac.uk/~sd601/papers/mov.pdf Abstract -------- It is well-known that the x86 instruction set is baroque, overcom- plicated, and redundantly redundant. We show just how much fluff it has by demonstrating that it remains Turing-complete when re- duced to just one instruction. The instruction we choose is mov, which can do both loads and stores. We use no unusual addressing modes, self-modifying code, or runtime code generation. Using just this instruction (and a single unconditional branch at the end of the program to make nontermination possible), we demonstrate how an arbitrary Turing machine can be simulated. -- Steven git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space. From eryksun at gmail.com Wed Oct 12 00:14:27 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 12 Oct 2016 04:14:27 +0000 Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: <93b381d3-25e4-4f30-819d-faa0ca17e17a@googlegroups.com> References: <93b381d3-25e4-4f30-819d-faa0ca17e17a@googlegroups.com> Message-ID: On Wed, Oct 12, 2016 at 2:36 AM, wrote: > > I had try to find the document of the _winapi module, but can't find any in my > installed Python directory. Can you give me a link to look for? _winapi is not documented. It contains light wrappers around a small set of Windows functions. It's ok for quick scripts if it has what you need. But for everything else you should prefer the publicly documented PyWin32 package, ctypes, or CFFI. >> This alone doesn't make the Windows API case sensitive, but it does >> enable individual CreateFile calls to be case sensitive, via the POSIX >> semantics flag. > > Does it means that an application program which uses the POSIX semantics flag in its > API call will perform case-sensitive automatically after the "obcaseinsensitive" flag was > modified by the user? FILE_FLAG_POSIX_SEMANTICS has to be specified in every CreateFile call. The real system call is NtCreateFile. Like all other named objects in the NT kernel, a file path is passed to NtCreateFile in a OBJECT_ATTRIBUTES structure, which can be flagged as OBJ_CASE_INSENSITIVE. When WinAPI CreateFile calls NtCreateFile, it almost always uses the OBJ_CASE_INSENSITIVE flag. The exception is when the caller requests FILE_FLAG_POSIX_SEMANTICS. However, the latter is no longer enough in Windows XP and later (NT 5.1+), for which the default was switched to case insensitive for all object manager and I/O manager objects such as devices and files. To get the old behavior back, and thus fix the now broken FILE_FLAG_POSIX_SEMANTICS flag, you have to disable the ObCaseInsensitive setting, as I showed. This modification is best for administrative and short-term use, or for systems running an NFS server. As a long-term setting it may be problematic. Notably SQL Server and the Windows 10 Subsystem for Linux (WSL) don't work properly if ObCaseInsensitive is disabled. From torriem at gmail.com Wed Oct 12 00:17:07 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 11 Oct 2016 22:17:07 -0600 Subject: OFF TOPIC mov is Turing complete In-Reply-To: <57fdabde$0$11091$c3e8da3@news.astraweb.com> References: <57fdabde$0$11091$c3e8da3@news.astraweb.com> Message-ID: <0e8ed86c-97f4-9e2f-b002-58f3ea0bd388@gmail.com> On 10/11/2016 09:19 PM, Steven D'Aprano wrote: > Completely off-topic, but too awesome not to share: > > The x86 assembly language "mov" instruction is Turing complete: > > https://www.cl.cam.ac.uk/~sd601/papers/mov.pdf And apparently someone has taken this and made a small C compiler that compiles to nothing but mov instructions. Amazing: https://github.com/xoreaxeaxeax/movfuscator Now I'm sure malware writers will take full advantage of this to make reverse-engineering of their code difficult! From greg.ewing at canterbury.ac.nz Wed Oct 12 00:42:05 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 12 Oct 2016 17:42:05 +1300 Subject: Is it possible Python can distinguish capital letter and small letter in the path of Windows? In-Reply-To: References: <4fkpvbtb9uauoc4chl2uqjfo8i6cciccef@4ax.com> <36uqvb10efntpqacjv5o9sf89vp466ue5m@4ax.com> Message-ID: Dennis Lee Bieber wrote: > VMS documentation set was around 60 linear inches (two 30" shelves) of > 3-ring binders. Amiga RKMs were five volumes with the main documentation in > fine print -- two "pages" side-by-side on a landscape page. Apparently if you're writing software for the US Air Force, minimum documentation requirements are measured in metres. Although it seems it doesn't matter much whether all of those metres are actually useful... http://thedailywtf.com/articles/Very,_Very_Well_Documented -- Greg From zachary.ware+pylist at gmail.com Wed Oct 12 00:45:36 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Tue, 11 Oct 2016 23:45:36 -0500 Subject: Pip error on installing Python 3.5 In-Reply-To: <57fd7ed0$0$1619$c3e8da3$5496439d@news.astraweb.com> References: <57fd7ed0$0$1619$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Tue, Oct 11, 2016 at 7:07 PM, Steve D'Aprano wrote: > I've just installed Python 3.5 from the source tarball and received an > unexpected error related to pip. On Linux, as a regular user (except for > the last line): > > > wget https://www.python.org/ftp/python/3.5.2/Python-3.5.2.tgz > tar xvf Python-3.5.2.tgz > cd Python-3.5.2 > ./configure > make > sudo make altinstall > > > There were no obvious errors until the end: > > The directory '/home/steve/.cache/pip/http' or its parent directory is not > owned by the current user and the cache has been disabled. Please check the > permissions and owner of that directory. If executing pip with sudo, you > may want sudo's -H flag. > > Ignoring indexes: https://pypi.python.org/simple > > The directory '/home/steve/.cache/pip' or its parent directory is not owned > by the current user and caching wheels has been disabled. check the > permissions and owner of that directory. If executing pip with sudo, you > may want sudo's -H flag. > > > > Can anyone explain what is going on, why I should care, and how I should fix > it? Those aren't errors, and pip should be perfectly usable. The message about `/home/steve/.cache/pip` is just because pip is being run via sudo; pip's cache dir is based on $HOME, which is still set to `/home/steve` and pip doesn't want to screw up its cache for the `steve` user. The "Ignoring indexes" message is due to ensurepip explicitly disabling pulling from PyPI when installing pip and setuptools. The messages about the cache do look a bit unnecessarily scary, it might be nice if they didn't appear when pip is run by ensurepip. I'm not sure what removing the message for that particular case might entail, but it might be worth opening an issue for it. -- Zach From greg.ewing at canterbury.ac.nz Wed Oct 12 01:09:34 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 12 Oct 2016 18:09:34 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> Message-ID: Anuradha Laxminarayan wrote: > seq f g h = f (\s1 -> g h s1) > > better be written as > > seq f g x = f (\s1 -> g x s1) > > because naming conventions imply that h is function. Well, for the subset of monads I'm talking about, it always is a function -- it's the continuation to be run after f and g. -- Greg From greg.ewing at canterbury.ac.nz Wed Oct 12 01:14:59 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 12 Oct 2016 18:14:59 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <87k2dest3b.fsf@nightsong.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <35b23711-dafd-4b43-a957-9399d3e3be0e@googlegroups.com> <87k2dest3b.fsf@nightsong.com> Message-ID: Paul Rubin wrote: > Also if this operation is what it looks like, it's usually called > "bind". seq is something else entirely. Ah, I hadn't realised there was already a function in Haskell called seq -- sorry about that! I don't really want to call the Python version 'bind', because it seems a bit obscure. Also, the way I'm using it there, it's not actually binding anything (it corresponds to >> rather than >>=), so calling it 'bind' would just confuse matters. I'll see if I can come up with a better name for it that doesn't clash with anything. -- Greg From storchaka at gmail.com Wed Oct 12 02:26:43 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Wed, 12 Oct 2016 09:26:43 +0300 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: <57FD8DD7.80003@stoneleaf.us> Message-ID: On 12.10.16 04:30, Skip Montanaro wrote: >> https://docs.python.org/3/whatsnew/3.1.html >> >> It's the third hit when searching for 'float'. Assuming I understand > what it's saying. ;) > > Thanks. Is that the "David Gay's algorithm"? That seems to apply only to > repr(), while the change I observed was in str(). In Python 2 str() for floats was equivalent to '%.12g' % x, and repr() -- to '%.17g' % x. In Python 3.0 str() becomes the same as repr() (this added too much noise). In Python 3.1 repr() (and therefore str()) was changed to produce more compact representation if possible. This change was backported to 2.7, but str() left different. From steve+comp.lang.python at pearwood.info Wed Oct 12 04:29:04 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 12 Oct 2016 19:29:04 +1100 Subject: Python-based monads essay (Re: Assignment versus binding) References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <50a8d925-2bf7-4e34-ac92-d3288f529b6b@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> Message-ID: <57fdf452$0$1532$c3e8da3$5496439d@news.astraweb.com> On Sunday 09 October 2016 18:48, Gregory Ewing wrote: > Here's the first part of the essay I said I'd write about > monads: > > http://www.cosc.canterbury.ac.nz/greg.ewing/essays/monads/DemystifyingMonads.html Quoting from the essay: "the implementation is free to use in-place mutations of the state object ? which obviously allows considerable gains in efficiency, both in time and memory usage ? without giving up any functional purity." Surely that should be as follows? "the implementation is free to use in-place mutations of the state object ? which obviously allows considerable gains in efficiency, both in time and memory usage ? without letting anyone know that the implementation has given up any functional purity." I'm inclined to agree with James Hague (via John Cook): functional programming is great, until you try to be strict about it. The closer you get to 100% functional, the less you can use it. He reckons the sweet spot is about 85% functional: http://www.johndcook.com/blog/2010/04/15/85-functional-language-purity/ -- Steven git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space. From jobmattcon at gmail.com Wed Oct 12 04:47:14 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 12 Oct 2016 01:47:14 -0700 (PDT) Subject: why solveset and dsolve do not have _fields for ast.walk? Message-ID: Traceback (most recent call last): File "astgraphforsympy.py", line 86, in print get_func_calls(tree) File "astgraphforsympy.py", line 48, in get_func_calls for node in ast.walk(tree): File "C:\Python27\lib\ast.py", line 213, in walk todo.extend(iter_child_nodes(node)) File "C:\Python27\lib\ast.py", line 178, in iter_child_nodes for name, field in iter_fields(node): File "C:\Python27\lib\ast.py", line 166, in iter_fields for field in node._fields: AttributeError: Module instance has no attribute '_fields' From jobmattcon at gmail.com Wed Oct 12 05:32:58 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 12 Oct 2016 02:32:58 -0700 (PDT) Subject: how to evaluate a ast tree and change Add to Multiply ? Message-ID: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> import sys import parser import ast from collections import deque class ChangeAddtoMultiply(ast.NodeTransformer): """Wraps all integers in a call to Integer()""" def visit_Num(self, node): if isinstance(node.n, int): return ast.Call(func=ast.Name(id='Add', ctx=ast.Load()), args=[node], keywords=[]) return node tree = ast.parse('2+5') for node in ast.walk(tree): print(str(node)) eval(tree) From rosuav at gmail.com Wed Oct 12 05:46:56 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 12 Oct 2016 20:46:56 +1100 Subject: how to evaluate a ast tree and change Add to Multiply ? In-Reply-To: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> References: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> Message-ID: On Wed, Oct 12, 2016 at 8:32 PM, meInvent bbird wrote: > class ChangeAddtoMultiply(ast.NodeTransformer): > """Wraps all integers in a call to Integer()""" > def visit_Num(self, node): > if isinstance(node.n, int): > return ast.Call(func=ast.Name(id='Add', ctx=ast.Load()), > args=[node], keywords=[]) > return node > > Your class name, docstring, and functionality are all different, and you have given no explanation of what you want us to do. I can't read minds quite that well. ChrisA From jobmattcon at gmail.com Wed Oct 12 05:51:01 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 12 Oct 2016 02:51:01 -0700 (PDT) Subject: how to faster to know which ast name or id we want to change? Message-ID: <30de1d45-e87e-4329-a708-2c309fb98249@googlegroups.com> how to faster to know which ast name or id we want to change? because i use ast.walk to print all nodes then i know the name + is Add if using cparser to parse linux kernel, it will be a very large From bc at freeuk.com Wed Oct 12 05:53:28 2016 From: bc at freeuk.com (BartC) Date: Wed, 12 Oct 2016 10:53:28 +0100 Subject: while loop (Reposting On Python-List Prohibited) In-Reply-To: <5d3512a6-c921-4317-b942-166df17ff93c@googlegroups.com> References: <29beec92-3097-4f43-9c41-bb356302aa6a@googlegroups.com> <5d3512a6-c921-4317-b942-166df17ff93c@googlegroups.com> Message-ID: On 12/10/2016 05:30, Lawrence D?Oliveiro wrote: > On Wednesday, October 12, 2016 at 11:23:48 AM UTC+13, BartC wrote: >> while n>=x: >> n=n-1 >> print "*"* n >> else: >> print ("2nd loop exit n=",n,"x=",x) > > What is the difference between that and > > while n>=x: > n=n-1 > print "*"* n > print ("2nd loop exit n=",n,"x=",x) > > ? > > None at all. > Not so much in this specific example: that message will be shown whether there have been 0 or more iterations of the loop body. But with 'else', if you see the message it means the while statement has been entered. Here: if cond: while n>=x: n=n-1 print "*"* n else: print ("2nd loop exit n=",n,"x=",x) when cond is false, nothing will be printed. You then know the while statement hasn't been entered, so it's not looping for some other reason than its loop condition being false from the start. Another situation is when the loop body contains 'break'; then it will bypass the 'else' part. -- Bartc From jobmattcon at gmail.com Wed Oct 12 05:53:33 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 12 Oct 2016 02:53:33 -0700 (PDT) Subject: how to evaluate a ast tree and change Add to Multiply ? In-Reply-To: References: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> Message-ID: i use example here http://greentreesnakes.readthedocs.io/en/latest/examples.html On Wednesday, October 12, 2016 at 5:47:12 PM UTC+8, Chris Angelico wrote: > On Wed, Oct 12, 2016 at 8:32 PM, meInvent bbird wrote: > > class ChangeAddtoMultiply(ast.NodeTransformer): > > """Wraps all integers in a call to Integer()""" > > def visit_Num(self, node): > > if isinstance(node.n, int): > > return ast.Call(func=ast.Name(id='Add', ctx=ast.Load()), > > args=[node], keywords=[]) > > return node > > > > > > Your class name, docstring, and functionality are all different, and > you have given no explanation of what you want us to do. I can't read > minds quite that well. > > ChrisA From jobmattcon at gmail.com Wed Oct 12 05:55:54 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Wed, 12 Oct 2016 02:55:54 -0700 (PDT) Subject: how to evaluate a ast tree and change Add to Multiply ? In-Reply-To: References: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> Message-ID: <2d3bf63e-9638-4e33-a50a-01a3face8b65@googlegroups.com> i just expect to rewrite + become multiply by edit the example in the link provided but search no examples about this, feel unknown about args and keywords etc, do not know how to write this ast.Call(func=ast.Name(id='Add', ctx=ast.Load()), args=[node], keywords=[]) On Wednesday, October 12, 2016 at 5:53:42 PM UTC+8, meInvent bbird wrote: > i use example here > > http://greentreesnakes.readthedocs.io/en/latest/examples.html > > > > > On Wednesday, October 12, 2016 at 5:47:12 PM UTC+8, Chris Angelico wrote: > > On Wed, Oct 12, 2016 at 8:32 PM, meInvent bbird wrote: > > > class ChangeAddtoMultiply(ast.NodeTransformer): > > > """Wraps all integers in a call to Integer()""" > > > def visit_Num(self, node): > > > if isinstance(node.n, int): > > > return ast.Call(func=ast.Name(id='Add', ctx=ast.Load()), > > > args=[node], keywords=[]) > > > return node > > > > > > > > > > Your class name, docstring, and functionality are all different, and > > you have given no explanation of what you want us to do. I can't read > > minds quite that well. > > > > ChrisA From mr.puneet.goyal at gmail.com Wed Oct 12 05:59:59 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Wed, 12 Oct 2016 02:59:59 -0700 (PDT) Subject: How to process syntax errors In-Reply-To: References: Message-ID: Hi All Its really good to see that some discussion happening around this topic. Sorry I was out from my work for sometime so couldn't follow up but I really find it useful. It gives me good opportunity to know python better as I recently started learning python. Ok so I tell you why I need to catch syntax error during compilation and process them. See below example, # I created a platform class with different method in a file and making it as a package. class platform: def connect(self): # connect device def destroy(self): # destroy device def config(self, command): # Send command to configure device def show(self, command): # check device health Now person who wants to write a script using above package can simply use below approach. Which does not make him to have knowledge in python. DUT = platform() DUT connect DUT config {commands} DUT show {commands} DUT destroy But I know this is not easy to do in python. As they all invalid syntax for python language. So I thought of grabing syntax errors and manipulate them as below internally to call them while code is compiling. DUT = platform() DUT.connect() DUT.config(commands) DUT.show(commands) DUT.destroy() Hope you understand my need of doing this. If there is another solution to achieve what I am trying to. Please let me know. Thanks, Puneet From mr.puneet.goyal at gmail.com Wed Oct 12 06:08:43 2016 From: mr.puneet.goyal at gmail.com (mr.puneet.goyal at gmail.com) Date: Wed, 12 Oct 2016 03:08:43 -0700 (PDT) Subject: How to process syntax errors In-Reply-To: References: Message-ID: On Monday, October 10, 2016 at 7:45:40 PM UTC+5:30, mr.pune... at gmail.com wrote: > Hi > > Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. > > For example, > > inside example.py > > I just mention below line > > > Obj = myClass() > Obj xyz > > Obj is instance of a class. But there is syntax error on xyz. So I want to grab that error and process. > > Regards, Puneet Steve, You are absolutely right. I am trying to eliminate the method of using parenthesis while calling in my file. Especially when I call it from a instance. From __peter__ at web.de Wed Oct 12 06:15:56 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 12 Oct 2016 12:15:56 +0200 Subject: while loop (Reposting On Python-List Prohibited) References: <29beec92-3097-4f43-9c41-bb356302aa6a@googlegroups.com> <5d3512a6-c921-4317-b942-166df17ff93c@googlegroups.com> Message-ID: BartC wrote: > On 12/10/2016 05:30, Lawrence D?Oliveiro wrote: >> On Wednesday, October 12, 2016 at 11:23:48 AM UTC+13, BartC wrote: >>> while n>=x: >>> n=n-1 >>> print "*"* n >>> else: >>> print ("2nd loop exit n=",n,"x=",x) >> >> What is the difference between that and >> >> while n>=x: >> n=n-1 >> print "*"* n >> print ("2nd loop exit n=",n,"x=",x) >> >> ? >> >> None at all. >> > > Not so much in this specific example: that message will be shown whether > there have been 0 or more iterations of the loop body. > > But with 'else', if you see the message it means the while statement has > been entered. Here: > > if cond: > while n>=x: > n=n-1 > print "*"* n > else: > print ("2nd loop exit n=",n,"x=",x) Lawrence is right. The enclosing if doesn't make a difference. > when cond is false, nothing will be printed. You then know the while > statement hasn't been entered, so it's not looping for some other reason > than its loop condition being false from the start. > > Another situation is when the loop body contains 'break'; then it will > bypass the 'else' part. This is the only case where while...else makes sense: the code in the else suite is executed if and only if the break was not reached. From skip.montanaro at gmail.com Wed Oct 12 06:26:21 2016 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 12 Oct 2016 05:26:21 -0500 Subject: repr/str diff between Python 2 and 3 In-Reply-To: References: <57FD8DD7.80003@stoneleaf.us> Message-ID: Thanks everyone. I'm not going to try to be too cute, and will just change my test case. I'm leaving Python 2 behind in this particular case for now anyway. I can always return to the issue if I decide I need Python 2.7 support at some point in the future. Skip From bc at freeuk.com Wed Oct 12 06:53:24 2016 From: bc at freeuk.com (BartC) Date: Wed, 12 Oct 2016 11:53:24 +0100 Subject: while loop (Reposting On Python-List Prohibited) In-Reply-To: References: <29beec92-3097-4f43-9c41-bb356302aa6a@googlegroups.com> <5d3512a6-c921-4317-b942-166df17ff93c@googlegroups.com> Message-ID: On 12/10/2016 11:15, Peter Otten wrote: > BartC wrote: > >> On 12/10/2016 05:30, Lawrence D?Oliveiro wrote: >>> On Wednesday, October 12, 2016 at 11:23:48 AM UTC+13, BartC wrote: >>>> while n>=x: >>>> n=n-1 >>>> print "*"* n >>>> else: >>>> print ("2nd loop exit n=",n,"x=",x) >>> >>> What is the difference between that and >>> >>> while n>=x: >>> n=n-1 >>> print "*"* n >>> print ("2nd loop exit n=",n,"x=",x) >>> >>> ? >>> >>> None at all. >>> >> >> Not so much in this specific example: that message will be shown whether >> there have been 0 or more iterations of the loop body. >> >> But with 'else', if you see the message it means the while statement has >> been entered. Here: >> >> if cond: >> while n>=x: >> n=n-1 >> print "*"* n >> else: >> print ("2nd loop exit n=",n,"x=",x) > > Lawrence is right. The enclosing if doesn't make a difference. The idea is to detect whether the while loop has been entered. With while-else-print, it will always execute the else (assuming no break). With while then print, you can't tell if it has attempted to or not. My example above has wrapped the if-cond around the whole of while-else because it has to (that's the advantage). With a separate print it need not do that: if cond: while n>=x: n=n-1 print "*"* n print ("2nd loop exit n=",n,"x=",x) With real code it may not be as easy to see. 'else' adds structure. -- bartc From robin at reportlab.com Wed Oct 12 06:56:21 2016 From: robin at reportlab.com (Robin Becker) Date: Wed, 12 Oct 2016 11:56:21 +0100 Subject: [RELEASE] Python 3.6.0b2 is now available In-Reply-To: <4E66D65C-0EFF-41B9-9B5E-1B3FF60151B4@python.org> References: <4E66D65C-0EFF-41B9-9B5E-1B3FF60151B4@python.org> Message-ID: <1704f083-617a-ecec-bc79-c0ba85189bd1@chamonix.reportlab.co.uk> On 11/10/2016 02:12, Ned Deily wrote: > On behalf of the Python development community and the Python 3.6 release > team, I'm happy to announce the availability of Python 3.6.0b2. 3.6.0b2 > is the second of four planned beta releases of Python 3.6, the next major > release of Python, and marks the end of the feature development phase > for 3.6. ....... thanks > -- > Ned Deily > nad at python.org -- [] > I notice an extra space at the windows command prompt compared with 3.5, is that deliberate? > C:\ux\XB33>\python35\python > Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> > C:\ux\XB33\repos\pyRXP>..\..\py36_amd64\Scripts\python.exe > Python 3.6.0b2 (default, Oct 10 2016, 21:15:32) [MSC v.1900 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys -- Robin Becker From steve+python at pearwood.info Wed Oct 12 07:02:29 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 12 Oct 2016 22:02:29 +1100 Subject: How to process syntax errors References: Message-ID: <57fe1847$0$1601$c3e8da3$5496439d@news.astraweb.com> On Wed, 12 Oct 2016 08:59 pm, mr.puneet.goyal at gmail.com wrote: > Now person who wants to write a script using above package can simply use > below approach. Which does not make him to have knowledge in python. > > DUT = platform() > DUT connect > DUT config {commands} > DUT show {commands} > DUT destroy > > > But I know this is not easy to do in python. This is *impossible* to do in Python. You cannot expect to write something in a language which is not Python, and have the Python interpreter run it as Python code. Either your users have to write Python code, or you have to write your own interpreter for whatever language it is that they wish to write. The interpreter doesn't need to be very complicated, and it can be written in Python. The Python standard library includes support for writing *interactive* command interpreters: https://docs.python.org/3/library/cmd.html You can study that to get some hints for writing a non-interactive interpreter. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From nomail at com.invalid Wed Oct 12 07:18:16 2016 From: nomail at com.invalid (ast) Date: Wed, 12 Oct 2016 13:18:16 +0200 Subject: Local variables to a function doesn't disapear after function execution. Why ? Message-ID: <57fe1bfe$0$5287$426a74cc@news.free.fr> Hello, here is the small program: from tkinter import * class Test: def __init__(self): root = Tk() label = Label(root, text="this is a test") label.pack() root.mainloop() test=Test() I dont understand why this program works. After execution of function __init__, local variables 'root' and 'label' should disapear and the widgets Tk and Label should be garbage collected. But it's not the case, the window Tk is still on the screen with the label inside it. Why ? Usually, we write: class Test: def __init__(self): self.root = Tk() self.label = Label(self.root, text="this is a test") self.label.pack() self.root.mainloop() test = Test() to keep some references to Tk() and Label() From auriocus at gmx.de Wed Oct 12 07:25:28 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 12 Oct 2016 13:25:28 +0200 Subject: Local variables to a function doesn't disapear after function execution. Why ? In-Reply-To: <57fe1bfe$0$5287$426a74cc@news.free.fr> References: <57fe1bfe$0$5287$426a74cc@news.free.fr> Message-ID: Am 12.10.16 um 13:18 schrieb ast: > Hello, here is the small program: > > from tkinter import * > > class Test: > def __init__(self): > root = Tk() > label = Label(root, text="this is a test") > label.pack() > root.mainloop() > > test=Test() > > I dont understand why this program works. After execution > of function __init__, local variables 'root' and 'label' should disapear > and the widgets Tk and Label should be garbage collected. But it's not > the case, the window Tk is still on the screen with the label inside it. > Why ? Because the Test() call does never terminate. You have the mainloop inside of your constructor. As long as this loop runs, your program exists. Try it by putting test=Test() print("Now I am here") and you'll see. Christian From K.Hoening at gmx.net Wed Oct 12 08:24:46 2016 From: K.Hoening at gmx.net (Karlheinz Hoening) Date: Wed, 12 Oct 2016 14:24:46 +0200 Subject: an other issue when installing python under Windows OS Message-ID: Hello all, on my iMac I have installed Windows (7) and I am now trying to install Python 3.5.2 (32-bit) under Windows OS. Every time after having 'repaired' with the installation procedere I receive the following message when starting Python 3.5: ---------------------------------------------------------------------------------------------------------------------------- python.exe - Sytemfehler Das Programm kann nicht gestartet werden, da api-ms-win-crt-I1-1-0.dll auf dem Computer fehlt. Installieren Sie das Programm erneut ... ---------------------------------------------------------------------------------------------------------------------------- Can you give any hint (support)? Where can I find the above mentionned driver? Thanks a lot in advance. cheers Kalle (k.hoening at gmx.net) From nomail at com.invalid Wed Oct 12 08:27:58 2016 From: nomail at com.invalid (ast) Date: Wed, 12 Oct 2016 14:27:58 +0200 Subject: Local variables to a function doesn't disapear after function execution. Why ? In-Reply-To: References: <57fe1bfe$0$5287$426a74cc@news.free.fr> Message-ID: <57fe2c55$0$4280$426a34cc@news.free.fr> "Christian Gollwitzer" a ?crit dans le message de news:ntl6in$on5$1 at dont-email.me... > Am 12.10.16 um 13:18 schrieb ast: > Because the Test() call does never terminate. You have the mainloop inside of your constructor. As > long as this loop runs, your program exists. Try it by putting > > test=Test() > print("Now I am here") > > and you'll see. > > Christian Ah yes, mainloop() is an endless loop. Thanks From greg.ewing at canterbury.ac.nz Wed Oct 12 08:52:50 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 13 Oct 2016 01:52:50 +1300 Subject: Python-based monads essay (Re: Assignment versus binding) In-Reply-To: <57fdf452$0$1532$c3e8da3$5496439d@news.astraweb.com> References: <788a06d3-ee8c-460c-8ab6-1d72da300a38@googlegroups.com> <57ef9ec3$0$1593$c3e8da3$5496439d@news.astraweb.com> <57f08b51$0$1618$c3e8da3$5496439d@news.astraweb.com> <57f1b734$0$1585$c3e8da3$5496439d@news.astraweb.com> <57f316da$0$1584$c3e8da3$5496439d@news.astraweb.com> <87k2doeg74.fsf@handshake.de> <87a8ekqzp5.fsf@elektro.pacujo.net> <87a8ek2taf.fsf@bsb.me.uk> <57f3dc6b$0$1589$c3e8da3$5496439d@news.astraweb.com> <87bmz0c4md.fsf@elektro.pacujo.net> <57F9F63D.8080300@canterbury.ac.nz> <57fdf452$0$1532$c3e8da3$5496439d@news.astraweb.com> Message-ID: Steven D'Aprano wrote: > "the implementation is free to use in-place mutations of the state object ? > ... without letting anyone know that the implementation has given up > any functional purity." If it's impossible to tell that functional purity has been given up, then in what sense has it been given up at all? -- Greg From vlastimil.brom at gmail.com Wed Oct 12 09:04:27 2016 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Wed, 12 Oct 2016 15:04:27 +0200 Subject: an other issue when installing python under Windows OS In-Reply-To: References: Message-ID: 2016-10-12 14:24 GMT+02:00 Karlheinz Hoening : > Hello all, > > on my iMac I have installed Windows (7) and I am now trying to install > Python 3.5.2 (32-bit) under Windows OS. > Every time after having 'repaired' with the installation procedere I > receive the following message when starting Python 3.5: > ---------------------------------------------------------------------------------------------------------------------------- > python.exe - Sytemfehler > Das Programm kann nicht gestartet werden, da api-ms-win-crt-I1-1-0.dll auf > dem Computer fehlt. > Installieren Sie das Programm erneut ... > ---------------------------------------------------------------------------------------------------------------------------- > Can you give any hint (support)? > Where can I find the above mentionned driver? > > Thanks a lot in advance. > cheers > Kalle > (k.hoening at gmx.net) > -- > https://mail.python.org/mailman/listinfo/python-list Hello, I believe, that file is a part of Windows runtime libraries and may be downloaded from the Microsoft webpage: https://support.microsoft.com/de-de/kb/2999226 (for multiple windows versions) However, there might be some specificities in handling this update on your computer. hth, vbr From Joaquin.Alzola at lebara.com Wed Oct 12 09:21:41 2016 From: Joaquin.Alzola at lebara.com (Joaquin Alzola) Date: Wed, 12 Oct 2016 13:21:41 +0000 Subject: smtplib TimeoutError not catch Message-ID: Hi Guys Try to connect to the smtp via smtplib. The connection is down because the Firewall has not been open yet so the exception is something that should appear. Now I want to catch this timeout in case error happens in future. Here the exception trace. [2016-10-12 14:14:06,289] ERROR in app: Exception on /api/barredMSISDN [POST] Traceback (most recent call last): File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1641, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1544, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.4/site-packages/flask/_compat.py", line 33, in reraise raise value File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1639, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1625, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/root/main.py", line 34, in barredMSISDN smtp_con(http_json) File "/root/main.py", line 134, in smtp_con server = smtplib.SMTP('smtp.lebara.com') File "/usr/local/lib/python3.4/smtplib.py", line 242, in __init__ (code, msg) = self.connect(host, port) File "/usr/local/lib/python3.4/smtplib.py", line 321, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/local/lib/python3.4/smtplib.py", line 292, in _get_socket self.source_address) File "/usr/local/lib/python3.4/socket.py", line 516, in create_connection raise err File "/usr/local/lib/python3.4/socket.py", line 507, in create_connection sock.connect(sa) TimeoutError: [Errno 110] Connection timed out How to catch such exception? ... have been trying with a lot of combinations (of course except the correct one :)) try: server.sendmail(sender, receivers, msg.as_string()) except Exception as e: <-- have try with many combinations here print("SMTP could not be contacted: %s",e) pass finally: server.quit() Joaquin This email is confidential and may be subject to privilege. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt. From pdorange at pas-de-pub-merci.mac.com Wed Oct 12 09:46:54 2016 From: pdorange at pas-de-pub-merci.mac.com (Pierre-Alain Dorange) Date: Wed, 12 Oct 2016 15:46:54 +0200 Subject: How to process syntax errors References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> Message-ID: <1mv03m2.2py2mkwzknd8N%pdorange@pas-de-pub-merci.mac.com> Terry Reedy wrote: > > Using this function, the code is "compiled". > > I do not think this function is often used and most python project > > simply use the interpreter (which do a small translation into byte-code > > to be faster and check syntax error before running interpretation > > You seem to be confusing CPython with, for instance, simple BASIC > interpreters that tokenized the code, translated keywords to function > numbers, and did other 'small translations' before execution. > > The CPython compiler lexes (tokenizes), ll(1) parses to a syntax tree, > does some analysis and transformation of the tree, and translates it to > the bytecode for an stack machine. All done using standard compiler theory. Yes i'm probably confusing things ; i've not explore Python implentation, i'm just an amateur developer. But what confuse me, is that Python require "real live" interpratation of the code to work properly (or perhaps i also confuse on that but Python rely on interpretation of the code to conform to its own standard, ie variables can change type during execution...) -- Pierre-Alain Dorange Moof Ce message est sous licence Creative Commons "by-nc-sa-2.0" From rosuav at gmail.com Wed Oct 12 11:00:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 13 Oct 2016 02:00:04 +1100 Subject: How to process syntax errors In-Reply-To: <1mv03m2.2py2mkwzknd8N%pdorange@pas-de-pub-merci.mac.com> References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <1mv03m2.2py2mkwzknd8N%pdorange@pas-de-pub-merci.mac.com> Message-ID: On Thu, Oct 13, 2016 at 12:46 AM, Pierre-Alain Dorange wrote: > Terry Reedy wrote: > >> > Using this function, the code is "compiled". >> > I do not think this function is often used and most python project >> > simply use the interpreter (which do a small translation into byte-code >> > to be faster and check syntax error before running interpretation >> >> You seem to be confusing CPython with, for instance, simple BASIC >> interpreters that tokenized the code, translated keywords to function >> numbers, and did other 'small translations' before execution. >> >> The CPython compiler lexes (tokenizes), ll(1) parses to a syntax tree, >> does some analysis and transformation of the tree, and translates it to >> the bytecode for an stack machine. All done using standard compiler theory. > > Yes i'm probably confusing things ; i've not explore Python > implentation, i'm just an amateur developer. > But what confuse me, is that Python require "real live" interpratation > of the code to work properly (or perhaps i also confuse on that but > Python rely on interpretation of the code to conform to its own > standard, ie variables can change type during execution...) Variables don't have types; objects do. A variable in Python always holds a thing of type 'object', and everything in Python is a subclass of object. This doesn't stop Python from being compiled - you could do the exact same thing in C++ or Java. ChrisA From eryksun at gmail.com Wed Oct 12 11:25:28 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 12 Oct 2016 15:25:28 +0000 Subject: an other issue when installing python under Windows OS In-Reply-To: References: Message-ID: On Wed, Oct 12, 2016 at 12:24 PM, Karlheinz Hoening wrote: > > on my iMac I have installed Windows (7) and I am now trying to install > Python 3.5.2 (32-bit) under Windows OS. > Every time after having 'repaired' with the installation procedere I > receive the following message when starting Python 3.5: > ---------------------------------------------------------------------------------------------------------------------------- > python.exe - Sytemfehler > Das Programm kann nicht gestartet werden, da api-ms-win-crt-I1-1-0.dll auf > dem Computer fehlt. > Installieren Sie das Programm erneut ... > ---------------------------------------------------------------------------------------------------------------------------- > Can you give any hint (support)? > Where can I find the above mentionned driver? It's not a driver; it's an API set DLL for the C runtime library. The C runtime is now an OS component, so preferably you should get this from Windows Update to install the most recent version available. It may be listed under optional updates. From chris at simplistix.co.uk Wed Oct 12 12:06:40 2016 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 12 Oct 2016 17:06:40 +0100 Subject: testfixtures 4.11.0 Released! Message-ID: Hi All, I'm pleased to announce the release of testfixtures 4.10.1 featuring the following: - Allow the attributes returned in LogCapture.actual() rows to be specified. - Allow a default to be specified for encoding in TempDirectory.read() and TempDirectory.write(). The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: https://github.com/Simplistix/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris From chris at withers.org Wed Oct 12 12:18:34 2016 From: chris at withers.org (Chris Withers) Date: Wed, 12 Oct 2016 17:18:34 +0100 Subject: [TIP] testfixtures 4.11.0 Released! In-Reply-To: References: Message-ID: <2a1efe3f-1751-4af1-c3a6-f47549b5ea27@withers.org> *sigh*, it's 4.11.0 that was released with these changes... On 12/10/2016 17:06, Chris Withers wrote: > Hi All, > > I'm pleased to announce the release of testfixtures 4.10.1 featuring > the following: > > - Allow the attributes returned in LogCapture.actual() rows to be > specified. > > - Allow a default to be specified for encoding in TempDirectory.read() > and TempDirectory.write(). > > The package is on PyPI and a full list of all the links to docs, issue > trackers and the like can be found here: > > https://github.com/Simplistix/testfixtures > > Any questions, please do ask on the Testing in Python list or on the > Simplistix open source mailing list... > > cheers, > > Chris > > _______________________________________________ > testing-in-python mailing list > testing-in-python at lists.idyll.org > http://lists.idyll.org/listinfo/testing-in-python From steve+python at pearwood.info Wed Oct 12 12:35:12 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 13 Oct 2016 03:35:12 +1100 Subject: How to process syntax errors References: <1muwzso.1ihpise13oukc8N%pdorange@pas-de-pub-merci.mac.com> <1muy91k.14c6xf6i2jrhsN%pdorange@pas-de-pub-merci.mac.com> <1mv03m2.2py2mkwzknd8N%pdorange@pas-de-pub-merci.mac.com> Message-ID: <57fe6642$0$1591$c3e8da3$5496439d@news.astraweb.com> On Thu, 13 Oct 2016 12:46 am, Pierre-Alain Dorange wrote: > Terry Reedy wrote: > >> > Using this function, the code is "compiled". >> > I do not think this function is often used and most python project >> > simply use the interpreter (which do a small translation into byte-code >> > to be faster and check syntax error before running interpretation >> >> You seem to be confusing CPython with, for instance, simple BASIC >> interpreters that tokenized the code, translated keywords to function >> numbers, and did other 'small translations' before execution. >> >> The CPython compiler lexes (tokenizes), ll(1) parses to a syntax tree, >> does some analysis and transformation of the tree, and translates it to >> the bytecode for an stack machine. All done using standard compiler >> theory. > > Yes i'm probably confusing things ; i've not explore Python > implentation, i'm just an amateur developer. > But what confuse me, is that Python require "real live" interpratation > of the code to work properly That is true, but not as much as you would think. Only two things in Python absolutely require runtime interpretation: eval() and exec(). But that depends on what you mean by "real live" interpretation. Compiled languages like Java and C++ also include runtime features like runtime dispatch, V-tables, > (or perhaps i also confuse on that but > Python rely on interpretation of the code to conform to its own > standard, ie variables can change type during execution...) Variables changing type is the least important part of this. The Rust programming language is strongly typed, compiled to efficient machine code like C, and it allows you to change variable types. Say you have code like this in Rust: let x = 1; f(x); let x = 'this is a string'; g(x); The Rust compiler is clever enough to tell that when you call f(), x is an integer, and when you call f(), x is a string. It can do that at compile time, not just runtime. If you can read a piece of source code, and work out what type a variable has, then so can the compiler -- if it is smart enough. The type systems of older languages like Pascal and C are not very smart, so they are very restrictive: they insist that variables have only a single type, which must be explicitly declared. But modern languages like Rust have very smart compilers with a very powerful type system, and they don't have that restriction. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rgaddi at highlandtechnology.invalid Wed Oct 12 12:57:05 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 12 Oct 2016 16:57:05 -0000 (UTC) Subject: How to process syntax errors References: Message-ID: mr.puneet.goyal at gmail.com wrote: > On Monday, October 10, 2016 at 7:45:40 PM UTC+5:30, mr.pune... at gmail.com wrote: >> Hi >> >> Is there any way to capture syntax errors and process them ? I want to write a function which calls every time whenever there is syntax error in the program. >> >> For example, >> >> inside example.py >> >> I just mention below line >> >> >> Obj = myClass() >> Obj xyz >> >> Obj is instance of a class. But there is syntax error on xyz. So I want to grab that error and process. >> >> Regards, Puneet > > Steve, You are absolutely right. I am trying to eliminate the method of using parenthesis while calling in my file. Especially when I call it from a instance. There's a simple solution then. Don't do that. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From eryksun at gmail.com Wed Oct 12 13:39:02 2016 From: eryksun at gmail.com (eryk sun) Date: Wed, 12 Oct 2016 17:39:02 +0000 Subject: an other issue when installing python under Windows OS In-Reply-To: References: Message-ID: On Wed, Oct 12, 2016 at 3:25 PM, eryk sun wrote: > On Wed, Oct 12, 2016 at 12:24 PM, Karlheinz Hoening wrote: >> >> on my iMac I have installed Windows (7) and I am now trying to install >> Python 3.5.2 (32-bit) under Windows OS. >> Every time after having 'repaired' with the installation procedere I >> receive the following message when starting Python 3.5: >> ---------------------------------------------------------------------------------------------------------------------------- >> python.exe - Sytemfehler >> Das Programm kann nicht gestartet werden, da api-ms-win-crt-I1-1-0.dll auf >> dem Computer fehlt. >> Installieren Sie das Programm erneut ... >> ---------------------------------------------------------------------------------------------------------------------------- >> Can you give any hint (support)? >> Where can I find the above mentionned driver? > > It's not a driver; it's an API set DLL for the C runtime library. The > C runtime is now an OS component, so preferably you should get this > from Windows Update to install the most recent version available. It > may be listed under optional updates. I just checked a Windows 7 system's update history. KB3118401 [1] was installed in March of this year as a recommended update. Typical Windows users just ignore and put off installing updates. That's why Microsoft switched to compulsory updates in Windows 10. [1]: https://support.microsoft.com/en-us/kb/3118401 From sohcahtoa82 at gmail.com Wed Oct 12 13:54:14 2016 From: sohcahtoa82 at gmail.com (sohcahtoa82 at gmail.com) Date: Wed, 12 Oct 2016 10:54:14 -0700 (PDT) Subject: How to process syntax errors In-Reply-To: References: Message-ID: <439579f1-007c-4e0d-931e-fdbbef0a25fa@googlegroups.com> On Wednesday, October 12, 2016 at 3:01:26 AM UTC-7, mr.pune... at gmail.com wrote: > Hi All > > Its really good to see that some discussion happening around this topic. Sorry I was out from my work for sometime so couldn't follow up but I really find it useful. It gives me good opportunity to know python better as I recently started learning python. > > Ok so I tell you why I need to catch syntax error during compilation and process them. See below example, > > # I created a platform class with different method in a file and making it as a package. > class platform: > def connect(self): > # connect device > def destroy(self): > # destroy device > def config(self, command): > # Send command to configure device > def show(self, command): > # check device health > > Now person who wants to write a script using above package can simply use below approach. Which does not make him to have knowledge in python. > > DUT = platform() > DUT connect > DUT config {commands} > DUT show {commands} > DUT destroy > > > But I know this is not easy to do in python. As they all invalid syntax for python language. So I thought of grabing syntax errors and manipulate them as below internally to call them while code is compiling. > > DUT = platform() > DUT.connect() > DUT.config(commands) > DUT.show(commands) > DUT.destroy() > > Hope you understand my need of doing this. If there is another solution to achieve what I am trying to. Please let me know. > > Thanks, Puneet You have two possible solutions: 1. Write an interpreter that can read an interpret the commands without parentheses. 2. Tell your users to use the dot "." operator and parentheses. You don't need to teach them Python, just how to properly format the script. Personally, I'd go with option #2. But as others have said, you can't get Python to just "work" without parentheses. Sure, you COULD attempt to import the script the users have written and catch the SyntaxError exception, but what would you do with it? At that point, you're going to have to interpret the line of code in the script yourself, and at that point, you should just be writing an interpreter and not attempting to treat the script as Python. From python at mrabarnett.plus.com Wed Oct 12 15:26:21 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 12 Oct 2016 20:26:21 +0100 Subject: smtplib TimeoutError not catch In-Reply-To: References: Message-ID: <8da319c4-7caf-22f0-a976-060455a24502@mrabarnett.plus.com> On 2016-10-12 14:21, Joaquin Alzola wrote: > Hi Guys > > Try to connect to the smtp via smtplib. > > The connection is down because the Firewall has not been open yet so the exception is something that should appear. > Now I want to catch this timeout in case error happens in future. > > Here the exception trace. > > [2016-10-12 14:14:06,289] ERROR in app: Exception on /api/barredMSISDN [POST] > Traceback (most recent call last): > File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1988, in wsgi_app > response = self.full_dispatch_request() > File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1641, in full_dispatch_request > rv = self.handle_user_exception(e) > File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1544, in handle_user_exception > reraise(exc_type, exc_value, tb) > File "/usr/local/lib/python3.4/site-packages/flask/_compat.py", line 33, in reraise > raise value > File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1639, in full_dispatch_request > rv = self.dispatch_request() > File "/usr/local/lib/python3.4/site-packages/flask/app.py", line 1625, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File "/root/main.py", line 34, in barredMSISDN > smtp_con(http_json) > File "/root/main.py", line 134, in smtp_con > server = smtplib.SMTP('smtp.lebara.com') > File "/usr/local/lib/python3.4/smtplib.py", line 242, in __init__ > (code, msg) = self.connect(host, port) > File "/usr/local/lib/python3.4/smtplib.py", line 321, in connect > self.sock = self._get_socket(host, port, self.timeout) > File "/usr/local/lib/python3.4/smtplib.py", line 292, in _get_socket > self.source_address) > File "/usr/local/lib/python3.4/socket.py", line 516, in create_connection > raise err > File "/usr/local/lib/python3.4/socket.py", line 507, in create_connection > sock.connect(sa) > TimeoutError: [Errno 110] Connection timed out > > > How to catch such exception? ... have been trying with a lot of combinations (of course except the correct one :)) > > try: > server.sendmail(sender, receivers, msg.as_string()) > except Exception as e: <-- have try with many combinations here > print("SMTP could not be contacted: %s",e) > pass > finally: > server.quit() > Not surprisingly, the exception you should catch is simply TimeoutError: try: server.sendmail(sender, receivers, msg.as_string()) except TimeoutError as e: print("SMTP could not be contacted: %s" % e) finally: server.quit() From trolleri.trollface at gmail.com Wed Oct 12 16:37:23 2016 From: trolleri.trollface at gmail.com (LongHairLuke) Date: Wed, 12 Oct 2016 13:37:23 -0700 (PDT) Subject: Scripting Help please Message-ID: <29b9e43e-d7a3-4587-913a-707a7f0ef517@googlegroups.com> Hi l am on my way to make a bot for the game Piano Tiles 2. But the code l have written so far saids invalid syntax at 2nd line. Here is my code: while True: If active then FFSnapShot(areaX, areaY + height - offsetBottom, areaX + width, areaY + height - offsetBottom, sid) For row = 0 To norows - 1 c = FFGetPixel(areaX + row * rowWidth + 5, areaY + height - offsetBottom, sid) cs = _ColorGetRGB(c) b = ( cs[0] * 0.3 ) + ( cs[1] * 0.59 ) + ( cs[2] * 0.11 ) If lock[row] Then If b < darkness Then ContinueLoop Else lock[row] = False EndIf EndIf If b < darkness Then MouseUp("Left") lock[row] = True MouseMove(areaX + row * rowWidth + rowWidth / 2, areaY + height - offsetBottom, 1) MouseDown("Left") EndIf Next EndIf End Can someone help me please? From alister.ware at ntlworld.com Wed Oct 12 16:48:31 2016 From: alister.ware at ntlworld.com (alister) Date: Wed, 12 Oct 2016 20:48:31 GMT Subject: Scripting Help please References: <29b9e43e-d7a3-4587-913a-707a7f0ef517@googlegroups.com> Message-ID: On Wed, 12 Oct 2016 13:37:23 -0700, LongHairLuke wrote: > Hi l am on my way to make a bot for the game Piano Tiles 2. > But the code l have written so far saids invalid syntax at 2nd line. > Here is my code: > > > > while True: > If active then > FFSnapShot(areaX, areaY + height - offsetBottom, areaX + width, > areaY + height - offsetBottom, sid) > For row = 0 To norows - 1 > c = FFGetPixel(areaX + row * rowWidth + 5, areaY + height - > offsetBottom, sid) > cs = _ColorGetRGB(c) > b = ( cs[0] * 0.3 ) + ( cs[1] * 0.59 ) + ( cs[2] * 0.11 ) > If lock[row] Then > If b < darkness Then > ContinueLoop > Else > lock[row] = False > EndIf > EndIf If b < darkness Then > MouseUp("Left") > lock[row] = True MouseMove(areaX + row * rowWidth + rowWidth > / 2, areaY + height - offsetBottom, 1) > MouseDown("Left") > EndIf > Next > EndIf > End > > Can someone help me please? Is observation this is not Python code it looks more like some variety of BASIC but I could easily be wrong -- Regarding astral projection, Woody Allen once wrote, "This is not a bad way to travel, although there is usually a half-hour wait for luggage." From tjreedy at udel.edu Wed Oct 12 17:28:48 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 12 Oct 2016 17:28:48 -0400 Subject: How to process syntax errors In-Reply-To: References: Message-ID: On 10/12/2016 5:59 AM, mr.puneet.goyal at gmail.com wrote: > # I created a platform class with different method in a file > # and making it as a package. > class platform: > def connect(self): > # connect device > def destroy(self): > # destroy device > def config(self, command): > # Send command to configure device > def show(self, command): > # check device health > Now person who wants to write a script using above package can simply > use below approach. Which does not make him to have knowledge in python. > > DUT = platform() > DUT connect > DUT config {commands} > DUT show {commands} > DUT destroy > But I know this is not easy to do in python. Actually, as long as args (commands) continue to be space separated, it is. One way would be to add the following after the class statement. def execute(file): with open(file) as f: ns = {} for line in f: if '=' in line: exec(line, ns) else: fields = line.split() exec('%s.%s(*%s)' % (fields[0], fields[1], fields[2:]), nd) if __name__ = __main__: import sys execute(sys.argv[1]) -- If the module is platform.py, then the command line usage would be > platform ... Please, can You me to aim on the right way to obtain into a Python data structure everything that JS code give back to browser. Thank in advance From bgailer at gmail.com Fri Oct 21 10:32:39 2016 From: bgailer at gmail.com (Bob Gailer) Date: Fri, 21 Oct 2016 10:32:39 -0400 Subject: Obtain javascript result In-Reply-To: References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> Message-ID: On Oct 21, 2016 9:30 AM, wrote: > > Hello NG. > > I'm new in Python for fun. > > I have a html page (I load it by BeautifulSoap) that contain > also this javascript code: > ... > > ... > Please, can You me to aim on the right way > to obtain into a Python data structure > everything that JS code give back to browser. Not without seeing the code for matchdetails_init. Likely we'd also need to see the page source. What do you expect to get? From __peter__ at web.de Fri Oct 21 10:45:39 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 21 Oct 2016 16:45:39 +0200 Subject: tkinter callbacks stop working References: Message-ID: namenobodywants at gmail.com wrote: > hello pythonistas > > the script below plays tictactoe; everything works; but if i replace the > block at the bottom > > if True: > > > with this instead > > def function(): Below I use main() instead. > > function() > > then the tkinter callbacks don't work anymore; can anybody make sense of > this? thanks if you can help > > peace > stm > > ps: here's the code... When you invoke your script on the command line you'll get detailed information about the error, e. g.: $ python3 tictactoe.py Traceback (most recent call last): File "tictactoe.py", line 113, in main() File "tictactoe.py", line 108, in main newgame() File "tictactoe.py", line 57, in newgame show(board) File "tictactoe.py", line 70, in show = lambda board: status.set(getstatus(board)) or [squares[i].set(board[i]) for i in range(9)] NameError: name 'status' is not defined This is called "traceback" and the thing that you should study carefully when debugging and always provide when you are asking for help. The NameError is a hint that the show callback cannot find a variable called status. This is of course because after your change status is bound inside a function and thus local to that function. Some ways to fix the problem are (1) make status (and squares) global by putting the declaration global status, squares into your function. This is the beginner's approach and for more complex programs you tend to end up in a big mess where everything is global. (2) make the lambda (which doesn't have to be a lambda because it's not acually used as an expression) local def main(): global show def show(board): ... ... so that it sees the names local to the enclosing function. We still need to declare the function as global because it will be invoked by other global functions (of course you can try and make those local, too). (4) Pass the dependencies explicitly as additional parameters. show() would become def show(board, status, squares): ... and you have to change all places where it is called and provide the extra arguments. This gets tedious when there are multiple levels of function calls because the calling function must often be changed in a similar way. (5) Use a class and turn all functions that implicitly access global state into methods. Store what used to be global state as instance attributes of the class. This seems to be the most popular approach for GUI applications. It allows relatively clean designs if you are careful that your classes don't become too big. From maiqtefan at hotmail.com Fri Oct 21 11:00:58 2016 From: maiqtefan at hotmail.com (mohammed iqtefan) Date: Fri, 21 Oct 2016 15:00:58 +0000 Subject: dll files missings ....how to solve this problem In-Reply-To: References: Message-ID: hi i installed python 3.5.2 but i coudln't run it because i always have this msg api-ms-win-crt-conio-l1-1-0.dll is missing and its not the only dll file that missing there is dozens more help me plz From eproser at gmail.com Fri Oct 21 11:07:41 2016 From: eproser at gmail.com (eproser at gmail.com) Date: Fri, 21 Oct 2016 08:07:41 -0700 (PDT) Subject: Obtain javascript result In-Reply-To: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> Message-ID: <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> Yes, the page is http://www.betexplorer.com/next/soccer/ and You have to open any match You want. This pop-up new windows with match detail and odds (if present). I try to extract home team, away team, results, and bet365's bookmaker odds. I can't continue in my fun analyses because the odds are returned through that javascript code and I don't know how manage it in python to take that results. From bgailer at gmail.com Fri Oct 21 13:04:42 2016 From: bgailer at gmail.com (bob gailer) Date: Fri, 21 Oct 2016 13:04:42 -0400 Subject: Obtain javascript result In-Reply-To: <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> Message-ID: <30a83067-be35-ab7e-c978-f3dc4aa36a07@gmail.com> On 10/21/2016 11:07 AM, eproser at gmail.com wrote: > Yes, > > the page is http://www.betexplorer.com/next/soccer/ > and You have to open any match You want. > > This pop-up new windows with match detail and odds > (if present). > > I try to extract home team, away team, results, and > bet365's bookmaker odds. > > I can't continue in my fun analyses because the odds > are returned through that javascript code and > I don't know how manage it in python > to take that results. What version of Python are you using, and what operating system? I would use splinter (https://pypi.python.org/pypi/splinter) From a command prompt: pip install splinter The Python program (this is not tested; your mileage may vary): from splinter import Browser browser = Browser(browser) page = browser.visit('http://www.betexplorer.com/soccer/russia/youth-league/matchdetails.php?matchid=rLu2Xsdi') div = browser.find_by_id('be') text = div.text text looks like " Soccer ? Russia ? Youth League 2016/201720.10.2016 Cruzeiro-Corinthians 4:2 (1:1, 3:1) 14.Abila Ramon (penalty kick) 59.Abila Ramon 63.Rodrigo Bruno 83.De Arrascaeta Giorgian 35.Rodriguinho 86.Rildo 1X2 Odds (25) O/U (100) AH (33) DNB (11) DC (20) BTS (14) Bookmakers: 251X2 10Bet1.573.555.05 12BET1.753.554.40 188BET1.723.654.40 888sport1.683.704.80 bet-at-home1.673.404.42 bet3651.733.604.75 Betclic1.653.404.60 Betfair1.663.404.40 Betsafe1.683.504.25 Betsson1.683.504.25 BetVictor1.703.605.00 Betway1.703.404.75 bwin1.673.704.40 ComeOn1.573.555.05 Expekt1.653.404.60 Interwetten1.703.454.50 Ladbrokes1.733.605.00 mybet1.753.504.00 Pinnacle1.753.815.25 SBOBET1.723.504.70 Sportingbet1.673.604.60 Tipico1.653.504.70 Unibet1.683.704.80 William Hill1.733.404.40 youwin1.673.604.60 Average odds 1.693.544.63 close window $(document).ready(function() { matchdetails_init('rLu2Xsdi', '1x2'); }); " I hope you know enough to extract the data you want. There are probably better ways to drill down to the relevant table rows, but I can't do that right now as I am having my own problem with splinter. splinter is the only tool I know that will do this. There may be others. From best_lay at yahoo.com Fri Oct 21 13:06:48 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 21 Oct 2016 12:06:48 -0500 Subject: How to handle errors? References: <8500044a-c8d1-43ad-91d9-e836d52bd3f0@googlegroups.com> <5809a443$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, 21 Oct 2016 16:14:41 +1100, Steve D'Aprano wrote: > On Fri, 21 Oct 2016 11:03 am, Wildman wrote: > >> On Thu, 20 Oct 2016 12:48:28 -0700, SS wrote: >> >>> The following script works fine: >>> >>> #!/bin/python >> >> I meant to include this with my other post but I forgot it. >> >> Using a direct path to the Python interpreter can cause problems >> on some systems because it is not always installed to the same >> directory. > > Then you change the path and fix it. You are assuming that I, the user, knows what a shebang is and what it is used for and what Python is and where the interpreter is located. If you are targeting developers none of that will be a problem but, if your program is for "average users", you may have a serious problem. Personally I would not want to distribute a program that may require the user to fix the code before it will run. >> On my Debian-based system Python is installed in >> /usr/bin. So your code as written will not run on my system. >> A workaround for this is to use env in the shebang/hashbang. > > That's not a work-around. That's a bug waiting to happen. > > One of the problems with of env is that it will use whatever python > executable appears first in the user's $PATH, regardless of whether it is > the right Python or not -- or even whether it is actually Python, or just > some random executable file called "python". For example, you might have > compiled your own experimental Python executable, and adjusted your PATH > environment variable to find it. Now your env scripts will use your > unstable, experimental Python interpreter instead of the system Python. A developer might have complied experimental versions of Python but the average user will not. Here again it depends on your target. > Another serious problem with using env in the hash-bang line is that you > cannot pass commandline options to the Python executable. Not true. I made a test script with this code: #!/usr/bin/env python import sys print sys.argv Then I ran it: ~$ python test.py argument1 argument2 ['test.py', 'argument1', 'argument2'] > Using env in this way is a hack that happens to mostly work. Its arguably an > abuse of env, and its not as portable as people often think (not all older > Unix systems even have env). If you are targeting older Unixes than yes that could be a problem. >> For Python 2: #!/usr/bin/env python >> For Python 3: #!/usr/bin/env python3 >> >> It will not matter where Python is installed. 'env' will always >> know where it is. > > That's not correct: env only searches the PATH, so if your python is not in > the path, it won't be found. Here's env on my system with the default PATH: > > [steve at ando ~]$ /usr/bin/env python -c "import sys; print(sys.version)" > 2.4.3 (#1, Jan 9 2013, 06:49:54) > [GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] > > > But if I change the search path (or if I move the Python executable > somewhere off the path): > > [steve at ando ~]$ PATH="/tmp" /usr/bin/env python -c "import sys; > print(sys.version)" > /usr/bin/env: python: No such file or directory > > > Even if env finds something called "python", you can't be sure that it is > the right version of Python, or even Python at all. All you know is that it > is something called "python" on the search path. Not likely an average user is going muck around with the path in the way you describe. -- GNU/Linux user #557453 The cow died so I don't need your bull! From walters.justin01 at gmail.com Fri Oct 21 13:13:15 2016 From: walters.justin01 at gmail.com (justin walters) Date: Fri, 21 Oct 2016 10:13:15 -0700 Subject: Obtain javascript result In-Reply-To: <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> Message-ID: On Fri, Oct 21, 2016 at 8:07 AM, wrote: > Yes, > > the page is http://www.betexplorer.com/next/soccer/ > and You have to open any match You want. > > This pop-up new windows with match detail and odds > (if present). > > I try to extract home team, away team, results, and > bet365's bookmaker odds. > > I can't continue in my fun analyses because the odds > are returned through that javascript code and > I don't know how manage it in python > to take that results. > The data is being obtained through an ajax call: jQuery.ajax({ url: '/gres/ajax-matchdetails.php', type: 'GET', cache: false, dataType: 'json', success: function(data) { if (data.user_logged) { user.logged = true; user.oddsformat = data.oddsformat; } bookmaker_urls = data.bookmaker_urls; ajax_to_call--; if (ajax_to_call == 0) { matchdetails_finish(); $('#odds-all-loader').remove(); $('#odds-all').show(); } } }); jQuery.ajax({ url: '/gres/ajax-matchodds.php', type: 'GET', data: 't=d&e=' + eventid + '&b=' + bettype, success: function(data) { $('#odds-all').html(data); ajax_to_call--; if (ajax_to_call == 0) { matchdetails_finish(); $('#odds-all-loader').remove(); $('#odds-all').show(); } } You can see this by opening up your developer tools (chrome or Firefox) and navigating to the debugger/scripts tab where you will find the "matchdetails_init" function. However, I tried fetching the "ajax-matchdetails.php" endpoint via httpie, and I received a 404 page. Investigating the request via the developer console led to some strange results as well. There was a small piece of json data returned containing the keys "user_logged" and "bookmaker_urls". There was no data about the match at all. There are several other ajax calls in the script, so you may want to check those out. However, it seems like the match data itself is actually populated with data server-side via a PHP script. You should be able to get the link for a match, point beautiful soup at it, parse it, and get the data. From torriem at gmail.com Fri Oct 21 14:34:17 2016 From: torriem at gmail.com (Michael Torrie) Date: Fri, 21 Oct 2016 12:34:17 -0600 Subject: dll files missings ....how to solve this problem In-Reply-To: References: Message-ID: <658a4e96-0845-e712-08b8-d14267f7dec8@gmail.com> On 10/21/2016 09:00 AM, mohammed iqtefan wrote: > hi > > i installed python 3.5.2 but i coudln't run it because i always have > this msg > > api-ms-win-crt-conio-l1-1-0.dll is missing > > and its not the only dll file that missing there is dozens more The api-ms-win-crt-conio dll is part of the new Microsoft universal C runtime library. Most computers after Windows 7 should already have it via windows update. From torriem at gmail.com Fri Oct 21 14:37:19 2016 From: torriem at gmail.com (Michael Torrie) Date: Fri, 21 Oct 2016 12:37:19 -0600 Subject: Obtain javascript result In-Reply-To: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> Message-ID: <5dc42b0e-44f5-f0f6-f420-ba0afe19131a@gmail.com> On 10/21/2016 07:29 AM, eproser at gmail.com wrote: > Hello NG. > > I'm new in Python for fun. > > I have a html page (I load it by BeautifulSoap) that contain > also this javascript code: > ... > > ... > Please, can You me to aim on the right way > to obtain into a Python data structure > everything that JS code give back to browser. All BeautifulSoup does is parse html. It doesn't run javascript. To obtain the html after the javascript has run, or to interact with the Javascript you'll need a tool like Selenium[1], which has drivers for Python and also Chrome or Firefox. In other words you need a real browser running that can execute the javascript. After that you can extract the html results using BeautifulSoup. Google for many tutorials on using Selenium with python. [1] https://pypi.python.org/pypi/selenium From lucho5798 at gmail.com Fri Oct 21 16:07:17 2016 From: lucho5798 at gmail.com (...) Date: Fri, 21 Oct 2016 13:07:17 -0700 (PDT) Subject: ReGex with python Message-ID: <19b929a9-d479-4c5b-8a89-d893b837e6be@googlegroups.com> Hi, I have a problem I could not solve, ask for your cooperation. "show_app_install": true, "static_root": "//instagramstatic-a.akamaihd.net/h1", "platform": "web", "activity_counts": null, "hostname": "www.instagram.com", "entry_data": {"PostPage": [{"media": {"caption_is_edited": false, "code": "BL1X_KlhEgp", "date": 1477072256, "dimensions": {"width": 750, "height": 750}, "usertags": {"nodes": []}, "comments_disabled": false, "comments": {"count": 1, "page_info": {"has_previous_page": false, "start_cursor": null, "end_cursor": null, "has_next_page": false}, "nodes": [{"text": "Muy buena.ja ja", "created_at": 1477073804.0, "id": "17854331140105338", "user": {"username": "jairo_becerra_castano", "profile_pic_url": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/s150x150/14052608_540744196124877_178395402_n.jpg", "id": "2311473855"}}]}, "id": "1366103557740447785", "caption": "Jajajajajajajajaja, uyyyyyyy Echeverry, a cuantos enga\\u00f1aron con este cuento? #plebiscito #paz #colombia", "likes": {"count": 9, "viewer_has_liked": false, "nodes": [{"user": {"username": "lpintor7", "profile_pic_url": "https://igcdn-photos-h-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/11257245_514557058701967_834362393_a.jpg", "id": "184969394"}}, {"user": {"username": "lination24", "profile_pic_url": "https://igcdn-photos-g-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/s150x150/14027341_154551878317334_55370841_a.jpg", "id": "177966087"}}, {"user": {"username": "maccamontero", "profile_pic_url": "https://igcdn-photos-g-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/s150x150/14359472_532259770300854_7780250093252247552_a.jpg", "id": "595746059"}}, {"user": {"username": "jairo_becerra_castano", "profile_pic_url": "https://igcdn-photos-f-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/s150x150/14052608_540744196124877_178395402_n.jpg", "id": "2311473855"}}, {"user": {"username": "dieguito_kintero", "profile_pic_url": "https://igcdn-photos-g-a.akamaihd.net/hphotos-ak-xpa1/t51.2885-19/10499068_1393117677674150_1052081019_a.jpg", "id": "1777663318"}}, I need only select the text and caption, in this case: Muy buena.ja ja Jajajajajajajajaja, uyyyyyyy Echeverry, a cuantos enga\\u00f1aron con este cuento? #plebiscito #paz #colombia Thanks!. From eryksun at gmail.com Fri Oct 21 19:01:56 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 21 Oct 2016 23:01:56 +0000 Subject: dll files missings ....how to solve this problem In-Reply-To: References: Message-ID: On Fri, Oct 21, 2016 at 4:05 PM, Dennis Lee Bieber wrote: > On Fri, 21 Oct 2016 15:00:58 +0000, mohammed iqtefan > declaimed the following: > >>i installed python 3.5.2 >> ... >>api-ms-win-crt-conio-l1-1-0.dll is missing >>and its not the only dll file that missing there is dozens more > > While there has been a common problem with one of the more recent M$ > DLLs not being readily available, "dozens more" would seem to imply that > your computer is missing a lot of stuff... Or is running WinXP. By design, 3.5.2 should fail to install in Windows XP. The Universal CRT is an OS component in Windows Vista and later. Mohammed should run Windows Update and ensure that all recommended updates are installed. > The newest Python builds require Windows 7 or newer. 3.5 and 3.6 support Vista. Each feature release of Python supports versions of Windows that have yet to reach EOL as of the release date. EOL Python Version ---------------------------------- XP 2014-04-08 3.4 (2014-03-16) Vista 2017-04-11 3.6 (2016-12-16) Estimates: 7 2020-01-14 3.8 8.1 2023-01-10 4.0 10 2025-10-14 4.2 From patrick.mckendree.young at gmail.com Fri Oct 21 19:15:51 2016 From: patrick.mckendree.young at gmail.com (Patrick Young) Date: Fri, 21 Oct 2016 17:15:51 -0600 Subject: Building manylinux wheels & auditwheel Message-ID: Hello, I've been exploring building manylinux wheels and trying to use auditwheel to vendorize my shared libraries. Essentially, I can't seem to get "auditwheel repair" to fix up my wheel that has some dependencies that need to be packed up. I've described my problem at https://github.com/pypa/auditwheel/issues/52, but to summarize, I've built the demo wheel found at https://github.com/pypa/python-manylinux-demo. From inside the container, after the wheel's have been built but not repaired, I run auditwheel -vv show python_manylinux_demo-1.0-cp27-cp27mu-linux_x86_64.whl which gives the following: DEBUG:auditwheel.wheel_abi:{} DEBUG:auditwheel.policy.versioned_symbols:Required symbol versions: {} INFO:auditwheel.wheel_abi:{ "manylinux1_x86_64": { "priority": 100, "libs": {} }, "linux_x86_64": { "priority": 0, "libs": {} } } DEBUG:auditwheel.wheel_abi:external referene info DEBUG:auditwheel.wheel_abi:{ "manylinux1_x86_64": { "priority": 100, "libs": {} }, "linux_x86_64": { "priority": 0, "libs": {} } } python_manylinux_demo-1.0-cp27-cp27mu-linux_x86_64.whl is consistent with the following platform tag: "manylinux1_x86_64". The wheel references no external versioned symbols from system- provided shared libraries. The wheel requires no external shared libraries! :) but unzipping that wheel and runnning ldd tells a different story [root at 1965501e43ea pymanylinuxdemo]# ldd extension.so linux-vdso.so.1 => (0x00007ffc52bd8000) libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007fdcd467e000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fdcd4461000) libc.so.6 => /lib64/libc.so.6 (0x00007fdcd4108000) libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007fdcd3813000) libm.so.6 => /lib64/libm.so.6 (0x00007fdcd358f000) /lib64/ld-linux-x86-64.so.2 (0x00005619467b3000) Any help would be greatly appreciated! I'm experiencing the same thing across the board with other builds I've been trying to manylinuxize. Thanks! -Patrick From hanzer at riseup.net Fri Oct 21 22:43:55 2016 From: hanzer at riseup.net (Adam Jensen) Date: Fri, 21 Oct 2016 22:43:55 -0400 Subject: =?UTF-8?Q?Internet_Data_Handling_=c2=bb_mailbox?= Message-ID: The mailbox library documentation seems to be a little weak. In this example: https://docs.python.org/2.7/library/mailbox.html#examples import mailbox for message in mailbox.mbox('~/mbox'): subject = message['subject'] # Could possibly be None. if subject and 'python' in subject.lower(): print subject What is the structure of "message"? I guess it's a dictionary but what is its structure? What are the keys? Is the structure constant or does it change depending upon the content of the mbox? I'm a bit new to python documentation. How would a potential user of the "mailbox" library determine these things? Or did I miss something? From python at mrabarnett.plus.com Fri Oct 21 23:22:45 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 22 Oct 2016 04:22:45 +0100 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: References: Message-ID: <33b7ab59-0344-0b42-cbdc-c9e8e88bd32f@mrabarnett.plus.com> On 2016-10-22 03:43, Adam Jensen wrote: > The mailbox library documentation seems to be a little weak. In this > example: > > https://docs.python.org/2.7/library/mailbox.html#examples > > import mailbox > for message in mailbox.mbox('~/mbox'): > subject = message['subject'] # Could possibly be None. > if subject and 'python' in subject.lower(): > print subject > > What is the structure of "message"? I guess it's a dictionary but what > is its structure? What are the keys? Is the structure constant or does > it change depending upon the content of the mbox? > > I'm a bit new to python documentation. How would a potential user of the > "mailbox" library determine these things? Or did I miss something? > The docs say that it's subclass of the email.message module's Message. You can get the email's header fields like it's a dict, except that the field names are case-insensitive. The author(s) of the module couldn't use a true dict because of the need for additional functionality. From ben+python at benfinney.id.au Fri Oct 21 23:45:10 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 22 Oct 2016 14:45:10 +1100 Subject: Internet Data Handling =?utf-8?Q?=C2=BB?= mailbox References: Message-ID: <85zilxdp2x.fsf@benfinney.id.au> Adam Jensen writes: > import mailbox > for message in mailbox.mbox('~/mbox'): > subject = message['subject'] # Could possibly be None. > if subject and 'python' in subject.lower(): > print subject > > What is the structure of "message"? You're binding that name to each item from the collection returned from ?mailbox.mbox?. So, let's look at the documentation for that function: class mailbox.mbox(path, factory=None, create=True) A subclass of ``Mailbox`` for mailboxes in mbox format. Parameter `factory` is a callable object that accepts a file-like message representation (which behaves as if opened in binary mode) and returns a custom representation. If `factory` is ``None``, ``mboxMessage`` is used as the default message representation. [?] So the above usage doesn't specify a `factory` to create instances, which means the instances will be instances of ``mboxMessage`` type. > I guess it's a dictionary but what is its structure? What are the > keys? Is the structure constant or does it change depending upon the > content of the mbox? >From the same documentation you can follow the link to the documentation for ``mailbox.mboxMessage``. There you'll find it inherits from ``mailbox.Message``, which itself inherits ``email.message.Message``. So each instance you're getting has (a superset of) the API of ``email.message.Message``, which is a lot of behaviour including being able to interrogate the message header for its fields, by name, using the get-an-item ?foo[bar]? syntax. (The message has exactly one header, the header has multiple fields. The documentation and API erroneously refer to those fields as ?headers?, but that is a common confusion to a lot of software and the Python standard library makes it too.) > I'm a bit new to python documentation. How would a potential user of the > "mailbox" library determine these things? Or did I miss something? The library reference documentation must assume you at least know the Python language, including that collections contain items, those items are each objects, every object has a type, the types inherit in a tree, etc. So you'd need to read the in the knowledge that a return value's behaviour is determined by its type, and that the behaviour of a type is determined by other behaviour it inherits, etc. -- \ ?Some forms of reality are so horrible we refuse to face them, | `\ unless we are trapped into it by comedy. To label any subject | _o__) unsuitable for comedy is to admit defeat.? ?Peter Sellers | Ben Finney From eproser at gmail.com Sat Oct 22 00:40:42 2016 From: eproser at gmail.com (eproser at gmail.com) Date: Fri, 21 Oct 2016 21:40:42 -0700 (PDT) Subject: Obtain javascript result In-Reply-To: References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> Message-ID: <08cc64fa-5567-4de3-aefc-ad5931199b51@googlegroups.com> Many Thanks to everybody. From dieter at handshake.de Sat Oct 22 03:24:34 2016 From: dieter at handshake.de (dieter) Date: Sat, 22 Oct 2016 09:24:34 +0200 Subject: Internet Data Handling =?iso-8859-1?Q?=BB?= mailbox References: Message-ID: <874m447snh.fsf@handshake.de> Adam Jensen writes: > ... > https://docs.python.org/2.7/library/mailbox.html#examples > > import mailbox > for message in mailbox.mbox('~/mbox'): > subject = message['subject'] # Could possibly be None. > if subject and 'python' in subject.lower(): > print subject > > What is the structure of "message"? I guess it's a dictionary but what > is its structure? What are the keys? In addition to the previous (excellent) responses: A "message" models a MIME (RFC1521 Multipurpose Internet Mail Extensions) message (the international standard for the structure of emails). The standard tells you that a message consists essentially of two parts: a set of headers and a body and describes standard headers and their intended meaning (e.g. "To", "From", "Subject", ...). It allows a message to contain non-standard headers as well. With this knowledge, your "keys" related question can be answered: there is a (case insensitive) key for each header actually present in your message. If the message contains several headers with the same name, the subscription access gives you the first one; there is an alternative method to access all of them. From chenyong20000 at gmail.com Sat Oct 22 04:39:02 2016 From: chenyong20000 at gmail.com (chenyong20000 at gmail.com) Date: Sat, 22 Oct 2016 01:39:02 -0700 (PDT) Subject: function call questions In-Reply-To: References: <8a0a7843-9b7c-4543-83b0-cd6ebd5c8fc3@googlegroups.com> <26abbecc-a7eb-4544-8bf1-5df8ec458fe9@googlegroups.com> <17e0008e-697e-49c5-96b7-137e48b85a6f@googlegroups.com> <5506e4d8-bd1d-4e56-8d1b-f71fa8293393@googlegroups.com> <01cfd810-0561-40b1-a834-95a73dad6e56@googlegroups.com> Message-ID: <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> ? 2016?10?20???? UTC+8??11:04:38?Frank Millman??? > wrote in message > news:01cfd810-0561-40b1-a834-95a73dad6e56 at googlegroups.com... > > ? 2016?10?20???? UTC+8??1:32:18?Frank Millman??? > > wrote in message > > news:5506e4d8-bd1d-4e56-8d1b-f71fa8293393 at googlegroups.com... > > > > > Let's see if I can explain. I am using 't' and 'r' instead of 'tree' and > > > 'root', but otherwise it is the same as your original example. > > > > > > >>> t = {} > > > >>> r = t > > > >>> id(t) > > > 2542235910088 > > > >>> id(r) > > > 2542235910088 > > > > > > At this point, t and r are both references to the same empty dictionary. > > > > > > >>> r = r.setdefault('a', {}) > > > > > > This has done two things. > > > > > > It has inserted the key 'a' into the dictionary, and set its value to > > > {}. > > > > > > >>> t > > > {'a': {}} > > > >>> id(t) > > > 2542235910088 > > > > > > It has also rebound 'r' so that it now references the new empty > > > dictionary > > > that has been inserted. > > > > > > >>> r > > > {} > > > >>> id(r) > > > 2542234429896 > > > >>>t['a'] > > > {} > > > >>> id(t['a']) > > > 2542234429896 > > > > > > Now continue this process with r = r.setdefault('b', {}), and watch what > > > happens. > > > > thanks very much for your kind help. Your reply is clear. But I'm hindered > > by those you've not explained. > > > > I'm always confused by "r = r.setdefault('a', {})". when the first loop > > finished, as what you have pointed out, > > > >>> t > > > {'a': {}} > > > >>> r > > > {} > > > > Then next "r = r.setdefault('b', {})" will run again. Here what is "r" in > > "r.setdefault('b',{})"? According to final result, it should be "t['a']", > > which I can't understand. I thought the command is r.setdefault, so it > > should still be last "r", i.e., empty {}. Could you please let me know > > what I missed? thanks. > > Firstly, I want to explain more clearly what I am doing here. > > Instead of running your loop 3 times, I am running your command three times > one step at a time (though I only showed the first one). > > >>> t = {} > >>> r = t > >>> r = r.setdefault('a', {}) > >>> r = r.setdefault('b', {}) > >>> r = r.setdefault('c', {}) > > This should give exactly the same result as your loop. The benefit of > running it this way is that you can check the values after each step. > > May I suggest that you do this, and try to understand the contents of 't' > and 'r' at each point. If you are still unsure, let us know at which point > the values are not what you expect, and I will try to explain further. > > It is important that you understand that you are rebinding 'r' at each step, > so after each command, 'r' is no longer referencing the same object that it > was referencing in the previous step. > > To see the difference, try running it it this way - > > >>> t = {} > >>> r = t > >>> r.setdefault('a', {}) > >>> r.setdefault('b', {}) > >>> r.setdefault('c', {}) > > Hope this helps. > > Frank Hi Frank, thanks for your kind help. What confused me is at this line: > >>> r = r.setdefault('b', {}) and its previous one > >>> r = r.setdefault('a', {}) When r.setdefault('a',{}) is run, I understand it will return an empty {}. At this time both r & t reference to {'a':{}}, right? So when "r = r.setdefault('a',{})" is run, r reference to {} while t keeps the same as {'a':{}}. then comes r.setdefault('b',{}). What hinder me is here. since r has changed its reference to {}, r.setdefault('b',{}) will return {} again. So what does this done to t? why at this time t changes to {'a':'b':{}}? Sorry for my silly here. Thanks From frank at chagford.com Sat Oct 22 05:05:32 2016 From: frank at chagford.com (Frank Millman) Date: Sat, 22 Oct 2016 11:05:32 +0200 Subject: function call questions In-Reply-To: <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> References: <8a0a7843-9b7c-4543-83b0-cd6ebd5c8fc3@googlegroups.com> <26abbecc-a7eb-4544-8bf1-5df8ec458fe9@googlegroups.com> <17e0008e-697e-49c5-96b7-137e48b85a6f@googlegroups.com> <5506e4d8-bd1d-4e56-8d1b-f71fa8293393@googlegroups.com> <01cfd810-0561-40b1-a834-95a73dad6e56@googlegroups.com> <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> Message-ID: wrote in message news:2853d778-857e-46fc-96a0-8d164c098499 at googlegroups.com... ? 2016?10?20???? UTC+8??11:04:38?Frank Millman??? > wrote in message > news:01cfd810-0561-40b1-a834-95a73dad6e56 at googlegroups.com... > > Hi Frank, > > thanks for your kind help. What confused me is at this line: > > >>> r = r.setdefault('b', {}) > > and its previous one > > >>> r = r.setdefault('a', {}) > > When r.setdefault('a',{}) is run, I understand it will return an empty {}. > At this time both r & t reference to {'a':{}}, right? So when "r = > r.setdefault('a',{})" is run, r reference to {} while t keeps the same as > {'a':{}}. > > then comes r.setdefault('b',{}). What hinder me is here. since r has > changed its reference to {}, r.setdefault('b',{}) will return {} again. So > what does this done to t? why at this time t changes to {'a':'b':{}}? > Sorry for my silly here. Thanks > I don't know if you have been following the other posts in this thread. There were some posts from Anssi Saari who was also confused by this, but then yesterday he sent a post saying that he has now 'got it'. I don't think I can do better than quote his explanation of what is happening - """ OK, so what happens is that now t references the dictionary with {'a': {}} and r references the empty dict inside that. So when we assign to r again, it's the empty dict inside t (the one accessed by key 'a') that changes to {'b': {}} and t becomes {'a': {'b': {}}}. """ That is exactly right. Does that help? Frank From not at available.local Sat Oct 22 05:47:41 2016 From: not at available.local (andy) Date: Sat, 22 Oct 2016 11:47:41 +0200 Subject: Internet Data Handling =?iso-8859-1?B?uw==?= mailbox References: Message-ID: <4d76b$580b35bd$548175db$1672@news1.surfino.com> Fri, 21 Oct 2016 22:43:55 -0400 wrote Adam Jensen: > The mailbox library documentation seems to be a little weak. In this > example: > > https://docs.python.org/2.7/library/mailbox.html#examples > > import mailbox for message in mailbox.mbox('~/mbox'): > subject = message['subject'] # Could possibly be None. > if subject and 'python' in subject.lower(): > print subject > > What is the structure of "message"? I guess it's a dictionary but what > is its structure? What are the keys? Is the structure constant or does > it change depending upon the content of the mbox? > > I'm a bit new to python documentation. How would a potential user of the > "mailbox" library determine these things? Or did I miss something? I would type: help(mailbox) after importing it. best regards andy From bgailer at gmail.com Sat Oct 22 07:07:45 2016 From: bgailer at gmail.com (Bob Gailer) Date: Sat, 22 Oct 2016 07:07:45 -0400 Subject: Obtain javascript result In-Reply-To: <08cc64fa-5567-4de3-aefc-ad5931199b51@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> <883b6baf-b128-4d2b-aa4d-1db578e1a16b@googlegroups.com> <08cc64fa-5567-4de3-aefc-ad5931199b51@googlegroups.com> Message-ID: On Oct 22, 2016 12:45 AM, wrote: > > Many Thanks to everybody. You're welcome. It's fun to help in a challenging problem. Please let us know what solution(s), if any, you have adopted, what problems, if any, you've encountered, what Python version and operating system. From chenyong20000 at gmail.com Sat Oct 22 08:26:04 2016 From: chenyong20000 at gmail.com (chenyong20000 at gmail.com) Date: Sat, 22 Oct 2016 05:26:04 -0700 (PDT) Subject: function call questions In-Reply-To: References: <8a0a7843-9b7c-4543-83b0-cd6ebd5c8fc3@googlegroups.com> <26abbecc-a7eb-4544-8bf1-5df8ec458fe9@googlegroups.com> <17e0008e-697e-49c5-96b7-137e48b85a6f@googlegroups.com> <5506e4d8-bd1d-4e56-8d1b-f71fa8293393@googlegroups.com> <01cfd810-0561-40b1-a834-95a73dad6e56@googlegroups.com> <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> Message-ID: <9c91a4cf-1f3e-43b3-b75c-afc96b0b406e@googlegroups.com> ? 2016?10?22???? UTC+8??5:06:22?Frank Millman??? > wrote in message > news:2853d778-857e-46fc-96a0-8d164c098499 at googlegroups.com... > > ? 2016?10?20???? UTC+8??11:04:38?Frank Millman??? > > wrote in message > > news:01cfd810-0561-40b1-a834-95a73dad6e56 at googlegroups.com... > > > > Hi Frank, > > > > thanks for your kind help. What confused me is at this line: > > > > >>> r = r.setdefault('b', {}) > > > > and its previous one > > > > >>> r = r.setdefault('a', {}) > > > > When r.setdefault('a',{}) is run, I understand it will return an empty {}. > > At this time both r & t reference to {'a':{}}, right? So when "r = > > r.setdefault('a',{})" is run, r reference to {} while t keeps the same as > > {'a':{}}. > > > > then comes r.setdefault('b',{}). What hinder me is here. since r has > > changed its reference to {}, r.setdefault('b',{}) will return {} again. So > > what does this done to t? why at this time t changes to {'a':'b':{}}? > > Sorry for my silly here. Thanks > > > > I don't know if you have been following the other posts in this thread. > > There were some posts from Anssi Saari who was also confused by this, but > then yesterday he sent a post saying that he has now 'got it'. > > I don't think I can do better than quote his explanation of what is > happening - > > """ > OK, so what happens is that now t references the dictionary with {'a': {}} > and r references the empty dict inside that. > > So when we assign to r again, it's the empty dict inside t (the one accessed > by key 'a') that changes to {'b': {}} and t becomes {'a': {'b': {}}}. > """ > > That is exactly right. > > Does that help? > > Frank Hi Frank, I have read Anssi's post already before I sent the post. To be frankly, I can't understand why he got the right answer. I'm sorry for my silly. "So when we assign to r again, it's the empty dict inside t (the one accessed by key 'a')". I do can't understand why this happens. that is the reason why I have asked for this once again and again. There must be some import point I missed but I don't what is it. From frank at chagford.com Sat Oct 22 09:14:29 2016 From: frank at chagford.com (Frank Millman) Date: Sat, 22 Oct 2016 15:14:29 +0200 Subject: function call questions In-Reply-To: <9c91a4cf-1f3e-43b3-b75c-afc96b0b406e@googlegroups.com> References: <8a0a7843-9b7c-4543-83b0-cd6ebd5c8fc3@googlegroups.com> <26abbecc-a7eb-4544-8bf1-5df8ec458fe9@googlegroups.com> <17e0008e-697e-49c5-96b7-137e48b85a6f@googlegroups.com> <5506e4d8-bd1d-4e56-8d1b-f71fa8293393@googlegroups.com> <01cfd810-0561-40b1-a834-95a73dad6e56@googlegroups.com> <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> <9c91a4cf-1f3e-43b3-b75c-afc96b0b406e@googlegroups.com> Message-ID: wrote in message news:9c91a4cf-1f3e-43b3-b75c-afc96b0b406e at googlegroups.com... > I have read Anssi's post already before I sent the post. To be frankly, I can't understand why he got the right answer. I'm sorry for my silly. "So when we assign to r again, it's the empty dict inside t (the one accessed by key 'a')". I do can't understand why this happens. that is the reason why I have asked for this once again and again. There must be some import point I missed but I don't what is it. Let's try this - >>> t = {} >>> r = t >>> r = r.setdefault('a', {}) >>> t {'a': {}} I think you are happy up to this point. We now have three objects - "t" is a dictionary 'a' is a key in the dictionary {} is the value associated with the key 'a' in "t" I think you are happy up to this point. The question is, what is "r"? Before the assignment, "r" was a reference to the dictionary referenced by "t". After the assignment, "r" no longer refers to "t". It is now a reference to the third object listed above, the {} that is the value associated with the key 'a'. >>> t {'a': {}} >>> t['a'] {} >>> r {} >>> t['a] is r True Keep looking at this until it sinks in. "r" and "t['a']" are *the same object*. We just have two ways of accessing it. Try adding some key/values to the empty dictionary - >>> r['x'] = 99 >>> r {'x': 99} >>> t['a'] {'x': 99} >>> t {'a': {'x': 99}} I will pause at this point, and give you a moment to absorb that. Hopefully, the penny will drop and everything will become clear. If not, let us know which of the above steps you do not understand. Good luck - keep plugging away, and you will get there :-) Frank P.S. I assume you understand that the lines prefixed with '>>>' are to be entered while in the python interpreter. It is really important that you type these lines in yourself and examine the results. From gordon at panix.com Sat Oct 22 11:01:46 2016 From: gordon at panix.com (John Gordon) Date: Sat, 22 Oct 2016 15:01:46 +0000 (UTC) Subject: How to handle errors? References: <8500044a-c8d1-43ad-91d9-e836d52bd3f0@googlegroups.com> <5809a443$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: In Wildman writes: > > Another serious problem with using env in the hash-bang line is that you > > cannot pass commandline options to the Python executable. > Not true. I made a test script with this code: > #!/usr/bin/env python > import sys > print sys.argv > Then I ran it: > ~$ python test.py argument1 argument2 > ['test.py', 'argument1', 'argument2'] Options cannot be passed *on the hash-bang line*. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From gerald.britton at gmail.com Sat Oct 22 11:06:21 2016 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 22 Oct 2016 11:06:21 -0400 Subject: Question about abstract base classes and abstract properties -- Python 2.7 Message-ID: I was rereading the 2.7 docs about abstract base classes the other day. I found this line in the usage section of the abc.abstractproperty function: "This defines a read-only property; you can also define a read-write abstract property using the ?long? form of property declaration:" along with an example. so I copied the example and put in a little surrounding code: from abc import ABCMeta, abstractproperty class C(object): __metaclass__ = ABCMeta def getx(self): pass def setx(self, value): pass x = abstractproperty(getx, setx) class D(C): @property def x(self):self._x d = D() print(d) When I ran this, I expected an exception, since I defined a read/write abstract property but only implemented the read operation. However, the example runs fine. That is the class D can be instantiated without error. Of course I cannot set the property since I didn't implement that part. Now, If I don't implement the property at all, I can' instantiate the class. I get: "TypeError: Can't instantiate abstract class D with abstract methods x" which is what I would expect. What I don't understand is why I don't get a similar error when I implement the read operation for the property but not the write operation. If this actually doesn't work (catching the non-implementation at instantiation time), then why is it documented this way. To me at least the doc implies that it *will* raise on the missing write property implementation (otherwise, what's the point of the example?) Is this a doc bug, an ABC bug or just me? (I've been known to be buggy from time to time!) -- Gerald Britton From best_lay at yahoo.com Sat Oct 22 13:57:13 2016 From: best_lay at yahoo.com (Wildman) Date: Sat, 22 Oct 2016 12:57:13 -0500 Subject: How to handle errors? References: <8500044a-c8d1-43ad-91d9-e836d52bd3f0@googlegroups.com> <5809a443$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, 22 Oct 2016 15:01:46 +0000, John Gordon wrote: > In Wildman writes: > >> > Another serious problem with using env in the hash-bang line is that you >> > cannot pass commandline options to the Python executable. > >> Not true. I made a test script with this code: > >> #!/usr/bin/env python >> import sys >> print sys.argv > >> Then I ran it: > >> ~$ python test.py argument1 argument2 >> ['test.py', 'argument1', 'argument2'] > > Options cannot be passed *on the hash-bang line*. OK, I misunderstood Steve's statement. I stand corrected. -- GNU/Linux user #557453 "Well, that's quite different. Never mind." -Emily Litella From hanzer at riseup.net Sat Oct 22 19:36:18 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 22 Oct 2016 19:36:18 -0400 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: References: <85zilxdp2x.fsf@benfinney.id.au> Message-ID: On 10/21/2016 11:45 PM, Ben Finney wrote: > So each instance you're getting has (a superset of) the API of > ``email.message.Message``, which is a lot of behaviour > > including being able to interrogate the message header for its fields, > by name, using the get-an-item ?foo[bar]? syntax. Thanks, the list of functions on that page made it much clearer. From hanzer at riseup.net Sat Oct 22 19:41:45 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 22 Oct 2016 19:41:45 -0400 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: <4d76b$580b35bd$548175db$1672@news1.surfino.com> References: <4d76b$580b35bd$548175db$1672@news1.surfino.com> Message-ID: On 10/22/2016 05:47 AM, andy wrote: > I would type: help(mailbox) after importing it. I guess the output of that might be more meaningful once I understand the underlying structures and conventions. From hanzer at riseup.net Sat Oct 22 19:49:29 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 22 Oct 2016 19:49:29 -0400 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: References: <874m447snh.fsf@handshake.de> Message-ID: On 10/22/2016 03:24 AM, dieter wrote: > In addition to the previous (excellent) responses: > > A "message" models a MIME (RFC1521 Multipurpose Internet Mail Extensions) > message (the international standard for the structure of emails). > The standard tells you that a message consists essentially of two > parts: a set of headers and a body and describes standard headers > and their intended meaning (e.g. "To", "From", "Subject", ...). > It allows a message to contain non-standard headers as well. > > With this knowledge, your "keys" related question can be answered: > there is a (case insensitive) key for each header actually present > in your message. If the message contains several headers with > the same name, the subscription access gives you the first one; > there is an alternative method to access all of them. Thanks. I needed to search for emails to/from a specific person and extract them from a [Google mail archive][1]. [1]: https://takeout.google.com/settings/takeout This is my quick and dirty little one-shot script to get the job done. search_mbox.py -------------------------------------------------------------- #!/usr/bin/env python2.7 import mailbox import sys name = sys.argv[2].lower() for message in mailbox.mbox(sys.argv[1]): if message.has_key("From") and message.has_key("To"): addrs = message.get_all("From") addrs.extend(message.get_all("To")) for addr in addrs: addrl = addr.lower() if addrl.find(name) > 0: print message break -------------------------------------------------------------- Usage: ./search_mbox.py archive.mbox hanzer > hanzer.mbox From hanzer at riseup.net Sat Oct 22 19:52:26 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 22 Oct 2016 19:52:26 -0400 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: References: <33b7ab59-0344-0b42-cbdc-c9e8e88bd32f@mrabarnett.plus.com> Message-ID: On 10/21/2016 11:22 PM, MRAB wrote: > The docs say that it's subclass of the email.message module's Message. > > You can get the email's header fields like it's a dict, except that the > field names are case-insensitive. The author(s) of the module couldn't > use a true dict because of the need for additional functionality. I've only looked at python once or twice in the last ten years, and I jumped into a little project last night with almost no preparation. Thanks for helping out! From rosuav at gmail.com Sat Oct 22 19:57:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 10:57:46 +1100 Subject: Odd name shadowing in comprehension Message-ID: This surprised me. Python 3.4.2 (default, Oct 8 2014, 10:45:20) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> y=6 >>> [(x,y) for x in range(y) for y in range(3)] [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2), (5, 0), (5, 1), (5, 2)] >>> [(x,y) for x in range(3) for z in range(y) for y in range(3)] Traceback (most recent call last): File "", line 1, in File "", line 1, in UnboundLocalError: local variable 'y' referenced before assignment Normally, a comprehension is described as being equivalent to an unrolled loop, inside a nested function. That would be like this: def temp(): ret = [] for x in range(y): for y in range(3): ret.append((x,y)) return ret temp() But it seems that the first iterator (and only that one) is evaluated in the parent context: def temp(iter): ret = [] for x in iter: for y in range(3): ret.append((x, y)) return ret temp(iter(range(y))) Why is this? It seems rather curious. ChrisA From tjreedy at udel.edu Sat Oct 22 20:43:51 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 22 Oct 2016 20:43:51 -0400 Subject: Odd name shadowing in comprehension In-Reply-To: References: Message-ID: On 10/22/2016 7:57 PM, Chris Angelico wrote: > This surprised me. > > Python 3.4.2 (default, Oct 8 2014, 10:45:20) > [GCC 4.9.1] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> y=6 >>>> [(x,y) for x in range(y) for y in range(3)] > [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, > 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2), (5, 0), (5, 1), > (5, 2)] >>>> [(x,y) for x in range(3) for z in range(y) for y in range(3)] > Traceback (most recent call last): > File "", line 1, in > File "", line 1, in > UnboundLocalError: local variable 'y' referenced before assignment > > Normally, a comprehension is described as being equivalent to an > unrolled loop, inside a nested function. That would be like this: > > def temp(): > ret = [] > for x in range(y): > for y in range(3): > ret.append((x,y)) > return ret > temp() This would make the first example fail, which would not be nice. > But it seems that the first iterator (and only that one) is evaluated > in the parent context: Because the first iterator *can* always be evaluated. > def temp(iter): > ret = [] > for x in iter: > for y in range(3): > ret.append((x, y)) > return ret > temp(iter(range(y))) > > Why is this? It seems rather curious. Guido explained this somewhere some time ago. Not sure it is documented very well. In general, subordinate clauses depend on the initial loop variable, hence cannot be evaluated. -- Terry Jan Reedy From eryksun at gmail.com Sat Oct 22 20:51:50 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 23 Oct 2016 00:51:50 +0000 Subject: Odd name shadowing in comprehension In-Reply-To: References: Message-ID: On Sat, Oct 22, 2016 at 11:57 PM, Chris Angelico wrote: > > Normally, a comprehension is described as being equivalent to an > unrolled loop, inside a nested function. > ... > But it seems that the first iterator (and only that one) is evaluated > in the parent context: > > Why is this? It seems rather curious. It matches the behavior of generator expressions, for which Guido gives the following example, as quoted in PEP 289: Consider sum(x for x in foo()). Now suppose there's a bug in foo() that raises an exception, and a bug in sum() that raises an exception before it starts iterating over its argument. Which exception would you expect to see? I'd be surprised if the one in sum() was raised rather the one in foo(), since the call to foo() is part of the argument to sum(), and I expect arguments to be processed before the function is called. From rosuav at gmail.com Sat Oct 22 21:26:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 12:26:08 +1100 Subject: Odd name shadowing in comprehension In-Reply-To: References: Message-ID: On Sun, Oct 23, 2016 at 11:43 AM, Terry Reedy wrote: >> Normally, a comprehension is described as being equivalent to an >> unrolled loop, inside a nested function. That would be like this: >> >> def temp(): >> ret = [] >> for x in range(y): >> for y in range(3): >> ret.append((x,y)) >> return ret >> temp() > > > This would make the first example fail, which would not be nice. > Actually, I discovered this while trying to prove that order of the loops in a comprehension was an easily-detected problem - that if you got them wrong, you'd get a quick UnboundLocalError. While this is true if you have three loops, it's NOT true of the one that's evaluated first. (This in relation to the python-ideas thread about wanting to be able to switch the order. Why bother, if getting it wrong gives instant feedback?) It's a bit of an edge case, as created by the nested function. Lambda functions don't have this issue, because they can't assign to anything other than the parameters (which are up front); genexp and comprehension functions, however, have internal assignments caused by the 'for' loops. As such, it's possible to trigger UnboundLocalError - *except* in that very first loop iterable, which is now evaluated as a parameter. Needs to be documented somewhere, methinks. ChrisA From rosuav at gmail.com Sat Oct 22 21:28:08 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 12:28:08 +1100 Subject: Odd name shadowing in comprehension In-Reply-To: References: Message-ID: On Sun, Oct 23, 2016 at 11:51 AM, eryk sun wrote: > It matches the behavior of generator expressions, for which Guido > gives the following example, as quoted in PEP 289: > > Consider sum(x for x in foo()). Now suppose there's a bug in foo() > that raises an exception, and a bug in sum() that raises an > exception before it starts iterating over its argument. Which > exception would you expect to see? I'd be surprised if the one in > sum() was raised rather the one in foo(), since the call to foo() > is part of the argument to sum(), and I expect arguments to be > processed before the function is called. Fair enough, except that a generator expression is syntactic sugar for a generator function, and the return value of a generator function is a generator object that hasn't yet been started. So where the boundary is, then, is a bit of a fuzzy line. Thanks for digging that up, at least. ChrisA From eryksun at gmail.com Sat Oct 22 22:15:32 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 23 Oct 2016 02:15:32 +0000 Subject: Odd name shadowing in comprehension In-Reply-To: References: Message-ID: On Sun, Oct 23, 2016 at 1:28 AM, Chris Angelico wrote: > > Fair enough, except that a generator expression is syntactic sugar for > a generator function, and the return value of a generator function is > a generator object that hasn't yet been started. So where the boundary > is, then, is a bit of a fuzzy line. I meant the behavior seems to have been copied to align with generator expressions, even though the cited rationale doesn't apply. I'm not saying this is wrong. It's useful that the expression for the outer iterator is evaluated in the defining scope. However, it's only documented for generator expressions, in 6.2.8. The documentation for comprehensions in 6.2.4 makes no mention of it. Actually, it states without qualification that a comprehension is evaluated in a separate scope, which could be misleading: >>> class A: ... a = [x for x in range(locals().setdefault('y', 2))] ... >>> A.y 2 From steve+python at pearwood.info Sat Oct 22 22:42:13 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 23 Oct 2016 13:42:13 +1100 Subject: Odd name shadowing in comprehension References: Message-ID: <580c2386$0$1602$c3e8da3$5496439d@news.astraweb.com> On Sun, 23 Oct 2016 10:57 am, Chris Angelico wrote: > This surprised me. > > Python 3.4.2 (default, Oct 8 2014, 10:45:20) > [GCC 4.9.1] on linux > Type "help", "copyright", "credits" or "license" for more information. >>>> y=6 >>>> [(x,y) for x in range(y) for y in range(3)] > [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, > 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2), (5, 0), (5, 1), > (5, 2)] That surprises me too. I wouldn't have expected that y is both global and local to the comprehension at the same time. I think that this is a bug in list comprehensions and should at least give a warning that y is being used as both local and non-local. This is what happens if you try to access a local before it exists: py> [(y, x) for x in (1, y) for y in (10, 20)] Traceback (most recent call last): File "", line 1, in NameError: name 'y' is not defined Swap the order, and it works: py> [(y, x) for y in (10, 20) for x in (1, y)] [(10, 1), (10, 10), (20, 1), (20, 20)] But if y happens to exist as a global, the first version mysteriously works! py> y = 999 py> [(y, x) for x in (1, y) for y in (10, 20)] [(10, 1), (20, 1), (10, 999), (20, 999)] Let's expand the list comp to a function: y = 999 def list_comp(): result = [] for x in (1, y): for y in (10, 20): result.append((y, x)) return result Calling that function gives a NameError, specifically: UnboundLocalError: local variable 'y' referenced before assignment > Normally, a comprehension is described as being equivalent to an > unrolled loop, inside a nested function. I don't think that's quite the right description, but something like that. > That would be like this: > > def temp(): > ret = [] > for x in range(y): > for y in range(3): > ret.append((x,y)) > return ret > temp() Indeed. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 22 23:14:37 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 23 Oct 2016 14:14:37 +1100 Subject: Odd name shadowing in comprehension References: Message-ID: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> On Sun, 23 Oct 2016 11:43 am, Terry Reedy wrote: > On 10/22/2016 7:57 PM, Chris Angelico wrote: >> This surprised me. >> >> Python 3.4.2 (default, Oct 8 2014, 10:45:20) >> [GCC 4.9.1] on linux >> Type "help", "copyright", "credits" or "license" for more information. >>>>> y=6 >>>>> [(x,y) for x in range(y) for y in range(3)] >> [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, >> 2), (3, 0), (3, 1), (3, 2), (4, 0), (4, 1), (4, 2), (5, 0), (5, 1), >> (5, 2)] >>>>> [(x,y) for x in range(3) for z in range(y) for y in range(3)] >> Traceback (most recent call last): >> File "", line 1, in >> File "", line 1, in >> UnboundLocalError: local variable 'y' referenced before assignment >> >> Normally, a comprehension is described as being equivalent to an >> unrolled loop, inside a nested function. That would be like this: >> >> def temp(): >> ret = [] >> for x in range(y): >> for y in range(3): >> ret.append((x,y)) >> return ret >> temp() > > This would make the first example fail, which would not be nice. > > >> But it seems that the first iterator (and only that one) is evaluated >> in the parent context: > > Because the first iterator *can* always be evaluated. I don't understand what you mean by that. If I take you literally, it is obviously not true: py> [x for x in garglebarblewarble] Traceback (most recent call last): File "", line 1, in NameError: name 'garglebarblewarble' is not defined but I'm sure you know that, so I don't understand what you mean by "always". According to the normal Python scoping rules[1], variables only come from a single scope at a time. Lua has different rules: translating into Python, Lua functions work like this: x = 'global' def foo(): print x # here, x will be the global x x = 'local' print x # but now it is the local x and foo() will print "global" then "local". But according to Python's scoping rules, foo must raise NameError, specifically UnboundLocalError: local variable 'x' referenced before assignment So it seems strange that a little bit of Lua's behaviour has crept into list comprehensions. I doubt that's intentional. There's definitely something strange going on. Compare the what happens when the semi-global variable is in the first loop iterable versus the second loop iterable. In this first example, y refers to both the global and the local, yet strangely there's no error: py> y = 999 py> [(y, z, x) for x in (1, y) for z in (10, 20) for y in (100,)] [(100, 10, 1), (100, 20, 1), (100, 10, 999), (100, 20, 999)] but if we move the reference to y into the second loop, the usual rule about undefined local variables is used: py> [(y, z, x) for x in (1, 2) for z in (10, y) for y in (100,)] Traceback (most recent call last): File "", line 1, in File "", line 1, in UnboundLocalError: local variable 'y' referenced before assignment Of course there's no problem with accessing globals in the second loop, so long as the name doesn't clash with a local: py> Y = 999 py> [(y, z, x) for x in (1, 2) for z in (10, Y) for y in (100,)] [(100, 10, 1), (100, 999, 1), (100, 10, 2), (100, 999, 2)] [1] Function declarations are *slightly* different, so we can write this: def func(a, b=b) to define a parameter (local variable) "b" that takes its default value from b in the surrounding scope. But that's a declaration, not an expression. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Sat Oct 22 23:24:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 14:24:44 +1100 Subject: Odd name shadowing in comprehension In-Reply-To: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 23, 2016 at 2:14 PM, Steve D'Aprano wrote: >>> But it seems that the first iterator (and only that one) is evaluated >>> in the parent context: >> >> Because the first iterator *can* always be evaluated. > > I don't understand what you mean by that. If I take you literally, it is > obviously not true: > > py> [x for x in garglebarblewarble] > Traceback (most recent call last): > File "", line 1, in > NameError: name 'garglebarblewarble' is not defined > > but I'm sure you know that, so I don't understand what you mean by "always". AIUI, what he means is this: 1) Sometimes, all the iterables can be evaluated in advance. dice_2d6 = [a+b for a in range(1,7) for b in range(1,7)] 2) But sometimes, subsequent iterables depend on the outer loop. triangle = [a+b for a in range(1, 7) for b in range(1, a+1)] So in case #2, you cannot evaluate the second range until you're actually in the loop - but in both cases, the first one can be pre-evaluated. Also, the most-outside loop's iterable gets evaluated exactly once, whereas inner loops might be evaluated more often (or less, for that matter). Only in extreme edge cases involving name scoping can the evaluation of the first iterable depend on whether it's inside or outside the invisible function. ChrisA From rosuav at gmail.com Sat Oct 22 23:26:58 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 14:26:58 +1100 Subject: Odd name shadowing in comprehension In-Reply-To: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> References: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 23, 2016 at 2:14 PM, Steve D'Aprano wrote: > There's definitely something strange going on. Compare the what happens when > the semi-global variable is in the first loop iterable versus the second > loop iterable. In this first example, y refers to both the global and the > local, yet strangely there's no error: > Right. Disassembly shows that it's something like this: y = 6 val1 = [(x,y) for x in range(y) for y in range(3)] def temp(iter): ret = [] for x in iter: for y in range(3): ret.append((x, y)) return ret val2 = temp(iter(range(y))) That first iterable is evaluated in the outer context, then passed as a parameter. ChrisA From steve+python at pearwood.info Sat Oct 22 23:44:21 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 23 Oct 2016 14:44:21 +1100 Subject: Odd name shadowing in comprehension References: Message-ID: <580c3216$0$1621$c3e8da3$5496439d@news.astraweb.com> On Sun, 23 Oct 2016 11:51 am, eryk sun wrote: > On Sat, Oct 22, 2016 at 11:57 PM, Chris Angelico wrote: >> >> Normally, a comprehension is described as being equivalent to an >> unrolled loop, inside a nested function. >> ... >> But it seems that the first iterator (and only that one) is evaluated >> in the parent context: >> >> Why is this? It seems rather curious. > > It matches the behavior of generator expressions, for which Guido > gives the following example, as quoted in PEP 289: > > Consider sum(x for x in foo()). Now suppose there's a bug in foo() > that raises an exception, and a bug in sum() that raises an > exception before it starts iterating over its argument. Which > exception would you expect to see? I'd be surprised if the one in > sum() was raised rather the one in foo(), since the call to foo() > is part of the argument to sum(), and I expect arguments to be > processed before the function is called. I don't think that this is relevant. Or rather, it might explain why the behaviour is the way it is, but it isn't justification for it. Here's a link to the PEP, showing the quote in context: https://www.python.org/dev/peps/pep-0289/#early-binding-versus-late-binding But this isn't a question about early or late binding, it is asking why the variable y is treated as both global and local in the same comprehension. This may be an unexpected side-effect of other choices, but I don't see any discussion or consideration of this specific issue. [Aside: Guido's quote in the PEP is unsourced; there's a reference given, but it goes to a different email from Guido, not one that includes the claimed explanation.] -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From jsf80238 at gmail.com Sat Oct 22 23:56:40 2016 From: jsf80238 at gmail.com (Jason Friedman) Date: Sat, 22 Oct 2016 21:56:40 -0600 Subject: =?UTF-8?Q?Re=3A_Internet_Data_Handling_=C2=BB_mailbox?= In-Reply-To: References: <874m447snh.fsf@handshake.de> Message-ID: > > for message in mailbox.mbox(sys.argv[1]): > if message.has_key("From") and message.has_key("To"): > addrs = message.get_all("From") > addrs.extend(message.get_all("To")) > for addr in addrs: > addrl = addr.lower() > if addrl.find(name) > 0: > print message > break > ------------------------------------------------------------- I usually see if addrl.find(name) > 0: written as if name in addrl: From walters.justin01 at gmail.com Sun Oct 23 00:47:38 2016 From: walters.justin01 at gmail.com (justin walters) Date: Sat, 22 Oct 2016 21:47:38 -0700 Subject: New REST API Library for Django Message-ID: Hi everyone! I am working on a new REST API library for Django. You can see it on: Github: https://github.com/FFX01/django-restup PyPI: https://pypi.python.org/pypi?name=django-restup&version=0.1.1&:action=display I know there is already a couple great packages out there for REST API development with Django. However, I found them to be difficult to use for very granular data manipulation and permissions for my most recent project. So, I set out to make that a bit easier for myself. I am not using this package in production anywhere as I just released this pre-alpha today for people to mess around with if they want. I just want to see what you all think. I could use any criticism, feedback, and ideas you may have for me. If you want to contribute, that would be even better! This is my first time putting something out on PyPI. It's also my first "serious" open source project. Let me know what you would like to see from a REST API library and I can add it to the schedule. Thank you, - Justin From eryksun at gmail.com Sun Oct 23 01:08:01 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 23 Oct 2016 05:08:01 +0000 Subject: Odd name shadowing in comprehension In-Reply-To: <580c3216$0$1621$c3e8da3$5496439d@news.astraweb.com> References: <580c3216$0$1621$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 23, 2016 at 3:44 AM, Steve D'Aprano wrote: > > https://www.python.org/dev/peps/pep-0289/#early-binding-versus-late-binding > > But this isn't a question about early or late binding, it is asking why the > variable y is treated as both global and local in the same comprehension. > This may be an unexpected side-effect of other choices, but I don't see any > discussion or consideration of this specific issue. For generator expressions, it's about early binding of the outer iterator. This makes the result of the expression for the outer iterator behave like a function parameter. Obviously it has to be evaluated in the defining scope. Comprehensions have adopted the same behavior, and it's not a bug or a flawed design. However it does need to be documented better in section 6.2.4. If the expression for the outer iterator is `range(y)`, the compiler emits whatever operation is required to load y in the defining scope. At the module or class level it emits a LOAD_NAME. At function scope, it uses LOAD_FAST when y is local, LOAD_DEREF when y is a free variable, and otherwise LOAD_GLOBAL. The interpreter evaluates the expression (e.g. it calls range) and executes GET_ITER on the result. The resulting iterator is passed as parameter ".0" to the [generator] function that implements the comprehension or generator expression. From tjreedy at udel.edu Sun Oct 23 01:16:23 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 23 Oct 2016 01:16:23 -0400 Subject: Odd name shadowing in comprehension In-Reply-To: References: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/22/2016 11:24 PM, Chris Angelico wrote: > On Sun, Oct 23, 2016 at 2:14 PM, Steve D'Aprano > wrote: >>>> But it seems that the first iterator (and only that one) is evaluated >>>> in the parent context: >>> >>> Because the first iterator *can* always be evaluated. >> >> I don't understand what you mean by that. If I take you literally, it is >> obviously not true: >> >> py> [x for x in garglebarblewarble] >> Traceback (most recent call last): >> File "", line 1, in >> NameError: name 'garglebarblewarble' is not defined >> >> but I'm sure you know that, so I don't understand what you mean by "always". > > AIUI, what he means is this: Right. You put it better than I did. > 1) Sometimes, all the iterables can be evaluated in advance. > > dice_2d6 = [a+b for a in range(1,7) for b in range(1,7)] > > 2) But sometimes, subsequent iterables depend on the outer loop. > > triangle = [a+b for a in range(1, 7) for b in range(1, a+1)] > > So in case #2, you cannot evaluate the second range until you're > actually in the loop - but in both cases, the first one can be > pre-evaluated. Also, the most-outside loop's iterable gets evaluated > exactly once, whereas inner loops might be evaluated more often (or > less, for that matter). > > Only in extreme edge cases involving name scoping can the evaluation > of the first iterable depend on whether it's inside or outside the > invisible function. > > ChrisA > -- Terry Jan Reedy From rosuav at gmail.com Sun Oct 23 01:47:44 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 23 Oct 2016 16:47:44 +1100 Subject: Odd name shadowing in comprehension In-Reply-To: References: <580c3216$0$1621$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 23, 2016 at 4:08 PM, eryk sun wrote: > For generator expressions, it's about early binding of the outer > iterator. This makes the result of the expression for the outer > iterator behave like a function parameter. Obviously it has to be > evaluated in the defining scope. Comprehensions have adopted the same > behavior, and it's not a bug or a flawed design. However it does need > to be documented better in section 6.2.4. It may not be flawed design, but it's certainly unintuitive, given the normal explanation: gen = ((x,y) for x in range(y) for y in range(z)) def g(): for x in range(y): for y in range(z): yield (x,y) gen = g() It's actually this, which is definitely unintuitive: def g(it): for x in it: for y in range(z): yield (x,y) gen = g(iter(range(y))) Do you ever see them explained like that? Me neither. So if it's intentional, it needs to be explained somewhere, unless the assumption is that it'll almost never matter. ChrisA From steve+python at pearwood.info Sun Oct 23 02:15:29 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 23 Oct 2016 17:15:29 +1100 Subject: Odd name shadowing in comprehension References: Message-ID: <580c5583$0$1604$c3e8da3$5496439d@news.astraweb.com> On Sun, 23 Oct 2016 01:15 pm, eryk sun wrote: > I meant the behavior seems to have been copied to align with generator > expressions, even though the cited rationale doesn't apply. I'm not > saying this is wrong. It's useful that the expression for the outer > iterator is evaluated in the defining scope. However, it's only > documented for generator expressions, in 6.2.8. The documentation for > comprehensions in 6.2.4 makes no mention of it. 6.2.8? 6.2.4? What are these references to? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sun Oct 23 02:26:08 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 23 Oct 2016 17:26:08 +1100 Subject: Odd name shadowing in comprehension References: <580c2b1f$0$1593$c3e8da3$5496439d@news.astraweb.com> Message-ID: <580c5802$0$22140$c3e8da3$5496439d@news.astraweb.com> On Sun, 23 Oct 2016 02:24 pm, Chris Angelico wrote: > 1) Sometimes, all the iterables can be evaluated in advance. > > dice_2d6 = [a+b for a in range(1,7) for b in range(1,7)] > > 2) But sometimes, subsequent iterables depend on the outer loop. > > triangle = [a+b for a in range(1, 7) for b in range(1, a+1)] > > So in case #2, you cannot evaluate the second range until you're > actually in the loop Obviously not. > - but in both cases, the first one can be pre-evaluated. That doesn't follow. Consider: gen = (x for x in [time.time()]) Should next(gen) return the time that the generator expression was created, or the time when you first call next()? The answer to that depends on whether you want early binding or late binding. If that's not clear, consider this instead: y = 'first' gen = (x for x in [y]) y = 'second' What will next(gen) return? With *early binding*, it will return "first". This is how function parameter defaults work. With *late binding*, it will return "second", which is how access to globals normally work. But that's a separate issue to the question of what happens if y is a local variable: gen = (x for x in [y] for y in [999]) I can see how it happens and why it happens, but I think it is still weird and deserves at least a warning. > Also, the most-outside loop's iterable gets evaluated > exactly once, whereas inner loops might be evaluated more often (or > less, for that matter). Again, that's quite straight forward. > Only in extreme edge cases involving name scoping can the evaluation > of the first iterable depend on whether it's inside or outside the > invisible function. I don't think that this is an extreme edge case. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From pic8690 at gmail.com Sun Oct 23 03:10:35 2016 From: pic8690 at gmail.com (pic8690 at gmail.com) Date: Sun, 23 Oct 2016 00:10:35 -0700 (PDT) Subject: multiprocess passing arguments double asterisks Message-ID: import multiprocessing as mp def bar(**kwargs): for a in kwargs: print a,kwargs[a] arguments={'name':'Joe','age':20} p=mp.Pool(processes=4) p.map(bar,**arguments) p.close() p.join() Errors: Traceback (most recent call last): File "post.py", line 9, in p.map(bar,**arguments) TypeError: map() got an unexpected keyword argument 'age' How do I pass the dictionary arguments? From pic8690 at gmail.com Sun Oct 23 03:12:10 2016 From: pic8690 at gmail.com (pic8690 at gmail.com) Date: Sun, 23 Oct 2016 00:12:10 -0700 (PDT) Subject: multiprocess passing arguments double asterisks Message-ID: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> import multiprocessing as mp def bar(**kwargs): for a in kwargs: print a,kwargs[a] arguments={'name':'Joe','age':20} p=mp.Pool(processes=4) p.map(bar,**arguments) p.close() p.join() Errors: Traceback (most recent call last): File "post.py", line 9, in p.map(bar,**arguments) TypeError: map() got an unexpected keyword argument 'age' How do I pass the dictionary arguments? From not at available.local Sun Oct 23 03:33:55 2016 From: not at available.local (andy) Date: Sun, 23 Oct 2016 09:33:55 +0200 Subject: Internet Data Handling =?iso-8859-1?B?uw==?= mailbox References: <4d76b$580b35bd$548175db$1672@news1.surfino.com> Message-ID: <185da$580c67e3$54817004$7491@news1.surfino.com> Sat, 22 Oct 2016 19:41:45 -0400 wrote Adam Jensen: > On 10/22/2016 05:47 AM, andy wrote: >> I would type: help(mailbox) after importing it. > > I guess the output of that might be more meaningful once I understand > the underlying structures and conventions. yes - you are right. fortunatelly python autors have thought about 'documntation strings' and 'coding style', the syntax of python itself helps reading source code (indentation). this allows using auto- documentation features like help(...). when i don't know enough about a module like 'mailbox' , i first try a file search for the source code on the local system: i.e. 'locate mailbox.py' on a linux system. possibly i have to install the module first when there is nothing found (using pip or package manager). this yields on my system ('sudo updatedb' - for updating the db) to this result: /usr/lib/python2.7/mailbox.py /usr/lib/python2.7/mailbox.pyc /usr/lib/python3.5/mailbox.py i can read the source file with: 'less /usr/lib/python3.5/mailbox.py'. within the sourcefile i can study the imports and data structures. Other sources of information: doc.python.org - even with search: https://docs.python.org/3/search.html?q=mailbox and finally all these mail-modules should follow the RFCs ;-) https://tools.ietf.org/html/rfc2822 best regards andy From eryksun at gmail.com Sun Oct 23 05:56:46 2016 From: eryksun at gmail.com (eryk sun) Date: Sun, 23 Oct 2016 09:56:46 +0000 Subject: Odd name shadowing in comprehension In-Reply-To: <580c5583$0$1604$c3e8da3$5496439d@news.astraweb.com> References: <580c5583$0$1604$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 23, 2016 at 6:15 AM, Steve D'Aprano wrote: > On Sun, 23 Oct 2016 01:15 pm, eryk sun wrote: > >> I meant the behavior seems to have been copied to align with generator >> expressions, even though the cited rationale doesn't apply. I'm not >> saying this is wrong. It's useful that the expression for the outer >> iterator is evaluated in the defining scope. However, it's only >> documented for generator expressions, in 6.2.8. The documentation for >> comprehensions in 6.2.4 makes no mention of it. > > 6.2.8? 6.2.4? What are these references to? They're section numbers in the Python 3 language reference. Chapter 6 covers expressions. https://docs.python.org/3/reference/expressions From tomuxiong at gmx.com Sun Oct 23 08:51:58 2016 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Sun, 23 Oct 2016 08:51:58 -0400 Subject: multiprocess passing arguments double asterisks In-Reply-To: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> Message-ID: <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> On 10/23/2016 03:12 AM, pic8690 at gmail.com wrote: > import multiprocessing as mp > > def bar(**kwargs): > for a in kwargs: > print a,kwargs[a] > > arguments={'name':'Joe','age':20} > p=mp.Pool(processes=4) > p.map(bar,**arguments) > p.close() > p.join() What are you trying to do? The map method is similar to the map built-in: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.pool.multiprocessing.Pool.map https://docs.python.org/2/library/functions.html#map map(function, iterable, ...) Apply function to every item of iterable and return a list of the results... You can't apply it to keyword arguments like this. There are some different SO threads talking about this sort of thing: http://stackoverflow.com/questions/13499824/using-python-map-function-with-keyword-arguments http://stackoverflow.com/questions/10212445/python-map-list-item-to-function-with-arguments http://stackoverflow.com/questions/16874244/python-map-and-arguments-unpacking Maybe those (especially the last one) are helpful. Cheers, Thomas From chenyong20000 at gmail.com Sun Oct 23 09:27:47 2016 From: chenyong20000 at gmail.com (chenyong20000 at gmail.com) Date: Sun, 23 Oct 2016 06:27:47 -0700 (PDT) Subject: function call questions In-Reply-To: References: <8a0a7843-9b7c-4543-83b0-cd6ebd5c8fc3@googlegroups.com> <26abbecc-a7eb-4544-8bf1-5df8ec458fe9@googlegroups.com> <17e0008e-697e-49c5-96b7-137e48b85a6f@googlegroups.com> <5506e4d8-bd1d-4e56-8d1b-f71fa8293393@googlegroups.com> <01cfd810-0561-40b1-a834-95a73dad6e56@googlegroups.com> <2853d778-857e-46fc-96a0-8d164c098499@googlegroups.com> <9c91a4cf-1f3e-43b3-b75c-afc96b0b406e@googlegroups.com> Message-ID: ? 2016?10?22???? UTC+8??9:15:06?Frank Millman??? > wrote in message > news:9c91a4cf-1f3e-43b3-b75c-afc96b0b406e at googlegroups.com... > > > I have read Anssi's post already before I sent the post. To be frankly, I > can't understand why he got the right answer. I'm sorry for my silly. "So > when we assign to r again, it's the empty dict inside t (the one accessed > by key 'a')". I do can't understand why this happens. that is the reason why > I have asked for this once again and again. There must be some import point > I missed but I don't what is it. > > Let's try this - > > >>> t = {} > >>> r = t > >>> r = r.setdefault('a', {}) > >>> t > {'a': {}} > > I think you are happy up to this point. > > We now have three objects - > > "t" is a dictionary > 'a' is a key in the dictionary > {} is the value associated with the key 'a' in "t" > > I think you are happy up to this point. > > The question is, what is "r"? > > Before the assignment, "r" was a reference to the dictionary referenced by > "t". > > After the assignment, "r" no longer refers to "t". It is now a reference to > the > third object listed above, the {} that is the value associated with the key > 'a'. > > >>> t > {'a': {}} > >>> t['a'] > {} > >>> r > {} > >>> t['a] is r > True > > Keep looking at this until it sinks in. "r" and "t['a']" are *the same > object*. We just have two ways of accessing it. > > Try adding some key/values to the empty dictionary - > > >>> r['x'] = 99 > >>> r > {'x': 99} > >>> t['a'] > {'x': 99} > >>> t > {'a': {'x': 99}} > > I will pause at this point, and give you a moment to absorb that. > > Hopefully, the penny will drop and everything will become clear. > > If not, let us know which of the above steps you do not understand. > > Good luck - keep plugging away, and you will get there :-) > > Frank > > P.S. I assume you understand that the lines prefixed with '>>>' are to be > entered while in the python interpreter. It is really important that you > type these lines in yourself and examine the results. Hi Frank, I got it this time. Thanks very much for your help. I thought r is an empty dictionary without any connection to t before. Now I know that happened. Thanks. regards skyworld From jcasale at activenetwerx.com Sun Oct 23 09:59:41 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Sun, 23 Oct 2016 13:59:41 +0000 Subject: MySQL connector issue Message-ID: I have some code that I am testing on Windows without c extensions which runs on a RHEL server with c extensions. In a simplified test case as follows: connection = mysql.connector.connect(...) cursor = connection.cursor(cursor_class=MySQLCursorDict) while True: cursor.execute('SELECT foo,biz FROM bar WHERE baz IS NULL) rows = cursor.fetchall() print(rows) cursor.execute('UPDATE bar SET baz=42 WHERE baz IS NULL') connection.commit() sleep(.5) This works on Windows, the select query consistently returns new results as they appear in the database when generated by other applications. However on the RHEL server, the initial select only produces a result on the first iteration and then as new results are written to the database, the select does not find results? What might be the issue? Thanks, jlc From __peter__ at web.de Sun Oct 23 12:14:33 2016 From: __peter__ at web.de (Peter Otten) Date: Sun, 23 Oct 2016 18:14:33 +0200 Subject: MySQL connector issue References: Message-ID: Joseph L. Casale wrote: > I have some code that I am testing on Windows without c extensions which > runs on a RHEL server with c extensions. In a simplified test case as > follows: > > connection = mysql.connector.connect(...) > cursor = connection.cursor(cursor_class=MySQLCursorDict) > while True: > cursor.execute('SELECT foo,biz FROM bar WHERE baz IS NULL) > rows = cursor.fetchall() > print(rows) > > cursor.execute('UPDATE bar SET baz=42 WHERE baz IS NULL') > connection.commit() > > sleep(.5) > > This works on Windows, the select query consistently returns new results > as they appear in the database when generated by other applications. > > However on the RHEL server, the initial select only produces a result on > the first iteration and then as new results are written to the database, > the select does not find results? > > What might be the issue? Perhaps you simplified too much, but changes between the select and the update could be lost. I think you need at least three states: 1 mark rows where baz is null (by setting baz to some value other than NULL or 42, 24, say: set baz = 24 where baz is NULL) 2 show marked rows (select ... where baz = 24) 3 mark rows as seen (set baz = 42 where baz = 24) From jcasale at activenetwerx.com Sun Oct 23 12:46:02 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Sun, 23 Oct 2016 16:46:02 +0000 Subject: MySQL connector issue In-Reply-To: References: Message-ID: <62906fb1b1cc42c580095033f51f01f8@activenetwerx.com> > Perhaps you simplified too much, but changes between the select and the update could be lost. I think you need at least three states: > > 1 mark rows where baz is null (by setting baz to some value other than NULL or 42, 24, say: set baz = 24 where baz is NULL) > 2 show marked rows (select ... where baz = 24) > 3 mark rows as seen (set baz = 42 where baz = 24) Hi Peter, The UPDATE happens only if rows were found, the sleep is much longer as it is expected that another application adds rows satisfying the SELECT. It really is that simple which is why I am baffled. Given the throughput is so low, if I close the cursor and connection at the end of loop and instantiate them both at the start of the loop, it works as expected but that's obviously not optimal. Thanks, jlc From rosuav at gmail.com Sun Oct 23 12:53:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 24 Oct 2016 03:53:12 +1100 Subject: MySQL connector issue In-Reply-To: <62906fb1b1cc42c580095033f51f01f8@activenetwerx.com> References: <62906fb1b1cc42c580095033f51f01f8@activenetwerx.com> Message-ID: On Mon, Oct 24, 2016 at 3:46 AM, Joseph L. Casale wrote: > It really is that simple which is why I am baffled. Given the throughput is so > low, if I close the cursor and connection at the end of loop and instantiate them > both at the start of the loop, it works as expected but that's obviously not optimal. Interesting. Generally, I allocate cursors exactly at the same time as I open transactions; not sure if this works with the mysql connector, but with psycopg2 (PostgreSQL), my code looks like this: with conn, conn.cursor() as cur: cur.execute(...) ... = cur.fetchall() The 'with' block guarantees that (a) the cursor will be closed and all resources freed, and (b) the transaction will be committed or rolled back (on exception, roll back, otherwise commit), at the unindent. It's a nice, clean way to operate. Never had problems with it. ChrisA From eproser at gmail.com Sun Oct 23 14:09:17 2016 From: eproser at gmail.com (eproser at gmail.com) Date: Sun, 23 Oct 2016 11:09:17 -0700 (PDT) Subject: Obtain javascript result In-Reply-To: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> References: <7b27e2ef-4abd-4d90-86b9-2c1c428b74eb@googlegroups.com> Message-ID: <5b51f0f9-bc69-4b9a-bd55-9fa0a9117341@googlegroups.com> Ok, I solved to this way: from bs4 import BeautifulSoup from selenium import webdriver driver = webdriver.Chrome() driver.get('http://www.betexplorer.com/soccer/russia/youth-\league/matchdetails.php?matchid=rLu2Xsdi') pg_src = driver.page_source driver.close() soup = BeautifulSoup(pg_src, 'html.parser') # start from here I do something with soup ... Windows 10 / Python 3.5.2 Thanks From jcasale at activenetwerx.com Sun Oct 23 15:44:53 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Sun, 23 Oct 2016 19:44:53 +0000 Subject: MySQL connector issue In-Reply-To: References: <62906fb1b1cc42c580095033f51f01f8@activenetwerx.com> Message-ID: <684763915d994d8bbf6af83772ad1fc5@activenetwerx.com> > Interesting. Generally, I allocate cursors exactly at the same time as I open transactions; > not sure if this works with the mysql connector, but with psycopg2 (PostgreSQL), my code looks like this: > > with conn, conn.cursor() as cur: > cur.execute(...) > ... = cur.fetchall() > > The 'with' block guarantees that (a) the cursor will be closed and all resources freed, and (b) the > transaction will be committed or rolled back (on exception, roll back, otherwise commit), at the unindent. > It's a nice, clean way to operate. Never had problems with it. Ditto, however the official Oracle python module lacks context managers. Regardless, using try/finally and creating and closing cursors for both the query and the update still did not help. The connection must be reset in order for the query to see new results. That sounds ridiculous, certainly I overlooking something in a connection parameter possibly? Thanks, jlc From rtomek at ceti.pl Sun Oct 23 16:51:58 2016 From: rtomek at ceti.pl (Tomasz Rola) Date: Sun, 23 Oct 2016 22:51:58 +0200 Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? In-Reply-To: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> Message-ID: <20161023205158.GA977@tau1.ceti.pl> On Wed, Oct 05, 2016 at 01:56:59PM -0400, Malcolm Greene wrote: > Looking for a quick way to calculate lines of code/comments in a > collection of Python scripts. This isn't a LOC per day per developer > type analysis - I'm looking for a metric to quickly judge the complexity > of a set of scripts I'm inheriting. > > Thank you, > Malcolm A bit more than what you asked for (and sorry for being late) but I find sloccount quite good. Or at least interesting (computes sloc and some stats about project, given project dir or a single file with code): http://www.dwheeler.com/sloccount/ -- Regards, Tomasz Rola -- ** A C programmer asked whether computer had Buddha's nature. ** ** As the answer, master did "rm -rif" on the programmer's home ** ** directory. And then the C programmer became enlightened... ** ** ** ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** From jon+usenet at unequivocal.eu Sun Oct 23 17:15:19 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sun, 23 Oct 2016 21:15:19 -0000 (UTC) Subject: Internet Data Handling =?UTF-8?Q?=C2=BB?= mailbox References: <874m447snh.fsf@handshake.de> Message-ID: On 2016-10-23, Jason Friedman wrote: >> >> for message in mailbox.mbox(sys.argv[1]): >> if message.has_key("From") and message.has_key("To"): >> addrs = message.get_all("From") >> addrs.extend(message.get_all("To")) >> for addr in addrs: >> addrl = addr.lower() >> if addrl.find(name) > 0: >> print message >> break >> ------------------------------------------------------------- > > I usually see > > if addrl.find(name) > 0: > > written as > > if name in addrl: I suppose technically it would be: iaddrf name in addrl[1:]: From jon+usenet at unequivocal.eu Sun Oct 23 17:15:50 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sun, 23 Oct 2016 21:15:50 -0000 (UTC) Subject: Internet Data Handling =?UTF-8?Q?=C2=BB?= mailbox References: <874m447snh.fsf@handshake.de> Message-ID: On 2016-10-23, Jon Ribbens wrote: > On 2016-10-23, Jason Friedman wrote: >>> >>> for message in mailbox.mbox(sys.argv[1]): >>> if message.has_key("From") and message.has_key("To"): >>> addrs = message.get_all("From") >>> addrs.extend(message.get_all("To")) >>> for addr in addrs: >>> addrl = addr.lower() >>> if addrl.find(name) > 0: >>> print message >>> break >>> ------------------------------------------------------------- >> >> I usually see >> >> if addrl.find(name) > 0: >> >> written as >> >> if name in addrl: > > I suppose technically it would be: > > iaddrf name in addrl[1:]: s/iaddrf/if/ obviously! From alindikris77 at gmail.com Sun Oct 23 17:34:29 2016 From: alindikris77 at gmail.com (chris alindi) Date: Sun, 23 Oct 2016 14:34:29 -0700 (PDT) Subject: exist loop by pressing esc Message-ID: <6e030fd0-93c1-4d23-8656-e06c411b61aa@googlegroups.com> simple while loop range(10) if user press esc exits loop From grant.a.orchard at gmail.com Sun Oct 23 18:46:06 2016 From: grant.a.orchard at gmail.com (grant.a.orchard at gmail.com) Date: Sun, 23 Oct 2016 15:46:06 -0700 (PDT) Subject: Has any one automated the vmware-vra setup using python? In-Reply-To: References: Message-ID: On Thursday, 6 October 2016 04:36:15 UTC+11, Robert Clove wrote: > Not yet. There are a few people working towards it though. Grant From steve+python at pearwood.info Sun Oct 23 21:36:26 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 24 Oct 2016 12:36:26 +1100 Subject: exist loop by pressing esc References: <6e030fd0-93c1-4d23-8656-e06c411b61aa@googlegroups.com> Message-ID: <580d659b$0$1593$c3e8da3$5496439d@news.astraweb.com> On Mon, 24 Oct 2016 08:34 am, chris alindi wrote: > simple while loop range(10) if user press esc exits loop Your post is not a question nor even a grammatical sentence? Would you like us to guess what you mean? Or perhaps you could ask your question in actual proper sentences? If English is not your first language, please say so, and try your best. It may help if you show some code, and explain what result you want. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From best_lay at yahoo.com Mon Oct 24 00:18:50 2016 From: best_lay at yahoo.com (Wildman) Date: Sun, 23 Oct 2016 23:18:50 -0500 Subject: exist loop by pressing esc References: <6e030fd0-93c1-4d23-8656-e06c411b61aa@googlegroups.com> Message-ID: <1O-dnR7fSvY3FpDFnZ2dnUU7-LmdnZ2d@giganews.com> On Sun, 23 Oct 2016 14:34:29 -0700, chris alindi wrote: > simple while loop range(10) if user press esc exits loop If I understand you correctly you want to exit a while loop with the ESC key. That can be done but it depends on the platform. For Windows use this: (not tested) import msvcrt while something: if msvcrt.kbhit() and msvcrt.getch() == chr(27): break In Linux you need to install getch: https://pypi.python.org/pypi/getch Download it here: https://pypi.python.org/pypi/getch#downloads Extract the tar.gz file. A directory called getch-1.0 will be created. Open a terminal in that directory and enter this... sudo python setup.py install Here is a code example: (tested) import getch while something: if getch.getch() == '\x1b': break -- GNU/Linux user #557453 May the Source be with you. From stephen_tucker at sil.org Mon Oct 24 05:03:29 2016 From: stephen_tucker at sil.org (Stephen Tucker) Date: Mon, 24 Oct 2016 10:03:29 +0100 Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? In-Reply-To: <20161023205158.GA977@tau1.ceti.pl> References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> <20161023205158.GA977@tau1.ceti.pl> Message-ID: Tomasz, How about using the command prompt command FIND /C on each of your source files as follows: FIND/C "#" >NumbersOfLinesContainingPythonComments.dat FIND/C /V "#" >NumbersOfLinesNotContainingPythonComments.dat You would end up with two files each with a column of line counts; Import these lines into an Excel Spreadsheet and calculate whatever you like with them. Stephen. On Sun, Oct 23, 2016 at 9:51 PM, Tomasz Rola wrote: > On Wed, Oct 05, 2016 at 01:56:59PM -0400, Malcolm Greene wrote: > > Looking for a quick way to calculate lines of code/comments in a > > collection of Python scripts. This isn't a LOC per day per developer > > type analysis - I'm looking for a metric to quickly judge the complexity > > of a set of scripts I'm inheriting. > > > > Thank you, > > Malcolm > > A bit more than what you asked for (and sorry for being late) but I > find sloccount quite good. Or at least interesting (computes sloc and > some stats about project, given project dir or a single file with > code): > > http://www.dwheeler.com/sloccount/ > > -- > Regards, > Tomasz Rola > > -- > ** A C programmer asked whether computer had Buddha's nature. ** > ** As the answer, master did "rm -rif" on the programmer's home ** > ** directory. And then the C programmer became enlightened... ** > ** ** > ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** > -- > https://mail.python.org/mailman/listinfo/python-list > From gordon at panix.com Mon Oct 24 11:05:54 2016 From: gordon at panix.com (John Gordon) Date: Mon, 24 Oct 2016 15:05:54 +0000 (UTC) Subject: exist loop by pressing esc References: <6e030fd0-93c1-4d23-8656-e06c411b61aa@googlegroups.com> Message-ID: In <6e030fd0-93c1-4d23-8656-e06c411b61aa at googlegroups.com> chris alindi writes: > simple while loop range(10) if user press esc exits loop range() is typically used with for loops, not while loops. what is your while condition? what use is the range() value? -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From pic8690 at gmail.com Mon Oct 24 12:45:03 2016 From: pic8690 at gmail.com (pic8690 at gmail.com) Date: Mon, 24 Oct 2016 09:45:03 -0700 (PDT) Subject: multiprocess passing arguments double asterisks In-Reply-To: References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> Message-ID: <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> On Sunday, October 23, 2016 at 3:44:16 PM UTC-5, Thomas Nyberg wrote: > On 10/23/2016 03:12 AM, pic8690 at gmail.com wrote: > > import multiprocessing as mp > > > > def bar(**kwargs): > > for a in kwargs: > > print a,kwargs[a] > > > > arguments={'name':'Joe','age':20} > > p=mp.Pool(processes=4) > > p.map(bar,**arguments) > > p.close() > > p.join() > > What are you trying to do? The map method is similar to the map built-in: > > https://docs.python.org/2/library/multiprocessing.html#multiprocessing.pool.multiprocessing.Pool.map > https://docs.python.org/2/library/functions.html#map > > map(function, iterable, ...) > Apply function to every item of iterable and return a list of the results... > > You can't apply it to keyword arguments like this. There are some > different SO threads talking about this sort of thing: > > http://stackoverflow.com/questions/13499824/using-python-map-function-with-keyword-arguments > http://stackoverflow.com/questions/10212445/python-map-list-item-to-function-with-arguments > http://stackoverflow.com/questions/16874244/python-map-and-arguments-unpacking > > Maybe those (especially the last one) are helpful. > > Cheers, > Thomas Thanks for the reply. The code snippet given by Peter is not very clear I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value. This example does not return anything Would you please use my example for the map function? I appreciate your help, From tomuxiong at gmx.com Mon Oct 24 13:07:12 2016 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Mon, 24 Oct 2016 13:07:12 -0400 Subject: multiprocess passing arguments double asterisks In-Reply-To: <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> Message-ID: <870c7a5e-e5ee-bc25-087f-ee9cf35a39e3@gmx.com> On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote: > Thanks for the reply. > > The code snippet given by Peter is not very clear > > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value. This example does not return anything > > Would you please use my example for the map function? > > I appreciate your help, > I'm honestly not totally sure what you want to do. However, say you want to do the following (btw this is basically what Dennis said i nhis last email, but maybe I can help clarify): kwargs = {'param1': val1, 'param2': val2}) Then you'd like to have the following two operations performed in separate processes: bar(param1=val1) bar(param2=val2) In that case, I guess I would do something like the following. First define bar_wrapper as follows (*I haven't tested any code here!): def bar_wrapper(pair): key, val = pair return bar(**{key: val}) Then I would probably do something like map(bar_wrapper, kwargs.items()) I.e. basically what I'm doing is taking the key-val pairs and producing a list of them (as tuples). This is something that you can apply map too, but not with the original function. So then the wrapper function converts the tuple back to what you want originally. Hopefully I'm understanding correctly and hopefully this helps. Cheers, Thomas From duncan at invalid.invalid Mon Oct 24 13:36:44 2016 From: duncan at invalid.invalid (duncan smith) Date: Mon, 24 Oct 2016 18:36:44 +0100 Subject: retain dimensions for numpy slice Message-ID: Hello, I have several arrays that I need to combine elementwise in various fashions. They are basically probability tables and there is a mapping of axes to variables. I have code for transposing and reshaping that aligns the variables / axes so the usual broadcasting rules achieve the desired objective. But for a specific application I want to avoid the transposing and reshaping. So I've specified arrays that contain the full dimensionality (dimensions equal to the total number of variables). e.g. Arrays with shape, [1,3,3] and [2,3,1] to represent probability tables with variables [B,C] and [A,B]. One operation that I need that is not elementwise is summing over axes, but I can use numpy.sum with keepdims=True to retain the appropriate shape. The problem I have is with slicing. This drops dimensions. Does anyone know of a solution to this so that I can e.g. take an array with shape [2,3,1] and generate a slice with shape [2,1,1]? I'm hoping to avoid having to manually reshape it. Thanks. Duncan From __peter__ at web.de Mon Oct 24 14:05:47 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 24 Oct 2016 20:05:47 +0200 Subject: retain dimensions for numpy slice References: Message-ID: duncan smith wrote: > Hello, > I have several arrays that I need to combine elementwise in > various fashions. They are basically probability tables and there is a > mapping of axes to variables. I have code for transposing and reshaping > that aligns the variables / axes so the usual broadcasting rules achieve > the desired objective. But for a specific application I want to avoid > the transposing and reshaping. So I've specified arrays that contain the > full dimensionality (dimensions equal to the total number of variables). > e.g. > > Arrays with shape, > > [1,3,3] and [2,3,1] > > to represent probability tables with variables > > [B,C] and [A,B]. > > One operation that I need that is not elementwise is summing over axes, > but I can use numpy.sum with keepdims=True to retain the appropriate > shape. > > The problem I have is with slicing. This drops dimensions. Does anyone > know of a solution to this so that I can e.g. take an array with shape > [2,3,1] and generate a slice with shape [2,1,1]? I'm hoping to avoid > having to manually reshape it. Thanks. Can you clarify your requirement or give an example of what you want? Given an array >>> a.shape (2, 3, 1) you can get a slice with shape (2,1,1) with (for example) >>> a[:,:1,:].shape (2, 1, 1) or even >>> newshape = (2, 1, 1) >>> a[tuple(slice(d) for d in newshape)].shape (2, 1, 1) but that's probably not what you are asking for... From jladasky at itu.edu Mon Oct 24 14:14:05 2016 From: jladasky at itu.edu (jladasky at itu.edu) Date: Mon, 24 Oct 2016 11:14:05 -0700 (PDT) Subject: Why doesn't Python include non-blocking keyboard input function? Message-ID: After reading this rather vague thread... https://groups.google.com/forum/#!topic/comp.lang.python/FVnTe2i0UTY ... I find myself asking why Python doesn't include a standard, non-blocking keyboard input function. I have often wanted one myself. The only way that I've ever achieved this behavior is: 1) by restricting the user to pressing Ctrl-C while the program is running, and catching a KeyboardInterrupt; or 2) loading a heavyweight GUI like wxPython or PyQt, and using its event loop to intercept keyboard events. I gather that non-blocking keyboard input functions aren't the easiest thing to implement. They seem to depend on the operating system. Still, ease of use is a primary goal of Python, and the need for this feature must be common. From MrJean1 at gmail.com Mon Oct 24 16:20:12 2016 From: MrJean1 at gmail.com (MrJean1) Date: Mon, 24 Oct 2016 13:20:12 -0700 (PDT) Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? In-Reply-To: References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> Message-ID: <247ea9ab-a7a6-421c-9d84-aaa6b12c591b@googlegroups.com> On Wednesday, October 5, 2016 at 1:57:14 PM UTC-4, Malcolm Greene wrote: > Looking for a quick way to calculate lines of code/comments in a > collection of Python scripts. This isn't a LOC per day per developer > type analysis - I'm looking for a metric to quickly judge the complexity > of a set of scripts I'm inheriting. > > Thank you, > Malcolm Here is a basic LOC counter for Python source files References: <87h98kvg40.fsf@handshake.de> Message-ID: On Mon, Oct 10, 2016 at 12:05 AM, dieter wrote: > Dan Stromberg writes: >> I have a program http://stromberg.dnsalias.org/~dstromberg/looper/ >> that I use and maintain. >> >> It's like GNU parallel or similar - yet another "run n processes, m at >> a time" implementation. Interestingly, I've only used/tested it on >> Linux, but it's under a Microsoft copyright because Microsoft acquired >> a Linux company I was working for. >> >> Anyway, it seems to work well, except one annoying bug. >> >> That bug is: if you control-C the top-level process, all the >> subprocesses are left running. >> >> I've been thinking about making it catch SIGINT, SIGTERM and SIGHUP, >> and having it SIGKILL its active subprocesses upon receiving one of >> these signals. >> >> However, it's multithreaded, and I've heard that in CPython, threads >> and signals don't mix well. > > I would not state it this way. > > Signals are delivered to processes (not process threads). > That leads to the question which thread (in a multi thread application) > will handle a signal when it arrives. > In Python, the design decision has been not to use the currently > running thread (which may not be a Python thread at all or > may have executed Python code but at the moment runs C code outside > of Python) but let the signal be handled deterministically by > the main thread (once it starts again to execute Python code). > This looks like a sane decision to me. Thank you for the clear explanation. From drsalists at gmail.com Mon Oct 24 17:00:25 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 24 Oct 2016 14:00:25 -0700 Subject: Sphinx + autodoc + apidoc Message-ID: Hi folks. I'm attempting to set up Sphinx to document several API's based on docstrings. I've got something browseable for one example API using Sphinx + autodoc + apidoc. However, we aren't really a PEP8 shop; we use hard tabs expanded to 4 columns, and we use 120 columns total width (and sometimes a little more :). I'd love to switch to spaces, but it's not my decision, so... And it's hard for me to imagine going back to 80 columns - it's so confining, especially when you use good, descriptive identifiers. Anyway, in the Sphinx pages that say "Source code for ", and that show every line of the particular python module (syntax highlighted), the tabs are all expanded to 8 columns, and the total width is 80 columns. I can scroll around to see everything, but it's a little like looking at a football field through a microscope. Is there any good way of making Sphinx use 4 column tabs and 120 column text? I can imagine recursively applying *ix "expand -4" to the .py files to get a for-doc-only tree before passing them to Sphinx, but what about the 80 column limit? And is there a nicer way of dealing with the tab situation? I googled quite a bit, and found some stuff suggesting that the Sphinx devs would be OK with doing hard tabs, but Sphinx depends on Docutils, and the Docutils project is perhaps more into PEP8. Thanks! From drsalists at gmail.com Mon Oct 24 17:28:09 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 24 Oct 2016 14:28:09 -0700 Subject: sphinx (or other means to document python) In-Reply-To: References: <0e8c5dd4-6a64-45dc-adc9-e2dd58790ca2@googlegroups.com> <377e472c-f7f1-4947-be5e-72a19de2470c@googlegroups.com> Message-ID: On Sat, Sep 24, 2016 at 6:30 PM, Yann Kaiser wrote: > pydoctor may be something you're looking for. I don't know if it supports > exporting to PDF like Sphinx does. > > As you've no doubt figured out by now, Sphinx doesn't revolve around the > Python files themselves, but rather .rst files in which you can indeed > instruct Sphinx to just go and document a module. I just evaluated a bunch of docstring-extracting-for-documentation tools, including Sphinx and pydoctor. Sphinx isn't that hard anymore, with autodoc and apidoc. But pydoctor appears to be Python 2.x only - that's a problem in 2016. Please correct me if I'm wrong. From ned at nedbatchelder.com Mon Oct 24 17:39:40 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 24 Oct 2016 14:39:40 -0700 (PDT) Subject: Sphinx + autodoc + apidoc In-Reply-To: References: Message-ID: On Monday, October 24, 2016 at 5:00:47 PM UTC-4, Dan Stromberg wrote: > Hi folks. > > I'm attempting to set up Sphinx to document several API's based on docstrings. > > I've got something browseable for one example API using Sphinx + > autodoc + apidoc. > > However, we aren't really a PEP8 shop; we use hard tabs expanded to 4 > columns, and we use 120 columns total width (and sometimes a little > more :). > > I'd love to switch to spaces, but it's not my decision, so... And > it's hard for me to imagine going back to 80 columns - it's so > confining, especially when you use good, descriptive identifiers. > > Anyway, in the Sphinx pages that say "Source code for ", > and that show every line of the particular python module (syntax > highlighted), the tabs are all expanded to 8 columns, and the total > width is 80 columns. I can scroll around to see everything, but it's > a little like looking at a football field through a microscope. > > Is there any good way of making Sphinx use 4 column tabs and 120 column text? > > I can imagine recursively applying *ix "expand -4" to the .py files to > get a for-doc-only tree before passing them to Sphinx, but what about > the 80 column limit? And is there a nicer way of dealing with the tab > situation? > > I googled quite a bit, and found some stuff suggesting that the Sphinx > devs would be OK with doing hard tabs, but Sphinx depends on Docutils, > and the Docutils project is perhaps more into PEP8. Are you sure your tabs are being changed to eight spaces? It's possible they are still tabs in the browser, and the browser is choosing to display them as eight spaces. If that's the case, you can change the width using CSS in the theme. Likely the page width of 80 characters is also manipulable with CSS. --Ned. From dkoleary at olearycomputers.com Mon Oct 24 17:41:05 2016 From: dkoleary at olearycomputers.com (Doug OLeary) Date: Mon, 24 Oct 2016 14:41:05 -0700 (PDT) Subject: lxml and xpath(?) Message-ID: <9ceddc7b-a3d0-479a-a009-29c4bdc15e72@googlegroups.com> Hey; Reasonably new to python and incredibly new to xml much less trying to parse it. I need to identify cluster nodes from a series of weblogic xml configuration files. I've figured out how to get 75% of them; now, I'm going after the edge case and I'm unsure how to proceed. Weblogic xml config files start with namespace definitions then a number of child elements some of which have children of their own. The element that I'm interested in is which will usually have a subelement called containing the hostname that I'm looking for. Following the paradigm of "we love standards, we got lots of them", this model doesn't work everywhere. Where it doesn't work, I need to look for a subelement of called . That element contains an alias which is expanded in a different root child, at the same level as . So, picture worth a 1000 words: < [[ heinous namespace xml snipped ]] > [[text]] ... EDIServices_MS1 ... EDIServices_MC1 ... EDIServices_MS2 ... EDIServices_MC2 ... EDIServices_MC1 EDIServices_MC1 SSL host001 7001 EDIServices_MC2 EDIServices_MC2 host002 7001 So, running it on 'normal' config, I get: $ ./lxml configs/EntsvcSoa_Domain_config.xml EntsvcSoa_CS => host003.myco.com EntsvcSoa_CS => host004.myco.com Running it against the abi-normal config, I'm currently getting: $ ./lxml configs/EDIServices_Domain_config.xml EDIServices_CS => EDIServices_MC1 EDIServices_CS => EDIServices_MC2 Using the examples above, I would like to translate EDIServices_MC1 and EDIServices_MC2 to host001 and host002 respectively. The primary loop is: for server in root.findall('ns:server', namespaces): cs = server.find('ns:cluster', namespaces) if cs is None: continue # cluster_name = server.find('ns:cluster', namespaces).text cluster_name = cs.text listen_address = server.find('ns:listen-address', namespaces) server_name = listen_address.text if server_name is None: machine = server.find('ns:machine', namespaces) if machine is None: continue else: server_name = machine.text print("%-15s => %s" % (cluster_name, server_name)) (it's taken me days to write 12 lines of code... good thing I don't do this for a living :) ) Rephrased, I need to find the under the child who's name matches the name under the corresponding child. From some of the examples on the web, I believe xpath might help but I've not been able to get even the simple examples working. Go figure, I just figured out what a namespace is... Any hints/tips/suggestions greatly appreciated especially with complete noob tutorials for xpath. Thanks for your time. Doug O'Leary From hanzer at riseup.net Mon Oct 24 17:46:51 2016 From: hanzer at riseup.net (Adam Jensen) Date: Mon, 24 Oct 2016 17:46:51 -0400 Subject: =?UTF-8?Q?Re:_Internet_Data_Handling_=c2=bb_mailbox?= In-Reply-To: References: <874m447snh.fsf@handshake.de> Message-ID: On 10/22/2016 11:56 PM, Jason Friedman wrote: >> >> for message in mailbox.mbox(sys.argv[1]): >> if message.has_key("From") and message.has_key("To"): >> addrs = message.get_all("From") >> addrs.extend(message.get_all("To")) >> for addr in addrs: >> addrl = addr.lower() >> if addrl.find(name) > 0: >> print message >> break >> ------------------------------------------------------------- > > > I usually see > > if addrl.find(name) > 0: > > written as > > if name in addrl: > Yeah, that would be more consistent with the 'for addr in addrs' construct. But I've never been able to take a serious look at Python, or develop an understanding and consistent style. The catastrophic battology[1] present in every book I've encountered is too much of an obstacle. [1]: http://grammar.about.com/od/ab/g/battologyterm.htm From rtomek at ceti.pl Mon Oct 24 18:05:32 2016 From: rtomek at ceti.pl (Tomasz Rola) Date: Tue, 25 Oct 2016 00:05:32 +0200 Subject: Quick way to calculate lines of code/comments in a collection of Python scripts? In-Reply-To: References: <1475690219.707576.746931233.6BCD55C1@webmail.messagingengine.com> <20161023205158.GA977@tau1.ceti.pl> Message-ID: <20161024220532.GA29119@tau1.ceti.pl> On Mon, Oct 24, 2016 at 10:03:29AM +0100, Stephen Tucker wrote: > Tomasz, > > How about using the command prompt command FIND /C on each of your source > files as follows: > > FIND/C "#" >NumbersOfLinesContainingPythonComments.dat > FIND/C /V "#" >NumbersOfLinesNotContainingPythonComments.dat > > You would end up with two files each with a column of line counts; > > Import these lines into an Excel Spreadsheet and calculate whatever you > like with them. If this is what you really want to do, then why not. Albeit I would rather go with sh script for this, with ability to process either a directory or single file of Python code. Also, I tend to avoid tools that are "click to work" as much as possible, so for me, this is not good. Using "find/c" or "grep|wc" might look like simple and quick and good solution, but it may soon turn out to be too little, which is why I do not consider sloccount to be an overkill (which you seem to suggest). Especially that OP mentioned something about code complexity, if memory serves. On my system, all it takes is: (as root) apt-get install sloccount and: sloccount alioth_nbody.lsp (or .py or what you like) I guess it is similarly easy to install under other OSes, even under Windows - I would try cygwin installer for this. On the other hand, using Office (or equivalent) only to count lines seems like royal excess. And if I want to calculate, I use lisp interpreter interactively (believe it or not). Spritesheep, like Excel, has some merits but I consider them poor choice for computing anything important (I see no formula, I see no errors). -- Regards, Tomasz Rola -- ** A C programmer asked whether computer had Buddha's nature. ** ** As the answer, master did "rm -rif" on the programmer's home ** ** directory. And then the C programmer became enlightened... ** ** ** ** Tomasz Rola mailto:tomasz_rola at bigfoot.com ** From drsalists at gmail.com Mon Oct 24 19:01:52 2016 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 24 Oct 2016 16:01:52 -0700 Subject: Sphinx + autodoc + apidoc In-Reply-To: References: Message-ID: On Mon, Oct 24, 2016 at 2:39 PM, Ned Batchelder wrote: > On Monday, October 24, 2016 at 5:00:47 PM UTC-4, Dan Stromberg wrote: >> However, we aren't really a PEP8 shop; we use hard tabs expanded to 4 >> columns, and we use 120 columns total width (and sometimes a little >> more :). >> Is there any good way of making Sphinx use 4 column tabs and 120 column text? > Are you sure your tabs are being changed to eight spaces? It's possible they > are still tabs in the browser, and the browser is choosing to display them > as eight spaces. If that's the case, you can change the width using CSS in > the theme. Likely the page width of 80 characters is also manipulable with > CSS. It turned out CSS was the answer. I fixed it with: echo 'pre {tab-size: 4;width: 80em;}' > doc/_build/html/_static/custom.css echo 'div.document {width: 100em; margin-top: 0px; margin-bottom: 0px; margin-right: 0px; margin-left: 0px;}' >> doc/_build/html/_static/custom.css Though it remains to be seen if it'll look as good in other people's browsers. It bothers me that 120 columns of text fit so easily into 80em. I thought 80em would be the width of 80 character cells... It seems like a fixed pitch font, but maybe it's really proportionate. Using 120em was huge. Thanks! From duncan at invalid.invalid Mon Oct 24 19:39:02 2016 From: duncan at invalid.invalid (duncan smith) Date: Tue, 25 Oct 2016 00:39:02 +0100 Subject: retain dimensions for numpy slice In-Reply-To: References: Message-ID: On 24/10/16 19:05, Peter Otten wrote: > duncan smith wrote: > >> Hello, >> I have several arrays that I need to combine elementwise in >> various fashions. They are basically probability tables and there is a >> mapping of axes to variables. I have code for transposing and reshaping >> that aligns the variables / axes so the usual broadcasting rules achieve >> the desired objective. But for a specific application I want to avoid >> the transposing and reshaping. So I've specified arrays that contain the >> full dimensionality (dimensions equal to the total number of variables). >> e.g. >> >> Arrays with shape, >> >> [1,3,3] and [2,3,1] >> >> to represent probability tables with variables >> >> [B,C] and [A,B]. >> >> One operation that I need that is not elementwise is summing over axes, >> but I can use numpy.sum with keepdims=True to retain the appropriate >> shape. >> >> The problem I have is with slicing. This drops dimensions. Does anyone >> know of a solution to this so that I can e.g. take an array with shape >> [2,3,1] and generate a slice with shape [2,1,1]? I'm hoping to avoid >> having to manually reshape it. Thanks. > > Can you clarify your requirement or give an example of what you want? > > Given an array > >>>> a.shape > (2, 3, 1) > > you can get a slice with shape (2,1,1) with (for example) > >>>> a[:,:1,:].shape > (2, 1, 1) > > or even > >>>> newshape = (2, 1, 1) >>>> a[tuple(slice(d) for d in newshape)].shape > (2, 1, 1) > > but that's probably not what you are asking for... > Thanks. I think that's exactly what I wanted. Duncan From rgaddi at highlandtechnology.invalid Mon Oct 24 19:49:42 2016 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 24 Oct 2016 23:49:42 -0000 (UTC) Subject: System-wide module path Message-ID: On a Linux (Ubuntu) system, with no concerns for Python < 3.4, how do I a) Add a directory to the system-wide (rather than per-user) module path? b) Tell pip that I would like to install a given module to there. What's going on is that we're trying to set up an internal software distribution system to make sure that internally-written software gets installed on a bunch of identical machines. The easiest mechanism to work with in all of this is a single directory on the server that can simply be rsync'd to all of the target machines. So if I could do my installs to /usr/highland/python3.4/site-packages, and put all the executable stubs that pip creates into /usr/highland/bin, then I can just rsync all of /usr/highland. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From ned at nedbatchelder.com Mon Oct 24 19:51:00 2016 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 24 Oct 2016 16:51:00 -0700 (PDT) Subject: Sphinx + autodoc + apidoc In-Reply-To: References: Message-ID: <715352dd-5477-4041-9e92-9c49377ecae7@googlegroups.com> On Monday, October 24, 2016 at 7:02:11 PM UTC-4, Dan Stromberg wrote: > On Mon, Oct 24, 2016 at 2:39 PM, Ned Batchelder wrote: > > On Monday, October 24, 2016 at 5:00:47 PM UTC-4, Dan Stromberg wrote: > > >> However, we aren't really a PEP8 shop; we use hard tabs expanded to 4 > >> columns, and we use 120 columns total width (and sometimes a little > >> more :). > > >> Is there any good way of making Sphinx use 4 column tabs and 120 column text? > > > Are you sure your tabs are being changed to eight spaces? It's possible they > > are still tabs in the browser, and the browser is choosing to display them > > as eight spaces. If that's the case, you can change the width using CSS in > > the theme. Likely the page width of 80 characters is also manipulable with > > CSS. > > It turned out CSS was the answer. I fixed it with: > > echo 'pre {tab-size: 4;width: 80em;}' > doc/_build/html/_static/custom.css > echo 'div.document {width: 100em; margin-top: 0px; margin-bottom: > 0px; margin-right: 0px; margin-left: 0px;}' >> > doc/_build/html/_static/custom.css > > Though it remains to be seen if it'll look as good in other people's > browsers. It bothers me that 120 columns of text fit so easily into > 80em. I thought 80em would be the width of 80 character cells... It > seems like a fixed pitch font, but maybe it's really proportionate. > Using 120em was huge. An em is a unit as wide as the font size. So with 12-point text, an em is 12 points wide. Most characters are much narrower than that. The unit is called "em" because it's roughly the width of a capital M in a proportional font. Your monospace font would have 80 characters in 80em if the characters were all square, but they are not. Much more common is a character width of about .6em. Typography has a long history spanning multiple technologies, and close connections with people and how they read, so it isn't always obvious, though I personally find it fascinating. --Ned. From starr22 at sbcglobal.net Mon Oct 24 20:11:21 2016 From: starr22 at sbcglobal.net (Kenneth L Stege) Date: Tue, 25 Oct 2016 00:11:21 +0000 (UTC) Subject: 3.5.2 References: <1001351358.47441.1477354281809.ref@mail.yahoo.com> Message-ID: <1001351358.47441.1477354281809@mail.yahoo.com> Im running windows 7 pro, 64 bit. I downloaded 3.5.2 64 bit and when I try to run I get the error message? api-ms-win-crt-runtime-l1-1-0.dll is missing. I loaded that file and still will not run.?? suggestions? thanks From zachary.ware+pylist at gmail.com Mon Oct 24 23:42:54 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Mon, 24 Oct 2016 22:42:54 -0500 Subject: System-wide module path In-Reply-To: References: Message-ID: On Mon, Oct 24, 2016 at 6:49 PM, Rob Gaddi wrote: > On a Linux (Ubuntu) system, with no concerns for Python < 3.4, how do I > a) Add a directory to the system-wide (rather than per-user) module > path? This is the trickier part. There are a couple of ways to do it, but which is better is a matter of some debate (at least in my head). The first option is to stick a `.pth` file in the regular site-packages directory that points to /usr/highland/lib/python3.4/site-packages. Another option is to set PYTHONPATH; this is one of the very few exceptions to the rule that you never want to permanently set PYTHONPATH. The main differences between the two options are which end of sys.path the directory is added to, and whether it's added when the -S interpreter option is used (say, by system utilities). The `.pth` option is cleaner and safer, but `.pth` files just feel icky. > b) Tell pip that I would like to install a given module to there. Use the --prefix option: `pip install --prefix /usr/highland/ alembic`. This tells pip to install the package(s) as though Python was installed at /usr/highland/, so libraries go in /usr/highland/lib/python3.4/site-packages and executable stubs go in /usr/highland/bin/. Hope this helps, -- Zach From steve+comp.lang.python at pearwood.info Tue Oct 25 02:39:10 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 25 Oct 2016 17:39:10 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: Message-ID: <580efe10$0$11101$c3e8da3@news.astraweb.com> On Tuesday 25 October 2016 05:14, jladasky at itu.edu wrote: > After reading this rather vague thread... > > https://groups.google.com/forum/#!topic/comp.lang.python/FVnTe2i0UTY > > ... I find myself asking why Python doesn't include a standard, non-blocking > keyboard input function. I have often wanted one myself. The only way that > I've ever achieved this behavior is: > > 1) by restricting the user to pressing Ctrl-C while the program is running, > and catching a KeyboardInterrupt; or > > 2) loading a heavyweight GUI like wxPython or PyQt, and using its event loop > to intercept keyboard events. > > I gather that non-blocking keyboard input functions aren't the easiest thing > to implement. They seem to depend on the operating system. Still, ease of > use is a primary goal of Python, and the need for this feature must be > common. Not really. I think that lots of people think they need it, but once they write a little utility, they often realise that it's not that useful. That's just my opinion, and I'm one of those guys who wrote one: http://code.activestate.com/recipes/577977-get-single-keypress/?in=user-4172944 Me and ten thousand others. If you (generic you, not you specifically) are telling the user "press any key to continue", then you probably shouldn't. *Any* key may not do anything. E.g. if the user hits the Shift key. A much better interface is to specify a specific key, and ignore anything else... in which case, why not specify the Enter key? raw_input('Press the Enter key to continue... ') If you are doing something more complex, waiting on different keys to do different things, then you probably should use an existing text UI like Curses, or a GUI like wxPython etc, rather than trying to reinvent the wheel badly. -- Steven git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space. From johnpote at jptechnical.co.uk Tue Oct 25 05:51:28 2016 From: johnpote at jptechnical.co.uk (John Pote) Date: Tue, 25 Oct 2016 10:51:28 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <580efe10$0$11101$c3e8da3@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <6fcfe093-91ed-fd9f-59b3-15101d085a56@jptechnical.co.uk> If you are on a windows platform you could use kbhit() from the msvcrt module as Steven D does in his solution on the activestate site which also works for xNIX. Worth a look at his code to see how these sort of things are done on xNIX. Alternatively you could use a couple of threads. One for your worker code and one for the keyboard entry code. Use a message queue to pass received lines from the keyboard thread to the worker thread. OK you still have to press Enter but you don't need the windows msvcrt library or xNIX termios modules. Regards John On 25/10/2016 07:39, Steven D'Aprano wrote: > On Tuesday 25 October 2016 05:14, jladasky at itu.edu wrote: > >> After reading this rather vague thread... >> >> https://groups.google.com/forum/#!topic/comp.lang.python/FVnTe2i0UTY >> >> ... I find myself asking why Python doesn't include a standard, non-blocking >> keyboard input function. I have often wanted one myself. The only way that >> I've ever achieved this behavior is: >> >> 1) by restricting the user to pressing Ctrl-C while the program is running, >> and catching a KeyboardInterrupt; or >> >> 2) loading a heavyweight GUI like wxPython or PyQt, and using its event loop >> to intercept keyboard events. >> >> I gather that non-blocking keyboard input functions aren't the easiest thing >> to implement. They seem to depend on the operating system. Still, ease of >> use is a primary goal of Python, and the need for this feature must be >> common. > > Not really. I think that lots of people think they need it, but once they write > a little utility, they often realise that it's not that useful. That's just my > opinion, and I'm one of those guys who wrote one: > > http://code.activestate.com/recipes/577977-get-single-keypress/?in=user-4172944 > > Me and ten thousand others. > > If you (generic you, not you specifically) are telling the user "press any key > to continue", then you probably shouldn't. *Any* key may not do anything. E.g. > if the user hits the Shift key. A much better interface is to specify a > specific key, and ignore anything else... in which case, why not specify the > Enter key? > > raw_input('Press the Enter key to continue... ') > > > If you are doing something more complex, waiting on different keys to do > different things, then you probably should use an existing text UI like Curses, > or a GUI like wxPython etc, rather than trying to reinvent the wheel badly. > > > From bc at freeuk.com Tue Oct 25 06:02:31 2016 From: bc at freeuk.com (BartC) Date: Tue, 25 Oct 2016 11:02:31 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <580efe10$0$11101$c3e8da3@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: On 25/10/2016 07:39, Steven D'Aprano wrote: >> I gather that non-blocking keyboard input functions aren't the easiest thing >> to implement. They seem to depend on the operating system. Still, ease of >> use is a primary goal of Python, and the need for this feature must be >> common. > > > Not really. I think that lots of people think they need it, but once they write > a little utility, they often realise that it's not that useful. > If you (generic you, not you specifically) are telling the user "press any key > to continue", then you probably shouldn't. *Any* key may not do anything. E.g. > if the user hits the Shift key. A much better interface is to specify a > specific key, and ignore anything else... in which case, why not specify the > Enter key? > > raw_input('Press the Enter key to continue... ') Which doesn't work on Python 3. So even here, making it easy by using line-input, it's not so straightforward. > If you are doing something more complex, waiting on different keys to do > different things, You mean, something as sophisticated as press Enter to continue, or Escape to quit? Or Page Up and Page Down? then you probably should use an existing text UI like Curses, > or a GUI like wxPython etc, rather than trying to reinvent the wheel badly. But why the need to have to use someone else's massive great wheel? Both Curses and wxPython are completely over the top IMO for something so fundamental. I started coding in 1976 (Christ, 40 years ago!), using teletypes. Getting line input was easy (apparently easier than in Python now!). What was hard was: (1) Reading a single key-press without the user having to press Enter (or Return as it was then) (2) Reading that single key-press without the character being echoed to the output It seems like little has changed! You needed this to do what I considered cool things. A bit limited on a TTY but on a video display, a lot more was possible (editors, games, etc). As for actually reading from the keyboard, the first board I made that actually had one (it was gorgeous, with all sorts of exotic keys you didn't find on typewriters like !, [], {}, + and Ctrl), involved this: * Poll the byte at some specific port until the top bit changed from 0 to 1. Then a key had just been pressed. * The lower 7 bits was the ASCII value of the character (I don't know what it did about cursor keys, but maybe it didn't have any.) This gives you the ability to do (2) above. From that, you could do (1) (echoing) and go on to build full line-orientated input. But you had complete control. So, why has it all become so difficult? -- Bartc From bc at freeuk.com Tue Oct 25 07:14:38 2016 From: bc at freeuk.com (BartC) Date: Tue, 25 Oct 2016 12:14:38 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> Message-ID: On 25/10/2016 11:39, Dennis Lee Bieber wrote: > On Tue, 25 Oct 2016 11:02:31 +0100, BartC declaimed the > following: > >> This gives you the ability to do (2) above. From that, you could do (1) >> (echoing) and go on to build full line-orientated input. But you had >> complete control. >> >> So, why has it all become so difficult? > > Because you now have an interrupt driven operating system with buffered > line handling between you and the hardware. The OS is handling things like > back-space/delete, (from a single keypress -- for those systems > that use both for line ending), previous line recall/history. > > Your polling loop on a status register basically meant nothing else was > happening on that computer until you retrieved a character Only if you were specifically waiting for a key press. For the requirement in the other thread (see link in OP), it just needed to know if a key had been pressed; that would be very quick (I can't remember if that status bit latched or not.) Anyway the next step would have been to wire up the status bit to an interrupt line. and passed it on > to whatever program had requested it. Something doable on a > single-user/single-task system, but totally at odds with any system running > a few dozen separate processes. I don't agree. Each single process shouldn't need to be aware of any of the others. In the same way that the raw_input() example doesn't need to take account of the other half-dozen Python programs all waiting on raw_input() at the same time (which are all waiting for the same keyboard). > Hardware peripherals are now the domain of the OS, and mere users are > supposed to request the OS for data... That means OS specific methods for > bypassing the convenience mode of "cooked" input. Fine, then let the OS provide the appropriate means if the user program is not allowed to directly access the hardware. getch() and kbhit() are crude but they will do for simple programs. But they are not part of the OS. Implementing the equivalent via calls to Win32 is horrendous (and to Linux not much better). (BTW the computer I was using in 1976 had 160 simultaneous users. And each had their own keyboard. Now usually there is just one keyboard...) -- bartc From rosuav at gmail.com Tue Oct 25 07:25:23 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Oct 2016 22:25:23 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: On Tue, Oct 25, 2016 at 9:02 PM, BartC wrote: >> raw_input('Press the Enter key to continue... ') > > > Which doesn't work on Python 3. So even here, making it easy by using > line-input, it's not so straightforward. So you use input() instead. Big deal. The concept is still the same. >> If you are doing something more complex, waiting on different keys to do >> different things, > > > You mean, something as sophisticated as press Enter to continue, or Escape > to quit? Or Page Up and Page Down? Enter to continue or Ctrl-C to quit. Just as easy. > But why the need to have to use someone else's massive great wheel? Both > Curses and wxPython are completely over the top IMO for something so > fundamental. Fundamental? What exactly is fundamental about key-based input? Much more fundamental is character-based input, where you request *text* from the user. How often do you care whether someone pressed '1' on the top row as compared to '1' on the numeric keypad? How often do you actually need to handle Backspace as a key, rather than simply having it remove one character? Do you actually want to distinguish between typed keys and pasted text? Much more commonly, you simply ask for input from the user, and get back a line of text. The user might have edited that line before submitting it, and that's a feature, not a bug. When you want more flexibility than "Enter to continue or Ctrl-C to abort", it probably *is* time to get curses or a GUI toolkit, because you're doing something that's fundamentally different from "request text from the user". ChrisA From rosuav at gmail.com Tue Oct 25 07:28:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Oct 2016 22:28:14 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> Message-ID: On Tue, Oct 25, 2016 at 10:14 PM, BartC wrote: > I don't agree. Each single process shouldn't need to be aware of any of the > others. In the same way that the raw_input() example doesn't need to take > account of the other half-dozen Python programs all waiting on raw_input() > at the same time (which are all waiting for the same keyboard). > > Fine, then let the OS provide the appropriate means if the user program is > not allowed to directly access the hardware. getch() and kbhit() are crude > but they will do for simple programs. But they are not part of the OS. > Implementing the equivalent via calls to Win32 is horrendous (and to Linux > not much better). There's a huge difference between a loop that calls a blocking function like (raw_)input and one that calls a non-blocking function like kbhit(). One of them is polite to other processes; the other is not. In fact, even in a single-user single-process system, polling is a bad idea - it means that nothing can go to sleep. Unless, of course, you want to reimplement the whole concept of blocking calls on top of non-blocking ones, in which case... why?!? ChrisA From marko at pacujo.net Tue Oct 25 08:09:50 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 25 Oct 2016 15:09:50 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> Message-ID: <87y41c635d.fsf@elektro.pacujo.net> Chris Angelico : > There's a huge difference between a loop that calls a blocking > function like (raw_)input and one that calls a non-blocking function > like kbhit(). One of them is polite to other processes; the other is > not. Each process can have its own PTY with a separate virtual keyboard interface. The main problem is that the Unix terminal interface is quite old and crude. I'm guessing the raw inputs come from the /dev/input/* drivers. Unfortunately the PTYs don't come with their faked /dev/input drivers, and they would be reserved for root anyway. > In fact, even in a single-user single-process system, polling is a bad > idea - it means that nothing can go to sleep. Yes, spinning is bad. However, there are nicer ways to poll: select, poll, epoll... > Unless, of course, you want to reimplement the whole concept of > blocking calls on top of non-blocking ones, in which case... why?!? Blocking calls are evil. Marko From bc at freeuk.com Tue Oct 25 08:35:22 2016 From: bc at freeuk.com (BartC) Date: Tue, 25 Oct 2016 13:35:22 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: On 25/10/2016 12:25, Chris Angelico wrote: > On Tue, Oct 25, 2016 at 9:02 PM, BartC wrote: >>> raw_input('Press the Enter key to continue... ') >> >> >> Which doesn't work on Python 3. So even here, making it easy by using >> line-input, it's not so straightforward. > > So you use input() instead. Big deal. The concept is still the same. > >>> If you are doing something more complex, waiting on different keys to do >>> different things, >> >> >> You mean, something as sophisticated as press Enter to continue, or Escape >> to quit? Or Page Up and Page Down? > > Enter to continue or Ctrl-C to quit. Just as easy. Ctrl-C is not the same; that will just abort (without doing proper termination such as saving your data, or even just asking the user to confirm). > >> But why the need to have to use someone else's massive great wheel? Both >> Curses and wxPython are completely over the top IMO for something so >> fundamental. > > Fundamental? What exactly is fundamental about key-based input? Much > more fundamental is character-based input, where you request *text* > from the user. How often do you care whether someone pressed '1' on > the top row as compared to '1' on the numeric keypad? I don't think I've made that distinction. A very basic model of an interactive text-based computer has input at one end and output at the other. And a keyboard is the obvious choice for input (a bit easier than voice and less crude than punched cards or tape). You can do a lot of stuff with line-based input, but how do you think /that/ gets implemented? At some point it needs to be a character at a time or a key at a time; the requirement is there. Take away a keyboard, real or virtual, from a development computer (or even one used to check your bank account or go on forums), and see how far you get. > Much more commonly, you simply ask for > input from the user, and get back a line of text. The user might have > edited that line before submitting it, and that's a feature, not a > bug. Yeah, 'kill dwarf with axe' and such. It works but it's limited. Try and write a program where the keys represent the notes on a piano, and pressing each key plays the corresponding note from speaker. Having to press Enter after each one is going to cramp your style a bit! And don't tell me this is advanced because I was doing stuff like this decades ago. (Of course now I wouldn't have a clue how to make it generate sounds.) > When you want more flexibility than "Enter to continue or Ctrl-C to > abort", it probably *is* time to get curses or a GUI toolkit, because > you're doing something that's fundamentally different from "request > text from the user". My IDE and editor that I use every day are based on 'getchx()', an extended version of getch(). Although it is implemented on top of Win32 which is a kind of complex GUI. (getchx returns key and shift-state information as well as character codes.) But the point is, it is just the one function; how it's implemented is not relevant. It's the kind of function that it would be nice to have /as standard/ in any language without being told that being able to deal with key-at-a-time input is an advanced topic! -- Bartc From rosuav at gmail.com Tue Oct 25 08:40:14 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Oct 2016 23:40:14 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87y41c635d.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> <87y41c635d.fsf@elektro.pacujo.net> Message-ID: On Tue, Oct 25, 2016 at 11:09 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> There's a huge difference between a loop that calls a blocking >> function like (raw_)input and one that calls a non-blocking function >> like kbhit(). One of them is polite to other processes; the other is >> not. > > Each process can have its own PTY with a separate virtual keyboard > interface. The main problem is that the Unix terminal interface is quite > old and crude. Or no PTY at all. And yes, it is crude... all it can do is allow input of arbitrary text. You can't draw a self-portrait, you can't take a photograph, you can't enter a mouse gesture. Terribly crude. And perfect for anything that uses text. >> In fact, even in a single-user single-process system, polling is a bad >> idea - it means that nothing can go to sleep. > > Yes, spinning is bad. However, there are nicer ways to poll: select, > poll, epoll... What's the point of using select etc when you care about only one input? >> Unless, of course, you want to reimplement the whole concept of >> blocking calls on top of non-blocking ones, in which case... why?!? > > Blocking calls are evil. Oh, that's why. Got it. So because blocking calls are fundamentally evil, we have to... what? What's so bad about them? Remember, not every program is a server handling myriad clients. ChrisA From marko at pacujo.net Tue Oct 25 08:43:05 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 25 Oct 2016 15:43:05 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <87shrk61ly.fsf@elektro.pacujo.net> BartC : > Ctrl-C is not the same; that will just abort (without doing proper > termination such as saving your data, or even just asking the user to > confirm). Ctrl-C is not the same, but it does let you intercept it and even ignore it. Just handle signal.SIGINT. > A very basic model of an interactive text-based computer has input at > one end and output at the other. And a keyboard is the obvious choice > for input (a bit easier than voice and less crude than punched cards > or tape). Text and keyboard are completely different entities. You don't find [F1] or [Insert] or [Alt] in text nor do you distinguish between button up and button down. Also, you can't measure the timing between the characters in text. Most Unix entities are right at home with text. It's the keyboard that's problematic. The keyboard does not communicate with characters or text. Rather it emits events with keycodes. Marko From marko at pacujo.net Tue Oct 25 08:45:03 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 25 Oct 2016 15:45:03 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> <87y41c635d.fsf@elektro.pacujo.net> Message-ID: <87oa2861io.fsf@elektro.pacujo.net> Chris Angelico : > On Tue, Oct 25, 2016 at 11:09 PM, Marko Rauhamaa wrote: >> Blocking calls are evil. > > Oh, that's why. Got it. So because blocking calls are fundamentally > evil, we have to... what? What's so bad about them? Remember, not > every program is a server handling myriad clients. Myriads or not, we are talking about interactive (or reactive) programs. The paradigm of choice is event-driven programming. Marko From rosuav at gmail.com Tue Oct 25 08:49:24 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Oct 2016 23:49:24 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: On Tue, Oct 25, 2016 at 11:35 PM, BartC wrote: > On 25/10/2016 12:25, Chris Angelico wrote: >>> You mean, something as sophisticated as press Enter to continue, or >>> Escape >>> to quit? Or Page Up and Page Down? >> >> >> Enter to continue or Ctrl-C to quit. Just as easy. > > > Ctrl-C is not the same; that will just abort (without doing proper > termination such as saving your data, or even just asking the user to > confirm). In Python, Ctrl-C raises KeyboardInterrupt. If you want to save your data, use standard exception handling. (And asking to confirm? Isn't that exactly what "Press Enter to continue or Ctrl-C to abort" *is*?) >>> But why the need to have to use someone else's massive great wheel? Both >>> Curses and wxPython are completely over the top IMO for something so >>> fundamental. >> >> >> Fundamental? What exactly is fundamental about key-based input? Much >> more fundamental is character-based input, where you request *text* >> from the user. How often do you care whether someone pressed '1' on >> the top row as compared to '1' on the numeric keypad? > > > I don't think I've made that distinction. No, but that's exactly what happens when you switch from character-based input to key-based. You can't have both. Either you care about text, or you care about buttons being pressed. > A very basic model of an interactive text-based computer has input at one > end and output at the other. And a keyboard is the obvious choice for input > (a bit easier than voice and less crude than punched cards or tape). > > You can do a lot of stuff with line-based input, but how do you think /that/ > gets implemented? At some point it needs to be a character at a time or a > key at a time; the requirement is there. > > Take away a keyboard, real or virtual, from a development computer (or even > one used to check your bank account or go on forums), and see how far you > get. So? My programs don't care about that. They care about text. Where that text comes from is immaterial - it could be the keyboard, it could be the mouse (middle-click paste), it could be a file (stream redirection), it could be anything else. >> Much more commonly, you simply ask for >> input from the user, and get back a line of text. The user might have >> edited that line before submitting it, and that's a feature, not a >> bug. > > > Yeah, 'kill dwarf with axe' and such. It works but it's limited. > > Try and write a program where the keys represent the notes on a piano, and > pressing each key plays the corresponding note from speaker. > > Having to press Enter after each one is going to cramp your style a bit! And that's where you should be using a proper UI library - I would do this with a GUI, but you could also use ncurses. > And don't tell me this is advanced because I was doing stuff like this > decades ago. (Of course now I wouldn't have a clue how to make it generate > sounds.) I've been doing stuff like this for decades too - and decades ago, I was doing pretty advanced stuff. False line of argument. It sounds like "I'm older than you, and must know better", and that wasn't much good for Alice and the Lory either. > But the point is, it is just the one function; how it's implemented is not > relevant. It's the kind of function that it would be nice to have /as > standard/ in any language without being told that being able to deal with > key-at-a-time input is an advanced topic! How often do you really need this functionality, and what are you willing to give up for it? Normally, if you want that kind of thing, you need to reach for a GUI/TUI toolkit. Yaknow, like Steve said in the first place. ChrisA From rosuav at gmail.com Tue Oct 25 08:55:42 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 25 Oct 2016 23:55:42 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87oa2861io.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> <87y41c635d.fsf@elektro.pacujo.net> <87oa2861io.fsf@elektro.pacujo.net> Message-ID: On Tue, Oct 25, 2016 at 11:45 PM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Tue, Oct 25, 2016 at 11:09 PM, Marko Rauhamaa wrote: >>> Blocking calls are evil. >> >> Oh, that's why. Got it. So because blocking calls are fundamentally >> evil, we have to... what? What's so bad about them? Remember, not >> every program is a server handling myriad clients. > > Myriads or not, we are talking about interactive (or reactive) programs. > The paradigm of choice is event-driven programming. Have you watched "Tron"? A program goes to the I/O tower to receive a message from the User. It's an active operation on the part of the program. The user cannot initiate it, only the program can. Tron is extremely accurate in this way. ChrisA From auriocus at gmx.de Tue Oct 25 09:03:27 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Tue, 25 Oct 2016 15:03:27 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87oa2861io.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> <87y41c635d.fsf@elektro.pacujo.net> <87oa2861io.fsf@elektro.pacujo.net> Message-ID: Am 25.10.16 um 14:45 schrieb Marko Rauhamaa: > Chris Angelico : > >> On Tue, Oct 25, 2016 at 11:09 PM, Marko Rauhamaa wrote: >>> Blocking calls are evil. >> >> Oh, that's why. Got it. So because blocking calls are fundamentally >> evil, we have to... what? What's so bad about them? Remember, not >> every program is a server handling myriad clients. > > Myriads or not, we are talking about interactive (or reactive) programs. > The paradigm of choice is event-driven programming. > I agree, which means (basically) GUI, which means event loop and all that stuff, making programming more inconvenient or complicated. That reminds me of a standard "problem" with introductory programming texts. Many books describe the first programs along the lines of x=input('Please enter x: ') y=input('Please enter y: ') print('The product is ', x*y) Now, such programs are not useful in practice. Once you try, you will inevitably make errors entering the data, restart that thing multiple times and curse the author badly. "Real" programs either provide a GUI or a command line parser which allows for the correction of errors and entering the data in arbitrary order. Both lead to more complex programs. Maybe some (extremely simple) option parser or GUI thingy could be implemented into Python directly, so that the novice doesn't have to bother with the details, in the sense of: parameters({'x', double, 'y', double}) output('The product is ', x*y) which becomes either a command line thingy with -x and -y options or a GUI with input fields and an output line. Christian From tjreedy at udel.edu Tue Oct 25 09:23:24 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 25 Oct 2016 09:23:24 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: Message-ID: On 10/24/2016 2:14 PM, jladasky at itu.edu wrote: > After reading this rather vague thread... > > https://groups.google.com/forum/#!topic/comp.lang.python/FVnTe2i0UTY > > ... I find myself asking why Python doesn't include a standard, > non-blocking keyboard input function. I have often wanted one > myself. The only way that I've ever achieved this behavior is: > > 1) by restricting the user to pressing Ctrl-C while the program is > running, and catching a KeyboardInterrupt; or > > 2) loading a heavyweight GUI like wxPython or PyQt, and using its > event loop to intercept keyboard events. Or load the lighter weight cross-platform tkinter GUI that comes with Python. One can either make the GUI invisible or use at least a Text or Entry widget instead of the OS console. The text widget comes with numerous key and mouse bindings, including the generic "Display glyph in response to press of graphics key". Either way, if one wants to do a prolonged computation, one must either put it another thread or split it into chunks of limited duration, such 50 milleseconds (1/20 second), with breaks in between that allow user input handling. > I gather that non-blocking keyboard input functions aren't the > easiest thing to implement. They seem to depend on the operating > system. Still, ease of use is a primary goal of Python, and the need > for this feature must be common. What is not common today is to only want asynchronous keypress handing without the use of any GUI widget (other than the OS-supplied console). -- Terry Jan Reedy From random832 at fastmail.com Tue Oct 25 10:36:01 2016 From: random832 at fastmail.com (Random832) Date: Tue, 25 Oct 2016 10:36:01 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <580efe10$0$11101$c3e8da3@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <1477406161.3036984.766774953.79E4788A@webmail.messagingengine.com> On Tue, Oct 25, 2016, at 02:39, Steven D'Aprano wrote: > Not really. I think that lots of people think they need it, but > once they write a little utility, they often realise that it's not > that useful. That's just my opinion, and I'm one of those guys who > wrote one: > > http://code.activestate.com/recipes/577977-get-single-keypress/?in=user-4172944 Non-blocking (which your example here doesn't even do) isn't the same thing as character-at-a-time. It doesn't even imply it, technically - you could want to do other stuff and occasionally check if the user has entered a line, though *that* is even *more* involved on Windows because it means you can't do it with msvcrt.kbhit. From steve+python at pearwood.info Tue Oct 25 12:05:54 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 26 Oct 2016 03:05:54 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <580f82e4$0$1598$c3e8da3$5496439d@news.astraweb.com> On Tue, 25 Oct 2016 11:49 pm, Chris Angelico wrote: > In Python, Ctrl-C raises KeyboardInterrupt. If you want to save your > data, use standard exception handling. (And asking to confirm? Isn't > that exactly what "Press Enter to continue or Ctrl-C to abort" *is*?) $ Fire missiles? Press Enter to continue or Ctrl-C to abort. ^C $ Are you sure you want to abort? Press Enter to abort or Ctrl-C to abort the abort. ENTER $ Did you mean to abort? Press Enter to continue aborting, or Ctrl-C to abort aborting the abort and continue. ENTER $ Confirm abort: Enter to abort, Ctrl-C to abort. ENTER $ Abort aborted. Missiles fired. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Tue Oct 25 12:22:02 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 26 Oct 2016 03:22:02 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <1477406161.3036984.766774953.79E4788A@webmail.messagingengine.com> Message-ID: <580f86ac$0$1599$c3e8da3$5496439d@news.astraweb.com> On Wed, 26 Oct 2016 01:36 am, Random832 wrote: > On Tue, Oct 25, 2016, at 02:39, Steven D'Aprano wrote: >> Not really. I think that lots of people think they need it, but >> once they write a little utility, they often realise that it's not >> that useful. That's just my opinion, and I'm one of those guys who >> wrote one: >> >> http://code.activestate.com/recipes/577977-get-single-keypress/?in=user-4172944 > > Non-blocking (which your example here doesn't even do) isn't the same > thing as character-at-a-time. Heh, I just realised exactly the same thing. I wondered how many responses this thread would get before somebody noticed. Thanks Random. You're right, of course. My code blocks. So how would you do non-blocking keyboard input? How would it work? What would be the interface? > It doesn't even imply it, technically - > you could want to do other stuff and occasionally check if the user has > entered a line, though *that* is even *more* involved on Windows because > it means you can't do it with msvcrt.kbhit. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From no.email at nospam.invalid Tue Oct 25 12:32:29 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Tue, 25 Oct 2016 09:32:29 -0700 Subject: Why doesn't Python include non-blocking keyboard input function? References: Message-ID: <87lgxcmlsy.fsf@nightsong.com> jladasky at itu.edu writes: > ... I find myself asking why Python doesn't include a standard, > non-blocking keyboard input function. I have often wanted one myself. I agree this would be useful. Forth has a standard word KEY to read a key, and I used it in a simple game that I wrote a few months ago (you enter a key indicating which way you want to move). > The only way that I've ever achieved this behavior is: ... In *nix you should be able to set the tty modes with fcntl.ioctl so that reading from stdin returns immediately when you hit a key. You should be able to use select.select with zero timeout to see whether input is available. From python.list at tim.thechases.com Tue Oct 25 13:06:12 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Tue, 25 Oct 2016 12:06:12 -0500 Subject: Reversing \N{...} notation? Message-ID: <20161025120612.02194da4@bigbox.christie.dr> I like the clarity of using the "\N{...}" notation when creating string literals involving Unicode chars. Is there a built-in way to get such strings back from Python? >>> s = 'ma\N{LATIN SMALL LETTER N WITH TILDE}ana' >>> s 'ma?ana' >>> magic(s) 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' I can manually rip the string apart and put it back together with something like >>> import unicodedata as ud >>> ''.join(c if ord(c) < 128 else '\\N{%s}' % ud.name(c) for c in s) 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' but was wondering if there was some sort of .encode() trick or other corner of the library I hadn't found. -tkc From irmen.NOSPAM at xs4all.nl Tue Oct 25 13:14:07 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Tue, 25 Oct 2016 19:14:07 +0200 Subject: 3.5.2 In-Reply-To: References: <1001351358.47441.1477354281809.ref@mail.yahoo.com> <1001351358.47441.1477354281809@mail.yahoo.com> Message-ID: <580f92df$0$892$e4fe514c@news.xs4all.nl> On 25-10-2016 2:11, Kenneth L Stege wrote: > Im running windows 7 pro, 64 bit. I downloaded 3.5.2 64 bit and when I try to run I get the error message api-ms-win-crt-runtime-l1-1-0.dll is missing. I loaded that file and still will not run. > suggestions? > thanks > http://lmgtfy.com/?q=api-ms-win-crt-runtime-l1-1-0.dll+missing+windows+7 From mandalmanas786 at gmail.com Tue Oct 25 14:06:14 2016 From: mandalmanas786 at gmail.com (mandalmanas786 at gmail.com) Date: Tue, 25 Oct 2016 11:06:14 -0700 (PDT) Subject: 3rd command never executed in remote machine using paramiko Message-ID: I have written below code to run 3 command in remote server interactively But when i checked 3rd command never executed and code stuck here is my code def execute(): ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect('ipaddress',username='user', password='pw') chan=ssh.invoke_shell() # start the shell before sending commands chan.send('cd /path to folder/test') chan.send('\n') time.sleep(3) chan.send("ls -l") chan.send('\n') buff='' while not buff.endswith(">"): resp = chan.recv(9999) # code stuck here after 'path to folder/test >' comes in shell prompt buff+=resp print resp print "test" chan.send("ls -lh") chan.send('\n') time.sleep(5) buff='' while not buff.endswith(">"): resp = chan.recv(9999) buff+=resp print resp if __name__ == "__main__": execute() When i ran i got output of ls -l but ls -lh never executed my code stuck in first while loop. Anyone please help to resolve my issue When i reduced bytes to read size then it executed 3rd script. From marko at pacujo.net Tue Oct 25 14:06:50 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Tue, 25 Oct 2016 21:06:50 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <1477406161.3036984.766774953.79E4788A@webmail.messagingengine.com> <580f86ac$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87r374xpz9.fsf@elektro.pacujo.net> Steve D'Aprano : > So how would you do non-blocking keyboard input? How would it work? > What would be the interface? https://tronche.com/gui/x/xlib/events/keyboard-pointer/keyboard-pointer.html Marko From __peter__ at web.de Tue Oct 25 14:14:14 2016 From: __peter__ at web.de (Peter Otten) Date: Tue, 25 Oct 2016 20:14:14 +0200 Subject: Reversing \N{...} notation? References: <20161025120612.02194da4@bigbox.christie.dr> Message-ID: Tim Chase wrote: > I like the clarity of using the "\N{...}" notation when creating > string literals involving Unicode chars. > > Is there a built-in way to get such strings back from Python? > > >>> s = 'ma\N{LATIN SMALL LETTER N WITH TILDE}ana' > >>> s > 'ma?ana' > >>> magic(s) > 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' > > I can manually rip the string apart and put it back together with > something like > > >>> import unicodedata as ud > >>> ''.join(c if ord(c) < 128 else '\\N{%s}' % ud.name(c) for c in s) > 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' > > but was wondering if there was some sort of .encode() trick or other > corner of the library I hadn't found. >>> 'ma?ana'.encode("ascii", "namereplace").decode() 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' (requires Python 3.5) From rosuav at gmail.com Tue Oct 25 14:22:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Oct 2016 05:22:51 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <580f82e4$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <580f82e4$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Wed, Oct 26, 2016 at 3:05 AM, Steve D'Aprano wrote: > On Tue, 25 Oct 2016 11:49 pm, Chris Angelico wrote: > >> In Python, Ctrl-C raises KeyboardInterrupt. If you want to save your >> data, use standard exception handling. (And asking to confirm? Isn't >> that exactly what "Press Enter to continue or Ctrl-C to abort" *is*?) > > > $ Fire missiles? Press Enter to continue or Ctrl-C to abort. ^C > $ Are you sure you want to abort? > Press Enter to abort or Ctrl-C to abort the abort. ENTER > $ Did you mean to abort? Press Enter to continue aborting, > or Ctrl-C to abort aborting the abort and continue. ENTER > $ Confirm abort: Enter to abort, Ctrl-C to abort. ENTER > $ Abort aborted. Missiles fired. http://www.gotterdammerung.org/humor/boh13.html (Sadly, the official bofh.ntk.net is down, or I'd link to it there.) ChrisA From nobody at nowhere.invalid Tue Oct 25 15:57:46 2016 From: nobody at nowhere.invalid (Nobody) Date: Tue, 25 Oct 2016 20:57:46 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? References: Message-ID: On Mon, 24 Oct 2016 11:14:05 -0700, jladasky wrote: > I gather that non-blocking keyboard input functions aren't the easiest > thing to implement. They seem to depend on the operating system. Indeed. It's somewhat harder to implement one on an OS which doesn't take it for granted that the system actually *has* a keyboard (i.e. Unix). If you're willing to compromise and accept the use of a terminal rather than a keyboard, the next question is whether to use the process' controlling terminal, or the terminal associated with stdin (if both exist, they're probably the same terminal, but aren't required to be). Other complications include the fact that, if the process isn't part of the terminal's foreground process group, attempting to read from the terminal (even a non-blocking read) will typically suspend the process (unless you ignore SIGTTIN). And also the fact that the terminal itself may be line buffered, so the computer has no idea of what's being typed on it until Return/Enter (or Send, etc) is pressed. Aside from that, receiving key presses as they are entered means disabling canonical mode in the tty driver (which buffers input until Return or Ctrl-D are pressed, so that you can edit the input line with Backspace or Ctrl-U). That affects all processes using the terminal. If the current process is in the foreground process group, then processes in other groups probably won't be reading from the terminal ... at least until you suspend the forground process group with Ctrl-Z. So you need to install signal handlers for SIGTSTP and SIGCONT to restore the terminal settings when the process is suspended. But what should you do about any existing handlers for those signals? All things considered, requiring the user to use one of the keys that generates a signal might be simpler. Or at least not using Esc, which is about the worst possible choice, given that its normal function is as a prefix for, well, just about every control sequence (i.e. what is sent when you press a key which doesn't correspond to a printable character). tl;dr: Unix is not MS-DOS. From marko at pacujo.net Tue Oct 25 17:24:55 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 00:24:55 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: Message-ID: <87lgxcxgt4.fsf@elektro.pacujo.net> Nobody : > On Mon, 24 Oct 2016 11:14:05 -0700, jladasky wrote: >> I gather that non-blocking keyboard input functions aren't the >> easiest thing to implement. They seem to depend on the operating >> system. > > Indeed. It's somewhat harder to implement one on an OS which doesn't > take it for granted that the system actually *has* a keyboard (i.e. > Unix). I don't think that's very relevant. > [...] > controlling terminal, or the terminal associated with stdin [...] > > Other complications include the fact that, if the process isn't part > of the terminal's foreground process group, attempting to read from > the terminal (even a non-blocking read) will typically suspend the > process (unless you ignore SIGTTIN). And also the fact that the > terminal itself may be line buffered, so the computer has no idea of > what's being typed on it until Return/Enter (or Send, etc) is pressed. Arcane schemes that reflect ancient realities. Unix came about when wasting CPU cycles processing terminal commands was considered extravagant -- CPU cycles were charged to the customers. Nowadays, of course, the CPU processes all keyboard events and commits even more atrocious things like micromanages individual pixels on the screen and encrypts and decrypts all external communication. > [...] > > tl;dr: Unix is not MS-DOS. Thankfully, you don't need to run your program from a terminal. You can interpret keyboard events any way you want if your program is an X11 or Wayland client, and forget all about TTYs, baud rates, parity bits, hangups etc. In fact, that's what emacs does, for example. It can operate in a terminal (and I take advantage of that every day) but I'm typing this in emacs running under X11. Marko From rosuav at gmail.com Tue Oct 25 17:57:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Oct 2016 08:57:37 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87lgxcxgt4.fsf@elektro.pacujo.net> References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On Wed, Oct 26, 2016 at 8:24 AM, Marko Rauhamaa wrote: > > Thankfully, you don't need to run your program from a terminal. You can > interpret keyboard events any way you want if your program is an X11 or > Wayland client, and forget all about TTYs, baud rates, parity bits, > hangups etc. That still doesn't answer the fundamental question: Are you looking for KEYBOARD input or TEXT input? Until you figure that out, nothing matters. Personally, I'd much rather work with text than with actual keys; in the uncommon case where I want a keystroke to trigger an action, I'm always working in a GUI already, and I can create a menu item with an accelerator. A visible one. Just because you *can* poll the keyboard for events, don't think that you *should*. ChrisA From bc at freeuk.com Tue Oct 25 18:30:09 2016 From: bc at freeuk.com (BartC) Date: Tue, 25 Oct 2016 23:30:09 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On 25/10/2016 22:57, Chris Angelico wrote: > On Wed, Oct 26, 2016 at 8:24 AM, Marko Rauhamaa wrote: >> >> Thankfully, you don't need to run your program from a terminal. You can >> interpret keyboard events any way you want if your program is an X11 or >> Wayland client, and forget all about TTYs, baud rates, parity bits, >> hangups etc. > > That still doesn't answer the fundamental question: > > Are you looking for KEYBOARD input or TEXT input? Does it matter that much? Because even if you opt for TEXT, the input (when interactive which is what we're talking about) is usually chopped up into LINE events. What's the difference if we want to concentrate on CHAR or KEY events instead? I've used enough line-buffered editors on teletypes and serial VDUs to remember them as horrible to use. Was the input considered TEXT or KEYBOARD? I can't remember but it wasn't really important. And no one has the answered the question of how Curses or a GUI solves the problem of getting char or key events. Same machine, same OS, same keyboard, but one piece of software has apparently discovered the secret which is then denied to other software. > Until you figure that out, nothing matters. Personally, I'd much > rather work with text than with actual keys; in the uncommon case > where I want a keystroke to trigger an action, I'm always working in a > GUI already, and I can create a menu item with an accelerator. A > visible one. Some people want to work at low level, without needing to drag in a GUI, and want to do something as simple as finding out if a button has been pressed on a standard peripheral that nearly every computer has. It can't be that hard! > Just because you *can* poll the keyboard for events, don't think that > you *should*. With my rare forays into GUI, that's exactly what I end up doing: in a loop waiting for events. -- Bartc From rosuav at gmail.com Tue Oct 25 18:58:51 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Oct 2016 09:58:51 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On Wed, Oct 26, 2016 at 9:30 AM, BartC wrote: >> That still doesn't answer the fundamental question: >> >> Are you looking for KEYBOARD input or TEXT input? > > > Does it matter that much? > > Because even if you opt for TEXT, the input (when interactive which is what > we're talking about) is usually chopped up into LINE events. What's the > difference if we want to concentrate on CHAR or KEY events instead? Yes, it does. Text does not include "Home" or "Delete", but it does include all manner of symbols that aren't on everyone's keyboards. It makes a huge difference. Of course, you might want to stick your head in the sand and pretend that every character has a button on your keyboard. ChrisA From bc at freeuk.com Tue Oct 25 19:45:39 2016 From: bc at freeuk.com (BartC) Date: Wed, 26 Oct 2016 00:45:39 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On 25/10/2016 23:58, Chris Angelico wrote: > On Wed, Oct 26, 2016 at 9:30 AM, BartC wrote: >>> That still doesn't answer the fundamental question: >>> >>> Are you looking for KEYBOARD input or TEXT input? >> >> >> Does it matter that much? >> >> Because even if you opt for TEXT, the input (when interactive which is what >> we're talking about) is usually chopped up into LINE events. What's the >> difference if we want to concentrate on CHAR or KEY events instead? > > Yes, it does. Text does not include "Home" or "Delete", but it does > include all manner of symbols that aren't on everyone's keyboards. It > makes a huge difference. Actually TXT files can include codes such as Carriage Return, Backspace and Tab. > Of course, you might want to stick your head in the sand and pretend > that every character has a button on your keyboard. I'm not sure of the relevance of that. There isn't a 1:1 correspondence between key and character code. So 'a' and 'A' might both be entered by pressing the key marked 'A'. With, possibly, some underlying translation so that the 'A' key can vary locations by locale. That isn't new. It doesn't seem to me that anyone is denying that key-at-a-time access is useful in a wide range of programs. They just seem to be unwilling to let it be it available in a simple, standard form. We are expected to download wxPython first. -- Bartc From travisgriggs at gmail.com Tue Oct 25 19:50:37 2016 From: travisgriggs at gmail.com (Travis Griggs) Date: Tue, 25 Oct 2016 16:50:37 -0700 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <53du0cp29hsds6emf094uekak7bs21sbhg@4ax.com> <87y41c635d.fsf@elektro.pacujo.net> <87oa2861io.fsf@elektro.pacujo.net> Message-ID: > On Oct 25, 2016, at 5:55 AM, Chris Angelico wrote: > > On Tue, Oct 25, 2016 at 11:45 PM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> On Tue, Oct 25, 2016 at 11:09 PM, Marko Rauhamaa wrote: >>>> Blocking calls are evil. >>> >>> Oh, that's why. Got it. So because blocking calls are fundamentally >>> evil, we have to... what? What's so bad about them? Remember, not >>> every program is a server handling myriad clients. >> >> Myriads or not, we are talking about interactive (or reactive) programs. >> The paradigm of choice is event-driven programming. > > Have you watched "Tron"? A program goes to the I/O tower to receive a > message from the User. It's an active operation on the part of the > program. The user cannot initiate it, only the program can. > > Tron is extremely accurate in this way. Thanks for this ChrisA. Rest of this thread has been meh for me, but this one post, definitely won my MostValueablePost for the thread. :) Still chuckling. From steve+python at pearwood.info Tue Oct 25 21:02:19 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Wed, 26 Oct 2016 12:02:19 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> On Tue, 25 Oct 2016 09:02 pm, BartC wrote: >> raw_input('Press the Enter key to continue... ') > > Which doesn't work on Python 3. So even here, making it easy by using > line-input, it's not so straightforward. Really, Bart? You're stymied by the change of raw_input() to input() in Python 3? A programmer of your many years experience and skill can't work out how to conditionally change "raw_input" to "input" according to the version of the interpreter running. I don't think so. >> If you are doing something more complex, waiting on different keys to do >> different things, > > You mean, something as sophisticated as press Enter to continue, or > Escape to quit? Or Page Up and Page Down? This discussion is a red herring, because the OP is talking about non-blocking input. So "Press enter to continue" is not a good model for what he had in mind. But regardless, yes, I'd consider using an existing text UI at this point, rather than try re-inventing the wheel. Unless I *like* re-inventing the wheel, or that's what I'm being paid for. If I've got two keys to choose between, I'll probably have more soon: Page Up, Page Down, Line Up, Line Down, Backwards, Forwards, Refresh, Quit, Load File, Reload File ... Somebody once said, there's really only three numbers you need care about. Zero, one and infinity. Zero is easy to handle -- you don't do anything. One is easy, because there's only one thing to do. But once you have two things, you might as well be prepared to handle an indefinitely large number of things, because you're surely going to need to. YMMV. >> then you probably should use an existing text UI like Curses, >> or a GUI like wxPython etc, rather than trying to reinvent the wheel >> badly. > > But why the need to have to use someone else's massive great wheel? Both > Curses and wxPython are completely over the top IMO for something so > fundamental. *shrug* Then find a more lightweight solution. Tkinter? Something even lighter? Maybe there is no such lightweight solution? Then that tells you that nobody else needed this enough to build a solution. Perhaps you're the first! Congratulations! Or perhaps you're looking at the question the wrong way. How many thousands of man-hours, and deaths, came about because people thought that the only way to fly was by flapping wings like a bird? Maybe there are other ways to fly... maybe there are other ways to get a good text UI other than collecting raw keyboard events. I don't know. I've never needed this enough to care to investigate. Like I said, it *seems* like the sort of obvious functionality every programmer should need all the time, but in 15 years I've never, not once, actually needs a non-blocking way to check for a keyboard event in a situation where I wasn't using something like curses or a GUI framework. YMMV. [...] > This gives you the ability to do (2) above. From that, you could do (1) > (echoing) and go on to build full line-orientated input. But you had > complete control. > > So, why has it all become so difficult? Build line oriented input? Why would I do that, when the OS does it? Okay, sure, if you're programming for some sort of primitive system with no OS or such a feeble one that it didn't even offer line-oriented text I/O, then needs must, and you have to do what the OS doesn't provide. But that sort of low-level I/O is precisely what operating systems are for. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Tue Oct 25 22:25:33 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 25 Oct 2016 22:25:33 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On 10/25/2016 7:45 PM, BartC wrote: > On 25/10/2016 23:58, Chris Angelico wrote: >> Yes, it does. Text does not include "Home" or "Delete", but it does >> include all manner of symbols that aren't on everyone's keyboards. It >> makes a huge difference. > > Actually TXT files can include codes such as Carriage Return, Backspace > and Tab. These are called 'control *characters*' because, like characters, they are represented as encoded bytes or, now, as unicode codepoints. 'Home' is not a control character. (I don't know if there is a unicode codepoint for it.) >> Of course, you might want to stick your head in the sand and pretend >> that every character has a button on your keyboard. > > I'm not sure of the relevance of that. There isn't a 1:1 correspondence > between key and character code. So 'a' and 'A' might both be entered by > pressing the key marked 'A'. With, possibly, some underlying translation > so that the 'A' key can vary locations by locale. That isn't new. The translation used to be done by a physically separate terminal and communication between computer and and terminal was via encoded bytes. A program could not access keystrokes because the computer never saw them. Python normally runs in a terminal or console that translates keycodes to encoded bytes > It doesn't seem to me that anyone is denying that key-at-a-time access > is useful in a wide range of programs. They just seem to be unwilling to > let it be it available in a simple, standard form. The simple, standard form for Python, as well as for tcl, perl, and ruby, is as tk events. Python accesses then via tkinter. Other languages have their own version of the API. > We are expected to download wxPython first. Since this is not true, why do you keep repeating it? -- Terry Jan Reedy From torriem at gmail.com Tue Oct 25 22:37:25 2016 From: torriem at gmail.com (Michael Torrie) Date: Tue, 25 Oct 2016 20:37:25 -0600 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <580f86ac$0$1599$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <1477406161.3036984.766774953.79E4788A@webmail.messagingengine.com> <580f86ac$0$1599$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87be7e70-9748-f2c2-3034-a341c0ab5c64@gmail.com> On 10/25/2016 10:22 AM, Steve D'Aprano wrote: > So how would you do non-blocking keyboard input? How would it work? What > would be the interface? Curses must allow you to do this because I've seen text-mode games made in curses and you could do things with arrow keys, etc, all while ascii animation was going on. Now it could be that curses forces you into an event-driven paradigm. Not sure about that. But even if it did, your event loop could run a callback that would deposit keystrokes (if any) into another buffer of some kind that your non-event-driven thread could peak into. From jobmattcon at gmail.com Tue Oct 25 22:55:23 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Tue, 25 Oct 2016 19:55:23 -0700 (PDT) Subject: is it possible use python to run another main point or function in some range of memory in executable file Message-ID: is it possible python to run another main point or function in some range of memory in executable file From jobmattcon at gmail.com Tue Oct 25 22:59:37 2016 From: jobmattcon at gmail.com (meInvent bbird) Date: Tue, 25 Oct 2016 19:59:37 -0700 (PDT) Subject: how to evaluate a ast tree and change Add to Multiply ? In-Reply-To: References: <76cba98f-01db-44e9-9450-a64fedf3e5e3@googlegroups.com> <2d3bf63e-9638-4e33-a50a-01a3face8b65@googlegroups.com> Message-ID: <1183377b-23bd-42ee-9c9b-28ea135d5034@googlegroups.com> Hi Jerry, how about custom function? i change to node.op = ast.op2() import ast def op2(a,b): return a*b+a class ChangeAddToMultiply(ast.NodeTransformer): """Wraps all integers in a call to Integer()""" def visit_BinOp(self, node): if isinstance(node.op, ast.Add): node.op = ast.op2() return node code = 'print(2+5)' tree = ast.parse(code) tree = ChangeAddToMultiply().visit(tree) ast.fix_missing_locations(tree) co = compile(tree, '', "exec") exec(code) exec(co) On Thursday, October 20, 2016 at 12:34:55 AM UTC+8, Jerry Hill wrote: > On Wed, Oct 12, 2016 at 5:55 AM, meInvent bbird wrote: > > i just expect to > > rewrite + become multiply > > by edit the example in the link provided > > This seems to work. You need to define visit_BinOp instead of > visit_Num (since you want to mess with the binary operations, not the > numbers). Then,in visit_BinOp, we just replace the ast.Add node with > an ast.Mult node. > > import ast > > class ChangeAddToMultiply(ast.NodeTransformer): > """Wraps all integers in a call to Integer()""" > def visit_BinOp(self, node): > if isinstance(node.op, ast.Add): > node.op = ast.Mult() > return node > > code = 'print(2+5)' > tree = ast.parse(code) > tree = ChangeAddToMultiply().visit(tree) > ast.fix_missing_locations(tree) > co = compile(tree, '', "exec") > > exec(code) > exec(co) > > -- > Jerry From julimadoz at gmail.com Tue Oct 25 23:19:49 2016 From: julimadoz at gmail.com (julimadoz at gmail.com) Date: Tue, 25 Oct 2016 20:19:49 -0700 (PDT) Subject: UDP decode Message-ID: <53d78569-b6f9-45c4-98bf-9b3f556dd8ec@googlegroups.com> Hi! I'm using Kinect with OSCeleton and it is sending the data thru UDP. I can receive it in python, but i can't decode it. I have this: -------------------------------------------------- import socket UDP_IP = "127.0.0.1" UDP_PORT = 7110 sock = socket.socket(socket.AF_INET, # Internet socket.SOCK_DGRAM) # UDP sock.bind((UDP_IP, UDP_PORT)) while True: data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes print(data) -------------------------------------------------- And I'm receiving something like this: -------------------------------------------------- b'/osceleton2/joint\x00\x00\x00,siiffffd\x00\x00\x00head\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05C\xec\xad&C\xa0\x81vDG\x84N?\x80\x00\x00Bu\x7f\xedh5f\x9b' -------------------------------------------------- It supouse to be this: -------------------------------------------------- Address pattern: "/osceleton2/joint" Type tag: "siiffffd" s: Joint name, check out the full list of joints below i: The ID of the sensor i: The ID of the user f: X coordinate of joint in real world coordinates (centimetres) f: Y coordinate of joint in real world coordinates (centimetres) f: Z coordinate of joint in real world coordinates (centimetres) f: confidence value in interval [0.0, 1.0] d: time stamp in milliseconds since Unix epoch example: /osceleton2/joint head 0 1 109.07692 54.557518 666.81543 1. 0. -------------------------------------------------- Can somebody help me? I worked with OSCeleton in Max/MSP and was fantastic, but here i have this problem. Thanks in advice! OSCeleton: https://github.com/Zillode/OSCeleton-KinectSDK2 From rosuav at gmail.com Tue Oct 25 23:41:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 26 Oct 2016 14:41:22 +1100 Subject: UDP decode In-Reply-To: <53d78569-b6f9-45c4-98bf-9b3f556dd8ec@googlegroups.com> References: <53d78569-b6f9-45c4-98bf-9b3f556dd8ec@googlegroups.com> Message-ID: On Wed, Oct 26, 2016 at 2:19 PM, wrote: > b'/osceleton2/joint\x00\x00\x00,siiffffd\x00\x00\x00head\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05C\xec\xad&C\xa0\x81vDG\x84N?\x80\x00\x00Bu\x7f\xedh5f\x9b' > -------------------------------------------------- > > > It supouse to be this: > -------------------------------------------------- > Address pattern: "/osceleton2/joint" > Type tag: "siiffffd" > s: Joint name, check out the full list of joints below > i: The ID of the sensor > i: The ID of the user > f: X coordinate of joint in real world coordinates (centimetres) > f: Y coordinate of joint in real world coordinates (centimetres) > f: Z coordinate of joint in real world coordinates (centimetres) > f: confidence value in interval [0.0, 1.0] > d: time stamp in milliseconds since Unix epoch > > example: > > /osceleton2/joint head 0 1 109.07692 54.557518 666.81543 1. 0. Well, the first bit clearly matches. Then you have four bytes ending with a comma (0x2C) and then siiffffd. A few more NULs and "head". So really, the interesting part starts after that. Can you find information you recognize in any of the bytes that follow that? ChrisA From marko at pacujo.net Wed Oct 26 00:44:35 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 07:44:35 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: <87d1inyb0s.fsf@elektro.pacujo.net> BartC : > And no one has the answered the question of how Curses or a GUI solves > the problem of getting char or key events. Same machine, same OS, same > keyboard, but one piece of software has apparently discovered the > secret which is then denied to other software. Curses, emacs, vi, bash, CPython, nethack etc "solve the problem" by setting the terminal mode. You can, too: https://docs.python.org/3/library/termios.html https://docs.python.org/3/library/tty.html However, that only gives you access to the interpreted characters. For example, you can't see when a [Shift] key has been pressed. > Some people want to work at low level, without needing to drag in a GUI, > and want to do something as simple as finding out if a button has been > pressed on a standard peripheral that nearly every computer has. It > can't be that hard! I don't consider that to be very low level. If you want to get to the low level, open /dev/input/by-id/*-event-kbd See: http://stackoverflow.com/questions/3662368/dev-input-keyboard-format Marko From pozzugno at gmail.com Wed Oct 26 03:13:51 2016 From: pozzugno at gmail.com (pozz) Date: Wed, 26 Oct 2016 09:13:51 +0200 Subject: How to use two threads (GUI and backend) Message-ID: I'm designing a GUI application in Python (with pyGObject, so GTK). The application communicates with a remote device (connected through RS232, but it could be on Internet) to retrieve its status and set/get its configuration. When the user press "Start" button, the application starts sending "GET STATUS" requests to the remote device, waiting its response. When the response arrives, the GUI widgets are refreshed with the new status. The "GET STATUS" requests are send at a regular frequency (polling mode). I thought two split the application in two threads: the GUI main thread that manages graphical widgets and user interaction; the backend thread that manages low-level communication. When the user press Start button (the pressed handler is in the GUI class): self.comm_active = True threading.Thread(target=self.comm_thread).start() The backend thread is: def comm_thread(self): while self.comm_active: self.device.get_status() GLib.idle_add(self.polling_received) time.sleep(1) self.m.close() self.device.get_status() is blocking. It is executed in backend thread, so the GUI isn't blocked. self.polling_received() function will be executed in main thread (thanks to GLib.idle_add), because it will change widgets properties. Now the get_stats() of self.comm object: def get_status(self): self.property1 = self.property2 = return And self.polling_received() of GUI class: def polling_received(self): txtEntry1.set_text(self.comm.property1) txtEntry2.set_text(self.comm.property2) I didn't fully tested this, but it seems it works well. However I have some concerns, mainly for thread syncronizations. self.polling_received() is executed in GUI thread and reads properties (self.comm.property1, ...) that are changed during parsing of responses in self.comm.get_status() function that is executed in the backend thread. So the two threads use the same variables/objects without synchronization. Is this a problem? What is the best approach to use in my scenario (GUI and backend communication)? From pozzugno at gmail.com Wed Oct 26 03:44:19 2016 From: pozzugno at gmail.com (pozz) Date: Wed, 26 Oct 2016 09:44:19 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: Il 26/10/2016 09:13, pozz ha scritto: > [...] > What is the best approach to use in my scenario (GUI and backend > communication)? I just found this[1] page, where the thread approach is explained with the following code: --- import threading import time from gi.repository import GLib, Gtk, GObject def app_main(): win = Gtk.Window(default_height=50, default_width=300) win.connect("delete-event", Gtk.main_quit) progress = Gtk.ProgressBar(show_text=True) win.add(progress) def update_progess(i): progress.pulse() progress.set_text(str(i)) return False def example_target(): for i in range(50): GLib.idle_add(update_progess, i) time.sleep(0.2) win.show_all() thread = threading.Thread(target=example_target) thread.daemon = True thread.start() if __name__ == "__main__": # Calling GObject.threads_init() is not needed for PyGObject 3.10.2+ GObject.threads_init() app_main() --- This is similar to my approach, with a main difference: the callback update_progress() added to the GLib idle loop (so executed in the main GUI thread) receives all the data as arguments (the value i to write as text in the progress widget). In my case, I have many many properties of the remote device. So my first idea is to get directly the value by accessing variables changed during backend thread... I think this is wrong. [1] https://wiki.gnome.org/Projects/PyGObject/Threading From lucaberto at libero.it Wed Oct 26 05:36:51 2016 From: lucaberto at libero.it (luca72) Date: Wed, 26 Oct 2016 02:36:51 -0700 (PDT) Subject: Qtimer and extra argument Message-ID: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> Hello i hope that yo can reply to this question also if the argument is pyqt i have a simple test: def start_timer(self): self.timer = QTimer() testo = 'pressed' self.timer.singleShot(1000, self.metto_testo) def test(self, testo): self.lineEdit.setText(testo) How i can pass the variable testo to the def test? If i use : self.timer.singleShot(1000, self.metto_testo(testo) i get error Thanks From daiyueweng at gmail.com Wed Oct 26 05:38:34 2016 From: daiyueweng at gmail.com (Daiyue Weng) Date: Wed, 26 Oct 2016 10:38:34 +0100 Subject: ImportError: No module named 'simstring' Message-ID: Hi, I am trying to install simstring on Windows 10 through pip/conda, but couldn't find the package. I am using Pycharm + Anaconda, so I am wondering is it possible to install simstring on Windows, and how. Cheers From ben+python at benfinney.id.au Wed Oct 26 05:47:29 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 26 Oct 2016 20:47:29 +1100 Subject: ImportError: No module named 'simstring' References: Message-ID: <85insfcuha.fsf@benfinney.id.au> Daiyue Weng writes: > Hi, I am trying to install simstring on Windows 10 through pip/conda, but > couldn't find the package. Pip installs packages listed on the Python Package Index (PyPI) , do any of the search results there match what you want? -- \ ?Pinky, are you pondering what I'm pondering?? ?I think so, | `\ Brain, but can the Gummi Worms really live in peace with the | _o__) Marshmallow Chicks?? ?_Pinky and The Brain_ | Ben Finney From phil at riverbankcomputing.com Wed Oct 26 05:55:55 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Wed, 26 Oct 2016 10:55:55 +0100 Subject: Qtimer and extra argument In-Reply-To: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> References: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> Message-ID: On 26 Oct 2016, at 10:36 am, luca72 via Python-list wrote: > > Hello i hope that yo can reply to this question also if the argument is pyqt > > i have a simple test: > def start_timer(self): > self.timer = QTimer() > testo = 'pressed' > self.timer.singleShot(1000, self.metto_testo) > > def test(self, testo): > self.lineEdit.setText(testo) > > > How i can pass the variable testo to the def test? > If i use : > self.timer.singleShot(1000, self.metto_testo(testo) > i get error Use a lambda... self.timer.singleShot(1000, lambda s: self.metto_testo(testo)) Phil From pozzugno at gmail.com Wed Oct 26 06:22:53 2016 From: pozzugno at gmail.com (pozz) Date: Wed, 26 Oct 2016 12:22:53 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: Il 26/10/2016 09:13, pozz ha scritto: > [...] > When the user press Start button (the pressed handler is in the GUI class): > > self.comm_active = True > threading.Thread(target=self.comm_thread).start() > > The backend thread is: > > def comm_thread(self): > while self.comm_active: > self.device.get_status() > GLib.idle_add(self.polling_received) > time.sleep(1) > self.m.close() > [...] Now I have some concerns even in using self.comm_active. It is a boolean variable accessed by the GUI thread (inside Start/Stop buttons handler) and backend thread (in the "while self.comm_active" instruction). Is it safe to access this variable from two different threads? Should I implement a safer and more complex mechanism? If yes, what mechanism? From ben+python at benfinney.id.au Wed Oct 26 06:37:04 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 26 Oct 2016 21:37:04 +1100 Subject: [RELEASE] =?utf-8?Q?=E2=80=98python-daemon?= =?utf-8?B?4oCZ?= version 2.1.2 released Message-ID: <20161026103704.uqxhufpcgde4hfn3@benfinney.id.au> Howdy all, I am pleased to announce the release of version 2.1.2 of the ?python-daemon? library. The current release is always available at . Significant changes since the previous version ============================================== Additions: * Add a README document for the code base. Changes: * Migrate code project hosting to Pagure. Record the change of homepage URL in PyPI metadata. * Raise a warning that the ?runner? module is pending deprecation. This has been an unofficial example module from the beginning, and it will be removed in a future version. Bug Fixes: * Ensure custom types are part of the Python type hierarchy. * Avoid a circular dependency for the version string at install time. Thanks to Maarten van Gompel for the reproducible test case. What is the ?python-daemon? library? ==================================== ?python-daemon? is a Python library to implement a well-behaved Unix daemon process. -- \ ?The best is the enemy of the good.? ?Voltaire, _Dictionnaire | `\ Philosophique_ | _o__) | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: not available URL: From jeanmichel at sequans.com Wed Oct 26 07:16:25 2016 From: jeanmichel at sequans.com (jmp) Date: Wed, 26 Oct 2016 13:16:25 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On 10/26/2016 12:22 PM, pozz wrote: > Il 26/10/2016 09:13, pozz ha scritto: > > [...] >> When the user press Start button (the pressed handler is in the GUI >> class): >> >> self.comm_active = True >> threading.Thread(target=self.comm_thread).start() >> >> The backend thread is: >> >> def comm_thread(self): >> while self.comm_active: >> self.device.get_status() >> GLib.idle_add(self.polling_received) >> time.sleep(1) >> self.m.close() > > [...] > > > Now I have some concerns even in using self.comm_active. It is a > boolean variable accessed by the GUI thread (inside Start/Stop buttons > handler) and backend thread (in the "while self.comm_active" instruction). > > Is it safe to access this variable from two different threads? Should I > implement a safer and more complex mechanism? If yes, what mechanism? > from http://nedbatchelder.com/blog/201204/two_problems.html Some people, when confronted with a problem, think, "I know, I'll use threads," and then two they hav erpoblesms. I suggest you write a GUI that make synchronous calls to a remote application, if possible. If the remote app is in python, you have access to remote protocols already written for you, Pyro is one of them, you can skip the low level communication part. jm From bc at freeuk.com Wed Oct 26 07:18:56 2016 From: bc at freeuk.com (BartC) Date: Wed, 26 Oct 2016 12:18:56 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 26/10/2016 02:02, Steve D'Aprano wrote: > On Tue, 25 Oct 2016 09:02 pm, BartC wrote: > >>> raw_input('Press the Enter key to continue... ') >> >> Which doesn't work on Python 3. So even here, making it easy by using >> line-input, it's not so straightforward. > > Really, Bart? You're stymied by the change of raw_input() to input() in > Python 3? A programmer of your many years experience and skill can't work > out how to conditionally change "raw_input" to "input" according to the > version of the interpreter running. I don't think so. I could probably figure it out. But how about a beginner? (When I post Python code I try and make sure it works on both, or specify which version.) >> But why the need to have to use someone else's massive great wheel? Both >> Curses and wxPython are completely over the top IMO for something so >> fundamental. > > *shrug* Then find a more lightweight solution. Tkinter? Something even > lighter? Can tkinter do it without creating a distracting pop-up window at the same time? If not then that's too intrusive. Curses? If I try 'import curses' (on Windows), I get 'No module named '_curses'. Same with ncurses or Curses. Now instead of getting on with it I have to go chasing some add-on. And if I want to share a bit of code with someone else, then /they/ have to do the same! My point is that this simple stuff just be included in a language. > Maybe there is no such lightweight solution? Then that tells you that nobody > else needed this enough to build a solution. I find this a lot with stuff that originates on Unix or Linux. (Do you know when I create executables that run on Linux, I have to try and cram all the support files within the executable itself - because it appears to be impossible to discover the path the executable was started from. With a view to using the same or adjoining path for associated files. When I ask about this, Oh it's never been needed! Same with basic keyboard stuff that any microcomputer from the 80s could do in an instant.) > I don't know. I've never needed this enough to care to investigate. Like I > said, it *seems* like the sort of obvious functionality every programmer > should need all the time, but in 15 years I've never, not once, actually > needs a non-blocking way to check for a keyboard event in a situation where > I wasn't using something like curses or a GUI framework. By non-blocking you mean checking if a key has been pressed rather than waiting for it to be pressed? I use the latter ALL THE TIME when debugging. The former less often, because if I wanted to use it to abort huge amounts of output, I just abort the program (with Ctrl Break). Nevertheless, it is still used sometimes, and it's there when I need it (NOT Python): repeat print "A" until testkey() >> So, why has it all become so difficult? > > Build line oriented input? Why would I do that, when the OS does it? Sometimes you want your own line-input functions because they need to be customised to do special things. Assign special meanings to certain key events for example. You make it sound completely crazy like building your own OS. But I've done this dozens of times (and yes including when there /was/ no OS), it really isn't a big deal. Except when a language strives to make it so. > I don't know. I've never needed this enough to care to investigate. Try this little task. In the late 70s I was fascinated by the ability of a teletype to do this: it would ask a Yes/No question, perhaps: .CONFIRM (YES/NO): But then, instead of typing YES or NO, if either Y or N was pressed, it would auto-complete it, printing 'ES' or 'O' and doing a Return for you ("." is the prompt; you have to imagine the clatter): .CONFIRM (YES/NO): YES . How hard is it to do this nearly 40 years On a machines a million times more powerful? And in Python (although I'd imagine any difficulties it has originate outside the language and probably affect others). Bartc From antoon.pardon at rece.vub.ac.be Wed Oct 26 07:27:02 2016 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Wed, 26 Oct 2016 13:27:02 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> Op 26-10-16 om 12:22 schreef pozz: > Il 26/10/2016 09:13, pozz ha scritto: > > [...] >> When the user press Start button (the pressed handler is in the GUI >> class): >> >> self.comm_active = True >> threading.Thread(target=self.comm_thread).start() >> >> The backend thread is: >> >> def comm_thread(self): >> while self.comm_active: >> self.device.get_status() >> GLib.idle_add(self.polling_received) >> time.sleep(1) >> self.m.close() > > [...] > > Now I have some concerns even in using self.comm_active. It is a boolean variable > accessed by the GUI thread (inside Start/Stop buttons handler) and backend thread > (in the "while self.comm_active" instruction). > Is it safe to access this variable from two different threads? > Should I implement a safer and more complex mechanism? If yes, what mechanism? Accessing from multiple thread shouldn't be a problem. As long as you only change it in one thread. From bc at freeuk.com Wed Oct 26 07:34:30 2016 From: bc at freeuk.com (BartC) Date: Wed, 26 Oct 2016 12:34:30 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87d1inyb0s.fsf@elektro.pacujo.net> References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> Message-ID: On 26/10/2016 05:44, Marko Rauhamaa wrote: > BartC : >> Some people want to work at low level, without needing to drag in a GUI, >> and want to do something as simple as finding out if a button has been >> pressed on a standard peripheral that nearly every computer has. It >> can't be that hard! > > I don't consider that to be very low level. I think working in text mode (character-based display) and a key at a time is enough to be called low-level. Lower than that would be directly working with the hardware. But it gets more specific as keyboards work in different ways. That's not so useful or interesting. (I've implemented 'keyboards' both on-screen, and on the surface of digitising tablets (also with a hacked Casio calculator pcb when I couldn't afford a real one). With all of those I was mainly interested in key events, not the details.) If you want to get to the > low level, open > > /dev/input/by-id/*-event-kbd > > See: > > http://stackoverflow.com/questions/3662368/dev-input-keyboard-format That's not what I'd call low-level. It's more building a mountain of complexity around something that ought to be straightforward. Apparently when you need to dig very deeply to get through to the fundamentals, that's now called 'low-level'! -- bartc From lucaberto at libero.it Wed Oct 26 08:29:27 2016 From: lucaberto at libero.it (luca72) Date: Wed, 26 Oct 2016 05:29:27 -0700 (PDT) Subject: Qtimer and extra argument In-Reply-To: References: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> Message-ID: <90557b54-bf2c-4c0c-9665-ae0355a27dba@googlegroups.com> I get () missing 1 required positional argument: 's' From marko at pacujo.net Wed Oct 26 08:33:15 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 15:33:15 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> Message-ID: <87insf5lys.fsf@elektro.pacujo.net> BartC : > On 26/10/2016 05:44, Marko Rauhamaa wrote: > (I've implemented 'keyboards' both on-screen, and on the surface of > digitising tablets (also with a hacked Casio calculator pcb when I > couldn't afford a real one). With all of those I was mainly interested > in key events, not the details.) Say you want to implement a simple, character-based shooting game where the two guns are operated by the [Shift] keys. Unfortunately, the Unix terminal API doesn't make that possible. You need to get the keyboard events from some other API. In practice, your only choice is X11/Wayland (on Linux). > It's more building a mountain of complexity around something that > ought to be straightforward. Maybe there should be some way to get the raw events from the PTY. However, next you'd start wanting the mouse events and pixel-level color controls. It starts to look like a GUI application. But what would be wrong in a GUI PTY API? No windowing, just a regular character display where you could draw pictures and interpret the inputs directly ? la Apple II or Commodore 64. It would make teaching programming much more fun, too. Marko From julimadoz at gmail.com Wed Oct 26 08:33:42 2016 From: julimadoz at gmail.com (Julian Madoz) Date: Wed, 26 Oct 2016 05:33:42 -0700 (PDT) Subject: UDP decode In-Reply-To: References: <53d78569-b6f9-45c4-98bf-9b3f556dd8ec@googlegroups.com> Message-ID: <7437d47c-dc5e-4487-ae6c-583dc4b5800b@googlegroups.com> Thanks for response! I don't know what can appear because the values changes 30 times per second. I only know the format. From pozzugno at gmail.com Wed Oct 26 08:34:58 2016 From: pozzugno at gmail.com (pozz) Date: Wed, 26 Oct 2016 14:34:58 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> Message-ID: Il 26/10/2016 13:27, Antoon Pardon ha scritto: > Op 26-10-16 om 12:22 schreef pozz: >> Il 26/10/2016 09:13, pozz ha scritto: >>> [...] >>> When the user press Start button (the pressed handler is in the GUI >>> class): >>> >>> self.comm_active = True >>> threading.Thread(target=self.comm_thread).start() >>> >>> The backend thread is: >>> >>> def comm_thread(self): >>> while self.comm_active: >>> self.device.get_status() >>> GLib.idle_add(self.polling_received) >>> time.sleep(1) >>> self.m.close() >>> [...] >> >> Now I have some concerns even in using self.comm_active. It is a boolean variable >> accessed by the GUI thread (inside Start/Stop buttons handler) and backend thread >> (in the "while self.comm_active" instruction). > >> Is it safe to access this variable from two different threads? >> Should I implement a safer and more complex mechanism? If yes, what mechanism? > > Accessing from multiple thread shouldn't be a problem. As long as you only change > it in one thread. I don't want to doubt what you have written, but... are you definitevely sure? I tried to search for some authoritative documentation about this topic, but I couldn't find any. I have many years of experiece in embedded firmware written in C for small microcontrollers, so I know the problems that could occur when a variable is read in one ISR (interrupt service routine) and written in the main loop (or viceversa). ISR and main loop can be considered two threads. If the variable is 32-bits and the microcontroller can't write atomically (without any interruption) a 32-bit variable, bad things could occur. If the main loop is updating the variable from 0x01020304 to 0xA1A2A3A4 and the change happens on a byte basis, the ISR could access a completely wrong value, for example 0x0102A3A4. So the main question here is: does python *specification/standard* guarantees atomic operations? If yes, what are they? From pozzugno at gmail.com Wed Oct 26 08:45:48 2016 From: pozzugno at gmail.com (pozz) Date: Wed, 26 Oct 2016 14:45:48 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: Il 26/10/2016 13:16, jmp ha scritto: > [...] > I suggest you write a GUI that make synchronous calls to a remote > application, if possible. If the remote app is in python, you have > access to remote protocols already written for you, Pyro is one of them, > you can skip the low level communication part. I'm not sure Pyro (or similar alternatives) helps in my case. The real problem is that retrieving status from remote device is a slow operation. If the GUI thread blocks waiting for the answer, the GUI blocks and the user complains. From Pyro documentation: --- Normal method calls always block until the response is returned. This can be any normal return value, None, or an error in the form of a raised exception. The client code execution is suspended until the method call has finished and produced its result. --- So, even with Pyro, I need to have another thread that manages Pyro communication (instead of serial communication)... additional problems. From marko at pacujo.net Wed Oct 26 08:58:45 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 15:58:45 +0300 Subject: How to use two threads (GUI and backend) References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> Message-ID: <87eg335ksa.fsf@elektro.pacujo.net> pozz : > Il 26/10/2016 13:27, Antoon Pardon ha scritto: >> Op 26-10-16 om 12:22 schreef pozz: >>> Is it safe to access this variable from two different threads? >>> Should I implement a safer and more complex mechanism? If yes, what >>> mechanism? >> >> Accessing from multiple thread shouldn't be a problem. As long as you >> only change it in one thread. > > I don't want to doubt what you have written, but... are you > definitevely sure? I tried to search for some authoritative > documentation about this topic, but I couldn't find any. I didn't check but I would guess you are right. That kind of authoritative statement is not made explicitly. Java, on the other hand, has been documented nicely: > If the main loop is updating the variable from 0x01020304 to > 0xA1A2A3A4 and the change happens on a byte basis, the ISR could > access a completely wrong value, for example 0x0102A3A4. > > So the main question here is: does python *specification/standard* > guarantees atomic operations? If yes, what are they? Python guarantees that even a pathological Python application program that only employs ordinary, safe operations cannot crash Python. It follows (de facto) that "pointers" must be protected against race conditions and other artifacts. Similarly, you can't render lists, dicts and other complex data structures incoherent with any ordinary means regardless of race conditions. Since Python's integers are (really or conceptually) objects behind pointers, any Python implementation would be considered out of compliance if it didn't guarantee either 0x01020304 or 0xa1a2a3a4 in your example. In practice, this coherency has been implemented in CPython with a global lock (GIL). CPython programs are effectively single-threaded. They only let go of the lock when they are performing a system call. I can't think of a valid program that could take advantage of this primitive guarantee of Python's. For example, there is no "volatile" in Python so you can't coordinate Python threads safely without proper synchronization. If you set a variable in one thread and read it in another thread, the latter might never see the change. Marko From marko at pacujo.net Wed Oct 26 09:02:46 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 16:02:46 +0300 Subject: How to use two threads (GUI and backend) References: Message-ID: <87a8dr5kll.fsf@elektro.pacujo.net> pozz : > The real problem is that retrieving status from remote device is a > slow operation. If the GUI thread blocks waiting for the answer, the > GUI blocks and the user complains. Correct. Obnoxious, blocking APIs abound. However, I have usually used processes (instead of threads) to encapsulate blocking APIs. Processes have neater resource isolation and a better-behaving life cycle. For example, you can actually kill a process while you can't kill a thread. Marko From rosuav at gmail.com Wed Oct 26 09:05:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 00:05:41 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: <87eg335ksa.fsf@elektro.pacujo.net> References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> Message-ID: On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa wrote: > In practice, this coherency has been implemented in CPython with a > global lock (GIL). CPython programs are effectively single-threaded. > They only let go of the lock when they are performing a system call. > > I can't think of a valid program that could take advantage of this > primitive guarantee of Python's. For example, there is no "volatile" in > Python so you can't coordinate Python threads safely without proper > synchronization. If you set a variable in one thread and read it in > another thread, the latter might never see the change. Incorrect. If you set something in one thread and read it in another, it WILL see it, just as it would with any other way of running two functions. (Obviously function locals won't be seen, because they never will.) Global state is shared across all threads. pozz, a classic worker model like you're suggesting will be fine IMO. Go for it. ChrisA From phil at riverbankcomputing.com Wed Oct 26 09:12:25 2016 From: phil at riverbankcomputing.com (Phil Thompson) Date: Wed, 26 Oct 2016 14:12:25 +0100 Subject: Qtimer and extra argument In-Reply-To: <90557b54-bf2c-4c0c-9665-ae0355a27dba@googlegroups.com> References: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> <90557b54-bf2c-4c0c-9665-ae0355a27dba@googlegroups.com> Message-ID: On 26 Oct 2016, at 1:29 pm, luca72 via Python-list wrote: > > I get () missing 1 required positional argument: 's' Sorry, it should have been... lambda: self.metto_testo(testo) Phil From rosuav at gmail.com Wed Oct 26 09:18:50 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 00:18:50 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: <87a8dr5kll.fsf@elektro.pacujo.net> References: <87a8dr5kll.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 27, 2016 at 12:02 AM, Marko Rauhamaa wrote: > pozz : > >> The real problem is that retrieving status from remote device is a >> slow operation. If the GUI thread blocks waiting for the answer, the >> GUI blocks and the user complains. > > Correct. Obnoxious, blocking APIs abound. > > However, I have usually used processes (instead of threads) to > encapsulate blocking APIs. Processes have neater resource isolation and > a better-behaving life cycle. For example, you can actually kill a > process while you can't kill a thread. Why is there so much FUD against threads? Processes involve a lot more overhead, and are completely unnecessary for this task. Threads will work just fine. ChrisA From marko at pacujo.net Wed Oct 26 09:37:36 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 16:37:36 +0300 Subject: How to use two threads (GUI and backend) References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> Message-ID: <8760of5izj.fsf@elektro.pacujo.net> Chris Angelico : > On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa wrote: >> I can't think of a valid program that could take advantage of this >> primitive guarantee of Python's. For example, there is no "volatile" >> in Python so you can't coordinate Python threads safely without >> proper synchronization. If you set a variable in one thread and read >> it in another thread, the latter might never see the change. > > Incorrect. If you set something in one thread and read it in another, > it WILL see it, just as it would with any other way of running two > functions. (Obviously function locals won't be seen, because they > never will.) Global state is shared across all threads. I don't know what "Global state is shared across all threads" means in this context. It sounds like something that would be true for, say, Java and C as well. However, those languages don't promise to propagate improperly synchronized changes between threads. Now I would like to ask for some documentation. Marko From rosuav at gmail.com Wed Oct 26 09:51:32 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 00:51:32 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: <8760of5izj.fsf@elektro.pacujo.net> References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 27, 2016 at 12:37 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> On Wed, Oct 26, 2016 at 11:58 PM, Marko Rauhamaa wrote: >>> I can't think of a valid program that could take advantage of this >>> primitive guarantee of Python's. For example, there is no "volatile" >>> in Python so you can't coordinate Python threads safely without >>> proper synchronization. If you set a variable in one thread and read >>> it in another thread, the latter might never see the change. >> >> Incorrect. If you set something in one thread and read it in another, >> it WILL see it, just as it would with any other way of running two >> functions. (Obviously function locals won't be seen, because they >> never will.) Global state is shared across all threads. > > I don't know what "Global state is shared across all threads" means in > this context. It sounds like something that would be true for, say, Java > and C as well. However, those languages don't promise to propagate > improperly synchronized changes between threads. > > Now I would like to ask for some documentation. Here you have two functions and a global: active = True def func1(): while active: # do work time.sleep(1) func2() def func2(): global active if random.random() < 0.1: active = False I'm sure you understand that these functions share the global state of the 'active' flag. One changes it, the other sees the change. So far, nothing controversial or difficult. It's exactly the same with threads. If you remove the func2() call from func1 and have it operate on a separate thread, and then call func2 from the main thread (or another secondary thread), func1 will notice the change the very next time it gets to the top of the loop. The two functions are executing in the same process, the same module, the same everything except their call stack (ie locals). ChrisA From __peter__ at web.de Wed Oct 26 09:56:42 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 26 Oct 2016 15:56:42 +0200 Subject: lxml and xpath(?) References: <9ceddc7b-a3d0-479a-a009-29c4bdc15e72@googlegroups.com> Message-ID: Doug OLeary wrote: > Hey; > > Reasonably new to python and incredibly new to xml much less trying to > parse it. I need to identify cluster nodes from a series of weblogic xml > configuration files. I've figured out how to get 75% of them; now, I'm > going after the edge case and I'm unsure how to proceed. > > Weblogic xml config files start with namespace definitions then a number > of child elements some of which have children of their own. > > The element that I'm interested in is which will usually have a > subelement called containing the hostname that I'm > looking for. > > Following the paradigm of "we love standards, we got lots of them", this > model doesn't work everywhere. Where it doesn't work, I need to look for a > subelement of called . That element contains an alias > which is expanded in a different root child, at the same level as > . > > So, picture worth a 1000 words: > > > < [[ heinous namespace xml snipped ]] > > [[text]] > ... > > EDIServices_MS1 > ... > EDIServices_MC1 > ... > > > EDIServices_MS2 > ... > EDIServices_MC2 > ... > > > EDIServices_MC1 > > EDIServices_MC1 > SSL > host001 > 7001 > > > > EDIServices_MC2 > > EDIServices_MC2 > host002 > 7001 > > > > > So, running it on 'normal' config, I get: > > $ ./lxml configs/EntsvcSoa_Domain_config.xml > EntsvcSoa_CS => host003.myco.com > EntsvcSoa_CS => host004.myco.com > > Running it against the abi-normal config, I'm currently getting: > > $ ./lxml configs/EDIServices_Domain_config.xml > EDIServices_CS => EDIServices_MC1 > EDIServices_CS => EDIServices_MC2 > > Using the examples above, I would like to translate EDIServices_MC1 and > EDIServices_MC2 to host001 and host002 respectively. > > The primary loop is: > > for server in root.findall('ns:server', namespaces): > cs = server.find('ns:cluster', namespaces) > if cs is None: > continue > # cluster_name = server.find('ns:cluster', namespaces).text > cluster_name = cs.text > listen_address = server.find('ns:listen-address', namespaces) > server_name = listen_address.text > if server_name is None: > machine = server.find('ns:machine', namespaces) > if machine is None: > continue > else: > server_name = machine.text > > print("%-15s => %s" % (cluster_name, server_name)) > > (it's taken me days to write 12 lines of code... good thing I don't do > this for a living :) ) You tend to get more efficient when you read the tutorial before you start writing code. Hard-won advice that I still not always follow myself ;) > > Rephrased, I need to find the under the child > who's name matches the name under the corresponding child. From > some of the examples on the web, I believe xpath might help but I've not > been able to get even the simple examples working. Go figure, I just > figured out what a namespace is... > > Any hints/tips/suggestions greatly appreciated especially with complete > noob tutorials for xpath. Use your favourite search engine. One advantage of XPath is that it's not limited to Python. I did not completely follow your question, so the example below is my interpretation of what you are asking for. It may still help you get started... $ cat lxml_translate_host.py from lxml import etree s = """\ text EDIServices_MS1 EDIServices_MC1 EDIServices_MS2 EDIServices_MC2 EDIServices_MC1 EDIServices_MC1 SSL host001 7001 EDIServices_MC2 EDIServices_MC2 host002 7001 """.encode() root = etree.fromstring(s) for server in root.xpath("./server"): servername = server.xpath("./name/text()")[0] print("server", servername) if not servername.isidentifier(): raise ValueError("Kind regards to Bobby Tables' Mom") machine = server.xpath("./machine/text()")[0] print("machine", machine) path = ("../machine[name='{}']/node-manager/" "listen-address/text()").format(machine) host = server.xpath(path)[0] print("host", host) print() $ python3 lxml_translate_host.py server EDIServices_MS1 machine EDIServices_MC1 host host001 server EDIServices_MS2 machine EDIServices_MC2 host host002 $ From jeanmichel at sequans.com Wed Oct 26 10:18:18 2016 From: jeanmichel at sequans.com (jmp) Date: Wed, 26 Oct 2016 16:18:18 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On 10/26/2016 02:45 PM, pozz wrote: > Il 26/10/2016 13:16, jmp ha scritto: >> [...] >> I suggest you write a GUI that make synchronous calls to a remote >> application, if possible. If the remote app is in python, you have >> access to remote protocols already written for you, Pyro is one of them, >> you can skip the low level communication part. > > I'm not sure Pyro (or similar alternatives) helps in my case. > > The real problem is that retrieving status from remote device is a slow > operation. If the GUI thread blocks waiting for the answer, the GUI > blocks and the user complains. > > From Pyro documentation: > --- > Normal method calls always block until the response is returned. This > can be any normal return value, None, or an error in the form of a > raised exception. The client code execution is suspended until the > method call has finished and produced its result. > --- > > So, even with Pyro, I need to have another thread that manages Pyro > communication (instead of serial communication)... additional problems. > Also from the Pyro doc: You can execute a remote method call and tell Pyro: ?hey, I don?t need the results right now. Go ahead and compute them, I?ll come back later once I need them?. The call will be processed in the background and you can collect the results at a later time. [...] It is possible to define one or more callables (the ?call chain?) that should be invoked automatically by Pyro as soon as the result value becomes available. jm From marko at pacujo.net Wed Oct 26 10:21:39 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 17:21:39 +0300 Subject: How to use two threads (GUI and backend) References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> Message-ID: <87y41b42do.fsf@elektro.pacujo.net> Chris Angelico : > On Thu, Oct 27, 2016 at 12:37 AM, Marko Rauhamaa wrote: >> I don't know what "Global state is shared across all threads" means >> in this context. It sounds like something that would be true for, >> say, Java and C as well. However, those languages don't promise to >> propagate improperly synchronized changes between threads. >> >> Now I would like to ask for some documentation. > > Here you have two functions and a global: > > active = True > > def func1(): > while active: > # do work > time.sleep(1) > func2() > > def func2(): > global active > if random.random() < 0.1: > active = False The thread version: ======================================================================== active = True class T1(threading.Thread): def run(self): while active: # do work time.sleep(1) class T2(threading.Thread): def run(self): global active if random.random() < 0.1: active = False t1, t2 = T1(), T2() t1.start() t2.start() t1.join() t2.join ======================================================================== Analogous code in C or Java would not be guaranteed to finish if func1() and func2() were in different execution contexts. In fact, it would be almost guaranteed to hang. That is because the compiler can see that "active" cannot change within T1.run() and would rewrite the code as: ======================================================================== class T1(threading.Thread): def run(self): if active: while True: # do work time.sleep(1) ======================================================================== Similarly, setting a flag in a signal handler might not be noticed by the main program if it were written in C. (You need to mark the variable as volatile.) > I'm sure you understand that these functions share the global state of > the 'active' flag. One changes it, the other sees the change. So far, > nothing controversial or difficult. > > It's exactly the same with threads. Extraordinary claims require extraordinary evidence. Now, Jython developers guarantee the volatility of all memory access. They even state this as normative for Python: The fundamental thing to know about Python, and what we have implemented in Jython, is that setting any attribute in Python is a volatile write; and getting any attribute is a volatile read. [...] This means that safe publication is pretty much trivial in Python, when compared to Java. Safe publication means the thread safe association of an object with a name. [...] this is always a memory-fenced operation in Python Marko From rosuav at gmail.com Wed Oct 26 10:33:34 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 01:33:34 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: <87y41b42do.fsf@elektro.pacujo.net> References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> <87y41b42do.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 27, 2016 at 1:21 AM, Marko Rauhamaa wrote: > Analogous code in C or Java would not be guaranteed to finish if func1() > and func2() were in different execution contexts. In fact, it would be > almost guaranteed to hang. > > That is because the compiler can see that "active" cannot change within > T1.run() and would rewrite the code... And since Python doesn't rewrite the code, you don't have a problem. Optimization has costs. ChrisA From lucaberto at libero.it Wed Oct 26 10:34:23 2016 From: lucaberto at libero.it (luca72) Date: Wed, 26 Oct 2016 07:34:23 -0700 (PDT) Subject: Qtimer and extra argument In-Reply-To: References: <3afc08b2-f76a-4a42-becc-e7576df3fd35@googlegroups.com> <90557b54-bf2c-4c0c-9665-ae0355a27dba@googlegroups.com> Message-ID: <4aaee1f0-4333-45f3-9ef0-f43963cc33da@googlegroups.com> thanks From marko at pacujo.net Wed Oct 26 10:42:26 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 17:42:26 +0300 Subject: How to use two threads (GUI and backend) References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> <87y41b42do.fsf@elektro.pacujo.net> Message-ID: <87twbz41f1.fsf@elektro.pacujo.net> Chris Angelico : > And since Python doesn't rewrite the code, you don't have a problem. Do you mean Python or CPython? And how do you know? Marko From rosuav at gmail.com Wed Oct 26 10:51:04 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 01:51:04 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: <87twbz41f1.fsf@elektro.pacujo.net> References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> <87y41b42do.fsf@elektro.pacujo.net> <87twbz41f1.fsf@elektro.pacujo.net> Message-ID: On Thu, Oct 27, 2016 at 1:42 AM, Marko Rauhamaa wrote: > Chris Angelico : >> And since Python doesn't rewrite the code, you don't have a problem. > > Do you mean Python or CPython? > > And how do you know? Both, and I know because Python-the-language doesn't permit those kinds of rewrites. PyPy does do a whole lot of optimization, but it very carefully ensures that semantics are retained with reference to object identities and execution order and so on. ChrisA From marko at pacujo.net Wed Oct 26 10:59:16 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Wed, 26 Oct 2016 17:59:16 +0300 Subject: How to use two threads (GUI and backend) References: <85221f44-e59f-2b12-b8a1-d527d6c95236@rece.vub.ac.be> <87eg335ksa.fsf@elektro.pacujo.net> <8760of5izj.fsf@elektro.pacujo.net> <87y41b42do.fsf@elektro.pacujo.net> <87twbz41f1.fsf@elektro.pacujo.net> Message-ID: <87pomn40mz.fsf@elektro.pacujo.net> Chris Angelico : > Python-the-language doesn't permit those kinds of rewrites. [Citation needed] Is there something here, perhaps? Marko From bc at freeuk.com Wed Oct 26 11:00:12 2016 From: bc at freeuk.com (BartC) Date: Wed, 26 Oct 2016 16:00:12 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87insf5lys.fsf@elektro.pacujo.net> References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> Message-ID: On 26/10/2016 13:33, Marko Rauhamaa wrote: > BartC : > >> On 26/10/2016 05:44, Marko Rauhamaa wrote: >> (I've implemented 'keyboards' both on-screen, and on the surface of >> digitising tablets (also with a hacked Casio calculator pcb when I >> couldn't afford a real one). With all of those I was mainly interested >> in key events, not the details.) > > Say you want to implement a simple, character-based shooting game where > the two guns are operated by the [Shift] keys. Unfortunately, the Unix > terminal API doesn't make that possible. You need to get the keyboard > events from some other API. In practice, your only choice is X11/Wayland > (on Linux). That sort of thing is possible to build by directly calling OS-specific functions in a similar manner to Steven D'Aprano's way of implementing getch(). But it's something everyone would have to code themselves. (I just tried it using my 'getchx' function where it ought to have worked. Unfortunately MS' interface to key events doesn't seem to distinguish between left and right shift keys. But it was doable with left/right ctrl keys. That's a blocking function it it means having to wait for input. But a version that just tests for status shouldn't be hard.) -- Bartc From python.list at tim.thechases.com Wed Oct 26 14:35:10 2016 From: python.list at tim.thechases.com (Tim Chase) Date: Wed, 26 Oct 2016 13:35:10 -0500 Subject: Reversing \N{...} notation? In-Reply-To: References: <20161025120612.02194da4@bigbox.christie.dr> Message-ID: <20161026133510.7c828825@bigbox.christie.dr> On 2016-10-25 20:14, Peter Otten wrote: > Tim Chase wrote: > > I like the clarity of using the "\N{...}" notation when creating > > string literals involving Unicode chars. > > > > Is there a built-in way to get such strings back from Python? > > >>> 'ma?ana'.encode("ascii", "namereplace").decode() > 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' Wonderful! Thanks for knowing about that corner of Python and sharing it. > (requires Python 3.5) Sorry it wasn't back-ported, but glad it's there now. Thanks! -tkc From pic8690 at gmail.com Wed Oct 26 16:44:56 2016 From: pic8690 at gmail.com (pic8690 at gmail.com) Date: Wed, 26 Oct 2016 13:44:56 -0700 (PDT) Subject: multiprocess passing arguments double asterisks In-Reply-To: References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> <870c7a5e-e5ee-bc25-087f-ee9cf35a39e3@gmx.com> Message-ID: <376918e0-15a9-4072-a9dd-42d49398ab95@googlegroups.com> On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote: > On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote: > > Thanks for the reply. > > > > The code snippet given by Peter is not very clear > > > > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value. This example does not return anything > > > > Would you please use my example for the map function? > > > > I appreciate your help, > > > I'm honestly not totally sure what you want to do. However, say you want > to do the following (btw this is basically what Dennis said i nhis last > email, but maybe I can help clarify): > > kwargs = {'param1': val1, 'param2': val2}) > > Then you'd like to have the following two operations performed in > separate processes: > > bar(param1=val1) > bar(param2=val2) > > In that case, I guess I would do something like the following. First > define bar_wrapper as follows (*I haven't tested any code here!): > > def bar_wrapper(pair): > key, val = pair > return bar(**{key: val}) > > Then I would probably do something like > > map(bar_wrapper, kwargs.items()) > > I.e. basically what I'm doing is taking the key-val pairs and producing > a list of them (as tuples). This is something that you can apply map > too, but not with the original function. So then the wrapper function > converts the tuple back to what you want originally. > > Hopefully I'm understanding correctly and hopefully this helps. > > Cheers, > Thomas Thomas, I have strings & numpy.ndarray as arguments. The wrapper function works great for strings. Here's what I'm trying to do... ************************** I have a serial python fn of the form def bar(**kwargs): a=kwargs.get("name") print a self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float) (****more calculations--takes a long time to compute returns an object) I am trying to run this function in parallel. Here's my Main program import numpy as np import numpy.ma as ma from delegate import parallelize from hashlib import sha1 a1d=np.zeros((32)) b1d=np.zeros((32)) p=open("/tmp/pdata","rb") pdata=np.load(p) for i in range(0,10): a1d=pdata['t1d'] b1d=pdata['gz1d'] print a1d,b1d kwargs={'name':'special','a':a1d,'b':b1d} val=parallelize(bar,kwargs) *************************************** Error: line 30, in val=parallelize(bar,kwargs) delegate.py", line 311, in process item, items = items[0], items[1:] KeyError: 0 ************************************** Error: (with the wrapper function) val=parallelize(bar_wrapper,kwargs.items()) TypeError: unhashable type: 'numpy.ndarray' *************************************** How do I pass the string & numpy.ndarray to the function bar(**kwargs)? Thank you for suggestions & help, a1d=np.zeros((32)) b1d=np.zeros((32)) p=open("/tmp/pdata","rb") pdata=np.load(p) for i in range(0,100): a1d=pdata['t1d'] b1d=pdata['gz1d'] print a1d,b1d kwargs={'name':'special','a':a1d,'b':b1d} val=parallelize(bar,kwargs) From __peter__ at web.de Wed Oct 26 16:47:29 2016 From: __peter__ at web.de (Peter Otten) Date: Wed, 26 Oct 2016 22:47:29 +0200 Subject: Reversing \N{...} notation? References: <20161025120612.02194da4@bigbox.christie.dr> <20161026133510.7c828825@bigbox.christie.dr> Message-ID: Tim Chase wrote: > On 2016-10-25 20:14, Peter Otten wrote: >> Tim Chase wrote: >> > I like the clarity of using the "\N{...}" notation when creating >> > string literals involving Unicode chars. >> > >> > Is there a built-in way to get such strings back from Python? >> >> >>> 'ma?ana'.encode("ascii", "namereplace").decode() >> 'ma\\N{LATIN SMALL LETTER N WITH TILDE}ana' > > Wonderful! Thanks for knowing about that corner of Python and > sharing it. > >> (requires Python 3.5) > > Sorry it wasn't back-ported, but glad it's there now. Thanks! My original plan was to write a custom error handler, and I found namereplace when I tried to understand the required function signature. Completing the initial effort: $ cat codecs_mynamereplace.py # -*- coding: utf-8 -*- import codecs import unicodedata try: codecs.namereplace_errors except AttributeError: print("using mynamereplace") def mynamereplace(exc): return u"".join( "\\N{%s}" % unicodedata.name(c) for c in exc.object[exc.start:exc.end] ), exc.end codecs.register_error("namereplace", mynamereplace) print(u"ma?ana".encode("ascii", "namereplace").decode()) $ python3.5 codecs_mynamereplace.py ma\N{LATIN SMALL LETTER N WITH TILDE}ana $ python3.4 codecs_mynamereplace.py using mynamereplace ma\N{LATIN SMALL LETTER N WITH TILDE}ana $ python2.7 codecs_mynamereplace.py using mynamereplace ma\N{LATIN SMALL LETTER N WITH TILDE}ana From jcasale at activenetwerx.com Wed Oct 26 17:53:47 2016 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Wed, 26 Oct 2016 21:53:47 +0000 Subject: thread local storage Message-ID: <1772b0eb0f814ea5b6ded930e907c645@activenetwerx.com> Looks like the shipped implementation doesn't give access to all the discrete copies of tls for us in a case where a context manager needs to perform any cleanup. Does something exists where I can leverage this feature or do I need to roll my own? Thanks, jlc From python at mrabarnett.plus.com Wed Oct 26 18:27:54 2016 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 26 Oct 2016 23:27:54 +0100 Subject: multiprocess passing arguments double asterisks In-Reply-To: <376918e0-15a9-4072-a9dd-42d49398ab95@googlegroups.com> References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> <870c7a5e-e5ee-bc25-087f-ee9cf35a39e3@gmx.com> <376918e0-15a9-4072-a9dd-42d49398ab95@googlegroups.com> Message-ID: <09738f36-9291-ea51-c842-ab23535128ba@mrabarnett.plus.com> On 2016-10-26 21:44, pic8690 at gmail.com wrote: > On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote: >> On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote: >> > Thanks for the reply. >> > >> > The code snippet given by Peter is not very clear >> > >> > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value. This example does not return anything >> > >> > Would you please use my example for the map function? >> > >> > I appreciate your help, >> > >> I'm honestly not totally sure what you want to do. However, say you want >> to do the following (btw this is basically what Dennis said i nhis last >> email, but maybe I can help clarify): >> >> kwargs = {'param1': val1, 'param2': val2}) >> >> Then you'd like to have the following two operations performed in >> separate processes: >> >> bar(param1=val1) >> bar(param2=val2) >> >> In that case, I guess I would do something like the following. First >> define bar_wrapper as follows (*I haven't tested any code here!): >> >> def bar_wrapper(pair): >> key, val = pair >> return bar(**{key: val}) >> >> Then I would probably do something like >> >> map(bar_wrapper, kwargs.items()) >> >> I.e. basically what I'm doing is taking the key-val pairs and producing >> a list of them (as tuples). This is something that you can apply map >> too, but not with the original function. So then the wrapper function >> converts the tuple back to what you want originally. >> >> Hopefully I'm understanding correctly and hopefully this helps. >> >> Cheers, >> Thomas > > Thomas, > > I have strings & numpy.ndarray as arguments. The wrapper function works great for strings. > > Here's what I'm trying to do... > ************************** > > I have a serial python fn of the form > > def bar(**kwargs): > a=kwargs.get("name") > print a > self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float) > (****more calculations--takes a long time to compute > returns an object) > > I am trying to run this function in parallel. > > Here's my Main program > > import numpy as np > import numpy.ma as ma > from delegate import parallelize > from hashlib import sha1 > a1d=np.zeros((32)) > b1d=np.zeros((32)) > p=open("/tmp/pdata","rb") > pdata=np.load(p) > for i in range(0,10): > a1d=pdata['t1d'] > b1d=pdata['gz1d'] > print a1d,b1d > kwargs={'name':'special','a':a1d,'b':b1d} > val=parallelize(bar,kwargs) > *************************************** > Error: > line 30, in > val=parallelize(bar,kwargs) > delegate.py", line 311, in process > item, items = items[0], items[1:] > KeyError: 0 > ************************************** > Error: (with the wrapper function) > val=parallelize(bar_wrapper,kwargs.items()) > TypeError: unhashable type: 'numpy.ndarray' > *************************************** > > How do I pass the string & numpy.ndarray to the function bar(**kwargs)? > > Thank you for suggestions & help, > > > a1d=np.zeros((32)) > b1d=np.zeros((32)) > p=open("/tmp/pdata","rb") > pdata=np.load(p) > for i in range(0,100): > a1d=pdata['t1d'] > b1d=pdata['gz1d'] > print a1d,b1d > kwargs={'name':'special','a':a1d,'b':b1d} > val=parallelize(bar,kwargs) > 'parallelize' expects a function and a list of items. It calls the function with each item in the list in parallel, passing the item as the argument of the function. That _single_ argument item can be a tuple or a dict. From tjreedy at udel.edu Wed Oct 26 19:30:18 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 26 Oct 2016 19:30:18 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/26/2016 7:18 AM, BartC wrote: > Can tkinter do it without creating a distracting pop-up window at the > same time? Yes. I already showed how on this thread. Of course, for some text appications, one would be better off with a Text widget than with the system-specific console. Bart, you appear to have been fortunate enough to be spoiled by learning programming on microcomputers, where the terminal and computer are combined into one unit, so that the computer, and potentially the programmer, have access to user input actions. However, Python was not developed on, and in not limited to use on, such machines. Today, ethernet-connected *nix servers have no keyboard, mouse, or even a directly connected terminal. When Python was developed, standard C did not have keyboard and mouse functions. (I don't know about the most recent standards.) Keyboard functions on microcomputer C ports were non-standard OS-specific extensions. On Unix, X windows was and is optional. Over 20 years ago, tk was written in tcl and C to give tcl programmers access to X Windows graphics terminals, including user input actions. It has since been ported to MS Windows and Apple OSX and adopted by other languages, including Python, to give the same access. -- Terry Jan Reedy From tjreedy at udel.edu Wed Oct 26 20:33:44 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 26 Oct 2016 20:33:44 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87insf5lys.fsf@elektro.pacujo.net> References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> Message-ID: On 10/26/2016 8:33 AM, Marko Rauhamaa wrote: > BartC : > Say you want to implement a simple, character-based shooting game where > the two guns are operated by the [Shift] keys. Unfortunately, the Unix > terminal API doesn't make that possible. You need to get the keyboard > events from some other API. In practice, your only choice is X11/Wayland > (on Linux). Or tk(inter), which uses X11 on *nix and other stuff on OSX and Windows. I verified that it gives access to the Shift keys as keys in themselves, rather than just as modifiers. import tkinter as tk root = tk.Tk() def lshift(event): print('shift-l') def rshift(event): print('shift-r') root.bind('', lshift) root.bind('', rshift) root.mainloop() does what I hoped, with autorepeat when the key was held down. >> It's more building a mountain of complexity around something that >> ought to be straightforward. > > Maybe there should be some way to get the raw events from the PTY. PTY? Must be Linux-specific. Most beginners are not on Linux. > However, next you'd start wanting the mouse events and pixel-level color > controls. It starts to look like a GUI application. > > But what would be wrong in a GUI PTY API? No windowing, 'No windowing'? Python normally runs in a text widget in a window, programmed to emulate a dumb terminal. The request for non-blocking access to user actions is a request for something smarter. > just a regular > character display where you could draw pictures and interpret the inputs > directly ? la Apple II or Commodore 64. I never touched an Apple II and only briefly a Commodore 64, so either never knew or have forgotten the Basic commands they had. But it should be possible to emulate at least the text screen of either with tkinter. (Graphics modes with pixel peek and poke might also be possible with a Canvas, but I won't claim that without knowing more.) It should then be possible to translate old games into Python. Maybe something like this has been done? > It would make teaching programming much more fun, too. The logo-in-Python turtle module is already used for this. What would be doable and possibly useful would be a textscreen module with a Screen class that does as much boilerplate for people as possible. For my proof-of-concept above, something like from textscreen import Screen def lshift(event): print('shift-l') def rshift(event): print('shift-r') Screen().go should be enough. The go method should import tkinter, create root, add a Text, scan globals() for functions with key names (and names of mouse actions), bind any that are found, and start mainloop. This sort of thing would be even nicer if and when tcl/tk gains support for the full unicode character set so that all the non-BMP emoji are potentially available. (A font supporting such would also be needed.) -- Terry Jan Reedy From bc at freeuk.com Wed Oct 26 21:12:52 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 02:12:52 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 00:30, Terry Reedy wrote: > On 10/26/2016 7:18 AM, BartC wrote: > >> Can tkinter do it without creating a distracting pop-up window at the >> same time? > > Yes. I already showed how on this thread. Of course, for some text > appications, one would be better off with a Text widget than with the > system-specific console. > > Bart, you appear to have been fortunate enough to be spoiled by learning > programming on microcomputers, where the terminal and computer are > combined into one unit, so that the computer, and potentially the > programmer, have access to user input actions. Actually, I first used machines such as pdp10 and pdp11. Those mostly used serial terminals, a mystery to me, and they still are. It seems Unix is keen to propagate the mystery. However, Python was not > developed on, and in not limited to use on, such machines. Today, > ethernet-connected *nix servers have no keyboard, mouse, or even a > directly connected terminal. So how does your tkinter example work in such a server? As I don't understand the argument that a language shouldn't have a basic keyboard API because some computers it could run on might not have a keyboards. Anyway vast numbers of /consumer/ machines /do/ have displays and keyboards. Look at any computer in an office: it has a display and keyboard. Look at any laptop: display and keyboard. Even a tablet can have an on-screen keyboard or have a real one plugged in. > Over 20 years ago, tk was written in tcl and C to give tcl programmers > access to X Windows graphics terminals, including user input actions. It > has since been ported to MS Windows and Apple OSX and adopted by other > languages, including Python, to give the same access. OK. I was writing graphics applications before a lot of those things existed. They needed to use a display and they needed user input, and I had to make it work. I couldn't ask my customers to wait a decade or so until someone invented an API before they could fully use their keyboards! (I've looked at X Windows; bloody hell, it makes Win32/GDI look like child's play. It's quite frustrating see things that used to be so simple to do become next to impossible.) -- Bartc From tjreedy at udel.edu Wed Oct 26 22:15:24 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 26 Oct 2016 22:15:24 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> Message-ID: On 10/26/2016 11:00 AM, BartC wrote: > On 26/10/2016 13:33, Marko Rauhamaa wrote: >> Say you want to implement a simple, character-based shooting game where >> the two guns are operated by the [Shift] keys. Unfortunately, the Unix >> terminal API doesn't make that possible. You need to get the keyboard >> events from some other API. In practice, your only choice is X11/Wayland >> (on Linux). This is trivial with tkinter and practical if one uses a tk Text. See below for the problem with using tkinter and console. > That sort of thing is possible to build by directly calling OS-specific > functions in a similar manner to Steven D'Aprano's way of implementing > getch(). > > But it's something everyone would have to code themselves. > > (I just tried it using my 'getchx' function where it ought to have > worked. Unfortunately MS' interface to key events doesn't seem to > distinguish between left and right shift keys. But it was doable with > left/right ctrl keys. > > That's a blocking function it it means having to wait for input. But a > version that just tests for status shouldn't be hard.) In my answer to Marko, I posted code which worked as far as I tested it. Here I add a line to make the tk window invisible. I also replace rshift with a function that gets input from the user. import tkinter as tk root = tk.Tk() root.withdraw() # make tk window invisible def lshift(event): print('shift-l') def rshift(event): s = input('type something: ') print('received', s) root.bind('', lshift) root.bind('', rshift) root.mainloop() For input, the problem is input focus. When the tk window is created, the OS gives it input focus. Leaving itself invisible does not negate that. But to respond to input in a different window, the user must, as least on Windows, click on the input window. I do not know of any way for tkinter to give focus back to a parent window that is either not a tk window or is not in the same process. After input is received, or indeed after focus is moved by clicking on any other window, there is the problem of moving focus back to the tk window. If it is invisible, it cannot be clicked on. And without a binding to stop mainloop when the focus leaves, there is no way to stop it without killing the console, or with IDLE, restarting Shell. I conclude that if one uses tkinter to captures and process some keyboard events, one should do so for all. Python's input should be replaced by a tkinter simulation. I can think of a couple of ways this might be implemented. -- Terry Jan Reedy From eryksun at gmail.com Wed Oct 26 23:45:50 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 27 Oct 2016 03:45:50 +0000 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> Message-ID: On Wed, Oct 26, 2016 at 11:39 AM, Dennis Lee Bieber wrote: > Curses .... tends to not be available on Windows as M$ hasn't implemented a > compatible console driver. The PDCurses library supports the Windows console. Christoph Gohlke distributes a curses extension module based on it: http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses From no.email at nospam.invalid Thu Oct 27 00:08:43 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Wed, 26 Oct 2016 21:08:43 -0700 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87h97ymo1g.fsf@nightsong.com> Terry Reedy writes: > Today, ethernet-connected *nix servers have no > keyboard, mouse, or even a directly connected terminal. Usually you ssh into them and connect to a pty which supports the same ioctls that a real terminal would. I use screen editors over ssh all the time, not to mention filters like "more" where you press the space bar to scroll to the next page. It's sad that there's no easy way to do that in Python. From grant.b.edwards at gmail.com Thu Oct 27 00:26:42 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 27 Oct 2016 04:26:42 +0000 (UTC) Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87h97ymo1g.fsf@nightsong.com> Message-ID: On 2016-10-27, Paul Rubin wrote: > Terry Reedy writes: > >> Today, ethernet-connected *nix servers have no keyboard, mouse, or >> even a directly connected terminal. > > Usually you ssh into them and connect to a pty which supports the > same ioctls that a real terminal would. On all the Unixes/Linuxs I know of, a pty supports a _subset_ of those that a real tty supports. The fact that some of them are missing has annoyed me for decades because it prevents you from implementing a serial port in user-space[1]. I've offered a few times to extend the Linux pty driver to support the same set of ioctl calls that a tty does (so that it could be used in place of a tty generally), but I've never gotten any response. > I use screen editors over ssh all the time, not to mention filters > like "more" where you press the space bar to scroll to the next > page. It's sad that there's no easy way to do that in Python. Yep, among the ioctl calls that ptys do support are those that allow you to do raw and non-blocking input. [1] The fact that Windows allows you to implement a serial port in userspace and Linux doesn't just adds insult to injury. -- Grant From marko at pacujo.net Thu Oct 27 01:49:31 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 27 Oct 2016 08:49:31 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> Message-ID: <8737jixrx0.fsf@elektro.pacujo.net> Terry Reedy : > On 10/26/2016 8:33 AM, Marko Rauhamaa wrote: >> Maybe there should be some way to get the raw events from the PTY. > > PTY? Must be Linux-specific. Most beginners are not on Linux. A PTY is an emulated console (). I don't know Windows but I would guess cmd.exe does something similar there. Also, I have no statistics on most beginning programmers operating systems. >> But what would be wrong in a GUI PTY API? No windowing, > > 'No windowing'? Python normally runs in a text widget in a window, > programmed to emulate a dumb terminal. Must be Windows-specific. > I never touched an Apple II and only briefly a Commodore 64, so either > never knew or have forgotten the Basic commands they had. But it should > be possible to emulate at least the text screen of either with > tkinter. Everything is possible for an individual application (like Python). The TTY/PTY interface enhancement I was entertaining would be a new API provided by the OS to all programs. Marko From marko at pacujo.net Thu Oct 27 01:56:17 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 27 Oct 2016 08:56:17 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87h97ymo1g.fsf@nightsong.com> Message-ID: <87y41awd1a.fsf@elektro.pacujo.net> Grant Edwards : > I've offered a few times to extend the Linux pty driver to support the > same set of ioctl calls that a tty does (so that it could be used in > place of a tty generally), but I've never gotten any response. Ah, Linux kernel politics are Byzantine. It's virtually impossible to get a hearing at the linux-kernel main mailing list. Did you try one of the targeted mailing lists on ? Marko From steve+comp.lang.python at pearwood.info Thu Oct 27 02:51:57 2016 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 27 Oct 2016 17:51:57 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> On Thursday 27 October 2016 12:12, BartC wrote: > I don't > understand the argument that a language shouldn't have a basic keyboard > API because some computers it could run on might not have a keyboards. That's not the argument. The argument is that Python has a basic keyboard API: raw_input (in Python 2) or input (in 3). This supports 97% of keyboard-based interaction, namely blocking line-based text input. Ctrl-C (KeyboardInterrupt) can be co-opted to support maybe another one or two percent. The question is, what are the use-cases for the sorts of key APIs you are asking for? Who needs them? Why should it be a language feature? Those are not rhetoricial questions. Python falls neatly into the same niche of languages as (for example) Ruby, Javascript, Lua, Swift, Tcl, and (not quite as well) bash, Java, Julia. Which of these languages offer non-blocking keyboard input as a standard part of the language? Python has no "peek" and "poke" memory access commands either. It's not 1972 and programming has moved on. The utility of something like this feature is very low, the amount of effort needed for a cross-platform solution is very high. -- Steven git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space. From petef4+usenet at gmail.com Thu Oct 27 02:56:44 2016 From: petef4+usenet at gmail.com (Pete Forman) Date: Thu, 27 Oct 2016 07:56:44 +0100 Subject: lxml and xpath(?) References: <9ceddc7b-a3d0-479a-a009-29c4bdc15e72@googlegroups.com> Message-ID: Peter Otten <__peter__ at web.de> writes: > root = etree.fromstring(s) > for server in root.xpath("./server"): > servername = server.xpath("./name/text()")[0] When working with lxml I prefer to use this Python idiom. servername, = server.xpath("./name/text()") That enforces a single result. The original code will detect a lack of results but if the query returns multiple results when only one is expected then it silently returns the first. -- Pete Forman From tjreedy at udel.edu Thu Oct 27 03:05:58 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 27 Oct 2016 03:05:58 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <8737jixrx0.fsf@elektro.pacujo.net> References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> <8737jixrx0.fsf@elektro.pacujo.net> Message-ID: On 10/27/2016 1:49 AM, Marko Rauhamaa wrote: > Terry Reedy : > >> On 10/26/2016 8:33 AM, Marko Rauhamaa wrote: >>> Maybe there should be some way to get the raw events from the PTY. >> >> PTY? Must be Linux-specific. Most beginners are not on Linux. > > A PTY is an emulated console ( https://en.wikipedia.org/wiki/Pseudoterminal>). I don't know Windows but > I would guess cmd.exe does something similar there. > > Also, I have no statistics on most beginning programmers operating > systems. > >>> But what would be wrong in a GUI PTY API? No windowing, >> >> 'No windowing'? Python normally runs in a text widget in a window, >> programmed to emulate a dumb terminal. > > Must be Windows-specific. Not as I meant the above. When I used unix in the 1980s, the full screen ran csh until one started another full screen application. MSDOS was the same. Every contemporary photo of modern Linux or Mac I have seen has a desktop with windows just like Windows. Do people on Linux still commonly use full-screen, no window text editors like the one I had? On Windows, there are full screen games, but I have never seen a full-screen, no-window text application. Since modern screen are pixel graphics screens, rather than character screens, there must be a widget, whether standard with the OS or custom to the console, that emulates the old fixed-pitch character screens. At least on Windows, C Programs that run with the console still get characters entered by users and send characters to be displayed. -- Terry Jan Reedy From __peter__ at web.de Thu Oct 27 03:18:04 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 27 Oct 2016 09:18:04 +0200 Subject: lxml and xpath(?) References: <9ceddc7b-a3d0-479a-a009-29c4bdc15e72@googlegroups.com> Message-ID: Pete Forman wrote: > Peter Otten <__peter__ at web.de> writes: > >> root = etree.fromstring(s) >> for server in root.xpath("./server"): >> servername = server.xpath("./name/text()")[0] > > When working with lxml I prefer to use this Python idiom. > > servername, = server.xpath("./name/text()") > > That enforces a single result. The original code will detect a lack of > results but if the query returns multiple results when only one is > expected then it silently returns the first. Good suggestion, and for those who find the trailing comma easy to overlook: [servername] = server.xpath("./name/text()") From darcy at vex.net Thu Oct 27 03:56:14 2016 From: darcy at vex.net (D'Arcy Cain) Date: Thu, 27 Oct 2016 03:56:14 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> <8737jixrx0.fsf@elektro.pacujo.net> Message-ID: <3752cd9e-d1f2-6f45-d022-f8adec02545e@vex.net> On 2016-10-27 03:05 AM, Terry Reedy wrote: > When I used unix in the 1980s, the full screen ran csh until one started > another full screen application. MSDOS was the same. Every contemporary > photo of modern Linux or Mac I have seen has a desktop with windows just > like Windows. Do people on Linux still commonly use full-screen, no > window text editors like the one I had? On Windows, there are full > screen games, but I have never seen a full-screen, no-window text > application. You must lead a sheltered life then. > Since modern screen are pixel graphics screens, rather than character Not always. I have many Unix systems that don't run X. A GUI is not essential to running a useful Unix server. You may be correct that Linux systems tend to run GUIs but even then it isn't mandatory. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From storchaka at gmail.com Thu Oct 27 04:36:03 2016 From: storchaka at gmail.com (Serhiy Storchaka) Date: Thu, 27 Oct 2016 11:36:03 +0300 Subject: Reversing \N{...} notation? In-Reply-To: References: <20161025120612.02194da4@bigbox.christie.dr> <20161026133510.7c828825@bigbox.christie.dr> Message-ID: On 26.10.16 23:47, Peter Otten wrote: > def mynamereplace(exc): > return u"".join( > "\\N{%s}" % unicodedata.name(c) > for c in exc.object[exc.start:exc.end] > ), exc.end > codecs.register_error("namereplace", mynamereplace) Not all characters has standard name. This fails with say '\x80'. Use backslash escaping as a fallback: try: ascii except NameError: ascii = repr def mynamereplace(exc): repl = [] for c in exc.object[exc.start:exc.end]: try: repl.append("\\N{%s}" % unicodedata.name(c)) except ValueError: repl.append(ascii(c)[1:-1]) return u"".join(repl), exc.end From vek.m1234 at gmail.com Thu Oct 27 05:05:00 2016 From: vek.m1234 at gmail.com (Veek M) Date: Thu, 27 Oct 2016 14:35 +0530 Subject: Iteration, while loop, and for loop References: <079ff112-8f18-4a27-938f-12e4e55a4a5b@googlegroups.com> Message-ID: Elizabeth Weiss wrote: > words=["hello", "world", "spam", "eggs"] > counter=0 > max_index=len(words)-1 > > while counter<=max_index: > word=words[counter] > print(word + "!") > counter=counter + 1 while 0 < 10: get 0'th element do something with element increment 0 to 1 (repeat) words[0] gets the 0'th word (arrays/lists start at 0, not 1) -------------------- That example of his is badly presented.. 1. use i, x, y, z, count in that order for integers 2. do your dirty work in a function 3. keep loops clean and tidy 4. keep data far away from code 5. avoid " when ' does the trick 6. try to create black box functions - that work no matter what you throw at them.. without sacrificing readability - programming is about aesthetics and readability - quite artsy.. (I'm not an expert so.. some of this might be outright wrong) words=['hello', 'world', 'spam', 'eggs'] def display(txt, decoration='!'): message = str(txt) + str(decoration) print(message) i = 0 i_max = len(words) - 1 while i <= i_max: word = words[i] i += 1 display(word) languages that don't have 'for' like C, will use 'while' - in python 'for' is preferred - faster, readable - especially for the example you cited. From mantaselprj at gmail.com Thu Oct 27 05:34:20 2016 From: mantaselprj at gmail.com (mantaselprj at gmail.com) Date: Thu, 27 Oct 2016 02:34:20 -0700 (PDT) Subject: AsyncSSH and timeout Message-ID: <037f135b-9771-4302-adbd-6db4c3b80eae@googlegroups.com> Good day, I'm trying to run multiple SSH clients using AsyncSSH lib. This is an example from manual, which works fine: http://pastebin.com/zh4zymeQ The problem is, it is not possible to directly set connect timeout in run_client() function. However, the manual says: "asyncio calls can be wrapped in a call to asyncio.wait_for() or asyncio.wait() which takes a timeout and provides equivalent functionality." So I tried to modify run_client() like this: http://pastebin.com/mH7pBuuf But it spits out a syntax error. I also tried countless variations of it, but still no luck. Could someone more knowledgeable help me to set the damn timeout? From mantaselprj at gmail.com Thu Oct 27 05:37:19 2016 From: mantaselprj at gmail.com (Standard User) Date: Thu, 27 Oct 2016 02:37:19 -0700 (PDT) Subject: AsyncSSH and timeout Message-ID: <8093c81b-52ab-4b49-aa85-d38d2f1249cb@googlegroups.com> Good day, I'm trying to run multiple SSH clients using AsyncSSH lib. This is an example from manual, which works fine: http://pastebin.com/zh4zymeQ The problem is, it is not possible to directly set connect timeout in run_client() function. However, the manual says: "asyncio calls can be wrapped in a call to asyncio.wait_for() or asyncio.wait() which takes a timeout and provides equivalent functionality." So I tried to modify run_client() like this: http://pastebin.com/mH7pBuuf But it spits out a syntax error. I also tried countless variations of it, but still no luck. Could someone more knowledgeable help me to set the damn timeout? From tjreedy at udel.edu Thu Oct 27 06:07:04 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 27 Oct 2016 06:07:04 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/26/2016 9:12 PM, BartC wrote: > On 27/10/2016 00:30, Terry Reedy wrote: >> Bart, you appear to have been fortunate enough to be spoiled by learning >> programming on microcomputers, where the terminal and computer are >> combined into one unit, so that the computer, and potentially the >> programmer, have access to user input actions. > > Actually, I first used machines such as pdp10 and pdp11. Those mostly > used serial terminals, a mystery to me, and they still are. It seems > Unix is keen to propagate the mystery. OK, you got spoiled later ;-). I share your frustration a bit. >> However, Python was not >> developed on, and in not limited to use on, such machines. Today, >> ethernet-connected *nix servers have no keyboard, mouse, or even a >> directly connected terminal. > > So how does your tkinter example work in such a server? Without X-windows available, there would be no point, and it will not work. I presume including the X window subsystem on a linux (server) build is optional. Last I knew, all of the linux machines with a CPython buildbot either do not have X or prohibit the buildbot from using it. Compiling _tkinter.c is optional and many (most?) Linux distributions put tkinter.py, idlelib/*.py, and turtle.py in a separate package. If a buildbot tries to run gui tests on linux machines without X available, an exception is raised, something like 'Failed to connect to the X subsystem'. (It has been 3 years since I cause one of those.) If the server includes X and tkinter and one connects (likely as admin) with a X-terminal or emulator, I expect that tk and tkinter should work. Whether X will let an invisible window keep keyboard focus, I will not know until someone tries it. Perhaps 99+% of Tcl/tk works the same across platforms. > As I don't > understand the argument that a language shouldn't have a basic keyboard > API because some computers it could run on might not have a keyboards. As I and others have said, those keyboard functions are not available on text terminals. I predict that keyboard functions that so not work on all systems will never become built-ins. But some are available with an import. Python optionally comes with a sophisticated keyboard api. The PSF (python.org) CPython builds for Windows and Mac include that API. On Windows, so is the required tcl/tk build. The premise of the subject line, that Python does not include 'non-blocking keyboard input functions', is not true. Some things *can* be simplified. I gave one example previously. While writing this, I realized that with a little work, I could automate all the bindings for IDLE menu item event handlers. > (I've looked at X Windows; bloody hell, it makes Win32/GDI look like > child's play.) This may have helped persuade three different language groups to piggyback on the work of tcl folk. Keeping up with Apple's series of graphics systems has also been a hassle. -- Terry Jan Reedy From pozzugno at gmail.com Thu Oct 27 06:22:08 2016 From: pozzugno at gmail.com (pozz) Date: Thu, 27 Oct 2016 12:22:08 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: Il 26/10/2016 16:18, jmp ha scritto: > On 10/26/2016 02:45 PM, pozz wrote: >> Il 26/10/2016 13:16, jmp ha scritto: >>> [...] >>> I suggest you write a GUI that make synchronous calls to a remote >>> application, if possible. If the remote app is in python, you have >>> access to remote protocols already written for you, Pyro is one of them, >>> you can skip the low level communication part. >> >> I'm not sure Pyro (or similar alternatives) helps in my case. >> >> The real problem is that retrieving status from remote device is a slow >> operation. If the GUI thread blocks waiting for the answer, the GUI >> blocks and the user complains. >> >> From Pyro documentation: >> --- >> Normal method calls always block until the response is returned. This >> can be any normal return value, None, or an error in the form of a >> raised exception. The client code execution is suspended until the >> method call has finished and produced its result. >> --- >> >> So, even with Pyro, I need to have another thread that manages Pyro >> communication (instead of serial communication)... additional problems. >> > > Also from the Pyro doc: > > You can execute a remote method call and tell Pyro: ?hey, I don?t need > the results right now. Go ahead and compute them, I?ll come back later > once I need them?. The call will be processed in the background and you > can collect the results at a later time. > > [...] > > It is possible to define one or more callables (the ?call chain?) that > should be invoked automatically by Pyro as soon as the result value > becomes available. I already read that, it is the feature "Asynchronous ('future') remote calls & call chains". This approach can be taken also without pyro at all, just using pyserial module (and I think all the communication libraries). With pyserial, I can set a read timeout value of zero: timeout = 0: non-blocking mode, return immediately in any case, returning zero or more, up to the requested number of bytes In this way, I can implement exactly the same mechanism of pyro in asyncronous mode. With pyserial I could avoid setting timeout=0, using in_waiting property ("number of bytes in the input buffer"). Anyway I don't like this approach, because the main (and single) thread should check in_waiting every X milliseconds. If X is too high, I could wait for the answer even if it is already ready in the input buffer. If X is too low, the application consumes a lot of clocks to check in_waiting. I would prefer to have a callback automatically called when the read operation is complete. And I think the only method is using another (blocking) thread. The blocking function read returns *immediately* when all the bytes are received. And I think during blocking time, the thread isn't consuming CPU clocks. I could try with asyncio feature of pyserial, but it is classified as "experimental". From marko at pacujo.net Thu Oct 27 06:42:25 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 27 Oct 2016 13:42:25 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> <8737jixrx0.fsf@elektro.pacujo.net> Message-ID: <87h97y3wfi.fsf@elektro.pacujo.net> Terry Reedy : > Do people on Linux still commonly use full-screen, no window text > editors like the one I had? I occasionally switch on one of the alternate VTs, which are not running any GUI. However, I constantly use -- in fact, as I type, I'm using -- a program running in a PTY environment. IOW, the program "thinks" it's running on a dedicated text-only terminal. The news program (GNUS/emacs in my case) is not conscious of pixels, keycodes or the mouse. I must use keyboard commands to navigate. > On Windows, there are full screen games, but I have never seen a > full-screen, no-window text application. As I'm typing, I have two other, unrelated windows on the screen. I'm using LXDE, which is a classical GUI environment. It is my understanding that both Windows and Gnome are moving to a de-facto full-screen GUI. The GUI strongly prefers you opening windows in full-screen mode. Marko From bc at freeuk.com Thu Oct 27 06:42:41 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 11:42:41 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 07:51, Steven D'Aprano wrote: > On Thursday 27 October 2016 12:12, BartC wrote: > >> I don't >> understand the argument that a language shouldn't have a basic keyboard >> API because some computers it could run on might not have a keyboards. > > That's not the argument. The argument is that Python has a basic keyboard API: > raw_input (in Python 2) or input (in 3). > > This supports 97% of keyboard-based interaction, namely blocking line-based > text input. Ctrl-C (KeyboardInterrupt) can be co-opted to support maybe another > one or two percent. > > The question is, what are the use-cases for the sorts of key APIs you are > asking for? Who needs them? Why should it be a language feature? If a language (or more likely an OS such as Unix) doesn't natively provide a certain feature, then it's not surprising that 97% of applications don't use it! "There's a room in your house with no door to it; how do I get in?" "There's no need for a door because no one ever uses that room! But you can get in through the chimney - if you /have/ to." > Those are not rhetoricial questions. > > Python falls neatly into the same niche of languages as (for example) Ruby, > Javascript, Lua, Swift, Tcl, and (not quite as well) bash, Java, Julia. Which > of these languages offer non-blocking keyboard input as a standard part of the > language? That doesn't surprise me either. Even implementation languages such as C tend to shy away from the issue. And since C was intimately associated with Unix, it starts to get even less surprising! But I seem to remember in a previous thread that Python had some problem even with line-buffered input. Something to do with only getting a line of input as a string then needed to do some processing to read individual elements, IIRC. (FWIW my own language does strive to have this basic stuff built it. But getting full keyboard and console handling working as I want it across both Windows and Linux is challenging. It's a bit easier on Windows as, even though you're using a console, you have the full resources of the Win32 API to draw on. On Linux you can't assume any such resources except some apparently 1970s-style terminal handling, from what I can figure out.) > Python has no "peek" and "poke" memory access commands either. It's not 1972 > and programming has moved on. The utility of something like this feature is > very low, the amount of effort needed for a cross-platform solution is very > high. My language: function peek(p,t=byte)= return makeref(p,t)^ end ... print peek(0x400'000) It 'works' on both Windows and Linux, although you need to know what addresses to safely peek. (I didn't try poke but that's also easy, if more dangerous. But usually I use such pointer accesses for known data structures and memory blocks.) On a machine with a certain memory-mapped device, then this would be ideal. -- Bartc From demosthenesk at gmail.com Thu Oct 27 06:44:29 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Thu, 27 Oct 2016 13:44:29 +0300 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: <698f0b9f-f68e-f3c3-10a0-acd4ad4a5e25@gmail.com> Here is an example about threads and PyQT https://www.youtube.com/watch?v=ivcxZSHL7jM&index=2 On 10/27/2016 01:22 PM, pozz wrote: > Il 26/10/2016 16:18, jmp ha scritto: >> On 10/26/2016 02:45 PM, pozz wrote: >>> Il 26/10/2016 13:16, jmp ha scritto: >>>> [...] >>>> I suggest you write a GUI that make synchronous calls to a remote >>>> application, if possible. If the remote app is in python, you have >>>> access to remote protocols already written for you, Pyro is one of >>>> them, >>>> you can skip the low level communication part. >>> >>> I'm not sure Pyro (or similar alternatives) helps in my case. >>> >>> The real problem is that retrieving status from remote device is a slow >>> operation. If the GUI thread blocks waiting for the answer, the GUI >>> blocks and the user complains. >>> >>> From Pyro documentation: >>> --- >>> Normal method calls always block until the response is returned. This >>> can be any normal return value, None, or an error in the form of a >>> raised exception. The client code execution is suspended until the >>> method call has finished and produced its result. >>> --- >>> >>> So, even with Pyro, I need to have another thread that manages Pyro >>> communication (instead of serial communication)... additional problems. >>> >> >> Also from the Pyro doc: >> >> You can execute a remote method call and tell Pyro: ?hey, I don?t need >> the results right now. Go ahead and compute them, I?ll come back later >> once I need them?. The call will be processed in the background and you >> can collect the results at a later time. >> >> [...] >> >> It is possible to define one or more callables (the ?call chain?) that >> should be invoked automatically by Pyro as soon as the result value >> becomes available. > > I already read that, it is the feature "Asynchronous ('future') remote > calls & call chains". > > This approach can be taken also without pyro at all, just using > pyserial module (and I think all the communication libraries). > With pyserial, I can set a read timeout value of zero: > > timeout = 0: non-blocking mode, return immediately in > any case, returning zero or more, up to the requested > number of bytes > > In this way, I can implement exactly the same mechanism of pyro in > asyncronous mode. With pyserial I could avoid setting timeout=0, > using in_waiting property ("number of bytes in the input buffer"). > > Anyway I don't like this approach, because the main (and single) > thread should check in_waiting every X milliseconds. > If X is too high, I could wait for the answer even if it is already > ready in the input buffer. > If X is too low, the application consumes a lot of clocks to check > in_waiting. > > I would prefer to have a callback automatically called when the read > operation is complete. And I think the only method is using another > (blocking) thread. The blocking function read returns *immediately* > when all the bytes are received. And I think during blocking time, > the thread isn't consuming CPU clocks. > > I could try with asyncio feature of pyserial, but it is classified as > "experimental". > > From marko at pacujo.net Thu Oct 27 06:50:14 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 27 Oct 2016 13:50:14 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87bmy63w2h.fsf@elektro.pacujo.net> BartC : > "There's a room in your house with no door to it; how do I get in?" > > "There's no need for a door because no one ever uses that room! But > you can get in through the chimney - if you /have/ to." +1 > On Linux you can't assume any such resources except some apparently > 1970s-style terminal handling, from what I can figure out.) Correct. > My language: > > function peek(p,t=byte)= > return makeref(p,t)^ > end > ... > print peek(0x400'000) > > It 'works' on both Windows and Linux So how do you draw a big, yellow happy face in the top right corner using your language? Marko From bc at freeuk.com Thu Oct 27 07:15:42 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 12:15:42 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 11:07, Terry Reedy wrote: > On 10/26/2016 9:12 PM, BartC wrote: >> On 27/10/2016 00:30, Terry Reedy wrote: >> So how does your tkinter example work in such a server? > > Without X-windows available, there would be no point, and it will not > work. I presume including the X window subsystem on a linux (server) > build is optional. > Compiling _tkinter.c is optional and many (most?) Linux distributions > put tkinter.py, idlelib/*.py, and turtle.py in a separate package. I tried your example. It sort of worked but was a bit temperamental when pressing Rshift. But I also tried it in a virtual Ubuntu which said that tkinter was not installed. And I tried it on Debian on a raspberry pi without X-windows running, and there it also failed even though it didn't use a window. So even with a supposedly standard library such as tkinter, relying on it can give problems. (Apart from to switch between between Tkinter and tkinter depending on Python version.) -- Bartc From jeanmichel at sequans.com Thu Oct 27 07:33:03 2016 From: jeanmichel at sequans.com (jmp) Date: Thu, 27 Oct 2016 13:33:03 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On 10/27/2016 12:22 PM, pozz wrote: > Anyway I don't like this approach, because the main (and single) thread > should check in_waiting every X milliseconds. > If X is too high, I could wait for the answer even if it is already > ready in the input buffer. > If X is too low, the application consumes a lot of clocks to check > in_waiting. > > I would prefer to have a callback automatically called when the read > operation is complete. And I think the only method is using another > (blocking) thread. The blocking function read returns *immediately* when > all the bytes are received. And I think during blocking time, the > thread isn't consuming CPU clocks. Threads do consume CPU clocks. An operation within a thread will not consume less CPU clocks, however, the scheduler will interrupt the thread and give other threads/operations a chance to process as well. Threads implement paralellism, not performances. From what I understand of your context, you don't want you GUI to "freeze" when waiting for the remote application. That's a valid concern. You can use threads to fix that(or you can use already written working python libraries that would mask this low level programing, it's up to you). What you should not do is focus on gaining "CPU clocks". You just don't care. It's probably not an issue. If it is, drop python and implement your app in C. JM From steve+python at pearwood.info Thu Oct 27 07:41:38 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Thu, 27 Oct 2016 22:41:38 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> On Thu, 27 Oct 2016 09:42 pm, BartC wrote: > On 27/10/2016 07:51, Steven D'Aprano wrote: >> On Thursday 27 October 2016 12:12, BartC wrote: >> >>> I don't >>> understand the argument that a language shouldn't have a basic keyboard >>> API because some computers it could run on might not have a keyboards. >> >> That's not the argument. The argument is that Python has a basic keyboard >> API: raw_input (in Python 2) or input (in 3). >> >> This supports 97% of keyboard-based interaction, namely blocking >> line-based text input. Ctrl-C (KeyboardInterrupt) can be co-opted to >> support maybe another one or two percent. >> >> The question is, what are the use-cases for the sorts of key APIs you are >> asking for? Who needs them? Why should it be a language feature? > > If a language (or more likely an OS such as Unix) doesn't natively > provide a certain feature, then it's not surprising that 97% of > applications don't use it! Perhaps you should read my statement again. My claim (plucked from thin air, of course, but nevertheless I'm sticking by it) is that for 97% of use-cases, when a non-GUI program needs text input from the user, the *right solution* is blocking line-based text input. I don't need one character at a time. I want to pause everything else, ask the user a question, and wait for them to enter an entire line. And no, I don't want to reinvent the wheel and build up line editing from character editing myself. I don't want to have to handle backspacing and navigation myself. >> Those are not rhetoricial questions. Feel free to give an answer. Apart from reinventing the wheel and building functionality that Python already supports, what do you use non-blocking keyboard input for? Can you give some examples of how you might use it? >> Python falls neatly into the same niche of languages as (for example) >> Ruby, Javascript, Lua, Swift, Tcl, and (not quite as well) bash, Java, >> Julia. Which of these languages offer non-blocking keyboard input as a >> standard part of the language? > > That doesn't surprise me either. That was a question, not a statement. You claim that this sort of low-level non-blocking keyboard input is a "basic" API. Okay, then which other languages offer this? If Python is the odd one out, if every other programming language bar Python provides this API, then I'll cheerfully acknowledge that I'm terribly wrong. I don't understand what this function is good for myself, but obviously thousands of others do, so I'll learn something from them. Or... if no other language offers this "basic" API, then maybe its not that useful or simple, and perhaps not that basic. > Even implementation languages such as C > tend to shy away from the issue. And since C was intimately associated > with Unix, it starts to get even less surprising! How about Lisp? Scheme? Fortran? Java? C#? Objective-C? Dylan? Forth? Well I'd completely believe Forth has this, I think you'd like Chuck Moore, I think the two of you think in similar ways. Cobol? Hypertalk? Inform 7? Bash? Haskell? There must be *some* language other than your own that offers this feature, if it is as obvious, useful, simple and basic and you claim. > But I seem to remember in a previous thread that Python had some problem > even with line-buffered input. Something to do with only getting a line > of input as a string then needed to do some processing to read > individual elements, IIRC. O_o Um, of course it gets a line of input as a text string. What would you expect to get the user's text as? A bitmap? > (FWIW my own language does strive to have this basic stuff built it. But > getting full keyboard and console handling working as I want it across > both Windows and Linux is challenging. It's a bit easier on Windows as, > even though you're using a console, you have the full resources of the > Win32 API to draw on. > > On Linux you can't assume any such resources except some apparently > 1970s-style terminal handling, from what I can figure out.) And possibly not even that. If your script is running as a cron job, I believe that there's no terminal attached, and possibly no stdin or stdout. I don't remember the details. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Thu Oct 27 07:43:33 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 22:43:33 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 10:33 PM, jmp wrote: > On 10/27/2016 12:22 PM, pozz wrote: >> >> Anyway I don't like this approach, because the main (and single) thread >> should check in_waiting every X milliseconds. >> If X is too high, I could wait for the answer even if it is already >> ready in the input buffer. >> If X is too low, the application consumes a lot of clocks to check >> in_waiting. >> >> I would prefer to have a callback automatically called when the read >> operation is complete. And I think the only method is using another >> (blocking) thread. The blocking function read returns *immediately* when >> all the bytes are received. And I think during blocking time, the >> thread isn't consuming CPU clocks. > > > Threads do consume CPU clocks. > An operation within a thread will not consume less CPU clocks, however, the > scheduler will interrupt the thread and give other threads/operations a > chance to process as well. > Threads implement paralellism, not performances. Blocked threads don't consume CPU time. Why would they? Remember, folks, *EVERY* program has at least one thread. Threads aren't some weird and magical thing that you have to be scared of. They're things you use *all the time*, and they are exactly what you're used to. ChrisA From pozzugno at gmail.com Thu Oct 27 07:56:36 2016 From: pozzugno at gmail.com (pozz) Date: Thu, 27 Oct 2016 13:56:36 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: Il 27/10/2016 13:33, jmp ha scritto: > On 10/27/2016 12:22 PM, pozz wrote: >> Anyway I don't like this approach, because the main (and single) thread >> should check in_waiting every X milliseconds. >> If X is too high, I could wait for the answer even if it is already >> ready in the input buffer. >> If X is too low, the application consumes a lot of clocks to check >> in_waiting. >> >> I would prefer to have a callback automatically called when the read >> operation is complete. And I think the only method is using another >> (blocking) thread. The blocking function read returns *immediately* when >> all the bytes are received. And I think during blocking time, the >> thread isn't consuming CPU clocks. > > Threads do consume CPU clocks. > An operation within a thread will not consume less CPU clocks, however, > the scheduler will interrupt the thread and give other > threads/operations a chance to process as well. > Threads implement paralellism, not performances. Yes of course, but when the backend thread calls the *blocking* function pyserial.read(), it *doesn't* consume CPU clocks (at least, I hope). The low-level implementation of pyserial.read() should move the thread in a "suspend" or "waiting" state, so the thread scheduler shouldn't activate it. The suspend state is exited (automatically from OS, I think) when one or more bytes are ready in the input buffer. > From what I understand of your context, you don't want you GUI to > "freeze" when waiting for the remote application. That's a valid concern. > > You can use threads to fix that(or you can use already written working > python libraries that would mask this low level programing, it's up to > you). > > What you should not do is focus on gaining "CPU clocks". You just don't > care. It's probably not an issue. If it is, drop python and implement > your app in C. Because I don't want to drop python, I want to learn the best tecnique to use to have the best performance. From rosuav at gmail.com Thu Oct 27 08:02:12 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 23:02:12 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 10:56 PM, pozz wrote: > Yes of course, but when the backend thread calls the *blocking* function > pyserial.read(), it *doesn't* consume CPU clocks (at least, I hope). > The low-level implementation of pyserial.read() should move the thread in a > "suspend" or "waiting" state, so the thread scheduler shouldn't activate it. > The suspend state is exited (automatically from OS, I think) when one or > more bytes are ready in the input buffer. Exactly. ChrisA From jeanmichel at sequans.com Thu Oct 27 08:33:22 2016 From: jeanmichel at sequans.com (jmp) Date: Thu, 27 Oct 2016 14:33:22 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On 10/27/2016 01:43 PM, Chris Angelico wrote: > Blocked threads don't consume CPU time. Why would they? > > ChrisA > Agreed. My point being that a blocked thread achieve nothing, except parallelism, i.e. other threads can be processed. To be more specific, if you compute factorial(51354) in a thread, it will still require approx. the same amount of CPU clocks than in a main thread (probably slightly more due to the scheduler overhead). jm From rosuav at gmail.com Thu Oct 27 08:55:06 2016 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 27 Oct 2016 23:55:06 +1100 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 11:33 PM, jmp wrote: > On 10/27/2016 01:43 PM, Chris Angelico wrote: >> >> Blocked threads don't consume CPU time. Why would they? >> >> ChrisA >> > > Agreed. My point being that a blocked thread achieve nothing, except > parallelism, i.e. other threads can be processed. > > To be more specific, if you compute factorial(51354) in a thread, it will > still require approx. the same amount of CPU clocks than in a main thread > (probably slightly more due to the scheduler overhead). > > jm Of course. But the OP wants to do blocking calls, which don't cost you like that. So it's fine. ChrisA From bc at freeuk.com Thu Oct 27 09:13:30 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 14:13:30 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: [repost as original disappeared] On 27/10/2016 12:41, Steve D'Aprano wrote: > On Thu, 27 Oct 2016 09:42 pm, BartC wrote: > I don't need one character at a time. I want to pause everything else, ask > the user a question, and wait for them to enter an entire line. > > And no, I don't want to reinvent the wheel and build up line editing from > character editing myself. I don't want to have to handle backspacing and > navigation myself. You're just throwing the ball into someone else's court then. YOU don't want such features to be part of a language, but you expect others to be able to write text-mode editors, IDEs, file-managers, games etc without them? Presumably you're OK with those features being available in a GUI environment? So why the discrimination? Text-apps aren't so common now but I get the impression that you would never have found this stuff useful. > Feel free to give an answer. Apart from reinventing the wheel and building > functionality that Python already supports, what do you use non-blocking > keyboard input for? Can you give some examples of how you might use it? I didn't even know what non-blocking meant until this thread. But suppose you're writing an app such as a debugger, or interpreter, or anything that keeps going until interrupted or controlled by some key event. If you're executing a billion instructions per second you don't want to keep stopping every N instructions to ask the user for any special requests, or to press Enter to continue. Is that good enough? Doubtless your solution would be some large sledgehammer to crack this particular nut. > How about Lisp? Scheme? Fortran? Java? C#? Objective-C? Dylan? Forth? Well > I'd completely believe Forth has this, I think you'd like Chuck Moore, I > think the two of you think in similar ways. > > Cobol? Hypertalk? Inform 7? Bash? Haskell? > There must be *some* language other than your own that offers this feature, > if it is as obvious, useful, simple and basic and you claim. I'm not familiar with that many languages. However if you google for ' non-blocking keyboard' then it seems quite a few people are interested in the feature! >> But I seem to remember in a previous thread that Python had some problem >> even with line-buffered input. Something to do with only getting a line >> of input as a string then needed to do some processing to read >> individual elements, IIRC. > > O_o > > Um, of course it gets a line of input as a text string. What would you > expect to get the user's text as? A bitmap? print "Enter 3 numbers: " readln a,b,c I'm fairly sure this kind of input was common in plenty of other languages. But maybe even that is being frowned on now; /that/ wouldn't surprise me either. > >> (FWIW my own language does strive to have this basic stuff built it. But >> getting full keyboard and console handling working as I want it across >> both Windows and Linux is challenging. It's a bit easier on Windows as, >> even though you're using a console, you have the full resources of the >> Win32 API to draw on. >> >> On Linux you can't assume any such resources except some apparently >> 1970s-style terminal handling, from what I can figure out.) > > And possibly not even that. If your script is running as a cron job, I > believe that there's no terminal attached, and possibly no stdin or stdout. > I don't remember the details. The same argument as before. So we don't have print() as part of the language either before there might not be a stdout?! The fact is that if I want to run an editor under Linux, I will need a display, and a keyboard, and I want to have the means to use them. -- bartc From jeanmichel at sequans.com Thu Oct 27 09:14:19 2016 From: jeanmichel at sequans.com (jmp) Date: Thu, 27 Oct 2016 15:14:19 +0200 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: On 10/27/2016 02:55 PM, Chris Angelico wrote: > On Thu, Oct 27, 2016 at 11:33 PM, jmp wrote: >> On 10/27/2016 01:43 PM, Chris Angelico wrote: >>> >>> Blocked threads don't consume CPU time. Why would they? >>> >>> ChrisA >>> >> >> Agreed. My point being that a blocked thread achieve nothing, except >> parallelism, i.e. other threads can be processed. >> >> To be more specific, if you compute factorial(51354) in a thread, it will >> still require approx. the same amount of CPU clocks than in a main thread >> (probably slightly more due to the scheduler overhead). >> >> jm > > Of course. But the OP wants to do blocking calls, which don't cost you > like that. So it's fine. > > ChrisA Sure but the OP is very focus on performance(that's a mistake imo). "Because I don't want to drop python, I want to learn the best technique to use to have the best performance. " I just wanted to point that using thread implements parallelism, not performance. And that's probably what its gui needs. And that's probably why using a higher level API would have been acceptable. JM Best performance is achieved by sacrificing a lot in python. A better technique than polling threads would be sleeping thread where the thread is put to hold until a hardware interrupt wakes up the thread. From marko at pacujo.net Thu Oct 27 09:24:17 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Thu, 27 Oct 2016 16:24:17 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87vawd3oxq.fsf@elektro.pacujo.net> BartC : > If you're executing a billion instructions per second you don't want > to keep stopping every N instructions to ask the user for any special > requests, or to press Enter to continue. In mobile computing, such wakeups drain the battery. Marko From grant.b.edwards at gmail.com Thu Oct 27 10:07:45 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 27 Oct 2016 14:07:45 +0000 (UTC) Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87h97ymo1g.fsf@nightsong.com> <87y41awd1a.fsf@elektro.pacujo.net> Message-ID: On 2016-10-27, Marko Rauhamaa wrote: > Grant Edwards : >> I've offered a few times to extend the Linux pty driver to support the >> same set of ioctl calls that a tty does (so that it could be used in >> place of a tty generally), but I've never gotten any response. > > Ah, Linux kernel politics are Byzantine. It's virtually impossible to > get a hearing at the linux-kernel main mailing list. Did you try one of > the targeted mailing lists on http://vger.kernel.org/vger-lists.html>? Yes, I posted my proposal to the linux-serial list -- which I _think_ is the right one for the pty driver, but that may not have been the right list. I suspect proposals like mine generally just get ignored unless you can show code to back them up. -- Grant Edwards grant.b.edwards Yow! My face is new, my at license is expired, and I'm gmail.com under a doctor's care!!!! From grant.b.edwards at gmail.com Thu Oct 27 10:18:43 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 27 Oct 2016 14:18:43 +0000 (UTC) Subject: Why doesn't Python include non-blocking keyboard input function? References: <87lgxcxgt4.fsf@elektro.pacujo.net> <87d1inyb0s.fsf@elektro.pacujo.net> <87insf5lys.fsf@elektro.pacujo.net> <8737jixrx0.fsf@elektro.pacujo.net> Message-ID: On 2016-10-27, Terry Reedy wrote: > When I used unix in the 1980s, the full screen ran csh until one started > another full screen application. MSDOS was the same. Every contemporary > photo of modern Linux or Mac I have seen has a desktop with windows just > like Windows. Do people on Linux still commonly use full-screen, It depends on your definition of "commonly". I do it fairly regularly, but only for short periods of time while doing system maintenance stuff. > no window text editors like the one I had? Don't know what you mean by "window text editors" > On Windows, there are full screen games, but I have never seen a > full-screen, no-window text application. Just con't conflate "full-screen" with "command-line" or "terminal" applications. I do use terminal applications all day, every day, but I mostly run them in terminal emulator windows on top of X11 (that way I can have lots of terminals of various sizes and shapes). > Since modern screen are pixel graphics screens, rather than character > screens, there must be a widget, whether standard with the OS or custom > to the console, that emulates the old fixed-pitch character screens. Yes. On Unix they're called terminal emulators, and I use lots of them them constantly. This post is being edited in one. > At least on Windows, C Programs that run with the console still get > characters entered by users and send characters to be displayed. On Unix, a terminal emulator on an X11/Wayland desktop, a linux video console, a real serial terminal connected to an RS-232C serial port, or a "screen" session all behave pretty much the same. The API supported by the pty driver, the video console tty driver, and the real serial-port tty driver, all provide a common set of API calls. -- Grant Edwards grant.b.edwards Yow! I wish I was on a at Cincinnati street corner gmail.com holding a clean dog! From darcy at vex.net Thu Oct 27 10:52:52 2016 From: darcy at vex.net (D'Arcy Cain) Date: Thu, 27 Oct 2016 10:52:52 -0400 Subject: How to use two threads (GUI and backend) In-Reply-To: References: Message-ID: <91343fac-a53d-67eb-bd63-795ee2e97f3f@vex.net> On 2016-10-27 07:33 AM, jmp wrote: > On 10/27/2016 12:22 PM, pozz wrote: >> (blocking) thread. The blocking function read returns *immediately* when >> all the bytes are received. And I think during blocking time, the >> thread isn't consuming CPU clocks. > > Threads do consume CPU clocks. Sometimes they do but read what pozz wrote. "during blocking time, the thread isn't consuming CPU clocks". That means that if you have two threads with one computing ? and the other blocked waiting for user input then the first one is using CPU cycles - as many as it can get - and the other is using none until the user enters something. This is kind of the point of threads - only things that need cycles get cycles. > An operation within a thread will not consume less CPU clocks, however, It will if it blocks for something. A simple sleep will also give up the processor. > From what I understand of your context, you don't want you GUI to > "freeze" when waiting for the remote application. That's a valid concern. And that's why you use threads. You can also use select in a state machine loop. Depends on your model and somewhat on your preferences. > What you should not do is focus on gaining "CPU clocks". You just don't > care. It's probably not an issue. If it is, drop python and implement > your app in C. In fact, going to C often is less of a win than you may think. Python is pretty damn efficient if you write good code. -- D'Arcy J.M. Cain System Administrator, Vex.Net http://www.Vex.Net/ IM:darcy at Vex.Net VoIP: sip:darcy at Vex.Net From daiyueweng at gmail.com Thu Oct 27 11:11:13 2016 From: daiyueweng at gmail.com (Daiyue Weng) Date: Thu, 27 Oct 2016 16:11:13 +0100 Subject: Windows switch between python 2 and 3 Message-ID: Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. Under C:\Python35 C:\Python27 Both have been set in environment variable Path. When I type python in cmd, it only gives me python 2.7, I am wondering how to switch between 2 and 3 in command prompt. cheers From ricemom at gmail.com Thu Oct 27 11:14:36 2016 From: ricemom at gmail.com (ricemom at gmail.com) Date: Thu, 27 Oct 2016 08:14:36 -0700 (PDT) Subject: multiprocess passing arguments double asterisks In-Reply-To: References: <332a77c3-8cce-4d1a-846d-f474a7e0fc41@googlegroups.com> <6f9293c5-252f-160f-5e3b-44bdc702e717@gmx.com> <3a8394da-1a09-4e08-8b80-d30987d342b9@googlegroups.com> <870c7a5e-e5ee-bc25-087f-ee9cf35a39e3@gmx.com> <376918e0-15a9-4072-a9dd-42d49398ab95@googlegroups.com> <09738f36-9291-ea51-c842-ab23535128ba@mrabarnett.plus.com> Message-ID: <3b52cc7c-cc99-4efd-8413-19d9cc8f1a3a@googlegroups.com> On Wednesday, October 26, 2016 at 5:31:18 PM UTC-5, MRAB wrote: > On 2016-10-26 21:44, pic8690 at gmail.com wrote: > > On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote: > >> On 10/24/2016 12:45 PM, pic8690 at gmail.com wrote: > >> > Thanks for the reply. > >> > > >> > The code snippet given by Peter is not very clear > >> > > >> > I would like to multiprocess a function which is written in python of the form bar(**kwargs) which returns a value. This example does not return anything > >> > > >> > Would you please use my example for the map function? > >> > > >> > I appreciate your help, > >> > > >> I'm honestly not totally sure what you want to do. However, say you want > >> to do the following (btw this is basically what Dennis said i nhis last > >> email, but maybe I can help clarify): > >> > >> kwargs = {'param1': val1, 'param2': val2}) > >> > >> Then you'd like to have the following two operations performed in > >> separate processes: > >> > >> bar(param1=val1) > >> bar(param2=val2) > >> > >> In that case, I guess I would do something like the following. First > >> define bar_wrapper as follows (*I haven't tested any code here!): > >> > >> def bar_wrapper(pair): > >> key, val = pair > >> return bar(**{key: val}) > >> > >> Then I would probably do something like > >> > >> map(bar_wrapper, kwargs.items()) > >> > >> I.e. basically what I'm doing is taking the key-val pairs and producing > >> a list of them (as tuples). This is something that you can apply map > >> too, but not with the original function. So then the wrapper function > >> converts the tuple back to what you want originally. > >> > >> Hopefully I'm understanding correctly and hopefully this helps. > >> > >> Cheers, > >> Thomas > > > > Thomas, > > > > I have strings & numpy.ndarray as arguments. The wrapper function works great for strings. > > > > Here's what I'm trying to do... > > ************************** > > > > I have a serial python fn of the form > > > > def bar(**kwargs): > > a=kwargs.get("name") > > print a > > self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float) > > (****more calculations--takes a long time to compute > > returns an object) > > > > I am trying to run this function in parallel. > > > > Here's my Main program > > > > import numpy as np > > import numpy.ma as ma > > from delegate import parallelize > > from hashlib import sha1 > > a1d=np.zeros((32)) > > b1d=np.zeros((32)) > > p=open("/tmp/pdata","rb") > > pdata=np.load(p) > > for i in range(0,10): > > a1d=pdata['t1d'] > > b1d=pdata['gz1d'] > > print a1d,b1d > > kwargs={'name':'special','a':a1d,'b':b1d} > > val=parallelize(bar,kwargs) > > *************************************** > > Error: > > line 30, in > > val=parallelize(bar,kwargs) > > delegate.py", line 311, in process > > item, items = items[0], items[1:] > > KeyError: 0 > > ************************************** > > Error: (with the wrapper function) > > val=parallelize(bar_wrapper,kwargs.items()) > > TypeError: unhashable type: 'numpy.ndarray' > > *************************************** > > > > How do I pass the string & numpy.ndarray to the function bar(**kwargs)? > > > > Thank you for suggestions & help, > > > > > > a1d=np.zeros((32)) > > b1d=np.zeros((32)) > > p=open("/tmp/pdata","rb") > > pdata=np.load(p) > > for i in range(0,100): > > a1d=pdata['t1d'] > > b1d=pdata['gz1d'] > > print a1d,b1d > > kwargs={'name':'special','a':a1d,'b':b1d} > > val=parallelize(bar,kwargs) > > > 'parallelize' expects a function and a list of items. It calls the > function with each item in the list in parallel, passing the item as the > argument of the function. > > That _single_ argument item can be a tuple or a dict. --if the dict has numpy arrays it fails.. In the above example if kwargs={'name':'special','a':2} val=parallelize(bar_wrapper,kwargs.items()) it works fine But if kwargs={'name':'special','a':a1d,'b':b1d} it fails (a1d & b1d are numpy arrays) From vek.m1234 at gmail.com Thu Oct 27 11:26:32 2016 From: vek.m1234 at gmail.com (Veek M) Date: Thu, 27 Oct 2016 20:56:32 +0530 Subject: Windows switch between python 2 and 3 References: Message-ID: Daiyue Weng wrote: > Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. > Under > > C:\Python35 > > C:\Python27 > > Both have been set in environment variable Path. > > When I type python in cmd, it only gives me python 2.7, I am wondering > how to switch between 2 and 3 in command prompt. > > cheers you could try typing the path to the interpreter: c:\Python35\bin\python.exe foo.py or changing the path ordering using 'set /?' i think - google From __peter__ at web.de Thu Oct 27 11:37:51 2016 From: __peter__ at web.de (Peter Otten) Date: Thu, 27 Oct 2016 17:37:51 +0200 Subject: Windows switch between python 2 and 3 References: Message-ID: Daiyue Weng wrote: > Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. Under > > C:\Python35 > > C:\Python27 > > Both have been set in environment variable Path. > > When I type python in cmd, it only gives me python 2.7, I am wondering how > to switch between 2 and 3 in command prompt. > > cheers https://docs.python.org/3.5/using/windows.html#python-launcher-for-windows From steve+python at pearwood.info Thu Oct 27 11:41:26 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 28 Oct 2016 02:41:26 +1100 Subject: Windows switch between python 2 and 3 References: Message-ID: <58122027$0$1587$c3e8da3$5496439d@news.astraweb.com> On Fri, 28 Oct 2016 02:11 am, Daiyue Weng wrote: > Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. Under > > C:\Python35 > > C:\Python27 > > Both have been set in environment variable Path. > > When I type python in cmd, it only gives me python 2.7, I am wondering how > to switch between 2 and 3 in command prompt. I don't use Windows, so I am guessing, but you could try: python2 python27 python3 python35 -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From zachary.ware+pylist at gmail.com Thu Oct 27 11:49:35 2016 From: zachary.ware+pylist at gmail.com (Zachary Ware) Date: Thu, 27 Oct 2016 10:49:35 -0500 Subject: Windows switch between python 2 and 3 In-Reply-To: References: Message-ID: On Thu, Oct 27, 2016 at 10:11 AM, Daiyue Weng wrote: > Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. Under > > C:\Python35 > > C:\Python27 > > Both have been set in environment variable Path. > > When I type python in cmd, it only gives me python 2.7, I am wondering how > to switch between 2 and 3 in command prompt. Along with Python 3.5, the Python Launcher for Windows (py.exe) was installed in C:\Windows, unless you unchecked it. You can choose which interpreter you want to run by an argument to py.exe; for Python 2.7, use `py -2.7`; for Python 3.5, use `py -3.5`. There are also shortcuts for "latest Python 2" and "latest Python 3", `py -2` and `py -3` respectively. Calling `py` on its own will default to matching `py -2`, but settings can be changed in C:\Windows\py.ini. See https://docs.python.org/using/windows.html#launcher for more details. -- Zach From vek.m1234 at gmail.com Thu Oct 27 11:52:09 2016 From: vek.m1234 at gmail.com (Veek M) Date: Thu, 27 Oct 2016 21:22:09 +0530 Subject: problem using pickle References: <85wpl4u0gt.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > "Veek. M" writes: > >> class Foo(object): >> pass >> >> object is a keyword and you're using it as an identifier > > Python does not have ?object? as a keyword. ?and? is a keyword. > > Here's the difference:: > > >>> object > > >>> object = "Lorem ipsum" > >>> object > 'Lorem ipsum' > > >>> and > File "", line 1 > and > ^ > SyntaxError: invalid syntax > >>> and = "Lorem ipsum" > File "", line 1 > and = "Lorem ipsum" > ^ > SyntaxError: invalid syntax > > Here is how you can test whether a word is a Python keyword:: > > >>> import keyword > >>> keyword.iskeyword('object') > False > >>> keyword.iskeyword('and') > True > > The set of keywords in Python is quite small. > > >>> len(keyword.kwlist) > 33 > Oh awesome - thanks - that's a neat module. Yeah, basically keywords are part of the language but not punctuation - kind of like how you had to do: print 'foo' in 2.x - part of the grammar of the language. Dunno why i muffed that.. 'object's a class anyhow so it couldn't be a keyword (inherited with its methods etc) - thanks for catching that and sorry for the delay.. saw that today. From vek.m1234 at gmail.com Thu Oct 27 11:53:10 2016 From: vek.m1234 at gmail.com (Veek M) Date: Thu, 27 Oct 2016 21:23:10 +0530 Subject: problem using pickle References: <7e4ba6e5-eb3f-4c07-8c3a-6c3c24991ca0@googlegroups.com> Message-ID: Rustom Mody wrote: > On Saturday, July 2, 2016 at 9:17:01 AM UTC+5:30, Veek. M wrote: >> object is a keyword and you're using it as an identifier > > keyword and builtin are different > In this case though the advice remains the same > In general maybe not... > Just sayin' np - feel free to correct me From steve+python at pearwood.info Thu Oct 27 12:13:12 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 28 Oct 2016 03:13:12 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> On Fri, 28 Oct 2016 12:13 am, BartC wrote: > [repost as original disappeared] > On 27/10/2016 12:41, Steve D'Aprano wrote: >> On Thu, 27 Oct 2016 09:42 pm, BartC wrote: > >> I don't need one character at a time. I want to pause everything else, >> ask the user a question, and wait for them to enter an entire line. >> >> And no, I don't want to reinvent the wheel and build up line editing from >> character editing myself. I don't want to have to handle backspacing and >> navigation myself. > > You're just throwing the ball into someone else's court then. Well of course I am. I don't program my code from bare metal. Nobody with any sense does that, at least not on a PC. Maybe on embedded systems. I don't write my own memory management, or my own file system, or my own process scheduler, or any of the other dozens of things that the OS does much, much better than anything I can do. Including line editing. > YOU don't want such features to be part of a language, but you expect > others to be able to write text-mode editors, IDEs, file-managers, games > etc without them? Ah, now we're getting somewhere: I'm starting to get a hint of what this feature might be useful for. Okay. Sure, I'll accept that perhaps this feature of non-blocking keyboard input does have its use-cases. I might even revise my plucked-from-thin-air figure of 97% down to allow for "text-mode editors, IDEs, file-managers, games, etc.". All of those are big, complex applications. Not Microsoft Office big, or Oracle database big, but still, you need things like windows, dialog boxes, animation, sprites, etc. If I were writing any of those, I would absolutely want to start with an existing framework or library that provides things like windows, menus, icons, and text input widgets. Whether I'm doing a GUI application or a text-based application, I wouldn't be writing this stuff from scratch like it was 1973. And yes, that's "throwing the ball into someone else's court". There's no shame in that. > Presumably you're OK with those features being available in a GUI > environment? So why the discrimination? What discrimination? They're available in text environments too. You just have to use a good text-based UI library, like curses. > Text-apps aren't so common now but I get the impression that you would > never have found this stuff useful. > >> Feel free to give an answer. Apart from reinventing the wheel and >> building functionality that Python already supports, what do you use >> non-blocking keyboard input for? Can you give some examples of how you >> might use it? > > I didn't even know what non-blocking meant until this thread. When you call (raw_)input(), Python stops processing until the user hits Enter, at which point it returns whatever they typed. Earlier, I mistakenly posted a recipe that works similarly: it pauses until the user hits a key. It *blocks* until the user hits a key, then returns that key and continues. A non-blocking function might return None if there is no character in the input buffer, or that character. I don't know -- I'm not sure how the API would be designed. > But suppose you're writing an app such as a debugger, or interpreter, or > anything that keeps going until interrupted or controlled by some key > event. If you're executing a billion instructions per second you don't > want to keep stopping every N instructions to ask the user for any > special requests, or to press Enter to continue. > > Is that good enough? > > Doubtless your solution would be some large sledgehammer to crack this > particular nut. *shrug* Python's a pretty high-level language. Not every low-level feature needs to be part of the Python language. This sounds like something you would write in a low-level language like C or Rust, turn it into a extension file, and call it from Python. You could always check the source code to the Python debugger. Presumably it already solves this. There has to be *some* solution. After all, operating systems, text editors, shells, GUI frameworks, etc. all support this. >> How about Lisp? Scheme? Fortran? Java? C#? Objective-C? Dylan? Forth? >> Well I'd completely believe Forth has this, I think you'd like Chuck >> Moore, I think the two of you think in similar ways. >> >> Cobol? Hypertalk? Inform 7? Bash? Haskell? > >> There must be *some* language other than your own that offers this >> feature, if it is as obvious, useful, simple and basic and you claim. > > I'm not familiar with that many languages. However if you google for > ' non-blocking keyboard' then it seems quite a few people are > interested in the feature! *shrug* My very first response suggested that many people ask for this feature, but then after writing code to implement it, they find they don't actually need it. Maybe I'm wrong. But if this is so useful, outside of specialist areas, why do so few languages support it? >>> But I seem to remember in a previous thread that Python had some problem >>> even with line-buffered input. Something to do with only getting a line >>> of input as a string then needed to do some processing to read >>> individual elements, IIRC. >> >> O_o >> >> Um, of course it gets a line of input as a text string. What would you >> expect to get the user's text as? A bitmap? > > print "Enter 3 numbers: " > readln a,b,c How is the interpreter supposed to know that a, b, c are numbers? What sort of numbers? 16-bit integers, 80-bit floats, Bignums, complex, Fractions, or something else? That's a fairly simple thing for the compiler to do in a language with static types. It knows that a is a float, b an int, and c another float. So it can automatically call the appropriate input routines, or conversion routines, or both. But in a dynamically typed language, the compiler has no idea what you expect a, b and c to be. So it returns text, and you can convert it yourself. > I'm fairly sure this kind of input was common in plenty of other > languages. But maybe even that is being frowned on now; /that/ wouldn't > surprise me either. Nah, I'm pretty sure that's just a simple issue of whether or not the compiler knows the type. If it does, then you can say: x: float; x = read(); and the compiler can choose the right input routine. Otherwise you have to do so yourself: x = read_float(); [...] >>> On Linux you can't assume any such resources except some apparently >>> 1970s-style terminal handling, from what I can figure out.) >> >> And possibly not even that. If your script is running as a cron job, I >> believe that there's no terminal attached, and possibly no stdin or >> stdout. I don't remember the details. > > The same argument as before. So we don't have print() as part of the > language either before there might not be a stdout?! I didn't say that. I said that print might not work, if you call it from a cron job. (Or it may -- as I said, I don't remember the details. All I remember is that cron jobs are run under very restrictive environments.) > The fact is that if I want to run an editor under Linux, I will need a > display, and a keyboard, and I want to have the means to use them. That's nice, but Python is not an Editor Construction Kit that specialises in writing editors. It can operate under all sorts of conditions, including some where no editor is possible because there's no keyboard input. If this feature is so basic, so popular, and so necessary, why hasn't anyone written the code to provide it and release it for others to use? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From daiyueweng at gmail.com Thu Oct 27 12:16:26 2016 From: daiyueweng at gmail.com (Daiyue Weng) Date: Thu, 27 Oct 2016 17:16:26 +0100 Subject: Windows switch between python 2 and 3 In-Reply-To: References: Message-ID: python windows launcher seems like the best option here. thanks On 27 October 2016 at 16:49, Zachary Ware wrote: > On Thu, Oct 27, 2016 at 10:11 AM, Daiyue Weng > wrote: > > Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. > Under > > > > C:\Python35 > > > > C:\Python27 > > > > Both have been set in environment variable Path. > > > > When I type python in cmd, it only gives me python 2.7, I am wondering > how > > to switch between 2 and 3 in command prompt. > > Along with Python 3.5, the Python Launcher for Windows (py.exe) was > installed in C:\Windows, unless you unchecked it. You can choose > which interpreter you want to run by an argument to py.exe; for Python > 2.7, use `py -2.7`; for Python 3.5, use `py -3.5`. There are also > shortcuts for "latest Python 2" and "latest Python 3", `py -2` and `py > -3` respectively. Calling `py` on its own will default to matching > `py -2`, but settings can be changed in C:\Windows\py.ini. See > https://docs.python.org/using/windows.html#launcher for more details. > > -- > Zach > From demosthenesk at gmail.com Thu Oct 27 12:56:38 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Thu, 27 Oct 2016 19:56:38 +0300 Subject: [PyQT] After MessageBox app quits...why? Message-ID: <4111aac5-a0c0-928d-bfa4-7d93c6523d85@gmail.com> Hello, i have a PyQT systray app with a menu and two actions. Action1 is Exit and action2 display a MessageBox with Hello World message. When i click OK to MessageBox app quits...why? http://pastebin.com/bVA49k1C From bc at freeuk.com Thu Oct 27 14:09:48 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 19:09:48 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 17:13, Steve D'Aprano wrote: > On Fri, 28 Oct 2016 12:13 am, BartC wrote: >> Doubtless your solution would be some large sledgehammer to crack this >> particular nut. > > *shrug* > > Python's a pretty high-level language. Not every low-level feature needs to > be part of the Python language. This sounds like something you would write > in a low-level language like C or Rust, turn it into a extension file, and > call it from Python. I tend not to separate out which applications which are suitable for low-level and which are better for high-level languages. And I've seen things like 'import msvcrt', 'import winapi' in Python code, and then there's all that stuff with ctypes. It look like other people don't worry about that either. Although if the language is really unsuitable then they'll find out when it runs too slowly to be practical. > You could always check the source code to the Python debugger. Presumably it > already solves this. There has to be *some* solution. After all, operating > systems, text editors, shells, GUI frameworks, etc. all support this. Yes, all those kinds of application need these basic features. Isn't it odd then that not a single mainstream language provides them as standard? (From my point of view the discussion has been about any key-at-a-time events rather than blocking or not. Those are also problematical.) >> I'm not familiar with that many languages. However if you google for >> ' non-blocking keyboard' then it seems quite a few people are >> interested in the feature! > > *shrug* My very first response suggested that many people ask for this > feature, but then after writing code to implement it, they find they don't > actually need it. Maybe I'm wrong. But if this is so useful, outside of > specialist areas, why do so few languages support it? For years I've had discussions in comp.lang.c about things that C should or should not have. What usually happens is that someone comes up with some crude workaround using macros, so there is less need to have some feature built-in (and the language fails to a acquire a slick new enhancement). But with the usual trouble that everyone then has to re-invent the same macro but in a different way to everyone else. The same thing probably happens with a low-level keyboard API. There are half a dozen ways of achieving the same functionality (with varying degrees of hassle), so maybe the language doesn't need to provide a standard solution after all. You talk about not wanting to re-invent things, but that's exactly what everyone ends up doing! >> print "Enter 3 numbers: " >> readln a,b,c > > How is the interpreter supposed to know that a, b, c are numbers? What sort > of numbers? 16-bit integers, 80-bit floats, Bignums, complex, Fractions, or > something else? > But in a dynamically typed language, the compiler has no idea what you > expect a, b and c to be. So it returns text, and you can convert it > yourself. So you tell it what to expect. The above example is from a dynamic language, and will assume integers unless told otherwise. This: readln a:"h", b:"r", c:"s" reads a hex integer, floating point number and a string respectively (with a default set of separators delimiting the string, or "..." can be used). Or values can be read one at a time: readln read a:"h" I think this came up when someone wanted to switch from Visual Basic to Python. The above is not a very sophisticated approach but it is very simple and intuitive. -- bartc From eryksun at gmail.com Thu Oct 27 16:13:50 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 27 Oct 2016 20:13:50 +0000 Subject: Windows switch between python 2 and 3 In-Reply-To: <58122027$0$1587$c3e8da3$5496439d@news.astraweb.com> References: <58122027$0$1587$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Thu, Oct 27, 2016 at 3:41 PM, Steve D'Aprano wrote: > On Fri, 28 Oct 2016 02:11 am, Daiyue Weng wrote: > >> Hi, I installed Python 2.7 and Python 3.5 64 bit versions on Win 10. Under >> >> C:\Python35 >> >> C:\Python27 >> >> Both have been set in environment variable Path. >> >> When I type python in cmd, it only gives me python 2.7, I am wondering how >> to switch between 2 and 3 in command prompt. > > I don't use Windows, so I am guessing, but you could try: > > python2 > python27 > > python3 > python35 These links aren't created by the installer, but you can create them if that's your preference. Execute the following in an admin command prompt: mklink C:\Python27\python2.7.exe C:\Python27\python.exe mklink C:\Python27\python2.exe C:\Python27\python.exe mklink C:\Python35\python3.5.exe C:\Python35\python.exe mklink C:\Python35\python3.exe C:\Python35\python.exe From demosthenesk at gmail.com Thu Oct 27 17:30:52 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Fri, 28 Oct 2016 00:30:52 +0300 Subject: How to execute "gksudo umount VirtualDVD" Message-ID: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> I want to execute the command "gksudo umount VirtualDVD" My code is this but it fails: def umount(self): '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() print proc It pops up the gksudo dialog, and then fails. But i don't get any stdout or stderror. I tried cmd='gksudo ls' and it succeeds. From greg.ewing at canterbury.ac.nz Thu Oct 27 17:51:16 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Fri, 28 Oct 2016 10:51:16 +1300 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> Message-ID: BartC wrote: > "There's a room in your house with no door to it; how do I get in?" > > "There's no need for a door because no one ever uses that room! But you > can get in through the chimney - if you /have/ to." It's not like that. The room *does* have a door, it's just that it's in different places in different houses, and may require a different key to open it. > function peek(p,t=byte)= > return makeref(p,t)^ > end You can achieve similar things in Python using ctypes if you really need to live that dangerously. -- Greg From ian.g.kelly at gmail.com Thu Oct 27 17:54:59 2016 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Thu, 27 Oct 2016 15:54:59 -0600 Subject: How to execute "gksudo umount VirtualDVD" In-Reply-To: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> Message-ID: On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis wrote: > I want to execute the command "gksudo umount VirtualDVD" > > My code is this but it fails: > > def umount(self): > '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = > subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() > print proc It looks like your code pasted incorrectly. > It pops up the gksudo dialog, and then fails. But i don't get any stdout or > stderror. Fails how? Is there an error? Does it hang? Does nothing happen at all? My initial thought is that you might want to try using Popen.communicate instead of stdout.read in case you're getting deadlocked. See the big red warning below https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout From bc at freeuk.com Thu Oct 27 18:02:24 2016 From: bc at freeuk.com (BartC) Date: Thu, 27 Oct 2016 23:02:24 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 19:09, BartC wrote: > On 27/10/2016 17:13, Steve D'Aprano wrote: >> On Fri, 28 Oct 2016 12:13 am, BartC wrote: >>> print "Enter 3 numbers: " >>> readln a,b,c >> >> How is the interpreter supposed to know that a, b, c are numbers? What >> sort >> of numbers? 16-bit integers, 80-bit floats, Bignums, complex, >> Fractions, or >> something else? > >> But in a dynamically typed language, the compiler has no idea what you >> expect a, b and c to be. So it returns text, and you can convert it >> yourself. I notice that when it comes to reading command-line arguments, then Python's sys.argv presents them as a list, not one long string. And the list is just a series of strings, so needing to know whether any parameter was a number or whatever obviously wasn't a problem. It just makes each item into a string (actually that might be a better default than mine). This is a very similar issue to reading items from a line of user input. So why doesn't sys.argv just return a single string if a line is so easy to parse? (That's exactly what Windows' WinMain() function does - optional entry point for executables under Windows. But C's main() entry point chops the command line up into separate strings like Python. Also - this might some bearing on why Python does it that way - under Linux, a parameter such as *.py is replaced by the names of ALL the files that end in .py. (I was rather astonished when I find out. But I've recently had to deal with a directory containing 3,400,000 files so having a single "*" converted into 3.4 million filenames would be unwelcome.)) -- Bartc From karenjhermann at gmail.com Thu Oct 27 18:10:31 2016 From: karenjhermann at gmail.com (Karen Hermann) Date: Thu, 27 Oct 2016 17:10:31 -0500 Subject: Installing Python on Windows 7 Message-ID: <73FE32AF-093D-4A9D-8A5C-E2DD86F34476@gmail.com> I just downloaded Python 3.5.2 for Windows, which I installed on a Windows 7 laptop. Disclaimer up front, I am a former lapsed programmer that has been away from it and all things Windows since Windows XP. :) I?m back to being a bit of a newbie again. It?s a relatively clean laptop, just got it, and rarely use it, and there?s nothing else on it except Microsoft Office 2013. I downloaded Python, installed it, tried to run it, and got this message: The program can?t start because api-ms-win-crd-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem. I?ve redownloaded it and reinstalled it several times, each time deleting all traces of it before trying again. I?ve even disabled my virus-scanner (Sophos), thinking that might be interfering but that didn?t help either. Nothing is helping, I can?t get past this. Can you help please, is there something wrong with my system / setup, should I be downloading a different Python version? Thanks in advance for any help you can give! From rosuav at gmail.com Thu Oct 27 18:31:00 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Oct 2016 09:31:00 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 28, 2016 at 9:02 AM, BartC wrote: > > I notice that when it comes to reading command-line arguments, then Python's > sys.argv presents them as a list, not one long string. > > And the list is just a series of strings, so needing to know whether any > parameter was a number or whatever obviously wasn't a problem. It just makes > each item into a string (actually that might be a better default than mine). > > This is a very similar issue to reading items from a line of user input. > > So why doesn't sys.argv just return a single string if a line is so easy to > parse? Because the OS provides a series of strings, not just one string. When you exec to a process, you provide multiple arguments, not a single combined string. ChrisA From bc at freeuk.com Thu Oct 27 19:45:03 2016 From: bc at freeuk.com (BartC) Date: Fri, 28 Oct 2016 00:45:03 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 27/10/2016 23:31, Chris Angelico wrote: > On Fri, Oct 28, 2016 at 9:02 AM, BartC wrote: >> >> I notice that when it comes to reading command-line arguments, then Python's >> sys.argv presents them as a list, not one long string. >> >> And the list is just a series of strings, so needing to know whether any >> parameter was a number or whatever obviously wasn't a problem. It just makes >> each item into a string (actually that might be a better default than mine). >> >> This is a very similar issue to reading items from a line of user input. >> >> So why doesn't sys.argv just return a single string if a line is so easy to >> parse? > > Because the OS provides a series of strings, not just one string. I don't think that's the case on Windows. Perhaps on Unix. When > you exec to a process, you provide multiple arguments, not a single > combined string. Really, there could be dozens of arguments? Windows' CreateProcess() (if that's the same thing) has ten of which one is the command-line as a single string, while C's system() just has one. This might just be one of those Unixisms that doesn't apply on all platforms. -- Bartc From eryksun at gmail.com Thu Oct 27 19:57:54 2016 From: eryksun at gmail.com (eryk sun) Date: Thu, 27 Oct 2016 23:57:54 +0000 Subject: Installing Python on Windows 7 In-Reply-To: <73FE32AF-093D-4A9D-8A5C-E2DD86F34476@gmail.com> References: <73FE32AF-093D-4A9D-8A5C-E2DD86F34476@gmail.com> Message-ID: On Thu, Oct 27, 2016 at 10:10 PM, Karen Hermann wrote: > > Can you help please, is there something wrong with my system / setup, should I be > downloading a different Python version? The system is missing the new C runtime. Enable Windows Update and install the recommended updates. From rosuav at gmail.com Thu Oct 27 20:08:34 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Oct 2016 11:08:34 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 28, 2016 at 10:45 AM, BartC wrote: > On 27/10/2016 23:31, Chris Angelico wrote: >> >> When >> you exec to a process, you provide multiple arguments, not a single >> combined string. > > > Really, there could be dozens of arguments? Windows' CreateProcess() (if > that's the same thing) has ten of which one is the command-line as a single > string, while C's system() just has one. system() passes its argument through to the shell for parsing. In the same way, Python's Popen constructor can either take a list of strings, or a single string. > This might just be one of those Unixisms that doesn't apply on all > platforms. Or maybe the single-string form is a Windowsism that doesn't apply on any other platforms. Let me go dig up my OS/2 Assembly Language Programming Reference... ChrisA From bc at freeuk.com Thu Oct 27 20:20:58 2016 From: bc at freeuk.com (BartC) Date: Fri, 28 Oct 2016 01:20:58 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 28/10/2016 01:08, Chris Angelico wrote: > On Fri, Oct 28, 2016 at 10:45 AM, BartC wrote: >> On 27/10/2016 23:31, Chris Angelico wrote: >>> >>> When >>> you exec to a process, you provide multiple arguments, not a single >>> combined string. >> >> >> Really, there could be dozens of arguments? Windows' CreateProcess() (if >> that's the same thing) has ten of which one is the command-line as a single >> string, while C's system() just has one. > > system() passes its argument through to the shell for parsing. In the > same way, Python's Popen constructor can either take a list of > strings, or a single string. > >> This might just be one of those Unixisms that doesn't apply on all >> platforms. > > Or maybe the single-string form is a Windowsism that doesn't apply on > any other platforms. Let me go dig up my OS/2 Assembly Language > Programming Reference... OK. What comes out of this is that single- or multi-string ways passing the contents of a line of input are both workable. So perhaps my way of allowing more general line-input to be read an item-at-a-time instead of as a single string isn't that off-the-wall either. -- Bartc From torriem at gmail.com Fri Oct 28 01:05:02 2016 From: torriem at gmail.com (Michael Torrie) Date: Thu, 27 Oct 2016 23:05:02 -0600 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/27/2016 04:07 AM, Terry Reedy wrote: > As I and others have said, those keyboard functions are not available on > text terminals. I predict that keyboard functions that so not work on > all systems will never become built-ins. But some are available with an > import. Sure you can't get a keyboard scancode when you're in terminal. But you can get "keystrokes" as it were, without having to read an entire line from standard in. I use editors and programs all the time which are interactive (they don't buffer keyboard input into lines) in the terminal. vim, nano, pico, mc, etc. Is this not what BartC is talking about? A way of reading in "keystrokes" in a terminal. Whether this is coming from a canned file via a pipe or a real keyboard and screen attached to a TTY doesn't really matter. There are implementations of curses for the Windows console, but I doubt they are supported by the python curses module. But at least curses provides a means of doing this for programmers in a unix environment. For mostly nostalgic reasons I play around with the FreeBASIC compiler and it tries to faithfully emulate the old key input that BASICs of yore did. Such as inkey$ (which returns nothing if no keystroke is available). It runs more or less as advertised in a terminal window on Linux, or the console on Windows. They even go so far as to emulate the old MS-DOS keycodes for things like arrow keys so that old code can still run faithfully. I think this is the sort of thing BartC would like to see in Python. It's certainly possible, even on terminals, but unlikely to happen for reasons that have been well-stated already by others. > Python optionally comes with a sophisticated keyboard api. The PSF > (python.org) CPython builds for Windows and Mac include that API. On > Windows, so is the required tcl/tk build. The premise of the subject > line, that Python does not include 'non-blocking keyboard input > functions', is not true. I would think curses would provide the needed functionality without tkinter or an X server. It's limited in that only key combinations that have control code representation can be detected, but you can definitely use arrow keys, function keys, etc. And I've seen curses interactive games so I know they can be read without blocking the whole program. From torriem at torriefamily.org Fri Oct 28 01:10:43 2016 From: torriem at torriefamily.org (Michael L Torrie) Date: Thu, 27 Oct 2016 23:10:43 -0600 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <7500adeb-236a-69c1-8726-34060f30f1f4@torriefamily.org> On 10/27/2016 11:05 PM, Michael Torrie wrote: > On 10/27/2016 04:07 AM, Terry Reedy wrote: >> As I and others have said, those keyboard functions are not available on >> text terminals. I predict that keyboard functions that so not work on >> all systems will never become built-ins. But some are available with an >> import. > Is this not what BartC is talking about? I see that BartC is wanting to detect keys like left shift and right shift. Even back in QB MS-DOS days when the program was running very close to bare metal that wasn't possible using a QB construct. From auriocus at gmx.de Fri Oct 28 01:34:33 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 28 Oct 2016 07:34:33 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 28.10.16 um 07:05 schrieb Michael Torrie: > On 10/27/2016 04:07 AM, Terry Reedy wrote: >> As I and others have said, those keyboard functions are not available on >> text terminals. I predict that keyboard functions that so not work on >> all systems will never become built-ins. But some are available with an >> import. > > Sure you can't get a keyboard scancode when you're in terminal. But you > can get "keystrokes" as it were, without having to read an entire line > from standard in. I use editors and programs all the time which are > interactive (they don't buffer keyboard input into lines) in the > terminal. vim, nano, pico, mc, etc. > > Is this not what BartC is talking about? A way of reading in > "keystrokes" in a terminal. You can do this, if you want, by setting the terminal to raw mode. On Linux and OSX you can call "stty" to do that which should not be too difficult. Then you need a second thread or switch to non-blocking I/O from stdin, maybe asyncio can do that. On Windows OTOH you have to switch to a whole different API instead of reading from a file descripor AFAIK. I still believe that it is not a "basic functionality". You need it, if you want to program a text editor or similar thing, but without using a real GUI. This is a small niche. It is both easier and more functional to use a real GUI library. Christian From marko at pacujo.net Fri Oct 28 03:33:44 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 28 Oct 2016 10:33:44 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <87poml2ahz.fsf@elektro.pacujo.net> Christian Gollwitzer : > I still believe that it is not a "basic functionality". You need it, > if you want to program a text editor or similar thing, but without > using a real GUI. This is a small niche. I disagree. It's a very large group of programs. For example, CPython's input() function puts the terminal in character mode to support emacs-style editing keys on Linux. And of course, CPython's REPL reader does it as well. CPython does that via GNU readline: > It is both easier and more functional to use a real GUI library. Well, CPython doesn't do that... Marko From auriocus at gmx.de Fri Oct 28 03:51:52 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 28 Oct 2016 09:51:52 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87poml2ahz.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> Message-ID: Am 28.10.16 um 09:33 schrieb Marko Rauhamaa: > Christian Gollwitzer : >> I still believe that it is not a "basic functionality". You need it, >> if you want to program a text editor or similar thing, but without >> using a real GUI. This is a small niche. > > I disagree. It's a very large group of programs. For example, CPython's > input() function puts the terminal in character mode to support > emacs-style editing keys on Linux. And of course, CPython's REPL reader > does it as well. > > CPython does that via GNU readline: > > You are right, readline is a very important application that uses raw terminal input. What Bart is complaining, currently you can't easily implement something like readline in Python from scratch - where you intercept all keystrokes - as opposed to using readline as it is. I am just not convinced that so many people need to implement something like readline. >> It is both easier and more functional to use a real GUI library. > > Well, CPython doesn't do that... But it doesn't intercept each keystroke, either. You could argue that readline IS a "GUI" library, and you get all the benefits from using it - it is used in multiple places, so the user is familiar with it, you don't need to fiddle with low-leve stuff like different terminal escape sequences etc. Christian From demosthenesk at gmail.com Fri Oct 28 04:05:17 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Fri, 28 Oct 2016 11:05:17 +0300 Subject: How to execute "gksudo umount VirtualDVD" In-Reply-To: References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> Message-ID: <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> Yes it was pasted wrong... def umount(self): '''unmounts VirtualDVD''' cmd = 'gksudo umount VirtualDVD' proc = subprocess.Popen(str(cmd), shell=True, stdout=subprocess.PIPE).stdout.read() print proc it fails silently.... the gksudo runs correctly. I can input the password. But the umount does nothing. I keep have mounted the VirtualDVD folder. On 10/28/2016 12:54 AM, Ian Kelly wrote: > On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis > wrote: >> I want to execute the command "gksudo umount VirtualDVD" >> >> My code is this but it fails: >> >> def umount(self): >> '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = >> subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() >> print proc > It looks like your code pasted incorrectly. > >> It pops up the gksudo dialog, and then fails. But i don't get any stdout or >> stderror. > Fails how? Is there an error? Does it hang? Does nothing happen at all? > > My initial thought is that you might want to try using > Popen.communicate instead of stdout.read in case you're getting > deadlocked. See the big red warning below > https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout From demosthenesk at gmail.com Fri Oct 28 04:47:38 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Fri, 28 Oct 2016 11:47:38 +0300 Subject: How to execute "gksudo umount VirtualDVD" In-Reply-To: <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> Message-ID: I figure it out.... umount accepts full path of mounting point. i changed my function as following and now it works. def umount(self): '''unmounts VirtualDVD''' #get virtualdvd folder home = QtCore.QDir.homePath() vpath = home + "/VirtualDVD" cmd = 'gksudo umount ' + vpath proc = subprocess.Popen(str(cmd), shell=True, stdout=subprocess.PIPE).stdout.read() print proc On 10/28/2016 11:05 AM, Demosthenes Koptsis wrote: > Yes it was pasted wrong... > > def umount(self): > '''unmounts VirtualDVD''' > cmd = 'gksudo umount VirtualDVD' > proc = subprocess.Popen(str(cmd), shell=True, > stdout=subprocess.PIPE).stdout.read() > print proc > > it fails silently.... the gksudo runs correctly. I can input the > password. > > But the umount does nothing. I keep have mounted the VirtualDVD folder. > > > On 10/28/2016 12:54 AM, Ian Kelly wrote: >> On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis >> wrote: >>> I want to execute the command "gksudo umount VirtualDVD" >>> >>> My code is this but it fails: >>> >>> def umount(self): >>> '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = >>> subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() >>> >>> print proc >> It looks like your code pasted incorrectly. >> >>> It pops up the gksudo dialog, and then fails. But i don't get any >>> stdout or >>> stderror. >> Fails how? Is there an error? Does it hang? Does nothing happen at all? >> >> My initial thought is that you might want to try using >> Popen.communicate instead of stdout.read in case you're getting >> deadlocked. See the big red warning below >> https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout >> > From marko at pacujo.net Fri Oct 28 04:59:23 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 28 Oct 2016 11:59:23 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> Message-ID: <87k2cs3l3o.fsf@elektro.pacujo.net> Christian Gollwitzer : > Am 28.10.16 um 09:33 schrieb Marko Rauhamaa: > I am just not convinced that so many people need to implement > something like readline. I don't know. How would you implement "less" in Python? How would you implement "nethack" in Python? Well, Python does offer it all. You do it like you would in C. You set the terminal mode. When in Linux, do as the Linux system programmers do. I'm glad Python hasn't tried to hide Linux under shoddy abstractions -- or at least I'm not forced to use them. Marko From auriocus at gmx.de Fri Oct 28 05:22:10 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 28 Oct 2016 11:22:10 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87k2cs3l3o.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> Message-ID: Am 28.10.16 um 10:59 schrieb Marko Rauhamaa: > Christian Gollwitzer : > >> Am 28.10.16 um 09:33 schrieb Marko Rauhamaa: >> I am just not convinced that so many people need to implement >> something like readline. > > I don't know. How would you implement "less" in Python? How would you > implement "nethack" in Python? On my system: Apfelkiste:~ chris$ otool -L /usr/bin/less /usr/bin/less: /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.0.0) So "less" in C uses ncurses. You can do the same in Python, "import curses" - instead of fiddling with terminal escape characters you leave that to a library, just like less does it. > Well, Python does offer it all. You do it like you would in C. You set > the terminal mode. > > When in Linux, do as the Linux system programmers do. I'm glad Python > hasn't tried to hide Linux under shoddy abstractions -- or at least I'm > not forced to use them. Agreed. Christian From bc at freeuk.com Fri Oct 28 06:02:23 2016 From: bc at freeuk.com (BartC) Date: Fri, 28 Oct 2016 11:02:23 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> Message-ID: On 28/10/2016 10:22, Christian Gollwitzer wrote: > Am 28.10.16 um 10:59 schrieb Marko Rauhamaa: > So "less" in C uses ncurses. You can do the same in Python, "import > curses" - instead of fiddling with terminal escape characters you leave > that to a library, just like less does it. As I commented further up the thread, I tried 'import curses' in Windows and it said it couldn't find the module. (Looking at docs.python.org: "No one has made a Windows port of the curses module".) If you intend this keyboard handling to be a very minor part of a larger application, you don't want it falling over the first time someone tries to use it. -- Bartc From marko at pacujo.net Fri Oct 28 06:30:18 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 28 Oct 2016 13:30:18 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> Message-ID: <87a8do3gw5.fsf@elektro.pacujo.net> Christian Gollwitzer : > Am 28.10.16 um 10:59 schrieb Marko Rauhamaa: >> I don't know. How would you implement "less" in Python? How would you >> implement "nethack" in Python? > > On my system: > > Apfelkiste:~ chris$ otool -L /usr/bin/less > /usr/bin/less: > /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, > current version 5.4.0) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current > version 1225.0.0) > > So "less" in C uses ncurses. On mine (Fedora 24): ======================================================================== $ ldd $(which less) linux-vdso.so.1 (0x00007ffdbbb79000) libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f8d72678000) libc.so.6 => /lib64/libc.so.6 (0x00007f8d722b6000) /lib64/ld-linux-x86-64.so.2 (0x000055e31be7e000) ======================================================================== Notably missing is: /usr/lib64/libncurses.so.6 Marko From marko at pacujo.net Fri Oct 28 06:35:08 2016 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 28 Oct 2016 13:35:08 +0300 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> Message-ID: <8760oc3go3.fsf@elektro.pacujo.net> BartC : > On 28/10/2016 10:22, Christian Gollwitzer wrote: >> So "less" in C uses ncurses. You can do the same in Python, "import >> curses" - instead of fiddling with terminal escape characters you >> leave that to a library, just like less does it. > > As I commented further up the thread, I tried 'import curses' in > Windows and it said it couldn't find the module. I can't comment on Windows. I'm only talking about Linux. Christian was talking about OSX. As for why there isn't a facility that would do it on all platforms, that question is akin to: Oh, why can't everybody just speak English? > If you intend this keyboard handling to be a very minor part of a > larger application, you don't want it falling over the first time > someone tries to use it. Didn't someone in this thread already mention a way to do it in Windows? Marko From bc at freeuk.com Fri Oct 28 06:57:44 2016 From: bc at freeuk.com (BartC) Date: Fri, 28 Oct 2016 11:57:44 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <8760oc3go3.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> <8760oc3go3.fsf@elektro.pacujo.net> Message-ID: On 28/10/2016 11:35, Marko Rauhamaa wrote: > BartC : > >> On 28/10/2016 10:22, Christian Gollwitzer wrote: >>> So "less" in C uses ncurses. You can do the same in Python, "import >>> curses" - instead of fiddling with terminal escape characters you >>> leave that to a library, just like less does it. >> >> As I commented further up the thread, I tried 'import curses' in >> Windows and it said it couldn't find the module. > > I can't comment on Windows. I'm only talking about Linux. Christian was > talking about OSX. > > As for why there isn't a facility that would do it on all platforms, > that question is akin to: Oh, why can't everybody just speak English? > >> If you intend this keyboard handling to be a very minor part of a >> larger application, you don't want it falling over the first time >> someone tries to use it. > > Didn't someone in this thread already mention a way to do it in Windows? There are a million ways to do it. But you don't want a million, you want one. Otherwise "A" will now need to create a new library which presents a common API to the application but internally takes care of the differences between platforms. But now you have an extra dependency for your app. Your own code is not a problem; you just bundle it with the app. But platform-dependent dependencies (if that makes sense) can be a problem. And "B" who wants to use the same functionality will have to find their own solution, and so on. This is exactly the sort of 're-inventing the wheel' that not including the functionality in the language was supposed to avoid! -- Bartc From steve+python at pearwood.info Fri Oct 28 07:04:27 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Fri, 28 Oct 2016 22:04:27 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <581330bd$0$1585$c3e8da3$5496439d@news.astraweb.com> On Fri, 28 Oct 2016 09:02 am, BartC wrote: > I notice that when it comes to reading command-line arguments, then > Python's sys.argv presents them as a list, not one long string. Yes, just like the shell presents it to Python. It would be silly for Python to take the list of strings it receives, join them into a single string, and then have the user split it up again. Not to mention the possible ambiguity if any of the arguments contain spaces. > And the list is just a series of strings, so needing to know whether any > parameter was a number or whatever obviously wasn't a problem. It just > makes each item into a string (actually that might be a better default > than mine). > > This is a very similar issue to reading items from a line of user input. > > So why doesn't sys.argv just return a single string if a line is so easy > to parse? I don't understand your question. Who said strings are necessarily easy to parse? Strings *might* be easy to parse if you know what the string represents ahead of time: "a sequence of integers, separated by spaces" is easy. But not all data is that simple: "a valid Perl program" is notoriously difficult. As they say, nothing but Perl can parse Perl. Likewise for C++. The shell cannot make many assumptions about the data is it passing on. All it knows is that it has received a sequence of characters. It cannot know what the receiver intends to do with them, or what those characters represent. Just because the characters are "1234" doesn't mean that they represent the integer 1234. I say *many* assumptions rather than *any* because, of course, any shell is permitted to make whatever assumptions it likes! If you don't like the shell's rules, use a different shell! But in general, shells tend towards *minimal* interpretation of arguments: spaces separate arguments unless escaped, for example, and a handful of special characters like * ? & ! are given special meaning. > (That's exactly what Windows' WinMain() function does - optional entry > point for executables under Windows. But C's main() entry point chops > the command line up into separate strings like Python. > > Also - this might some bearing on why Python does it that way - under > Linux, a parameter such as *.py is replaced by the names of ALL the > files that end in .py. That's a feature of the shell. I expect you're probably using bash, as that's the most commonly used shell, but other shells do the same. There's probably a way to turn that off, but I don't know it. If you want to pass a literal star * you need to escape it so that the shell won't treat it as a glob and expanding it: \*.py or "*.py" will probably work. (And yes, shell escaping is one of the more arcane and tricky part of Unix shell scripting.) -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tjreedy at udel.edu Fri Oct 28 08:12:00 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 28 Oct 2016 08:12:00 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/28/2016 1:05 AM, Michael Torrie wrote: > Sure you can't get a keyboard scancode when you're in terminal. As you note in your followup, Marko and Bart want to be able to respond, for instance, to left and right shift, separately and differently. Ascii terminals only send and receive ascii characters, including control 'characters'. Enhanced ('smart') terminals also receive and, I presume, send escape sequences representing editing actions Last I knew a couple of decades ago, there were no ansi code sequences for shift key presses. So responding to these requires that something in the process have access to them. > But you can get "keystrokes" as it were, without having to read an entire line > from standard in. ... > Is this not what BartC is talking about? A way of reading in > "keystrokes" in a terminal. The only specification he has given is reference to the BASIC INKEY$ variable. I don't know how consistent this was across different BASICs. I looked in Microsoft's GW-BASIC reference and it says that it returns '', 'x', or '0x'. This latter represents an extended code "described in Appendix C'. However, Appendix C only lists the standard ASCII codes 000 to 128. So I do not know what else was available and would not know from this book how to emulate GW-BASIC INKEY$. With tk, everything is available that is standard on any of the major systems. A key event includes widget name, key name (a single char for char keys, a capitalized name otherwise), keycode, and x,y pixel position of the mouse relative to the widget. A tkinter inkey() function might work as follows: #------------------------------------------------------------ """tk_inkey.py, 2016 Oct 26 (C) Terry Jan Reedy Emulate BASIC INKEY$ function/variable as Python inkey() function. """ # Create inkey function from collections import deque import tkinter as tk root = tk.Tk() def setup_inkey(widget): # Key values must be stored so inkey can access them. # If inkey were always called faster than a person can type, # a single nonlocal would suffice. However, since update() # processes all pending keys, a queue is needed. q = deque() def storekeyname(event): q.append(event.keysym) # or other key description return 'break' # swallow the event widget.bind('', storekeyname) def inkey(): widget.update() # process all pending keys return q.popleft() if q else '' return inkey inkey = setup_inkey(root) # Test inkey in simulated use. import time display = tk.Label(root, text='No Key', width=30) display.pack() root.update() while True: time.sleep(.3) # 'do some calculation' try: c = inkey() except tk.TclError: break if c: display['text'] = c #--------------------------------------------------------------- This runs *without* blocking root.mainloop(), but rather with root.update() within inkey(). So it can be used within normal code. As I said before, this could be coupled with a tk Text that emulates a console, with custom print and input functions, so it would look like and work like a console application that responds to keypresses. The use model of INKEY$ is that the programmer must branch to code blocks or functions according to the value returned. An alternative 'non-blocking' use model is that the programmer binds functions to particular events and lets the framework do the dispatching. One difference is when the function is called. A generalized version of the key handler above could triage key presses into those handled immediately, those enqueued for later handling, and those ignored. The decision whether to swallow or pass on events could be different for different events. -- Terry Jan Reedy From rosuav at gmail.com Fri Oct 28 08:28:37 2016 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 28 Oct 2016 23:28:37 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Fri, Oct 28, 2016 at 11:12 PM, Terry Reedy wrote: > The only specification he has given is reference to the BASIC INKEY$ > variable. I don't know how consistent this was across different BASICs. I > looked in Microsoft's GW-BASIC reference and it says that it returns '', > 'x', or '0x'. This latter represents an extended code "described in > Appendix C'. However, Appendix C only lists the standard ASCII codes 000 to > 128. So I do not know what else was available and would not know from this > book how to emulate GW-BASIC INKEY$. The return values from INKEY$ are CHR$(0) + "H" for up-arrow, "P" for down-arrow, and I think "K" and "L" for left and right. F1 is CHR$(0) + ";", and the next nine function keys are the subsequent ASCII characters (";<=>?@ABCD"), although F11 and F12 are different, and I don't remember what they are. The codes don't really have any meaning in ASCII - they're just the scan codes, as represented in strings. (CHR$(0) is equivalent to "\0" in saner languages.) This is from memory, but I spent a *ton* of time in BASIC in my earlier days, and there are things you never forget :) ChrisA From nomail at com.invalid Fri Oct 28 09:24:51 2016 From: nomail at com.invalid (ast) Date: Fri, 28 Oct 2016 15:24:51 +0200 Subject: __prepare__ metaclass's method Message-ID: <581351a6$0$5415$426a74cc@news.free.fr> Hi On python doc here: https://docs.python.org/3.4/reference/datamodel.html it is said about __prepare__ metaclass's method: If the metaclass has a __prepare__ attribute, it is called as namespace = metaclass.__prepare__(name, bases, **kwds) where the additional keyword arguments, if any, come from the class definition. I don't understand what they call the "class definition". So I took their example and add a print(kwds) class OrderedClass(type): @classmethod def __prepare__(metacls, name, bases, **kwds): print(kwds) return collections.OrderedDict() def __new__(cls, name, bases, namespace, **kwds): result = type.__new__(cls, name, bases, dict(namespace)) result.members = tuple(namespace) return result class A(metaclass=OrderedClass): def one(self): pass def two(self): pass def three(self): pass def four(self): pass but print(kwds) outputs an empty dictionnary {} So what kwds is supposed to contains ? Thx From auriocus at gmx.de Fri Oct 28 09:50:59 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 28 Oct 2016 15:50:59 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <87a8do3gw5.fsf@elektro.pacujo.net> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <87poml2ahz.fsf@elektro.pacujo.net> <87k2cs3l3o.fsf@elektro.pacujo.net> <87a8do3gw5.fsf@elektro.pacujo.net> Message-ID: Am 28.10.16 um 12:30 schrieb Marko Rauhamaa: > Christian Gollwitzer : > >> Am 28.10.16 um 10:59 schrieb Marko Rauhamaa: >>> I don't know. How would you implement "less" in Python? How would you >>> implement "nethack" in Python? >> >> On my system: >> >> Apfelkiste:~ chris$ otool -L /usr/bin/less >> /usr/bin/less: >> /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, >> current version 5.4.0) >> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current >> version 1225.0.0) >> >> So "less" in C uses ncurses. > > On mine (Fedora 24): > > ======================================================================== > $ ldd $(which less) > linux-vdso.so.1 (0x00007ffdbbb79000) > libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f8d72678000) > libc.so.6 => /lib64/libc.so.6 (0x00007f8d722b6000) > /lib64/ld-linux-x86-64.so.2 (0x000055e31be7e000) > ======================================================================== > > Notably missing is: /usr/lib64/libncurses.so.6 > Interesting. So your less does it in a different way than mine. I peeked into the sources, and it seems that less uses either tinfo, xcurses, ncursesw, ncurses, curses, termcap or termlib on nNix-like systems. On Windows it uses WIN32getch(). So there is no "one obvious way" to do this in C, but a large variety of options - just like in Python. Christian From __peter__ at web.de Fri Oct 28 10:09:49 2016 From: __peter__ at web.de (Peter Otten) Date: Fri, 28 Oct 2016 16:09:49 +0200 Subject: __prepare__ metaclass's method References: <581351a6$0$5415$426a74cc@news.free.fr> Message-ID: ast wrote: > Hi > > On python doc here: > > https://docs.python.org/3.4/reference/datamodel.html > > it is said about __prepare__ metaclass's method: > > If the metaclass has a __prepare__ attribute, it is called as > namespace = metaclass.__prepare__(name, bases, **kwds) > where the additional keyword arguments, if any, come from > the class definition. > > I don't understand what they call the "class definition". > > So I took their example and add a print(kwds) > > class OrderedClass(type): > > @classmethod > def __prepare__(metacls, name, bases, **kwds): > print(kwds) > return collections.OrderedDict() > > def __new__(cls, name, bases, namespace, **kwds): > result = type.__new__(cls, name, bases, dict(namespace)) > result.members = tuple(namespace) > return result > > class A(metaclass=OrderedClass): > def one(self): pass > def two(self): pass > def three(self): pass > def four(self): pass > > but print(kwds) outputs an empty dictionnary {} > > So what kwds is supposed to contains ? > > Thx >>> class T(type): ... def __new__(*args, **kw): return type.__new__(*args) ... def __prepare__(*args, **kw): ... print(kw) ... return {} ... def __init__(*args, **kw): ... pass ... >>> class A(metaclass=T, answer=42): ... pass ... {'answer': 42} Adapted from . From grant.b.edwards at gmail.com Fri Oct 28 10:28:21 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 28 Oct 2016 14:28:21 +0000 (UTC) Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 2016-10-27, BartC wrote: > On 27/10/2016 23:31, Chris Angelico wrote: >> When you exec to a process, you provide multiple arguments, not a >> single combined string. > > Really, there could be dozens of arguments? Windows' CreateProcess() (if > that's the same thing) has ten of which one is the command-line as a > single string, while C's system() just has one. He didn't say "when you CreateProcess()" or "when you system()", he said "when you exec". Take a look at the exec man page: http://man7.org/linux/man-pages/man3/exec.3.html Note that the arguments are passed as arbitrary length arrays of character pointers. > This might just be one of those Unixisms that doesn't apply on all > platforms. By using the name of a Unix system call, one might think so. -- Grant Edwards grant.b.edwards Yow! I'm a fuschia bowling at ball somewhere in Brittany gmail.com From janzen.jens at gmail.com Fri Oct 28 10:39:07 2016 From: janzen.jens at gmail.com (Jens Janzen) Date: Fri, 28 Oct 2016 07:39:07 -0700 (PDT) Subject: Installing Python on Windows 7 In-Reply-To: References: <73FE32AF-093D-4A9D-8A5C-E2DD86F34476@gmail.com> Message-ID: Am Freitag, 28. Oktober 2016 01:24:18 UTC+2 schrieb Karen Hermann: > I just downloaded Python 3.5.2 for Windows, which I installed on a Windows 7 laptop. Disclaimer up front, I am a former lapsed programmer that has been away from it and all things Windows since Windows XP. :) I?m back to being a bit of a newbie again. > > It?s a relatively clean laptop, just got it, and rarely use it, and there?s nothing else on it except Microsoft Office 2013. I downloaded Python, installed it, tried to run it, and got this message: The program can?t start because api-ms-win-crd-runtime-l1-1-0.dll is missing from your computer. Try reinstalling the program to fix this problem. > > I?ve redownloaded it and reinstalled it several times, each time deleting all traces of it before trying again. I?ve even disabled my virus-scanner (Sophos), thinking that might be interfering but that didn?t help either. > > Nothing is helping, I can?t get past this. > > Can you help please, is there something wrong with my system / setup, should I be downloading a different Python version? > > Thanks in advance for any help you can give! Hi Karen, i suggest the following steps: 1. Download C runtime Libraries from Microsoft https://support.microsoft.com/de-de/kb/2999226 for Windows 7 2. Install it 3. Try again to install Python. 4. Have fun. Greetings. JJ From nomail at com.invalid Fri Oct 28 10:55:32 2016 From: nomail at com.invalid (ast) Date: Fri, 28 Oct 2016 16:55:32 +0200 Subject: __prepare__ metaclass's method In-Reply-To: References: <581351a6$0$5415$426a74cc@news.free.fr> Message-ID: <581366ea$0$24777$426a74cc@news.free.fr> "Peter Otten" <__peter__ at web.de> a ?crit dans le message de news:mailman.34.1477663877.31204.python-list at python.org... > ast wrote: > > >>>> class T(type): > ... def __new__(*args, **kw): return type.__new__(*args) > ... def __prepare__(*args, **kw): > ... print(kw) > ... return {} > ... def __init__(*args, **kw): > ... pass > ... >>>> class A(metaclass=T, answer=42): > ... pass > ... > {'answer': 42} > > Adapted from > . > Thanks, Could these class-level keyword arguments be useful for standard classes (those whose metaclass is type) ? eg: class Test(option = True) make use of "option" here ? it seems no From rosuav at gmail.com Fri Oct 28 11:03:47 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 29 Oct 2016 02:03:47 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 29, 2016 at 1:54 AM, Dennis Lee Bieber wrote: > https://en.wikibooks.org/wiki/QBasic/Advanced_Input > indicates that INKEY$ doesn't even test the keyboard for presses, but is > only retrieving the next item from the keyboard input /buffer/ (which that > article goes on to mention used to be part of a dedicated hardware chip, > but now is a software/interrupt maintained buffer). True, but in practice, it's the same thing. The keyboard buffer ring gets populated by the interrupt handler, and INKEY$ looks at the buffer, sees if it's empty (return ""), and if not, consumes and returns one key. > And it defines the extended return value as the "scan code" for the > keyboard. Which would just be a number assigned to the key position, and > might (back in those days) have had different values going from machine to > machine. Most of the modifier keys (shift, control) in those days did not > produce their own codes. Yes, but who ever writes programs that run on multiple different computers? Just hard-code those scan codes everywhere, it'll be fine! ChrisA born in those lackadaisical days when "Extended ASCII" was a single thing, because the rest of the world didn't exist From torriem at gmail.com Fri Oct 28 11:16:48 2016 From: torriem at gmail.com (Michael Torrie) Date: Fri, 28 Oct 2016 09:16:48 -0600 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <596185bf-4dbc-3453-1762-66edaa43b051@gmail.com> On 10/28/2016 06:28 AM, Chris Angelico wrote: > On Fri, Oct 28, 2016 at 11:12 PM, Terry Reedy wrote: >> The only specification he has given is reference to the BASIC INKEY$ >> variable. I don't know how consistent this was across different BASICs. I >> looked in Microsoft's GW-BASIC reference and it says that it returns '', >> 'x', or '0x'. This latter represents an extended code "described in >> Appendix C'. However, Appendix C only lists the standard ASCII codes 000 to >> 128. So I do not know what else was available and would not know from this >> book how to emulate GW-BASIC INKEY$. > > The return values from INKEY$ are CHR$(0) + "H" for up-arrow, "P" for > down-arrow, and I think "K" and "L" for left and right. F1 is CHR$(0) > + ";", and the next nine function keys are the subsequent ASCII > characters (";<=>?@ABCD"), although F11 and F12 are different, and I > don't remember what they are. The codes don't really have any meaning > in ASCII - they're just the scan codes, as represented in strings. > (CHR$(0) is equivalent to "\0" in saner languages.) > > This is from memory, but I spent a *ton* of time in BASIC in my > earlier days, and there are things you never forget :) And FreeBASIC faithfully emulates this for all its supported platforms, terminals or graphics mode (fakes a screen buffer in a window). I'm sure someone could make a python module that offered inkey() on any platform in terminals and consoles based on raw input, or using win32 console api calls. But I am not sure it belongs in the standard library. Nor am I sure what codes should be returned. From bc at freeuk.com Fri Oct 28 11:28:00 2016 From: bc at freeuk.com (BartC) Date: Fri, 28 Oct 2016 16:28:00 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 28/10/2016 15:28, Grant Edwards wrote: > On 2016-10-27, BartC wrote: >> On 27/10/2016 23:31, Chris Angelico wrote: > >>> When you exec to a process, you provide multiple arguments, not a >>> single combined string. >> >> Really, there could be dozens of arguments? Windows' CreateProcess() (if >> that's the same thing) has ten of which one is the command-line as a >> single string, while C's system() just has one. > > He didn't say "when you CreateProcess()" or "when you system()", he > said "when you exec". Take a look at the exec man page: > > http://man7.org/linux/man-pages/man3/exec.3.html > > Note that the arguments are passed as arbitrary length arrays of > character pointers. > >> This might just be one of those Unixisms that doesn't apply on all >> platforms. > > By using the name of a Unix system call, one might think so. Python is cross-platform, isn't it? -- bartc From eryksun at gmail.com Fri Oct 28 12:12:06 2016 From: eryksun at gmail.com (eryk sun) Date: Fri, 28 Oct 2016 16:12:06 +0000 Subject: Installing Python on Windows 7 In-Reply-To: References: <73FE32AF-093D-4A9D-8A5C-E2DD86F34476@gmail.com> Message-ID: On Fri, Oct 28, 2016 at 2:39 PM, Jens Janzen wrote: > 1. Download C runtime Libraries from Microsoft > https://support.microsoft.com/de-de/kb/2999226 > for Windows 7 The more recent update is here [1], but don't use this. Use Windows Update to get the latest updates. [1]: https://support.microsoft.com/en-us/kb/3118401 From tjreedy at udel.edu Fri Oct 28 13:50:01 2016 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 28 Oct 2016 13:50:01 -0400 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 10/28/2016 11:03 AM, Chris Angelico wrote: > On Sat, Oct 29, 2016 at 1:54 AM, Dennis Lee Bieber > wrote: >> https://en.wikibooks.org/wiki/QBasic/Advanced_Input >> indicates that INKEY$ doesn't even test the keyboard for presses, but is >> only retrieving the next item from the keyboard input /buffer/ (which that >> article goes on to mention used to be part of a dedicated hardware chip, >> but now is a software/interrupt maintained buffer). > > True, but in practice, it's the same thing. The keyboard buffer ring > gets populated by the interrupt handler, and INKEY$ looks at the > buffer, sees if it's empty (return ""), and if not, consumes and > returns one key. 'ring bugger' implies a finite limit tot he size of the queue. Do either of you know what it is? >> And it defines the extended return value as the "scan code" for the >> keyboard. Which would just be a number assigned to the key position, and >> might (back in those days) have had different values going from machine to >> machine. Most of the modifier keys (shift, control) in those days did not >> produce their own codes. If I were to write inkey() for my own use or as part of a tk console module, I would stick with returning either the character or a mostly machine and OS independent name. -- Terry Jan Reedy From rosuav at gmail.com Fri Oct 28 13:55:57 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 29 Oct 2016 04:55:57 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 29, 2016 at 4:50 AM, Terry Reedy wrote: > On 10/28/2016 11:03 AM, Chris Angelico wrote: >> >> On Sat, Oct 29, 2016 at 1:54 AM, Dennis Lee Bieber >> wrote: >>> >>> https://en.wikibooks.org/wiki/QBasic/Advanced_Input >>> indicates that INKEY$ doesn't even test the keyboard for presses, but is >>> only retrieving the next item from the keyboard input /buffer/ (which >>> that >>> article goes on to mention used to be part of a dedicated hardware chip, >>> but now is a software/interrupt maintained buffer). >> >> >> True, but in practice, it's the same thing. The keyboard buffer ring >> gets populated by the interrupt handler, and INKEY$ looks at the >> buffer, sees if it's empty (return ""), and if not, consumes and >> returns one key. > > > 'ring bugger' implies a finite limit tot he size of the queue. Do either of > you know what it is? > 16 slots, each 16 bits (8 bits character, 8 bits scan code). I think it filled at 15, though, from memory; if the head and tail pointers are equal, the buffer is empty, ergo the buffer cannot completely fill. That's where the classic "bip" of keyboard-buffer-full came from. ChrisA From gordon at panix.com Fri Oct 28 17:24:11 2016 From: gordon at panix.com (John Gordon) Date: Fri, 28 Oct 2016 21:24:11 +0000 (UTC) Subject: After import, some submodules are accessible and some aren't Message-ID: After importing a module, I can access some of its submodules directly but others require an explicit import of the submodule. As an example, see ldap.dn and ldap.modlist: % python Python 2.7.8 (default, Aug 4 2016, 09:29:33) [GCC 4.4.7 20120313 (Red Hat 4.4.7-9)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ldap >>> ldap.__version__ '2.4.19' >>> ldap.modlist Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'modlist' >>> ldap.dn >>> import ldap.modlist >>> ldap.modlist Why the difference? -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From rosuav at gmail.com Fri Oct 28 17:46:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 29 Oct 2016 08:46:07 +1100 Subject: After import, some submodules are accessible and some aren't In-Reply-To: References: Message-ID: On Sat, Oct 29, 2016 at 8:24 AM, John Gordon wrote: > After importing a module, I can access some of its submodules directly > but others require an explicit import of the submodule. > > As an example, see ldap.dn and ldap.modlist: > > % python > Python 2.7.8 (default, Aug 4 2016, 09:29:33) > [GCC 4.4.7 20120313 (Red Hat 4.4.7-9)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import ldap >>>> ldap.__version__ > '2.4.19' >>>> ldap.modlist > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'module' object has no attribute 'modlist' >>>> ldap.dn > >>>> import ldap.modlist >>>> ldap.modlist > > > Why the difference? A package can leave some of its modules for lazy loading. By default, only the package itself gets loaded (making available any names in __init__.py itself, but no modules); if the package wishes to eagerly load some of the modules, it can import them itself ("from . import dn"). It's probably something along the lines of ldap.dn being critical but ldap.modlist being optional - you'd have to check the docs or ask the author to find out exactly why. ChrisA From best_lay at yahoo.com Fri Oct 28 18:19:17 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 28 Oct 2016 17:19:17 -0500 Subject: How to execute "gksudo umount VirtualDVD" References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> Message-ID: <2_GdnUt3nsx4U47FnZ2dnUU7-amdnZ2d@giganews.com> On Fri, 28 Oct 2016 11:05:17 +0300, Demosthenes Koptsis wrote: > Yes it was pasted wrong... > > def umount(self): > '''unmounts VirtualDVD''' > cmd = 'gksudo umount VirtualDVD' > proc = subprocess.Popen(str(cmd), shell=True, > stdout=subprocess.PIPE).stdout.read() > print proc > > it fails silently.... the gksudo runs correctly. I can input the password. > > But the umount does nothing. I keep have mounted the VirtualDVD folder. > > > On 10/28/2016 12:54 AM, Ian Kelly wrote: >> On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis >> wrote: >>> I want to execute the command "gksudo umount VirtualDVD" >>> >>> My code is this but it fails: >>> >>> def umount(self): >>> '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = >>> subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() >>> print proc >> It looks like your code pasted incorrectly. >> >>> It pops up the gksudo dialog, and then fails. But i don't get any stdout or >>> stderror. >> Fails how? Is there an error? Does it hang? Does nothing happen at all? >> >> My initial thought is that you might want to try using >> Popen.communicate instead of stdout.read in case you're getting >> deadlocked. See the big red warning below >> https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout Try this: def umount(self): '''unmounts VirtualDVD''' cmd = ["gksudo", "umount", VirtualDVD p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) proc = p.communicate() print proc -- GNU/Linux user #557453 The cow died so I don't need your bull! From best_lay at yahoo.com Fri Oct 28 18:22:23 2016 From: best_lay at yahoo.com (Wildman) Date: Fri, 28 Oct 2016 17:22:23 -0500 Subject: How to execute "gksudo umount VirtualDVD" References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> <2_GdnUt3nsx4U47FnZ2dnUU7-amdnZ2d@giganews.com> Message-ID: <2_GdnUp3nswCUo7FnZ2dnUU7-amdnZ2d@giganews.com> On Fri, 28 Oct 2016 17:19:17 -0500, Wildman wrote: > On Fri, 28 Oct 2016 11:05:17 +0300, Demosthenes Koptsis wrote: > >> Yes it was pasted wrong... >> >> def umount(self): >> '''unmounts VirtualDVD''' >> cmd = 'gksudo umount VirtualDVD' >> proc = subprocess.Popen(str(cmd), shell=True, >> stdout=subprocess.PIPE).stdout.read() >> print proc >> >> it fails silently.... the gksudo runs correctly. I can input the password. >> >> But the umount does nothing. I keep have mounted the VirtualDVD folder. >> >> >> On 10/28/2016 12:54 AM, Ian Kelly wrote: >>> On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis >>> wrote: >>>> I want to execute the command "gksudo umount VirtualDVD" >>>> >>>> My code is this but it fails: >>>> >>>> def umount(self): >>>> '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = >>>> subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() >>>> print proc >>> It looks like your code pasted incorrectly. >>> >>>> It pops up the gksudo dialog, and then fails. But i don't get any stdout or >>>> stderror. >>> Fails how? Is there an error? Does it hang? Does nothing happen at all? >>> >>> My initial thought is that you might want to try using >>> Popen.communicate instead of stdout.read in case you're getting >>> deadlocked. See the big red warning below >>> https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout > > Try this: > > def umount(self): > '''unmounts VirtualDVD''' > cmd = ["gksudo", "umount", VirtualDVD] ^ > p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) > proc = p.communicate() > print proc Oops! -- GNU/Linux user #557453 The cow died so I don't need your bull! From steve+python at pearwood.info Fri Oct 28 20:13:02 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 29 Oct 2016 11:13:02 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5813e991$0$22141$c3e8da3$5496439d@news.astraweb.com> On Sat, 29 Oct 2016 02:03 am, Chris Angelico wrote: > born in those lackadaisical days when "Extended ASCII" was a single > thing, because the rest of the world didn't exist I know you're being sarcastic, but "extended ASCII" *never* was a single thing, even for Americans who ignored the rest of the world. You had IBM PC extended ASCII, Apple Macintosh extended ASCII, Hewlett-Packard extended ASCII, Atari extended ASCII ("ATASCII"), Commodore extended ASCII ("PETSCII"), and more. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 28 20:24:24 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 29 Oct 2016 11:24:24 +1100 Subject: After import, some submodules are accessible and some aren't References: Message-ID: <5813ec3a$0$1617$c3e8da3$5496439d@news.astraweb.com> On Sat, 29 Oct 2016 08:24 am, John Gordon wrote: > After importing a module, I can access some of its submodules directly > but others require an explicit import of the submodule. [...] > Why the difference? Ask the author of the package. Try this experiment: create a package called "imptest" with this directory structure: imptest/ +-- __init__.py +-- spam.py +-- eggs.py That is, a single directory called "imptest", containing three files "__init__.py" (that's TWO leading and trailing underscores), "spam.py" and "eggs.py". spam and eggs can remain blank. Inside __init__.py put: import imptest.spam print(spam) Save and close the file, and then launch Python. Try: import imptest imptest.spam # this should work imptest.eggs # this should fail import imptest.eggs imptest.eggs # this should now work -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Fri Oct 28 20:32:08 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 29 Oct 2016 11:32:08 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5813ee0a$0$1619$c3e8da3$5496439d@news.astraweb.com> On Sat, 29 Oct 2016 02:28 am, BartC wrote: >> By using the name of a Unix system call, one might think so. > > Python is cross-platform, isn't it? The os module isn't. It is specifically designed for OS-specific functions. There's lots of OS-specific functionality in Python. In general, Python won't make any promises that the OS doesn't, nor will it generally offer any feature that the OS doesn't. In particular, system calls are only available on platforms that provide that system call. (Duh.) So no Windows system calls on Unix, and no Unix system calls on Windows. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From eryksun at gmail.com Fri Oct 28 20:41:14 2016 From: eryksun at gmail.com (eryk sun) Date: Sat, 29 Oct 2016 00:41:14 +0000 Subject: Doubled backslashes in Windows paths In-Reply-To: References: Message-ID: On Fri, Oct 28, 2016 at 8:04 PM, Gilmeh Serda wrote: > > You can use forward slash to avoid the messy problem. There are cases in which you need to use backslash, such as extended paths and command lines. Python 3's pathlib automatically normalizes a Windows path to use backslash. Otherwise you can use os.path.normpath(). >>>> target_dir = 'Desktop/2B_proc' >>>> full_target = os.path.join(os.path.expanduser('~'), target_dir) Don't assume the default location of a user's known folders. They can all be relocated, either by domain group policy or individually using the folder properties. Instead call SHGetKnownFolderPath or SHGetFolderPath, e.g. to look up the value of FOLDERID_Desktop or CSIDL_DESKTOP, respectively. expanduser('~') is also used to locate configuration and data files, such as "~\.python_history". On Windows, such files belong in a subfolder of the user's hidden AppData folder. If they should roam with a roaming profile, use %AppData%; otherwise use %LocalAppData%, such as "%LocalAppData%\Python\python_history.txt". I prefer calling SHGetKnownFolderPath instead of using the potentially stale environment variables, but relocating these folders in a session is uncommon. From steve+python at pearwood.info Fri Oct 28 21:04:22 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sat, 29 Oct 2016 12:04:22 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> On Fri, 28 Oct 2016 05:09 am, BartC wrote: > And I've seen things like 'import msvcrt', 'import winapi' in Python > code, and then there's all that stuff with ctypes. Right. Not everything needs to be a feature of the language itself, especially if it is operating system dependent and can be delegated to the OS. >> You could always check the source code to the Python debugger. Presumably >> it already solves this. There has to be *some* solution. After all, >> operating systems, text editors, shells, GUI frameworks, etc. all support >> this. > > Yes, all those kinds of application need these basic features. Isn't it > odd then that not a single mainstream language provides them as standard? Not really. I don't see this is as a useful language feature. > (From my point of view the discussion has been about any key-at-a-time > events rather than blocking or not. Those are also problematical.) Key-at-a-time events are solved. I've already linked to a working solution for that, which supports both Windows and Unix/Linux. Its not in the base language because its not as useful as you insist, and those who need it have alternatives. Not everything needs to be in the base language. >>> I'm not familiar with that many languages. However if you google for >>> ' non-blocking keyboard' then it seems quite a few people are >>> interested in the feature! >> >> *shrug* My very first response suggested that many people ask for this >> feature, but then after writing code to implement it, they find they >> don't actually need it. Maybe I'm wrong. But if this is so useful, >> outside of specialist areas, why do so few languages support it? > > For years I've had discussions in comp.lang.c about things that C should > or should not have. What usually happens is that someone comes up with > some crude workaround using macros, so there is less need to have some > feature built-in (and the language fails to a acquire a slick new > enhancement). Bart, don't be naive. The C language isn't going to "acquire a slick new enhancement" based on a few emails on compl.lang.c. C is an ISO-standard. Stability of the language, the fact that you have a known set of functionality, is precisely why it was made a standard in the first place: to discourage the sort of "wouldn't it be cool if..." pile up of features and bloat and ever-changing language specs. You are *incredibly* privileged to do all your work on your own custom programming languages, where you get to choose what platforms to support ("whatever PC I am using this year") and have a user-base of exactly one user ("me"). > But with the usual trouble that everyone then has to re-invent the same > macro but in a different way to everyone else. > > The same thing probably happens with a low-level keyboard API. There are > half a dozen ways of achieving the same functionality (with varying > degrees of hassle), so maybe the language doesn't need to provide a > standard solution after all. > > You talk about not wanting to re-invent things, but that's exactly what > everyone ends up doing! So what? That sounds harsh, and it is intended to. Screw 'em. Who cares if a dozen people have to write their own three line function to do something? The Python core developers are not a magic genie that you get to snap your fingers and they will do your coding for you. They are (for the most part, with one or two part-time exceptions) volunteers who take time out of their busy lives to work on Python. Time that they might otherwise be spending with their families or making a living. Have you seen the bug tracker? They don't have either the time or manpower to keep up with the number of enhancements and bug fixes requested. Any new feature has to be weighed up by these volunteers: - Nobody is paying me to do this. Do I care about this feature enough to implement it when I could work on something more important instead? - If I add this new feature, will people actually use it? Or is this something that sounds important but in practice will be used by hardly anyone? Am I wasting my time? - Even useful features have cost. They add to the cost of the Python compiler and standard library: more code, more bugs, more documentation, more tests, more features to learn, more complex code base, and larger downloads. Is it worth the cost? The size of Python isn't important to *you* on your fancy PC with entire gigabytes of hard drive, but to people interested in running Python on embedded hardware and micro-computers, every kilobyte counts. - And more important than the physical size is the size and shape of the learning curve, both to use Python and to maintain it. Every new feature makes the language bigger and raises the barrier to entry for new users before they can say they have mastered the language. So no, not every feature belongs in the core language, or even in the standard library. >>> print "Enter 3 numbers: " >>> readln a,b,c >> >> How is the interpreter supposed to know that a, b, c are numbers? What >> sort of numbers? 16-bit integers, 80-bit floats, Bignums, complex, >> Fractions, or something else? > >> But in a dynamically typed language, the compiler has no idea what you >> expect a, b and c to be. So it returns text, and you can convert it >> yourself. > > So you tell it what to expect. The above example is from a dynamic > language, and will assume integers unless told otherwise. This: > > readln a:"h", b:"r", c:"s" > > reads a hex integer, floating point number and a string respectively > (with a default set of separators delimiting the string, or "..." can be > used). Or values can be read one at a time: > > readln > read a:"h" Here's an alternative spelling: readh a readr b reads c Or if you prefer more Pythonic syntax: a = read_hexint() b = read_float() c = read_string() And we're back to what started with. > I think this came up when someone wanted to switch from Visual Basic to > Python. The above is not a very sophisticated approach but it is very > simple and intuitive. Intuitive to whom? To me, it just looks weird. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From rosuav at gmail.com Fri Oct 28 22:55:41 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 29 Oct 2016 13:55:41 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5813e991$0$22141$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5813e991$0$22141$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sat, Oct 29, 2016 at 11:13 AM, Steve D'Aprano wrote: > On Sat, 29 Oct 2016 02:03 am, Chris Angelico wrote: > >> born in those lackadaisical days when "Extended ASCII" was a single >> thing, because the rest of the world didn't exist > > I know you're being sarcastic, but "extended ASCII" *never* was a single > thing, even for Americans who ignored the rest of the world. You had IBM PC > extended ASCII, Apple Macintosh extended ASCII, Hewlett-Packard extended > ASCII, Atari extended ASCII ("ATASCII"), Commodore extended ASCII > ("PETSCII"), and more. > I know; the non-rest-of-world mentality is a consequence of writing code solely for an IBM PC, as well as solely for your locale. We used what is today called codepage 437 - why would anyone need anything else? I guess, in theory, people who speak Greek or Russian would need to use something else. But then they miss out on all these awesome box drawing characters, and.... bah, that doesn't matter anyway, does it. Took the rise of the web before I, as a programmer and as a person, got a proper appreciation for i18n. And even then it was slow going. Way better for someone to learn the right way first. ChrisA From hanzer at riseup.net Fri Oct 28 23:02:12 2016 From: hanzer at riseup.net (Adam Jensen) Date: Fri, 28 Oct 2016 23:02:12 -0400 Subject: distributed development methodology Message-ID: If one were to develop a Python application on multiple machines, what are some good methods for keeping them synchronized? For example, I develop on a FreeBSD machine and a CentOS machine, each with python2.7 and differing sets of site packages. On each machine, I can use virtualenv. But if I 'pip install xxx' on one, should I just try to remember to do the same on the other? I am not sure about the details but syncing the entire virtualenv directory with a DVCS seems like it might be problematic. (I prefer https://www.fossil-scm.org). So what are some of the more successful distributed. multi-platform, development models? From cs at zip.com.au Fri Oct 28 23:59:42 2016 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 29 Oct 2016 14:59:42 +1100 Subject: distributed development methodology In-Reply-To: References: Message-ID: <20161029035942.GA30229@cskk.homeip.net> On 28Oct2016 23:02, Adam Jensen wrote: >If one were to develop a Python application on multiple machines, what >are some good methods for keeping them synchronized? For example, I >develop on a FreeBSD machine and a CentOS machine, each with python2.7 >and differing sets of site packages. On each machine, I can use >virtualenv. But if I 'pip install xxx' on one, should I just try to >remember to do the same on the other? I am not sure about the details >but syncing the entire virtualenv directory with a DVCS seems like it >might be problematic. (I prefer https://www.fossil-scm.org). > >So what are some of the more successful distributed. multi-platform, >development models? Sync the virtualenv prerequisites file with your DVCS. Have a tiny script to update the local virtualenv prereq file and run its update command to honour any new prereqs. Cheers, Cameron Simpson From hanzer at riseup.net Sat Oct 29 00:11:12 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 29 Oct 2016 00:11:12 -0400 Subject: distributed development methodology In-Reply-To: <20161029035942.GA30229@cskk.homeip.net> References: <20161029035942.GA30229@cskk.homeip.net> Message-ID: <392149d2-f079-a884-66c1-98c6e10950d7@riseup.net> On 10/28/2016 11:59 PM, Cameron Simpson wrote: > Sync the virtualenv prerequisites file with your DVCS. Have a tiny > script to update the local virtualenv prereq file and run its update > command to honour any new prereqs. Cool. I didn't mention that I am a python n00b, did I? What/where is the "virtualenv prerequisites file" and what would the "update command" look like? :) From hanzer at riseup.net Sat Oct 29 00:31:48 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 29 Oct 2016 00:31:48 -0400 Subject: distributed development methodology In-Reply-To: References: <20161029035942.GA30229@cskk.homeip.net> Message-ID: On 10/28/2016 11:59 PM, Cameron Simpson wrote: > Sync the virtualenv prerequisites file with your DVCS. Have a tiny > script to update the local virtualenv prereq file and run its update > command to honour any new prereqs. Cool. I didn't mention that I am a python n00b, did I? What/where is the "virtualenv prerequisites file" and what would the "update command" look like? :) From demosthenesk at gmail.com Sat Oct 29 00:56:54 2016 From: demosthenesk at gmail.com (Demosthenes Koptsis) Date: Sat, 29 Oct 2016 07:56:54 +0300 Subject: How to execute "gksudo umount VirtualDVD" In-Reply-To: <2_GdnUp3nswCUo7FnZ2dnUU7-amdnZ2d@giganews.com> References: <52f88dea-2172-310b-be0d-5f403108a35e@gmail.com> <3ad94911-fbb5-7ced-1432-a02d5582e2a5@gmail.com> <2_GdnUt3nsx4U47FnZ2dnUU7-amdnZ2d@giganews.com> <2_GdnUp3nswCUo7FnZ2dnUU7-amdnZ2d@giganews.com> Message-ID: <7cfe5337-c021-9f23-545c-d33d40b4f418@gmail.com> The code is ok, i need to provide the absolute path to umount. def umount(self): '''unmounts VirtualDVD''' #get virtualdvd folder home = QtCore.QDir.homePath() vpath = home + "/VirtualDVD" cmd = 'gksudo umount ' + vpath subprocess.Popen(str(cmd), shell=True, stdout=subprocess.PIPE) This is strange, in terminal "gksudo umount VirtualDVD" worked fine but not from code... From code i have to "gksudo umount /home/user/VirtualDVD" Thanks all for your help. Regards, Dim On 10/29/2016 01:22 AM, Wildman via Python-list wrote: > On Fri, 28 Oct 2016 17:19:17 -0500, Wildman wrote: > >> On Fri, 28 Oct 2016 11:05:17 +0300, Demosthenes Koptsis wrote: >> >>> Yes it was pasted wrong... >>> >>> def umount(self): >>> '''unmounts VirtualDVD''' >>> cmd = 'gksudo umount VirtualDVD' >>> proc = subprocess.Popen(str(cmd), shell=True, >>> stdout=subprocess.PIPE).stdout.read() >>> print proc >>> >>> it fails silently.... the gksudo runs correctly. I can input the password. >>> >>> But the umount does nothing. I keep have mounted the VirtualDVD folder. >>> >>> >>> On 10/28/2016 12:54 AM, Ian Kelly wrote: >>>> On Thu, Oct 27, 2016 at 3:30 PM, Demosthenes Koptsis >>>> wrote: >>>>> I want to execute the command "gksudo umount VirtualDVD" >>>>> >>>>> My code is this but it fails: >>>>> >>>>> def umount(self): >>>>> '''unmounts VirtualDVD''' cmd ='gksudo umount VirtualDVD' proc = >>>>> subprocess.Popen(str(cmd),shell=True,stdout=subprocess.PIPE).stdout.read() >>>>> print proc >>>> It looks like your code pasted incorrectly. >>>> >>>>> It pops up the gksudo dialog, and then fails. But i don't get any stdout or >>>>> stderror. >>>> Fails how? Is there an error? Does it hang? Does nothing happen at all? >>>> >>>> My initial thought is that you might want to try using >>>> Popen.communicate instead of stdout.read in case you're getting >>>> deadlocked. See the big red warning below >>>> https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stdout >> Try this: >> >> def umount(self): >> '''unmounts VirtualDVD''' >> cmd = ["gksudo", "umount", VirtualDVD] > ^ > > >> p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) >> proc = p.communicate() >> print proc > Oops! > From rosuav at gmail.com Sat Oct 29 02:30:52 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 29 Oct 2016 17:30:52 +1100 Subject: distributed development methodology In-Reply-To: <20161029035942.GA30229@cskk.homeip.net> References: <20161029035942.GA30229@cskk.homeip.net> Message-ID: On Sat, Oct 29, 2016 at 2:59 PM, Cameron Simpson wrote: > On 28Oct2016 23:02, Adam Jensen wrote: >> >> If one were to develop a Python application on multiple machines, what >> are some good methods for keeping them synchronized? For example, I >> develop on a FreeBSD machine and a CentOS machine, each with python2.7 >> and differing sets of site packages. On each machine, I can use >> virtualenv. But if I 'pip install xxx' on one, should I just try to >> remember to do the same on the other? I am not sure about the details >> but syncing the entire virtualenv directory with a DVCS seems like it >> might be problematic. (I prefer https://www.fossil-scm.org). >> >> So what are some of the more successful distributed. multi-platform, >> development models? > > > Sync the virtualenv prerequisites file with your DVCS. Have a tiny script to > update the local virtualenv prereq file and run its update command to honour > any new prereqs. Easiest way that I've found is to use requirements.txt and "pip install -r requirements.txt". And as Cameron says, source control is the key, but you don't don't need to manage the entire venv, just the one key file. ChrisA From cs at zip.com.au Sat Oct 29 02:51:41 2016 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 29 Oct 2016 17:51:41 +1100 Subject: distributed development methodology In-Reply-To: <392149d2-f079-a884-66c1-98c6e10950d7@riseup.net> References: <392149d2-f079-a884-66c1-98c6e10950d7@riseup.net> Message-ID: <20161029065141.GA80059@cskk.homeip.net> On 29Oct2016 00:11, Adam Jensen wrote: >On 10/28/2016 11:59 PM, Cameron Simpson wrote: >> Sync the virtualenv prerequisites file with your DVCS. Have a tiny >> script to update the local virtualenv prereq file and run its update >> command to honour any new prereqs. > >Cool. I didn't mention that I am a python n00b, did I? What/where is the >"virtualenv prerequisites file" and what would the "update command" look >like? :) Have a read of this: https://www.dabapps.com/blog/introduction-to-pip-and-virtualenv-python/ and then this: https://pip.pypa.io/en/latest/user_guide/#requirements-files Then come back with specific questions :-) Cheers, Cameron Simpson From no.email at nospam.invalid Sat Oct 29 03:50:13 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Sat, 29 Oct 2016 00:50:13 -0700 Subject: distributed development methodology References: Message-ID: <878tt7mw5m.fsf@nightsong.com> Adam Jensen writes: > So what are some of the more successful distributed. multi-platform, > development models? Use an orchestration program to keep the systems in sync: I use ansible (ansible.com) which is written in Python and fairly simple once you get used to it, but there are lots of other programs in that space and everyone has their own preferences. If stuff doesn't change too often, you could also use Docker or whatever the current hotness is to make a container image and deploy it to your fleet. From vek.m1234 at gmail.com Sat Oct 29 04:17:57 2016 From: vek.m1234 at gmail.com (Veek M) Date: Sat, 29 Oct 2016 13:47:57 +0530 Subject: Need help in python program References: <2144eabe-bf8f-4fa8-833d-e41b0efdc932@googlegroups.com> Message-ID: id_1, clk, val = foo_function() id_2, key, units, delay = bar_function() if id_1 == id_2: print id_1, clk, val, key, units, delay From hanzer at riseup.net Sat Oct 29 05:06:13 2016 From: hanzer at riseup.net (Adam Jensen) Date: Sat, 29 Oct 2016 05:06:13 -0400 Subject: distributed development methodology In-Reply-To: References: <20161029035942.GA30229@cskk.homeip.net> Message-ID: On 10/29/2016 12:31 AM, Adam Jensen wrote: > On 10/28/2016 11:59 PM, Cameron Simpson wrote: >> Sync the virtualenv prerequisites file with your DVCS. Have a tiny >> script to update the local virtualenv prereq file and run its update >> command to honour any new prereqs. > > Cool. I didn't mention that I am a python n00b, did I? What/where is the > "virtualenv prerequisites file" and what would the "update command" look > like? :) > For posterity: pip freeze > requirements.txt pip install -r requirements.txt https://pip.pypa.io/en/latest/user_guide/#requirements-files From bc at freeuk.com Sat Oct 29 07:53:35 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 12:53:35 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/10/2016 02:04, Steve D'Aprano wrote: > On Fri, 28 Oct 2016 05:09 am, BartC wrote: >> For years I've had discussions in comp.lang.c about things that C should >> or should not have. > Bart, don't be naive. The C language isn't going to "acquire a slick new > enhancement" based on a few emails on compl.lang.c. C is an ISO-standard. > Stability of the language, C is a supposedly ultra-portable and apparently simple language. But take an application like CPython - you can't just grab the nearest C compiler and build it. You need to blindly run tens of thousands of lines of scripts and assorted utilities, and compile with /gcc/, and hope it works. And that's in Linux; on Windows it's different yet again. And take C compilers such as gcc and MSVC - they're monsters (how long would it take either of them to build itself? Now see below.) The language and the way it's used have a few problems - why shouldn't someone talk about them? the fact that you have a known set of > functionality, is precisely why it was made a standard in the first place: > to discourage the sort of "wouldn't it be cool if..." pile up of features > and bloat and ever-changing language specs. Most of the enhancements I talked about were trivial. The language is unlikely to change because of me but it would be nice for someone to acknowledge them rather than defend one of C's crazy quirks or lack of a feature to the death. (I use my own implementation language in place of C, which fixes most of those annoyances. Yesterday I was able to compile it entirely from scratch in 0.016 seconds. And I haven't misplaced a decimal point! More info: http://pastebin.com/yFKzs2eF) > You are *incredibly* privileged to do all your work on your own custom > programming languages, These days anyone (any coder) can do that if they wish. Apparently most people don't. I don't blame them. > The size of Python isn't important to *you* on your fancy PC with entire > gigabytes of hard drive, but to people interested in running Python on > embedded hardware and micro-computers, every kilobyte counts. The subject is about a feature that was available on any 1980s microcomputer with memory measured in KB. Anyway I know about small systems, and the ones I create are pretty small (they will still fit on a floppy disk), but they will no longer run on tiny systems. But I'd like to see Python running on a 64KB system (Micropython doesn't count!). >> I think this came up when someone wanted to switch from Visual Basic to >> Python. The above is not a very sophisticated approach but it is very >> simple and intuitive. > > Intuitive to whom? To me, it just looks weird. 'read a,b,c' is weird and unintuitive compared with its counterpart 'print a,b,c'. OK.... BTW what does reading three integers from the user look like in Python? -- Bartc From breamoreboy at gmail.com Sat Oct 29 09:48:10 2016 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 29 Oct 2016 06:48:10 -0700 (PDT) Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <580efe10$0$11101$c3e8da3@news.astraweb.com> Message-ID: <4bd7a5ad-d83a-4402-a015-6030be3fd5a1@googlegroups.com> On Tuesday, October 25, 2016 at 11:02:47 AM UTC+1, BartC wrote: > On 25/10/2016 07:39, Steven D'Aprano wrote: > > >> I gather that non-blocking keyboard input functions aren't the easiest thing > >> to implement. They seem to depend on the operating system. Still, ease of > >> use is a primary goal of Python, and the need for this feature must be > >> common. > > > > > > Not really. I think that lots of people think they need it, but once they write > > a little utility, they often realise that it's not that useful. > > > If you (generic you, not you specifically) are telling the user "press any key > > to continue", then you probably shouldn't. *Any* key may not do anything. E.g. > > if the user hits the Shift key. A much better interface is to specify a > > specific key, and ignore anything else... in which case, why not specify the > > Enter key? > > > > raw_input('Press the Enter key to continue... ') > > Which doesn't work on Python 3. So even here, making it easy by using > line-input, it's not so straightforward. > Just get this http://pythonhosted.org/six/ or similar. From steve+python at pearwood.info Sat Oct 29 10:19:20 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 30 Oct 2016 01:19:20 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5814afe9$0$22141$c3e8da3$5496439d@news.astraweb.com> On Sat, 29 Oct 2016 10:53 pm, BartC wrote: > On 29/10/2016 02:04, Steve D'Aprano wrote: >> On Fri, 28 Oct 2016 05:09 am, BartC wrote: > >>> For years I've had discussions in comp.lang.c about things that C should >>> or should not have. > >> Bart, don't be naive. The C language isn't going to "acquire a slick new >> enhancement" based on a few emails on compl.lang.c. C is an ISO-standard. >> Stability of the language, > > C is a supposedly ultra-portable and apparently simple language. I'm not sure if C *ever* was simple, but it certainly hasn't been simple for the last quarter of a century. The most recent version of the standard has a draft of 701 pages: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf The draft for the previous standard, C99, was 552 pages: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf See also: http://www.open-std.org/jtc1/sc22/wg14/www/standards http://stackoverflow.com/questions/81656/where-do-i-find-the-current-c-or-c-standard-documents Here's a draft of ANSI C (C89): http://flash-gordon.me.uk/ansi.c.txt My browser formats that to 290 pages. So even in 1989, C was significantly complex. [...] > The language and the way it's used have a few problems - why shouldn't > someone talk about them? Nobody is stopping you from discussing C's problems in the appropriate forum. But unlike you and your hand-made compilers, with a user-base of exactly 1 user (you), there are dozens of implementations of C, with tens of thousands of users or more, and an ISO standard defining what is and isn't C. > the fact that you have a known set of >> functionality, is precisely why it was made a standard in the first >> place: to discourage the sort of "wouldn't it be cool if..." pile up of >> features and bloat and ever-changing language specs. > > Most of the enhancements I talked about were trivial. The language is > unlikely to change because of me but it would be nice for someone to > acknowledge them rather than defend one of C's crazy quirks or lack of a > feature to the death. You won't get any defence of C's crazy quirks from me. [...] > But I'd like to see Python running on a 64KB system > (Micropython doesn't count!). Hmmm. So tell me... how do you expect Python to run on tiny systems by *adding* new features? Regardless of how "small" this feature is, it is not negative size. The implementation is at least 1 byte. The documentation is at least 1 byte. The tests are at least 1 byte. If you think Python is too big now, then adding this feature will make it at least 3 bytes bigger. >>> I think this came up when someone wanted to switch from Visual Basic to >>> Python. The above is not a very sophisticated approach but it is very >>> simple and intuitive. >> >> Intuitive to whom? To me, it just looks weird. > > 'read a,b,c' is weird and unintuitive compared with its counterpart > 'print a,b,c'. OK.... print as a statement was always an anomaly, and yes, it got pretty weird: print >>sys.stderr, spam, eggs So much nicer now that it is a function that takes proper keyword arguments. In any case, I wasn't specifically talking about read a, b, c Yes, it is a bit (actually a lot) unusual to assign a value to a variable by way of a statement other than assignment, it is not unprecedented in Python. We have at least three other binding statements: import name for name in ... del name even though one of them is actually an *unbinding* statement. What I said looked weird was your syntax: read a:"h", b:"r", c:"s" If you're going to introduce special syntax for types, why make the types strings rather than symbols? > BTW what does reading three integers from the user look like in Python? for i in range(3): n = int(input("")) You want a prompt? Change the empty string to your prompt. You want to assign to three different variables? Unroll the loop: n = int(input("")) p = int(input("")) q = int(input("")) If the user types something that isn't an integer, you want to try again? Write a loop. If you're sensible, of course you will put it into a helper function: _SENTINEL = object() def read_int_with_all_the_trimmings( prompt1='Enter an int: ', prompt2='Invalid value for an int; please try again: ', prompt3='Value out of range; please try again: ', max_times=2**31, low=None, high=None, default=_SENTINEL): msg = prompt1 for attempt in range(max_times): s = input(msg) try: n = int(s) except ValueError: msg = prompt2 else: out_of_range = (low is not None and n < low) or ( high is not None and n > high) if out_of_range: msg = prompt3 else: return n if default is _SENTINEL: raise ValueError('not a valid int') return default Does your `read` statement do all that? Why not? -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Sat Oct 29 10:32:16 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 15:32:16 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/10/2016 14:51, Dennis Lee Bieber wrote: > On Sat, 29 Oct 2016 12:53:35 +0100, BartC declaimed the > following: > >> >> BTW what does reading three integers from the user look like in Python? > > On one line, or on three lines? > > (Python 2.7) > > ln = raw_input("Enter three integers separated by spaces: ") > ints = [] > for i, wd in enumerate(ln.split()): > try: > anInt = int(wd) > except: #yes, I know -- the horrible bare except clause > print ("***** item %s: '%s' could not be converted to integer" > % (i, wd) ) > anInt = None > ints.append(anInt) Yes, that's the sort of thing I was expecting, that's why I asked. I still think a beginner would much prefer something along the lines of 'readln a,b,c' (and I still think that's more intuitive). (The first programming exercises I ever did involved typing in integers from the user, and sorting them or working out if they made a triangle or whatever. But in Python someone would first need to master for-loops, enumeration, string-processing, numeric conversion, exceptions and, your second example, functions. That's quite a steep learning curve! (And that second example starts to involve re-inventing things.) (BTW the functionality of my 'readln a,b,c' differs from the above. Separators can be anything reasonable. When eol is encountered, it will read zeros. And errors are not handled: any non-numeric will yield zero. When reading name, file and string items rather than integers, the split method doesn't deal with embedded quotes. Probably there are bigger guns you can bring out to deal with more elaborate input, but it starts to get further away from beginner level.) -- Bartc From rosuav at gmail.com Sat Oct 29 10:53:22 2016 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 30 Oct 2016 01:53:22 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: On Sun, Oct 30, 2016 at 1:32 AM, BartC wrote: > BTW the functionality of my 'readln a,b,c' differs from the above. > Separators can be anything reasonable. When eol is encountered, it will read > zeros. And errors are not handled: any non-numeric will yield zero. People will disagree as to what is "correct behaviour" in that kind of situation. If someone disagrees with Steve's function, s/he can write a different one. If someone disagrees with your language primitive.... s/he can write a different language? ChrisA From lepto.python at gmail.com Sat Oct 29 10:55:58 2016 From: lepto.python at gmail.com (oyster) Date: Sat, 29 Oct 2016 22:55:58 +0800 Subject: what does type(subprocess.Popen)== mean? Message-ID: why does not type(subprocess.Popen)==? Thanks [quote] Python 3.4.4 |Anaconda 2.3.0 (64-bit)| (default, Feb 16 2016, 09:54:04) [MSC v.1 600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> type(subprocess.Popen) >>> import os >>> type(os.path) [/quote] From mal at europython.eu Sat Oct 29 11:16:02 2016 From: mal at europython.eu (M.-A. Lemburg) Date: Sat, 29 Oct 2016 17:16:02 +0200 Subject: EuroPython 2017 will be held in Rimini, Italy Message-ID: <5814BD32.4060209@europython.eu> After carefully reviewing all proposals we had received and intense discussions with the teams, the EuroPython Society (EPS) is happy to announce the decision to accept the proposal from the Italian on-site team, backed by the Python Italia APS, to hold EuroPython 2017 in Rimini, Italy. The EPS would like to thank all teams who have entered bids for our Call for Interest (CFI): * Python Italia APS: Milan/Como/Genoa/Rimini, Italy * The local Czech Python community: Brno, the Czech Republic * Python San Sebastian Society (ACPySS): Bilbao, Basque Country, Spain The conference will be held at the Rimini PalaCongressi in July 2017. The exact dates are still subject to negotiations with the venue. We?ll announce them as soon as they are finalized. Until then, here?s the official EuroPython 2017 URL for you to bookmark, where we?ll open up the website in January 2017: *** EuroPython 2017 Pre-launch Website *** http://ep2017.europython.eu/ Sponsoring EuroPython --------------------- Companies who would like to signup as EuroPython 2017 sponsor are encouraged to contact the sponsor workgroup at sponsoring at europython.eu. Until we have the 2017 sponsor brochure in place, please have a look at our 2016 brochure: https://ep2016.europython.eu/media/conference/sponsor/brochure/ep2016_sponsor_brochure.pdf We will be preparing the launch of the website in January 2017. If you?d like to sign up early as launch sponsor, please contact us in the next two months. As with the past conferences, we will try to make EuroPython 2017 as effective as possible for sponsors by offering more booth space and sponsors slots than ever before. This is your chance to reach out to more than 1.100 enthusiastic and highly motivated EuroPython attendees ! EuroPython Workgroups --------------------- Organizing a EuroPython event is a lot of work and with the workgroup concept, we have opened up much of the organization for remote participation. If you want to help, please apply for one or more workgroups which you feel match your interests and experience. If you?d like to help, but don?t have enough experience, yet are willing to learn, please apply as well. The application process is described on our workgroups page: http://www.europython-society.org/workgroups Enjoy, -- EuroPython Society http://www.europython-society.org/ From steve+python at pearwood.info Sat Oct 29 11:22:40 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 30 Oct 2016 02:22:40 +1100 Subject: what does type(subprocess.Popen)== mean? References: Message-ID: <5814bec2$0$1607$c3e8da3$5496439d@news.astraweb.com> On Sun, 30 Oct 2016 01:55 am, oyster wrote: > why does not type(subprocess.Popen)==? Thanks Because Popen is not a module. Why do you expect it to be a module? py> type(int) py> type(float) py> type(str) py> class X(object): ... pass ... py> type(X) >>>> type(subprocess.Popen) > Popen is a class, also known as a "type". >>>> import os >>>> type(os.path) > os.path is a module, just like os. Inside the os module, you will find code that looks something like this: if the operating system is Windows: import ntpath as path elif the operating system is Unix: import posixpath as path else: import genericpath as path So os.path is a module. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From steve+python at pearwood.info Sat Oct 29 11:24:15 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Sun, 30 Oct 2016 02:24:15 +1100 Subject: Why doesn't Python include non-blocking keyboard input function? References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: <5814bf20$0$1607$c3e8da3$5496439d@news.astraweb.com> On Sun, 30 Oct 2016 01:32 am, BartC wrote: > (BTW the functionality of my 'readln a,b,c' differs from the above. > Separators can be anything reasonable. When eol is encountered, it will > read zeros. And errors are not handled: any non-numeric will yield zero. Ah, in other words it is a toy, utterly unsuitable for serious use by anyone who cares about data validity and error checking, only suitable for teaching bad habits to beginners. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From bc at freeuk.com Sat Oct 29 11:24:53 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 16:24:53 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5814afe9$0$22141$c3e8da3$5496439d@news.astraweb.com> References: <580efe10$0$11101$c3e8da3@news.astraweb.com> <5810009c$0$1598$c3e8da3$5496439d@news.astraweb.com> <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> <5814afe9$0$22141$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/10/2016 15:19, Steve D'Aprano wrote: > On Sat, 29 Oct 2016 10:53 pm, BartC wrote: >> But I'd like to see Python running on a 64KB system >> (Micropython doesn't count!). > > Hmmm. So tell me... how do you expect Python to run on tiny systems by > *adding* new features? > > Regardless of how "small" this feature is, it is not negative size. The > implementation is at least 1 byte. The documentation is at least 1 byte. > The tests are at least 1 byte. If you think Python is too big now, then > adding this feature will make it at least 3 bytes bigger. The /effective/ size could well be negative! If someone really needs the feature they might need to drag in 'curses' or require 'tkinter', substantial add-ons even though they only need one minor feature. > print as a statement was always an anomaly, and yes, it got pretty weird: > > print >>sys.stderr, spam, eggs > So much nicer now that it is a function that takes proper keyword arguments. I guess ">>" is what I used "#" or "@" for. But what does it look like in function form? Functions are great for adding general-purpose utility but their syntax is limited; sometimes dedicated syntax is better for things you use all the time. > What I said > looked weird was your syntax: > > read a:"h", b:"r", c:"s" > > > If you're going to introduce special syntax for types, why make the types > strings rather than symbols? (I can't remember where that came from. The colon syntax I think was from Pascal. I used strings for the Print counterpart as there can be other stuff in there: x:"z8hs'" might display x as "0001'E800". Read formats could have been single unquoted characters I suppose (they were once). But a string means the format can be dynamic: read a:fmt,b:fmt. But this is a detail. The main thing is trivially just listing the things you want to read in a similar manner to print.) >> BTW what does reading three integers from the user look like in Python? > > for i in range(3): > n = int(input("")) OK, one per line. Just so each input line corresponds to exactly one integer! (I can do this too but I write it as: n := strtoval(sreadln()) # int default x := strtoval(sreadln(),"r") # floating point ('real') ) > You want a prompt? Change the empty string to your prompt. > > You want to assign to three different variables? Unroll the loop: > > n = int(input("")) > p = int(input("")) > q = int(input("")) > > If the user types something that isn't an integer, you want to try again? The idea of read and print statements, in my languages at least, are for simple or throwaway programs, or when dealing with known file formats: r := "r" readln @file, x:r, y:r, z:r It's not /meant/ to be sophisticated. Just convenient. > Write a loop. If you're sensible, of course you will put it into a helper > function: > > > _SENTINEL = object() > > def read_int_with_all_the_trimmings( > prompt1='Enter an int: ', > prompt2='Invalid value for an int; please try again: ', > prompt3='Value out of range; please try again: ', > max_times=2**31, > low=None, > high=None, > default=_SENTINEL): > msg = prompt1 > for attempt in range(max_times): > s = input(msg) > try: > n = int(s) > except ValueError: > msg = prompt2 > else: > out_of_range = (low is not None and n < low) or ( > high is not None and n > high) > if out_of_range: > msg = prompt3 > else: > return n > if default is _SENTINEL: > raise ValueError('not a valid int') > return default > > Does your `read` statement do all that? Why not? I don't understand your point. No, it doesn't, but neither does Python (it doesn't have 'read' for a start). You've had to write it; I can too. And that example still reads one number per line! Suppose you have to read a ppm file which might start like this: P6 600 800 255 I can read this using 'read': (checking and comment-skipping code omitted): readln @f, sig:"s" readln @f, width, height readln @f, maxpix It doesn't need anything fancy and you don't want to bother with parsing the input lines and converting text to numbers. -- Bartc From auriocus at gmx.de Sat Oct 29 11:26:55 2016 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sat, 29 Oct 2016 17:26:55 +0200 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: Am 29.10.16 um 16:32 schrieb BartC: > I still think a beginner would much prefer something along the lines of > 'readln a,b,c' (and I still think that's more intuitive). > > (The first programming exercises I ever did involved typing in integers > from the user, and sorting them or working out if they made a triangle > or whatever. Yes, and that's exactly why they learn to produce such horrible interfaces which do not work for a serious program. Instead, teach them how to use the *environment* for I/O. That is, as a beginner put your function into a file, e.g. def is_triangle(a, b, c): if abs(a)+abs(b) > abs(c): print "Triangle!" else: print "Not a triangle" ..and then use IPython: %run triangle.py >>> is_triangle(1,1.7, 2) Triangle! >>> is_triangle(1,1.7, 8) Not a triangle >>> (I know the prog is incomplete, but that's not the point) This way you can concentrate on the algorithm and leave the I/O thing to python. Due to the built-in readline, you can recall previous arguments, edit them when you make mistakes, save the result (if there were any), .... much more than your readln function can accomplish. Christian From bc at freeuk.com Sat Oct 29 11:32:07 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 16:32:07 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: <5814bf20$0$1607$c3e8da3$5496439d@news.astraweb.com> References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> <5814bf20$0$1607$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/10/2016 16:24, Steve D'Aprano wrote: > On Sun, 30 Oct 2016 01:32 am, BartC wrote: > >> (BTW the functionality of my 'readln a,b,c' differs from the above. >> Separators can be anything reasonable. When eol is encountered, it will >> read zeros. And errors are not handled: any non-numeric will yield zero. > > Ah, in other words it is a toy, utterly unsuitable for serious use by anyone > who cares about data validity and error checking, only suitable for > teaching bad habits to beginners. So, how does Python differ? From what I've seen, you have to /write/ the validation code (as I said in my last post). ANY language can do that too! And the eol-handling behaviour is deliberate. If you want to read UP TO three numbers but only two are entered on a line, some languages will keep pestering you until you've entered that third number. -- Bartc From bc at freeuk.com Sat Oct 29 11:41:43 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 16:41:43 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> Message-ID: On 29/10/2016 15:53, Chris Angelico wrote: > On Sun, Oct 30, 2016 at 1:32 AM, BartC wrote: >> BTW the functionality of my 'readln a,b,c' differs from the above. >> Separators can be anything reasonable. When eol is encountered, it will read >> zeros. And errors are not handled: any non-numeric will yield zero. > > People will disagree as to what is "correct behaviour" in that kind of > situation. If someone disagrees with Steve's function, s/he can write > a different one. If someone disagrees with your language primitive.... > s/he can write a different language? No, they use a function similar to Steve's. Or just tweak the code: read an item as a string instead of a number. Then its validity can checked. This sounds like another one of those arguments: there's no point in having a simple feature A, because it doesn't do X, Y and Z. So we won't have it. But do we instead have feature B that includes X, Y and Z? No, of course not! No matter that 90% of the time, X, Y and Z are never used... -- Bartc From dishaacharya96 at gmail.com Sat Oct 29 15:38:20 2016 From: dishaacharya96 at gmail.com (dishaacharya96 at gmail.com) Date: Sat, 29 Oct 2016 12:38:20 -0700 (PDT) Subject: comapring 2 sequences of DNA ouput the silent and non mutations Message-ID: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> Code: A = 0 B= 0 i=0 j=0 # opening the files infile1 = open("CDSsrebf1.txt") infile2 = open("PROsrebf1.txt") infile3 = open("mutant.txt") print(" 1st line of WT SREBF1 (CDS):",infile1.readline()) print ("1st line of mutant protein of SREBF1: ", infile3.readline()) print ("1st line of protein of SREBF1: ",infile2.readline()) # ----------------------------------------------------- # reading the nucleotide sequence for WT SREBF1 seq1 = infile1.read() seq1 = seq1.replace('\n', '') len1 = len(seq1) # -------------------------------------------- # reading the mutant file mutant = infile3.read() mutant = mutant.replace('\n', '') #--------------------------------------- # reading the protein file # which is used to check our codon dictionary wtPRO = infile2.read() wtPRO = wtPRO.replace('\n', '') #--------------------------------------------------------- # setting up the dictionary letters = ('G', 'A', 'C', 'T') codes = [] for a in letters : for b in letters : for c in letters : codes.append(a + b + c) aa = 'ggggeeddaaaavvvvrrsskknnttttmiiirrrrqqhhppppllllwxccxxyyssssllff' aa = aa.upper() codons = {} for i in range(64) : codons[codes[i]] = aa[i] #------------------------------------------------------------------ # making the protein from the WT SREBF1, which is seq1 protein = '' for i in range(0, len(seq1), 3) : codon = seq1[i:i+3] aminoacid = codons[codon] protein += aminoacid # ----------------------------------------------------------- # making the protein from the mutant SREBF1, which is mutant mutantPRO = '' for i in range(0, len(mutant), 3) : codon = mutant[i:i+3] aminoacid = codons[codon] mutantPRO += aminoacid # ---------------------------------------------------------- # quick check if WT and mutant are the same for the protein if protein == mutantPRO : print ('The protein sequences are the same.') else : print ('The protein sequences are different.') # -------------------------------------------------------- # Printing the differences in the format XiY # which means WT amino acid X at position i changed to mutant amino acid Y print ('-------------------------') print ('The mutations are:') for i in range (len(protein) & len(seq1)) : if protein[i] != mutantPRO[i] : print (protein[i] + str(i) + mutantPRO[i]) A+= 1 else: if seq1[i:i+3] != mutant[i:i+3]: print(protein[i] + str(i) + mutantPRO[i] +' Silent mutation ') print(seq1[i:i+3] + mutant[i:i+3]) B+= 1 print("Number of non-silent mutations are: ",A) print("Number of silent mutations are: " , B) output should be The mutations are: M0I D1D silent mutation C5T V291L I dont know what to print the C5T part Thank you for helping me! From bc at freeuk.com Sat Oct 29 17:11:31 2016 From: bc at freeuk.com (BartC) Date: Sat, 29 Oct 2016 22:11:31 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5811a410$0$1586$c3e8da3$5496439d@news.astraweb.com> <5811e7f4$0$1603$c3e8da3$5496439d@news.astraweb.com> <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> <5ve91chsgqgikh6ifletbpirfdrhpb761p@4ax.com> Message-ID: On 29/10/2016 17:27, Dennis Lee Bieber wrote: > On Sat, 29 Oct 2016 15:32:16 +0100, BartC declaimed the > following: > > >> I still think a beginner would much prefer something along the lines of >> 'readln a,b,c' (and I still think that's more intuitive). >> > Then I would suggest using something like REXX or a K&K-based BASIC (I > have no idea, of the top of my head, what Visual BASIC uses). > > In REXX, everything is considered a string until it needs to be a > numeric type. And old BASIC would require one to put a data type code on > the variable name which would control the input parsing. > > -=-=-=-=- > /* */ > say "Enter three integers" > > parse pull a b c That could be workable if it was line-oriented. I strongly suspect it isn't, because C isn't, and C seems to have lot of influence. (Non-line-oriented would mean it just keeps sitting there until it's read three values, damn it, no matter how many times you press Enter, and it's not going to shift until it has them! In other words, user-unfriendly. This is related to my big bug-bear with most text editors where keys such as left, right, backspace and delete don't stop at the ends of a line but keep going to previous or next lines.) > The parse command has lots of fancy options allowing for defining fixed > column positions > > parse pull 0 a 5 b 8 c > parse pull 5 a 0 b "5" 4 c "9" That's easy string processing, you don't really need i/o system support: get the whole line as a string then slice it for the various fields. > For many years, FORTRAN couldn't even handle variable width input from > the keyboard. > READ(5, 1100, END=50, ERR=100) A, B, C > 1100 FORMAT("I8, X, I8, X, I8") I spent a year writing interactive Fortran IV, I'm fairly sure it was possible but I couldn't tell you how. > Ada.Text_IO.Put ("Enter three integers => "); > > Ada.Integer_Text_IO.Get (A); > Ada.Integer_Text_IO.Get (B); > Ada.Integer_Text_IO.Get (C); I admire Ada in many ways, but this ... >> When reading name, file and string items rather than integers, the split >> method doesn't deal with embedded quotes. Probably there are bigger guns >> you can bring out to deal with more elaborate input, but it starts to >> get further away from beginner level.) > > Typically, if doing interactive input, you ask for ONE item at a time, > and accept the entire line as the response. You can't really draw the line like that. This sort of input can used for live input from a keyboard, from an input file, or a input file redirected so that it appears to come from the keyboard. So embedded quotes aren't a > problem. OR you do your own parsing on the line if multiple items are > expected, only reading more from the console when you've run out of data. > > Ad hoc parsing of user input is going to be difficult in any language. But you wouldn't use this for that. I'd use these 'readln' methods for reading formatted text files; I showed a PPM header example; for text-mode images, 'read' can also be used. Then there are text configuration files, or project description files, or files contain lists of 3D data, or comma- or tab-separated data files, simple command scripts.... There is a huge range of possible uses for a crude read/readln feature where verification at the i/o level is not really necessary. (If the width, image values of a PPM out of range, then you check that at the application level.) Anything that is not line-oriented, or that needs very precise parsing such as code, would just use a different approach. -- Bartc From python at mrabarnett.plus.com Sat Oct 29 19:48:38 2016 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 30 Oct 2016 00:48:38 +0100 Subject: comapring 2 sequences of DNA ouput the silent and non mutations In-Reply-To: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> References: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> Message-ID: <41ecbaf2-30a7-7c1c-d62a-09546964a77e@mrabarnett.plus.com> On 2016-10-29 20:38, dishaacharya96 at gmail.com wrote: > Code: > > A = 0 > B= 0 > i=0 > j=0 > # opening the files > infile1 = open("CDSsrebf1.txt") > infile2 = open("PROsrebf1.txt") > infile3 = open("mutant.txt") > print(" 1st line of WT SREBF1 (CDS):",infile1.readline()) > print ("1st line of mutant protein of SREBF1: ", infile3.readline()) > print ("1st line of protein of SREBF1: ",infile2.readline()) > # ----------------------------------------------------- > # reading the nucleotide sequence for WT SREBF1 > seq1 = infile1.read() > seq1 = seq1.replace('\n', '') > len1 = len(seq1) > # -------------------------------------------- > # reading the mutant file > mutant = infile3.read() > mutant = mutant.replace('\n', '') > #--------------------------------------- > # reading the protein file > # which is used to check our codon dictionary > wtPRO = infile2.read() > wtPRO = wtPRO.replace('\n', '') > #--------------------------------------------------------- > # setting up the dictionary > letters = ('G', 'A', 'C', 'T') > codes = [] > for a in letters : > for b in letters : > for c in letters : > codes.append(a + b + c) > aa = 'ggggeeddaaaavvvvrrsskknnttttmiiirrrrqqhhppppllllwxccxxyyssssllff' > aa = aa.upper() > codons = {} > for i in range(64) : > codons[codes[i]] = aa[i] > #------------------------------------------------------------------ > # making the protein from the WT SREBF1, which is seq1 > protein = '' > for i in range(0, len(seq1), 3) : > codon = seq1[i:i+3] > aminoacid = codons[codon] > protein += aminoacid > # ----------------------------------------------------------- > # making the protein from the mutant SREBF1, which is mutant > mutantPRO = '' > for i in range(0, len(mutant), 3) : > codon = mutant[i:i+3] > aminoacid = codons[codon] > mutantPRO += aminoacid > # ---------------------------------------------------------- > # quick check if WT and mutant are the same for the protein > if protein == mutantPRO : > print ('The protein sequences are the same.') > else : > print ('The protein sequences are different.') > # -------------------------------------------------------- > # Printing the differences in the format XiY > # which means WT amino acid X at position i changed to mutant amino acid Y > print ('-------------------------') > print ('The mutations are:') > > for i in range (len(protein) & len(seq1)) : > > if protein[i] != mutantPRO[i] : > print (protein[i] + str(i) + mutantPRO[i]) > A+= 1 > else: > if seq1[i:i+3] != mutant[i:i+3]: > print(protein[i] + str(i) + mutantPRO[i] +' Silent mutation ') > print(seq1[i:i+3] + mutant[i:i+3]) > B+= 1 > > > print("Number of non-silent mutations are: ",A) > print("Number of silent mutations are: " , B) > > > output > > should be The mutations are: > M0I > D1D silent mutation C5T > V291L > > > I dont know what to print the C5T part > > Thank you for helping me! > I don't understand what the expression "len(protein) & len(seq1)" is supposed to be doing. It's a bitwise 'and' of the lengths, which just looks wrong! From bc at freeuk.com Sat Oct 29 20:16:51 2016 From: bc at freeuk.com (BartC) Date: Sun, 30 Oct 2016 01:16:51 +0100 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> <5ve91chsgqgikh6ifletbpirfdrhpb761p@4ax.com> <9j7a1cllv1l34uooaq9l2pstlidc8a40s4@4ax.com> Message-ID: On 29/10/2016 23:21, Dennis Lee Bieber wrote: > On Sat, 29 Oct 2016 22:11:31 +0100, BartC declaimed the > following: >> (Non-line-oriented would mean it just keeps sitting there until it's >> read three values, damn it, no matter how many times you press Enter, >> and it's not going to shift until it has them! In other words, >> user-unfriendly. > > There are those of us who, if we were given a prompt that said enter > three values, would want it to take three values regardless of how many > blank lines you enter. Such behaviour can be programmed if it is useful for the input to be free-format where N items of input can be spread over any number of lines. But it's not a good idea where line ends are meaningful (eg. marking the end of some input), or it is necessary to stay synchronised with line beginnings. An example of such input is the command prompt in console and terminal windows. If I want to enter type 'python program.py' then it will end badly if I enter is as: >> python program.py on two lines. >> This is related to my big bug-bear with most text editors where keys >> such as left, right, backspace and delete don't stop at the ends of a >> line but keep going to previous or next lines.) > > > DEC SOS died off in the early 80s. But VI might make you happy... Not > sure about VIM (VI IMproved). An editor is either line-oriented or it isn't. Free-flowing English text usually isn't, but most program code is. And a line-oriented editor should have hard stops at the line ends. (IMO which apparently isn't shared by anyone else on the planet.) (Not sure what the reference to SOS means. That was a TTY editor which didn't have navigation that I recall, while backspace and delete didn't really work on printed display.) -- Bartc From subbiahvishal at gmail.com Sat Oct 29 20:49:48 2016 From: subbiahvishal at gmail.com (Vishal Subbiah) Date: Sat, 29 Oct 2016 17:49:48 -0700 Subject: pip3 : command not found Message-ID: Hi, So I wads trying to install some packages for python3. when I run pip3 in terminal i receive the error "pip3 : command not found". When looking at the path, I noticed /usr/local/Cellar/python3/3.5.2_3 does not have pip3 in the directory while /usr/local/Cellar/python3/3.5.2_2 does have. I am guessing this is the issue. I tried using the get-pip.py and that didnt work either. When I search for pip I find pip, pip2 and pip2.7 all of which are for python 2.7. I tried uninstalling python3 and reinstalling but that didnt work either. I am running this on Mac OS Sierra. Please let me know how I can fix this. Looking forward to your response. Regards, Vishal Subbiah Institute for Computational and Mathematical Engineering Masters Student Stanford University From torriem at gmail.com Sat Oct 29 20:51:59 2016 From: torriem at gmail.com (Michael Torrie) Date: Sat, 29 Oct 2016 18:51:59 -0600 Subject: Why doesn't Python include non-blocking keyboard input function? In-Reply-To: References: <5812279a$0$1585$c3e8da3$5496439d@news.astraweb.com> <5813f597$0$1600$c3e8da3$5496439d@news.astraweb.com> <5ve91chsgqgikh6ifletbpirfdrhpb761p@4ax.com> <9j7a1cllv1l34uooaq9l2pstlidc8a40s4@4ax.com> Message-ID: On 10/29/2016 06:16 PM, BartC wrote: > An editor is either line-oriented or it isn't. Free-flowing English text > usually isn't, but most program code is. And a line-oriented editor > should have hard stops at the line ends. (IMO which apparently isn't > shared by anyone else on the planet.) ViM appears to be line-oriented, which makes sense given its ed heritage. Cursor movement will only get you to the end of the line and won't wrap around to the next line. So it seems that ViM works much as you prefer, except that you probably don't like the two modes in ViM. From john_ladasky at sbcglobal.net Sun Oct 30 00:36:16 2016 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Sat, 29 Oct 2016 21:36:16 -0700 (PDT) Subject: comapring 2 sequences of DNA ouput the silent and non mutations In-Reply-To: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> References: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> Message-ID: <09ad14e8-0c79-4ed7-b03f-71b4b8f73130@googlegroups.com> Disha, Before you struggle to reinvent the wheel, you might want to check out the Biopython package. http://biopython.org/wiki/Biopython I haven't used it for a few years, but the version that I did use was very comprehensive. From ethan at stoneleaf.us Sun Oct 30 01:34:16 2016 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 29 Oct 2016 22:34:16 -0700 Subject: modifying images in a pdf file Message-ID: <58158658.2080506@stoneleaf.us> I'd like to modify some images in pdf files -- specifically, extract the image, process it to give it a comic strip type of coloring (using pillow), and then put it back. I see there are several pdf libraries, but a cursory examination did not reveal which, if any, had a method for extraction and replacement of embedded images. Has anyone had experience with this, or can point me in the right direction? Thanks! -- ~Ethan~ From maxischmeii at gmail.com Sun Oct 30 05:01:30 2016 From: maxischmeii at gmail.com (Maxime S) Date: Sun, 30 Oct 2016 10:01:30 +0100 Subject: comapring 2 sequences of DNA ouput the silent and non mutations In-Reply-To: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> References: <5e01ecd0-561b-4ac9-bb86-a390a742917d@googlegroups.com> Message-ID: 2016-10-29 21:38 GMT+02:00 : > > Code: > > [...] > > for i in range (len(protein) & len(seq1)) : > > if protein[i] != mutantPRO[i] : > print (protein[i] + str(i) + mutantPRO[i]) > A+= 1 > else: > if seq1[i:i+3] != mutant[i:i+3]: > print(protein[i] + str(i) + mutantPRO[i] +' Silent mutation ') > print(seq1[i:i+3] + mutant[i:i+3]) > B+= 1 Hi, The problem here is that you try to mix two different index in one variable. Instead, you need to do something like this: #i index protein #j index DNA for i in range (len(protein)) : j = i*3 if protein[i] != mutantPRO[i] : print (protein[i] + str(i) + mutantPRO[i]) A+= 1 else: if seq1[j:j+3] != mutant[j:j+3]: print(protein[i] + str(i) + mutantPRO[i] +' Silent mutation ') print(seq1[j:j+3] + mutant[j:j+3]) B+=1 From ben+python at benfinney.id.au Sun Oct 30 20:37:50 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 31 Oct 2016 11:37:50 +1100 Subject: pip3 : command not found References: Message-ID: <85zill9wv5.fsf@benfinney.id.au> Vishal Subbiah writes: > So I wads trying to install some packages for python3. when I run pip3 > in terminal There is no guarantee that a command named ?pip3? will be installed. Instead, you should invoke the exact Python interpreter you want ? and, by extension, the Python environment into which you want packages installed. $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum If you already have a specific environment active and know that ?python3? is the correct Python interpreter from that environment, you can omit the explicit path. $ python3 -m pip install LoremIpsum ---- Why doesn't a command ?pip? or ?pip3? do the job? Because there's no guarantee that will use the specific Python environment you want. For many programs, it simply doesn't matter which Python interpreter ? or even *whether* a Python interpreter or Perl interpreter or etc. ? is the one that runs. So most Python-implemented programs you don't need to specify which interpreter; they know how to find it themselves, and your choice of a different environment should not affect their operation. But for a command that has the primary purpose of interacting with that environment ? by installing or removing packages, as Pip does ? it must obey your explicit instruction on which Python environment and interpreter to use. -- \ ?It ain't so much the things we don't know that get us in | `\ trouble. It's the things we know that ain't so.? ?Artemus Ward | _o__) (1834?1867), U.S. journalist | Ben Finney From subbiahvishal at gmail.com Sun Oct 30 21:15:52 2016 From: subbiahvishal at gmail.com (Vishal Subbiah) Date: Sun, 30 Oct 2016 18:15:52 -0700 Subject: pip3 : command not found In-Reply-To: <85zill9wv5.fsf@benfinney.id.au> References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: Hi, Thanks Ben for the response. I followed your suggestion and this is what i get: "python3 -m pip3 install LoremIpsum /usr/local/opt/python3/bin/python3.5: No module named pip3" "python3 -m pip install LoremIpsum /usr/local/opt/python3/bin/python3.5: No module named pip" How do I fix this? Regards, Vishal Subbiah Institute for Computational and Mathematical Engineering Masters Student Stanford University On Sun, Oct 30, 2016 at 5:37 PM, Ben Finney wrote: > Vishal Subbiah writes: > > > So I wads trying to install some packages for python3. when I run pip3 > > in terminal > > There is no guarantee that a command named ?pip3? will be installed. > > Instead, you should invoke the exact Python interpreter you want ? and, > by extension, the Python environment into which you want packages > installed. > > $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum > > If you already have a specific environment active and know that > ?python3? is the correct Python interpreter from that environment, you > can omit the explicit path. > > $ python3 -m pip install LoremIpsum > > ---- > > Why doesn't a command ?pip? or ?pip3? do the job? Because there's no > guarantee that will use the specific Python environment you want. > > For many programs, it simply doesn't matter which Python interpreter ? > or even *whether* a Python interpreter or Perl interpreter or etc. ? is > the one that runs. > > So most Python-implemented programs you don't need to specify which > interpreter; they know how to find it themselves, and your choice of a > different environment should not affect their operation. > > But for a command that has the primary purpose of interacting with that > environment ? by installing or removing packages, as Pip does ? it must > obey your explicit instruction on which Python environment and > interpreter to use. > > -- > \ ?It ain't so much the things we don't know that get us in | > `\ trouble. It's the things we know that ain't so.? ?Artemus Ward | > _o__) (1834?1867), U.S. journalist | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list > From ben+python at benfinney.id.au Sun Oct 30 21:46:33 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 31 Oct 2016 12:46:33 +1100 Subject: pip3 : command not found References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: <85vaw99tom.fsf@benfinney.id.au> Vishal Subbiah writes: > Hi, (Please don't top-post in responses. Instead, post your responses interleaved with only the quoted material you are responding to .) > Thanks Ben for the response. I followed your suggestion and this is > what i get: > > "python3 -m pip3 install LoremIpsum > /usr/local/opt/python3/bin/python3.5: No module named pip3" Do you know of anything which should provide a module named ?pip3?? I don't know of any distribution which makes such a module available for import, so unless you know it's available you shouldn't expect it. > "python3 -m pip install LoremIpsum > /usr/local/opt/python3/bin/python3.5: No module named pip" > > How do I fix this? You have revealed that the Python 3 interpreter at that path does not have a ?pip? module. That's a problem, but should be easily solved . -- \ ?Creativity can be a social contribution, but only in so far as | `\ society is free to use the results.? ?Richard M. Stallman | _o__) | Ben Finney From rosuav at gmail.com Sun Oct 30 21:59:07 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 31 Oct 2016 12:59:07 +1100 Subject: pip3 : command not found In-Reply-To: References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: On Mon, Oct 31, 2016 at 12:15 PM, Vishal Subbiah wrote: > "python3 -m pip install LoremIpsum > /usr/local/opt/python3/bin/python3.5: No module named pip" > > How do I fix this? This is the one that ought to work. Something's gone wrong with the installation; as part of the install of recent Pythons, ensurepip should be run, which will provide you with pip. How did you install Python? ChrisA From songofacandy at gmail.com Sun Oct 30 22:28:55 2016 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 31 Oct 2016 11:28:55 +0900 Subject: pip3 : command not found In-Reply-To: References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: > How do I fix this? Maybe: $ brew uninstall python3 $ brew install python3 From best_lay at yahoo.com Mon Oct 31 00:19:36 2016 From: best_lay at yahoo.com (Wildman) Date: Sun, 30 Oct 2016 23:19:36 -0500 Subject: Calling Bash Command From Python Message-ID: Python 2.7.9 on Linux Here is a bash command that I want to run from a python program: sudo grep "^user\:" /etc/shadow If I enter the command directly into a terminal it works perfectly. If I run it from a python program it returns an empty string. Below is the code I am using. Suggestions appreciated. cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] p = subprocess.Popen(cmdlist, stdout=subprocess.PIPE, stderr=subprocess.PIPE) shadow, err = p.communicate() print shadow -- GNU/Linux user #557453 From rosuav at gmail.com Mon Oct 31 00:31:27 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 31 Oct 2016 15:31:27 +1100 Subject: Calling Bash Command From Python In-Reply-To: References: Message-ID: On Mon, Oct 31, 2016 at 3:19 PM, Wildman via Python-list wrote: > Here is a bash command that I want to run from a python > program: sudo grep "^user\:" /etc/shadow > > If I enter the command directly into a terminal it works > perfectly. If I run it from a python program it returns an > empty string. Below is the code I am using. Suggestions > appreciated. > > cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] > p = subprocess.Popen(cmdlist, > stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > shadow, err = p.communicate() > print shadow Are you able to run that command without a password? sudo might be attempting to open the console, failing, and aborting the command. Is anything being printed to stderr? ChrisA From ben+python at benfinney.id.au Mon Oct 31 00:44:13 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 31 Oct 2016 15:44:13 +1100 Subject: Call a shell command from Python (was: Calling Bash Command From Python) References: Message-ID: <85r36x9lgi.fsf@benfinney.id.au> Wildman via Python-list writes: > Python 2.7.9 on Linux > > Here is a bash command that I want to run from a python > program: sudo grep "^user\:" /etc/shadow Some points to note: * Those commands are not special to Bash, or any particular shell. They invoke commands, without AFAIK any specific Bash features. So this is asking rather to invoke a shell command. Nothing wrong with that; but on that basis, I've changed the subject field. * You're asking to invoke the ?sudo? command, which itself is designed to switch to a separate user identity and run another program. * The above command is (I assume) typed into a shell, but your Python program never invokes Bash or any other shell. > If I enter the command directly into a terminal it works perfectly. Note that ?sudo? is specifically designed to be invoked interactively, seeking to verify that the current user has credentials to run the command. Note further that ?sudo? will record when the *current user session* last invoked ?sudo? and seek re-verification if that is too long in the past. Both of these are security measures, and are designed to avoid non-interactive use of ?sudo?. Rather, it's meant to be used interactively by a real, present human with credentials to run the command. > If I run it from a python program it returns an empty string. You can also check the exit status of a command; ?grep? will give different exit status for a match versus no match. > Below is the code I am using. Suggestions > appreciated. > > cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] One immediate difference I see is that you specify different arguments to ?grep?. You have a different pattern for each command. * The ?^user\:? pattern matches ?user\:? at the start of a line. * The ?^$USER\:? pattern I think won't match anything, since ?$? matches end-of-line and then you expect further characters *past* the end of the line. I think that will always fail to match any line. > p = subprocess.Popen(cmdlist, > stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > shadow, err = p.communicate() Maybe you are expecting Bash to be involved somehow (and so ?$USER? will be substituted by Bash with some other value). That's not what happens. Instead, the ?subprocess.Popen.communicate? method will invoke the program directly, without involving a shell. See the documentation for ?subprocess.Popen?. -- \ ?If you continue running Windows, your system may become | `\ unstable.? ?Microsoft, Windows 95 bluescreen error message | _o__) | Ben Finney From rosuav at gmail.com Mon Oct 31 00:55:21 2016 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 31 Oct 2016 15:55:21 +1100 Subject: Call a shell command from Python (was: Calling Bash Command From Python) In-Reply-To: <85r36x9lgi.fsf@benfinney.id.au> References: <85r36x9lgi.fsf@benfinney.id.au> Message-ID: On Mon, Oct 31, 2016 at 3:44 PM, Ben Finney wrote: > Note that ?sudo? is specifically designed to be invoked interactively, > seeking to verify that the current user has credentials to run the > command. > > Note further that ?sudo? will record when the *current user session* > last invoked ?sudo? and seek re-verification if that is too long in the > past. > > Both of these are security measures, and are designed to avoid > non-interactive use of ?sudo?. Rather, it's meant to be used > interactively by a real, present human with credentials to run the > command. I don't know that non-interactive sudo is so bad a thing. In fact, sudo has a --non-interactive option that appears specifically designed for this kind of thing - it causes the command to fail rather than prompt. You can configure a sudoers file to allow passwordless execution of specific commands, and then permit scripts to elevate privileges in very limited ways, safely. ChrisA From __peter__ at web.de Mon Oct 31 04:12:57 2016 From: __peter__ at web.de (Peter Otten) Date: Mon, 31 Oct 2016 09:12:57 +0100 Subject: Calling Bash Command From Python References: Message-ID: Wildman via Python-list wrote: > Python 2.7.9 on Linux > > Here is a bash command that I want to run from a python > program: sudo grep "^user\:" /etc/shadow > > If I enter the command directly into a terminal it works > perfectly. If I run it from a python program it returns an > empty string. Below is the code I am using. Suggestions > appreciated. > > cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] > p = subprocess.Popen(cmdlist, > stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > shadow, err = p.communicate() > print shadow What happens if you hardcode $USER? Compare: >>> subprocess.Popen(["sudo", "echo", "$USER"], stdout=subprocess.PIPE).communicate() ('$USER\n', None) That should explain the empty result. Possible fix: >>> subprocess.Popen(["sudo", "echo", os.environ["USER"]], stdout=subprocess.PIPE).communicate() ('user\n', None) While a shell should work, too, >>> subprocess.Popen("sudo echo $USER", stdout=subprocess.PIPE, shell=True).communicate() ('petto\n', None) I'd prefer the os.environ lookup. From jon+usenet at unequivocal.eu Mon Oct 31 04:13:54 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 31 Oct 2016 08:13:54 -0000 (UTC) Subject: Calling Bash Command From Python References: Message-ID: On 2016-10-31, Wildman wrote: > Here is a bash command that I want to run from a python > program: sudo grep "^user\:" /etc/shadow > > If I enter the command directly into a terminal it works > perfectly. If I run it from a python program it returns an > empty string. Below is the code I am using. Suggestions > appreciated. > > cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] > p = subprocess.Popen(cmdlist, > stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > shadow, err = p.communicate() > print shadow Slightly surprised that nobody's pointed out that in your bash invocation, the first argument to grep is: ^user\: and in the Python code it is: "$USER\:" Your cmdlist should read: ["sudo", "grep", r"^user\:", "/etc/shadow"] or if you really want it to do the same as the bash: ["sudo", "grep", "^" + os.environ["USER"] + r"\:", "/etc/shadow"] From jon+usenet at unequivocal.eu Mon Oct 31 04:21:15 2016 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 31 Oct 2016 08:21:15 -0000 (UTC) Subject: pip3 : command not found References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: On 2016-10-31, Ben Finney wrote: > Instead, you should invoke the exact Python interpreter you want ? and, > by extension, the Python environment into which you want packages > installed. > > $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum I'm slightly curious about that. /foo/bar/virtualenv/bin/python3 will just be a symbolic link to /usr/bin/python3, so how does invoking the intepreter that way make any difference? From steve+python at pearwood.info Mon Oct 31 06:55:23 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Mon, 31 Oct 2016 21:55:23 +1100 Subject: pip3 : command not found References: <85zill9wv5.fsf@benfinney.id.au> Message-ID: <5817231d$0$1593$c3e8da3$5496439d@news.astraweb.com> On Mon, 31 Oct 2016 07:21 pm, Jon Ribbens wrote: > On 2016-10-31, Ben Finney wrote: >> Instead, you should invoke the exact Python interpreter you want ? and, >> by extension, the Python environment into which you want packages >> installed. >> >> $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum > > I'm slightly curious about that. /foo/bar/virtualenv/bin/python3 > will just be a symbolic link to /usr/bin/python3, so how does > invoking the intepreter that way make any difference? It doesn't. If you read the rest of Ben's post, or for that matter the subject line of this thread, you will see he is comparing: path/to/python3 -m pip install LoremIpsum against: pip3 install LoremIpsum not a direct path to the executable versus a symbolic link to the executable. The problem here is the "pip3" command, which does not exist. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From tpqnnd01 at gmail.com Mon Oct 31 09:39:26 2016 From: tpqnnd01 at gmail.com (tpqnnd01 at gmail.com) Date: Mon, 31 Oct 2016 06:39:26 -0700 (PDT) Subject: Recursive generator in Python 3.5 Message-ID: I have some confuse about the recursive generator where the code mixing Yield and return keywork as below. I understand that "return" keywork just raise StopIteration exception but can not understanding in depth, so, could some one explain me about the actual function of two "return" keyworks in case it have other function and mechanism of for running this code segment below: def fg(args): if not args: yield "" return for i in args[0]: for tmp in fg(args[1:]): yield i + tmp return print(list(fg(['abc', 'xyz', '123']))) From rosuav at gmail.com Mon Oct 31 09:54:46 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Nov 2016 00:54:46 +1100 Subject: Recursive generator in Python 3.5 In-Reply-To: References: Message-ID: On Tue, Nov 1, 2016 at 12:39 AM, wrote: > I have some confuse about the recursive generator where the code mixing Yield and return keywork as below. I understand that "return" keywork just raise StopIteration exception but can not understanding in depth, so, could some one explain me about the actual function of two "return" keyworks in case it have other function and mechanism of for running this code segment below: > > > def fg(args): > if not args: > yield "" > return > for i in args[0]: > for tmp in fg(args[1:]): > yield i + tmp > return > print(list(fg(['abc', 'xyz', '123']))) > When you call list() on something, the list constructor will step through the generator until it returns. The generator function will run, just like any other function, until it hits a 'return' or falls off the end. Imagine that every 'yield ...' is actually 'some_list.append(...)'. That's really what's happening here. The list you get at the end consists of all the things yielded. Don't worry about the mechanism of StopIteration. The point of a generator is that it produces values by yielding, and when it returns, it no longer produces any values. This is exactly the same as you'd get if the function printed those values, or called some other function, or appended them to a list, or something - it's a way of generalizing a function so its values can be used in any way you choose. ChrisA From tpqnnd01 at gmail.com Mon Oct 31 10:03:18 2016 From: tpqnnd01 at gmail.com (tpqnnd01 at gmail.com) Date: Mon, 31 Oct 2016 07:03:18 -0700 (PDT) Subject: Recursive generator in Python 3.5 In-Reply-To: References: Message-ID: Hi ChrisA, Thank you so much for sharing this point, I just concern to the point that: normally, I found that the generator (it may be recursive generator also) do not apply the "return" keyword, just the yield only. But when I delete one or both of "return" keyword above, the code do not action. What is the concern here, could you share more experience on that, From best_lay at yahoo.com Mon Oct 31 10:14:06 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 09:14:06 -0500 Subject: Calling Bash Command From Python References: Message-ID: On Mon, 31 Oct 2016 15:31:27 +1100, Chris Angelico wrote: > On Mon, Oct 31, 2016 at 3:19 PM, Wildman via Python-list > wrote: >> Here is a bash command that I want to run from a python >> program: sudo grep "^user\:" /etc/shadow >> >> If I enter the command directly into a terminal it works >> perfectly. If I run it from a python program it returns an >> empty string. Below is the code I am using. Suggestions >> appreciated. >> >> cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] >> p = subprocess.Popen(cmdlist, >> stdout=subprocess.PIPE, >> stderr=subprocess.PIPE) >> shadow, err = p.communicate() >> print shadow > > Are you able to run that command without a password? sudo might be > attempting to open the console, failing, and aborting the command. Is > anything being printed to stderr? > > ChrisA No. The program is run in a terminal and sudo asks for a password. -- GNU/Linux user #557453 The cow died so I don't need your bull! From rosuav at gmail.com Mon Oct 31 10:21:31 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Nov 2016 01:21:31 +1100 Subject: Recursive generator in Python 3.5 In-Reply-To: References: Message-ID: On Tue, Nov 1, 2016 at 1:03 AM, wrote: > Hi ChrisA, > > Thank you so much for sharing this point, I just concern to the point that: normally, I found that the generator (it may be recursive generator also) do not apply the "return" keyword, just the yield only. But when I delete one or both of "return" keyword above, the code do not action. What is the concern here, could you share more experience on that, > They do use return, though. Can you show me an example of a generator that doesn't? Recursive generators may be slightly tricky, as you can't simply call them. Most likely you want to 'yield from' the recursive call. ChrisA From best_lay at yahoo.com Mon Oct 31 10:49:08 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 09:49:08 -0500 Subject: Call a shell command from Python (was: Calling Bash Command From Python) References: <85r36x9lgi.fsf@benfinney.id.au> Message-ID: On Mon, 31 Oct 2016 15:44:13 +1100, Ben Finney wrote: > Wildman via Python-list writes: > >> Python 2.7.9 on Linux >> >> Here is a bash command that I want to run from a python >> program: sudo grep "^user\:" /etc/shadow > > Some points to note: > > * Those commands are not special to Bash, or any particular shell. They > invoke commands, without AFAIK any specific Bash features. So this is > asking rather to invoke a shell command. Yes, I know. I perhaps should have used the word "shell" instead of bash. However, bash is a shell. > Nothing wrong with that; but on that basis, I've changed the subject > field. > > * You're asking to invoke the ?sudo? command, which itself is designed > to switch to a separate user identity and run another program. Yes, I know. > * The above command is (I assume) typed into a shell, but your Python > program never invokes Bash or any other shell. The program is run in a shell so invocation is not needed. >> If I enter the command directly into a terminal it works perfectly. > > Note that ?sudo? is specifically designed to be invoked interactively, > seeking to verify that the current user has credentials to run the > command. > > Note further that ?sudo? will record when the *current user session* > last invoked ?sudo? and seek re-verification if that is too long in the > past. > > Both of these are security measures, and are designed to avoid > non-interactive use of ?sudo?. Rather, it's meant to be used > interactively by a real, present human with credentials to run the > command. Yes, I know all that. >> If I run it from a python program it returns an empty string. > > You can also check the exit status of a command; ?grep? will give > different exit status for a match versus no match. > >> Below is the code I am using. Suggestions >> appreciated. >> >> cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] > > One immediate difference I see is that you specify different arguments > to ?grep?. You have a different pattern for each command. > > * The ?^user\:? pattern matches ?user\:? at the start of a line. > > * The ?^$USER\:? pattern I think won't match anything, since ?$? matches > end-of-line and then you expect further characters *past* the end of > the line. I think that will always fail to match any line. Yes, the '^' indicates the start of the line and the ':' indicates the character where to stop. The colon has a special meaning so it has to be escaped, '\:'. The dollar sign precedes a variable. In this case it is an environment variable. The Linux shadow file contains information about the users of the system. It has the user name, encrypted password, salt, encryption type and other information. The format is such that the user name is at the start of the line ending with a colon. Here is an example: wildman:$6$hODbsJJp$/NWGXZ3fMIVB4U.v/oLtAv.CnL0l0I39.IwsDx1ZAlKW3wUSjTfwJdnQvOMpYNbqNqqFfZ52vgYWBmnjsaX9R.:16177:0:99999:7::: >> p = subprocess.Popen(cmdlist, >> stdout=subprocess.PIPE, >> stderr=subprocess.PIPE) >> shadow, err = p.communicate() > > Maybe you are expecting Bash to be involved somehow (and so ?$USER? will > be substituted by Bash with some other value). That's not what happens. No, the shell is already running. And $USER will be substituted by the name of the user that invoked the shell. > Instead, the ?subprocess.Popen.communicate? method will invoke the > program directly, without involving a shell. See the documentation for > ?subprocess.Popen?. I will look into that. Thanks for the reply. -- GNU/Linux user #557453 The cow died so I don't need your bull! From tpqnnd01 at gmail.com Mon Oct 31 10:50:20 2016 From: tpqnnd01 at gmail.com (tpqnnd01 at gmail.com) Date: Mon, 31 Oct 2016 07:50:20 -0700 (PDT) Subject: Recursive generator in Python 3.5 In-Reply-To: References: Message-ID: <4b69583e-5ef5-4898-aa3d-afddce7a9d3e@googlegroups.com> Hi Chris, Please see the exampl I just got it below, this is the Tower of Hanoi with recursive generator but it do not have the 'return' here, do you have the advice for this term: def A001511(): yield 1 b=1 for x in A001511(): yield x+1 yield 1 trial=A001511() for i in range(10): a=next(trial) print(a) From best_lay at yahoo.com Mon Oct 31 10:55:06 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 09:55:06 -0500 Subject: Calling Bash Command From Python References: Message-ID: On Mon, 31 Oct 2016 09:12:57 +0100, Peter Otten wrote: > Wildman via Python-list wrote: > >> Python 2.7.9 on Linux >> >> Here is a bash command that I want to run from a python >> program: sudo grep "^user\:" /etc/shadow >> >> If I enter the command directly into a terminal it works >> perfectly. If I run it from a python program it returns an >> empty string. Below is the code I am using. Suggestions >> appreciated. >> >> cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] >> p = subprocess.Popen(cmdlist, >> stdout=subprocess.PIPE, >> stderr=subprocess.PIPE) >> shadow, err = p.communicate() >> print shadow > > What happens if you hardcode $USER? Compare: > >>>> subprocess.Popen(["sudo", "echo", "$USER"], > stdout=subprocess.PIPE).communicate() > ('$USER\n', None) > > That should explain the empty result. Possible fix: > >>>> subprocess.Popen(["sudo", "echo", os.environ["USER"]], > stdout=subprocess.PIPE).communicate() > ('user\n', None) > > While a shell should work, too, > >>>> subprocess.Popen("sudo echo $USER", stdout=subprocess.PIPE, > shell=True).communicate() > ('petto\n', None) > > I'd prefer the os.environ lookup. I have code using that approach but I am trying to save myself from having to parse the entire shadow file. Grep will do it for me if I can get code right. -- GNU/Linux user #557453 The cow died so I don't need your bull! From rosuav at gmail.com Mon Oct 31 11:02:30 2016 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Nov 2016 02:02:30 +1100 Subject: Recursive generator in Python 3.5 In-Reply-To: <4b69583e-5ef5-4898-aa3d-afddce7a9d3e@googlegroups.com> References: <4b69583e-5ef5-4898-aa3d-afddce7a9d3e@googlegroups.com> Message-ID: On Tue, Nov 1, 2016 at 1:50 AM, wrote: > > Please see the exampl I just got it below, this is the Tower of Hanoi with recursive generator but it do not have the 'return' here, do you have the advice for this term: > > def A001511(): > yield 1 > b=1 > for x in A001511(): > yield x+1 > yield 1 > > > trial=A001511() > for i in range(10): > a=next(trial) > print(a) As soon as you get to the end of the function, it returns. When you iterate over a generator, you run it to completion, taking all the values it yields. ChrisA From random832 at fastmail.com Mon Oct 31 11:05:23 2016 From: random832 at fastmail.com (Random832) Date: Mon, 31 Oct 2016 11:05:23 -0400 Subject: Calling Bash Command From Python In-Reply-To: References: Message-ID: <1477926323.1198764.772701521.15EF30FA@webmail.messagingengine.com> On Mon, Oct 31, 2016, at 10:55, Wildman via Python-list wrote: > I have code using that approach but I am trying to save myself > from having to parse the entire shadow file. Grep will do it > for me if I can get code right. Python already has built-in functions to parse the shadow file. https://docs.python.org/3/library/spwd.html#module-spwd But you can't use sudo this way if you use that. But why do you want to use sudo from within the python script instead of just running the python script with sudo? From best_lay at yahoo.com Mon Oct 31 12:16:51 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 11:16:51 -0500 Subject: Calling Bash Command From Python References: Message-ID: On Mon, 31 Oct 2016 08:13:54 +0000, Jon Ribbens wrote: > On 2016-10-31, Wildman wrote: >> Here is a bash command that I want to run from a python >> program: sudo grep "^user\:" /etc/shadow >> >> If I enter the command directly into a terminal it works >> perfectly. If I run it from a python program it returns an >> empty string. Below is the code I am using. Suggestions >> appreciated. >> >> cmdlist = ["sudo", "grep", '"^$USER\:"', "/etc/shadow"] >> p = subprocess.Popen(cmdlist, >> stdout=subprocess.PIPE, >> stderr=subprocess.PIPE) >> shadow, err = p.communicate() >> print shadow > > Slightly surprised that nobody's pointed out that in your bash > invocation, the first argument to grep is: > > ^user\: > > and in the Python code it is: > > "$USER\:" > > Your cmdlist should read: > > ["sudo", "grep", r"^user\:", "/etc/shadow"] > > or if you really want it to do the same as the bash: > > ["sudo", "grep", "^" + os.environ["USER"] + r"\:", "/etc/shadow"] The above line works perfectly. I didn't occur to me to use the 'r' modifier. Thank you. Still one thing is odd. No matter what, if I use the environment variable $USER in the code, it won't work. Just returns an empty string. At this point that is a non-issue tho. Thanks again. -- GNU/Linux user #557453 From steve+python at pearwood.info Mon Oct 31 12:46:33 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 01 Nov 2016 03:46:33 +1100 Subject: Recursive generator in Python 3.5 References: Message-ID: <5817756b$0$1591$c3e8da3$5496439d@news.astraweb.com> On Tue, 1 Nov 2016 12:39 am, tpqnnd01 at gmail.com wrote: > I have some confuse about the recursive generator where the code mixing > Yield and return keywork as below. I understand that "return" keywork just > raise StopIteration exception but can not understanding in depth, so, > could some one explain me about the actual function of two "return" > keyworks in case it have other function and mechanism of for running this > code segment below: What part confuses you? The recursion, or the generator? In a generator, "yield" outputs a value. "return" ends the generator. Return is optional: reaching the end of the generator is the same as return. So this generator: def gen(): yield 1 yield 2 return # optional will output 1, then 2, then stop. This generator: def gen(*args): if not args: yield "" return for a in args: yield a return # optional will either output "", then stop, or it will output each of the arguments given, then stop. Now let's look at your recursive call version: > def fg(args): > if not args: > yield "" > return > for i in args[0]: > for tmp in fg(args[1:]): > yield i + tmp > return Let's start with a simple call: list(fg([])) args is empty, so it will output "" and halt. list(fg(["ab"])) args[0] is "ab", so it will loop: for c in "ab": Inside this loop, it loops again, this time over the rest of the arguments. There are no more arguments, so the recursive call to fg() will yield "". So the generator outputs "a" + "" (which is "a"); then it outputs "b" + "" (which is "b"). Then it halts. Second example: list(fg(["xy", "ab"])) args[0] is "xy", so it will loop: for c in "xy": Inside this loop, it loops again, this time over the rest of the arguments. There is one more argument, "ab", so the recursive call to fg() is like: fg(["ab"]) But we have already seen what happens when you call fg(["ab"]): it outputs "a", then "b", then halts. So now we have: for c in "xy": for d in ("a", "b"): yield c + d so you will get "xa", "xb", "ya", "yb". Last example: list(fg(["PQR", "xy", "ab"])) will loop over "PQR", then as the inner loop it will loop over fg(["xy", "ab"]). But we already have seen that. So you will get: "Pxa", "Pxb", "Pya", "Pyb", "Qxa", "Qxb", "Qya", "Qyb", "Rxa", "Rxb", "Rya", "Ryb" and then halt. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From best_lay at yahoo.com Mon Oct 31 12:55:26 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 11:55:26 -0500 Subject: Calling Bash Command From Python References: <1477926323.1198764.772701521.15EF30FA@webmail.messagingengine.com> Message-ID: On Mon, 31 Oct 2016 11:05:23 -0400, Random832 wrote: > On Mon, Oct 31, 2016, at 10:55, Wildman via Python-list wrote: >> I have code using that approach but I am trying to save myself >> from having to parse the entire shadow file. Grep will do it >> for me if I can get code right. > > Python already has built-in functions to parse the shadow file. > > https://docs.python.org/3/library/spwd.html#module-spwd I didn't know about that module. Thanks, this can simplify things for me. > But you can't use sudo this way if you use that. But why do you want to > use sudo from within the python script instead of just running the > python script with sudo? In view of the module I just learned about, that would be a better approach. -- GNU/Linux user #557453 From hemla21 at gmail.com Mon Oct 31 13:20:18 2016 From: hemla21 at gmail.com (Heli) Date: Mon, 31 Oct 2016 10:20:18 -0700 (PDT) Subject: Reading Fortran Ascii output using python Message-ID: Hi all, I am trying to read an ascii file written in Fortran90 using python. I am reading this file by opening the input file and then reading using: inputfile.readline() On each line of the ascii file I have a few numbers like this: line 1: 1 line 2: 1000.834739 2000.38473 3000.349798 line 3: 1000 2000 5000.69394 99934.374638 54646.9784 The problem is when I have more than 3 numbers on the same line such as line 3, python seems to read this using two reads. This makes the above example will be read like this: line 1: 1 line 2: 1000.834739 2000.38473 3000.349798 line 3: 1000 2000 5000.69394 line 4: 99934.374638 54646.9784 How can I fix this for each fortran line to be read correctly using python? Thanks in Advance for your help, From irmen.NOSPAM at xs4all.nl Mon Oct 31 13:29:49 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 31 Oct 2016 18:29:49 +0100 Subject: Reading Fortran Ascii output using python In-Reply-To: References: Message-ID: <58177f8d$0$946$e4fe514c@news.xs4all.nl> On 31-10-2016 18:20, Heli wrote: > Hi all, > > I am trying to read an ascii file written in Fortran90 using python. I am reading this file by opening the input file and then reading using: > > inputfile.readline() > > On each line of the ascii file I have a few numbers like this: > > line 1: 1 > line 2: 1000.834739 2000.38473 3000.349798 > line 3: 1000 2000 5000.69394 99934.374638 54646.9784 > > The problem is when I have more than 3 numbers on the same line such as line 3, python seems to read this using two reads. This makes the above example will be read like this: > > line 1: 1 > line 2: 1000.834739 2000.38473 3000.349798 > line 3: 1000 2000 5000.69394 > line 4: 99934.374638 54646.9784 > > How can I fix this for each fortran line to be read correctly using python? > > Thanks in Advance for your help, > > You don't show any code so it's hard to say what is going on. My guess is that your file contains spurious newlines and/or CRLF combinations. Try opening the file in universal newline mode and see what happens? with open("fortranfile.txt", "rU") as f: for line in f: print("LINE:", line) Irmen From hemla21 at gmail.com Mon Oct 31 13:46:19 2016 From: hemla21 at gmail.com (Heli) Date: Mon, 31 Oct 2016 10:46:19 -0700 (PDT) Subject: Reading Fortran Ascii output using python In-Reply-To: <58177f8d$0$946$e4fe514c@news.xs4all.nl> References: <58177f8d$0$946$e4fe514c@news.xs4all.nl> Message-ID: On Monday, October 31, 2016 at 6:30:12 PM UTC+1, Irmen de Jong wrote: > On 31-10-2016 18:20, Heli wrote: > > Hi all, > > > > I am trying to read an ascii file written in Fortran90 using python. I am reading this file by opening the input file and then reading using: > > > > inputfile.readline() > > > > On each line of the ascii file I have a few numbers like this: > > > > line 1: 1 > > line 2: 1000.834739 2000.38473 3000.349798 > > line 3: 1000 2000 5000.69394 99934.374638 54646.9784 > > > > The problem is when I have more than 3 numbers on the same line such as line 3, python seems to read this using two reads. This makes the above example will be read like this: > > > > line 1: 1 > > line 2: 1000.834739 2000.38473 3000.349798 > > line 3: 1000 2000 5000.69394 > > line 4: 99934.374638 54646.9784 > > > > How can I fix this for each fortran line to be read correctly using python? > > > > Thanks in Advance for your help, > > > > > > You don't show any code so it's hard to say what is going on. > My guess is that your file contains spurious newlines and/or CRLF combinations. > > Try opening the file in universal newline mode and see what happens? > > with open("fortranfile.txt", "rU") as f: > for line in f: > print("LINE:", line) > > > Irmen Thanks Irmen, I tried with "rU" but that did not make a difference. The problem is a line that with one single write statement in my fortran code : write(UNIT=9,FMT="(99g20.8)") value seems to be read in two python inputfile.readline(). Any ideas how I should be fixing this? Thanks, From grant.b.edwards at gmail.com Mon Oct 31 14:18:03 2016 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 31 Oct 2016 18:18:03 +0000 (UTC) Subject: Call a shell command from Python (was: Calling Bash Command From Python) References: <85r36x9lgi.fsf@benfinney.id.au> Message-ID: On 2016-10-31, Wildman via Python-list wrote: > On Mon, 31 Oct 2016 15:44:13 +1100, Ben Finney wrote: >> Wildman via Python-list writes: >> >>> Python 2.7.9 on Linux >>> >>> Here is a bash command that I want to run from a python >>> program: sudo grep "^user\:" /etc/shadow >> >> Some points to note: >> >> * Those commands are not special to Bash, or any particular shell. They >> invoke commands, without AFAIK any specific Bash features. So this is >> asking rather to invoke a shell command. Actually it's not a shell command either. > Yes, I know. I perhaps should have used the word "shell" instead > of bash. However, bash is a shell. To most people, "bash command" or "shell command" refers to 1) a commands built in to bash (or other shell) or 2) a command line that must be interpreted by bash (or other shell) because it uses I/O redirection, variable substituion, globbing, etc. Your code does not use bash (or any other shell), nor does it need to. It's just running the sudo executable. There are no shells involved, so using the word shell isn't any better than using the word bash. -- Grant Edwards grant.b.edwards Yow! I have seen these EGG at EXTENDERS in my Supermarket gmail.com ... I have read the INSTRUCTIONS ... From best_lay at yahoo.com Mon Oct 31 14:20:27 2016 From: best_lay at yahoo.com (Wildman) Date: Mon, 31 Oct 2016 13:20:27 -0500 Subject: Calling Bash Command From Python References: <1477926323.1198764.772701521.15EF30FA@webmail.messagingengine.com> Message-ID: On Mon, 31 Oct 2016 11:55:26 -0500, Wildman wrote: > On Mon, 31 Oct 2016 11:05:23 -0400, Random832 wrote: > >> On Mon, Oct 31, 2016, at 10:55, Wildman via Python-list wrote: >>> I have code using that approach but I am trying to save myself >>> from having to parse the entire shadow file. Grep will do it >>> for me if I can get code right. >> >> Python already has built-in functions to parse the shadow file. >> >> https://docs.python.org/3/library/spwd.html#module-spwd > > I didn't know about that module. Thanks, this can simplify > things for me. > >> But you can't use sudo this way if you use that. But why do you want to >> use sudo from within the python script instead of just running the >> python script with sudo? > > In view of the module I just learned about, that would be > a better approach. I made a ?discovery that I thought I would share. When using sudo to run the script the environment variable $USER will always return 'root'. Not what I wanted. But this will work: user = os.environ["SUDO_USER"] shadow = spwd.getspnam(user) That will return the actual user name that invoked sudo. -- GNU/Linux user #557453 The cow died so I don't need your bull! From irmen.NOSPAM at xs4all.nl Mon Oct 31 14:24:10 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 31 Oct 2016 19:24:10 +0100 Subject: Reading Fortran Ascii output using python In-Reply-To: References: <58177f8d$0$946$e4fe514c@news.xs4all.nl> Message-ID: <58178c4a$0$889$e4fe514c@news.xs4all.nl> On 31-10-2016 18:46, Heli wrote: > Thanks Irmen, > > I tried with "rU" but that did not make a difference. The problem is a line that with one single write statement in my fortran code : > > write(UNIT=9,FMT="(99g20.8)") value > > seems to be read in two python inputfile.readline(). > > Any ideas how I should be fixing this? > > Thanks, > We don't speak Fortran here (at least I don't). Please show your Python code. What is 'inputfile'? Also, in Python, the readline method is defined as: >>> help(io.TextIOBase.readline) Help on method_descriptor: readline(...) Read until newline or EOF. Returns an empty string if EOF is hit immediately. So there must be something in that line in your file that it considers an EOF. Have you tried opening it in a regular text editor and inspecting what the characters are on that particular line? Irmen From irmen.NOSPAM at xs4all.nl Mon Oct 31 14:25:14 2016 From: irmen.NOSPAM at xs4all.nl (Irmen de Jong) Date: Mon, 31 Oct 2016 19:25:14 +0100 Subject: Reading Fortran Ascii output using python In-Reply-To: <58178c4a$0$889$e4fe514c@news.xs4all.nl> References: <58177f8d$0$946$e4fe514c@news.xs4all.nl> <58178c4a$0$889$e4fe514c@news.xs4all.nl> Message-ID: <58178c8b$0$889$e4fe514c@news.xs4all.nl> On 31-10-2016 19:24, Irmen de Jong wrote: > So there must be something in that line in your file that it considers an EOF. I meant to type EOL there. (end-of-line/newline). Irmen From tpqnnd01 at gmail.com Mon Oct 31 14:47:58 2016 From: tpqnnd01 at gmail.com (tpqnnd01 at gmail.com) Date: Mon, 31 Oct 2016 11:47:58 -0700 (PDT) Subject: Recursive generator in Python 3.5 In-Reply-To: <5817756b$0$1591$c3e8da3$5496439d@news.astraweb.com> References: <5817756b$0$1591$c3e8da3$5496439d@news.astraweb.com> Message-ID: Thanks for your detail explanation, my problem may be the 'return' keyword. I confuse at the point that return come after yield keyword, I wonder what if in case this code do not have the 1st return. So, when i try to delete the 1st return, the code run with error (the list out of range). This is actual clear from your example above, when the 1st return is delected, the statement flow shall be from (yield '') to (yield i + tmp) when the Next() is called. This behaviour shall reach to a statement is: for i in [][0] that shall raise an error: list index out of range. From python at mrabarnett.plus.com Mon Oct 31 15:03:24 2016 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 31 Oct 2016 19:03:24 +0000 Subject: Reading Fortran Ascii output using python In-Reply-To: References: <58177f8d$0$946$e4fe514c@news.xs4all.nl> Message-ID: <1ede9bc5-5897-b1cc-3875-46bc13c182a9@mrabarnett.plus.com> On 2016-10-31 17:46, Heli wrote: > On Monday, October 31, 2016 at 6:30:12 PM UTC+1, Irmen de Jong wrote: >> On 31-10-2016 18:20, Heli wrote: >> > Hi all, >> > >> > I am trying to read an ascii file written in Fortran90 using python. I am reading this file by opening the input file and then reading using: >> > >> > inputfile.readline() >> > >> > On each line of the ascii file I have a few numbers like this: >> > >> > line 1: 1 >> > line 2: 1000.834739 2000.38473 3000.349798 >> > line 3: 1000 2000 5000.69394 99934.374638 54646.9784 >> > >> > The problem is when I have more than 3 numbers on the same line such as line 3, python seems to read this using two reads. This makes the above example will be read like this: >> > >> > line 1: 1 >> > line 2: 1000.834739 2000.38473 3000.349798 >> > line 3: 1000 2000 5000.69394 >> > line 4: 99934.374638 54646.9784 >> > >> > How can I fix this for each fortran line to be read correctly using python? >> > >> > Thanks in Advance for your help, >> > >> > >> >> You don't show any code so it's hard to say what is going on. >> My guess is that your file contains spurious newlines and/or CRLF combinations. >> >> Try opening the file in universal newline mode and see what happens? >> >> with open("fortranfile.txt", "rU") as f: >> for line in f: >> print("LINE:", line) >> >> >> Irmen > > Thanks Irmen, > > I tried with "rU" but that did not make a difference. The problem is a line that with one single write statement in my fortran code : > > write(UNIT=9,FMT="(99g20.8)") value > > seems to be read in two python inputfile.readline(). > > Any ideas how I should be fixing this? > > Thanks, > What is actually in the file? Try opening it in binary mode and print using the ascii function: with open("fortranfile.txt", "rb") as f: contents = f.read() print("CONTENTS:", ascii(contents)) From devers.meetthebadger.jason at gmail.com Mon Oct 31 18:09:11 2016 From: devers.meetthebadger.jason at gmail.com (devers.meetthebadger.jason at gmail.com) Date: Mon, 31 Oct 2016 15:09:11 -0700 (PDT) Subject: Need help with coding a function in Python Message-ID: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> http://imgur.com/a/rfGhK#iVLQKSW How do I code a function that returns a list of the first n elements of the sequence defined in the link? I have no idea!!!!! So far this is my best shot at it (the problem with it is that the n that i'm subtracting or adding in the if/else part does not represent the element's position, but just the n that I am plugging into the function): def reca(n): rlist=[0] while len(rlist) < n: if (rlist[-1]-n) > 0 and (rlist[-1]-n) not in rlist: rlist.append(rlist[-1]-n) else: rlist.append(rlist[-1]+n) return(rlist) From greg.ewing at canterbury.ac.nz Mon Oct 31 18:25:33 2016 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 01 Nov 2016 11:25:33 +1300 Subject: Recursive generator in Python 3.5 In-Reply-To: References: Message-ID: tpqnnd01 at gmail.com wrote: > def fg(args): > if not args: > yield "" > return > for i in args[0]: > for tmp in fg(args[1:]): > yield i + tmp > return The final return is redundant, since there is an implicit return at the end, just like an ordinary function. The other one is just performing an early exit from the iteration. You could write it without any returns like this: def fg(args): if not args: yield "" else: for i in args[0]: for tmp in fg(args[1:]): yield i + tmp > this is the Tower of Hanoi with recursive generator but it do > not have the 'return' here -- Greg From python at mrabarnett.plus.com Mon Oct 31 18:37:19 2016 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 31 Oct 2016 22:37:19 +0000 Subject: Need help with coding a function in Python In-Reply-To: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> References: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> Message-ID: <13c147fa-9f64-2c85-af0b-e2b37b80fa53@mrabarnett.plus.com> On 2016-10-31 22:09, devers.meetthebadger.jason at gmail.com wrote: > http://imgur.com/a/rfGhK#iVLQKSW > > How do I code a function that returns a list of the first n elements of the sequence defined in the link? I have no idea!!!!! > > So far this is my best shot at it (the problem with it is that the n that i'm subtracting or adding in the if/else part does not represent the element's position, but just the n that I am plugging into the function): > > def reca(n): > rlist=[0] > while len(rlist) < n: > if (rlist[-1]-n) > 0 and (rlist[-1]-n) not in rlist: > rlist.append(rlist[-1]-n) > > else: > rlist.append(rlist[-1]+n) > > return(rlist) > When you're calculating a[n], what's n? Well: When you're calculating a[1], you already have a[0], or 1 element. When you're calculating a[2], you already have a[0] and a[1], or 2 elements. So, in general, when you're calculating a[n], you already have a[0] ... a[n - 1], or n elements. The value of n is the length of the list so far. From no.email at nospam.invalid Mon Oct 31 19:11:35 2016 From: no.email at nospam.invalid (Paul Rubin) Date: Mon, 31 Oct 2016 16:11:35 -0700 Subject: Need help with coding a function in Python References: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> Message-ID: <87bmy0ktaw.fsf@nightsong.com> devers.meetthebadger.jason at gmail.com writes: > http://imgur.com/a/rfGhK#iVLQKSW ... > So far this is my best shot at it (the problem with it is that the n > that i'm subtracting or adding in the if/else part does not represent > the element's position... Right, so can you figure out the element's position? If yes, calculate it and use it instead of n. From steve+python at pearwood.info Mon Oct 31 20:40:48 2016 From: steve+python at pearwood.info (Steve D'Aprano) Date: Tue, 01 Nov 2016 11:40:48 +1100 Subject: Need help with coding a function in Python References: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> Message-ID: <5817e493$0$1615$c3e8da3$5496439d@news.astraweb.com> On Tue, 1 Nov 2016 09:09 am, devers.meetthebadger.jason at gmail.com wrote: > http://imgur.com/a/rfGhK#iVLQKSW Why on earth are you posting a screen shot? Do you edit your code with Photoshop? As soon as you post an unnecessary screenshot, you cut the number of people willing and able to help you in half: - anyone who is blind or visually impaired and reading this with a screen reader cannot help you, even if they wanted to; - anyone reading this post somewhere where access to imgur is blocked (say, from work); - anyone who takes one look at the URL and says "F--k it, if they can't be bothered to copy and paste text, I can't be bothered to follow the link". We're volunteers. We're not paid to solve your problem, so if you make it hard for us, we simply won't bother. > How do I code a function that returns a list of the first n elements of > the sequence defined in the link? I have no idea!!!!! I have no idea either, because I cannot see the image. I'll leave you to guess why I can't. But the usual way to return a list of the first n elements of a sequence is with slicing: first_bunch = sequence[:n] # a slice from index 0 to just before index n If sequence is a list, then the slice sequence[:n] will also be a list. Otherwise you may want to convert it to a list: first_bunch = list(sequence[:n]) How does slicing work? You can think of it as a faster, more efficient way of something like this: def cut_slice(sequence, start=0, end=None): if end is None: end = len(sequence) result = [] for i in range(start, end): value = sequence[i] result.append(value) return result P.S. any time you think you want a while loop, 90% of the time you don't. While-loops have their uses, but they're unusual. Whenever you know ahead of time how many loops will be done, use a for-loop. While-loops are only for those unusual cases where you don't know how many times you need to loop. Use a while-loop for: - loop *until* something happens; - loop *while* something happens; Use a for-loop for: - loop over each element of a sequence; - loop a fixed number of times. -- Steve ?Cheer up,? they said, ?things could be worse.? So I cheered up, and sure enough, things got worse. From ben+python at benfinney.id.au Mon Oct 31 21:08:52 2016 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 01 Nov 2016 12:08:52 +1100 Subject: Call a shell command from Python References: <85r36x9lgi.fsf@benfinney.id.au> Message-ID: <85mvhk9fbv.fsf@benfinney.id.au> Wildman via Python-list writes: > On Mon, 31 Oct 2016 15:44:13 +1100, Ben Finney wrote: > > > One immediate difference I see is that you specify different > > arguments to ?grep?. You have a different pattern for each command. > > > > * The ?^user\:? pattern matches ?user\:? at the start of a line. > > > > * The ?^$USER\:? pattern I think won't match anything, since ?$? matches > > end-of-line and then you expect further characters *past* the end of > > the line. I think that will always fail to match any line. > > Yes, the '^' indicates the start of the line and the ':' indicates > the character where to stop. The colon has a special meaning so it > has to be escaped, '\:'. The dollar sign precedes a variable. In > this case it is an environment variable. The ?grep? program you're invoking knows nothing of such variables, and the ?$? sign means to ?grep? what I said above. > > Maybe you are expecting Bash to be involved somehow (and so ?$USER? > > will be substituted by Bash with some other value). That's not what > > happens. > > No, the shell is already running. I don't know what you mean by this. If you mean that some *other* instances of the shell ar running: that isn't relevant to how your Python program invokes a subprocess. The shell is not involved in the command as you invoke it directly as a subprocess, without asking for a shell. > And $USER will be substituted by the name of the user that invoked the > shell. It will not, because there is no shell involved: your Python program invokes ?sudo?, which invokes ?grep?. The shell is never involved in that chain, so its substitutions rules are irrelevant. -- \ ?Try adding ?as long as you don't breach the terms of service ? | `\ according to our sole judgement? to the end of any cloud | _o__) computing pitch.? ?Simon Phipps, 2010-12-11 | Ben Finney From orgnut at yahoo.com Mon Oct 31 22:26:03 2016 From: orgnut at yahoo.com (Larry Hudson) Date: Mon, 31 Oct 2016 19:26:03 -0700 Subject: Need help with coding a function in Python In-Reply-To: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> References: <01d2a11b-922c-4a07-a203-58238b2a7a82@googlegroups.com> Message-ID: <64qdnf-_homhYIrFnZ2dnUU7-TvNnZ2d@giganews.com> On 10/31/2016 03:09 PM, devers.meetthebadger.jason at gmail.com wrote: > http://imgur.com/a/rfGhK#iVLQKSW > > How do I code a function that returns a list of the first n elements of the sequence defined in the link? I have no idea!!!!! > > So far this is my best shot at it (the problem with it is that the n that i'm subtracting or adding in the if/else part does not represent the element's position, but just the n that I am plugging into the function): > > def reca(n): > rlist=[0] > while len(rlist) < n: > if (rlist[-1]-n) > 0 and (rlist[-1]-n) not in rlist: > rlist.append(rlist[-1]-n) > > else: > rlist.append(rlist[-1]+n) > > return(rlist) > I'm not looking at your link or your code, just your description of the problem. Paraphrased: "A list of the first n elements of a sequence" No function necessary, just use slicing... newlist = oldlist[:n] list: [1,2,3,4,5,6,7,8,9,10][:5] gives [1,2,3,4,5] tuple: (1,2,3,4,5,6,7,8,9,10)[:5] gives (1,2,3,4,5) string: "Hello there"[:5] gives "Hello" Works for sequences shorter than n as well, which gives the full (short) sequence. list: [1,2,3][:5] gives [1,2,3] etc... -- -=- Larry -=-