From breamoreboy at gmail.com Mon Jan 1 00:18:29 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 31 Dec 2017 21:18:29 -0800 (PST) Subject: Copy-on-write friendly Python garbage collection In-Reply-To: References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> Message-ID: On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > breamoreboy: > > An interesting write up on something that is incorporated into Python 3.7 https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > > Appearantly, Erlang is the way to go, when it comes to web frameworks. What has that got to do with the subject of this thread? -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Jan 1 00:29:22 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 31 Dec 2017 21:29:22 -0800 (PST) Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: <5pa2C.399820$bH7.185636@fx38.am4> References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <996f33d4-df8a-460b-bba1-8f43238588b0@googlegroups.com> <5pa2C.399820$bH7.185636@fx38.am4> Message-ID: On Sunday, December 31, 2017 at 6:56:16 PM UTC, bartc wrote: > On 31/12/2017 17:01, breamoreboy wrote: > > >Further I've never once in 17 years of using Python been tearing my hair out over the lack of goto > > Neither have I over all the advanced features of Python I never use, and > for double that number of years. I suggest that you steer well clear of all of Python's advanced features until you understand the Python philosophy as laid down in the Zen of Python. However I cannot see that happening as you seem to dispute everything about Python, flying in the face of any advice that you get from all the very experienced Pythonistas who frequent this place. You must have access to the PSU's time machine if you've 34 years experience of Python as it hasn't been out that long. > > Yet for some they will be as indispensable as they are incomprehensible > to others. Let's all go back to machine code. -- Kindest regards. Mark Lawrence. From tjreedy at udel.edu Mon Jan 1 00:50:37 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 1 Jan 2018 00:50:37 -0500 Subject: tkinter MP working like a charm In-Reply-To: References: Message-ID: On 12/31/2017 8:52 PM, Terry Reedy wrote: > To do everything in the main thread, one can replace 'root.mainloop' > with loop.run_forever (in the main thread) and repeated root.update calls. Wu Xi's example, rewritten more or less as suggested: from tkinter import * from tkinter import messagebox import asyncio import random async def one_url(url): """ One task. """ sec = random.randint(1, 8) await asyncio.sleep(sec ) return 'url: {} --- sec: {}'.format(url, sec) async def do_urls(): """ Creating and starting 10 tasks. """ tasks = [one_url(url) for url in range(10)] completed, pending = await asyncio.wait(tasks) results = [task.result() for task in completed] print('\n'.join(results)) def do_nofreeze(): messagebox.showinfo(message='see, Tkinter is still responsive') def widgets(master): Button(master, text='Frozen?', command=do_nofreeze).pack() # A production version of this should make it possible to cancel. # However, when loop stops, updates will stop. def tk_update(interval): root.update() loop.call_later(interval, tk_update) if __name__ == '__main__': root = Tk() widgets(root) loop = asyncio.get_event_loop() tk_update(.01) loop.run_until_complete(do_urls()) -- Terry Jan Reedy From tjreedy at udel.edu Mon Jan 1 01:10:58 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 1 Jan 2018 01:10:58 -0500 Subject: Scientific Python moving to Python 3 only. Message-ID: https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst Numpy people currently plan to stop adding features to their 2.7 branch after Dec 31, 2018 and stop adding bug fixes to the last version supporting 2.7 a year later, Dec 31, 2019. It will remain available for pip installation, and possible support by other (most likely commercial) parties. http://www.python3statement.org/ About 30 other projects, including other part of the python-science stack, have 'pledged' to so something similar. -- Terry Jan Reedy From arj.python at gmail.com Mon Jan 1 01:33:45 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 1 Jan 2018 10:33:45 +0400 Subject: tkinter MP working like a charm In-Reply-To: References: Message-ID: it seems that the original poster's mail went missing leugenpresse means lying press btw weird i doubt the author wanted to prank us by announcing i did this and that. Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 31 Dec 2017 5:00 am, "Wu Xi" wrote: > from tkinter import * # I cant see > anything wrong with this , it works like a charm > from tkinter import messagebox # is there a > serious concern doing things his way ? > import asyncio , threading , random # goal is > multi tasking with the GUI not freezing before loop completed > # which is > being achieved here ! > def _asyncio_thread(async_loop): > async_loop.run_until_complete(do_urls()) > > def do_work(async_loop): > """ Button-Event-Handler starting stuff """ > threading.Thread(target=_asyncio_thread, args=(async_loop,)).start() > > async def one_url(url): > """ One task. """ > sec = random.randint(1, 8) > await asyncio.sleep(sec ) > return 'url: {} --- sec: {}'.format(url, sec) > > async def do_urls(): > """ Creating and starting 10 tasks. """ > tasks = [one_url(url) for url in range(10)] > completed, pending = await asyncio.wait(tasks) > results = [task.result() for task in completed] > print('\n'.join(results)) > > > def do_nofreeze(): > messagebox.showinfo(message='see, Tkinter is still responsive') > > def submain(async_loop): > root = Tk() > b1 = Button(master=root, text='do work', command= > lambda:do_work( async_loop)).pack() > b2 = Button(master=root, text='Frozen?', command=do_nofreeze > ).pack() > root.mainloop() > > > if __name__ == '__main__': > async_loop = asyncio.get_event_loop() # all in this loop > submain(async_loop) > > > -- > https://mail.python.org/mailman/listinfo/python-list > From songofacandy at gmail.com Mon Jan 1 03:24:20 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Mon, 1 Jan 2018 17:24:20 +0900 Subject: Copy-on-write friendly Python garbage collection In-Reply-To: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> Message-ID: FYI: https://bugs.python.org/issue31558 INADA Naoki On Mon, Jan 1, 2018 at 12:39 AM, wrote: > An interesting write up on something that is incorporated into Python 3.7 https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > > -- > Kindest regards. > > Mark Lawrence. > -- > https://mail.python.org/mailman/listinfo/python-list From bc at freeuk.com Mon Jan 1 06:48:57 2018 From: bc at freeuk.com (bartc) Date: Mon, 1 Jan 2018 11:48:57 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: On 01/01/2018 00:40, MRAB wrote: > On 2017-12-31 23:21, bartc wrote: [Block delimiting] > proc fn2(int a)= ... > end > > (or possibly "inline f123="). > > [snip] > > OT: if "case ... esac" and "if ... fi", why not "proc ... corp"? :-) (I don't think Algol-68 used corp otherwise it might have been copied too. But I also have 'function', and 'noitcnuf' would be a bit much. Anyway in this variation of the syntax, there is a choice of block endings. So 'case' can be closed with 'end', 'endcase', 'end case' or 'esac'. With loops like while-do, either 'while' or 'do' can be the keyword. For some kinds of blocks, (...) can be used. But there can't be no block ending as in Python; there has to be something. Indentation here is not significant.) -- bartc From news at luegenpresse.edu Mon Jan 1 07:52:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 12:52:00 +0000 Subject: Copy-on-write friendly Python garbage collection References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> Message-ID: breamoreboy at gmail.com: > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: >> breamoreboy: >>> An interesting write up on something that is incorporated into Python 3.7 https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf >> >> Appearantly, Erlang is the way to go, when it comes to web frameworks. > > What has that got to do with the subject of this thread? Well, a little implicitly. Pointing out Erlang superiority in a pythonic newsgroup is, of course, heresy. From news at luegenpresse.edu Mon Jan 1 07:58:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 12:58:00 +0000 Subject: tkinter MP working like a charm References: Message-ID: Abdur-Rahmaan Janhangeer: > it seems that the original poster's mail went missing > > leugenpresse means lying press btw > > weird > > i doubt the author wanted to prank us by announcing i did this and that. > > Abdur-Rahmaan Janhangeer, > Mauritius > abdurrahmaanjanhangeer.wordpress.com > > On 31 Dec 2017 5:00 am, "Wu Xi" wrote: > >> from tkinter import * # I cant see L?gen Presse = lying press , fake news makers - indeed. Using a real news reader, changing the post's topic still keeps the posting chain intact. I hope ypu don't use soft not giving you that post concatenation feature. From news at luegenpresse.edu Mon Jan 1 08:33:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 13:33:00 +0000 Subject: tkinter MP working like a charm - almost... References: Message-ID: from tkinter import * ; from tkinter import messagebox import asyncio , random async def one_url(url): """ This is a multi threaded tkinter demo where tasks run in the background and aim to not freeze the tk GUI, py v3.7""" sec = random.randint(1, 8) await asyncio.sleep(sec ) return 'url: {} took {} seconds'.format(url, sec) async def do_urls(): """ Creating and starting 10 tasks. """ tasks = [one_url(url) for url in range(10)] completed, pending = await asyncio.wait(tasks) results = [task.result() for task in completed] print('\n'.join(results)) def do_nofreeze(): messagebox.showinfo(message='see, Tkinter is still responsive.') def widgets(master): Button(master, text='GUI Frozen?', command=do_nofreeze).pack() def tk_update(interval): """ A production version of this should make it possible to cancel. However, when loop stops, updates will stop. T. Reedy""" loop.call_later( interval, root.update ) root.update() if __name__ == '__main__': root = Tk() widgets(root) loop = asyncio.get_event_loop() tk_update(.05) loop.run_until_complete(do_urls()) # wow! no call to root.mainloop() anymore. somehow like in IDLE - but it freezes now :-( From news at luegenpresse.edu Mon Jan 1 08:51:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 13:51:00 +0000 Subject: Mr. Conway in 50 lines of code (Posting On Python-List Prohibited) References: <37655e02-18a3-4ae3-b54c-d0499fd3afbd@googlegroups.com> <87vagmtpxe.fsf@nightsong.com> Message-ID: the short versions are not equivalent, proggy won't work with them > def neighbours(point): > x,y = point > > yield x + 1 , y > yield x - 1 , y > yield x , y + 1 > yield x , y - 1 # this is proof that life can emerge inside of computers and cellular automatons, > > yield x + 1 , y + 1 # and evolve through memory, attack other cells and morph into toads, pulsars, etc.. > yield x + 1 , y - 1 > yield x - 1 , y + 1 # spray your memory with space alien patterns and life evolution will start in your machine ! > yield x - 1 , y - 1 > > ''' > for i in range(-1, 2) : > for j in range(-1, 2) : > if i != j : > yield x + i, y + j > > ''' > # yield from [(x+i, x+j) for i in [-1,1] for j in [-1,1]] From ned at nedbatchelder.com Mon Jan 1 09:13:12 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 1 Jan 2018 09:13:12 -0500 Subject: you shortened it, but you broke it too... ;-) In-Reply-To: References: <37655e02-18a3-4ae3-b54c-d0499fd3afbd@googlegroups.com> <87vagmtpxe.fsf@nightsong.com> Message-ID: <3d749b31-c92e-3871-d255-ce59535b08d8@nedbatchelder.com> On 12/31/17 8:15 PM, Wu Xi wrote: > def neighbours(point): > x,y = point > > yield x + 1 , y > yield x - 1 , y > yield x , y + 1 > yield x , y - 1 # this is proof that life can emerge inside of computers and cellular automatons, > > yield x + 1 , y + 1 # and evolve through memory, attack other cells and morph into toads, pulsars, etc.. > yield x + 1 , y - 1 > yield x - 1 , y + 1 # spray your memory with space alien patterns and life evolution will start in your machine ! > yield x - 1 , y - 1 This code correctly yields eight new coordinates > > ''' > for i in range(-1, 2) : > for j in range(-1, 2) : > if i != j : > yield x + i, y + j > > ''' This code only yields six of the eight needed. > # yield from [(x+i, x+j) for i in [-1,1] for j in [-1,1]] > This code only yields four! Perhaps this is what you are looking for: ??? def neighbors(point): ??????? x, y = point ??????? for dx in [-1, 0, 1]: ??????????? for dy in [-1, 0, 1]: ??????????????? if dx == dy == 0: ??????????????????? continue ??????????????? yield x + dx, y + dy --Ned. From I at n-e.mail Mon Jan 1 09:36:00 2018 From: I at n-e.mail (Ian) Date: Mon, 01 Jan 2018 14:36:00 +0000 Subject: Anaconda Navigator : Add App In-Reply-To: References: Message-ID: Anaconda is v3.6 very 2017'ish by now... ;-) From hjp-python at hjp.at Mon Jan 1 09:54:27 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 1 Jan 2018 15:54:27 +0100 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> Message-ID: <20180101145427.4fuwstpo5fhfbql7@hjp.at> On 2017-12-30 11:07:56 -0500, Dennis Lee Bieber wrote: > On Sat, 30 Dec 2017 13:46:14 +0100, "Peter J. Holzer" > declaimed the following: > > >I don't think this is correct. Structured programming is much older: > >ALGOL 60 was already a block structured language and Dijkstra wrote > >"goto considered harmful" in the late 1960s. Pascal appeared in 1970, C > >in 1974. To me (who learned to program in BASIC on a CP/M machine), C > >is very much a structured programming language. If structured > >programming gained traction around 1980, it might even have been because > >structured languages like C with good performance became widely > >available. > > > There is a slight difference between just being block structured and > full structured programming (at the time "one-entry/one-exit per > construct". I'd say there is a rather big difference: One is a programming paradigm, the other a syntactic aid for the former. You can do structured programming in assembler (I did in BASIC), but it is a lot easier and natural in a block-structured language. I wasn't present when ALGOL was invented (I wasn't even born then), but I am sure that the inventors shared much of the mindset of "structured programming" (although the name may not yet have been invented). > Even Pascal still had a GOTO statement, Yes. I don't know any language which enforces "pure" structured programming. They all have some constructs (goto, break, return, exceptions, ...) to leave a block early. I don't think that invalidates my point that the concept of structured programming predates Pascal. > and flow-charts were still part of documentation at school. We learned about them in the 80's, too (along with Nassi-Shneiderman diagrams). And in fact I still use them sometimes (although more frequently to document processes for humans than for computers). (I don't use NS diagrams at all anymore: They have no advantages over pseudo code for me.) > The /teaching/ of structured programming as a discipline didn't really > show up until my final year in college (79-80) and the first few years at > Lockheed I can't comment on how wide-spread teaching of structured programming was, since I am too young. But since Pascal was explicitely intended as a teaching language for structured programming, there must have been at least a few professors to teach it in the late 1960's. > -- which was also a time period when such abominations as TEXTFOR, > MORTRAN, and RATFOR were created to allow for doing structured programming > in FORTRAN-IV And regardless of the quality (or lack thereof) of these preprocessors I'd argue that they were created (and used) because at the time many programmers thought that structured programming was a good idea. Is there any mainstream (procedural) computer language invented after 1970 which doesn't try to encourage structured programming? hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From e-m at i.l Mon Jan 1 10:06:00 2018 From: e-m at i.l (From) Date: Mon, 01 Jan 2018 15:06:00 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: (Posting On Python-List Prohibited) why ? From bc at freeuk.com Mon Jan 1 12:06:18 2018 From: bc at freeuk.com (bartc) Date: Mon, 1 Jan 2018 17:06:18 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: On 01/01/2018 15:06, From wrote: > > > > > > (Posting On Python-List Prohibited) > > > > why ? > Huh? I'm posting to the usenet group comp.lang.python (an off-topic reply to an off-topic remark, but it happens). I've no idea what the prohibited part is about, if that's what you're posting about. But there have been dozens of other messages with the same subject. From bc at freeuk.com Mon Jan 1 12:17:15 2018 From: bc at freeuk.com (bartc) Date: Mon, 1 Jan 2018 17:17:15 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> <20180101145427.4fuwstpo5fhfbql7@hjp.at> Message-ID: On 01/01/2018 14:54, Peter J. Holzer wrote: > On 2017-12-30 11:07:56 -0500, Dennis Lee Bieber wrote: > Yes. I don't know any language which enforces "pure" structured > programming. They all have some constructs (goto, break, return, > exceptions, ...) to leave a block early. I don't think that invalidates > my point that the concept of structured programming predates Pascal. Functional languages? Some banish not only goto, but the concepts of assignments, and variables. And even functions have to be pure with no side-effects, which makes I/O a problem. They are really intent on making life difficult. -- bartc From news at luegenpresse.edu Mon Jan 1 13:28:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 18:28:00 +0000 Subject: ... (Posting On Python-List Prohibited) References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: >> ??? (Posting On Python-List Prohibited) >> ???? why ? > I'm posting to the usenet group comp.lang.python (an off-topic reply to an off-topic remark, but it happens). > I've no idea what the prohibited part is about, if that's what you're posting about. But there have been dozens of other messages with the same subject. that's right. on https://mail.python.org/pipermail/python-list/2018-January/subject.html#start which feeds into comp.lang.python , a lot of messages are missing. Appearantly there is some oppression scheme going on. From breamoreboy at gmail.com Mon Jan 1 13:41:19 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 10:41:19 -0800 (PST) Subject: Copy-on-write friendly Python garbage collection In-Reply-To: References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> Message-ID: <77f94872-c36c-47d2-ab16-f8e40f10d3f1@googlegroups.com> On Monday, January 1, 2018 at 12:53:03 PM UTC, Wu Xi wrote: > breamoreboy: > > On Sunday, December 31, 2017 at 6:19:13 PM UTC, Wu Xi wrote: > >> breamoreboy: > >>> An interesting write up on something that is incorporated into Python 3.7 https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf > >> > >> Appearantly, Erlang is the way to go, when it comes to web frameworks. > > > > What has that got to do with the subject of this thread? > > Well, a little implicitly. Pointing out Erlang superiority in a pythonic newsgroup is, of course, heresy. Python is fourth in the latest TIOBE index, Erlang doesn't even make the top 20, so in what way is it superior? Python doesn't need Pinky and the Brain in its quest to take over the world. -- Kindest regards. Mark Lawrence. From eternal-september at niles.xyz Mon Jan 1 13:49:10 2018 From: eternal-september at niles.xyz (Niles Rogoff) Date: Mon, 1 Jan 2018 18:49:10 -0000 (UTC) Subject: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited) References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> <87608mvifj.fsf@nightsong.com> <0fc798eb-783d-435b-8ddb-228f1d2a205c@googlegroups.com> <004b5ee1-62f5-41ba-9e78-8167c3d66c63@googlegroups.com> Message-ID: On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: > On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm... at gmail.com wrote: >> Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D?Oliveiro a ?crit?: >> > On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul Rubin wrote: >> > > I wonder if things would suffer if they tried a more aggressive >> > > approach and ditched refcounting completely. >> > >> > One thing that would suffer is Python?s well-behaved memory usage. >> > You would need to start imposing heap usage limits, like you do in >> > Java. >> >> Memory: >> >> >>> sys.getsizeof('abcdefghij' + '$') >> 36 >> >>> sys.getsizeof('abcdefghij' + '?') >> 60 >> >>> sys.getsizeof(('abcdefghij' + '?').encode('utf-8')) >> 30 >> >>> sys.getsizeof('abcdefghij' + '\U00010000') >> 84 >> >>> sys.getsizeof(('abcdefghij' + '\U00010000').encode('utf-8')) >> 31 >> >>> >> >>> >> Performance: >> "anti - utf-32" >> >> Buggyness: >> Better to not comment. >> >> Python is the single language, which is presenting the opposite of what >> Unicode.org offers on the side of memory *and* on the side of >> performance, utf-8 *and* utf-32 ! >> >> Happy new year. > He's right though. I would encourage anyone interested to check out http://utf8everywhere.org/ > Your usual drivel. When are you going to stop banging this drum, you've > done nothing else for the past five years? From eternal-september at niles.xyz Mon Jan 1 13:52:47 2018 From: eternal-september at niles.xyz (Niles Rogoff) Date: Mon, 1 Jan 2018 18:52:47 -0000 (UTC) Subject: How to exec a string which has an embedded '\n'? (Posting On Python-List Prohibited) References: <1514697936.456533.1220037056.708483ED@webmail.messagingengine.com> <31175816-d2ca-4e40-a2f3-ad27b63c616c@googlegroups.com> <4391a3d0-1ef2-457f-8044-52fb159f1849@googlegroups.com> Message-ID: > > I don't even dare to read through this code:-( > This kind of code is always fun to write and not at all fun to work with later. I just glanced through my own projects and found lots of code that works similarly (generates html + javascript, and the javascript generates more html). It's pretty ugly https://github.com/nilesr/index/blob/master/generate.py https://github.com/nilesr/formgen/blob/master/form_generator.py From python at mrabarnett.plus.com Mon Jan 1 13:55:51 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 1 Jan 2018 18:55:51 +0000 Subject: Scientific Python moving to Python 3 only. In-Reply-To: References: Message-ID: <844ebb4f-3564-f054-78ef-b98625734928@mrabarnett.plus.com> On 2018-01-01 06:10, Terry Reedy wrote: > https://github.com/numpy/numpy/blob/master/doc/neps/dropping-python2.7-proposal.rst > > Numpy people currently plan to stop adding features to their 2.7 branch > after Dec 31, 2018 and stop adding bug fixes to the last version > supporting 2.7 a year later, Dec 31, 2019. It will remain available for > pip installation, and possible support by other (most likely commercial) > parties. > > http://www.python3statement.org/ > About 30 other projects, including other part of the python-science > stack, have 'pledged' to so something similar. > FTR, I'll be dropping support for the regex module on Python 2 at about the same time. (In practice, there's a lot of code sharing with regex on Python 3, but if it's not a bug on Python 3, then it'll be "Won't fix".) From news at luegenpresse.edu Mon Jan 1 14:16:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 19:16:00 +0000 Subject: web fw performance with Erlang References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> <77f94872-c36c-47d2-ab16-f8e40f10d3f1@googlegroups.com> Message-ID: >> Well, a little implicitly. Pointing out Erlang superiority in a pythonic newsgroup is, of course, heresy. > > Python is fourth in the latest TIOBE index, Erlang doesn't even make the top 20, so in what way is it superior? > > Python doesn't need Pinky and the Brain in its quest to take over the world. Erlang is so fast, they had to ban erlang from competing here: https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=fortune worst loser is the python framework of course From tkadm30 at yandex.com Mon Jan 1 14:21:28 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Mon, 1 Jan 2018 14:21:28 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: <0c4a1c00-5a30-5f27-e75c-e8fb2535c757@yandex.com> So far I found the SWIG method quite good. But I really think that I will try patching CFFI to use libclang natively. Etienne Le 2017-12-29 ? 20:00, Etienne Robillard a ?crit?: > Hi all, > > I would like to extend uWSGI by creating a CPython extension module > for the libuwsgi.so shared library included in the distribution. > > My goal is to use this automatically generated CPython module for > extending uWSGI. > > I have summarized my current approach here: > > https://mail.python.org/pipermail/tutor/2017-December/112475.html > > So far, I have tried to use CFFI and pycparser to generate the Python > bindings, however CFFI and pycparser doesn't handle C directives like > #include and #define. > > I also attempted using clang to preprocess the "uwsgi.h" header found > in the uWSGI distro with pycparser.parse_file() to generate a AST, > however the generated AST object seems incorrect. > > Is there any way to reflect a shared library into a CPython module? > > I know that using "nm -D libuwsgi.so" I can get a list of available > functions for this module. However, I have very little experience with > ctypes and CFFI. > > Do I need to modify CFFI to allow it to parse a C header with libclang? > > I'm guessing CFFI/clang or ctypes should allow me to reflect the > shared library, but I'm not sure about which method is the most > appropriate. > > > What do you think? > > Sincerely, > > Etienne > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From zondervanz at gmail.com Mon Jan 1 15:00:13 2018 From: zondervanz at gmail.com (John Q Hacker) Date: Mon, 1 Jan 2018 20:00:13 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: <9Zz1C.185732$oD1.1720@fx44.am4> References: <9Zz1C.185732$oD1.1720@fx44.am4> Message-ID: >> I don?t use gotos in C code. Why should it be ?harder? in a higher-level >> language? > > Good for you. > > Looking at 14 million lines of Linux kernel sources, which are in C, > over 100,000 of them use 'goto'. About one every 120 lines. Most use of goto's implies a lack of understanding of the unseen architecture of the problem domain itself (otherwise, you wouldn't have structured your program with that architecture). The only remaining use is optimization, and most of that is probably premature, as use of gotos *can* make things hard to understand, but using labels is a pretty happy medium. Marxos From zondervanz at gmail.com Mon Jan 1 15:02:08 2018 From: zondervanz at gmail.com (John Q Hacker) Date: Mon, 1 Jan 2018 20:02:08 +0000 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> Message-ID: Sorry, delete string "n't". I mean that you would strcuture your code with that architecture. Hate that. marxos On 1/1/18, John Q Hacker wrote: >>> I don?t use gotos in C code. Why should it be ?harder? in a higher-level >>> language? >> >> Good for you. >> >> Looking at 14 million lines of Linux kernel sources, which are in C, >> over 100,000 of them use 'goto'. About one every 120 lines. > > Most use of goto's implies a lack of understanding of the unseen > architecture of the problem domain itself (otherwise, you wouldn't > have structured your program with that architecture). The only > remaining use is optimization, and most of that is probably premature, > as use of gotos *can* make things hard to understand, but using labels > is a pretty happy medium. > > Marxos > From cl at isbd.net Mon Jan 1 15:16:09 2018 From: cl at isbd.net (Chris Green) Date: Mon, 1 Jan 2018 20:16:09 +0000 Subject: Goto (Posting On Python-List Prohibited) References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> <20180101145427.4fuwstpo5fhfbql7@hjp.at> <072l4d1j2hmlrthp4bfl4vl2tpidrrrdji@4ax.com> Message-ID: <9hhqhe-qpc.ln1@esprimo.zbmc.eu> Dennis Lee Bieber wrote: > > Well... "break" does bypass the rest of the block, but it still exits > via the end of the block. I have a tendency to try for one "return" per > procedure (so I'm more likely to have an "if ...: break" then "if ...: > return"). I have always tried to enforce 'only one return per function'. If there are multiple returns it makes maintenance very difficult as 'clear up' code can get bypassed. -- Chris Green ? From ned at nedbatchelder.com Mon Jan 1 15:54:07 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 1 Jan 2018 15:54:07 -0500 Subject: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited) In-Reply-To: References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> <87608mvifj.fsf@nightsong.com> <0fc798eb-783d-435b-8ddb-228f1d2a205c@googlegroups.com> <004b5ee1-62f5-41ba-9e78-8167c3d66c63@googlegroups.com> Message-ID: <1bfb501e-d8a0-aa41-8d7f-4bf63179a45f@nedbatchelder.com> On 1/1/18 1:49 PM, Niles Rogoff wrote: > On Mon, 01 Jan 2018 10:42:58 -0800, breamoreboy wrote: > >> On Monday, January 1, 2018 at 10:14:59 AM UTC, wxjm... at gmail.com wrote: >>> Le lundi 1 janvier 2018 08:35:53 UTC+1, Lawrence D?Oliveiro a ?crit?: >>>> On Monday, January 1, 2018 at 7:52:48 AM UTC+13, Paul Rubin wrote: >>>>> I wonder if things would suffer if they tried a more aggressive >>>>> approach and ditched refcounting completely. >>>> One thing that would suffer is Python?s well-behaved memory usage. >>>> You would need to start imposing heap usage limits, like you do in >>>> Java. >>> Memory: >>> >>>>>> sys.getsizeof('abcdefghij' + '$') >>> 36 >>>>>> sys.getsizeof('abcdefghij' + '?') >>> 60 >>>>>> sys.getsizeof(('abcdefghij' + '?').encode('utf-8')) >>> 30 >>>>>> sys.getsizeof('abcdefghij' + '\U00010000') >>> 84 >>>>>> sys.getsizeof(('abcdefghij' + '\U00010000').encode('utf-8')) >>> 31 >>>>>> >>> Performance: >>> "anti - utf-32" >>> >>> Buggyness: >>> Better to not comment. >>> >>> Python is the single language, which is presenting the opposite of what >>> Unicode.org offers on the side of memory *and* on the side of >>> performance, utf-8 *and* utf-32 ! >>> >>> Happy new year. > He's right though. I would encourage anyone interested to check out > http://utf8everywhere.org/ > Niles, if you want to claim wxjmfauth is right, you'll have to present some actual evidence.? He's claimed for years that Python's Unicode support is buggy (as he does here), without ever demonstrating a bug.? We've long ago tired of trying to reason with him. The tradeoffs of memory use for algorithmic complexity are well understood, and have been endlessly discussed. There is not an "obviously right" answer to how to make those tradeoffs. --Ned. From tjreedy at udel.edu Mon Jan 1 16:19:54 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 1 Jan 2018 16:19:54 -0500 Subject: ... (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: On 1/1/2018 1:28 PM, Wu Xi wrote: > on > https://mail.python.org/pipermail/python-list/2018-January/subject.html#start > which feeds into comp.lang.python , > a lot of messages are missing. Appearantly there is some oppression scheme going on. Blocking of spamming and trolling prevents oppression of people who want to use the list, funded by PSF, for its purpose, discussion of Python. -- Terry Jan Reedy From news at luegenpresse.edu Mon Jan 1 16:27:00 2018 From: news at luegenpresse.edu (Wu Xi) Date: Mon, 01 Jan 2018 21:27:00 +0000 Subject: ... (Posting On Python-List Prohibited) References: <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: > Blocking of spamming and trolling prevents oppression of people who want to use the list, funded by PSF, for its purpose, discussion of Python. why are PSF funds privileged over anybody else's fund, which has zero privilege? From rosuav at gmail.com Mon Jan 1 16:34:45 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Jan 2018 08:34:45 +1100 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: <9hhqhe-qpc.ln1@esprimo.zbmc.eu> References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> <20180101145427.4fuwstpo5fhfbql7@hjp.at> <072l4d1j2hmlrthp4bfl4vl2tpidrrrdji@4ax.com> <9hhqhe-qpc.ln1@esprimo.zbmc.eu> Message-ID: On Tue, Jan 2, 2018 at 7:16 AM, Chris Green wrote: > Dennis Lee Bieber wrote: >> >> Well... "break" does bypass the rest of the block, but it still exits >> via the end of the block. I have a tendency to try for one "return" per >> procedure (so I'm more likely to have an "if ...: break" then "if ...: >> return"). > > I have always tried to enforce 'only one return per function'. If > there are multiple returns it makes maintenance very difficult as > 'clear up' code can get bypassed. > Isn't that why try/finally exists? No matter how many 'return' statements you have, there's always exceptions to bypass any naive cleanup code; and no matter how many returns you have, 'finally' blocks still execute before return. ChrisA From rosuav at gmail.com Mon Jan 1 16:37:22 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Jan 2018 08:37:22 +1100 Subject: ... (Posting On Python-List Prohibited) In-Reply-To: References: <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: On Tue, Jan 2, 2018 at 8:27 AM, Wu Xi wrote: > >> Blocking of spamming and trolling prevents oppression of people who want to use the list, funded by PSF, for its purpose, discussion of Python. > > why are PSF funds privileged over anybody else's fund, which has zero privilege? If you want to host your own mailing list on your own server, you're welcome to. All the software you need is open source. (In fact, I can and do host several mailing lists, though not for the discussion of Python.) ChrisA From hjp-python at hjp.at Mon Jan 1 16:39:33 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Mon, 1 Jan 2018 22:39:33 +0100 Subject: ... (Posting On Python-List Prohibited) In-Reply-To: References: Message-ID: <20180101213933.tsxzee247wzezqxj@hjp.at> On 2018-01-01 21:27:00 +0000, Wu Xi wrote: > > Blocking of spamming and trolling prevents oppression of people who > > want to use the list, funded by PSF, for its purpose, discussion of > > Python. > > why are PSF funds privileged over anybody else's fund, which has zero > privilege? Because they pay for the server which runs the list. Their server, their rules. If you set up a server which hosts a mailing list, a web forum, or whatever, you get to decide ther rules for your server. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From breamoreboy at gmail.com Mon Jan 1 17:11:13 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 14:11:13 -0800 (PST) Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> <20180101145427.4fuwstpo5fhfbql7@hjp.at> <072l4d1j2hmlrthp4bfl4vl2tpidrrrdji@4ax.com> <9hhqhe-qpc.ln1@esprimo.zbmc.eu> Message-ID: <3b1bfc06-4e2f-49ee-9db8-e9eec747072d@googlegroups.com> On Monday, January 1, 2018 at 9:35:06 PM UTC, Chris Angelico wrote: > On Tue, Jan 2, 2018 at 7:16 AM, Chris Green wrote: > > Dennis Lee Bieber wrote: > >> > >> Well... "break" does bypass the rest of the block, but it still exits > >> via the end of the block. I have a tendency to try for one "return" per > >> procedure (so I'm more likely to have an "if ...: break" then "if ...: > >> return"). > > > > I have always tried to enforce 'only one return per function'. If > > there are multiple returns it makes maintenance very difficult as > > 'clear up' code can get bypassed. > > > > Isn't that why try/finally exists? No matter how many 'return' > statements you have, there's always exceptions to bypass any naive > cleanup code; and no matter how many returns you have, 'finally' > blocks still execute before return. > > ChrisA What happened to context managers? -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Jan 1 17:12:23 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 14:12:23 -0800 (PST) Subject: ... (Posting On Python-List Prohibited) In-Reply-To: References: <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> Message-ID: <619cbf9d-66f5-4614-bd18-f497499c48b4@googlegroups.com> On Monday, January 1, 2018 at 9:28:01 PM UTC, Wu Xi wrote: > > Blocking of spamming and trolling prevents oppression of people who want to use the list, funded by PSF, for its purpose, discussion of Python. > > why are PSF funds privileged over anybody else's fund, which has zero privilege? Congratulations, after just a couple of days you're all ready pushing for a place in my dream team. -- Kindest regards. Mark Lawrence. From beliavsky at aol.com Mon Jan 1 17:23:08 2018 From: beliavsky at aol.com (beliavsky at aol.com) Date: Mon, 1 Jan 2018 14:23:08 -0800 (PST) Subject: Humble Book Bundle: Python by Packt Message-ID: One can purchase the following Python books and videos published by Packt for $15 at https://www.humblebundle.com/books/python-by-packt-book-bundle for about the next two weeks. Python Data Analysis Cookbook Mastering Python, Second Edition Learning Robotics using Python Python Programming with Raspberry Pi Web Development with Django Cookbook Expert Python Programming, Second Edition Learning Python Web Penetration Testing (Video) Python Data Science Essentials, Second Edition Learning Concurrency in Python Python Data Structures and Algorithms Beginning Python (Video) Building RESTful Python Web Services Mastering Python Networking Artificial Intelligence with Python Deep Learning with Python (Video) Python Machine Learning Projects (Video) Python Machine Learning Python Microservices Development Python Design Patterns (Video) Software Architecture with Python Modern Python Cookbook Python High Performance, Second Edition Python GUI Programming Cookbook, Second Edition Mastering Python (Video) From breamoreboy at gmail.com Mon Jan 1 17:27:22 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 14:27:22 -0800 (PST) Subject: stop prohibition of comp.lang.python ! In-Reply-To: References: Message-ID: <9b8d12b8-a4ac-46f4-ae5c-9d8b3a1e2bb5@googlegroups.com> On Monday, January 1, 2018 at 3:00:19 PM UTC, S. I. wrote: > stop prohibition of comp.lang.python ! > > it is childish to do this prohibition business ! > > don't you have spam filters ? The prohibition part of the subject line is added by Lawrence D'Oliveiro when he posts on google groups as he's been banned from the mailing list. Most people won't see it until someone replies to his post via their email to the list. -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Jan 1 17:34:45 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 14:34:45 -0800 (PST) Subject: ... (Posting On Python-List Prohibited) In-Reply-To: References: <8yV1C.295429$3J3.166915@fx37.am4> <87d12vc556.fsf@bsb.me.uk> <871sjaczxv.fsf@bsb.me.uk> <619cbf9d-66f5-4614-bd18-f497499c48b4@googlegroups.com> Message-ID: <21d7fe7f-dad8-464d-a783-28c7fa611a43@googlegroups.com> On Monday, January 1, 2018 at 10:21:15 PM UTC, P. timoriensis wrote: > >>> Blocking of spamming and trolling prevents oppression of people who want to use the list, funded by PSF, for its purpose, discussion of Python. > >> > >> why are PSF funds privileged over anybody else's fund, which has zero privilege? > > > > Congratulations, after just a couple of days you're all ready pushing for a place in my dream team. > > you sure would make for just another one of those self-appointed censors, who won't react to legit questions, when they realize how dangerous they are. I don't see a legitimate question, I see a completely stupid question. The PSF pays the money, the PSF takes their choice. If you don't like that you're perfectly free to use reddit, stackoverflow or any other forum. -- Kindest regards. Mark Lawrence. From breamoreboy at gmail.com Mon Jan 1 17:44:15 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 14:44:15 -0800 (PST) Subject: =?UTF-8?B?UmU6IOKcqPCfjbDinKggcHl0aG9uIDIwMTggd2lraSAtIGEgcGllY2Ugb2YgY2FrZSA=?= =?UTF-8?B?4pyo8J+NsOKcqA==?= In-Reply-To: References: Message-ID: <8cf390d9-4f32-46ba-bf7f-f7294418dd45@googlegroups.com> On Monday, January 1, 2018 at 2:54:26 PM UTC, S. I. wrote: > https://practical-scheme.net/wiliki/wiliki.cgi?python > > no register, no nothing ! just edit. > > ??? python - a piece of cake ??? > > just edit or enter a code.py entry with > > {{{ > > print(" oh yes, 2018 ") > > }}} >From the link "Look at those scumbag admins over at the py wiki.", "what utter pieces of s., man: https://mail.python.org/pipermail/pydotorg-www/2017-June/004316.html", "the scum admins are waging another wiki war, follow it here: https://mail.python.org/pipermail/pydotorg-www/", "more Lemburg-type bs:" and "Nothing but scum in the py wiki". Were you the author of "How To Win Friends And Influence People"? -- Kindest regards. Mark Lawrence. From rosuav at gmail.com Mon Jan 1 18:23:00 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Jan 2018 10:23:00 +1100 Subject: Goto (Posting On Python-List Prohibited) In-Reply-To: <3b1bfc06-4e2f-49ee-9db8-e9eec747072d@googlegroups.com> References: <9Zz1C.185732$oD1.1720@fx44.am4> <20171230124614.kv32v54rkg3yv5gz@hjp.at> <20180101145427.4fuwstpo5fhfbql7@hjp.at> <072l4d1j2hmlrthp4bfl4vl2tpidrrrdji@4ax.com> <9hhqhe-qpc.ln1@esprimo.zbmc.eu> <3b1bfc06-4e2f-49ee-9db8-e9eec747072d@googlegroups.com> Message-ID: On Tue, Jan 2, 2018 at 9:11 AM, wrote: > On Monday, January 1, 2018 at 9:35:06 PM UTC, Chris Angelico wrote: >> On Tue, Jan 2, 2018 at 7:16 AM, Chris Green wrote: >> > Dennis Lee Bieber wrote: >> >> >> >> Well... "break" does bypass the rest of the block, but it still exits >> >> via the end of the block. I have a tendency to try for one "return" per >> >> procedure (so I'm more likely to have an "if ...: break" then "if ...: >> >> return"). >> > >> > I have always tried to enforce 'only one return per function'. If >> > there are multiple returns it makes maintenance very difficult as >> > 'clear up' code can get bypassed. >> > >> >> Isn't that why try/finally exists? No matter how many 'return' >> statements you have, there's always exceptions to bypass any naive >> cleanup code; and no matter how many returns you have, 'finally' >> blocks still execute before return. >> >> ChrisA > > What happened to context managers? > They're a way of wrapping up repeatedly-used try/finally blocks. The try/finally construct is the underlying functionality. In any case, the same behaviour can be used by either, so yes, if you have cleanup code and you're using 'with x:' rather than try/finally, it still fits into the same concept that I'm describing, and still guarantees that the cleanup code is executed before return. (Yes, I know it's possible to bypass finally blocks. But the methods for doing so (eg hard-aborting) also bypass the actual return, so the "before return" part is still valid. If you're doing things that depend on cleanup outside the process, like deleting temporary files, you'll need something more than try/finally. But in that case, there's no other code layout that would help you anyway.) ChrisA From breamoreboy at gmail.com Mon Jan 1 18:30:21 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 1 Jan 2018 15:30:21 -0800 (PST) Subject: stop prohibition of comp.lang.python ! D'Oliveiro should wear this as a badge of honour. In-Reply-To: References: <9b8d12b8-a4ac-46f4-ae5c-9d8b3a1e2bb5@googlegroups.com> Message-ID: <19d84b14-045e-4f21-a8ca-0b5cad7eea43@googlegroups.com> On Monday, January 1, 2018 at 11:06:30 PM UTC, P. timoriensis wrote: > >> stop prohibition of comp.lang.python ! > >> > >> it is childish to do this prohibition business ! > >> > >> don't you have spam filters ? > > > > The prohibition part of the subject line is added by Lawrence D'Oliveiro when he posts on google groups as he's been banned from the mailing list. Most people won't see it until someone replies to his post via their email to the list. > > > Mark Lawrence. > > these cases are always the same: the censors in 99% of the cases are utterly wrong. > > > D'Oliveiro must be a good man, proof is here. > I know what to think about python-list censorship already. Like the case of the guy who was banned as he was using the list to spread antisemitic drivel through his signatures? If you don't like this facility you are perfectly welcome to leave and never come back. -- Kindest regards. Mark Lawrence. From rosuav at gmail.com Mon Jan 1 18:43:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Jan 2018 10:43:05 +1100 Subject: =?UTF-8?B?UmU6IOKcqPCfjbDinKggcHl0aG9uIDIwMTggd2lraSAtIGEgcGllY2Ugb2YgY2FrZSA=?= =?UTF-8?B?4pyo8J+NsOKcqA==?= In-Reply-To: <8cf390d9-4f32-46ba-bf7f-f7294418dd45@googlegroups.com> References: <8cf390d9-4f32-46ba-bf7f-f7294418dd45@googlegroups.com> Message-ID: On Tue, Jan 2, 2018 at 9:44 AM, wrote: > On Monday, January 1, 2018 at 2:54:26 PM UTC, S. I. wrote: >> https://practical-scheme.net/wiliki/wiliki.cgi?python >> >> no register, no nothing ! just edit. >> >> ??? python - a piece of cake ??? >> >> just edit or enter a code.py entry with >> >> {{{ >> >> print(" oh yes, 2018 ") >> >> }}} > > From the link "Look at those scumbag admins over at the py wiki.", "what utter pieces of s., man: https://mail.python.org/pipermail/pydotorg-www/2017-June/004316.html", "the scum admins are waging another wiki war, follow it here: https://mail.python.org/pipermail/pydotorg-www/", "more Lemburg-type bs:" and "Nothing but scum in the py wiki". Were you the author of "How To Win Friends And Influence People"? > Just FYI, the posts you're quoting here aren't making it to my inbox. I believe they're getting blocked at the news/mail gateway (though it could be elsewhere, for all I know). If you ignore them, we here in the tame land of the mailing list would be cheerfully unaware of the horrors ravaging the Wild West of the newsgroup. ChrisA From eternal-september at niles.xyz Mon Jan 1 19:03:38 2018 From: eternal-september at niles.xyz (Niles Rogoff) Date: Tue, 2 Jan 2018 00:03:38 -0000 (UTC) Subject: Copy-on-write friendly Python garbage collection (Posting On Python-List Prohibited) References: <10b9cfba-7611-4544-ae83-62969c47dcf4@googlegroups.com> <87608mvifj.fsf@nightsong.com> <0fc798eb-783d-435b-8ddb-228f1d2a205c@googlegroups.com> <004b5ee1-62f5-41ba-9e78-8167c3d66c63@googlegroups.com> <1bfb501e-d8a0-aa41-8d7f-4bf63179a45f@nedbatchelder.com> Message-ID: > Niles, if you want to claim wxjmfauth is right, you'll have to present > some actual evidence.? He's claimed for years that Python's Unicode > support is buggy (as he does here), without ever demonstrating a bug. > We've long ago tired of trying to reason with him. > > The tradeoffs of memory use for algorithmic complexity are well > understood, and have been endlessly discussed. There is not an > "obviously right" answer to how to make those tradeoffs. > > --Ned. Aah, I didn't know, I'm new here. I thought he was referring to UTF-16 being inefficient, which is mostly what I was agreeing with. From james at uplinkzero.com Tue Jan 2 05:06:07 2018 From: james at uplinkzero.com (James Chapman) Date: Tue, 2 Jan 2018 10:06:07 +0000 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: Hi Etienne, I'm not familiar with uSWGI, so I started here: https://uwsgi-docs.readthedocs.io/en/latest/Hooks.html AFAIK there are a number of hooks already exposed and available to Python apps. However, if I've understood your question correctly, you want to extend uWSGI itself by writing some python code which is then translated/generated into C code? You refer to AST generation but why are you doing this? Maybe my unfamiliarity with uWSGI is preventing me from understanding exactly what it is you are trying to do, but further clarification would go a long way. If your goal is actually to extend uWSGI via a plugin, then there's more to just writing some python, you'll need to write some C code (I wouldn't bother trying to generate this as it'll probably not work): https://github.com/unbit/uwsgi-docs/blob/master/tutorials/WritingPlugins.rst If you look here, there's a link to each 3rd party plugin (mostly GitHub) where you can see the source code for each plugin. This is a very good starting point as there are many examples here: https://uwsgi-docs.readthedocs.io/en/latest/ThirdPartyPlugins.html Perhaps you can clarify. James On 30 December 2017 at 01:00, Etienne Robillard wrote: > Hi all, > > I would like to extend uWSGI by creating a CPython extension module for > the libuwsgi.so shared library included in the distribution. > > My goal is to use this automatically generated CPython module for > extending uWSGI. > > From tkadm30 at yandex.com Tue Jan 2 05:17:39 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 2 Jan 2018 05:17:39 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: Hi James, Thanks for your input. I want to make a native Python application (uwsgictl) to dispatch FIFO commands to the master uWSGI process. uwsgictl would depend on libuwsgi.so, a uWSGI plugin included in the distribution. My goal is to use CFFI to generate python bindings for libuwsgi.so using clang interpreter. :) Best regards, Etienne Le 2018-01-02 ? 05:06, James Chapman a ?crit?: > Hi Etienne, > > I'm not familiar with uSWGI, so I started here: > https://uwsgi-docs.readthedocs.io/en/latest/Hooks.html > AFAIK there are a number of hooks already exposed and available to > Python apps. > > However, if I've understood your question correctly, you want to > extend uWSGI itself by writing some python code which is then > translated/generated into C code? You refer to AST generation but why > are you doing this? Maybe my unfamiliarity with uWSGI is preventing me > from understanding exactly what it is you are trying to do, but > further clarification would go a long way. > > If your goal is actually to extend uWSGI via a plugin, then there's > more to just writing some python, you'll need to write some C code (I > wouldn't bother trying to generate this as it'll probably not work): > https://github.com/unbit/uwsgi-docs/blob/master/tutorials/WritingPlugins.rst > > If you look here, there's a link to each 3rd party plugin (mostly > GitHub) where you can see the source code for each plugin. This is a > very good starting point as there are many examples here: > https://uwsgi-docs.readthedocs.io/en/latest/ThirdPartyPlugins.html > > Perhaps you can clarify. > > James > > > > > On 30 December 2017 at 01:00, Etienne Robillard > wrote: > > Hi all, > > I would like to extend uWSGI by creating a CPython extension > module for the libuwsgi.so shared library included in the > distribution. > > My goal is to use this automatically generated CPython module for > extending uWSGI. > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From tkadm30 at yandex.com Tue Jan 2 06:44:51 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 2 Jan 2018 06:44:51 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: Here's my ffi_build.py script: https://bitbucket.org/tkadm30/libuwsgictl/src/956f6ca24f9111c1d8b3ce90cf17173a6e5ae3e2/ffi_build.py?at=default&fileviewer=file-view-default Not really sure how to preprocess the uwsgi.h header with clang.cindex in cffi. I can load the shared lib with : >>> lib = clang.cindex.cdll.LoadLibrary(c_lib) Any ideas how to compile lib into a python extension module with cffi? Etienne Le 2018-01-02 ? 05:17, Etienne Robillard a ?crit?: > Hi James, > > Thanks for your input. > > I want to make a native Python application (uwsgictl) to dispatch FIFO > commands to the master uWSGI process. > > uwsgictl would depend on libuwsgi.so, a uWSGI plugin included in the > distribution. > > My goal is to use CFFI to generate python bindings for libuwsgi.so > using clang interpreter. :) > > > Best regards, > > Etienne > > > Le 2018-01-02 ? 05:06, James Chapman a ?crit?: >> Hi Etienne, >> >> I'm not familiar with uSWGI, so I started here: >> https://uwsgi-docs.readthedocs.io/en/latest/Hooks.html >> AFAIK there are a number of hooks already exposed and available to >> Python apps. >> >> However, if I've understood your question correctly, you want to >> extend uWSGI itself by writing some python code which is then >> translated/generated into C code? You refer to AST generation but why >> are you doing this? Maybe my unfamiliarity with uWSGI is preventing >> me from understanding exactly what it is you are trying to do, but >> further clarification would go a long way. >> >> If your goal is actually to extend uWSGI via a plugin, then there's >> more to just writing some python, you'll need to write some C code (I >> wouldn't bother trying to generate this as it'll probably not work): >> https://github.com/unbit/uwsgi-docs/blob/master/tutorials/WritingPlugins.rst >> >> >> If you look here, there's a link to each 3rd party plugin (mostly >> GitHub) where you can see the source code for each plugin. This is a >> very good starting point as there are many examples here: >> https://uwsgi-docs.readthedocs.io/en/latest/ThirdPartyPlugins.html >> >> Perhaps you can clarify. >> >> James >> >> >> >> >> On 30 December 2017 at 01:00, Etienne Robillard > > wrote: >> >> ??? Hi all, >> >> ??? I would like to extend uWSGI by creating a CPython extension >> ??? module for the libuwsgi.so shared library included in the >> ??? distribution. >> >> ??? My goal is to use this automatically generated CPython module for >> ??? extending uWSGI. >> >> > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From james at uplinkzero.com Tue Jan 2 09:30:06 2018 From: james at uplinkzero.com (James Chapman) Date: Tue, 2 Jan 2018 14:30:06 +0000 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: I have an example of loading and communicating with a dynamic library using ctypes in Windows here: https://github.com/James-Chapman/python-code-snippets/tree/master/DLL_C_funcs_w_callbacks It shouldn't be too dissimilar on Linux. What starts uWSGI? Is it started from a Python application or a webserver? Apologies for my lack of knowledge RE uWSGI. > Any ideas how to compile lib into a python extension module with cffi? No. Do you mean python binary extension? ( https://packaging.python.org/guides/packaging-binary-extensions/) Or Python package? (https://python-packaging.readthedocs.io/en/latest/) James From robin at reportlab.com Tue Jan 2 09:30:42 2018 From: robin at reportlab.com (Robin Becker) Date: Tue, 2 Jan 2018 14:30:42 +0000 Subject: unicode direction control characters Message-ID: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> I'm seeing some strange characters in web responses eg u'\u200e28\u200e/\u200e09\u200e/\u200e1962' for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to unicodedata.name. I tried unicodedata.normalize, but it leaves those characters there. Is there any standard way to deal with these? I assume that some browser+settings combination is putting these in eg perhaps the language is normally right to left but numbers are not. -- Robin Becker From bc at freeuk.com Tue Jan 2 09:51:32 2018 From: bc at freeuk.com (bartc) Date: Tue, 2 Jan 2018 14:51:32 +0000 Subject: Goto In-Reply-To: References: <7Mb1C.278680$gi3.141689@fx19.am4> Message-ID: On 29/12/2017 18:11, Chris Angelico wrote: > On Sat, Dec 30, 2017 at 4:13 AM, bartc wrote: >> If you want to translate code from one language to another, and the source >> language uses gotos, or uses control structures not available in the target >> language, then gotos would be very useful in the latter. >> > > As has already been said in this thread, a competent FORTRAN > programmer can write FORTRAN code in any language. But if you want to > actually write Python code, instead of writing FORTRAN code that gets > run through a Python interpreter, you have to grok the Python way of > doing things. That's not true. People should be able to write code as they like using the nearest available language and using the style they are most adept in and that they find comfortable. /Especially/ if they are coding for themselves rather as part of some organisation or within a team. The choice of language may enforce some changes, but that should be the extent of it. A lot of the so-called 'Pythonic' style I frankly don't care for. I like to write code in a simple, clean, universal style that everyone can understand. That doesn't mean it has to look like Fortran. > You can't do that by mechanically transforming source > code into the syntax of a different language. (Actually that's exactly what a project of mine from a few years ago was trying to do. Example of 'binary trees' benchmark: source syntax: https://pastebin.com/raw/bgfKw2pq generated Python: https://pastebin.com/raw/XJWjWzpw) Here, you normally had to be aware when writing in the source language, of what target language you were using. Although sometimes exactly the same source translated to several targets including Python and Lisp. Anyway, this was unsatisfactory as a universal syntax that could be translated to any language.) > People who try to tell you that all programming languages are > identical, just with different syntax, are doing you a disservice. > Lisp is not Python is not REXX is not C is not DeScribe Macro > Language. (Especially the latter. Do NOT try to write C code in DML. > It'll look ugly. Trust me. My experiment showed that such people have a point. -- bartc From tkadm30 at yandex.com Tue Jan 2 10:17:44 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 2 Jan 2018 10:17:44 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: Hi James, Le 2018-01-02 ? 09:30, James Chapman a ?crit?: > > > What starts uWSGI? Is it started from a Python application or a > webserver? Apologies for my lack of knowledge RE uWSGI. > uWSGI is typically managed by the web server. > > > Any ideas how to compile lib into a python extension module with cffi? > > No. Do you mean python binary extension? > (https://packaging.python.org/guides/packaging-binary-extensions/) > Or Python package? (https://python-packaging.readthedocs.io/en/latest/) > I just want CFFI to work with clang to produce automagically the python bindings for my app. > James > > Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Tue Jan 2 10:18:14 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Jan 2018 02:18:14 +1100 Subject: unicode direction control characters In-Reply-To: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> References: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> Message-ID: On Wed, Jan 3, 2018 at 1:30 AM, Robin Becker wrote: > I'm seeing some strange characters in web responses eg > > u'\u200e28\u200e/\u200e09\u200e/\u200e1962' > > for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to > unicodedata.name. I tried unicodedata.normalize, but it leaves those > characters there. Is there any standard way to deal with these? > > I assume that some browser+settings combination is putting these in eg > perhaps the language is normally right to left but numbers are not. Unicode normalization is a different beast altogether. You could probably just remove the LTR marks and run with the rest, though, as they don't seem to be important in this string. ChrisA From rosuav at gmail.com Tue Jan 2 10:20:04 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Jan 2018 02:20:04 +1100 Subject: Goto In-Reply-To: References: <7Mb1C.278680$gi3.141689@fx19.am4> Message-ID: On Wed, Jan 3, 2018 at 1:51 AM, bartc wrote: > On 29/12/2017 18:11, Chris Angelico wrote: >> >> On Sat, Dec 30, 2017 at 4:13 AM, bartc wrote: >>> >>> If you want to translate code from one language to another, and the >>> source >>> language uses gotos, or uses control structures not available in the >>> target >>> language, then gotos would be very useful in the latter. >>> >> >> As has already been said in this thread, a competent FORTRAN >> programmer can write FORTRAN code in any language. But if you want to >> actually write Python code, instead of writing FORTRAN code that gets >> run through a Python interpreter, you have to grok the Python way of >> doing things. > > > That's not true. People should be able to write code as they like using the > nearest available language and using the style they are most adept in and > that they find comfortable. /Especially/ if they are coding for themselves > rather as part of some organisation or within a team. > > The choice of language may enforce some changes, but that should be the > extent of it. > > A lot of the so-called 'Pythonic' style I frankly don't care for. > > I like to write code in a simple, clean, universal style that everyone can > understand. > > That doesn't mean it has to look like Fortran. Why are you using a Python interpreter then? Why are you here on python-list, talking about Python? There must be SOME reason for using this language rather than another. ChrisA From robin at reportlab.com Tue Jan 2 10:36:10 2018 From: robin at reportlab.com (Robin Becker) Date: Tue, 2 Jan 2018 15:36:10 +0000 Subject: unicode direction control characters In-Reply-To: References: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> Message-ID: On 02/01/2018 15:18, Chris Angelico wrote: > On Wed, Jan 3, 2018 at 1:30 AM, Robin Becker wrote: >> I'm seeing some strange characters in web responses eg >> >> u'\u200e28\u200e/\u200e09\u200e/\u200e1962' >> >> for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to >> unicodedata.name. I tried unicodedata.normalize, but it leaves those >> characters there. Is there any standard way to deal with these? >> >> I assume that some browser+settings combination is putting these in eg >> perhaps the language is normally right to left but numbers are not. > > Unicode normalization is a different beast altogether. You could > probably just remove the LTR marks and run with the rest, though, as > they don't seem to be important in this string. > > ChrisA > I guess I'm really wondering whether the BIDI control characters have any semantic meaning. Most numbers seem to be LTR. If I saw u'\u200f12' it seems to imply that the characters should be displayed '21', but I don't know whether the number is 12 or 21. -- Robin Becker From james at uplinkzero.com Tue Jan 2 10:45:48 2018 From: james at uplinkzero.com (James Chapman) Date: Tue, 2 Jan 2018 15:45:48 +0000 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: Again, apologies if I've dumbed this down, but if I understand this all correctly... The webserver starts uWSGI, the python application running within then does stuff via uWSGI. You want one of those things to be handled by a uWSGI extension that has been written in python, with CFFI, rather than in pure C. In other words, CFFI is going to generate the C part of the extension. Unfortunately I'm no CFFI expert, but the above sounds like something that might be better broken down into 2 parts. 1) Writing uWSGI extensions in Python and posted to a uWSGI development forum. 2) Generating python bindings for C calls with CFFI and clang and posted to a CFFI forum. (Why clang? GCC/G++ will work just as well and may be the default).? Hope that helps. James From rosuav at gmail.com Tue Jan 2 10:55:23 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Jan 2018 02:55:23 +1100 Subject: unicode direction control characters In-Reply-To: References: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> Message-ID: On Wed, Jan 3, 2018 at 2:36 AM, Robin Becker wrote: > On 02/01/2018 15:18, Chris Angelico wrote: >> >> On Wed, Jan 3, 2018 at 1:30 AM, Robin Becker wrote: >>> >>> I'm seeing some strange characters in web responses eg >>> >>> u'\u200e28\u200e/\u200e09\u200e/\u200e1962' >>> >>> for a date of birth. The code \u200e is LEFT-TO-RIGHT MARK according to >>> unicodedata.name. I tried unicodedata.normalize, but it leaves those >>> characters there. Is there any standard way to deal with these? >>> >>> I assume that some browser+settings combination is putting these in eg >>> perhaps the language is normally right to left but numbers are not. >> >> >> Unicode normalization is a different beast altogether. You could >> probably just remove the LTR marks and run with the rest, though, as >> they don't seem to be important in this string. >> >> ChrisA >> > I guess I'm really wondering whether the BIDI control characters have any > semantic meaning. Most numbers seem to be LTR. > > If I saw u'\u200f12' it seems to imply that the characters should be > displayed '21', but I don't know whether the number is 12 or 21. > In this particular situation, it's highly unlikely that they'll have any influence, and even if they do, I don't think there's any way for the *repeated* directionality markers to do anything. They look like something added automatically for the sake of paranoia. ChrisA From jason at apkudo.com Tue Jan 2 11:59:13 2018 From: jason at apkudo.com (jason at apkudo.com) Date: Tue, 2 Jan 2018 08:59:13 -0800 (PST) Subject: Where did csv.parser() go? Message-ID: <5c51c86d-54a6-4ace-8b5d-cef48b6897dd@googlegroups.com> I need record the starting offsets of csv rows in a database for fast seeking later. Unfortunately, using any csv.reader() (or DictReader) tries to cache, which means: example_Data = "'data 0123456789ABCDE 1123456789ABCDE 2123456789ABCDE 3123456789ABCDE ... ''' for line in reader: offsets[row] = f.tell() is not possible. With my example data , offsets are reported as [0, 260, 260, 260...] they should be [0x00, 0x00,0x15, 0x25, ...] (sample data is 16 byte rows after a 5 byte header (just for now)) I saw in one of PEP-305's references a mention of csv.parser() which won't return a row until parsing is complete. This is ideal since some lines will have quoted text containing commas and new lines. I don't want to re-write the parser, since later usage will use csvDictReader, so we need to identically parse rows. How can I do that with the Python 2.7 csv module? Or how can I accomplish this task through other means? From rustompmody at gmail.com Tue Jan 2 12:24:18 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 2 Jan 2018 09:24:18 -0800 (PST) Subject: Numpy and Terabyte data Message-ID: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> Someone who works in hadoop asked me: If our data is in terabytes can we do statistical (ie numpy pandas etc) analysis on it? I said: No (I dont think so at least!) ie I expect numpy (pandas etc) to not work if the data does not fit in memory Well sure *python* can handle (streams of) terabyte data I guess *numpy* cannot Is there a more sophisticated answer? ["Terabyte" is a just a figure of speech for "too large for main memory"] From bc at freeuk.com Tue Jan 2 12:34:06 2018 From: bc at freeuk.com (bartc) Date: Tue, 2 Jan 2018 17:34:06 +0000 Subject: Goto In-Reply-To: References: <7Mb1C.278680$gi3.141689@fx19.am4> Message-ID: On 02/01/2018 15:20, Chris Angelico wrote: > On Wed, Jan 3, 2018 at 1:51 AM, bartc wrote: >> I like to write code in a simple, clean, universal style that everyone can >> understand. >> >> That doesn't mean it has to look like Fortran. > > Why are you using a Python interpreter then? Why are you here on > python-list, talking about Python? There must be SOME reason for using > this language rather than another. I'm more familiar with Python than any other like languages. If I desperately needed a dynamic language and I couldn't use my own, then I would use Python. But I would need to use it on my own terms, regardless of whether the result is 'pythonic'. If the language had 'goto', then I would use it if I found it apt. However, the discussion here is academic, so it doesn't matter who uses what. Apart from anything else, Python is never going to officially adopt 'goto'. (Regardless of some version of it being available as an add-on library. Most things seem to be.) -- bartc From __peter__ at web.de Tue Jan 2 12:43:30 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 02 Jan 2018 18:43:30 +0100 Subject: Where did csv.parser() go? References: <5c51c86d-54a6-4ace-8b5d-cef48b6897dd@googlegroups.com> Message-ID: jason at apkudo.com wrote: > I need record the starting offsets of csv rows in a database for fast > seeking later. Unfortunately, using any csv.reader() (or DictReader) tries > to cache, which means: example_Data = "'data > 0123456789ABCDE > 1123456789ABCDE > 2123456789ABCDE > 3123456789ABCDE > ... > ''' > > for line in reader: > offsets[row] = f.tell() > > is not possible. With my example data , offsets are reported as [0, 260, > 260, 260...] they should be [0x00, 0x00,0x15, 0x25, ...] (sample data is > 16 byte rows after a 5 byte header (just for now)) > > I saw in one of PEP-305's references a mention of csv.parser() which won't > return a row until parsing is complete. This is ideal since some lines > will have quoted text containing commas and new lines. I don't want to > re-write the parser, since later usage will use csvDictReader, so we need > to identically parse rows. How can I do that with the Python 2.7 csv > module? > > Or how can I accomplish this task through other means? It's not the reader that performs the caching it's iteration over the file: $ python -c 'f = open("tmp.csv") > for line in f: print f.tell() > ' 73 73 73 73 73 73 You can work around that by using the file's readline() method: $ python -c 'f = open("tmp.csv") for line in iter(f.readline, ""): print f.tell() ' 5 21 37 53 69 73 Combined with csv.reader(): $ python -c 'import csv; f = open("tmp.csv") for row in csv.reader(iter(f.readline, "")): print f.tell(), row ' 5 ['data'] 21 ['0123456789ABCDE'] 37 ['1123456789ABCDE'] 53 ['2123456789ABCDE'] 69 ['3123456789ABCDE'] 73 ['...'] From jason at apkudo.com Tue Jan 2 13:06:02 2018 From: jason at apkudo.com (jason at apkudo.com) Date: Tue, 2 Jan 2018 10:06:02 -0800 (PST) Subject: Numpy and Terabyte data In-Reply-To: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> References: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> Message-ID: I'm not sure if I'll be laughed at, but a statistical sampling of a randomized sample should resemble the whole. If you need min/max then min ( min(each node) ) If you need average then you need sum( sum(each node)) sum(count(each node))* *You'll likely need to use log here, as you'll probably overflow. It doesn't really matter what numpy can nagle you just need to collate the data properly, defer the actual calculation until the node calculations are complete. Also, numpy should store values more densely than python itself. From jason at apkudo.com Tue Jan 2 13:08:07 2018 From: jason at apkudo.com (jason at apkudo.com) Date: Tue, 2 Jan 2018 10:08:07 -0800 (PST) Subject: Where did csv.parser() go? In-Reply-To: References: <5c51c86d-54a6-4ace-8b5d-cef48b6897dd@googlegroups.com> Message-ID: <86270948-da4a-4c7c-a6d1-e1b52c6042e2@googlegroups.com> Wow, awesome!!! Thank you! From rgaddi at highlandtechnology.invalid Tue Jan 2 13:16:23 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 2 Jan 2018 10:16:23 -0800 Subject: Python goto In-Reply-To: References: <474715f5-dfcc-67b2-6a30-9b977882f5fe@nedbatchelder.com> Message-ID: On 12/28/2017 04:35 AM, Skip Montanaro wrote: > Jorge> I would like to know if there is a goto command or something similar that > Jorge> I can use in Python. > > Ned> Python does not have a goto statement. You have to use structured > Ned> statements: for, while, try/except, yield, return, etc. > > Though it appears some wag has used function decorators to implement > goto statements: > > https://pypi.python.org/pypi/goto-statement/1.1 > > Rather clever, it seems. > > Skip > If only all that power had been used for good instead of evil... -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From rosuav at gmail.com Tue Jan 2 13:28:47 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 3 Jan 2018 05:28:47 +1100 Subject: Python goto In-Reply-To: References: <474715f5-dfcc-67b2-6a30-9b977882f5fe@nedbatchelder.com> Message-ID: On Wed, Jan 3, 2018 at 5:16 AM, Rob Gaddi wrote: > On 12/28/2017 04:35 AM, Skip Montanaro wrote: >> >> Jorge> I would like to know if there is a goto command or something >> similar that >> Jorge> I can use in Python. >> >> Ned> Python does not have a goto statement. You have to use structured >> Ned> statements: for, while, try/except, yield, return, etc. >> >> Though it appears some wag has used function decorators to implement >> goto statements: >> >> https://pypi.python.org/pypi/goto-statement/1.1 >> >> Rather clever, it seems. >> >> Skip >> > > If only all that power had been used for good instead of evil... > An evto statement? ChrisA From irving.duran at gmail.com Tue Jan 2 14:32:03 2018 From: irving.duran at gmail.com (Irving Duran) Date: Tue, 2 Jan 2018 13:32:03 -0600 Subject: Numpy and Terabyte data In-Reply-To: References: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> Message-ID: I've never heard or done that type of testing for a large dataset solely on python, so I don't know what's the cap from the memory standpoint that python can handle base on memory availability. Now, if I understand what you are trying to do, you can achieve that by leveraging Apache Spark and invoking "pyspark" where you can store data in memory and/or hard disk. Also, if you are working with Hadoop, you can use spark to move/transfer data back-and-forth. Thank You, Irving Duran On Tue, Jan 2, 2018 at 12:06 PM, wrote: > I'm not sure if I'll be laughed at, but a statistical sampling of a > randomized sample should resemble the whole. > > If you need min/max then min ( min(each node) ) > If you need average then you need sum( sum(each node)) sum(count(each > node))* > > *You'll likely need to use log here, as you'll probably overflow. > > It doesn't really matter what numpy can nagle you just need to collate the > data properly, defer the actual calculation until the node calculations are > complete. > > Also, numpy should store values more densely than python itself. > > > -- > https://mail.python.org/mailman/listinfo/python-list > From p.f.moore at gmail.com Tue Jan 2 15:13:08 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 2 Jan 2018 20:13:08 +0000 Subject: Numpy and Terabyte data In-Reply-To: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> References: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> Message-ID: On 2 January 2018 at 17:24, Rustom Mody wrote: > Someone who works in hadoop asked me: > > If our data is in terabytes can we do statistical (ie numpy pandas etc) > analysis on it? > > I said: No (I dont think so at least!) ie I expect numpy (pandas etc) > to not work if the data does not fit in memory > > Well sure *python* can handle (streams of) terabyte data I guess > *numpy* cannot > > Is there a more sophisticated answer? > > ["Terabyte" is a just a figure of speech for "too large for main memory"] You might want to look at Dask (https://pypi.python.org/pypi/dask, docs at http://dask.pydata.org/en/latest/). I've not used it myself, but I believe it's designed for very much the sort of use case you describe. Paul From gokoproject at gmail.com Tue Jan 2 15:29:55 2018 From: gokoproject at gmail.com (John Wong) Date: Tue, 02 Jan 2018 20:29:55 +0000 Subject: Goto In-Reply-To: References: <7Mb1C.278680$gi3.141689@fx19.am4> Message-ID: I think the point is there is not much to discuss. Goto is not going to be added. Furthermore, for every program language you want to translate from source, you have to find a workaround. Otherwise, your translation will only work for languages that have goto. Even so the implementation may not be exact what C goto is. Who knows. Much like in Go - the other day, slice there has a different behavior to Python?s slice. I brought this up to show that don?t expect everything to be equal. John On Tue, Jan 2, 2018 at 12:35 bartc wrote: > On 02/01/2018 15:20, Chris Angelico wrote: > > On Wed, Jan 3, 2018 at 1:51 AM, bartc wrote: > > >> I like to write code in a simple, clean, universal style that everyone > can > >> understand. > >> > >> That doesn't mean it has to look like Fortran. > > > > Why are you using a Python interpreter then? Why are you here on > > python-list, talking about Python? There must be SOME reason for using > > this language rather than another. > > I'm more familiar with Python than any other like languages. > > If I desperately needed a dynamic language and I couldn't use my own, > then I would use Python. > > But I would need to use it on my own terms, regardless of whether the > result is 'pythonic'. If the language had 'goto', then I would use it if > I found it apt. > > However, the discussion here is academic, so it doesn't matter who uses > what. > > Apart from anything else, Python is never going to officially adopt > 'goto'. (Regardless of some version of it being available as an add-on > library. Most things seem to be.) > > > -- > bartc > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sent from Jeff Dean's printf() mobile console From tkadm30 at yandex.com Tue Jan 2 16:21:50 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 2 Jan 2018 16:21:50 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: Message-ID: <5f328070-bdf2-0e9d-b0b4-ee792d95e6c6@yandex.com> Hi James, Part of the problem is because the CFFI and uWSGI developers aren't interested to support this. I need to modify CFFI to support preprocessing C headers with clang.cindex myself. I also need to make sure its possible to attach my Python script to the master uWSGI process to dispatch FIFO commands. Clang is needed because CFFI doesn't support preprocessing C headers with #define or #include directives. Best regards, Etienne Le 2018-01-02 ? 10:45, James Chapman a ?crit?: > Again, apologies if I've dumbed this down, but if I understand this > all correctly... > > The webserver starts uWSGI, the python application running within then > does stuff via uWSGI. You want one of those things to be handled by a > uWSGI extension that has been written in python, with CFFI, rather > than in pure C. In other words, CFFI is going to generate the C part > of the extension. > > Unfortunately I'm no CFFI expert, but the above sounds like something > that might be better broken down into 2 parts. > 1) Writing uWSGI extensions in Python and posted to a uWSGI > development forum. > 2) Generating python bindings for C calls with CFFI and clang and > posted to a CFFI forum. (Why clang? GCC/G++ will work just as well and > may be the default).? > > Hope that helps. > > James > > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From random832 at fastmail.com Tue Jan 2 16:24:39 2018 From: random832 at fastmail.com (Random832) Date: Tue, 02 Jan 2018 16:24:39 -0500 Subject: unicode direction control characters In-Reply-To: References: <8443deaa-37da-7ee5-7e03-a2d8cc8b681b@chamonix.reportlab.co.uk> Message-ID: <1514928279.4184868.1222169840.793A5710@webmail.messagingengine.com> On Tue, Jan 2, 2018, at 10:36, Robin Becker wrote: > >> u'\u200e28\u200e/\u200e09\u200e/\u200e1962' > > I guess I'm really wondering whether the BIDI control characters have any > semantic meaning. Most numbers seem to be LTR. > > If I saw u'\u200f12' it seems to imply that the characters should be displayed > '21', but I don't know whether the number is 12 or 21. No, 200F acts as a single R-L character (like an invisible letter), not an override for adjacent characters (as 202E would). LRM/RLM basically act like an invisible letter of the relevant directionality. European numerals have "weak" LTR directionality (to allow them to be used as part of e.g. a list of numbers in a sentence written in an RTL language), and don't affect some punctuation marks the same way as letters. I believe the purpose here is to ensure that it displays as 28/09/1962 instead of 1962/09/28 when the surrounding context is right-to-left. For the slash in particular, this was apparently a bug that was fixed in some recent version of unicode (this is mentioned briefly in UTR9, look for "solidus"), so earlier implementations or non-unicode implementations may not have supported it correctly. From rustompmody at gmail.com Tue Jan 2 23:07:45 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 2 Jan 2018 20:07:45 -0800 (PST) Subject: Numpy and Terabyte data In-Reply-To: References: <1ed64305-4719-449a-9432-a19d75689bd7@googlegroups.com> Message-ID: <3e6c82dc-3b9f-4ed4-99e4-436c2ab826b6@googlegroups.com> On Wednesday, January 3, 2018 at 1:43:40 AM UTC+5:30, Paul Moore wrote: > On 2 January 2018 at 17:24, Rustom Mody wrote: > > Someone who works in hadoop asked me: > > > > If our data is in terabytes can we do statistical (ie numpy pandas etc) > > analysis on it? > > > > I said: No (I dont think so at least!) ie I expect numpy (pandas etc) > > to not work if the data does not fit in memory > > > > Well sure *python* can handle (streams of) terabyte data I guess > > *numpy* cannot > > > > Is there a more sophisticated answer? > > > > ["Terabyte" is a just a figure of speech for "too large for main memory"] > > You might want to look at Dask (https://pypi.python.org/pypi/dask, > docs at http://dask.pydata.org/en/latest/). Thanks Looks like what I was asking about From james at uplinkzero.com Wed Jan 3 05:30:19 2018 From: james at uplinkzero.com (James Chapman) Date: Wed, 3 Jan 2018 10:30:19 +0000 Subject: How to create a python extension module from a shared library? In-Reply-To: <5f328070-bdf2-0e9d-b0b4-ee792d95e6c6@yandex.com> References: <5f328070-bdf2-0e9d-b0b4-ee792d95e6c6@yandex.com> Message-ID: In my opinion, just write your extension in C following the traditional extension development guidelines unless you plan to actively maintain and contribute your changes in the relevant projects (CFFI and uWSGI). Assuming you get this to work, you're going to have to support it and bug fix it. Something that initially will not be too difficult because it will all be fresh in your head, but 6 months down the line when you want to change something or discover a bug it's going to be very difficult. Will you be able to attach a debugger and step through the code? I've been down the route of trying to do something clever and even succeeded. I then later regretted it, because what seemed like a good idea at the time turned out to be a PITA to support. Go with what's supported, go with what's documented, don't modify core components if you don't absolutely have to, because you'll have to modify those components with each update and ultimately you just end up generating work for yourself. James On 2 January 2018 at 21:21, Etienne Robillard wrote: > Hi James, > > Part of the problem is because the CFFI and uWSGI developers aren't > interested to support this. I need to modify CFFI to support preprocessing > C headers with clang.cindex myself. > > I also need to make sure its possible to attach my Python script to the > master uWSGI process to dispatch FIFO commands. > > Clang is needed because CFFI doesn't support preprocessing C headers with > #define or #include directives. > > Best regards, > > Etienne > > From tkadm30 at yandex.com Wed Jan 3 06:12:25 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 3 Jan 2018 06:12:25 -0500 Subject: How to create a python extension module from a shared library? In-Reply-To: References: <5f328070-bdf2-0e9d-b0b4-ee792d95e6c6@yandex.com> Message-ID: <1b149998-a1a4-c94f-e1e3-9e4ce2bde13c@yandex.com> Hi James, I would love to write in C but I much prefer coding in Python :) I was thinking that I could use Cython to dlopen the shared library dynamically with ctypes. No need to compile anything. Best regards, Etienne Le 2018-01-03 ? 05:30, James Chapman a ?crit?: > In my opinion, just write your extension in C following the > traditional extension development guidelines unless you plan to > actively maintain and contribute your changes in the relevant projects > (CFFI and uWSGI). > > Assuming you get this to work, you're going to have to support it and > bug fix it. Something that initially will not be too difficult because > it will all be fresh in your head, but 6 months down the line when you > want to change something or discover a bug it's going to be very > difficult. Will you be able to attach a debugger and step through the > code? > > I've been down the route of trying to do something clever and even > succeeded. I then later regretted it, because what seemed like a good > idea at the time turned out to be a PITA to support. Go with what's > supported, go with what's documented, don't modify core components if > you don't absolutely have to, because you'll have to modify those > components with each update and ultimately you just end up generating > work for yourself. > > > James > > > > On 2 January 2018 at 21:21, Etienne Robillard > wrote: > > Hi James, > > Part of the problem is because the CFFI and uWSGI developers > aren't interested to support this. I need to modify CFFI to > support preprocessing C headers with clang.cindex myself. > > I also need to make sure its possible to attach my Python script > to the master uWSGI process to dispatch FIFO commands. > > Clang is needed because CFFI doesn't support preprocessing C > headers with #define or #include directives. > > Best regards, > > Etienne > > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From skip.montanaro at gmail.com Wed Jan 3 13:10:22 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 3 Jan 2018 12:10:22 -0600 Subject: 7z archive reader akin to zipfile? Message-ID: The zipfile module is kind of cool because you can access elements of the archive without explicitly uncompressing the entire archive and writing the structure to disk. I've got some 7z archives I'd like to treat the same way (read specific elements without first extractingg the entire tree to disk). I see the pylzma module for compressing and uncompressing files, but nothing slightly higher level. Does something like that exist? Thx, Skip From sjeik_appie at hotmail.com Wed Jan 3 15:03:30 2018 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Wed, 3 Jan 2018 20:03:30 +0000 Subject: Numpy and Terabyte data Message-ID: On Jan 2, 2018 18:27, Rustom Mody wrote: > > Someone who works in hadoop asked me: > > If our data is in terabytes can we do statistical (ie numpy pandas etc) > analysis on it? > > I said: No (I dont think so at least!) ie I expect numpy (pandas etc) > to not work if the data does not fit in memory > > Well sure *python* can handle (streams of) terabyte data I guess > *numpy* cannot > > Is there a more sophisticated answer? > > ["Terabyte" is a just a figure of speech for "too large for main memory"] Have a look at Pyspark and pyspark.ml. Pyspark has its own kind of DataFrame. Very, very cool stuff. Dask DataFrames have been mentioned already. numpy has memmapped arrays: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.memmap.html From harindudilshan95 at gmail.com Wed Jan 3 19:28:44 2018 From: harindudilshan95 at gmail.com (harindudilshan95 at gmail.com) Date: Wed, 3 Jan 2018 16:28:44 -0800 (PST) Subject: Progress on the Gilectomy In-Reply-To: References: <593bec11$0$1605$c3e8da3$5496439d@news.astraweb.com> <28555c12-7db9-3bd6-e62b-04706cb9ecd9@chamonix.reportlab.co.uk> <5947DDC1.8040706@stoneleaf.us> <5947EBA1.2080309@stoneleaf.us> <5947F251.6030005@stoneleaf.us> <730BD904-65C4-4EAA-8F2C-086346741EE3@gmail.com> <87d19z2q3n.fsf@nightsong.com> <6DE696B4-465F-4654-B7FF-40FBAB66EB5D@gmail.com> <874lv93o0i.fsf@nightsong.com> <72e05272-4811-4072-b890-609d04673930@googlegroups.com> <87r2yd1i5h.fsf@nightsong.com> <8c033515-9aaf-4ef7-ab3c-95d382b53710@googlegroups.com> <594af9f0$0$1618$c3e8da3$5496439d@news.astraweb.com> <594D2755.7010109@stoneleaf.us> Message-ID: <4e91d2a3-573f-4135-bf52-807be3ff7b21@googlegroups.com> Why not make the garbage collector check the reference count before freeing objects? Only c extensions would increment the ref count while python code would just use garbage collector making ref count = 0. That way even the existing c extensions would continue to work. Regarding to Java using all the memory, thats not really true. It has a default heap size which may exceed the total memory in a particular environment(Android ). From greg.ewing at canterbury.ac.nz Wed Jan 3 19:32:35 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Thu, 04 Jan 2018 13:32:35 +1300 Subject: 7z archive reader akin to zipfile? In-Reply-To: References: Message-ID: Skip Montanaro wrote: > I've got some 7z archives I'd like to > treat the same way (read specific elements without first extractingg > the entire tree to disk). If you're doing this a lot, it might be worth repackaging your 7z files as zip files. -- Greg From skip.montanaro at gmail.com Wed Jan 3 19:50:35 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Wed, 3 Jan 2018 18:50:35 -0600 Subject: 7z archive reader akin to zipfile? In-Reply-To: References: Message-ID: If you're doing this a lot, it might be worth repackaging your 7z files as > zip files. Good point. FWIW, these are the files: http://untroubled.org/spam Pretty static once a month or year is closed out... Skip From XPN at a.a Thu Jan 4 01:29:20 2018 From: XPN at a.a (XPN) Date: Thu, 4 Jan 2018 06:29:20 +0000 (UTC) Subject: when the new version of XPN py2 newsreader src-tarball hits alt.binaries, the world will hold it's breath Message-ID: when the new version of XPN py2 newsreader src-tarball hits alt.binaries, the world will hold it's breath. major usability overhaul is ongoing. release will be in style in usenet binary newsgroup. full autoconfigure, no bs asked. From a at a.a Thu Jan 4 01:31:46 2018 From: a at a.a (a) Date: Thu, 4 Jan 2018 06:31:46 +0000 (UTC) Subject: when the new version of XPN py2 newsreader src-tarball hits alt.binaries, the world will hold it's breath References: Message-ID: py2 now, gotta fix that one From x at x.x Thu Jan 4 01:35:58 2018 From: x at x.x (x at x.x) Date: Thu, 4 Jan 2018 06:35:58 +0000 (UTC) Subject: xpn References: Message-ID: need to fix those quirks. From tkadm30 at yandex.com Thu Jan 4 04:50:11 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 4 Jan 2018 04:50:11 -0500 Subject: Progress migrating cffi and pycparser to libclang Message-ID: Hi, I will be creating a repository for this: https://bitbucket.org/tkadm30/cffi-libclang The goal is to generate a AST object from a C header by preprocessing with clang -E then compile the python bindings with CFFI... ffi.cdef(open('uwsgi.h').read()) # <-- XXX need to modify internal parsing code to use clang.cindex! ffi.dlopen('/usr/local/lib/libuwsgi.so') Right now the parsing of the C source file is done by pycparser, but i would like to use the clang.cindex module. What do you think? Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From p.f.moore at gmail.com Thu Jan 4 06:41:43 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 4 Jan 2018 11:41:43 +0000 Subject: Progress migrating cffi and pycparser to libclang In-Reply-To: References: Message-ID: On 4 January 2018 at 09:50, Etienne Robillard wrote: > Hi, > > I will be creating a repository for this: > https://bitbucket.org/tkadm30/cffi-libclang > > The goal is to generate a AST object from a C header by preprocessing with > clang -E then compile the python bindings with CFFI... > > ffi.cdef(open('uwsgi.h').read()) # <-- XXX need to modify internal parsing > code to use clang.cindex! > > ffi.dlopen('/usr/local/lib/libuwsgi.so') > > > Right now the parsing of the C source file is done by pycparser, but i would > like to use the clang.cindex module. > > What do you think? Presumably that will introduce a dependency on some clang module? You mention clang.cindex - but the only clang package I can find on PyPI says "OBSOLETE. LLVM-CLANG NOW PUBLISHES PYTHON PACKAGE. JUST ADD THE OFFICIAL llvm-3.7 repo in your apt." but doesn't provide binaries or explain how to install clang on, say, Windows (to pick an example relevant to me :-)). As a fork/extension for cffi, I have no particular opinion (I'm unlikely to ever use it). But the advantage of pycparser is that it's cross-platform and pure Python, so I doubt this will be acceptable for inclusion into CFFI itself. Paul From tkadm30 at yandex.com Thu Jan 4 16:02:20 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 4 Jan 2018 16:02:20 -0500 Subject: Progress migrating cffi and pycparser to libclang In-Reply-To: References: Message-ID: <5414e199-35f3-2a51-298d-e7b94250a7a3@yandex.com> Hi Paul, Le 2018-01-04 ? 06:41, Paul Moore a ?crit?: > Presumably that will introduce a dependency on some clang module? You > mention clang.cindex - but the only clang package I can find on PyPI > says "OBSOLETE. LLVM-CLANG NOW PUBLISHES PYTHON PACKAGE. > JUST ADD THE OFFICIAL llvm-3.7 repo in your apt." but doesn't provide > binaries or explain how to install clang on, say, Windows (to pick an > example relevant to me :-)). I'm targeting the Linux/x86 machine class. On debian systems, it's pretty easy to install clang and the python bindings with apt-get. :-) > > As a fork/extension for cffi, I have no particular opinion (I'm > unlikely to ever use it). But the advantage of pycparser is that it's > cross-platform and pure Python, so I doubt this will be acceptable for > inclusion into CFFI itself. CFFI/pycparser definitely need to be patched to support parsing standard C directives like #define and #include in the ffi.cdef() function. The easiest solution is to migrate the internal parsing code to libclang, a state-of-the art C/C++ compiler based on LLVM. Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From p.f.moore at gmail.com Thu Jan 4 16:25:27 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 4 Jan 2018 21:25:27 +0000 Subject: Progress migrating cffi and pycparser to libclang In-Reply-To: <5414e199-35f3-2a51-298d-e7b94250a7a3@yandex.com> References: <5414e199-35f3-2a51-298d-e7b94250a7a3@yandex.com> Message-ID: On 4 January 2018 at 21:02, Etienne Robillard wrote: >> As a fork/extension for cffi, I have no particular opinion (I'm >> unlikely to ever use it). But the advantage of pycparser is that it's >> cross-platform and pure Python, so I doubt this will be acceptable for >> inclusion into CFFI itself. > > CFFI/pycparser definitely need to be patched to support parsing standard C > directives like #define and #include in the ffi.cdef() function. > > The easiest solution is to migrate the internal parsing code to libclang, a > state-of-the art C/C++ compiler based on LLVM. I would strongly object to adding a dependency to cffi that couldn't be automatically installed by pip as part of standard dependency resolution (i.e., a PyPI hosted Python project with wheels available for all common platforms - Linux, Mac OS and Windows). But ultimately if you're proposing this as a change to cffi, you should be getting the opinions of the cffi devs, not just asking on this list. (I notice you have posted to the cffi mailing list, but haven't had any response yet). Paul From dieter at handshake.de Fri Jan 5 01:52:25 2018 From: dieter at handshake.de (dieter) Date: Fri, 05 Jan 2018 07:52:25 +0100 Subject: Problems with imports on multiple threads, with embedded Python References: <48311546-1f0a-4f86-8882-79ae3ba387fa@googlegroups.com> Message-ID: <87incger12.fsf@handshake.de> geoff.bache at gmail.com writes: > I have a multithreaded application using an embedded Python 3.6.4 (upgraded from 3.6.2 today in the hope that the problem was now solved: it doesn't seem to be). The standard library is in a zip file. So long as only one thread is running Python at a time it seems to work fine. But there seems to be a problem with the module importing when several Python threads are active. > > I get a variety of errors indeterministically, usually indicating that some symbol hasn't been imported. This occurs both in my own code and in the standard library. The most frequent is probably this line: I do not know your specific problem (up to now, I used only Python 2), but I know about import problems in multi threaded applications. The problem is easily understood: import affects the global object "sys.modules". To protect this object, Python must do something. In former Python versions (somewhere before Python 3), it was using a a thread lock: this could cause a deadlock in the case of some import dependencies. The recoomendation to avoid the problem has been to avoid module level imports for modules imported by threads. Python 3 may have changed the way to protect "sys.modules". In case of recursive import dependencies, you may see that some symbols are not defined even in single threaded applications -- however then usually deterministically. At your place, I would start to check whether your application has recursive import dependancies -- and in this case, remove them. From dieter at handshake.de Fri Jan 5 01:55:51 2018 From: dieter at handshake.de (dieter) Date: Fri, 05 Jan 2018 07:55:51 +0100 Subject: Regarding the error: TypeError: can’t pickle _thread.lock objects References: <4328A44A7D09CB4B818774C0B09D7DE7E320E1D8@SRVEXCH-01.gsr-inc.local> Message-ID: <87efn4eqvc.fsf@handshake.de> Winston Manuel Vijay writes: > It would be of immense help, if someone could provide a suitable solution or related information that helps to sort out the below stated issue- > > > ? I had installed the Python version 3.6.4 > > ? Then I installed the package: Tensorflow > > ? Installed g2p.exe by downloading from GitHub > > ? Then tried running the below command- > > g2p-seq2seq --interactive --model (model_folder_path: is the path to an English model 2-layer LSTM with 512 hidden units CMU Sphinx dictionary downloaded from the CMU Sphinx website) > > Following the above procedure, I encountered the following error: TypeError: can.t pickle _thread.lock objects-please find the attached screenshot for your reference. This looks like a programming error in "g2p-seq2sec": contact its authors or try a different version. From tkadm30 at yandex.com Fri Jan 5 04:15:56 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 5 Jan 2018 04:15:56 -0500 Subject: Fwd: Re: Progress migrating cffi and pycparser to libclang In-Reply-To: References: Message-ID: Forwarding? this thread to the CFFI developers... Re Paul: Thanks for your feedback. My intended audience are developers who can use hg to fetch/build source code without pip. Best regards, Etienne -------- Message transf?r? -------- Sujet?: Re: Progress migrating cffi and pycparser to libclang Date?: Thu, 4 Jan 2018 21:25:27 +0000 De?: Paul Moore Pour?: Etienne Robillard Copie ??: Python On 4 January 2018 at 21:02, Etienne Robillard wrote: >> As a fork/extension for cffi, I have no particular opinion (I'm >> unlikely to ever use it). But the advantage of pycparser is that it's >> cross-platform and pure Python, so I doubt this will be acceptable for >> inclusion into CFFI itself. > > CFFI/pycparser definitely need to be patched to support parsing standard C > directives like #define and #include in the ffi.cdef() function. > > The easiest solution is to migrate the internal parsing code to libclang, a > state-of-the art C/C++ compiler based on LLVM. I would strongly object to adding a dependency to cffi that couldn't be automatically installed by pip as part of standard dependency resolution (i.e., a PyPI hosted Python project with wheels available for all common platforms - Linux, Mac OS and Windows). But ultimately if you're proposing this as a change to cffi, you should be getting the opinions of the cffi devs, not just asking on this list. (I notice you have posted to the cffi mailing list, but haven't had any response yet). Paul From iranna.gani28 at gmail.com Fri Jan 5 05:04:39 2018 From: iranna.gani28 at gmail.com (Iranna Mathapati) Date: Fri, 5 Jan 2018 15:34:39 +0530 Subject: sendline buffer Message-ID: Hi Team, I have faced fallowing issue:: dev.sendline("*show version*") <<< its printing "show version output" dev.sendline("*show module*") <<< its printing "shoe module output" *Runing again* dev.sendline("show veriosn") <<< its runing 2nd time again dev.before <<<<<<<< *output is not consistence means some time it printing "shoe version output" or some time its printing only "show module output"* can you please let me know , how to resolve the issue. Thanks, Iranna M From armin.rigo at gmail.com Fri Jan 5 05:59:05 2018 From: armin.rigo at gmail.com (Armin Rigo) Date: Fri, 5 Jan 2018 11:59:05 +0100 Subject: Progress migrating cffi and pycparser to libclang In-Reply-To: References: Message-ID: Hi Etienne, On 5 January 2018 at 10:15, Etienne Robillard wrote: > Forwarding this thread to the CFFI developers... > If you're asking whether we could add libclang as a dependency to CFFI, the answer is no, sorry. I feel that I've already explained exactly this to you several times in private e-mails, so I'm writing down this fact here on the public python-cffi mailing list. Please stop asking the same question. For reference, here's my answer again. CFFI is meant to be used in a certain way. I know from experience that a few people keep thinking about it in a different way---sometimes for good reasons: it is tedious to wrap a very large library even if it is easy to copy-paste-tweak individual function definitions. So to automate the process these people usually want to parse real .h files, using libclang or better pycparser integration or something else; and that's cool. As I said there are already third-party projects that try to do something similar. Usually it is never completely general, but more focused on a particular style of .h files coming from a particular large project. Maybe you'll run into similar issues and make something that works in your case but not in others. Or maybe you'll manage to make it general enough. In all cases this won't be included inside CFFI, but could nevertheless be a successful separate project. Please think about it as a project *above* CFFI, maybe something you'll import and use inside a CFFI build script in order to provide content to ffi.cdef(). A bient?t, Armin. From kw at codebykevin.com Fri Jan 5 08:29:21 2018 From: kw at codebykevin.com (Kevin Walzer) Date: Fri, 5 Jan 2018 08:29:21 -0500 Subject: Linux/Windows GUI programming: GUI-fy a CLI using pyInstaller In-Reply-To: References: Message-ID: On 1/1/18 11:45 AM, X. wrote: > Ulli Horlacher: >> I have to transfer a python 2.7 CLI programm into one with a (simple) GUI. >> The program must run on Linux and Windows and must be compilable with >> pyinstall, because I have to ship a standalone windows.exe >> Any kind of installer is not acceptable. >> >> Reading https://github.com/pyinstaller/pyinstaller/wiki/Supported-Packages >> supported GUI packages are PyGTK, PyQt4, PyQt5, wxPython >> I have tested tkinter by myself and it works, too. >> I do not like GTK and Qt, because they are too complex. >> >> I want to do VERY simple things and I prefer a simple GUI toolkit :-) > > > me too ! > Try easygui: https://pypi.python.org/pypi/easygui -- Kevin Walzer Code by Kevin/Mobile Code by Kevin http://www.codebykevin.com http://www.wtmobilesoftware.com From ian.g.kelly at gmail.com Fri Jan 5 11:43:01 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 5 Jan 2018 09:43:01 -0700 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: On Fri, Jan 5, 2018 at 9:27 AM, Kim of K. wrote: > > "Background > > We feel that the world still produces way too much software that is > frankly substandard. The reasons for this are pretty simple: software > producers do not pay enough attention [...]" > > > quote from http://texttest.sourceforge.net/index.php?page=about > > > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > > why is no one complaining ? It is what it is. What would be the utility of complaining about it? That's not going to fix anything. And what does this have to do with Python? From larry.martell at gmail.com Fri Jan 5 12:13:11 2018 From: larry.martell at gmail.com (Larry Martell) Date: Fri, 5 Jan 2018 12:13:11 -0500 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: On Fri, Jan 5, 2018 at 11:27 AM, Kim of K. wrote: > > "Background > > We feel that the world still produces way too much software that is > frankly substandard. The reasons for this are pretty simple: software > producers do not pay enough attention [...]" > > > quote from http://texttest.sourceforge.net/index.php?page=about > > > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > > why is no one complaining ? You want to complain? Look at these shoes. I've only had them three weeks and the heels are worn right through. From grant.b.edwards at gmail.com Fri Jan 5 12:21:28 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 5 Jan 2018 17:21:28 +0000 (UTC) Subject: has sourceforge exposed the dirty little secret ? References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: On 2018-01-05, Kim of K. wrote: > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > > why is no one complaining ? Because complaining doesn't have any effect? If you care, shut up and fix something. -- Grant Edwards grant.b.edwards Yow! ! I'm in a very at clever and adorable INSANE gmail.com ASYLUM!! From gordon at panix.com Fri Jan 5 13:58:57 2018 From: gordon at panix.com (John Gordon) Date: Fri, 5 Jan 2018 18:58:57 +0000 (UTC) Subject: has sourceforge exposed the dirty little secret ? References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: In <151516966665.348096.18338899180412170014.XPN at Welt.Netz> "Kim of K." writes: > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > why is no one complaining ? 90% of everything is crap. Why should software be any different? -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From gordon at panix.com Fri Jan 5 14:02:58 2018 From: gordon at panix.com (John Gordon) Date: Fri, 5 Jan 2018 19:02:58 +0000 (UTC) Subject: has sourceforge exposed the dirty little secret ? References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151517499074.367782.5631796281028137729.XPN@Welt.Netz> <151517608506.368831.5093080329614058603.XPN@Welt.Netz> Message-ID: In <151517608506.368831.5093080329614058603.XPN at Welt.Netz> "Kim of K." writes: > print(emo('now you see emos')) > OF COURSE THIS SHIT DOES NOT WORK. What device did you run this on? Your average terminal window isn't going to support emojis... -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From ikorot01 at gmail.com Fri Jan 5 14:21:36 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Fri, 5 Jan 2018 13:21:36 -0600 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: Hi, On Fri, Jan 5, 2018 at 12:58 PM, John Gordon wrote: > In <151516966665.348096.18338899180412170014.XPN at Welt.Netz> "Kim of K." writes: > > >> In other words: most sites like SF and github offer tons of crap. >> download and break is the overwhelming theme here. > >> why is no one complaining ? > > 90% of everything is crap. Why should software be any different? Moreover, most of the time life sucks. Every morning you have to go to school/work instead of staying home and enjoying the life. And you are lucky if the school/work is in the same city where you family is... Thank you. > > -- > John Gordon A is for Amy, who fell down the stairs > gordon at panix.com B is for Basil, assaulted by bears > -- Edward Gorey, "The Gashlycrumb Tinies" > > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Fri Jan 5 14:25:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Jan 2018 06:25:54 +1100 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151517499074.367782.5631796281028137729.XPN@Welt.Netz> <151517608506.368831.5093080329614058603.XPN@Welt.Netz> Message-ID: On Sat, Jan 6, 2018 at 6:02 AM, John Gordon wrote: > In <151517608506.368831.5093080329614058603.XPN at Welt.Netz> "Kim of K." writes: > >> print(emo('now you see emos')) >> OF COURSE THIS SHIT DOES NOT WORK. > > What device did you run this on? Your average terminal window isn't > going to support emojis... > I dunno about yours. Mine does. It's the default XFCE terminal on Debian GNU/Linux. It doesn't respect the Unicode colour designations (since it does colour in other ways and it would conflict), but AFAIK it supports everything else. ChrisA From ikorot01 at gmail.com Fri Jan 5 15:27:51 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Fri, 5 Jan 2018 14:27:51 -0600 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151518309421.370396.344401823089251819.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151518309421.370396.344401823089251819.XPN@Welt.Netz> Message-ID: Hi, On Fri, Jan 5, 2018 at 2:11 PM, Kim of K. wrote: > Igor Korot wrote: > >> Hi, >> >> On Fri, Jan 5, 2018 at 12:58 PM, John Gordon wrote: >>> In <151516966665.348096.18338899180412170014.XPN at Welt.Netz> "Kim of K." writes: >>> >>> >>>> In other words: most sites like SF and github offer tons of crap. >>>> download and break is the overwhelming theme here. >>> >>>> why is no one complaining ? >>> >>> 90% of everything is crap. Why should software be any different? >> >> Moreover, most of the time life sucks. >> Every morning you have to go to school/work instead of staying home >> and enjoying the life. >> And you are lucky if the school/work is in the same city where you family is... >> >> Thank you. > > > let me tell you... > > Once you're done with that school crap, you realize it was the pefect waste of time. > > At work or in life you need less than 2% of that school crap they funnelled into your head. I seriously doubt you had the same opinion when you were 7 y.o. I meant school as "junior/middle/high school/university/college" school, because you still have to wake up early in the morning and go to school. So yes, life sucks most of the time anyway... Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list From torriem at gmail.com Fri Jan 5 16:15:31 2018 From: torriem at gmail.com (Michael Torrie) Date: Fri, 5 Jan 2018 14:15:31 -0700 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <151517499074.367782.5631796281028137729.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151517499074.367782.5631796281028137729.XPN@Welt.Netz> Message-ID: <96159bc9-6c99-d00f-1941-7c938cb212a8@gmail.com> On 01/05/2018 10:56 AM, Kim of K. wrote: > wow! Yup that's what I said when I read your ramblings. > even you are defensive about publishing non-working garbage. Absolutely. You have absolutely no right to make demands of any of the folks who toss their half-baked personal projects up on sourceforge or github. What makes you think you do? This entitlement mentality is probably the biggest threat to open source software. Look I can publish non-working garbage if I want. Who are you to say otherwise? > btw., I'm trying to fix emojis for so one can use emoji in this group, using a > python newsreader, of course. Please, no! We don't need emoji in this group. Fortunately the vast majority of posters use plain text (as is the etiquette) and so we don't have to worry about that kind of nonsense. Anyway, it sounds like the software you are trying to get running is defective. Better ask for a refund. Anyway, time to plonk. From ian.g.kelly at gmail.com Fri Jan 5 16:54:42 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Fri, 5 Jan 2018 14:54:42 -0700 Subject: is comp.lang.python still the pinnacle of the py community ? In-Reply-To: <151518639229.370912.11192365825591600060.XPN@Welt.Netz> References: <151518639229.370912.11192365825591600060.XPN@Welt.Netz> Message-ID: On Fri, Jan 5, 2018 at 2:06 PM, Kim of K. wrote: > > post frequency is down to a precarious level It's true that compared to ten years ago, the quantity of posts here has diminished by a significant fraction, maybe even by an order of magnitude. This is still a great place for discussion however, as long as that discussion is about Python. If this post is (as it appears to be) just a backhanded way of complaining that not many are responding to your excessive and trollish posts, then try making higher-quality posts that are actually on topic. From gheskett at shentel.net Fri Jan 5 17:33:06 2018 From: gheskett at shentel.net (Gene Heskett) Date: Fri, 5 Jan 2018 17:33:06 -0500 Subject: is comp.lang.python still the pinnacle of the py community ? In-Reply-To: <151518639229.370912.11192365825591600060.XPN@Welt.Netz> References: <151518639229.370912.11192365825591600060.XPN@Welt.Netz> Message-ID: <201801051733.06613.gheskett@shentel.net> On Friday 05 January 2018 16:06:34 Kim of K. wrote: > post frequency is down to a precarious level Thats because the huge majority of us who are here to learn a tidbit here and there, shove that stuff off to a spamassassin training directory, where its studied by sa-learn --spam for a second or two, and deleted the next day. . And your posts are about to be added to that. Say something helpfull to the python knowledge base, or STFU, this is not intended to be your private podium to call the rest of the world un-enlightened or names. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From rgaddi at highlandtechnology.invalid Fri Jan 5 19:01:50 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Fri, 5 Jan 2018 16:01:50 -0800 Subject: Native object exposing buffer protocol Message-ID: I'd like to create a native Python object that exposes the buffer protocol. Basically, something with a ._data member which is a bytearray that I can still readinto, make directly into a numpy array, etc. I can do it by inheriting the entire thing from bytearray directly, but that gives me a whole lot of methods that are unsuitable for what I'm doing with it, which is reading the contents of a binary file, allowing them to be lightly malleable in memory, and then sending them on to a device. Not the end of the world (the file's less than 2KB), but it seems like something that should be doable easily without having to throw around a lot of extraneous copies. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From Irv at furrypants.com Fri Jan 5 19:11:35 2018 From: Irv at furrypants.com (Irv Kalb) Date: Fri, 5 Jan 2018 16:11:35 -0800 Subject: Python Inheritance Terminology Message-ID: <3C7576D1-FC53-4EB1-B09D-BF4FD3677779@furrypants.com> I'm doing some writing for an upcoming course on OOP using Python. I have been doing OOP programming for many years in many different languages, and I want make sure that I'm using the appropriate terminology in Python. I'd like to know if there are "official" or even standard terms that are used to describe a class that is inherited from, and the class that is doing the inheriting. From my reading (especially the PSF docs.python.org ), it looks like the terms would be "base class" and "subclass". However, in books about Python and other languages, I have also seen the terms: base class & derived class parent class & child class superclass & subclass So, are base class & subclass the proper terms? Thanks, Irv From breamoreboy at gmail.com Fri Jan 5 19:12:10 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 5 Jan 2018 16:12:10 -0800 (PST) Subject: Native object exposing buffer protocol In-Reply-To: References: Message-ID: <64e8ac2e-44c4-4057-b9ee-196f283b3d78@googlegroups.com> On Saturday, January 6, 2018 at 12:02:18 AM UTC, Rob Gaddi wrote: > I'd like to create a native Python object that exposes the buffer > protocol. Basically, something with a ._data member which is a > bytearray that I can still readinto, make directly into a numpy array, etc. > > I can do it by inheriting the entire thing from bytearray directly, but > that gives me a whole lot of methods that are unsuitable for what I'm > doing with it, which is reading the contents of a binary file, allowing > them to be lightly malleable in memory, and then sending them on to a > device. > > Not the end of the world (the file's less than 2KB), but it seems like > something that should be doable easily without having to throw around a > lot of extraneous copies. > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. Could you use memoryviews instead of making the copies? -- Kindest regards. Mark Lawrence. From ben+python at benfinney.id.au Fri Jan 5 19:27:04 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 06 Jan 2018 11:27:04 +1100 Subject: Native object exposing buffer protocol References: Message-ID: <85lghbltlz.fsf@benfinney.id.au> Rob Gaddi writes: > I'd like to create a native Python object that exposes the buffer > protocol. Basically, something with a ._data member which is a > bytearray that I can still readinto, make directly into a numpy array, > etc. The ?etc.? seems pretty important, there. You want the behaviour of ?bytearray? without actually inheriting that behaviour from the ?bytearray? type. So, it seems your options are: * Enumerate all the things, specifically, that you do want your new type to do. Don't hide anything in ?etc.?, so that you know exactly what behaviours need to be implemented. Implement all those behaviours, without benefit of inheriting from ?bytearray?. * Inherit from ?bytearray?, but ameliorate the problems you want to avoid. This will require enumerating all those problems, so that you can know whether you have avoided them. Don't hide any of them in an ?etc.?. > Not the end of the world (the file's less than 2KB), but it seems like > something that should be doable easily without having to throw around > a lot of extraneous copies. I look forward to your report from having tried it :-) -- \ ?A lie can be told in a few words. Debunking that lie can take | `\ pages. That is why my book? is five hundred pages long.? ?Chris | _o__) Rodda, 2011-05-05 | Ben Finney From ben+python at benfinney.id.au Fri Jan 5 19:32:42 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 06 Jan 2018 11:32:42 +1100 Subject: Python Inheritance Terminology References: <3C7576D1-FC53-4EB1-B09D-BF4FD3677779@furrypants.com> Message-ID: <85h8rzltcl.fsf@benfinney.id.au> Irv Kalb writes: > I'm doing some writing for an upcoming course on OOP using Python. Welcome, and congratulations for using Python in this work. > I'd like to know if there are "official" or even standard terms that > are used to describe a class that is inherited from, and the class > that is doing the inheriting. From my reading (especially the PSF > docs.python.org ), it looks like the terms > would be "base class" and "subclass". Standard (?official?) terms are most likely to be had from the language reference . I would recommend the glossary , but with the caveat that many flaws have been found in recent years. > However, in books about Python and other languages, I have also seen the terms: > > base class & derived class > parent class & child class > superclass & subclass The only term I take issue with there is ?superclass?. In a multiple-inheritance system, such as provided by Python, the superclass is *not* necessarily the base class. See this article from 2011 . > So, are base class & subclass the proper terms? In my opinion you will be correct to use those terms. Which is not to say that other terms aren't also good. -- \ ?The greatest tragedy in mankind's entire history may be the | `\ hijacking of morality by religion.? ?Arthur C. Clarke, 1991 | _o__) | Ben Finney From Richard at Damon-Family.org Fri Jan 5 19:57:44 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Fri, 5 Jan 2018 19:57:44 -0500 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151518309421.370396.344401823089251819.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151518309421.370396.344401823089251819.XPN@Welt.Netz> Message-ID: <8c999ec7-fe89-c7a9-2db1-044a74cbe062@Damon-Family.org> On 1/5/18 3:11 PM, Kim of K. wrote: > let me tell you... > > Once you're done with that school crap, you realize it was the pefect waste of time. > > At work or in life you need less than 2% of that school crap they funnelled into your head. My experience is that while I found little use for much of the 'facts' that I learned in school, there has been a lot of use for the basic methods that were learned while processing those facts in school. -- Richard Damon From bob.martin at excite.com Sat Jan 6 03:00:34 2018 From: bob.martin at excite.com (Bob Martin) Date: Sat, 06 Jan 2018 08:00:34 GMT Subject: Linux/Windows GUI programming: GUI-fy a CLI using pyInstaller References: Message-ID: in 788357 20180105 132921 Kevin Walzer wrote: >On 1/1/18 11:45 AM, X. wrote: >> Ulli Horlacher: >>> I have to transfer a python 2.7 CLI programm into one with a (simple) GUI. >>> The program must run on Linux and Windows and must be compilable with >>> pyinstall, because I have to ship a standalone windows.exe >>> Any kind of installer is not acceptable. >>> >>> Reading https://github.com/pyinstaller/pyinstaller/wiki/Supported-Packages >>> supported GUI packages are PyGTK, PyQt4, PyQt5, wxPython >>> I have tested tkinter by myself and it works, too. >>> I do not like GTK and Qt, because they are too complex. >>> >>> I want to do VERY simple things and I prefer a simple GUI toolkit :-) >> >> >> me too ! >> > >Try easygui: > >https://pypi.python.org/pypi/easygui Looks like Zenity. From arj.python at gmail.com Sat Jan 6 04:45:24 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 6 Jan 2018 13:45:24 +0400 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: github? On Fri, Jan 5, 2018 at 8:27 PM, Kim of K. wrote: > > "Background > > We feel that the world still produces way too much software that is > frankly substandard. The reasons for this are pretty simple: software > producers do not pay enough attention [...]" > > > quote from http://texttest.sourceforge.net/index.php?page=about > > > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > > why is no one complaining ? > -- > https://mail.python.org/mailman/listinfo/python-list > From hjp-python at hjp.at Sat Jan 6 05:39:06 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 6 Jan 2018 11:39:06 +0100 Subject: 7z archive reader akin to zipfile? In-Reply-To: References: Message-ID: <20180106103906.27emxpmh57xwl4rb@hjp.at> On 2018-01-03 12:10:22 -0600, Skip Montanaro wrote: > The zipfile module is kind of cool because you can access elements of > the archive without explicitly uncompressing the entire archive and > writing the structure to disk. I've got some 7z archives I'd like to > treat the same way (read specific elements without first extractingg > the entire tree to disk). I see the pylzma module for compressing and > uncompressing files, but nothing slightly higher level. Does something > like that exist? Have you looked at libarchive (https://pypi.python.org/pypi/libarchive)? I think it does what you want and it transparently supports several archive types (including zip and 7z). hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From tkadm30 at yandex.com Sat Jan 6 05:42:55 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sat, 6 Jan 2018 05:42:55 -0500 Subject: Spectre/Meltdown bug affecting Python ? Message-ID: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> Hi all, What do you think about the latest Spectre/Meltdown security flaw found in Intel processors and Apple smartphones? Are Python 2.7 and 3.6 affected by speculative execution side-channel attacks when using the Linux kernel and Intel CPUs? Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From tkadm30 at yandex.com Sat Jan 6 07:43:24 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sat, 6 Jan 2018 07:43:24 -0500 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> Message-ID: <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> My understanding of this vulnerability is that speculative indirect calls in Linux kernel can be used to extract/filter memory content via side-channels. So, is it time to implement --enable-retpoline to CPython ? [1] Etienne 1. https://www.bleepingcomputer.com/news/google/google-unveils-new-retpoline-coding-technique-for-mitigating-spectre-attacks/ Le 2018-01-06 ? 05:42, Etienne Robillard a ?crit?: > Hi all, > > What do you think about the latest Spectre/Meltdown security flaw > found in Intel processors and Apple smartphones? > > Are Python 2.7 and 3.6 affected by speculative execution side-channel > attacks when using the Linux kernel and Intel CPUs? > > > Best regards, > > Etienne > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From skip.montanaro at gmail.com Sat Jan 6 09:29:48 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sat, 6 Jan 2018 08:29:48 -0600 Subject: 7z archive reader akin to zipfile? In-Reply-To: <20180106103906.27emxpmh57xwl4rb@hjp.at> References: <20180106103906.27emxpmh57xwl4rb@hjp.at> Message-ID: > Have you looked at libarchive (https://pypi.python.org/pypi/libarchive)? Thanks, was completely unaware of its existence. I will take a look. I've been repackaging the 7z archives as zips, but the result is 3-5x larger. Skip From wanderer at dialup4less.com Sat Jan 6 11:20:44 2018 From: wanderer at dialup4less.com (Wanderer) Date: Sat, 6 Jan 2018 08:20:44 -0800 (PST) Subject: 7z archive reader akin to zipfile? In-Reply-To: References: Message-ID: <8c2bf767-c82e-479a-8566-c4932b4d3552@googlegroups.com> On Wednesday, January 3, 2018 at 1:11:31 PM UTC-5, Skip Montanaro wrote: > The zipfile module is kind of cool because you can access elements of > the archive without explicitly uncompressing the entire archive and > writing the structure to disk. I've got some 7z archives I'd like to > treat the same way (read specific elements without first extractingg > the entire tree to disk). I see the pylzma module for compressing and > uncompressing files, but nothing slightly higher level. Does something > like that exist? > > Thx, > > Skip I made this wrapper class for 7zip. It might be useful for you. #python wrapper for 7zip import os import zlib from subprocess import Popen, PIPE """ p = Popen(['program', 'arg1'], stdin=PIPE, stdout=PIPE, stderr=PIPE) output, err = p.communicate(b"input data that is passed to subprocess' stdin") rc = p.returncode """ SEVEN_ZIP_PATH = "C:/Program Files/7-Zip/7z.exe" class SevenZip(object): def __new__(cls, ZipProgram=SEVEN_ZIP_PATH): if os.path.isfile(ZipProgram): return super(SevenZip, cls).__new__(cls) else: raise ValueError("7zip program not found in %s" %(ZipProgram)) def __init__(self, ZipProgram=SEVEN_ZIP_PATH): self.ZipProgram = ZipProgram self.archive = None self.outputDir = None self.fileList = [] self.archiveType = "zip" def call(self, cmdList=None): """ Used by the other methods to call the 7zip command line. Can be used directly to run 7zip if the wrapper methods don't suffice. cmdList -- Subprocess style list of command line options with the first item in the list being self.ZipProgram """ if cmdList is not None: zip7 = Popen(cmdList, stdin=PIPE, stdout=PIPE, stderr=PIPE ) output ,err = zip7.communicate() rc = zip7.returncode print "output" , output print "return code", rc if len(err) > 0: print "errors found", err def modify(self, archive=None, fileList=None, cmd=None): """ Modify an archive (add, delete or update) [optional] archive -- the zip file fileList -- a list of file paths or a single filepath cmd -- 'a': add, 'd': delete or 'u': update """ if not cmd in ['a','u','d']: raise ValueError("Invalid command %s" %cmd) if fileList is not None: if type(fileList) is list: self.fileList = fileList else: self.fileList = [fileList] """ for f in self.fileList: if not (os.path.isfile(f) or os.path.isdir(f)): raise ValueError("File %s not found" %f) """ if archive is not None: self.archive = archive if self.archive is not None: if os.path.isfile(self.archive) or cmd == 'a': cmdList = [self.ZipProgram, cmd, '-y'] if self.archiveType is not None: cmdList.append("-t"+self.archiveType) cmdList.append(self.archive) cmdList.extend(self.fileList) print cmdList self.call(cmdList) else: raise ValueError("Archive not found in %s" %(self.archive)) def usage(self): """ Returns the 7zip command line usage text. These options can be accessed directly with call. 7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Usage: 7z [...] [...] [<@listfiles...>] a: Add files to archive b: Benchmark d: Delete files from archive e: Extract files from archive (without using directory names) l: List contents of archive t: Test integrity of archive u: Update files to archive x: eXtract files with full paths -ai[r[-|0]]{@listfile|!wildcard}: Include archives -ax[r[-|0]]{@listfile|!wildcard}: eXclude archives -bd: Disable percentage indicator -i[r[-|0]]{@listfile|!wildcard}: Include filenames -m{Parameters}: set compression Method -o{Directory}: set Output directory -p{Password}: set Password -r[-|0]: Recurse subdirectories -scs{UTF-8 | WIN | DOS}: set charset for list files -sfx[{name}]: Create SFX archive -si[{name}]: read data from stdin -slt: show technical information for l (List) command -so: write data to stdout -ssc[-]: set sensitive case mode -ssw: compress shared files -t{Type}: Set type of archive -u[-][p#][q#][r#][x#][y#][z#][!newArchiveName]: Update options -v{Size}[b|k|m|g]: Create volumes -w[{path}]: assign Work directory. Empty path means a temporary directory -x[r[-|0]]]{@listfile|!wildcard}: eXclude filenames -y: assume Yes on all queries """ cmdList = [self.ZipProgram] self.call(cmdList) def add(self, archive=None, fileList=None): """ Add a file or list of files to an archive [optional] archive -- the zip file fileList -- a list of file paths or a single filepath """ self.modify(archive, fileList, 'a') def benchmark(self, archive): print "not implemented" def delete(self, archive=None, fileList=None): """ delete a file or list of files to an archive [optional] archive -- the zip file fileList -- a list of file paths or a single filepath """ self.modify(archive, fileList, 'd') def list(self, archive=None): """ List the contents of an archive [optional] archive -- the zip file """ if archive is not None: self.archive = archive if self.archive is not None: if os.path.isfile(self.archive): cmdList = [self.ZipProgram, "l", self.archive] self.call(cmdList) else: raise ValueError("Archive not found in %s" %(self.archive)) def test(self, archive=None): """ Test an archive for errors [optional] archive -- the zip file """ if archive is not None: self.archive = archive if self.archive is not None: if os.path.isfile(self.archive): cmdList = [self.ZipProgram, "t", self.archive] self.call(cmdList) else: raise ValueError("Archive not found in %s" %(self.archive)) def update(self, archive=None, fileList=None): """ Update a file or list of files to an archive only if the file does not exist or is newer than the existing file [optional] archive -- the zip file fileList -- a list of file paths or a single filepath """ self.modify(archive, fileList, 'u') def extract(self, archive=None, fullpath=True, outputDir=None): """ extract the contents of an archive [optional] archive -- the zip file fullpath -- extract with fullpaths outputDir -- specify the output directory """ cmdList = [self.ZipProgram] if fullpath: cmdList.append('x') else: cmdList.append('e') cmdList.append('-y') if outputDir is not None: self.outputDir = outputDir if self.outputDir is not None: cmdList.append('-o'+self.outputDir) if archive is not None: self.archive = archive if self.archive is not None: if os.path.isfile(self.archive): cmdList.append(self.archive) print cmdList self.call(cmdList) else: raise ValueError("Archive not found in %s" %(self.archive)) def crc(self, archive=None): """ Return the checksum of the archive archive -- the zip file """ if archive is not None: self.archive = archive if self.archive is not None: if os.path.isfile(self.archive): prev = 0 f = open(self.archive,"rb") for eachLine in f: prev = zlib.crc32(eachLine, prev) f.close() return "%X"%(prev & 0xFFFFFFFF) else: raise ValueError("Archive not found in %s" %(self.archive)) From eliben at gmail.com Sat Jan 6 12:21:42 2018 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 6 Jan 2018 09:21:42 -0800 Subject: [python-cffi] Fwd: Re: Progress migrating cffi and pycparser to libclang In-Reply-To: References: Message-ID: On Fri, Jan 5, 2018 at 1:15 AM, Etienne Robillard wrote: > Forwarding this thread to the CFFI developers... > > Re Paul: Thanks for your feedback. > > My intended audience are developers who can use hg to fetch/build source > code without pip. > > Best regards, > > Etienne > I'd like to understand the requirements here better, from the point of view of pycparser. What are you trying to accomplish? One of the issues with parsing "original" headers (with #defines and #includes and all that jazz) is that you have to match the compilation environment very precisely. When a compiler compiles a large project it could set dozens of -Ds, as well as -Is pointing to directories with additional headers. This is usually specified in Makefiles or something similar. Clang and libclang deal with this by having the concept of a compilation database which remembers for each file what the exact flags to compile it are (https://clang.llvm.org/docs/JSONCompilationDatabase.html) To make this *really* work for CFFI you'd have to take all of this into account -- do you really want to deal with this on the CFFI level? Eli > > -------- Message transf?r? -------- > Sujet : Re: Progress migrating cffi and pycparser to libclang > Date : Thu, 4 Jan 2018 21:25:27 +0000 > De : Paul Moore > Pour : Etienne Robillard > Copie ? : Python > > On 4 January 2018 at 21:02, Etienne Robillard wrote: > >> As a fork/extension for cffi, I have no particular opinion (I'm > >> unlikely to ever use it). But the advantage of pycparser is that it's > >> cross-platform and pure Python, so I doubt this will be acceptable for > >> inclusion into CFFI itself. > > > > CFFI/pycparser definitely need to be patched to support parsing standard C > > directives like #define and #include in the ffi.cdef() function. > > > > The easiest solution is to migrate the internal parsing code to libclang, a > > state-of-the art C/C++ compiler based on LLVM. > > I would strongly object to adding a dependency to cffi that couldn't > be automatically installed by pip as part of standard dependency > resolution (i.e., a PyPI hosted Python project with wheels available > for all common platforms - Linux, Mac OS and Windows). But ultimately > if you're proposing this as a change to cffi, you should be getting > the opinions of the cffi devs, not just asking on this list. (I notice > you have posted to the cffi mailing list, but haven't had any response > yet). > > Paul > > > -- > -- python-cffi: To unsubscribe from this group, send email to > python-cffi+unsubscribe at googlegroups.com. For more options, visit this > group at https://groups.google.com/d/forum/python-cffi?hl=en > --- > You received this message because you are subscribed to the Google Groups > "python-cffi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to python-cffi+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > From user at example.net Sat Jan 6 15:49:52 2018 From: user at example.net (J.O. Aho) Date: Sat, 6 Jan 2018 21:49:52 +0100 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> Message-ID: On 01/06/18 13:43, Etienne Robillard wrote: > My understanding of this vulnerability is that speculative indirect > calls in Linux kernel can be used to extract/filter memory content via > side-channels. Not just Linux, but all other OS:es, Microsoft and Apple been patching in secret as they have a closed source approach, but ms-windows needs at least one more patch before it can breath out, which will be released on Tuesday. From tkadm30 at yandex.com Sat Jan 6 16:23:31 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sat, 6 Jan 2018 16:23:31 -0500 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> Message-ID: <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> Le 2018-01-06 ? 15:49, J.O. Aho a ?crit?: > On 01/06/18 13:43, Etienne Robillard wrote: >> My understanding of this vulnerability is that speculative indirect >> calls in Linux kernel can be used to extract/filter memory content via >> side-channels. > Not just Linux, but all other OS:es, Microsoft and Apple been patching > in secret as they have a closed source approach, but ms-windows needs at > least one more patch before it can breath out, which will be released on > Tuesday. It's unclear to me whether AMD CPUs are affected by theses design flaws. Furthermore, I'd like to know if Python can mitigate hardware-specific timing attacks. Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From tomuxiong at gmx.com Sat Jan 6 16:35:02 2018 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Sat, 6 Jan 2018 22:35:02 +0100 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> Message-ID: <9ab963ce-f869-b8b6-301b-c86d551cf88a@gmx.com> On 01/06/2018 10:23 PM, Etienne Robillard wrote: > It's unclear to me whether AMD CPUs are affected by theses design flaws. As far as I understand, AMD (and possibly ARM) is unaffected by Meltdown (except for possibly some very new processors). It seems like basically all modern out of order processors are affected by spectre. Obviously there are many details/caveats. Here is a starting point: Quoted from: https://spectreattack.com/ --------------------------------------------- Which systems are affected by Meltdown? Desktop, Laptop, and Cloud computers may be affected by Meltdown. More technically, every Intel processor which implements out-of-order execution is potentially affected, which is effectively every processor since 1995 (except Intel Itanium and Intel Atom before 2013). We successfully tested Meltdown on Intel processor generations released as early as 2011. Currently, we have only verified Meltdown on Intel processors. At the moment, it is unclear whether ARM and AMD processors are also affected by Meltdown. Which systems are affected by Spectre? Almost every system is affected by Spectre: Desktops, Laptops, Cloud Servers, as well as Smartphones. More specifically, all modern processors capable of keeping many instructions in flight are potentially vulnerable. In particular, we have verified Spectre on Intel, AMD, and ARM processors. --------------------------------------------- Cheers, Thomas From grant.b.edwards at gmail.com Sat Jan 6 16:39:49 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 6 Jan 2018 21:39:49 +0000 (UTC) Subject: Spectre/Meltdown bug affecting Python ? References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> Message-ID: On 2018-01-06, Etienne Robillard wrote: > > > Le 2018-01-06 ? 15:49, J.O. Aho a ?crit?: >> On 01/06/18 13:43, Etienne Robillard wrote: >>> My understanding of this vulnerability is that speculative indirect >>> calls in Linux kernel can be used to extract/filter memory content via >>> side-channels. >> Not just Linux, but all other OS:es, Microsoft and Apple been patching >> in secret as they have a closed source approach, but ms-windows needs at >> least one more patch before it can breath out, which will be released on >> Tuesday. > > It's unclear to me whether AMD CPUs are affected by theses design flaws. Everybody seems to agree that AMD CPUs are not affected by Meltdown. The consensus is that AMD CPUs are probably affected by 2 of the 3 Spectre variants. > Furthermore, I'd like to know if Python can mitigate hardware-specific > timing attacks. For CPython, probably not. Anything that Cpython tried to do could be trivially defeated by using something like ctypes to make calls to arbitrary machine code that was written to a file. -- Grant Edwards grant.b.edwards Yow! Do I have a lifestyle at yet? gmail.com From ian.g.kelly at gmail.com Sat Jan 6 17:18:42 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sat, 06 Jan 2018 22:18:42 +0000 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> Message-ID: On Sat, Jan 6, 2018, 4:45 PM Grant Edwards wrote: > On 2018-01-06, Etienne Robillard wrote: > > > > > > Le 2018-01-06 ? 15:49, J.O. Aho a ?crit : > >> On 01/06/18 13:43, Etienne Robillard wrote: > >>> My understanding of this vulnerability is that speculative indirect > >>> calls in Linux kernel can be used to extract/filter memory content via > >>> side-channels. > >> Not just Linux, but all other OS:es, Microsoft and Apple been patching > >> in secret as they have a closed source approach, but ms-windows needs at > >> least one more patch before it can breath out, which will be released on > >> Tuesday. > > > > It's unclear to me whether AMD CPUs are affected by theses design flaws. > > Everybody seems to agree that AMD CPUs are not affected by Meltdown. > The consensus is that AMD CPUs are probably affected by 2 of the 3 > Spectre variants. > > > Furthermore, I'd like to know if Python can mitigate hardware-specific > > timing attacks. > > For CPython, probably not. Anything that Cpython tried to do could be > trivially defeated by using something like ctypes to make calls to > arbitrary machine code that was written to a file. > It sounds like you're talking about the case where the malicious code is hosted by Python. I agree that's probably not realistic to do anything about -- if you can run malicious code then you're probably not restricted to Python (and without knowing a lot about the attacks, I'm doubtful that it's possible to implement them in pure Python anyway). I think the OP was talking about protecting the data of Python programs from other malicious processes, however. The mitigation seems to be like it could reasonably be accomplished (at least for core Python -- extension code would be on its own). > From grant.b.edwards at gmail.com Sat Jan 6 17:47:25 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 6 Jan 2018 22:47:25 +0000 (UTC) Subject: Spectre/Meltdown bug affecting Python ? References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> <7189958a-8298-2b99-7c0b-cf28a29afc0d@yandex.com> Message-ID: On 2018-01-06, Ian Kelly wrote: > >> > Furthermore, I'd like to know if Python can mitigate hardware-specific >> > timing attacks. >> >> For CPython, probably not. Anything that Cpython tried to do could be >> trivially defeated by using something like ctypes to make calls to >> arbitrary machine code that was written to a file. >> > > It sounds like you're talking about the case where the malicious code is > hosted by Python. I agree that's probably not realistic to do anything > about -- if you can run malicious code then you're probably not restricted > to Python (and without knowing a lot about the attacks, I'm doubtful that > it's possible to implement them in pure Python anyway). Yes, that's what I was talking about. > I think the OP was talking about protecting the data of Python programs > from other malicious processes, however. The mitigation seems to be like it > could reasonably be accomplished (at least for core Python -- extension > code would be on its own). Ah, yes. Eventually it seems that just compiling CPython with a compiler that uses something like Google's "retpoline" should help: https://support.google.com/faqs/answer/7625886 Though I think I understand what the retpoline _is_, I don't really understand enough about the Spectre vulnerability say much else. -- Grant Edwards grant.b.edwards Yow! I'm having an at emotional outburst!! gmail.com From listes at salort.eu Sun Jan 7 09:00:07 2018 From: listes at salort.eu (Julien Salort) Date: Sun, 7 Jan 2018 15:00:07 +0100 Subject: Spectre/Meltdown bug affecting Python ? In-Reply-To: References: <80dfe3c8-1629-9615-b44c-df09a104e2ec@yandex.com> <33ff2ec0-68cd-3317-387d-1cc3bcda77be@yandex.com> Message-ID: Le 06/01/2018 ? 21:49, J.O. Aho a ?crit?: > Not just Linux, but all other OS:es, Microsoft and Apple been patching > in secret as they have a closed source approach, but ms-windows needs at > least one more patch before it can breath out, which will be released on > Tuesday. As a matter of fact, Apple kernel, xnu, is not closed source, https://opensource.apple.com/source/xnu/ From Irv at furrypants.com Sun Jan 7 13:08:43 2018 From: Irv at furrypants.com (Irv Kalb) Date: Sun, 7 Jan 2018 10:08:43 -0800 Subject: Python Inheritance Terminology In-Reply-To: <85h8rzltcl.fsf@benfinney.id.au> References: <3C7576D1-FC53-4EB1-B09D-BF4FD3677779@furrypants.com> <85h8rzltcl.fsf@benfinney.id.au> Message-ID: Thanks for the confirmation, and for the link. Irv > On Jan 5, 2018, at 4:32 PM, Ben Finney wrote: > > Irv Kalb writes: > >> I'm doing some writing for an upcoming course on OOP using Python. > > Welcome, and congratulations for using Python in this work. > >> I'd like to know if there are "official" or even standard terms that >> are used to describe a class that is inherited from, and the class >> that is doing the inheriting. From my reading (especially the PSF >> docs.python.org ), it looks like the terms >> would be "base class" and "subclass". > > Standard (?official?) terms are most likely to be had from the language > reference . I would recommend > the glossary , but with the > caveat that many flaws have been found in recent years. > >> However, in books about Python and other languages, I have also seen the terms: >> >> base class & derived class >> parent class & child class >> superclass & subclass > > The only term I take issue with there is ?superclass?. In a > multiple-inheritance system, such as provided by Python, the superclass > is *not* necessarily the base class. See this article from 2011 > . > >> So, are base class & subclass the proper terms? > > In my opinion you will be correct to use those terms. Which is not to > say that other terms aren't also good. > > -- > \ ?The greatest tragedy in mankind's entire history may be the | > `\ hijacking of morality by religion.? ?Arthur C. Clarke, 1991 | > _o__) | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Sun Jan 7 14:48:10 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 06:48:10 +1100 Subject: Why does __ne__ exist? Message-ID: When you create a Python class, you can create dunder methods to define how your objects respond to the standard operators. With comparison operators, Python will happily switch the operands around to find a method to call: >>> class Spam(): ... def __lt__(self, other): ... print("%s is less than %s" % (self, other)) ... return True ... >>> Spam() < 2 <__main__.Spam object at 0x7fb7557b1fd0> is less than 2 True >>> 3 > Spam() <__main__.Spam object at 0x7fb7557b1fd0> is less than 3 True >>> 4 > Spam() < 5 <__main__.Spam object at 0x7fb7557b1fd0> is less than 4 <__main__.Spam object at 0x7fb7557b1fd0> is less than 5 True But Python will not automatically assume the converse: >>> Spam() >= 6 Traceback (most recent call last): File "", line 1, in TypeError: '>=' not supported between instances of 'Spam' and 'int' >>> Spam() > 7 Traceback (most recent call last): File "", line 1, in TypeError: '>' not supported between instances of 'Spam' and 'int' This is good. This is correct. For inequalities, you can't assume that >= is the exact opposite of < (for example, sets don't behave like numbers, so "x <= y" is very different from ) From rosuav at gmail.com Sun Jan 7 14:55:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 06:55:34 +1100 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: Whoops, premature send. Picking up from the last paragraph. This is good. This is correct. For inequalities, you can't assume that >= is the exact opposite of < or the combination of < and == (for example, sets don't behave like numbers, so "x <= y" is very different from "x < y or x == y"). But the one that confuses me is != or __ne__. If you don't create it, you get default behaviour: >>> class Ham: ... def __eq__(self, other): ... print("%s equals %s" % (self, other)) ... return True ... >>> Ham() == 1 <__main__.Ham object at 0x7fb7557c0278> equals 1 True >>> 2 == Ham() <__main__.Ham object at 0x7fb7557c0278> equals 2 True >>> Ham() != 3 <__main__.Ham object at 0x7fb7557c0278> equals 3 False >>> 4 != Ham() <__main__.Ham object at 0x7fb7557c0278> equals 4 False >>> x = Ham() >>> x == x <__main__.Ham object at 0x7fb7557b80f0> equals <__main__.Ham object at 0x7fb7557b80f0> True >>> x != x <__main__.Ham object at 0x7fb7557b80f0> equals <__main__.Ham object at 0x7fb7557b80f0> False Under what circumstances would you want "x != y" to be different from "not (x == y)" ? How would this make for sane behaviour? Even when other things go weird with equality checks, that basic parallel is always maintained: >>> z = float("nan") >>> z == z False >>> z != z True Python gives us a "not in" operator that uses __contains__ and then negates the result. There is no way for "x not in y" to be anything different from "not (x in y)", as evidenced by the peephole optimizer: >>> dis.dis("x not in y") 1 0 LOAD_NAME 0 (x) 2 LOAD_NAME 1 (y) 4 COMPARE_OP 7 (not in) 6 RETURN_VALUE >>> dis.dis("not (x in y)") 1 0 LOAD_NAME 0 (x) 2 LOAD_NAME 1 (y) 4 COMPARE_OP 7 (not in) 6 RETURN_VALUE So why isn't != done the same way? Is it historical? ChrisA From tjol at tjol.eu Sun Jan 7 15:13:25 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 7 Jan 2018 21:13:25 +0100 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> On 07/01/18 20:55, Chris Angelico wrote: > Under what circumstances would you want "x != y" to be different from > "not (x == y)" ? In numpy, __eq__ and __ne__ do not, in general, return bools. Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as np >>> a = np.array([1,2,3,4]) >>> b = np.array([0,2,0,4]) >>> a == b array([False, True, False, True], dtype=bool) >>> a != b array([ True, False, True, False], dtype=bool) >>> not (a == b) Traceback (most recent call last): File "", line 1, in ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() >>> ~(a == b) array([ True, False, True, False], dtype=bool) >>> I couldn't tell you why this was originally allowed, but it does turn out to be strangely useful. (As far as the numpy API is concerned, it would be even nicer if 'not' could be overridden, IMHO) Cheers Thomas From rosuav at gmail.com Sun Jan 7 15:33:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 07:33:54 +1100 Subject: Why does __ne__ exist? In-Reply-To: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> Message-ID: On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: > On 07/01/18 20:55, Chris Angelico wrote: >> Under what circumstances would you want "x != y" to be different from >> "not (x == y)" ? > > In numpy, __eq__ and __ne__ do not, in general, return bools. > >>>> a = np.array([1,2,3,4]) >>>> b = np.array([0,2,0,4]) >>>> a == b > array([False, True, False, True], dtype=bool) >>>> a != b > array([ True, False, True, False], dtype=bool) Thanks, that's the kind of example I was looking for. Though numpy doesn't drive the core language development much, so the obvious next question is: was this why __ne__ was implemented, or was there some other reason? This example shows how it can be useful, but not why it exists. >>>> not (a == b) > Traceback (most recent call last): > File "", line 1, in > ValueError: The truth value of an array with more than one element is > ambiguous. Use a.any() or a.all() Which means that this construct is still never going to come up in good code. >>>> ~(a == b) > array([ True, False, True, False], dtype=bool) >>>> > > I couldn't tell you why this was originally allowed, but it does turn > out to be strangely useful. (As far as the numpy API is concerned, it > would be even nicer if 'not' could be overridden, IMHO) I'm glad 'not' can't be overridden; it'd be too hard to reason about a piece of code if even the basic boolean operations could change. If you want overridables, you have &|~ for the bitwise operators (which is how numpy does things). Has numpy ever asked for a "not in" dunder method (__not_contained__ or something)? It's a strange anomaly that "not (x in y)" can be perfectly safely optimized to "x not in y", yet basic equality has to have separate handling. The default handling does mean that you can mostly ignore __ne__ and expect things to work, but if you subclass something that has both, it'll break: class Foo: def __eq__(self, other): print("Foo: %s == %s" % (self, other)) return True def __ne__(self, other): print("Foo: %s != %s" % (self, other)) return False class Bar(Foo): def __eq__(self, other): print("Bar: %s == %s" % (self, other)) return False >>> Bar() == 1 Bar: <__main__.Bar object at 0x7f40ebf3a128> == 1 False >>> Bar() != 1 Foo: <__main__.Bar object at 0x7f40ebf3a128> != 1 False Obviously this trivial example looks stupid, but imagine if the equality check in the subclass USUALLY gives the same result as the superclass, but differs in rare situations. Maybe you create a "bag" class that functions a lot like collections.Counter but ignores zeroes when comparing: >>> class Bag(collections.Counter): ... def __eq__(self, other): ... # Disregard any zero entries when comparing to another Bag ... return {k:c for k,c in self.items() if c} == {k:c for k,c in other.items() if c} ... >>> b1 = Bag("aaabccdd") >>> b2 = Bag("aaabccddq") >>> b2["q"] -= 1 >>> b1 == b2 True >>> b1 != b2 True >>> dict(b1) == dict(b2) False >>> dict(b1) != dict(b2) True The behaviour of __eq__ is normal and sane. But since there's no __ne__, the converse comparison falls back on dict.__ne__, not on object.__ne__. ChrisA From bc at freeuk.com Sun Jan 7 15:41:08 2018 From: bc at freeuk.com (bartc) Date: Sun, 7 Jan 2018 20:41:08 +0000 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: On 07/01/2018 19:55, Chris Angelico wrote: > Under what circumstances would you want "x != y" to be different from > "not (x == y)" ? How would this make for sane behaviour? Presumably so that any behaviour any be programmed when overriding these operators. Maybe someone wants to do weird stuff with == that doesn't yield a true or false result, so that you can't just reverse it for !=. For example (perhaps this is similar to what was suggested in another post): (10,20,30) == (10,20,40) yields (1,1,0) (10,20,30) != (10,20,40) yields (0,0,1) Although here, you would probably define 'not' so that 'not (1,1,0)' does actually yield '(0,0,1)'. So clearly I need a weirder example. Even when > other things go weird with equality checks, that basic parallel is > always maintained: > >>>> z = float("nan") >>>> z == z > False >>>> z != z > True > > Python gives us a "not in" operator that uses __contains__ and then > negates the result. There is no way for "x not in y" to be anything > different from "not (x in y)", as evidenced by the peephole optimizer: > >>>> dis.dis("x not in y") > 1 0 LOAD_NAME 0 (x) > 2 LOAD_NAME 1 (y) > 4 COMPARE_OP 7 (not in) > 6 RETURN_VALUE >>>> dis.dis("not (x in y)") > 1 0 LOAD_NAME 0 (x) > 2 LOAD_NAME 1 (y) > 4 COMPARE_OP 7 (not in) I get '4 COMPARE OP 6 (in)' here. So they are distinct ops. 'not in' doesn't just call 'in', then apply 'not'. Not here anyway. -- bartc From breamoreboy at gmail.com Sun Jan 7 16:06:21 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 7 Jan 2018 13:06:21 -0800 (PST) Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> On Sunday, January 7, 2018 at 7:55:57 PM UTC, Chris Angelico wrote: > Whoops, premature send. Picking up from the last paragraph. > > This is good. This is correct. For inequalities, you can't assume that > >= is the exact opposite of < or the combination of < and == (for > example, sets don't behave like numbers, so "x <= y" is very different > from "x < y or x == y"). But the one that confuses me is != or __ne__. > If you don't create it, you get default behaviour: > > >>> class Ham: > ... def __eq__(self, other): > ... print("%s equals %s" % (self, other)) > ... return True > ... > >>> Ham() == 1 > <__main__.Ham object at 0x7fb7557c0278> equals 1 > True > >>> 2 == Ham() > <__main__.Ham object at 0x7fb7557c0278> equals 2 > True > >>> Ham() != 3 > <__main__.Ham object at 0x7fb7557c0278> equals 3 > False > >>> 4 != Ham() > <__main__.Ham object at 0x7fb7557c0278> equals 4 > False > >>> x = Ham() > >>> x == x > <__main__.Ham object at 0x7fb7557b80f0> equals <__main__.Ham object at > 0x7fb7557b80f0> > True > >>> x != x > <__main__.Ham object at 0x7fb7557b80f0> equals <__main__.Ham object at > 0x7fb7557b80f0> > False > > Under what circumstances would you want "x != y" to be different from > "not (x == y)" ? How would this make for sane behaviour? Even when > other things go weird with equality checks, that basic parallel is > always maintained: > > >>> z = float("nan") > >>> z == z > False > >>> z != z > True > > Python gives us a "not in" operator that uses __contains__ and then > negates the result. There is no way for "x not in y" to be anything > different from "not (x in y)", as evidenced by the peephole optimizer: > > >>> dis.dis("x not in y") > 1 0 LOAD_NAME 0 (x) > 2 LOAD_NAME 1 (y) > 4 COMPARE_OP 7 (not in) > 6 RETURN_VALUE > >>> dis.dis("not (x in y)") > 1 0 LOAD_NAME 0 (x) > 2 LOAD_NAME 1 (y) > 4 COMPARE_OP 7 (not in) > 6 RETURN_VALUE > > So why isn't != done the same way? Is it historical? > > ChrisA I'd say this is certainly historical, remembering that in Python 2 you used to be able to compare all sorts of things, whereas in Python 3 you'll get:- >>> 1 < "a" Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'int' and 'str' >>> This seems to be confirmed by the following. From the third paragraph at https://docs.python.org/2/reference/datamodel.html#object.__ne__ "There are no implied relationships among the comparison operators. The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected...". Compare that with the Python 3 equivalent "By default, __ne__() delegates to __eq__() and inverts the result unless it is NotImplemented. There are no other implied relationships among the comparison operators, for example, the truth of (x References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <151517499074.367782.5631796281028137729.XPN@Welt.Netz> <96159bc9-6c99-d00f-1941-7c938cb212a8@gmail.com> Message-ID: Am 05.01.18 um 22:15 schrieb Michael Torrie: > Please, no! We don't need emoji in this group. Fortunately the vast > majority of posters use plain text (as is the etiquette) and so we don't > have to worry about that kind of nonsense. It's not needed, but shouldn't pose any big problems with modern software? I'm using Thunderbird over NNTP, and this shows up as graphics to me: ? ? (a snake and a computer) Unless this message is blocked at the NNTP-List boundary, it should show the same on any recent mail reader. Christian From rosuav at gmail.com Sun Jan 7 16:51:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 08:51:08 +1100 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: On Mon, Jan 8, 2018 at 7:41 AM, bartc wrote: > On 07/01/2018 19:55, Chris Angelico wrote: > >> Under what circumstances would you want "x != y" to be different from >> "not (x == y)" ? How would this make for sane behaviour? > > > Presumably so that any behaviour any be programmed when overriding these > operators. > > Maybe someone wants to do weird stuff with == that doesn't yield a true or > false result, so that you can't just reverse it for !=. > > For example (perhaps this is similar to what was suggested in another post): > > (10,20,30) == (10,20,40) yields (1,1,0) > (10,20,30) != (10,20,40) yields (0,0,1) > > Although here, you would probably define 'not' so that 'not (1,1,0)' does > actually yield '(0,0,1)'. > > So clearly I need a weirder example. With tuples, I absolutely agree with Python's current behaviour: the tuples you give are simply not equal. A tuple doesn't represent a vector; it represents a specific collection of values, like the coordinates of a point in 2D or 3D space. If you look at the two points (1,5) and (3,5), they aren't "half equal". They're different points, at different locations. They happen to have the same elevation, but that's just a point of curiosity. > Even when >> >> other things go weird with equality checks, that basic parallel is >> always maintained: >> >>>>> z = float("nan") >>>>> z == z >> >> False >>>>> >>>>> z != z >> >> True >> >> Python gives us a "not in" operator that uses __contains__ and then >> negates the result. There is no way for "x not in y" to be anything >> different from "not (x in y)", as evidenced by the peephole optimizer: >> >>>>> dis.dis("x not in y") >> >> 1 0 LOAD_NAME 0 (x) >> 2 LOAD_NAME 1 (y) >> 4 COMPARE_OP 7 (not in) >> 6 RETURN_VALUE >>>>> >>>>> dis.dis("not (x in y)") >> >> 1 0 LOAD_NAME 0 (x) >> 2 LOAD_NAME 1 (y) >> 4 COMPARE_OP 7 (not in) > > > I get '4 COMPARE OP 6 (in)' here. So they are distinct ops. 'not in' > doesn't just call 'in', then apply 'not'. Not here anyway. > The fact that your Python doesn't optimize it is actually beside the point; if _any_ Python interpreter can optimize this down, it must be semantically identical. I did this on CPython 3.7, fwiw, but it doesn't really matter. ChrisA From gheskett at shentel.net Sun Jan 7 17:27:02 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 17:27:02 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: <201801071727.02798.gheskett@shentel.net> On Sunday 07 January 2018 16:22:57 Christian Gollwitzer wrote: > Am 05.01.18 um 22:15 schrieb Michael Torrie: > > Please, no! We don't need emoji in this group. Fortunately the vast > > majority of posters use plain text (as is the etiquette) and so we > > don't have to worry about that kind of nonsense. > > It's not needed, but shouldn't pose any big problems with modern > software? I'm using Thunderbird over NNTP, and this shows up as > graphics to me: > > ? ? > But here its broken and I am looking at two pairs of vertical boxes because it is not properly mime'd. If you use chars or gliphs from a non-default charset, it needs to demarcated with a mime-boundary marker followed by the new type definition. Your email/news agent did not do that. > (a snake and a computer) Unless this message is blocked at the > NNTP-List boundary, it should show the same on any recent mail reader. > > Christian Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From bc at freeuk.com Sun Jan 7 17:32:30 2018 From: bc at freeuk.com (bartc) Date: Sun, 7 Jan 2018 22:32:30 +0000 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: On 07/01/2018 21:51, Chris Angelico wrote: > On Mon, Jan 8, 2018 at 7:41 AM, bartc wrote: >> Maybe someone wants to do weird stuff with == that doesn't yield a true or >> false result, so that you can't just reverse it for !=. >> >> For example (perhaps this is similar to what was suggested in another post): >> >> (10,20,30) == (10,20,40) yields (1,1,0) >> (10,20,30) != (10,20,40) yields (0,0,1) > With tuples, I absolutely agree with Python's current behaviour: the > tuples you give are simply not equal. A tuple doesn't represent a > vector; it represents a specific collection of values, like the > coordinates of a point in 2D or 3D space. If you look at the two > points (1,5) and (3,5), they aren't "half equal". They're different > points, at different locations. They happen to have the same > elevation, but that's just a point of curiosity. My (10,20,30) were meant to represent some user-defined type, not an ordinary tuple. And someone might intend that == operates on two instances of that type as thought they were vectors. Or any other kind of behaviour as I said. But not necessarily some logical inverse of !=. >>>>>> dis.dis("not (x in y)") >>> >>> 1 0 LOAD_NAME 0 (x) >>> 2 LOAD_NAME 1 (y) >>> 4 COMPARE_OP 7 (not in) >> >> >> I get '4 COMPARE OP 6 (in)' here. So they are distinct ops. 'not in' >> doesn't just call 'in', then apply 'not'. Not here anyway. >> > > The fact that your Python doesn't optimize it is actually beside the > point; if _any_ Python interpreter can optimize this down, it must be > semantically identical. Actually I didn't see the 'not' on the outside of the brackets. I thought the two expressions were 'not in' and 'in' and that you might have transcribed the '7 (not in)' part wrongly. But this reduction isn't necessarily an optimisation. It might just be a syntactical transformation from 'not (a in b)' to '(a not in b)' The disassembly for 'in' and 'not in' suggests that these are two independent operators, which could indeed have behaviours that are not complements of each other. On the other hand, when you /did/ want to evaluate 'in' followed by 'not', then you want: not (a in b) # compare using 'not in' to do the same thing as: temp = a in b # compare using 'in' not temp # apply unary not Then there might not be the freedom to have in/not in have independent behaviours. -- bartc From random832 at fastmail.com Sun Jan 7 17:37:14 2018 From: random832 at fastmail.com (Random832) Date: Sun, 07 Jan 2018 17:37:14 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071727.02798.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> Message-ID: <1515364634.3033237.1227282920.74621B94@webmail.messagingengine.com> On Sun, Jan 7, 2018, at 17:27, Gene Heskett wrote: > > > > ? ? > > > But here its broken and I am looking at two pairs of vertical boxes > because it is not properly mime'd. If you use chars or gliphs from a > non-default charset, it needs to demarcated with a mime-boundary marker > followed by the new type definition. Your email/news agent did not do > that. UTF-8 is the default character set, and anyway his message does have a content-type of 'text/plain; charset="utf-8"; Format="flowed"'. Your environment not having font support and/or support for non-BMP characters is not a deficiency in the message. From rosuav at gmail.com Sun Jan 7 17:41:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 09:41:35 +1100 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071727.02798.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> Message-ID: On Mon, Jan 8, 2018 at 9:27 AM, Gene Heskett wrote: > On Sunday 07 January 2018 16:22:57 Christian Gollwitzer wrote: > >> Am 05.01.18 um 22:15 schrieb Michael Torrie: >> > Please, no! We don't need emoji in this group. Fortunately the vast >> > majority of posters use plain text (as is the etiquette) and so we >> > don't have to worry about that kind of nonsense. >> >> It's not needed, but shouldn't pose any big problems with modern >> software? I'm using Thunderbird over NNTP, and this shows up as >> graphics to me: >> >> ? ? >> > But here its broken and I am looking at two pairs of vertical boxes > because it is not properly mime'd. If you use chars or gliphs from a > non-default charset, it needs to demarcated with a mime-boundary marker > followed by the new type definition. Your email/news agent did not do > that. > Non-default charset? Content-Type: text/plain; charset="utf-8"; Format="flowed" I'm pretty sure UTF-8 is a very default charset. Had no problems at all vieweing the original email. ChrisA From Richard at Damon-Family.org Sun Jan 7 17:47:38 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 7 Jan 2018 17:47:38 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> Message-ID: On 1/7/18 5:27 PM, Gene Heskett wrote: > On Sunday 07 January 2018 16:22:57 Christian Gollwitzer wrote: > >> Am 05.01.18 um 22:15 schrieb Michael Torrie: >>> Please, no! We don't need emoji in this group. Fortunately the vast >>> majority of posters use plain text (as is the etiquette) and so we >>> don't have to worry about that kind of nonsense. >> >> It's not needed, but shouldn't pose any big problems with modern >> software? I'm using Thunderbird over NNTP, and this shows up as >> graphics to me: >> >> ? ? >> > But here its broken and I am looking at two pairs of vertical boxes > because it is not properly mime'd. If you use chars or gliphs from a > non-default charset, it needs to demarcated with a mime-boundary marker > followed by the new type definition. Your email/news agent did not do > that. The headers say: Content-Type: text/plain; charset=UTF-8; format=flowed So it DOES declare the right character set. But it also says: Content-Transfer-Encoding: 7bit Which is incorrect, as the message is actually 8bit encoded (since the Emoji aren't in the first 127 characters, so their UTF-8 encoding isn't 7-bit. Some software might have messed up the message in transit due to that error. It came up on the mailing list correct, and the gateway converted the encoding to Base64 which should work. If you don't have a font installed that your reader can find those characters, that could be another issue. > >> (a snake and a computer) Unless this message is blocked at the >> NNTP-List boundary, it should show the same on any recent mail reader. >> >> Christian > > > Cheers, Gene Heskett > From rosuav at gmail.com Sun Jan 7 17:49:49 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 09:49:49 +1100 Subject: Why does __ne__ exist? In-Reply-To: References: Message-ID: On Mon, Jan 8, 2018 at 9:32 AM, bartc wrote: > On 07/01/2018 21:51, Chris Angelico wrote: >>>>>>> dis.dis("not (x in y)") >>>> >>>> >>>> 1 0 LOAD_NAME 0 (x) >>>> 2 LOAD_NAME 1 (y) >>>> 4 COMPARE_OP 7 (not in) >>> >>> >>> >>> I get '4 COMPARE OP 6 (in)' here. So they are distinct ops. 'not in' >>> doesn't just call 'in', then apply 'not'. Not here anyway. >>> >> >> The fact that your Python doesn't optimize it is actually beside the >> point; if _any_ Python interpreter can optimize this down, it must be >> semantically identical. > > > Actually I didn't see the 'not' on the outside of the brackets. I thought > the two expressions were 'not in' and 'in' and that you might have > transcribed the '7 (not in)' part wrongly. That's why I don't transcribe - I copy and paste. It's way WAY safer that way. > But this reduction isn't necessarily an optimisation. It might just be a > syntactical transformation from 'not (a in b)' to '(a not in b)' > > The disassembly for 'in' and 'not in' suggests that these are two > independent operators, which could indeed have behaviours that are not > complements of each other. Uhm, if the peephole optimizer does a syntactical transformation, it MUST retain the semantics. The disassembly for "in" and "not in" shows that they are independent, but the disassembly for "not (x in y)" proves that they are semantically linked. > On the other hand, when you /did/ want to evaluate 'in' followed by 'not', > then you want: > > not (a in b) # compare using 'not in' > > to do the same thing as: > > temp = a in b # compare using 'in' > not temp # apply unary not > > Then there might not be the freedom to have in/not in have independent > behaviours. And the whole point of my post is that there is no such freedom - that "not in" MUST always give the exact converse of "in". (And if __contains__ returns something other than a boolean, it is coerced before the operator returns it.) Yet equality is not like that. Hence my post. ChrisA From random832 at fastmail.com Sun Jan 7 18:25:52 2018 From: random832 at fastmail.com (Random832) Date: Sun, 07 Jan 2018 18:25:52 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> Message-ID: <1515367552.3042049.1227312192.461EC928@webmail.messagingengine.com> On Sun, Jan 7, 2018, at 17:47, Richard Damon wrote: > But it also says: > > Content-Transfer-Encoding: 7bit > > Which is incorrect, as the message is actually 8bit encoded (since the > Emoji aren't in the first 127 characters, so their UTF-8 encoding isn't > 7-bit. Some software might have messed up the message in transit due to > that error. Well, the fact that the emoji survived the round-trip and showed up properly in his reply (and yours) led me to rule out the possibility that anything like that had happened. Plus, if that had happened, the result wouldn't be boxes, but a series of ASCII characters (some of which are control characters, and some of which are printable). From rosuav at gmail.com Sun Jan 7 18:43:33 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 10:43:33 +1100 Subject: Why does __ne__ exist? In-Reply-To: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> Message-ID: On Mon, Jan 8, 2018 at 8:06 AM, wrote: > From the third paragraph at https://docs.python.org/2/reference/datamodel.html#object.__ne__ "There are no implied relationships among the comparison operators. The truth of x==y does not imply that x!=y is false. Accordingly, when defining __eq__(), one should also define __ne__() so that the operators will behave as expected...". Compare that with the Python 3 equivalent "By default, __ne__() delegates to __eq__() and inverts the result unless it is NotImplemented. There are no other implied relationships among the comparison operators, for example, the truth of (x Ah, I forgot to check the Py2 docs. So, yeah, sounds like it's basically historical. I'm still not sure why it was done in the first place, but it looks like it's the sort of thing that wouldn't be done now. ChrisA From gheskett at shentel.net Sun Jan 7 18:50:01 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 18:50:01 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <1515364634.3033237.1227282920.74621B94@webmail.messagingengine.com> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> <1515364634.3033237.1227282920.74621B94@webmail.messagingengine.com> Message-ID: <201801071850.01222.gheskett@shentel.net> On Sunday 07 January 2018 17:37:14 Random832 wrote: > On Sun, Jan 7, 2018, at 17:27, Gene Heskett wrote: > > > ? ? > > > > But here its broken and I am looking at two pairs of vertical boxes > > because it is not properly mime'd. If you use chars or gliphs from a > > non-default charset, it needs to demarcated with a mime-boundary > > marker followed by the new type definition. Your email/news agent > > did not do that. > > UTF-8 is the default character set, and anyway his message does have a > content-type of 'text/plain; charset="utf-8"; Format="flowed"'. Your > environment not having font support and/or support for non-BMP > characters is not a deficiency in the message. That, now that you mention it, could also effect this as I see it, my default kmail message body font is hack 14 in deference to the age of my eyes. My system default font is I believe utf-8. That is not a kmail settable option. But if I uncheck the "use custom fonts", it is still two pair of character outlines. So to what family of fonts do these characters belong? Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From ethan at stoneleaf.us Sun Jan 7 18:53:23 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 07 Jan 2018 15:53:23 -0800 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> Message-ID: <5A52B2F3.60801@stoneleaf.us> On 01/07/2018 12:33 PM, Chris Angelico wrote: > On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: >> On 07/01/18 20:55, Chris Angelico wrote: >>> Under what circumstances would you want "x != y" to be different from >>> "not (x == y)" ? >> >> In numpy, __eq__ and __ne__ do not, in general, return bools. >> >>>>> a = np.array([1,2,3,4]) >>>>> b = np.array([0,2,0,4]) >>>>> a == b >> array([False, True, False, True], dtype=bool) >>>>> a != b >> array([ True, False, True, False], dtype=bool) > > Thanks, that's the kind of example I was looking for. Though numpy > doesn't drive the core language development much, so the obvious next > question is: was this why __ne__ was implemented, or was there some > other reason? This example shows how it can be useful, but not why it > exists. Actually, I think it is why it exists. If I recall correctly, the addition of the six comparative operators* was added at the behest of the scientific/numerical community. -- ~Ethan~ * Yeah, I can't remember the cool name for those six operators at the moment. :( From ben+python at benfinney.id.au Sun Jan 7 18:55:15 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 08 Jan 2018 10:55:15 +1100 Subject: Why does __ne__ exist? References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> Message-ID: <858td9kyvw.fsf@benfinney.id.au> Chris Angelico writes: > So, yeah, sounds like it's basically historical. I'm still not sure > why it was done in the first place, but it looks like it's the sort of > thing that wouldn't be done now. I'm not understanding why you speculate that it wouldn't be done today. We've established that it is useful to allow data types to define their own meaning of ?equal? and ?not equal?, like many other operations. Is that not good enough reason to allow it still? -- \ ?Science is what we understand well enough to explain to a | `\ computer. Art is everything else we do.? ?Donald Knuth, 1996 | _o__) | Ben Finney From rosuav at gmail.com Sun Jan 7 19:04:12 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 11:04:12 +1100 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071850.01222.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> <1515364634.3033237.1227282920.74621B94@webmail.messagingengine.com> <201801071850.01222.gheskett@shentel.net> Message-ID: On Mon, Jan 8, 2018 at 10:50 AM, Gene Heskett wrote: > On Sunday 07 January 2018 17:37:14 Random832 wrote: > >> On Sun, Jan 7, 2018, at 17:27, Gene Heskett wrote: >> > > ? ? >> > >> > But here its broken and I am looking at two pairs of vertical boxes >> > because it is not properly mime'd. If you use chars or gliphs from a >> > non-default charset, it needs to demarcated with a mime-boundary >> > marker followed by the new type definition. Your email/news agent >> > did not do that. >> >> UTF-8 is the default character set, and anyway his message does have a >> content-type of 'text/plain; charset="utf-8"; Format="flowed"'. Your >> environment not having font support and/or support for non-BMP >> characters is not a deficiency in the message. > > That, now that you mention it, could also effect this as I see it, my > default kmail message body font is hack 14 in deference to the age of my > eyes. > > My system default font is I believe utf-8. That is not a kmail settable > option. But if I uncheck the "use custom fonts", it is still two pair of > character outlines. So to what family of fonts do these characters > belong? > You're conflating a few different things here. The character set is the Universal Character Set, basically synonymous with "Unicode". The encoding is UTF-8 and is a way to represent Unicode characters as bytes. The transfer encoding is base 64 (sometimes called "MIME encoding"), at least in the email version of it - I don't know what the original NG post used, and it may have been different. The font used is a mapping from character codes to displayable glyphs. Everything except the font is under the sender's control, and (at least in the mailing list version) was all fine. If your font can't handle those characters, the font renderer should still recognize that they are characters, and put boxes (maybe with the codepoints written in them). On my Debian Linux systems, I can "sudo apt install unifont" to grab a font that's used as a fallback for any characters not found in other fonts. Quoting from the package's description: "The philosophy behind this font, though, is that anything meaningful is better than an empty box for an unknown glyph." It's not perfect, but it's way better than nothing. I don't know how many of the emoji are included, but it's worth a try. ChrisA From gheskett at shentel.net Sun Jan 7 19:07:48 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 19:07:48 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <1515367552.3042049.1227312192.461EC928@webmail.messagingengine.com> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <1515367552.3042049.1227312192.461EC928@webmail.messagingengine.com> Message-ID: <201801071907.48721.gheskett@shentel.net> On Sunday 07 January 2018 18:25:52 Random832 wrote: > On Sun, Jan 7, 2018, at 17:47, Richard Damon wrote: > > But it also says: > > > > Content-Transfer-Encoding: 7bit > > > > Which is incorrect, as the message is actually 8bit encoded (since > > the Emoji aren't in the first 127 characters, so their UTF-8 > > encoding isn't 7-bit. Some software might have messed up the message > > in transit due to that error. > > Well, the fact that the emoji survived the round-trip and showed up > properly in his reply (and yours) led me to rule out the possibility > that anything like that had happened. Plus, if that had happened, the > result wouldn't be boxes, but a series of ASCII characters (some of > which are control characters, and some of which are printable). Its just boxes here, and I just spent the better part of half an hour trying all the fonts available to kmail, without see anything but variable sizes of twin boxes. Looking at the raw message its also marked transfer content encoding = base64, which I'd assume destroys any semblance of an 8 bit encoding. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From rosuav at gmail.com Sun Jan 7 19:09:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 11:09:18 +1100 Subject: Why does __ne__ exist? In-Reply-To: <858td9kyvw.fsf@benfinney.id.au> References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> Message-ID: On Mon, Jan 8, 2018 at 10:55 AM, Ben Finney wrote: > Chris Angelico writes: > >> So, yeah, sounds like it's basically historical. I'm still not sure >> why it was done in the first place, but it looks like it's the sort of >> thing that wouldn't be done now. > > I'm not understanding why you speculate that it wouldn't be done today. > > We've established that it is useful to allow data types to define their > own meaning of ?equal? and ?not equal?, like many other operations. Is > that not good enough reason to allow it still? The fact that container types can define "contains" but can't define "doesn't contain", and that (as of Py3) there's proper default handling, suggests that it's not as big a priority now. Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, just like with __contains__. Go ahead: sell the notion of __ne__. Pitch it, show why we absolutely need to allow this. Make sure you mention the potential confusion when subclassing. Be sure to show why it's okay for "not in" to force to boolean but "==" should allow any return value. ChrisA From rosuav at gmail.com Sun Jan 7 19:16:35 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 11:16:35 +1100 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071907.48721.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <1515367552.3042049.1227312192.461EC928@webmail.messagingengine.com> <201801071907.48721.gheskett@shentel.net> Message-ID: On Mon, Jan 8, 2018 at 11:07 AM, Gene Heskett wrote: > On Sunday 07 January 2018 18:25:52 Random832 wrote: > >> On Sun, Jan 7, 2018, at 17:47, Richard Damon wrote: >> > But it also says: >> > >> > Content-Transfer-Encoding: 7bit >> > >> > Which is incorrect, as the message is actually 8bit encoded (since >> > the Emoji aren't in the first 127 characters, so their UTF-8 >> > encoding isn't 7-bit. Some software might have messed up the message >> > in transit due to that error. >> >> Well, the fact that the emoji survived the round-trip and showed up >> properly in his reply (and yours) led me to rule out the possibility >> that anything like that had happened. Plus, if that had happened, the >> result wouldn't be boxes, but a series of ASCII characters (some of >> which are control characters, and some of which are printable). > > Its just boxes here, and I just spent the better part of half an hour > trying all the fonts available to kmail, without see anything but > variable sizes of twin boxes. Looking at the raw message its also marked > transfer content encoding = base64, which I'd assume destroys any > semblance of an 8 bit encoding. Proper font substitution would mean that, no matter which font you have selected, unknown glyphs will be drawn from some other font that supports them. So you probably don't have any font installed that has those glyphs. That should be a solvable problem, though it'll depend on finding one that you like. Transfer encoding of base64 just means that eight-bit encodings get wrapped up in a four-for-three carriage method. You take three bytes and represent them in four letters. It's an encoding that can be used for binary files or eight-bit text alike. Doesn't change the content itself; once you decode base64, you get back a series of bytes, which can be decoded into Unicode text by the normal methods. ChrisA From gheskett at shentel.net Sun Jan 7 19:25:41 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 19:25:41 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071850.01222.gheskett@shentel.net> Message-ID: <201801071925.41568.gheskett@shentel.net> On Sunday 07 January 2018 19:04:12 Chris Angelico wrote: > On Mon, Jan 8, 2018 at 10:50 AM, Gene Heskett wrote: > > On Sunday 07 January 2018 17:37:14 Random832 wrote: > >> On Sun, Jan 7, 2018, at 17:27, Gene Heskett wrote: > >> > > ? ? > >> > > >> > But here its broken and I am looking at two pairs of vertical > >> > boxes because it is not properly mime'd. If you use chars or > >> > gliphs from a non-default charset, it needs to demarcated with a > >> > mime-boundary marker followed by the new type definition. Your > >> > email/news agent did not do that. > >> > >> UTF-8 is the default character set, and anyway his message does > >> have a content-type of 'text/plain; charset="utf-8"; > >> Format="flowed"'. Your environment not having font support and/or > >> support for non-BMP characters is not a deficiency in the message. > > > > That, now that you mention it, could also effect this as I see it, > > my default kmail message body font is hack 14 in deference to the > > age of my eyes. > > > > My system default font is I believe utf-8. That is not a kmail > > settable option. But if I uncheck the "use custom fonts", it is > > still two pair of character outlines. So to what family of fonts do > > these characters belong? > > You're conflating a few different things here. The character set is > the Universal Character Set, basically synonymous with "Unicode". The > encoding is UTF-8 and is a way to represent Unicode characters as > bytes. The transfer encoding is base 64 (sometimes called "MIME > encoding"), at least in the email version of it - I don't know what > the original NG post used, and it may have been different. The font > used is a mapping from character codes to displayable glyphs. > That was looking at the original message. And its body was indeed a blob of base64. > Everything except the font is under the sender's control, and (at > least in the mailing list version) was all fine. If your font can't > handle those characters, the font renderer should still recognize that > they are characters, and put boxes (maybe with the codepoints written > in them). > > On my Debian Linux systems, I can "sudo apt install unifont" to grab a > font that's used as a fallback for any characters not found in other > fonts. Quoting from the package's description: > > "The philosophy behind this font, though, is that anything meaningful > is better than an empty box for an unknown glyph." > > It's not perfect, but it's way better than nothing. I don't know how > many of the emoji are included, but it's worth a try. > > ChrisA Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From Richard at Damon-Family.org Sun Jan 7 19:27:04 2018 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 7 Jan 2018 19:27:04 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071907.48721.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <1515367552.3042049.1227312192.461EC928@webmail.messagingengine.com> <201801071907.48721.gheskett@shentel.net> Message-ID: <60783167-ea36-223d-191d-a793c8d8f4e4@Damon-Family.org> On 1/7/18 7:07 PM, Gene Heskett wrote: > On Sunday 07 January 2018 18:25:52 Random832 wrote: > >> On Sun, Jan 7, 2018, at 17:47, Richard Damon wrote: >>> But it also says: >>> >>> Content-Transfer-Encoding: 7bit >>> >>> Which is incorrect, as the message is actually 8bit encoded (since >>> the Emoji aren't in the first 127 characters, so their UTF-8 >>> encoding isn't 7-bit. Some software might have messed up the message >>> in transit due to that error. >> Well, the fact that the emoji survived the round-trip and showed up >> properly in his reply (and yours) led me to rule out the possibility >> that anything like that had happened. Plus, if that had happened, the >> result wouldn't be boxes, but a series of ASCII characters (some of >> which are control characters, and some of which are printable). > Its just boxes here, and I just spent the better part of half an hour > trying all the fonts available to kmail, without see anything but > variable sizes of twin boxes. Looking at the raw message its also marked > transfer content encoding = base64, which I'd assume destroys any > semblance of an 8 bit encoding. > > Cheers, Gene Heskett If you see base64, then something converted it. The 7bit was what I got from the source comp.language.python, and it looks like the gateway saw that it wasn't 7 bit, and fixed the encoding to base64, which maintains the full 8 bit content of the message (with an encoding overhead of using 4 characters to encode 3 bytes of data, plus line overhead). If you got the message from the list, it was intact, and the lack of characters says you machine just doesn't have a full unicode font. (One of the issues with Unicode, a FULL font is huge) -- Richard Damon From storchaka at gmail.com Sun Jan 7 19:28:41 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 8 Jan 2018 02:28:41 +0200 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> Message-ID: 07.01.18 22:33, Chris Angelico ????: > On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: >> On 07/01/18 20:55, Chris Angelico wrote: >>> Under what circumstances would you want "x != y" to be different from >>> "not (x == y)" ? >> >> In numpy, __eq__ and __ne__ do not, in general, return bools. >> >>>>> a = np.array([1,2,3,4]) >>>>> b = np.array([0,2,0,4]) >>>>> a == b >> array([False, True, False, True], dtype=bool) >>>>> a != b >> array([ True, False, True, False], dtype=bool) > > Thanks, that's the kind of example I was looking for. Though numpy > doesn't drive the core language development much, so the obvious next > question is: was this why __ne__ was implemented, or was there some > other reason? This example shows how it can be useful, but not why it > exists. AFAIK this was the main reason. This can be also used for creating queries. NumPy inspired 4 or 5 core features which are rarely used outside of NumPy. They include the possibility of comparison operators to return non-booleans. From breamoreboy at gmail.com Sun Jan 7 19:31:40 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 7 Jan 2018 16:31:40 -0800 (PST) Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> Message-ID: On Monday, January 8, 2018 at 12:02:09 AM UTC, Ethan Furman wrote: > On 01/07/2018 12:33 PM, Chris Angelico wrote: > > On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: > >> On 07/01/18 20:55, Chris Angelico wrote: > >>> Under what circumstances would you want "x != y" to be different from > >>> "not (x == y)" ? > >> > >> In numpy, __eq__ and __ne__ do not, in general, return bools. > >> > >>>>> a = np.array([1,2,3,4]) > >>>>> b = np.array([0,2,0,4]) > >>>>> a == b > >> array([False, True, False, True], dtype=bool) > >>>>> a != b > >> array([ True, False, True, False], dtype=bool) > > > > Thanks, that's the kind of example I was looking for. Though numpy > > doesn't drive the core language development much, so the obvious next > > question is: was this why __ne__ was implemented, or was there some > > other reason? This example shows how it can be useful, but not why it > > exists. > > Actually, I think it is why it exists. If I recall correctly, the addition of the six comparative operators* was added > at the behest of the scientific/numerical community. > > -- > ~Ethan~ > > * Yeah, I can't remember the cool name for those six operators at the moment. :( The six rich comparison methods were added to 2.1 as a result of PEP 207, which confirms that you're correct, they were added at the request of the numpyites. -- Kindest regards. Mark Lawrence. From gheskett at shentel.net Sun Jan 7 19:33:32 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 19:33:32 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071850.01222.gheskett@shentel.net> Message-ID: <201801071933.32111.gheskett@shentel.net> On Sunday 07 January 2018 19:04:12 Chris Angelico wrote: > On Mon, Jan 8, 2018 at 10:50 AM, Gene Heskett wrote: > > On Sunday 07 January 2018 17:37:14 Random832 wrote: > >> On Sun, Jan 7, 2018, at 17:27, Gene Heskett wrote: > >> > > ? ? > >> > > >> > But here its broken and I am looking at two pairs of vertical > >> > boxes because it is not properly mime'd. If you use chars or > >> > gliphs from a non-default charset, it needs to demarcated with a > >> > mime-boundary marker followed by the new type definition. Your > >> > email/news agent did not do that. > >> > >> UTF-8 is the default character set, and anyway his message does > >> have a content-type of 'text/plain; charset="utf-8"; > >> Format="flowed"'. Your environment not having font support and/or > >> support for non-BMP characters is not a deficiency in the message. > > > > That, now that you mention it, could also effect this as I see it, > > my default kmail message body font is hack 14 in deference to the > > age of my eyes. > > > > My system default font is I believe utf-8. That is not a kmail > > settable option. But if I uncheck the "use custom fonts", it is > > still two pair of character outlines. So to what family of fonts do > > these characters belong? > > You're conflating a few different things here. The character set is > the Universal Character Set, basically synonymous with "Unicode". The > encoding is UTF-8 and is a way to represent Unicode characters as > bytes. The transfer encoding is base 64 (sometimes called "MIME > encoding"), at least in the email version of it - I don't know what > the original NG post used, and it may have been different. The font > used is a mapping from character codes to displayable glyphs. > > Everything except the font is under the sender's control, and (at > least in the mailing list version) was all fine. If your font can't > handle those characters, the font renderer should still recognize that > they are characters, and put boxes (maybe with the codepoints written > in them). > > On my Debian Linux systems, I can "sudo apt install unifont" to grab a > font that's used as a fallback for any characters not found in other > fonts. Quoting from the package's description: > > "The philosophy behind this font, though, is that anything meaningful > is better than an empty box for an unknown glyph." And here, unifont showed them as empty boxes. So does that point the finger of guilt to kmail? This is the TDE, R14.0.5 version. Hundreds of bugs fixed since the fork at KDE-3.5. > It's not perfect, but it's way better than nothing. I don't know how > many of the emoji are included, but it's worth a try. > > ChrisA I'll leave it set to use this unifont & see how it displays the smileys ;-) Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From ben+python at benfinney.id.au Sun Jan 7 19:35:54 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 08 Jan 2018 11:35:54 +1100 Subject: Why does __ne__ exist? References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> Message-ID: <854lnxkx05.fsf@benfinney.id.au> Chris Angelico writes: > On Mon, Jan 8, 2018 at 10:55 AM, Ben Finney wrote: > > We've established that it is useful to allow data types to define > > their own meaning of ?equal? and ?not equal?, like many other > > operations. Is that not good enough reason to allow it still? > > The fact that container types can define "contains" but can't define > "doesn't contain", and that (as of Py3) there's proper default > handling, suggests that it's not as big a priority now. That is an inconsistency, I agree. > Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, > just like with __contains__. Go ahead: sell the notion of __ne__. > Pitch it, show why we absolutely need to allow this. I think ?reject unless absolutely needed? is an unreasonably high bar, which would disqualify most Python language features. So I don't know why you expect this to be so especially strongly argued. > Make sure you mention the potential confusion when subclassing. For example, that would alsop be a problem for multiple inheritance. Not ?absolutely needed?, and high risk of confusion when subclassing. Do you think that multiple inheritance would thereby also not be allowed today? If you consider that a different case, why? -- \ ?First they came for the verbs, and I said nothing, for verbing | `\ weirds language. Then, they arrival for the nouns and I speech | _o__) nothing, for I no verbs.? ?Peter Ellis | Ben Finney From rosuav at gmail.com Sun Jan 7 19:38:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 11:38:37 +1100 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071933.32111.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071850.01222.gheskett@shentel.net> <201801071933.32111.gheskett@shentel.net> Message-ID: On Mon, Jan 8, 2018 at 11:33 AM, Gene Heskett wrote: > And here, unifont showed them as empty boxes. So does that point the > finger of guilt to kmail? This is the TDE, R14.0.5 version. Hundreds of > bugs fixed since the fork at KDE-3.5. > Huh. I've no idea, then, but it's entirely possible that (a) the font actually doesn't have those characters, or (b) kmail uses a 16-bit renderer. The latter is why Idle can't render astral characters - Tcl/Tk is limited to the BMP. ChrisA From rosuav at gmail.com Sun Jan 7 19:57:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 11:57:36 +1100 Subject: Why does __ne__ exist? In-Reply-To: <854lnxkx05.fsf@benfinney.id.au> References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> <854lnxkx05.fsf@benfinney.id.au> Message-ID: On Mon, Jan 8, 2018 at 11:35 AM, Ben Finney via Python-list wrote: > Chris Angelico writes: > >> On Mon, Jan 8, 2018 at 10:55 AM, Ben Finney wrote: >> > We've established that it is useful to allow data types to define >> > their own meaning of ?equal? and ?not equal?, like many other >> > operations. Is that not good enough reason to allow it still? >> >> The fact that container types can define "contains" but can't define >> "doesn't contain", and that (as of Py3) there's proper default >> handling, suggests that it's not as big a priority now. > > That is an inconsistency, I agree. > >> Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, >> just like with __contains__. Go ahead: sell the notion of __ne__. >> Pitch it, show why we absolutely need to allow this. > > I think ?reject unless absolutely needed? is an unreasonably high bar, > which would disqualify most Python language features. So I don't know > why you expect this to be so especially strongly argued. True, I exaggerated a bit. But do you think that, had __ne__ not existed for years, its addition could be justified? >> Make sure you mention the potential confusion when subclassing. > > For example, that would alsop be a problem for multiple inheritance. Not > ?absolutely needed?, and high risk of confusion when subclassing. Do you > think that multiple inheritance would thereby also not be allowed today? > > If you consider that a different case, why? There's a LOT that you can do usefully with MI that you can't do without it. Having spent a few years (many years ago) working with Java, I appreciate the ability to inherit from more than one class. Does it have to be done the way Python currently does it? No. But one way or another, it's a massively useful feature. (You're right that "absolutely needed" is too high a bar, but hyperbole aside, I do think that MI hits a higher mark than __ne__ does.) ChrisA From ethan at stoneleaf.us Sun Jan 7 20:24:08 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 07 Jan 2018 17:24:08 -0800 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> Message-ID: <5A52C838.2040903@stoneleaf.us> On 01/07/2018 04:31 PM, breamoreboy at gmail.com wrote: > On Monday, January 8, 2018 at 12:02:09 AM UTC, Ethan Furman wrote: >> On 01/07/2018 12:33 PM, Chris Angelico wrote: >>> On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: >>>> On 07/01/18 20:55, Chris Angelico wrote: >>>>> Under what circumstances would you want "x != y" to be different from >>>>> "not (x == y)" ? >>>> >>>> In numpy, __eq__ and __ne__ do not, in general, return bools. >>>> >>>>>>> a = np.array([1,2,3,4]) >>>>>>> b = np.array([0,2,0,4]) >>>>>>> a == b >>>> array([False, True, False, True], dtype=bool) >>>>>>> a != b >>>> array([ True, False, True, False], dtype=bool) >>> >>> Thanks, that's the kind of example I was looking for. Though numpy >>> doesn't drive the core language development much, so the obvious next >>> question is: was this why __ne__ was implemented, or was there some >>> other reason? This example shows how it can be useful, but not why it >>> exists. >> >> Actually, I think it is why it exists. If I recall correctly, the addition of the six comparative operators* was added >> at the behest of the scientific/numerical community. >> >> -- >> ~Ethan~ >> >> * Yeah, I can't remember the cool name for those six operators at the moment. :( > > The six rich comparison methods were added to 2.1 as a result of PEP 207, which confirms that you're correct, they were added at the request of the numpyites. Cool, thanks for tracking that down! -- ~Ethan~ From gheskett at shentel.net Sun Jan 7 21:10:37 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 7 Jan 2018 21:10:37 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071933.32111.gheskett@shentel.net> Message-ID: <201801072110.37346.gheskett@shentel.net> On Sunday 07 January 2018 19:38:37 Chris Angelico wrote: > On Mon, Jan 8, 2018 at 11:33 AM, Gene Heskett wrote: > > And here, unifont showed them as empty boxes. So does that point the > > finger of guilt to kmail? This is the TDE, R14.0.5 version. Hundreds > > of bugs fixed since the fork at KDE-3.5. > > Huh. I've no idea, We are in the same boat. :) > then, but it's entirely possible that (a) the font > actually doesn't have those characters, or (b) kmail uses a 16-bit > renderer. Both of which are unk to me. I like the rest of how kmail works, and have written automatic scripts to reduce the actual work involved, so I am not likely to change agents without a better reason than a couple boxes where someone, think its cute, decides to test the rest of the systems. I believe there is a french saying about such that boils down to a shrug. ;-) But its also something I've not seen in something approaching 70 years, last in some required school reading in the middle 40's IIRC, so its still a shrug. I thought maybe I'd gain a clue by starting a discussion, but either it didn't or it flew by without even the loss of a feather. Solution buried deeper in the system than my bash speaking shovel can reach. ;) Take care all, and have a better 2018. Orders from Grandpa Gene. > The latter is why Idle can't render astral characters - > Tcl/Tk is limited to the BMP. > > ChrisA Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From ben+python at benfinney.id.au Sun Jan 7 22:08:30 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 08 Jan 2018 14:08:30 +1100 Subject: Why does __ne__ exist? References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> <854lnxkx05.fsf@benfinney.id.au> Message-ID: <85zi5pjbdd.fsf@benfinney.id.au> Chris Angelico writes: > On Mon, Jan 8, 2018 at 11:35 AM, Ben Finney via Python-list > wrote: > > I think ?reject unless absolutely needed? is an unreasonably high > > bar, which would disqualify most Python language features. So I > > don't know why you expect this to be so especially strongly argued. > > True, I exaggerated a bit. But do you think that, had __ne__ not > existed for years, its addition could be justified? I'm not the one making pronouncements on what would or would not be allowed, in a counterfactual universe where things had been different for so many years. So, because I don't need to speculate about that, I won't :-) -- \ ?Corporation, n. An ingenious device for obtaining individual | `\ profit without individual responsibility.? ?Ambrose Bierce, | _o__) _The Devil's Dictionary_, 1906 | Ben Finney From random832 at fastmail.com Sun Jan 7 23:43:32 2018 From: random832 at fastmail.com (Random832) Date: Sun, 07 Jan 2018 23:43:32 -0500 Subject: [OT] Re: has sourceforge exposed the dirty little secret ? In-Reply-To: <201801071850.01222.gheskett@shentel.net> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> <201801071727.02798.gheskett@shentel.net> <1515364634.3033237.1227282920.74621B94@webmail.messagingengine.com> <201801071850.01222.gheskett@shentel.net> Message-ID: <1515386612.3106142.1227506328.2FEA3CD8@webmail.messagingengine.com> On Sun, Jan 7, 2018, at 18:50, Gene Heskett wrote: > That, now that you mention it, could also effect this as I see it, my > default kmail message body font is hack 14 in deference to the age of my > eyes. > > My system default font is I believe utf-8. That is not a kmail settable > option. But if I uncheck the "use custom fonts", it is still two pair of > character outlines. So to what family of fonts do these characters > belong? If it supports truetype fonts (and picking fonts based on what fonts have a character available, rather than one font for the whole message), you might try the https://www.google.com/get/noto/ family - it has a black-and-white Emoji font available (color fonts require special application support, but the black and white one is just plain truetype) - of course, that won't help if it's limited to 16 bits (the fact that they are *pairs* of boxes unfortunately suggests this, but maybe it'll work properly if a font is available). From ethan at stoneleaf.us Mon Jan 8 00:21:37 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 07 Jan 2018 21:21:37 -0800 Subject: Why does __ne__ exist? In-Reply-To: References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> <854lnxkx05.fsf@benfinney.id.au> Message-ID: <5A52FFE1.6000401@stoneleaf.us> On 01/07/2018 04:57 PM, Chris Angelico wrote: > On Mon, Jan 8, 2018 at 11:35 AM, Ben Finney wrote: >> Chris Angelico writes: >>> Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, >>> just like with __contains__. Go ahead: sell the notion of __ne__. >>> Pitch it, show why we absolutely need to allow this. >> >> I think ?reject unless absolutely needed? is an unreasonably high bar, >> which would disqualify most Python language features. So I don't know >> why you expect this to be so especially strongly argued. > > True, I exaggerated a bit. But do you think that, had __ne__ not > existed for years, its addition could be justified? Considering we just recently added a matrix-multiplication operator, yes. -- ~Ethan~ From rosuav at gmail.com Mon Jan 8 00:26:51 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 8 Jan 2018 16:26:51 +1100 Subject: Why does __ne__ exist? In-Reply-To: <5A52FFE1.6000401@stoneleaf.us> References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> <854lnxkx05.fsf@benfinney.id.au> <5A52FFE1.6000401@stoneleaf.us> Message-ID: On Mon, Jan 8, 2018 at 4:21 PM, Ethan Furman wrote: > On 01/07/2018 04:57 PM, Chris Angelico wrote: >> >> On Mon, Jan 8, 2018 at 11:35 AM, Ben Finney wrote: >>> >>> Chris Angelico writes: > > >>>> Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, >>>> just like with __contains__. Go ahead: sell the notion of __ne__. >>>> Pitch it, show why we absolutely need to allow this. >>> >>> >>> I think ?reject unless absolutely needed? is an unreasonably high bar, >>> which would disqualify most Python language features. So I don't know >>> why you expect this to be so especially strongly argued. >> >> >> True, I exaggerated a bit. But do you think that, had __ne__ not >> existed for years, its addition could be justified? > > > Considering we just recently added a matrix-multiplication operator, yes. > That has approximately zero consequences on class authors. If you were unaware of __matmul__, it wouldn't have the chance to randomly break your __mul__ semantics. And even with that excellent backward compatibility, it STILL took many years to get accepted. ChrisA From tjol at tjol.eu Mon Jan 8 06:36:45 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 8 Jan 2018 12:36:45 +0100 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> Message-ID: <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> On 2018-01-08 01:31, breamoreboy at gmail.com wrote: > On Monday, January 8, 2018 at 12:02:09 AM UTC, Ethan Furman wrote: >> On 01/07/2018 12:33 PM, Chris Angelico wrote: >>> On Mon, Jan 8, 2018 at 7:13 AM, Thomas Jollans wrote: >>>> On 07/01/18 20:55, Chris Angelico wrote: >>>>> Under what circumstances would you want "x != y" to be different from >>>>> "not (x == y)" ? >>>> >>>> In numpy, __eq__ and __ne__ do not, in general, return bools. >>>> >>>>>>> a = np.array([1,2,3,4]) >>>>>>> b = np.array([0,2,0,4]) >>>>>>> a == b >>>> array([False, True, False, True], dtype=bool) >>>>>>> a != b >>>> array([ True, False, True, False], dtype=bool) >>> >>> Thanks, that's the kind of example I was looking for. Though numpy >>> doesn't drive the core language development much, so the obvious next >>> question is: was this why __ne__ was implemented, or was there some >>> other reason? This example shows how it can be useful, but not why it >>> exists. >> >> Actually, I think it is why it exists. If I recall correctly, the addition of the six comparative operators* was added >> at the behest of the scientific/numerical community. >> >> -- >> ~Ethan~ >> >> * Yeah, I can't remember the cool name for those six operators at the moment. :( > > The six rich comparison methods were added to 2.1 as a result of PEP 207, which confirms that you're correct, they were added at the request of the numpyites. Interesting sentence from that PEP: "3. The == and != operators are not assumed to be each other's complement (e.g. IEEE 754 floating point numbers do not satisfy this)." Does anybody here know how IEE 754 floating point numbers need __ne__? > > -- > Kindest regards. > > Mark Lawrence. > From jorge.conrado at cptec.inpe.br Mon Jan 8 07:33:07 2018 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Mon, 08 Jan 2018 10:33:07 -0200 Subject: Plot map wit a white and black box Message-ID: <74722196045e25deddc67864d6869fe6@cptec.inpe.br> Hi, Please, I woudl like to plot a map like this figure. How can I do this using Python2.7 Thanks, Conrado From arj.python at gmail.com Mon Jan 8 08:49:55 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jan 2018 17:49:55 +0400 Subject: has sourceforge exposed the dirty little secret ? In-Reply-To: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> References: <151516966665.348096.18338899180412170014.XPN@Welt.Netz> Message-ID: fuel the troll ** poor py ** On 5 Jan 2018 20:30, "Kim of K." wrote: > > "Background > > We feel that the world still produces way too much software that is > frankly substandard. The reasons for this are pretty simple: software > producers do not pay enough attention [...]" > > > quote from http://texttest.sourceforge.net/index.php?page=about > > > In other words: most sites like SF and github offer tons of crap. > download and break is the overwhelming theme here. > > why is no one complaining ? > -- > https://mail.python.org/mailman/listinfo/python-list > From tomuxiong at gmx.com Mon Jan 8 09:01:33 2018 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Mon, 8 Jan 2018 15:01:33 +0100 Subject: Why does __ne__ exist? In-Reply-To: <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> Message-ID: On 01/08/2018 12:36 PM, Thomas Jollans wrote: > > Interesting sentence from that PEP: > > "3. The == and != operators are not assumed to be each other's > complement (e.g. IEEE 754 floating point numbers do not satisfy this)." > > Does anybody here know how IEE 754 floating point numbers need __ne__? That's very interesting. I'd also like an answer to this. I can't wrap my head around why it would be true. I've just spent 15 minutes playing with the interpreter (i.e. checking operations on 0, -0, 7, float('nan'), float('inf'), etc.) and then also reading a bit about IEEE 754 online and I can't find any combination of examples where == and != are not each others' complement. Cheers, Thomas From python-oren at ben-kiki.org Mon Jan 8 09:25:58 2018 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 8 Jan 2018 16:25:58 +0200 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> Message-ID: I don't see a case in IEEE where (x == y) != !(x != y). There _is_ a case where (x != x) is true (when x is NaN), but for such an x, (x == x) will be false. I am hard pressed to think of a case where __ne__ is actually useful. That said, while it is true you only need one of (__eq__, __ne__), you could make the same claim about (__lt__, __ge__) and (__le__, __gt__). That is, in principle you could get by with only (__eq__, __le__, and __ge__) or, if you prefer, (__ne__, __lt__, __gt__), or any other combination you prefer. Or you could go where C++ is doing and say that _if_ one specifies a single __cmp__ method, it should return one of LT, EQ, GT, and this will automatically give rise to all the comparison operators. "Trade-offs... trafe-offs as far as the eye can see" ;-) On Mon, Jan 8, 2018 at 4:01 PM, Thomas Nyberg wrote: > On 01/08/2018 12:36 PM, Thomas Jollans wrote: > > > > Interesting sentence from that PEP: > > > > "3. The == and != operators are not assumed to be each other's > > complement (e.g. IEEE 754 floating point numbers do not satisfy this)." > > > > Does anybody here know how IEE 754 floating point numbers need __ne__? > > That's very interesting. I'd also like an answer to this. I can't wrap > my head around why it would be true. I've just spent 15 minutes playing > with the interpreter (i.e. checking operations on 0, -0, 7, > float('nan'), float('inf'), etc.) and then also reading a bit about IEEE > 754 online and I can't find any combination of examples where == and != > are not each others' complement. > > Cheers, > Thomas > -- > https://mail.python.org/mailman/listinfo/python-list > From breamoreboy at gmail.com Mon Jan 8 09:36:20 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 8 Jan 2018 06:36:20 -0800 (PST) Subject: Plot map wit a white and black box In-Reply-To: References: <74722196045e25deddc67864d6869fe6@cptec.inpe.br> Message-ID: On Monday, January 8, 2018 at 1:16:08 PM UTC, jorge.... at cptec.inpe.br wrote: > Hi, > > Please, I woudl like to plot a map like this figure. How can I do this > using Python2.7 > > Thanks, > > Conrado Figures don't get through and you've all ready asked this question, possibly on another forum. What was wrong with the replies that you got then? -- Kindest regards. Mark Lawrence. From tomuxiong at gmx.com Mon Jan 8 09:36:47 2018 From: tomuxiong at gmx.com (Thomas Nyberg) Date: Mon, 8 Jan 2018 15:36:47 +0100 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> Message-ID: <9130099d-f505-d753-1522-38d4c70e8d6e@gmx.com> On 01/08/2018 03:25 PM, Oren Ben-Kiki wrote: > I am hard pressed to think of a case where __ne__ is actually useful. Assuming you're talking about a case specifically for IEEE 754, I'm starting to agree. In general, however, it certainly is useful for some numpy objects (as mentioned elsewhere in this thread). > That said, while it is true you only need one of (__eq__, __ne__), you > could make the same claim about (__lt__, __ge__) and (__le__, __gt__). > That is, in principle you could get by with only (__eq__, __le__, and > __ge__) or, if you prefer, (__ne__, __lt__, __gt__), or any other > combination you prefer. This isn't true for IEEE 754. For example: >>> float('nan') < 0 False >>> float('nan') > 0 False >>> float('nan') == 0 False Also there are many cases where you don't have a < b OR a >= b. For example, subsets don't follow this. > "Trade-offs... trafe-offs as far as the eye can see" ;-) Yes few things in life are free. :) From python-oren at ben-kiki.org Mon Jan 8 09:48:27 2018 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 8 Jan 2018 16:48:27 +0200 Subject: Why does __ne__ exist? In-Reply-To: <9130099d-f505-d753-1522-38d4c70e8d6e@gmx.com> References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> <9130099d-f505-d753-1522-38d4c70e8d6e@gmx.com> Message-ID: Ugh, right, for NaN you can have (x < y) != (x >= y) - both would be false if one of x and y is a NaN. But __ne__ is still useless ;-) On Mon, Jan 8, 2018 at 4:36 PM, Thomas Nyberg wrote: > On 01/08/2018 03:25 PM, Oren Ben-Kiki wrote: > > I am hard pressed to think of a case where __ne__ is actually useful. > > Assuming you're talking about a case specifically for IEEE 754, I'm > starting to agree. In general, however, it certainly is useful for some > numpy objects (as mentioned elsewhere in this thread). > > > That said, while it is true you only need one of (__eq__, __ne__), you > > could make the same claim about (__lt__, __ge__) and (__le__, __gt__). > > That is, in principle you could get by with only (__eq__, __le__, and > > __ge__) or, if you prefer, (__ne__, __lt__, __gt__), or any other > > combination you prefer. > > This isn't true for IEEE 754. For example: > > >>> float('nan') < 0 > False > >>> float('nan') > 0 > False > >>> float('nan') == 0 > False > > Also there are many cases where you don't have a < b OR a >= b. For > example, subsets don't follow this. > > > "Trade-offs... trafe-offs as far as the eye can see" ;-) > > Yes few things in life are free. :) > -- > https://mail.python.org/mailman/listinfo/python-list > From tjol at tjol.eu Mon Jan 8 09:51:33 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 8 Jan 2018 15:51:33 +0100 Subject: Why does __ne__ exist? In-Reply-To: References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> Message-ID: <38ee3bd6-7fd5-944b-024c-2edf4e2f1ddf@tjol.eu> On 2018-01-08 15:25, Oren Ben-Kiki wrote: > I don't see a case in IEEE where (x == y) != !(x != y). > There _is_ a case where (x != x) is true (when x is NaN), but for such an > x, (x == x) will be false. > > I am hard pressed to think of a case where __ne__ is actually useful. See my earlier email and/or PEP 207. (tl;dr: non-bool return values) > > That said, while it is true you only need one of (__eq__, __ne__), you > could make the same claim about (__lt__, __ge__) and (__le__, __gt__). > That is, in principle you could get by with only (__eq__, __le__, and > __ge__) or, if you prefer, (__ne__, __lt__, __gt__), or any other > combination you prefer. PEP 207: "The above mechanism is such that classes can get away with not implementing either __lt__ and __le__ or __gt__ and __ge__." > > Or you could go where C++ is doing and say that _if_ one specifies a single > __cmp__ method, it should return one of LT, EQ, GT, and this will > automatically give rise to all the comparison operators. This used to be the case. (from version 2.1 to version 2.7, AFAICT) > > "Trade-offs... trafe-offs as far as the eye can see" ;-) > > > On Mon, Jan 8, 2018 at 4:01 PM, Thomas Nyberg wrote: > >> On 01/08/2018 12:36 PM, Thomas Jollans wrote: >>> >>> Interesting sentence from that PEP: >>> >>> "3. The == and != operators are not assumed to be each other's >>> complement (e.g. IEEE 754 floating point numbers do not satisfy this)." >>> >>> Does anybody here know how IEE 754 floating point numbers need __ne__? >> >> That's very interesting. I'd also like an answer to this. I can't wrap >> my head around why it would be true. I've just spent 15 minutes playing >> with the interpreter (i.e. checking operations on 0, -0, 7, >> float('nan'), float('inf'), etc.) and then also reading a bit about IEEE >> 754 online and I can't find any combination of examples where == and != >> are not each others' complement. >> >> Cheers, >> Thomas From python-oren at ben-kiki.org Mon Jan 8 10:04:30 2018 From: python-oren at ben-kiki.org (Oren Ben-Kiki) Date: Mon, 8 Jan 2018 17:04:30 +0200 Subject: Why does __ne__ exist? In-Reply-To: <38ee3bd6-7fd5-944b-024c-2edf4e2f1ddf@tjol.eu> References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> <07cf7c20-f358-1f19-68cc-4e0776d9506f@tjol.eu> <38ee3bd6-7fd5-944b-024c-2edf4e2f1ddf@tjol.eu> Message-ID: Good points. Well, this is pretty academic at this point - I don't think anyone would seriously choose to obsolete __ne__, regardless of whether it is absolutely necessary or not. On Mon, Jan 8, 2018 at 4:51 PM, Thomas Jollans wrote: > On 2018-01-08 15:25, Oren Ben-Kiki wrote: > > I don't see a case in IEEE where (x == y) != !(x != y). > > There _is_ a case where (x != x) is true (when x is NaN), but for such an > > x, (x == x) will be false. > > > > I am hard pressed to think of a case where __ne__ is actually useful. > > See my earlier email and/or PEP 207. (tl;dr: non-bool return values) > > > > > That said, while it is true you only need one of (__eq__, __ne__), you > > could make the same claim about (__lt__, __ge__) and (__le__, __gt__). > > That is, in principle you could get by with only (__eq__, __le__, and > > __ge__) or, if you prefer, (__ne__, __lt__, __gt__), or any other > > combination you prefer. > > PEP 207: "The above mechanism is such that classes can get away with not > implementing either __lt__ and __le__ or __gt__ and __ge__." > > > > > > Or you could go where C++ is doing and say that _if_ one specifies a > single > > __cmp__ method, it should return one of LT, EQ, GT, and this will > > automatically give rise to all the comparison operators. > > This used to be the case. (from version 2.1 to version 2.7, AFAICT) > > > > > > "Trade-offs... trafe-offs as far as the eye can see" ;-) > > > > > > On Mon, Jan 8, 2018 at 4:01 PM, Thomas Nyberg wrote: > > > >> On 01/08/2018 12:36 PM, Thomas Jollans wrote: > >>> > >>> Interesting sentence from that PEP: > >>> > >>> "3. The == and != operators are not assumed to be each other's > >>> complement (e.g. IEEE 754 floating point numbers do not satisfy this)." > >>> > >>> Does anybody here know how IEE 754 floating point numbers need __ne__? > >> > >> That's very interesting. I'd also like an answer to this. I can't wrap > >> my head around why it would be true. I've just spent 15 minutes playing > >> with the interpreter (i.e. checking operations on 0, -0, 7, > >> float('nan'), float('inf'), etc.) and then also reading a bit about IEEE > >> 754 online and I can't find any combination of examples where == and != > >> are not each others' complement. > >> > >> Cheers, > >> Thomas > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at vub.be Mon Jan 8 10:25:49 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Mon, 8 Jan 2018 16:25:49 +0100 Subject: Why does __ne__ exist? In-Reply-To: <5A52B2F3.60801@stoneleaf.us> References: <252bf50a-bc1e-1d52-9013-2e278eddaae8@tjol.eu> <5A52B2F3.60801@stoneleaf.us> Message-ID: Op 08-01-18 om 00:53 schreef Ethan Furman: > On 01/07/2018 12:33 PM, Chris Angelico wrote: > > Actually, I think it is why it exists.? If I recall correctly, the > addition of the six comparative operators* was added at the behest of > the scientific/numerical community. Which personnaly, I think was a mistake. I can understand it is useful for the scientific/numerical community to compare vectors with each other and get a vector of booleans. However how useful is it doing this with the normal boolean operators, instead of calling a function? And if doing it with an operator was so important, I think it would have been better to introduce boxed operators, like [+], [<] ... where the default behaviour would be an elementary wise application of the non-boxed operator. -- Antoon Pardon From arj.python at gmail.com Mon Jan 8 10:40:28 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jan 2018 19:40:28 +0400 Subject: =?UTF-8?B?UmU6IDog4pyo8J+NsOKcqCBweXRob24gMjAxOCB3aWtpIC0gYSBwaWVjZSBvZiBjYWtlIA==?= =?UTF-8?B?4pyo8J+NsOKcqCAtLS0g8J+ZhPCfmYTwn5mE?= In-Reply-To: <151518046810.369899.10245783977163770426.XPN@Welt.Netz> References: <151518046810.369899.10245783977163770426.XPN@Welt.Netz> Message-ID: there is a language called python by guido you can ask your questions here ! On 5 Jan 2018 23:30, "Kim of K." wrote: > OK now we have emoji in XPN > > > but not in colour like in torBrowser... > > > :-( > > > ????????? > -- > https://mail.python.org/mailman/listinfo/python-list > From cody.piersall at gmail.com Mon Jan 8 11:25:34 2018 From: cody.piersall at gmail.com (Cody Piersall) Date: Mon, 8 Jan 2018 10:25:34 -0600 Subject: Why does __ne__ exist? In-Reply-To: References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> Message-ID: > Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, > just like with __contains__. Go ahead: sell the notion of __ne__. > Pitch it, show why we absolutely need to allow this. Make sure you > mention the potential confusion when subclassing. Be sure to show why > it's okay for "not in" to force to boolean but "==" should allow any > return value. __ne__ and __eq__ are important for building mask arrays in NumPy, which allow complex indexing operations. A lot of NumPy's design was inspired by MATLAB, so being able to index the same way as in MATLAB is a pretty killer feature. Indexing an array using mask arrays like this is idiomatic: some_arr = np.array([-1, 0, 1, 2, 3, 4, 5, 2, -1, 3, -1, 6, 7, 3]) valid = some_arr[some_arr != -1] Anybody with familiarity with NumPy appreciates that this is possible. I imagine that ORMs like Django or SqlAlchemy also override __ne__ to provide nice APIs. Finally (and perhaps least imporant), there is a performance hit if only allowing __eq__ and then taking its inverse. Cody From antoon.pardon at rece.vub.ac.be Mon Jan 8 12:40:04 2018 From: antoon.pardon at rece.vub.ac.be (Antoon Pardon) Date: Mon, 8 Jan 2018 18:40:04 +0100 Subject: Why does __ne__ exist? In-Reply-To: References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> Message-ID: <1fc3bd36-d765-eff1-66ce-b6b3176d47f1@rece.vub.ac.be> Op 08-01-18 om 17:25 schreef Cody Piersall: >> Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, >> just like with __contains__. Go ahead: sell the notion of __ne__. >> Pitch it, show why we absolutely need to allow this. Make sure you >> mention the potential confusion when subclassing. Be sure to show why >> it's okay for "not in" to force to boolean but "==" should allow any >> return value. > > __ne__ and __eq__ are important for building mask arrays in NumPy, > which allow complex indexing operations. A lot of NumPy's design was > inspired by MATLAB, so being able to index the same way as in MATLAB > is a pretty killer feature. They are only important if you find it necessary to build these mask arrays wih an operator. > Indexing an array using mask arrays like this is idiomatic: > > some_arr = np.array([-1, 0, 1, 2, 3, 4, 5, 2, -1, 3, -1, 6, 7, 3]) > valid = some_arr[some_arr != -1] I guess it was rather useful because list comprehension wasn't included in the language at that moment, but I don't find it that much more useful than: valid = somearr[[number != -1 for number in somearr]] -- Antoon Pardon. From alister.ware at ntlworld.com Mon Jan 8 13:00:56 2018 From: alister.ware at ntlworld.com (alister) Date: Mon, 08 Jan 2018 18:00:56 GMT Subject: : =?UTF-8?B?4pyo8J+NsOKcqA==?= python 2018 wiki - a piece of cake =?UTF-8?B?4pyo8J+NsOKcqCAtLS0g8J+ZhPCfmYTwn5mE?= References: <151518046810.369899.10245783977163770426.XPN@Welt.Netz> Message-ID: On Mon, 08 Jan 2018 15:55:00 +0000, user net wrote: > Abdur-Rahmaan Janhangeer: >> there is a language called python by guido >> >> you can ask your questions here ! > > > > ??? python - a piece of cake ??? > > > when u read this post in thunderbird or torBrowser, you see colored > emoji font > > in other newsreaders, like XPN, pan, etc., it is black & white only. > > how to get colored emoji icons - like the above cake - everywhere ? I am using Pan & it is in colour - not that i would loose any sleep if it wasn't ? -- Wrinkles should merely indicate where smiles have been. -- Mark Twain From rosuav at gmail.com Mon Jan 8 13:28:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 9 Jan 2018 05:28:40 +1100 Subject: Why does __ne__ exist? In-Reply-To: References: <62f68d0c-1bc5-4f24-95f0-c813f217b879@googlegroups.com> <858td9kyvw.fsf@benfinney.id.au> Message-ID: On Tue, Jan 9, 2018 at 3:25 AM, Cody Piersall wrote: >> Let's put it this way. Suppose that __eq__ existed and __ne__ didn't, >> just like with __contains__. Go ahead: sell the notion of __ne__. >> Pitch it, show why we absolutely need to allow this. Make sure you >> mention the potential confusion when subclassing. Be sure to show why >> it's okay for "not in" to force to boolean but "==" should allow any >> return value. > > __ne__ and __eq__ are important for building mask arrays in NumPy, > which allow complex indexing operations. A lot of NumPy's design was > inspired by MATLAB, so being able to index the same way as in MATLAB > is a pretty killer feature. > > Indexing an array using mask arrays like this is idiomatic: > > some_arr = np.array([-1, 0, 1, 2, 3, 4, 5, 2, -1, 3, -1, 6, 7, 3]) > valid = some_arr[some_arr != -1] > > Anybody with familiarity with NumPy appreciates that this is possible. I've used it, and I'm familiar with it, and I'm still not sure that I appreciate it. But if it's there because of MATLAB, well, I guess that's what people want? ChrisA From tkadm30 at yandex.com Mon Jan 8 18:25:41 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Mon, 8 Jan 2018 18:25:41 -0500 Subject: fun with cwrap: Unknown type kind 114 Message-ID: <1f654e47-1248-47f5-aa99-02cf624ad606@yandex.com> Dear Eli, Can you please take a look at: https://bitbucket.org/tkadm30/libuwsgictl/raw/048978bf2b51b1185302da98c5063978061293df/tests/cwrap/error I'm playing around with cwrap: https://github.com/geggo/cwrap Looks like this *gem* can generate Cython pxd files from C headers using libclang. :) Traceback (most recent call last): File "/usr/local/bin/cwrap", line 46, in ast_items = clang_parser.parse([('input.h', inputfile)], include_dirs, '') File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 889, in parse parser.parse(cfile[0][0], include_dirs, language, unsaved_files=cfile) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 123, in parse self.parse_element(tu.cursor) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 306, in parse_element child = self.parse_element(c, level+1) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 259, in parse_element result = mth(cursor, level) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 496, in visit_VAR_DECL typ, id_ = self.type_to_c_ast_type(cursor.type, level) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang_parser.py", line 167, in type_to_c_ast_type level.show( 'in type to c_ast:', 'kind:', t.kind, repr(t.get_declaration().spelling)) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang/cindex.py", line 1467, in kind return TypeKind.from_id(self._kind_id) File "/usr/local/lib/python2.7/dist-packages/cwrap/frontends/clang/clang/cindex.py", line 1407, in from_id raise ValueError,'Unknown type kind %d' % id ValueError: Unknown type kind 114 What do you think about this error? I ran: $ cwrap -i . -i /usr/lib/llvm-3.8/lib/clang/3.8.1/include uwsgi.h libuwsgi.pxd Cheers, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rgaddi at highlandtechnology.invalid Mon Jan 8 19:59:32 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 8 Jan 2018 16:59:32 -0800 Subject: Native object exposing buffer protocol In-Reply-To: References: <85lghbltlz.fsf@benfinney.id.au> Message-ID: On 01/05/2018 04:27 PM, Ben Finney wrote: > Rob Gaddi writes: > >> I'd like to create a native Python object that exposes the buffer >> protocol. Basically, something with a ._data member which is a >> bytearray that I can still readinto, make directly into a numpy array, >> etc. > > The ?etc.? seems pretty important, there. You want the behaviour of > ?bytearray? without actually inheriting that behaviour from the > ?bytearray? type. > Well, one specific behavior. Ideally, what I want is, when calls like RawIOBase.readinto and ndarray.frombuffer ask my class "Hey, do you have any raw bytes that I can read and potentially write?" it can answer "Why certainly, here they are." Something like a: class Thingy: def __memoryview__(self): return memoryview(self._data) But it doesn't look as if there's a dunder for that. There's __bytes__, but that specifically requires you give back a bytes() object; even returning a bytearray causes an error. > So, it seems your options are: > > * Enumerate all the things, specifically, that you do want your new type > to do. Don't hide anything in ?etc.?, so that you know exactly what > behaviours need to be implemented. Implement all those behaviours, > without benefit of inheriting from ?bytearray?. > > * Inherit from ?bytearray?, but ameliorate the problems you want to > avoid. This will require enumerating all those problems, so that you > can know whether you have avoided them. Don't hide any of them in an > ?etc.?. That's ultimately the way I'm going about it, but since this is only going to get used in-house, I'm approaching it the Pythonic way. The problem is "Most methods of bytearray make no sense on a data structure representing a fixed length waveform." The solution is "Well, don't use them." > >> Not the end of the world (the file's less than 2KB), but it seems like >> something that should be doable easily without having to throw around >> a lot of extraneous copies. > > I look forward to your report from having tried it :-) > -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From breamoreboy at gmail.com Mon Jan 8 21:00:24 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 8 Jan 2018 18:00:24 -0800 (PST) Subject: Native object exposing buffer protocol In-Reply-To: References: Message-ID: On Saturday, January 6, 2018 at 12:02:18 AM UTC, Rob Gaddi wrote: > I'd like to create a native Python object that exposes the buffer > protocol. Basically, something with a ._data member which is a > bytearray that I can still readinto, make directly into a numpy array, etc. > > I can do it by inheriting the entire thing from bytearray directly, but > that gives me a whole lot of methods that are unsuitable for what I'm > doing with it, which is reading the contents of a binary file, allowing > them to be lightly malleable in memory, and then sending them on to a > device. > > Not the end of the world (the file's less than 2KB), but it seems like > something that should be doable easily without having to throw around a > lot of extraneous copies. > > -- > Rob Gaddi, Highland Technology -- www.highlandtechnology.com > Email address domain is currently out of order. See above to fix. Curiosity having got the better of me I did some searching and found this "Implementing the buffer protocol" http://cython.readthedocs.io/en/latest/src/userguide/buffer.html, any use to you? Note that at the bottom of the link the final section "References" states "The buffer interface used here is set out in PEP 3118, Revising the buffer protocol.", i.e. it is the new protocol and not the old Python 2 one. -- Kindest regards. Mark Lawrence. From zxymike93 at gmail.com Mon Jan 8 21:06:15 2018 From: zxymike93 at gmail.com (zxymike93 at gmail.com) Date: Mon, 8 Jan 2018 18:06:15 -0800 (PST) Subject: How do I send keystrokes to a console window in Windows XP? In-Reply-To: References: <1121361775.974232.302560@g44g2000cwa.googlegroups.com> Message-ID: <9e0a7551-fa48-487f-9e7e-3dda554418b3@googlegroups.com> ? 2005?7?16???? UTC+8??8:46:34?Benji York??? > GoogleGroups at garringer.net wrote: > > How do I use Python to send keystrokes to a console window in Windows > > XP? > > import win32com.client > > shell = win32com.client.Dispatch("WScript.Shell") > shell.AppActivate("Command Prompt") > > shell.SendKeys("cls{ENTER}") > shell.SendKeys("dir{ENTER}") > shell.SendKeys("echo Hi There{ENTER}") > -- > Benji York Recently, I tried `.AppActivate("Command Prompt")` but cannot catch the `cmd` on my Windows 7, the result is `False`. I know it has been some years since your reply, and the name of the window object may be different.(Still I'm pretty thankful to your answer.) Just wondering if there is a similar solution with another object name. From songofacandy at gmail.com Mon Jan 8 23:09:33 2018 From: songofacandy at gmail.com (INADA Naoki) Date: Tue, 9 Jan 2018 13:09:33 +0900 Subject: How to create "transitional" package? Message-ID: Hi, all. Yesterday, I released msgpack-0.5, which was msgpack-python. Both packages provide "msgpack" python package. I used msgpack in early days, but easy_install crawling website and download msgpack-1.0.0.tar.gz, which is msgpack for C instead of Python package I upload to PyPI. So I renamed msgpack-python but I really dislike it. Now pip doesn't such silly crawling so I decided to rename back to msgpack. To ease transition, I follow what uritemplate.py [1] did (which was migrated to uritemplate). I released msgpack-python 0.5. It is empty package. But it's metadata contains `install_requires=["msgpack>=0.5"]`. Sadly, this breaks upgrading from old version via `pip install -U msgpack-python`. It does: * Install msgpack-0.5 (overwrite msgpack-python 0.4.7) * Uninstall msgpack-python 0.4.7 (removes msgapck 0.5!) * Install msgpack-python 0.5 (empty!) I found uritemplate.py has same issue. Maybe pip's behavior was changed after migration of uritemplate.py to uritemplate. Now what can I do for smooth transition? I don't want to back to msgpack-python again. [1] https://pypi.python.org/pypi/uritemplate.py Regards, INADA Naoki From frank at chagford.com Tue Jan 9 02:55:49 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 9 Jan 2018 09:55:49 +0200 Subject: Dunder variables Message-ID: Hi all I have read that one should not call dunder methods in application code. Does the same apply to dunder variables? I am thinking of the instance attribute __dict__, which allows access to the contents of the instance. I only want to read from __dict__, not update it. Is this frowned upon? Thanks Frank Millman From steve+comp.lang.python at pearwood.info Tue Jan 9 03:34:59 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 9 Jan 2018 08:34:59 +0000 (UTC) Subject: Dunder variables References: Message-ID: On Tue, 09 Jan 2018 09:55:49 +0200, Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. "Should not" rather than "must not", but that's correct. In general, calling a dunder method directly is *not* the same as the operation you probably want. For example, `x + y` may not be the same as `x.__add__(y)` or `y.__radd__(x)`. Better to use `operator.add(x, y)` instead. > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? If you are trying to look up obj.attribute where you don't know the name of the attribute until runtime, the right way is to use getattr: name = 'spam' # this is unknown until runtime value = getattr(obj, name) # like obj.spam If you want to avoid inheritance and avoid looking up attributes on the class or any superclasses, but only look it up on the instance, then the nice way is to use vars(obj): vars(obj)['attribute'] But be aware that not all objects even have a __dict__ -- not even all those with data attributes. -- Steve From __peter__ at web.de Tue Jan 9 03:41:19 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 09 Jan 2018 09:41:19 +0100 Subject: Dunder variables References: Message-ID: Frank Millman wrote: > Hi all > > I have read that one should not call dunder methods in application code. > > Does the same apply to dunder variables? I am thinking of the instance > attribute __dict__, which allows access to the contents of the instance. > > I only want to read from __dict__, not update it. Is this frowned upon? Why would you care whether it is "frowned upon" if it's the best way to achieve something useful? So the real question is /what/ you are trying to do and what your options are. PS: vars(a) is the non-dunder equivalent of a.__dict__. Both have the same limitations, like failing with __slots__ and not seeing properties and class attributes. From frank at chagford.com Tue Jan 9 04:28:03 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 9 Jan 2018 11:28:03 +0200 Subject: Dunder variables In-Reply-To: References: Message-ID: "Peter Otten" wrote in message news:p31v3m$pji$1 at blaine.gmane.org... > Frank Millman wrote: > > > Hi all > > > > I have read that one should not call dunder methods in application code. > > > > Does the same apply to dunder variables? I am thinking of the instance > > attribute __dict__, which allows access to the contents of the instance. > > > > I only want to read from __dict__, not update it. Is this frowned upon? > > Why would you care whether it is "frowned upon" if it's the best way to > achieve something useful? > > So the real question is /what/ you are trying to do and what your options > are. > Here is a brief history of how I have got to where I am now. I have a class call Context containing only data, not methods. Instances are passed around a lot in my application, with various methods accessing various attributes. I wanted to allow various parts of my app to 'piggy back' on this by adding their own attributes at certain points, to be read back at various other points. My first attempt was to add a new attribute to Context called 'vars', which was an empty dictionary. Any method could add to it or read from it, but I would basically ignore it. This worked, but it was awkward. My entire app revolves around passing passing units of data around, and my address mechanism is always two-part - container name, attribute name. This introduced a third element - container name, attribute name, key name. To tidy this up, I changed it to allow other parts of the app to store attributes directly into Context. To protect my 'core' attributes, I made them read-only, using @property. This all works quite well. Now I have a situation where various processes are 'long-running', and I need to persist the data to disk so that it can be recovered in the event of a crash. I want to use Json to store the data as a dictionary. However, I have no idea which additional attributes have been added by other parts of the application. My solution is to use __dict__ to get at the data. If there are any other options, I will be interested to hear them. Thanks Frank From frank at chagford.com Tue Jan 9 05:01:16 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 9 Jan 2018 12:01:16 +0200 Subject: Dunder variables In-Reply-To: References: Message-ID: "Frank Millman" wrote in message news:p321rb$9ct$1 at blaine.gmane.org... "Peter Otten" wrote in message news:p31v3m$pji$1 at blaine.gmane.org... > Frank Millman wrote: > > > Hi all > > > > I have read that one should not call dunder methods in application code. > > > > Does the same apply to dunder variables? I am thinking of the instance > > attribute __dict__, which allows access to the contents of the instance. > > > > I only want to read from __dict__, not update it. Is this frowned upon? > > Why would you care whether it is "frowned upon" if it's the best way to > achieve something useful? > > So the real question is /what/ you are trying to do and what your options > are. > > Here is a brief history of how I have got to where I am now. > [...] > > My solution is to use __dict__ to get at the data. If there are any other > options, I will be interested to hear them. Now that I have read yours and Steve's replies properly, I realise that you were telling me the answer all along. I just have to use vars(...) and it gives me the information I am looking for. Thank you Frank From steve+comp.lang.python at pearwood.info Tue Jan 9 08:30:07 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 9 Jan 2018 13:30:07 +0000 (UTC) Subject: Dunder variables References: Message-ID: On Tue, 09 Jan 2018 11:28:03 +0200, Frank Millman wrote: > I have a class call Context containing only data, not methods. Instances > are passed around a lot in my application, with various methods > accessing various attributes. That contradicts itself... your Context class has data, but no methods, and yet it has methods accessing various attributes? If you have a class with only data, and you access the attributes via the instance's __dict__, why not use an ordinary dict? Alternatively, use a SimpleNamespace: py> from types import SimpleNamespace py> bag = SimpleNamespace() py> bag.spam = 1 py> bag.eggs = 2 py> bag namespace(eggs=2, spam=1) > I wanted to allow various parts of my app to 'piggy back' on this by > adding their own attributes at certain points, to be read back at > various other points. [...] > To tidy this up, I changed it to allow other parts of the app to store > attributes directly into Context. To protect my 'core' attributes, I > made them read-only, using @property. This all works quite well. Except it is no longer a class with data and no methods. > Now I have a situation where various processes are 'long-running', and I > need to persist the data to disk so that it can be recovered in the > event of a crash. I want to use Json to store the data as a dictionary. > However, I have no idea which additional attributes have been added by > other parts of the application. Does it have to be JSON? If this is just for your own use, pickle will probably do what you want: py> import pickle py> class Foo: ... pass ... py> x = Foo() py> x.spam = 1 py> s = pickle.dumps(x) py> y = pickle.loads(s) py> y.spam 1 (Warning: pickle is not secure if arbitrary, untrusted users have write- access to the pickle files.) -- Steve From frank at chagford.com Tue Jan 9 09:14:27 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 9 Jan 2018 16:14:27 +0200 Subject: Dunder variables In-Reply-To: References: Message-ID: "Steven D'Aprano" wrote in message news:p32g4v$v88$2 at blaine.gmane.org... > On Tue, 09 Jan 2018 11:28:03 +0200, Frank Millman wrote: > > > I have a class call Context containing only data, not methods. Instances > > are passed around a lot in my application, with various methods > > accessing various attributes. > > That contradicts itself... your Context class has data, but no methods, > and yet it has methods accessing various attributes? > Maybe I was not clear. The Context instance is passed as an argument to many methods. The methods can access the attributes of the instance. The instance has no methods of its own. > If you have a class with only data, and you access the attributes via the > instance's __dict__, why not use an ordinary dict? I don?t access them *via* the __dict__, I access them via normal object [dot] attribute syntax. > Alternatively, use a SimpleNamespace: I did play with that for a while, but AFAIK it does not allow you to define read-only attributes. > > I wanted to allow various parts of my app to 'piggy back' on this by > > adding their own attributes at certain points, to be read back at > > various other points. [...] > > To tidy this up, I changed it to allow other parts of the app to store > > attributes directly into Context. To protect my 'core' attributes, I > > made them read-only, using @property. This all works quite well. > > Except it is no longer a class with data and no methods. > If you are referring to the @property methods, I think this is a bit pedantic. > > Now I have a situation where various processes are 'long-running', and I > > need to persist the data to disk so that it can be recovered in the > > event of a crash. I want to use Json to store the data as a dictionary. > > However, I have no idea which additional attributes have been added by > > other parts of the application. > Does it have to be JSON? If this is just for your own use, pickle will > probably do what you want: I thought of that. But some of the attributes are things like database connections and context managers. Are they pickleable? What I do at the moment is make a copy of the attributes, using vars(...).copy(), delete the ones I do not want, and save the rest. Frank From robertoshea2k11 at gmail.com Tue Jan 9 10:21:41 2018 From: robertoshea2k11 at gmail.com (Robert O'Shea) Date: Tue, 09 Jan 2018 15:21:41 +0000 Subject: Tips or strategies to understanding how CPython works under the hood Message-ID: Hey all, Been subscribed to this thread for a while but haven't contributed much. One of my ultimate goals this year is to get under the hood of CPython and get a decent understanding of mechanics Guido and the rest of you wonderful people have designed and implemented. I've been programming in python for nearly 10 years now and while I can write a mean Python script, I've been becoming more and more interested in low level operations or complex C programs so I thought I could spread my love of both to make a difference for me and others. So besides just grabbing a chunk of CPython source code and digesting it, I was wondering if those of you have read and understood the source code, do you have any tips or good starting points? Robert From darcy at VybeNetworks.com Tue Jan 9 10:36:54 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Tue, 9 Jan 2018 09:36:54 -0600 Subject: Dunder variables In-Reply-To: References: Message-ID: On 01/09/2018 07:30 AM, Steven D'Aprano wrote: > If you have a class with only data, and you access the attributes via the > instance's __dict__, why not use an ordinary dict? Or even subclass dict. class MyClass(dict): VAR = 5 m = MyClass() m['newvar'] = "Something" I do this and wrap things like __getitem__, __call__, __del__, etc. with my own methods. For example... def __getitem__(self, key): # return self.attribute if given "_attribute_" mapping if key[0] == '_' and key[-1] == '_' and key[1] != '_': k = key[1:-1] if hasattr(self, k): return getattr(self, k) return dict.__getitem__(self, key) So, in the above example... >>>print("newvar = '%(newvar)s', VAR = '%(_VAR_)s'" % m newvar = 'Something', VAR = '5' -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From nad at python.org Tue Jan 9 10:48:01 2018 From: nad at python.org (Ned Deily) Date: Tue, 9 Jan 2018 10:48:01 -0500 Subject: [RELEASE] Python 3.7.0a4 is now available for testing Message-ID: Python 3.7.0a4 is the last of four planned alpha releases of Python 3.7, the next feature release of Python. During the alpha phase, Python 3.7 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next preview release, 3.7.0b1, is planned for 2018-01-29. You can find Python 3.7.0a4 and more information here: https://www.python.org/downloads/release/python-370a4/ -- Ned Deily nad at python.org -- [] From rosuav at gmail.com Tue Jan 9 11:18:22 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Jan 2018 03:18:22 +1100 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: On Wed, Jan 10, 2018 at 2:21 AM, Robert O'Shea wrote: > Hey all, > > Been subscribed to this thread for a while but haven't contributed much. > One of my ultimate goals this year is to get under the hood of CPython and > get a decent understanding of mechanics Guido and the rest of you wonderful > people have designed and implemented. > > I've been programming in python for nearly 10 years now and while I can > write a mean Python script, I've been becoming more and more interested in > low level operations or complex C programs so I thought I could spread my > love of both to make a difference for me and others. > > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? Cool! Let's see. The first thing I'd do is to explore the CPython byte code. Use the 'dis' module to examine the compiled version of a function, and then look at the source code for dis.py (and the things it imports, like opcode.py) to get a handle on what's happening in that byte code. CPython is a stack-based interpreter, which means it loads values onto an (invisible) internal stack, processes values at the top of the stack, and removes them when it's done. Once you've gotten a handle on the bytecode, I'd next dive into one particular core data type. Pick one of dict, list, str (note that, for hysterical raisins, it's called "unicode" in the source), int (for similar hysterical raisins, it's called "long"), etc. In the CPython source code, there's an Objects/ directory, Explore the functionality of that one object type, keeping in mind the interpreter's stack. Get an understanding of the different things you can do with it at the low level; some of them will be the same as you're used to from the high level, but some won't (for instance, Python code is never aware of a call to list_resize). Especially, read all the comments; the top few pages of dictobject.c are pretty much entirely English, and give a lot of insight into how Python avoids potential pitfalls in dict behaviour. >From there, it's all up to you! Don't hesitate to ask questions about stuff you see. Tinkering is strongly encouraged! Oh, one thing to keep an eye on is error handling. You might discover something because there's code to raise an exception if something happens... like raising ValueError("generator already executing"), which I found highly amusing. (I cannot imagine ANY sane code that would ever trigger that error!) Have fun with it! ChrisA not a CPython source code expert by any means, but has followed the above steps and greatly enjoyed the process From p.f.moore at gmail.com Tue Jan 9 11:29:06 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 9 Jan 2018 16:29:06 +0000 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: On 9 January 2018 at 16:18, Chris Angelico wrote: > On Wed, Jan 10, 2018 at 2:21 AM, Robert O'Shea > wrote: >> Hey all, >> >> Been subscribed to this thread for a while but haven't contributed much. >> One of my ultimate goals this year is to get under the hood of CPython and >> get a decent understanding of mechanics Guido and the rest of you wonderful >> people have designed and implemented. >> >> I've been programming in python for nearly 10 years now and while I can >> write a mean Python script, I've been becoming more and more interested in >> low level operations or complex C programs so I thought I could spread my >> love of both to make a difference for me and others. >> >> So besides just grabbing a chunk of CPython source code and digesting it, I >> was wondering if those of you have read and understood the source code, do >> you have any tips or good starting points? > > Cool! Let's see. > > The first thing I'd do is to explore the CPython byte code. Use the > 'dis' module to examine the compiled version of a function, and then > look at the source code for dis.py (and the things it imports, like > opcode.py) to get a handle on what's happening in that byte code. > CPython is a stack-based interpreter, which means it loads values onto > an (invisible) internal stack, processes values at the top of the > stack, and removes them when it's done. > > Once you've gotten a handle on the bytecode, I'd next dive into one > particular core data type. Pick one of dict, list, str (note that, for > hysterical raisins, it's called "unicode" in the source), int (for > similar hysterical raisins, it's called "long"), etc. In the CPython > source code, there's an Objects/ directory, Explore the functionality > of that one object type, keeping in mind the interpreter's stack. Get > an understanding of the different things you can do with it at the low > level; some of them will be the same as you're used to from the high > level, but some won't (for instance, Python code is never aware of a > call to list_resize). Especially, read all the comments; the top few > pages of dictobject.c are pretty much entirely English, and give a lot > of insight into how Python avoids potential pitfalls in dict > behaviour. > > From there, it's all up to you! Don't hesitate to ask questions about > stuff you see. Tinkering is strongly encouraged! > > Oh, one thing to keep an eye on is error handling. You might discover > something because there's code to raise an exception if something > happens... like raising ValueError("generator already executing"), > which I found highly amusing. (I cannot imagine ANY sane code that > would ever trigger that error!) > > Have fun with it! In addition to Chris' suggestions, it would probably be good to look at the documentation - the "Extending and Embedding" and "Python/C API" manuals, although focused more on people writing C code to interface with Python, nevertheless include a lot of information about how the C code that implements Python works. And a lot of the core data types (as well as anything in the standard library) are written using the same C API as 3rd party extensions use, so the documentation will give you a lot of help with those sections of the code. Paul From elchino at cnn.cn Tue Jan 9 14:20:03 2018 From: elchino at cnn.cn (ElChino) Date: Tue, 9 Jan 2018 20:20:03 +0100 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: Chris Angelico wrote: > CPython is a stack-based interpreter, which means it loads values onto > an (invisible) internal stack, processes values at the top of the > stack, and removes them when it's done. Is this similar to how Lua operates too? From rosuav at gmail.com Tue Jan 9 14:42:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 10 Jan 2018 06:42:54 +1100 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: On Wed, Jan 10, 2018 at 6:20 AM, ElChino wrote: > Chris Angelico wrote: > >> CPython is a stack-based interpreter, which means it loads values onto >> an (invisible) internal stack, processes values at the top of the >> stack, and removes them when it's done. > > > Is this similar to how Lua operates too? I don't know (haven't looked into Lua's implementation at all), but it would be highly likely. Stack-based interpreters are a pest to actually program against, as it's too easy to mess something up, but they're beautifully easy to implement; so it's very common to compile to a stack-based bytecode. ChrisA From bingbong3334 at gmail.com Tue Jan 9 15:07:29 2018 From: bingbong3334 at gmail.com (bingbong3334 at gmail.com) Date: Tue, 9 Jan 2018 12:07:29 -0800 (PST) Subject: python server socket file transfer Message-ID: <81b959e9-793a-4d24-8ee9-dc989da572bc@googlegroups.com> how much client can i handel whit this code what the amount of client that i can handel the size of the file is 716 kb import socket from threading import Thread from SocketServer import ThreadingMixIn ## TCP_IP = '' TCP_PORT = 3156 BUFFER_SIZE = 1024 class ClientThread(Thread): def __init__(self,ip,port,sock): Thread.__init__(self) self.ip = ip self.port = port self.sock = sock print " New thread started for "+ip+":"+str(port) def run(self): filename='System.exe' f = open(filename,'rb') while True: l = f.read(BUFFER_SIZE) while (l): self.sock.send(l) #print('Sent ',repr(l)) l = f.read(BUFFER_SIZE) if not l: f.close() self.sock.close() break tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) tcpsock.bind((TCP_IP, TCP_PORT)) threads = [] while True: tcpsock.listen(10000) print "Waiting for incoming connections..." (conn, (ip,port)) = tcpsock.accept() print 'Got connection from ', (ip,port) newthread = ClientThread(ip,port,conn) newthread.start() threads.append(newthread) for t in threads: t.join() and the client side is import socket TCP_IP = '' TCP_PORT = 3156 BUFFER_SIZE = 1024 try: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((TCP_IP, TCP_PORT)) with open('Sys.exe', 'wb') as f: print 'file opened' while True: #print('receiving data...') data = s.recv(BUFFER_SIZE) if not data: f.close() print 'file close()' break # write data to a file f.write(data) except: print "problem" print('Successfully get the file') s.close() print('connection closed') From alain at universite-de-strasbourg.fr.invalid Tue Jan 9 15:12:49 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Tue, 09 Jan 2018 21:12:49 +0100 Subject: Tips or strategies to understanding how CPython works under the hood References: Message-ID: <87zi5mbxku.fsf@universite-de-strasbourg.fr.invalid> ElChino writes: > Chris Angelico wrote: > >> CPython is a stack-based interpreter, which means it loads values onto >> an (invisible) internal stack, processes values at the top of the >> stack, and removes them when it's done. > > Is this similar to how Lua operates too? No. Lua uses a register-based (virtual) machine. See https://www.lua.org/doc/jucs05.pdf I think Lua was the first language in widespread use to move to a register-based machine. -- Alain. From tradingroompitalito at gmail.com Tue Jan 9 15:26:17 2018 From: tradingroompitalito at gmail.com (Manuel Rincon) Date: Tue, 9 Jan 2018 12:26:17 -0800 (PST) Subject: CSV file edition Message-ID: <5aa7e4e9-5e0b-409e-9ddf-36f3ba72ef95@googlegroups.com> Dear: With what function could you separate the numerical part of each column? MarketTime Price Type Type=0 MarketTime=11:18:26.549 Price=112.8300 Type=0 MarketTime=11:18:28.792 Price=112.8300 Type=0 MarketTime=11:18:28.792 Price=112.8400 Type=0 MarketTime=11:18:28.792 Price=112.8300 Type=0 MarketTime=11:18:45.798 Price=112.8500 Type=0 MarketTime=11:18:45.799 Price=112.8500 Type=0 MarketTime=11:18:45.880 Price=112.8400 I would need to filter only the numeric part of all the columns. From alain at universite-de-strasbourg.fr.invalid Tue Jan 9 15:41:54 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Tue, 09 Jan 2018 21:41:54 +0100 Subject: CSV file edition References: <5aa7e4e9-5e0b-409e-9ddf-36f3ba72ef95@googlegroups.com> Message-ID: <87vagabw8d.fsf@universite-de-strasbourg.fr.invalid> Manuel Rincon writes: [...] > Type=0 MarketTime=11:18:26.549 Price=112.8300 > Type=0 MarketTime=11:18:28.792 Price=112.8300 [...] > > I would need to filter only the numeric part of all the columns. I assume that by "numeric" you mean the value after Price= line.split()[2].split('=')[1] line.rsplit('=',1)[1] line.rpartition('=')[2] line[line.rfind('=')+1:] I prefer the last one. You can also use regexps, but it would be overkill if your lines are that simple. If your lines are really that simple, use line[39:] (check that 39 is correct). Your data look like fixed-width fields. -- Alain. From dstanek at dstanek.com Tue Jan 9 15:52:35 2018 From: dstanek at dstanek.com (David Stanek) Date: Tue, 9 Jan 2018 15:52:35 -0500 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: <20180109205235.GD28253@x1> On 09-Jan 15:21, Robert O'Shea wrote: > > Been subscribed to this thread for a while but haven't contributed much. +1. I'm a lurker too. > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? > There are a ton of videos on Youtube that talk about Python internals. I liked https://www.youtube.com/playlist?list=PLzV58Zm8FuBL6OAv1Yu6AwXZrnsFbbR0S quite a bit. Even though I knew a good portion of the material, there was still a lot of new stuff. -- david stanek web: https://dstanek.com twitter: https://twitter.com/dstanek From breamoreboy at gmail.com Tue Jan 9 15:52:37 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Tue, 9 Jan 2018 12:52:37 -0800 (PST) Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: On Tuesday, January 9, 2018 at 3:22:30 PM UTC, Robert O'Shea wrote: > Hey all, > > Been subscribed to this thread for a while but haven't contributed much. > One of my ultimate goals this year is to get under the hood of CPython and > get a decent understanding of mechanics Guido and the rest of you wonderful > people have designed and implemented. > > I've been programming in python for nearly 10 years now and while I can > write a mean Python script, I've been becoming more and more interested in > low level operations or complex C programs so I thought I could spread my > love of both to make a difference for me and others. > > So besides just grabbing a chunk of CPython source code and digesting it, I > was wondering if those of you have read and understood the source code, do > you have any tips or good starting points? > > Robert Hopefully you'll find this http://pgbovine.net/cpython-internals.htm of interest. -- Kindest regards. Mark Lawrence. From tjol at tjol.eu Tue Jan 9 18:21:05 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 10 Jan 2018 00:21:05 +0100 Subject: How to create "transitional" package? In-Reply-To: References: Message-ID: On 09/01/18 05:09, INADA Naoki wrote: > Hi, all. > > Yesterday, I released msgpack-0.5, which was msgpack-python. > Both packages provide "msgpack" python package. > > I used msgpack in early days, but easy_install crawling website > and download msgpack-1.0.0.tar.gz, which is msgpack for C instead > of Python package I upload to PyPI. > So I renamed msgpack-python but I really dislike it. > > Now pip doesn't such silly crawling so I decided to rename back > to msgpack. > To ease transition, I follow what uritemplate.py [1] did (which was > migrated to uritemplate). > > I released msgpack-python 0.5. It is empty package. > But it's metadata contains `install_requires=["msgpack>=0.5"]`. > > Sadly, this breaks upgrading from old version via `pip install -U > msgpack-python`. > It does: > > * Install msgpack-0.5 (overwrite msgpack-python 0.4.7) > * Uninstall msgpack-python 0.4.7 (removes msgapck 0.5!) > * Install msgpack-python 0.5 (empty!) Here's a thought: if you distribute a source package for msgpack-python-0.5, your setup.py is run during installation; you can do whatever you want. You could check, at msgpack-python installation time, whether msgpack is still installed correctly - and if it's not, reinstall. Doing that correctly (and without somehow confusing or angering pip) might, of course, be easier said than done. > > I found uritemplate.py has same issue. Maybe pip's behavior was > changed after migration of uritemplate.py to uritemplate. > > Now what can I do for smooth transition? > I don't want to back to msgpack-python again. > > [1] https://pypi.python.org/pypi/uritemplate.py > > Regards, > > INADA Naoki > From steve+comp.lang.python at pearwood.info Tue Jan 9 18:34:08 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 9 Jan 2018 23:34:08 +0000 (UTC) Subject: Dunder variables References: Message-ID: On Tue, 09 Jan 2018 16:14:27 +0200, Frank Millman wrote: > Maybe I was not clear. The Context instance is passed as an argument to > many methods. The methods can access the attributes of the instance. The > instance has no methods of its own. Ah, I see, I misunderstood. [...] >> Alternatively, use a SimpleNamespace: > > I did play with that for a while, but AFAIK it does not allow you to > define read-only attributes. Not directly, no, but you can subclass: py> class Spam(SimpleNamespace): ... @property ... def eggs(self): ... return "foo" ... py> bag = Spam() py> bag.eggs 'foo' py> bag.eggs = 1 Traceback (most recent call last): File "", line 1, in AttributeError: can't set attribute However, at the point you are subclassing and adding multiple properties, I'm not sure there's any advantage over just defining a class from scratch. While it is true that SimpleNamespace gives you a nice repr, the properties don't show up in that repr, and I assume it is those read-only attributes that you most care about. -- Steve From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Tue Jan 9 23:22:47 2018 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Wed, 10 Jan 2018 04:22:47 +0000 Subject: pyplot: change the number of x labels (from 6) Message-ID: Hi all. I want to have dates as major ticks labels of X axis. This fragment of code works fine except that I need more dates to appear instead the 6 I am getting. The number of dates in dtsd is for ex. 262. Thanks for any help. BTW, I got most of this code from some site on the internet. ... fig=plt.figure(figsize=(12,9)) gs=gridspec.GridSpec(2,1,height_ratios=[4,1]) ... ax0=plt.subplot(gs[0]) ... plt.xlim(-0.1,dts[-1]+0.1) dtsd=pd.to_datetime(ds.getIndexes()) def format_date(x,__pos=None): thisind=np.clip(int(x+0.5),0,dtslen-1) return dtsd[thisind].strftime('%Y-%m-%d') ax0.xaxis.set_major_formatter(ticker.FuncFormatter(format_date)) fig.autofmt_xdate() ... From dieter at handshake.de Wed Jan 10 02:06:55 2018 From: dieter at handshake.de (dieter) Date: Wed, 10 Jan 2018 08:06:55 +0100 Subject: python server socket file transfer References: <81b959e9-793a-4d24-8ee9-dc989da572bc@googlegroups.com> Message-ID: <87vaga5h0w.fsf@handshake.de> bingbong3334 at gmail.com writes: > how much client can i handel whit this code what the amount of client that i can handel > the size of the file is 716 kb > ... > self.sock.send(l) Please read the documentation for *send* in the "socket" module: it tells you that "send" (in contrast to "sendall") is *not* guarantied to send the complete *l* (there is no guarantee how much is sent); the return value of "send" tells you how many bytes have been sent. From bingbong3334 at gmail.com Wed Jan 10 03:57:30 2018 From: bingbong3334 at gmail.com (bingbong3334 at gmail.com) Date: Wed, 10 Jan 2018 00:57:30 -0800 (PST) Subject: python server socket file transfer In-Reply-To: References: <81b959e9-793a-4d24-8ee9-dc989da572bc@googlegroups.com> <87vaga5h0w.fsf@handshake.de> Message-ID: On Wednesday, January 10, 2018 at 9:07:33 AM UTC+2, dieter wrote: > bingbong3334 at gmail.com writes: > > how much client can i handel whit this code what the amount of client that i can handel > > the size of the file is 716 kb > > ... > > self.sock.send(l) > > Please read the documentation for *send* in the "socket" module: > it tells you that "send" (in contrast to "sendall") is *not* guarantied > to send the complete *l* (there is no guarantee how much is sent); > the return value of "send" tells you how many bytes have been sent. bro i dont ask about if it works or not or what you say is not right i ask how much he can handel... From tjol at tjol.eu Wed Jan 10 04:17:20 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 10 Jan 2018 10:17:20 +0100 Subject: pyplot: change the number of x labels (from 6) In-Reply-To: References: Message-ID: <8c850c7f-c9d0-63b0-e2b0-146331aa21c5@tjol.eu> On 2018-01-10 05:22, Paulo da Silva wrote: > Hi all. > > I want to have dates as major ticks labels of X axis. > This fragment of code works fine except that I need more dates to appear > instead the 6 I am getting. The number of dates in dtsd is for ex. 262. > > Thanks for any help. > > BTW, I got most of this code from some site on the internet. > > ... > fig=plt.figure(figsize=(12,9)) > gs=gridspec.GridSpec(2,1,height_ratios=[4,1]) > ... > ax0=plt.subplot(gs[0]) > ... > plt.xlim(-0.1,dts[-1]+0.1) > dtsd=pd.to_datetime(ds.getIndexes()) > def format_date(x,__pos=None): > ? thisind=np.clip(int(x+0.5),0,dtslen-1) > ? return dtsd[thisind].strftime('%Y-%m-%d') > ax0.xaxis.set_major_formatter(ticker.FuncFormatter(format_date)) > fig.autofmt_xdate() > ... > It's a bit hard to tell without a working example, but I think you'll want to set a tick locator, e.g. something like ax0.xaxis.set_major_locator(matplotlib.ticker.MultipleLocator(1)) Read this: https://matplotlib.org/api/ticker_api.html The old-fashioned way would be to set to tick locations manually with https://matplotlib.org/api/_as_gen/matplotlib.axes.Axes.set_xticks.html From robin at reportlab.com Wed Jan 10 06:36:52 2018 From: robin at reportlab.com (Robin Becker) Date: Wed, 10 Jan 2018 11:36:52 +0000 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: References: Message-ID: <55948cec-a4aa-39f4-4613-7a36419d1d2d@chamonix.reportlab.co.uk> http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html -- Robin Becker From bc at freeuk.com Wed Jan 10 07:08:30 2018 From: bc at freeuk.com (bartc) Date: Wed, 10 Jan 2018 12:08:30 +0000 Subject: Tips or strategies to understanding how CPython works under the hood In-Reply-To: <87zi5mbxku.fsf@universite-de-strasbourg.fr.invalid> References: <87zi5mbxku.fsf@universite-de-strasbourg.fr.invalid> Message-ID: On 09/01/2018 20:12, Alain Ketterlin wrote: > ElChino writes: > >> Chris Angelico wrote: >> >>> CPython is a stack-based interpreter, which means it loads values onto >>> an (invisible) internal stack, processes values at the top of the >>> stack, and removes them when it's done. >> >> Is this similar to how Lua operates too? > > No. Lua uses a register-based (virtual) machine. See > > https://www.lua.org/doc/jucs05.pdf "Registers are kept in the run-time stack ... an array". So it sounds like a stack is still used, but instructions directly access specific slots on the stack, within a particular register window. It means there need be fewer instructions to implement some code, but each has more operands. Also interesting is that the comparison operators include only EQ, LT and LE. There is no NE, so no issues such as those discussed recently. > > I think Lua was the first language in widespread use to move to a > register-based machine. I believe stack-based byte-code, which is very easy to generate, can be transformed to some 'register'-based version, if performance is the motivation. (But I'm not convinced that register-based is necessarily faster. Lua is quite fast, especially LuaJIT, but the language is also smaller and simpler.) -- bartc From lists at mostrom.pp.se Wed Jan 10 07:40:28 2018 From: lists at mostrom.pp.se (Jan Erik =?utf-8?q?Mostr=C3=B6m?=) Date: Wed, 10 Jan 2018 13:40:28 +0100 Subject: Simple graphic library for beginners Message-ID: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> I'm looking for a really easy to use graphic library. The target users are teachers who have never programmed before and is taking a first (and possible last) programming course. I would like to have the ability to draw lines, circles, etc. Nothing fancy, as little window management as possible (possible buttons), perhaps simple events like button presses or clicks on canvas - think: making simple diagrams or simple figures. Cross-platform Yes, I know about tkinter. Yes, I know about various block languages like Scratch but they are not relevant in this case. Any suggestions? = jem From larry.martell at gmail.com Wed Jan 10 07:51:16 2018 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 10 Jan 2018 12:51:16 +0000 Subject: Simple graphic library for beginners In-Reply-To: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> Message-ID: On Wed, Jan 10, 2018 at 7:42 AM Jan Erik Mostr?m wrote: > I'm looking for a really easy to use graphic library. The target users > are teachers who have never programmed before and is taking a first (and > possible last) programming course. > > I would like to have the ability to draw lines, circles, etc. Nothing > fancy, as little window management as possible (possible buttons), > perhaps simple events like button presses or clicks on canvas - think: > making simple diagrams or simple figures. Cross-platform > > > Yes, I know about tkinter. > > Yes, I know about various block languages like Scratch but they are not > relevant in this case. > > Any suggestions? > I really like plotly. > > From tjol at tjol.eu Wed Jan 10 08:38:28 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 10 Jan 2018 14:38:28 +0100 Subject: Simple graphic library for beginners In-Reply-To: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> Message-ID: On 2018-01-10 13:40, Jan Erik Mostr?m wrote: > I'm looking for a really easy to use graphic library. The target users > are teachers who have never programmed before and is taking a first (and > possible last) programming course. > > I would like to have the ability to draw lines, circles, etc. Nothing > fancy, as little window management as possible (possible buttons), > perhaps simple events like button presses or clicks on canvas - think: > making simple diagrams or simple figures. Cross-platform > > > Yes, I know about tkinter. > > Yes, I know about various block languages like Scratch but they are not > relevant in this case. > > Any suggestions? It depends on what you're planning to do; "simple graphics library" and "making [...] figures" are two requirements that don't necessarily fit together perfectly. Perhaps turtle (https://docs.python.org/3/library/turtle.html) would suit your needs? If that's not what you're looking for, just pick any popular library that does the specific things you want to use in your lesson, and teach only the relevant bits. If you want (scientific) graphs and figures, go matplotlib. It's not the nicest library, but it's powerful and well-represented on stack overflow. If you want buttons and stuff, go for PyQt or Tkinter. Both are messy and complicated, and Tkinter looks terrible by default, but that's just the way it is. -- Thomas From breamoreboy at gmail.com Wed Jan 10 09:08:58 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 10 Jan 2018 06:08:58 -0800 (PST) Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> Message-ID: <4a90d44c-2f2b-4858-9fb0-d7f8b499c6b7@googlegroups.com> On Wednesday, January 10, 2018 at 12:42:07 PM UTC, Jan Erik Mostr?m wrote: > I'm looking for a really easy to use graphic library. The target users > are teachers who have never programmed before and is taking a first (and > possible last) programming course. > > I would like to have the ability to draw lines, circles, etc. Nothing > fancy, as little window management as possible (possible buttons), > perhaps simple events like button presses or clicks on canvas - think: > making simple diagrams or simple figures. Cross-platform > > > Yes, I know about tkinter. > > Yes, I know about various block languages like Scratch but they are not > relevant in this case. > > Any suggestions? > > = jem Take a look at https://python-sfml.org/, https://www.ogre3d.org/about or mcsp.wartburg.edu/zelle/python/graphics.py -- Kindest regards. Mark Lawrence. From endarthur at gmail.com Wed Jan 10 09:49:56 2018 From: endarthur at gmail.com (Arthur Endlein Correia) Date: Wed, 10 Jan 2018 12:49:56 -0200 Subject: Simple graphic library for beginners In-Reply-To: <4a90d44c-2f2b-4858-9fb0-d7f8b499c6b7@googlegroups.com> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <4a90d44c-2f2b-4858-9fb0-d7f8b499c6b7@googlegroups.com> Message-ID: Tkinter can look nice and not terribly hard to use, as I found out in this tutorial: http://www.tkdocs.com/tutorial/onepage.html Having said that, nowadays I pretty much always use PyQt (or Pyside) together with qtdesigner to build GUI applications. ------- Arthur Endlein Correia arthur.correia at usp.br endarthur at gmail.com 55 11 968 606 174 IGc, USP On Wed, Jan 10, 2018 at 12:08 PM, wrote: > On Wednesday, January 10, 2018 at 12:42:07 PM UTC, Jan Erik Mostr?m wrote: > > I'm looking for a really easy to use graphic library. The target users > > are teachers who have never programmed before and is taking a first (and > > possible last) programming course. > > > > I would like to have the ability to draw lines, circles, etc. Nothing > > fancy, as little window management as possible (possible buttons), > > perhaps simple events like button presses or clicks on canvas - think: > > making simple diagrams or simple figures. Cross-platform > > > > > > Yes, I know about tkinter. > > > > Yes, I know about various block languages like Scratch but they are not > > relevant in this case. > > > > Any suggestions? > > > > = jem > > Take a look at https://python-sfml.org/, https://www.ogre3d.org/about or > mcsp.wartburg.edu/zelle/python/graphics.py > > -- > Kindest regards. > > Mark Lawrence. > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at vub.be Wed Jan 10 09:55:28 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Wed, 10 Jan 2018 15:55:28 +0100 Subject: Simple graphic library for beginners In-Reply-To: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> Message-ID: <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> On 10-01-18 13:40, Jan Erik Mostr?m wrote: > I'm looking for a really easy to use graphic library. The target users > are teachers who have never programmed before and is taking a first > (and possible last) programming course. > > I would like to have the ability to draw lines, circles, etc. Nothing > fancy, as little window management as possible (possible buttons), > perhaps simple events like button presses or clicks on canvas - think: > making simple diagrams or simple figures. Cross-platform > > > Yes, I know about tkinter. > > Yes, I know about various block languages like Scratch but they are > not relevant in this case. > > Any suggestions? > > = jem cairo? From bc at freeuk.com Wed Jan 10 10:43:09 2018 From: bc at freeuk.com (bartc) Date: Wed, 10 Jan 2018 15:43:09 +0000 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> Message-ID: On 10/01/2018 14:55, Antoon Pardon wrote: > On 10-01-18 13:40, Jan Erik Mostr?m wrote: >> I'm looking for a really easy to use graphic library. The target users >> are teachers who have never programmed before and is taking a first >> (and possible last) programming course. >> >> I would like to have the ability to draw lines, circles, etc. Nothing >> fancy, as little window management as possible (possible buttons), >> perhaps simple events like button presses or clicks on canvas - think: >> making simple diagrams or simple figures. Cross-platform >> >> >> Yes, I know about tkinter. >> >> Yes, I know about various block languages like Scratch but they are >> not relevant in this case. >> >> Any suggestions? >> >> = jem > > cairo? Isn't that output only? (And not necessarily to a screen, but to an image or PDF for example.) So that no interaction is possible. From oliver.schoenborn at gmail.com Wed Jan 10 11:16:49 2018 From: oliver.schoenborn at gmail.com (oliver) Date: Wed, 10 Jan 2018 16:16:49 +0000 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> Message-ID: Pyqt without hesitation. - I've used it in full-fledged desktop app (120k LOC) and it's just awesome, so well documented, versatile, and you can learn gradually (starting with their simpler tablewidget then later move to their tableview/model for added flexibility). - Qt is moving so fast into iot and automotive and is so popular on desktops, that it's a skill that gets noticed on resumes or in interviews (i'm talking from first hand experience). - It is available in multiple languages like Java, Go and C#, although I have not used those wrappers (the PyQt wrapper is based on sip, whereas these other ones are not, so I have no idea about stability etc). - the community is huge, there are powerful tools like pytest.qt plugin for unit testing GUI, pycharm IDE integrates with it. - there is dual license based on LGPL and commercial, with reasonable costs, perfect for learning/startup then eventually (if that's the goal medium or long term) distribute/sell. Oliver On Wed, 10 Jan 2018 at 10:46 bartc wrote: > On 10/01/2018 14:55, Antoon Pardon wrote: > > On 10-01-18 13:40, Jan Erik Mostr?m wrote: > >> I'm looking for a really easy to use graphic library. The target users > >> are teachers who have never programmed before and is taking a first > >> (and possible last) programming course. > >> > >> I would like to have the ability to draw lines, circles, etc. Nothing > >> fancy, as little window management as possible (possible buttons), > >> perhaps simple events like button presses or clicks on canvas - think: > >> making simple diagrams or simple figures. Cross-platform > >> > >> > >> Yes, I know about tkinter. > >> > >> Yes, I know about various block languages like Scratch but they are > >> not relevant in this case. > >> > >> Any suggestions? > >> > >> = jem > > > > cairo? > > Isn't that output only? (And not necessarily to a screen, but to an > image or PDF for example.) > > So that no interaction is possible. > > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Oliver My StackOverflow contributions My CodeProject articles My Github projects My SourceForget.net projects From torriem at gmail.com Wed Jan 10 12:04:51 2018 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Jan 2018 10:04:51 -0700 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> Message-ID: <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> On 01/10/2018 09:16 AM, oliver wrote: > Pyqt without hesitation. Except that people are forgetting the OP is not asking about a GUI library. The subject line reads "Simple graphic[s] library for beginners." He just wants a simple graphics drawing library for beginners. Create a canvas of a certain size, draw lines, circles, and other primitives to it. PyQt can do all these things, but it's overkill for what the OP requires. Personally I recommend he look at PyGame, or SDL with PySDL2. Both of these libraries are designed to do what the OP is asking. I recently needed to do some very simple line drawing to visualize a GPS path, and I found PyGame worked rather well, and was nearly as simple as the old QuickBasic graphics routines. From ikorot01 at gmail.com Wed Jan 10 12:22:29 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 10 Jan 2018 11:22:29 -0600 Subject: Simple graphic library for beginners In-Reply-To: <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> Message-ID: Hi, wxPython/Phoenix. It can do everything you need and more.... And it will look exactly as any other application on the system... Thank you. On Wed, Jan 10, 2018 at 11:04 AM, Michael Torrie wrote: > On 01/10/2018 09:16 AM, oliver wrote: >> Pyqt without hesitation. > > Except that people are forgetting the OP is not asking about a GUI > library. The subject line reads "Simple graphic[s] library for > beginners." He just wants a simple graphics drawing library for > beginners. Create a canvas of a certain size, draw lines, circles, and > other primitives to it. > > PyQt can do all these things, but it's overkill for what the OP requires. > > Personally I recommend he look at PyGame, or SDL with PySDL2. Both of > these libraries are designed to do what the OP is asking. I recently > needed to do some very simple line drawing to visualize a GPS path, and > I found PyGame worked rather well, and was nearly as simple as the old > QuickBasic graphics routines. > > -- > https://mail.python.org/mailman/listinfo/python-list From torriem at gmail.com Wed Jan 10 12:36:23 2018 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Jan 2018 10:36:23 -0700 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> Message-ID: <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> On 01/10/2018 10:22 AM, Igor Korot wrote: > Hi, > wxPython/Phoenix. > It can do everything you need and more.... But the OP isn't looking for a full-blown GUI toolkit. I went back and re-read his post to be sure I wasn't misunderstanding. Therefore I don't think the suggestion to use wxPython or PyQt is that helpful. Do you have any other suggestions? Even Cairo is pretty complicated (requiring pens and contexts) for what he's asking for. Personally I don't think it gets much easier or simpler than this sort of thing: https://github.com/Mekire/pygame-samples From endarthur at gmail.com Wed Jan 10 12:43:17 2018 From: endarthur at gmail.com (Arthur Endlein Correia) Date: Wed, 10 Jan 2018 15:43:17 -0200 Subject: Simple graphic library for beginners In-Reply-To: <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: Yes, considering the OP request, pygame would be better, or one of Mark's suggestions: mcsp.wartburg.edu/zelle/python/graphics.py Seems really easy to use. ------- Arthur Endlein Correia arthur.correia at usp.br endarthur at gmail.com 55 11 968 606 174 IGc, USP On Wed, Jan 10, 2018 at 3:36 PM, Michael Torrie wrote: > On 01/10/2018 10:22 AM, Igor Korot wrote: > > Hi, > > wxPython/Phoenix. > > It can do everything you need and more.... > > But the OP isn't looking for a full-blown GUI toolkit. I went back and > re-read his post to be sure I wasn't misunderstanding. Therefore I > don't think the suggestion to use wxPython or PyQt is that helpful. > > Do you have any other suggestions? > > Even Cairo is pretty complicated (requiring pens and contexts) for what > he's asking for. > > Personally I don't think it gets much easier or simpler than this sort > of thing: > https://github.com/Mekire/pygame-samples > -- > https://mail.python.org/mailman/listinfo/python-list > From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Wed Jan 10 13:01:43 2018 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Wed, 10 Jan 2018 18:01:43 +0000 Subject: pyplot: change the number of x labels (from 6) References: <8c850c7f-c9d0-63b0-e2b0-146331aa21c5@tjol.eu> Message-ID: ?s 09:17 de 10-01-2018, Thomas Jollans escreveu: > On 2018-01-10 05:22, Paulo da Silva wrote: >> Hi all. >> ... > > It's a bit hard to tell without a working example, but I think you'll > want to set a tick locator, e.g. something like > ax0.xaxis.set_major_locator(matplotlib.ticker.MultipleLocator(1)) Basically I have a list of hundred of dates (one per point) and I want few of them, to fill the X axis. This should be simple! The code I have (too complex for the task, btw), from the internet, does exactly that, including presenting them 45? oriented, but only presents 5 or 6 dates. I want more, perhaps 12. > > Read this: https://matplotlib.org/api/ticker_api.html > > The old-fashioned way would be to set to tick locations manually with > https://matplotlib.org/api/_as_gen/matplotlib.axes.Axes.set_xticks.html > Yes, I need to look at this ... Thank you. From ikorot01 at gmail.com Wed Jan 10 13:49:22 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 10 Jan 2018 12:49:22 -0600 Subject: Simple graphic library for beginners In-Reply-To: <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: Hi, Michael, On Wed, Jan 10, 2018 at 11:36 AM, Michael Torrie wrote: > On 01/10/2018 10:22 AM, Igor Korot wrote: >> Hi, >> wxPython/Phoenix. >> It can do everything you need and more.... > > But the OP isn't looking for a full-blown GUI toolkit. I went back and > re-read his post to be sure I wasn't misunderstanding. Therefore I > don't think the suggestion to use wxPython or PyQt is that helpful. Completely agree. However people started throwing PyQt for some unknown reason, so I had to respond. ;-) Thank you. > > Do you have any other suggestions? > > Even Cairo is pretty complicated (requiring pens and contexts) for what > he's asking for. > > Personally I don't think it gets much easier or simpler than this sort > of thing: > https://github.com/Mekire/pygame-samples > -- > https://mail.python.org/mailman/listinfo/python-list From bc at freeuk.com Wed Jan 10 15:13:55 2018 From: bc at freeuk.com (bartc) Date: Wed, 10 Jan 2018 20:13:55 +0000 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: On 10/01/2018 17:36, Michael Torrie wrote: > On 01/10/2018 10:22 AM, Igor Korot wrote: >> Hi, >> wxPython/Phoenix. >> It can do everything you need and more.... > > But the OP isn't looking for a full-blown GUI toolkit. I went back and > re-read his post to be sure I wasn't misunderstanding. Therefore I > don't think the suggestion to use wxPython or PyQt is that helpful. > > Do you have any other suggestions? > > Even Cairo is pretty complicated (requiring pens and contexts) for what > he's asking for. > > Personally I don't think it gets much easier or simpler than this sort > of thing: > https://github.com/Mekire/pygame-samples I couldn't see anything obviously simple there. A lot seems to do with interaction which is always much more complicated than just drawing stuff. 'Turtle' will do it (assuming there's a way of drawing things without having to watch an actual turtle symbol crawling around the screen). One simple library of my own (not for Python) would use one function call to create a window, and another to draw an element (say, a box) within that window. (Plus a third to keep it on the screen otherwise it will disappear when the program terminates.) The only way to get it simpler than that is where the display window is always present (like some old Basics). -- bartc From mikhailwas at gmail.com Wed Jan 10 19:55:47 2018 From: mikhailwas at gmail.com (Mikhail V) Date: Thu, 11 Jan 2018 01:55:47 +0100 Subject: Simple graphic library for beginners Message-ID: > > But the OP isn't looking for a full-blown GUI toolkit. I went back and > > re-read his post to be sure I wasn't misunderstanding. Therefore I > > don't think the suggestion to use wxPython or PyQt is that helpful. > > > > Do you have any other suggestions? > > > > Even Cairo is pretty complicated (requiring pens and contexts) for what > > he's asking for. > > > > Personally I don't think it gets much easier or simpler than this sort > > of thing: > > https://github.com/Mekire/pygame-samples > > I couldn't see anything obviously simple there. A lot seems to do with > interaction which is always much more complicated than just drawing stuff. > > 'Turtle' will do it (assuming there's a way of drawing things without > having to watch an actual turtle symbol crawling around the screen). > That's right. Cairo, IIRC is mainly used as a rasterizer backend, so it is not what OP wants. Pygame is a SDL wrapper (a low-level lib), and NOT a wrapper for GUI widgets. Almost everything must be made from ground up - keyborad/mouse input, buttons (OP told about buttons). So there not even such thing as "button" there - Pygame has nothing of that high-level concept by default, so it will require an 3d party module that implements those. But, for *programming* classes, I personally would still recommend Pygame. As for something quick and simple, Turtle seems to be good. BTW - openCV is an excellent lib for drawing and image manipulation. It has tons of ready-to-use functions and can show things up in a window without additional coding. But again: if you want *buttons*, then you are merely speaking about a framework, not a lib. IOW if you drop the beloved by people "buttons" idea, then you have much more good options. PyQT is probably overkill, but it has a tool for GUI construction (qtdesigner) and it makes it child-easy to draw a window with buttons, and some canvas object, and then auto-creates pieces of code with objects and interaction slots. So PyQT is imo better option for noobs than Pygame. So here you have buttons and lots of OOP bloat as a consequence. Mikhail From bc at freeuk.com Wed Jan 10 21:09:25 2018 From: bc at freeuk.com (bartc) Date: Thu, 11 Jan 2018 02:09:25 +0000 Subject: Tips or strategies to understanding how CPython works under the hood (Posting On Python-List Prohibited) In-Reply-To: <6344c637-d534-4509-915d-acfbc10478c3@googlegroups.com> References: <87zi5mbxku.fsf@universite-de-strasbourg.fr.invalid> <6344c637-d534-4509-915d-acfbc10478c3@googlegroups.com> Message-ID: On 10/01/2018 23:31, Lawrence D?Oliveiro wrote: > On Thursday, January 11, 2018 at 1:08:25 AM UTC+13, bartc wrote: >> >> But I'm not convinced that register-based is necessarily faster. > > Not if your code is dominated by memory accesses, as a dynamic language is likely to be. But ask the people who design machine architectures, and who write compilers for them for languages like C--they?ll tell you it makes a helluva difference. > I'm not sure what you mean here. The subject is byte-code interpreters not statically compiled languages running native code, which are obviously better off using real hardware registers. The 'registers' in this example - for actual byte-code execution not what happens in pypy and LuaJIT - don't appear to be actual hardware registers. They can't be when an implementation is 100% high level code. They are just a different scheme to address instruction operands stored in memory, which can offer some advantages, with a few downsides. (I've been writing byte-code interpreters for stack-based VMs for years. I've never had much luck moving away from that model. At the moment they generally run the same algorithms a bit faster than register-based Lua, but can also be trivially accelerated to be several times faster, while still executing sequential byte-code. Not as fast as LuaJIT, but what goes on in that is beyond the methods discussed here for CPython and such.) -- bartc From rustompmody at gmail.com Wed Jan 10 23:23:33 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Wed, 10 Jan 2018 20:23:33 -0800 (PST) Subject: Pandas printing in jupyter Message-ID: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> If I make a data-frame in pandas in jupyter notebook it prints very nicely ie it looks quite like a spreadsheet How does it do it? Who does it? The data-frame does not seem to have str/repr methods? From torriem at gmail.com Thu Jan 11 00:16:47 2018 From: torriem at gmail.com (Michael Torrie) Date: Wed, 10 Jan 2018 22:16:47 -0700 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: On 01/10/2018 01:13 PM, bartc wrote: > I couldn't see anything obviously simple there. A lot seems to do with > interaction which is always much more complicated than just drawing stuff. Yes the link didn't have the simple examples I hoped for. How's this: ----------------------------- import pygame import time pygame.init() screen = pygame.display.set_mode((1024, 768) ) red = (255,0,0) green = (0,255,0) screen.fill( (255,255,255) ) pygame.draw.lines(screen, red, False, ((0,0),(100,100))) pygame.draw.lines(screen, green, False, ((0,100),(100,0))) pygame.display.update() time.sleep(5) pygame.quit() ------------------------------ PyGame has other primitives like circles, ellipses, etc. Much like the old BASIC graphics primitives. > 'Turtle' will do it (assuming there's a way of drawing things without > having to watch an actual turtle symbol crawling around the screen). Yes I think it can. > One simple library of my own (not for Python) would use one function > call to create a window, and another to draw an element (say, a box) > within that window. (Plus a third to keep it on the screen otherwise it > will disappear when the program terminates.) Are you thinking of sprites? > The only way to get it simpler than that is where the display window is > always present (like some old Basics). This is sort of similar to PyGame's concepts. From dieter at handshake.de Thu Jan 11 02:23:21 2018 From: dieter at handshake.de (dieter) Date: Thu, 11 Jan 2018 08:23:21 +0100 Subject: python server socket file transfer References: <81b959e9-793a-4d24-8ee9-dc989da572bc@googlegroups.com> <87vaga5h0w.fsf@handshake.de> Message-ID: <87shbchn9y.fsf@handshake.de> bingbong3334 at gmail.com writes: > On Wednesday, January 10, 2018 at 9:07:33 AM UTC+2, dieter wrote: >> bingbong3334 at gmail.com writes: >> > how much client can i handel whit this code what the amount of client that i can handel >> > the size of the file is 716 kb >> > ... >> > self.sock.send(l) >> >> Please read the documentation for *send* in the "socket" module: >> it tells you that "send" (in contrast to "sendall") is *not* guarantied >> to send the complete *l* (there is no guarantee how much is sent); >> the return value of "send" tells you how many bytes have been sent. > > > bro i dont ask about if it works or not or what you say is not right i ask how much he can handel... This depends on implementation details of your network package (and maybe other even less stable things). Thus, there is no reliable anwser to your question. And you do not need an anwser: use "sendall" or look on the "send" return value to learn how much has been transmitted (and use subsequent "send"'s to transmit the rest). From p.f.moore at gmail.com Thu Jan 11 03:43:10 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 11 Jan 2018 08:43:10 +0000 Subject: Pandas printing in jupyter In-Reply-To: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> Message-ID: The HTML representation is supplied by the object's _repr_html_ method. See https://ipython.org/ipython-doc/3/config/integrating.html for some details. >>> import pandas as pd >>> df = pd.DataFrame() >>> df._repr_html_() '
\n\n\n \n \n \n \n \n \n \n
\n
' >>> help(df._repr_html_) Help on method _repr_html_ in module pandas.core.frame: _repr_html_() method of pandas.core.frame.DataFrame instance Return a html representation for a particular DataFrame. Mainly for IPython notebook. >>> Paul On 11 January 2018 at 04:23, Rustom Mody wrote: > If I make a data-frame in pandas in jupyter notebook it prints very nicely > ie it looks quite like a spreadsheet > > How does it do it? > Who does it? > > The data-frame does not seem to have str/repr methods? > -- > https://mail.python.org/mailman/listinfo/python-list From rustompmody at gmail.com Thu Jan 11 03:59:53 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 11 Jan 2018 00:59:53 -0800 (PST) Subject: Pandas printing in jupyter In-Reply-To: References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> Message-ID: On Thursday, January 11, 2018 at 2:13:46 PM UTC+5:30, Paul Moore wrote: > The HTML representation is supplied by the object's _repr_html_ > method. See https://ipython.org/ipython-doc/3/config/integrating.html > for some details. > > >>> import pandas as pd > >>> df = pd.DataFrame() > >>> df._repr_html_() > '
\n From rosuav at gmail.com Fri Jan 12 06:44:24 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Jan 2018 22:44:24 +1100 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <1pP5C.29853$R83.10295@fx45.am4> <1TP5C.257875$pC5.145768@fx41.am4> <4ef00edd-ceed-4d26-2dee-eecbef93c44e@timgolden.me.uk> Message-ID: On Fri, Jan 12, 2018 at 9:00 PM, Paul Moore wrote: > On 12 January 2018 at 09:12, Tim Golden wrote: >> I think the shame here is that there is a learning opportunity on both >> sides. As Paul says: by and large, the huge amount of work which the Python >> Packaging team, especially the pip developers, have put in has paid off. >> It's now usually possible to say: "pip install XXX" and have it work out of >> the box for any recentish version of Python on any recentish version of a >> mainstream OS. Once people understand the basics of using that "interface", >> many things become simple. >> >> Unfortunately, where that *doesn't* work, it probably won't be because of >> low-hanging fruit: it'll be because of some strange interaction of different >> versions, odd leftover PATH setups, obscure Windows C Runtime redistribution >> issues, poor encoding interactions between Python and the Windows console, >> and so on. > > Agreed. The other factor, and in my experience one of the most > frustrating to deal with, is when people *don't* start with "pip > install XXX", but instead find some complex process on the web, try > that, have it fail, and then we have to help them untangle whatever > mess that might have left for them. This is particularly common on Windows, where the normal way to get software is "go look on the web, download something, hope it's the right thing, and give it free reign to install itself on your computer". On Linux systems, people tend to be a bit more familiar with the concept of package managers, so they aren't surprised to learn that Python has one. (Of course, there is still the big question of "which package manager ought I to be using", very much an XKCD 927 situation, but at least that's easier to ask about. "How did you install Python?" "With apt-get." "Okay, then use apt-get to install the package." "It's not in apt's repositories." "No problem, pip it is.") ChrisA From breamoreboy at gmail.com Fri Jan 12 06:48:21 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 12 Jan 2018 03:48:21 -0800 (PST) Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: <7c7aaf57-ade1-4dc4-bbd1-eb349d31566c@googlegroups.com> On Friday, January 12, 2018 at 6:52:32 AM UTC, Steven D'Aprano wrote: > On Fri, 12 Jan 2018 12:45:04 +1300, Gregory Ewing wrote: > > > Seems to me it would help if pip were to announce which version of > > Python it's installing things into. And instead of just saying "not > > compatible with this version of Python", say "not compatible with Python > > X.Y.Z". That would make these kinds of problems much easier to diagnose. > > This. > > If pip is joined at the hip to a specific version of Python, I think that > we ought to be able to specify the version number like we can with Python. > > Something like: > > pip ... # use whichever version of pip comes first on the PATH > pip3.6 ... # use the pip installed with Python 3.6 > pip2.7 ... # use the pip installed with Python 2.7 The time machine has struck again. The scripts directory on Windows always gives you three executables. pip.exe is always one of them, and then (say) pip3.exe and pip3.6.exe. > etc. > > And don't say "use a venv" :-) > -- > Steven D'Aprano -- Kindest regards. Mark Lawrence. From bc at freeuk.com Fri Jan 12 09:52:42 2018 From: bc at freeuk.com (bartc) Date: Fri, 12 Jan 2018 14:52:42 +0000 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <1pP5C.29853$R83.10295@fx45.am4> <1TP5C.257875$pC5.145768@fx41.am4> Message-ID: On 12/01/2018 01:56, Chris Angelico wrote: > On Fri, Jan 12, 2018 at 12:21 PM, bartc wrote: >> On 11/01/2018 23:23, Chris Angelico wrote: >>> >>> On Fri, Jan 12, 2018 at 10:11 AM, bartc wrote: >> >> >>> I'm almost ready to plonk you, but I think there is still SOME value >>> in your posts. But please, stop denigrating what you don't understand. >> >> >> And please try to see things from the pointer of view of a beginner or >> outsider, where anything to do with Python seems to have a bewildering and >> conflicting array of solutions. >> > You mean the sort of person who goes to the front page The front page of what? and sees just > two options, 2.7 and 3.6, and won't see anything about 3.7 until it's > released? Google for 'download python windows'. The first hit is this site: https://www.python.org/downloads/windows/ But before you click on it, the google search summary includes two links in big type, to 2.7.14 and 3.7.0a2 (which are to information not downloads). That is a strong suggestion that those are the latest versions. When you do go to that page, you are looking for a link that has 'download' next to it. The first 6 such links on the page are all for 3.7. If you a beginner, outsider, or infrequent user of Python with no idea of what the latest version is, except that you already have 3.6 but it might have a problem, which would you choose? > Again, a number of people have put in a lot of hours to make > that easy. It appears that because of that I'm not allowed to make any observations, not ones that could be construed as criticism. But as another example, search for 'download pelles c', the first hit should be this page: http://www.pellesc.de/index.php?lang=en&page=download That I think is a better presentation than Python's. It's also clearer which is 32-bit and which is 64. (Will a beginner programmer who's not going to be coding low level really appreciate the difference between x86 and x86-64?) But here's another example that goes the other way; the first hit for 'download lua windows' is this: https://www.lua.org/download.html A rather puzzling page with some gobbledygook on it that appears to do with Linux. And the download link gives a list of tar.gz files; not very useful. The second hit isn't much more use; the third one is more promising. I do get the impression that applications that originate on Windows are generally more accessible. -- bartc From lele at metapensiero.it Fri Jan 12 10:23:46 2018 From: lele at metapensiero.it (Lele Gaifax) Date: Fri, 12 Jan 2018 16:23:46 +0100 Subject: Simple graphic library for beginners References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <1pP5C.29853$R83.10295@fx45.am4> <1TP5C.257875$pC5.145768@fx41.am4> Message-ID: <87k1wnaynx.fsf@metapensiero.it> bartc writes: > If you a beginner, outsider, or infrequent user of Python with no idea of > what the latest version is, except that you already have 3.6 but it might > have a problem, which would you choose? Unless you are also unable to read *and* understand, by any chance you'd follow the very first link related to 3.7, and seeing "This is an early developer preview of Python 3.7" you could *decide* if that's appropriate for your goal. But it seems you belong to yet a different category of people indeed, who blindly follows the "I'm feeling lucky" Google advices. ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From mikhailwas at gmail.com Fri Jan 12 12:25:22 2018 From: mikhailwas at gmail.com (Mikhail V) Date: Fri, 12 Jan 2018 18:25:22 +0100 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: On Fri, Jan 12, 2018 at 10:38 AM, Paul Moore wrote: > On 12 January 2018 at 06:47, Steven D'Aprano > wrote: >> If pip is joined at the hip to a specific version of Python, I think that >> we ought to be able to specify the version number like we can with Python. >> >> Something like: >> >> pip ... # use whichever version of pip comes first on the PATH >> pip3.6 ... # use the pip installed with Python 3.6 >> pip2.7 ... # use the pip installed with Python 2.7 > > Well, that's sort of how it's intended to work, but in practice it > doesn't seem to be as straightforward as it ought to be. I can't > really say why, as it's a Unix-only thing and I don't really > understand the reasons, but it's why I'm a fan of the "python -m pip" > approach. There's discussion on the pip tracker if you're interested > enough to go searching - I know it's something that's been debated, > but I don't recall the context. In general I think more than one app and executable with same name on a system already asks for problems. On Windows I had issues with pip and pygame and I have two Pythons - 2.7.14 and 3.6.2. The problem was that I could not make it install to 2.7, and tried many options, looking in SO there were so many different solutions, but finally I found something that reinstalled newer pip, but I don't remember frankly. My opinion (from Windows user's POV) - I'd prefer if there should be only one PIP in system, so running pip will unambigiosly mean that I run (or upgrade) THE pip. (and not something which I don't even know where it is located). Ideally also it should be installed in separate Directory. So I'd have folders: python 27 python 36 pip And the target Python where the package will be installed should be defined by a switch, e.g. 'pip -2', 'pip -3' (in analogy with 'py -2', 'py -3'). The question is though, how pip will know what version(s) of python I have, and if I installed them later? Hmm, not an easy problem. So in this case pip shoud track the multiple versions each time I install another version of python. Mikhail From p.f.moore at gmail.com Fri Jan 12 13:02:11 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 12 Jan 2018 18:02:11 +0000 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> <2d3df7b9-e955-ef8a-7a46-e26a09c0ff70@vub.be> <14085993-f705-ef4d-01e0-264abe20c25d@gmail.com> <45cd141f-79c9-0b31-38b1-0e9a7ff5695c@gmail.com> Message-ID: On 12 January 2018 at 17:25, Mikhail V wrote: > And the target Python where the package will be installed should be defined by > a switch, e.g. 'pip -2', 'pip -3' (in analogy with 'py -2', 'py -3'). > The question is though, how pip will know what version(s) of python I have, and > if I installed them later? Hmm, not an easy problem. So in this case pip shoud > track the multiple versions each time I install another version of python. If that's how you want it to behave, just use "py -2 -m pip" or "py -3 -m pip" or "py -3.6 -m pip" as required. Works now, no hassle. You still have to install pip (the package, not the executable) in each Python home, but that's just how Python packages work (and pip is already installed by default when you install Python on Windows anyway). Paul From torriem at gmail.com Fri Jan 12 23:13:52 2018 From: torriem at gmail.com (Michael Torrie) Date: Fri, 12 Jan 2018 21:13:52 -0700 Subject: Simple graphic library for beginners In-Reply-To: References: <287B9FB2-D264-4EAC-BECF-3CC17F1FF5A8@mostrom.pp.se> Message-ID: On 01/11/2018 11:48 PM, Jan Erik Mostr?m wrote: > On 10 Jan 2018, at 13:40, Jan Erik Mostr?m wrote: > >> I'm looking for a really easy to use graphic library. The target users >> are teachers who have never programmed before and is taking a first >> (and possible last) programming course. > > Thanks for all the suggestions, I'm going to take a look at them and see > what fits my requirements best. I'm glad you are willing to overlook all the noise in this thread! Good luck and let us know what path you end up taking. From zingy7.yz at gmail.com Sat Jan 13 04:13:08 2018 From: zingy7.yz at gmail.com (yimba) Date: Sat, 13 Jan 2018 01:13:08 -0800 (PST) Subject: anyway you can connect your reddit to your crypto account to send people coins? Message-ID: wondering if its possible to have a bot that is able to send coins to a person if you choose to From gandalf at shopzeus.com Sat Jan 13 05:34:18 2018 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Sat, 13 Jan 2018 11:34:18 +0100 Subject: requests / SSL blocks forever? Message-ID: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> Hi! I have a multi threaded Windows service written in Python. It is running on 3.6.2.? Sometimes I cannot stop the service, because on of the threads won't exit. I have narrowed down the problem to request and _lib.SSL_read. I have used a modified version of this gem: http://code.activestate.com/recipes/577334-how-to-debug-deadlocked-multi-threaded-programs/ So even thould I cannot stop the service (only kill it from task manager), I can tell that this is the rebellious thread: File: "C:\Python36-32\lib\site-packages\requests\api.py", line 112, in post ? return request('post', url, data=data, json=json, **kwargs) File: "C:\Python36-32\lib\site-packages\requests\api.py", line 58, in request ? return session.request(method=method, url=url, **kwargs) File: "C:\Python36-32\lib\site-packages\requests\sessions.py", line 508, in request ? resp = self.send(prep, **send_kwargs) File: "C:\Python36-32\lib\site-packages\requests\sessions.py", line 658, in send ? r.content File: "C:\Python36-32\lib\site-packages\requests\models.py", line 823, in content ? self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes() File: "C:\Python36-32\lib\site-packages\requests\models.py", line 745, in generate ? for chunk in self.raw.stream(chunk_size, decode_content=True): File: "C:\Python36-32\lib\site-packages\urllib3\response.py", line 436, in stream ? data = self.read(amt=amt, decode_content=decode_content) File: "C:\Python36-32\lib\site-packages\urllib3\response.py", line 384, in read ? data = self._fp.read(amt) File: "C:\Python36-32\lib\http\client.py", line 449, in read ? n = self.readinto(b) File: "C:\Python36-32\lib\http\client.py", line 493, in readinto ? n = self.fp.readinto(b) File: "C:\Python36-32\lib\socket.py", line 586, in readinto ? return self._sock.recv_into(b) File: "C:\Python36-32\lib\site-packages\urllib3\contrib\pyopenssl.py", line 280, in recv_into ? return self.connection.recv_into(*args, **kwargs) File: "C:\Python36-32\lib\site-packages\OpenSSL\SSL.py", line 1624, in recv_into ? result = _lib.SSL_read(self._ssl, buf, nbytes) It is possible that there are network outages on this machine, but I don't think that this should result in an infinite block. The bigest problem is that the only way to restart the service is to login remotely, start the task manager and terminate the pythonservice.exe process. Do any of you have a clue why is this happening? How to fix this? Thanks, ?? Laszlo From tjol at tjol.eu Sat Jan 13 07:01:07 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 13 Jan 2018 13:01:07 +0100 Subject: requests / SSL blocks forever? In-Reply-To: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> Message-ID: On 13/01/18 11:34, Nagy L?szl? Zsolt wrote: > Hi! > > I have a multi threaded Windows service written in Python. It is running > on 3.6.2.? Sometimes I cannot stop the service, because on of the > threads won't exit. I have narrowed down the problem to request and > _lib.SSL_read. I have used a modified version of this gem: > http://code.activestate.com/recipes/577334-how-to-debug-deadlocked-multi-threaded-programs/ > > So even thould I cannot stop the service (only kill it from task > manager), I can tell that this is the rebellious thread: > > > File: "C:\Python36-32\lib\site-packages\requests\api.py", line 112, in post Unfortunately, this traceback doesn't contain any of your own code, so we can't really tell what you're trying to do, or whether this call *should* be terminating. > [snip] > > File: "C:\Python36-32\lib\site-packages\OpenSSL\SSL.py", line 1624, in > recv_into > ? result = _lib.SSL_read(self._ssl, buf, nbytes) so - this is calling a C library. At that point, if the C function doesn't return, there's not really anything Python can do. My point is: if the server is simply not sending a response (because it's waiting for you to send more data, or whatever) but *is* keeping the TCP connection alive - this could make sense. Are you setting a timeout? -- Thomas > > It is possible that there are network outages on this machine, but I > don't think that this should result in an infinite block. > > The bigest problem is that the only way to restart the service is to > login remotely, start the task manager and terminate the > pythonservice.exe process. > > Do any of you have a clue why is this happening? How to fix this? > > Thanks, > > ?? Laszlo > > > From tjol at tjol.eu Sat Jan 13 07:54:25 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 13 Jan 2018 13:54:25 +0100 Subject: pip --user by default Message-ID: Hi, I recently discovered the wonders of pip.conf: if I create a file ~/.config/pip/pip.conf* with: [install] user = true then pip will install to the --user site-packages by default, rather than trying to install packages into system directories. The trouble is that this fails when you also use virtualenvs. In a virtualenv, --user doesn't work, so pip fails when trying to install anything in a virtualenv as long as the user pip.conf contains those lines. Short of adding a pip.conf to every single virtualenv, is there any way to work around this, and configure pip to install packages - into the user directories if possible - into the environment when in an environment by default? Thanks, Thomas * the path obviously depends on the OS: https://pip.pypa.io/en/stable/user_guide/#config-file From jon+usenet at unequivocal.eu Sat Jan 13 09:03:24 2018 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Sat, 13 Jan 2018 14:03:24 -0000 (UTC) Subject: requests / SSL blocks forever? References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> Message-ID: On 2018-01-13, Nagy L?szl? Zsolt wrote: > I have a multi threaded Windows service written in Python. It is running > on 3.6.2.? Sometimes I cannot stop the service, because on of the > threads won't exit. I have narrowed down the problem to request and > _lib.SSL_read. (a) are you setting daemon=True on the thread? (b) are you setting a timeout on the requests call? From boffi at questa.la.so Sat Jan 13 16:57:35 2018 From: boffi at questa.la.so (boffi at questa.la.so) Date: Sat, 13 Jan 2018 22:57:35 +0100 Subject: Pandas printing in jupyter References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> <130ddd25-f45a-4f23-a333-e5d1bb0b204a@googlegroups.com> Message-ID: <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> Rustom Mody writes: > Specifically and for starters, I want a numpy array ? lets say 2D to > start with ? to be displayed(displayable) as elegantly as sympy does > to (its) matrices ######################################################################## import numpy as np from IPython.display import Latex def prmat(mat): return (r'\begin{bmatrix}' + r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + r'\end{bmatrix}' a = np.arange(12).reshape((3, 4))+1 display(Latex(prmat(a))) ######################################################################## you could add optional arguments to modify the type of brackets and the formatting string From boffi at pentangle.appreciation.net Sat Jan 13 17:10:21 2018 From: boffi at pentangle.appreciation.net (boffi at pentangle.appreciation.net) Date: Sat, 13 Jan 2018 23:10:21 +0100 Subject: Pandas printing in jupyter References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> <130ddd25-f45a-4f23-a333-e5d1bb0b204a@googlegroups.com> <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> Message-ID: <878td1beb6.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> boffi at questa.la.so writes: > def prmat(mat): > return (r'\begin{bmatrix}' + > r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + > r'\end{bmatrix}' add a closing parenthesis here ^ From frank at chagford.com Sun Jan 14 03:30:31 2018 From: frank at chagford.com (Frank Millman) Date: Sun, 14 Jan 2018 10:30:31 +0200 Subject: Detecting a cycle in a graph Message-ID: Hi all I am adding a bpm (Business Process Management) module to my accounting app. A process is laid out as a flowchart, and is therefore a form of directed graph, so I am getting into a bit of graph theory. I got some good ideas from this essay - https://www.python.org/doc/essays/graphs/ I need to detect when a 'cycle' occurs - when a path loops back on itself and revisits a node previously visited. I have figured out a way to do this, but I have a problem. A cycle occurs as a result of a node with more than one path leading out from it. I am following the BPMN spec, and they call these nodes 'gateways', so I will use that terminology. A gateway can optionally have a boolean condition associated with it, which determines which path is followed. If a given path loops back to an earlier node, a cycle is created. I can detect a cycle in a path. It is possible for there to be more than one gateway in the path. I want to identify the gateway that actually triggered the cycle, but I have not figured out a way to do this. I scribbled down a pseudo process on the back of an envelope. It has a gateway that can cause a cycle. Before that, there is a gateway that checks for a 'fast-track' indicator. If true, it bypasses the next part of the process and jumps to the next stage. After that, in the path returning to the previous node, there is another gateway that checks a counter. If the boolean check fails three times, terminate the process. I can 'visually' identify the gateway that triggers the cycle, but I cannot figure out how to determine it 'logically'. My intention is that users can create their own processes, and we know that they can sometimes create a dog's breakfast, so I cannot rely on it being 'obvious'. Maybe there is no logical way of identifying it. I am sure you will need more details if you want to assist, but maybe there is some literature you can point me to that explains these things in more detail. Thanks Frank Millman From steve+comp.lang.python at pearwood.info Sun Jan 14 05:04:01 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 14 Jan 2018 10:04:01 +0000 (UTC) Subject: Detecting a cycle in a graph References: Message-ID: On Sun, 14 Jan 2018 10:30:31 +0200, Frank Millman wrote: > I can detect a cycle in a path. It is possible for there to be more than > one gateway in the path. I want to identify the gateway that actually > triggered the cycle, but I have not figured out a way to do this. You don't need a gateway to get a cycle. Suppose you have nodes A -> B -> C -> D -> B that's a cycle with no gateways. If you do have gateways, who is to say which one is to blame? Or even that *any* of them is to blame? A -> (B) -> C -> F -> E \ D -> E -> F Why say that the gateway (B) is to blame for the cycle E -> F -> E? I'm sure that if you draw some diagrams, you can come up with a scenario where a cycle is formed by the action of two gateways, either one on its own being harmless. Why blame one rather than the other? For that matter, why are cycles necessarily harmful? So long as you can escape a cycle eventually, that's just a loop with an exit: A -> B -> C -> (D) -> E -> B \ X Now the path will loop B -> ... -> E and back to B, until the gateway D takes the alternate path and escapes. The usual way of fixing this sort of thing for systems intended for non- expert use is to have a (user-configurable?) limit on the number of steps, and have the process raise an error once it reaches the limit. > I scribbled down a pseudo process on the back of an envelope. It has a > gateway that can cause a cycle. > > Before that, there is a gateway that checks for a 'fast-track' > indicator. If true, it bypasses the next part of the process and jumps > to the next stage. > > After that, in the path returning to the previous node, there is another > gateway that checks a counter. If the boolean check fails three times, > terminate the process. What boolean check? You mentioned a counter -- that's normally considered to be an integer. > I can 'visually' identify the gateway that triggers the cycle, but I > cannot figure out how to determine it 'logically'. My intention is that > users can create their own processes, and we know that they can > sometimes create a dog's breakfast, so I cannot rely on it being > 'obvious'. Maybe there is no logical way of identifying it. > > I am sure you will need more details if you want to assist, but maybe > there is some literature you can point me to that explains these things > in more detail. Try googling for "Halting problem" to learn why you cannot, in general, prove that all cycles will eventually terminate. You may be able to identify *some* non-terminating graphs, but you cannot determine all of them. -- Steve From __peter__ at web.de Sun Jan 14 05:31:34 2018 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Jan 2018 11:31:34 +0100 Subject: Generating SVG from turtle graphics References: Message-ID: Steven D'Aprano wrote: > I'd like to draw something with turtle, then generate a SVG file from it. > > Is this possible? If this is a one-off job consider creating Postscript from the underlying Canvas: >>> import turtle >>> for i in range(12): ... turtle.forward(100) ... turtle.left(150) ... >>> turtle.getcanvas().postscript(file="tmp_turtle.ps") [...] Then convert to SVG with an external tool. It looks like ghostscript can do that: $ gs -dBATCH -dNOPAUSE -sDEVICE=svg -sOutputFile=tmp_turtle.svg tmp_turtle.ps From frank at chagford.com Sun Jan 14 06:04:17 2018 From: frank at chagford.com (Frank Millman) Date: Sun, 14 Jan 2018 13:04:17 +0200 Subject: Detecting a cycle in a graph In-Reply-To: References: Message-ID: "Steven D'Aprano" wrote in message news:p3f9uh$ar4$1 at blaine.gmane.org... > On Sun, 14 Jan 2018 10:30:31 +0200, Frank Millman wrote: > > > I can detect a cycle in a path. It is possible for there to be more than > > one gateway in the path. I want to identify the gateway that actually > > triggered the cycle, but I have not figured out a way to do this. > > You don't need a gateway to get a cycle. Suppose you have nodes > > A -> B -> C -> D -> B > > that's a cycle with no gateways. > [skip more good examples] > > The usual way of fixing this sort of thing for systems intended for non- > expert use is to have a (user-configurable?) limit on the number of > steps, and have the process raise an error once it reaches the limit. > > > After that, in the path returning to the previous node, there is another > > gateway that checks a counter. If the boolean check fails three times, > > terminate the process. > > What boolean check? You mentioned a counter -- that's normally considered > to be an integer. > Sorry, I left out a couple of steps. In my imaginary process, every time the boolean check in the first gateway fails, it increments a counter. The second gateway checks the counter, and terminates the process if it exceeds a limit. > Try googling for "Halting problem" to learn why you cannot, in general, > prove that all cycles will eventually terminate. You may be able to > identify *some* non-terminating graphs, but you cannot determine all of > them. I will do so. Thanks for clearing up some confusion and giving me some pointers. Frank From arj.python at gmail.com Sun Jan 14 07:32:53 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 14 Jan 2018 16:32:53 +0400 Subject: Generating SVG from turtle graphics In-Reply-To: References: Message-ID: maybe save to .png then use another tool to svg On 11 Jan 2018 15:06, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > I'd like to draw something with turtle, then generate a SVG file from it. > > Is this possible? > > If not, is there something I can do which lets me plot lines, shapes and > curves and output to SVG? > > Ideally, I'd like to draw a figure pixel by pixel, and then have the SVG > library fit a bezier curve to it. > > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From storchaka at gmail.com Sun Jan 14 11:44:38 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sun, 14 Jan 2018 18:44:38 +0200 Subject: Generating SVG from turtle graphics In-Reply-To: References: Message-ID: 11.01.18 13:03, Steven D'Aprano ????: > I'd like to draw something with turtle, then generate a SVG file from it. > > Is this possible? > > If not, is there something I can do which lets me plot lines, shapes and > curves and output to SVG? You can translate the following Tcl/Tk recipe to Python/Tkinter: http://wiki.tcl.tk/4534. Or just run the code directly in the buildin Tcl interpreter. From breamoreboy at gmail.com Sun Jan 14 16:02:13 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 14 Jan 2018 13:02:13 -0800 (PST) Subject: Why does pylint give this warning? Message-ID: <02e57990-7f39-42c4-9407-bef553281768@googlegroups.com> The warning is 'C0103:Method name "__len__" doesn't conform to '_?_?[a-z][A-Za-z0-9]{1,30}$' pattern' but it doesn't complain about __repr__ or __str__. If there is an explanation out in the wild my search fu has missed it :-( My setup on Ubuntu 17.10 is:- $ pylint --version Using config file /home/mark/.pylintrc pylint 1.8.1, astroid 1.6.0 Python 3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] -- Kindest regards. Mark Lawrence. From auriocus at gmx.de Sun Jan 14 16:04:45 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 14 Jan 2018 22:04:45 +0100 Subject: Detecting a cycle in a graph In-Reply-To: References: Message-ID: Am 14.01.18 um 09:30 schrieb Frank Millman: > I need to detect when a 'cycle' occurs - when a path loops back on > itself and revisits a node previously visited. I have figured out a way > to do this, but I have a problem. I don't know if that helps, but there is a classic graph theory algorithm called "Floyd's cycle detector". The idea is to have a pointer move along the graph and a second one which runs at double the speed. If they meet, you found a cycle. It is not straight-forward to come up with this algorithm, which even runs in constant storage. ISTR that there are some variants which can give you the split point of the cycle, too. Christian From auriocus at gmx.de Sun Jan 14 16:14:44 2018 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 14 Jan 2018 22:14:44 +0100 Subject: Detecting a cycle in a graph In-Reply-To: References: Message-ID: Am 14.01.18 um 22:04 schrieb Christian Gollwitzer: > Am 14.01.18 um 09:30 schrieb Frank Millman: >> I need to detect when a 'cycle' occurs - when a path loops back on >> itself and revisits a node previously visited. I have figured out a >> way to do this, but I have a problem. > > I don't know if that helps, but there is a classic graph theory > algorithm called "Floyd's cycle detector". The idea is to have a pointer > move along the graph and a second one which runs at double the speed. If > they meet, you found a cycle. It is not straight-forward to come up with > this algorithm, which even runs in constant storage. ISTR that there are > some variants which can give you the split point of the cycle, too. And here is an algorithm to enumerate all cycles in a directed graph: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.5999&rep=rep1&type=pdf with an implementation in C++ here: https://github.com/hellogcc/circuit-finding-algorithm Christian From __peter__ at web.de Sun Jan 14 17:11:35 2018 From: __peter__ at web.de (Peter Otten) Date: Sun, 14 Jan 2018 23:11:35 +0100 Subject: Why does pylint give this warning? References: <02e57990-7f39-42c4-9407-bef553281768@googlegroups.com> Message-ID: breamoreboy at gmail.com wrote: > Why does pylint give this warning? No idea. > The warning is 'C0103:Method name "__len__" doesn't conform to > '_?_?[a-z][A-Za-z0-9]{1,30}$' pattern' but it doesn't complain about > __repr__ or __str__. If there is an explanation out in the wild my search > fu has missed it :-( > > My setup on Ubuntu 17.10 is:- > > $ pylint --version > Using config file /home/mark/.pylintrc > pylint 1.8.1, > astroid 1.6.0 > Python 3.6.3 (default, Oct 3 2017, 21:45:48) > [GCC 7.2.0] > > -- > Kindest regards. > > Mark Lawrence. I cannot replicate this with $ pylint --version Using config file /home/petto/.pylintrc pylint 1.8.1, astroid 1.6.0 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] $ cat pylint_fodder.py class FooBar: def __len__(self): return 42 def __repr__(self): return "FooBar(length={})".format(len(self)) $ pylint pylint_fodder.py Using config file /home/petto/.pylintrc ************* Module pylint_fodder C: 1, 0: Missing module docstring (missing-docstring) C: 1, 0: Missing class docstring (missing-docstring) R: 1, 0: Too few public methods (0/2) (too-few-public-methods) ----------------------------------- Your code has been rated at 4.00/10 $ so the complaint is probably caused by your configuration file. Rename it, regenerate the default with $ pylint --generate-rcfile > ~/.pylintrc and reapply your changes -- I hope there aren't too many of them ;) -- until you see the warning message again. From skip.montanaro at gmail.com Sun Jan 14 17:31:44 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 14 Jan 2018 16:31:44 -0600 Subject: Why does pylint give this warning? In-Reply-To: References: <02e57990-7f39-42c4-9407-bef553281768@googlegroups.com> Message-ID: > I cannot replicate this with > > $ pylint --version > Using config file /home/petto/.pylintrc > pylint 1.8.1, > astroid 1.6.0 > Python 3.4.0 (default, Apr 11 2014, 13:05:11) > [GCC 4.8.2] > > $ cat pylint_fodder.py > class FooBar: > def __len__(self): > return 42 > def __repr__(self): > return "FooBar(length={})".format(len(self)) > > $ pylint pylint_fodder.py > Using config file /home/petto/.pylintrc > ************* Module pylint_fodder > C: 1, 0: Missing module docstring (missing-docstring) > C: 1, 0: Missing class docstring (missing-docstring) > R: 1, 0: Too few public methods (0/2) (too-few-public-methods) Ditto. Mark, do you have any tweaks in your .pylintrc file which might affect it? Skip From eternal-september at niles.xyz Sun Jan 14 17:55:01 2018 From: eternal-september at niles.xyz (Niles Rogoff) Date: Sun, 14 Jan 2018 22:55:01 -0000 (UTC) Subject: Generating SVG from turtle graphics References: Message-ID: On Sun, 14 Jan 2018 16:32:53 +0400, Abdur-Rahmaan Janhangeer wrote: > maybe save to .png then use another tool to svg PNG is a bitmap format[1], so you can't covert it to an SVG (a vector format) without guessing things like the start/end points of the lines, their slopes, etc... not to mention things like arcs. [1] RFC2083 From eternal-september at niles.xyz Sun Jan 14 17:56:12 2018 From: eternal-september at niles.xyz (Niles Rogoff) Date: Sun, 14 Jan 2018 22:56:12 -0000 (UTC) Subject: anyway you can connect your reddit to your crypto account to send people coins? References: Message-ID: On Sat, 13 Jan 2018 01:13:08 -0800, yimba wrote: > wondering if its possible to have a bot that is able to send coins to a > person if you choose to There's a pretty good library for reddit (praw), so as long as you can send your coins via another python library, it should be really easy From pengyu.ut at gmail.com Sun Jan 14 18:01:43 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sun, 14 Jan 2018 17:01:43 -0600 Subject: Where is the usage of (list comprehension) documented? Message-ID: Hi, I see the following usage of list comprehension can generate a generator. Does anybody know where this is documented? Thanks. $ cat main.py #!/usr/bin/env python import sys lines = (line.rstrip('\n') for line in sys.stdin) print lines lines = [line.rstrip('\n') for line in sys.stdin] print lines $ seq 10 | ./main.py at 0x1101ecd70> ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'] -- Regards, Peng From rosuav at gmail.com Sun Jan 14 18:05:44 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Jan 2018 10:05:44 +1100 Subject: Where is the usage of (list comprehension) documented? In-Reply-To: References: Message-ID: On Mon, Jan 15, 2018 at 10:01 AM, Peng Yu wrote: > Hi, > > I see the following usage of list comprehension can generate a > generator. Does anybody know where this is documented? Thanks. > > $ cat main.py > #!/usr/bin/env python > > import sys > lines = (line.rstrip('\n') for line in sys.stdin) > print lines > > lines = [line.rstrip('\n') for line in sys.stdin] > print lines > $ seq 10 | ./main.py > at 0x1101ecd70> > ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'] > When you use square brackets, you're creating a generator, as in your second example. Your first example is a slightly different beast called a "generator expression". If you search for that in the docs or on the web, you'll find what you want. ChrisA From breamoreboy at gmail.com Sun Jan 14 18:38:51 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sun, 14 Jan 2018 15:38:51 -0800 (PST) Subject: Why does pylint give this warning? In-Reply-To: References: <02e57990-7f39-42c4-9407-bef553281768@googlegroups.com> Message-ID: On Sunday, January 14, 2018 at 10:32:44 PM UTC, Skip Montanaro wrote: > > I cannot replicate this with > > > > $ pylint --version > > Using config file /home/petto/.pylintrc > > pylint 1.8.1, > > astroid 1.6.0 > > Python 3.4.0 (default, Apr 11 2014, 13:05:11) > > [GCC 4.8.2] > > > > $ cat pylint_fodder.py > > class FooBar: > > def __len__(self): > > return 42 > > def __repr__(self): > > return "FooBar(length={})".format(len(self)) > > > > $ pylint pylint_fodder.py > > Using config file /home/petto/.pylintrc > > ************* Module pylint_fodder > > C: 1, 0: Missing module docstring (missing-docstring) > > C: 1, 0: Missing class docstring (missing-docstring) > > R: 1, 0: Too few public methods (0/2) (too-few-public-methods) > > Ditto. Mark, do you have any tweaks in your .pylintrc file which might > affect it? > > Skip Having followed Peter's advice everything worked a treat once I'd put my changes into the new configuration file. Having compared it to the old one it looks as if I'd upgraded pylint but didn't regenerate the config file, as there were a number of obvious changes. Thanks for the help guys, and if nothing else this is on the record for anyone with the same or similar problems. -- Kindest regards. Mark Lawrence. From pengyu.ut at gmail.com Sun Jan 14 20:27:26 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Sun, 14 Jan 2018 19:27:26 -0600 Subject: Where is the usage of (list comprehension) documented? In-Reply-To: References: Message-ID: > When you use square brackets, you're creating a generator, as in your > second example. Your first example is a slightly different beast > called a "generator expression". If you search for that in the docs or > on the web, you'll find what you want. Thanks. Can the documentation be found by `help()` in python? -- Regards, Peng From ben+python at benfinney.id.au Sun Jan 14 20:41:54 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 15 Jan 2018 12:41:54 +1100 Subject: Where is the usage of (list comprehension) documented? References: Message-ID: <85tvvnkie5.fsf@benfinney.id.au> Peng Yu writes: > > When you use square brackets, you're creating a generator, as in your > > second example. Your first example is a slightly different beast > > called a "generator expression". If you search for that in the docs or > > on the web, you'll find what you want. > > Thanks. Can the documentation be found by `help()` in python? The ?help? command is not intended to provide comprehensive user documentation. Rather, it is intended primarily to provide a reminder of syntax and purpose of specific API. It is not a replacement for having the Python documentation available. -- \ ?The history of Western science confirms the aphorism that the | `\ great menace to progress is not ignorance but the illusion of | _o__) knowledge.? ?Daniel J. Boorstin, historian, 1914?2004 | Ben Finney From drsalists at gmail.com Sun Jan 14 21:57:48 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 14 Jan 2018 18:57:48 -0800 Subject: Where is the usage of (list comprehension) documented? In-Reply-To: References: Message-ID: On Sun, Jan 14, 2018 at 3:01 PM, Peng Yu wrote: > Hi, > > I see the following usage of list comprehension can generate a > generator. Does anybody know where this is documented? Thanks. Here's the (a?) generator expression PEP: https://www.python.org/dev/peps/pep-0289/ Here's a presentation I put together on this and related topics a while back: http://stromberg.dnsalias.org/~strombrg/Intro-to-Python/Python%20Generators,%20Iterators%20and%20Comprehensions%202014.pdf FWIW, [a for a in range(2)] is a list comprehension; it's eager. And (a for a in range(2)) is a generator expression; it's lazy. HTH From frank at chagford.com Mon Jan 15 01:15:32 2018 From: frank at chagford.com (Frank Millman) Date: Mon, 15 Jan 2018 08:15:32 +0200 Subject: Detecting a cycle in a graph In-Reply-To: References: Message-ID: "Christian Gollwitzer" wrote in message news:p3gh84$kfm$1 at dont-email.me... > > Am 14.01.18 um 22:04 schrieb Christian Gollwitzer: > > Am 14.01.18 um 09:30 schrieb Frank Millman: > >> I need to detect when a 'cycle' occurs - when a path loops back on > >> itself and revisits a node previously visited. I have figured out a way > >> to do this, but I have a problem. > > > > I don't know if that helps, but there is a classic graph theory > > algorithm called "Floyd's cycle detector". The idea is to have a pointer > > move along the graph and a second one which runs at double the speed. If > > they meet, you found a cycle. It is not straight-forward to come up with > > this algorithm, which even runs in constant storage. ISTR that there are > > some variants which can give you the split point of the cycle, too. > > And here is an algorithm to enumerate all cycles in a directed graph: > > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.5999&rep=rep1&type=pdf > > with an implementation in C++ here: > > https://github.com/hellogcc/circuit-finding-algorithm > I appreciate the input, Christian, but I am afraid both of those were over my head :-( I think/hope that a business process graph does not require such a complex solution. Here is my cycle-detecting algorithm. In BPMN terms, each node can have 0->* incoming connections, and 0->* outgoing connections. Any node with 0 incoming is deemed to start the process. Normally there is only one such node. Any node with 0 outgoing represents the end of an 'active path'. If there is more than one such node, all 'active' ones must reach the end before the process is finished. There is no formal definition of an 'active path', and I can think of a few corner cases which could prove problematic, but normally the meaning is clear. I start my cycle-detection with a node with 0 incoming connections. def find_cycle(node, path): for output in node.outputs: if output in path: print('Cycle found in', path) else: new_path = path[:] new_path.append(output) find_cycle(output, new_path) find_cycle(node, [node]) This traverses every possible path in the graph. I think/hope that a typical business process will not grow so large as to cause a problem. If anyone can see a flaw in this, please let me know. Frank From ned at nedbatchelder.com Mon Jan 15 06:41:46 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 15 Jan 2018 06:41:46 -0500 Subject: Where is the usage of (list comprehension) documented? In-Reply-To: References: Message-ID: On 1/14/18 9:57 PM, Dan Stromberg wrote: > On Sun, Jan 14, 2018 at 3:01 PM, Peng Yu wrote: >> Hi, >> >> I see the following usage of list comprehension can generate a >> generator. Does anybody know where this is documented? Thanks. > Here's the (a?) generator expression PEP: > https://www.python.org/dev/peps/pep-0289/ > > Here's a presentation I put together on this and related topics a while back: > http://stromberg.dnsalias.org/~strombrg/Intro-to-Python/Python%20Generators,%20Iterators%20and%20Comprehensions%202014.pdf > > FWIW, [a for a in range(2)] is a list comprehension; it's eager. And > (a for a in range(2)) is a generator expression; it's lazy. > I really wish these were called generator comprehensions.? I don't understand why they are not. ??? [ 2*x for x in range(10) ]???? # makes a list ??? ( 2*x for x in range(10) )???? # makes a generator --Ned. From tjol at tjol.eu Mon Jan 15 07:16:04 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 15 Jan 2018 13:16:04 +0100 Subject: Where is the usage of (list comprehension) documented? In-Reply-To: References: Message-ID: On 2018-01-15 00:01, Peng Yu wrote: > Hi, > > I see the following usage of list comprehension can generate a > generator. Does anybody know where this is documented? Thanks. > > $ cat main.py > #!/usr/bin/env python > > import sys > lines = (line.rstrip('\n') for line in sys.stdin) > print lines > > lines = [line.rstrip('\n') for line in sys.stdin] > print lines > $ seq 10 | ./main.py > at 0x1101ecd70> > ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10'] > Generator expressions are of course documented in the language reference: https://docs.python.org/3/reference/expressions.html#generator-expressions The official docs.python.org tutorial also explains them: https://docs.python.org/3/tutorial/classes.html#generator-expressions From pengyu.ut at gmail.com Mon Jan 15 09:11:02 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Mon, 15 Jan 2018 08:11:02 -0600 Subject: Can utf-8 encoded character contain a byte of TAB? Message-ID: Hi, I use the following code to process TSV input. $ printf '%s\t%s\n' {1..10} | ./main.py ['1', '2'] ['3', '4'] ['5', '6'] ['7', '8'] ['9', '10'] $ cat main.py #!/usr/bin/env python # vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 fileencoding=utf-8: import sys for line in sys.stdin: fields=line.rstrip('\n').split('\t') print fields But I am not sure it will process utf-8 input correctly. Thus, I come up with this code. However, I am not sure if this is really necessary as my impression is that utf-8 character should not contain the ascii code for TAB. Is it so? Thanks. $ cat main1.py #!/usr/bin/env python # vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 fileencoding=utf-8: import sys for line in sys.stdin: #fields=line.rstrip('\n').split('\t') fields=line.rstrip('\n').decode('utf-8').split('\t') print [x.encode('utf-8') for x in fields] $ printf '%s\t%s\n' {1..10} | ./main1.py ['1', '2'] ['3', '4'] ['5', '6'] ['7', '8'] ['9', '10'] -- Regards, Peng From __peter__ at web.de Mon Jan 15 09:35:22 2018 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Jan 2018 15:35:22 +0100 Subject: Can utf-8 encoded character contain a byte of TAB? References: Message-ID: Peng Yu wrote: > Can utf-8 encoded character contain a byte of TAB? Yes; ascii is a subset of utf8. Python 2.7.6 (default, Nov 23 2017, 15:49:48) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> ascii = "".join(map(chr, range(128))) >>> uni = ascii.decode("utf-8") >>> len(uni) 128 >>> assert map(ord, uni) == range(128) If you want to allow fields containing TABs in a file where TAB is also the field separator you need a convention to escape the TABs occuring in the values. Nothing I see in your post can cope with that, but the csv module can, by quoting field containing the delimiter: >>> import csv, sys >>> csv.writer(sys.stdout, delimiter="\t").writerow(["foo", "bar\tbaz"]) foo "bar baz" >>> next(csv.reader(['foo\t"bar\tbaz"\n'], delimiter="\t")) ['foo', 'bar\tbaz'] > Hi, > > I use the following code to process TSV input. > > $ printf '%s\t%s\n' {1..10} | ./main.py > ['1', '2'] > ['3', '4'] > ['5', '6'] > ['7', '8'] > ['9', '10'] > $ cat main.py > #!/usr/bin/env python > # vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 > # fileencoding=utf-8: > > import sys > for line in sys.stdin: > fields=line.rstrip('\n').split('\t') > print fields > > But I am not sure it will process utf-8 input correctly. Thus, I come > up with this code. However, I am not sure if this is really necessary > as my impression is that utf-8 character should not contain the ascii > code for TAB. Is it so? Thanks. > > $ cat main1.py > #!/usr/bin/env python > # vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 > # fileencoding=utf-8: > > import sys > for line in sys.stdin: > #fields=line.rstrip('\n').split('\t') > fields=line.rstrip('\n').decode('utf-8').split('\t') > print [x.encode('utf-8') for x in fields] > > $ printf '%s\t%s\n' {1..10} | ./main1.py > ['1', '2'] > ['3', '4'] > ['5', '6'] > ['7', '8'] > ['9', '10'] > > From as at sci.fi Mon Jan 15 09:58:25 2018 From: as at sci.fi (Anssi Saari) Date: Mon, 15 Jan 2018 16:58:25 +0200 Subject: Generating SVG from turtle graphics References: Message-ID: Peter Otten <__peter__ at web.de> writes: > Then convert to SVG with an external tool. It looks like ghostscript can do > that: > > $ gs -dBATCH -dNOPAUSE -sDEVICE=svg -sOutputFile=tmp_turtle.svg tmp_turtle.ps And if not (I at least don't have svg output on three ghostscripts I tried), pstoedit can do it too. From random832 at fastmail.com Mon Jan 15 10:09:41 2018 From: random832 at fastmail.com (Random832) Date: Mon, 15 Jan 2018 10:09:41 -0500 Subject: Can utf-8 encoded character contain a byte of TAB? In-Reply-To: References: Message-ID: <1516028981.3063620.1235899488.4E91462D@webmail.messagingengine.com> On Mon, Jan 15, 2018, at 09:35, Peter Otten wrote: > Peng Yu wrote: > > > Can utf-8 encoded character contain a byte of TAB? > > Yes; ascii is a subset of utf8. > > If you want to allow fields containing TABs in a file where TAB is also the > field separator you need a convention to escape the TABs occuring in the > values. Nothing I see in your post can cope with that, but the csv module > can, by quoting field containing the delimiter: Just to be clear, TAB *only* appears in utf-8 as the encoding for the actual TAB character, not as a part of any other character's encoding. The only bytes that can appear in the utf-8 encoding of non-ascii characters are starting with 0xC2 through 0xF4, followed by one or more of 0x80 through 0xBF. From rgaddi at highlandtechnology.invalid Mon Jan 15 12:33:32 2018 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 15 Jan 2018 09:33:32 -0800 Subject: pip --user by default In-Reply-To: References: Message-ID: On 01/13/2018 04:54 AM, Thomas Jollans wrote: > Hi, > > I recently discovered the wonders of pip.conf: if I create a file > ~/.config/pip/pip.conf* with: > > [install] > user = true > > then pip will install to the --user site-packages by default, rather > than trying to install packages into system directories. > > The trouble is that this fails when you also use virtualenvs. In a > virtualenv, --user doesn't work, so pip fails when trying to install > anything in a virtualenv as long as the user pip.conf contains those lines. > > Short of adding a pip.conf to every single virtualenv, is there any way > to work around this, and configure pip to install packages > > - into the user directories if possible > - into the environment when in an environment > > by default? > > Thanks, > Thomas > > > > * the path obviously depends on the OS: > https://pip.pypa.io/en/stable/user_guide/#config-file > Inside of a virtualenv, what's the difference between a --user install and a system one? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From tjol at tjol.eu Mon Jan 15 13:29:24 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 15 Jan 2018 19:29:24 +0100 Subject: pip --user by default In-Reply-To: References: Message-ID: <3a8f05d2-ac76-2e6a-aea2-336b06aa62f6@tjol.eu> On 2018-01-15 18:33, Rob Gaddi wrote: > > Inside of a virtualenv, what's the difference between a --user install > and a system one? > It errors out: % pip install --user urllib3 Can not perform a '--user' install. User site-packages are not visible in this virtualenv. From gandalf at shopzeus.com Mon Jan 15 13:30:03 2018 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Mon, 15 Jan 2018 19:30:03 +0100 Subject: requests / SSL blocks forever? In-Reply-To: References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> Message-ID: <73194294-4613-e46f-81f0-18461f1d8b86@shopzeus.com> 2018. 01. 13. 15:03 keltez?ssel, Jon Ribbens ?rta: > On 2018-01-13, Nagy L?szl? Zsolt wrote: >> I have a multi threaded Windows service written in Python. It is running >> on 3.6.2.? Sometimes I cannot stop the service, because on of the >> threads won't exit. I have narrowed down the problem to request and >> _lib.SSL_read. > (a) are you setting daemon=True on the thread? > (b) are you setting a timeout on the requests call? The thread is not daemonic, but it can be stopped (usually) within a second by setting a threading.Event. I'm not setting a timeout on the call. Good point. I'm going to try it now. By the way: yes, I have to use a service that is almost not documented, and sometimes it disconnects in the middle of the request, only sending half of the response. I was not thinking about not sending a reponse but keeping the connection alive. But now that you pointed out, it is very well possible. ?? Laszlo From gandalf at shopzeus.com Mon Jan 15 13:34:13 2018 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Mon, 15 Jan 2018 19:34:13 +0100 Subject: requests / SSL blocks forever? In-Reply-To: References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> Message-ID: <3245beca-3d5f-aff9-3c5e-6add2892a486@shopzeus.com> > (a) are you setting daemon=True on the thread? > (b) are you setting a timeout on the requests call? Hmm setting the timeout might not be the solution. This is from the docs of the requests module: > > Note > > |timeout| is not a time limit on the entire response download; rather, > an exception is raised if the server has not issued a response for > |timeout| seconds (more precisely, if no bytes have been received on > the underlying socket for |timeout| seconds). If no timeout is > specified explicitly, requests do not time out. > In other words: if the server starts to send the response, but then stops sending it (without closing the connection), then this will block forever anyway. There seems to be no protection against this scenario, at least not with the requests module. From gandalf at shopzeus.com Mon Jan 15 13:45:14 2018 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Mon, 15 Jan 2018 19:45:14 +0100 Subject: requests / SSL blocks forever? In-Reply-To: <3245beca-3d5f-aff9-3c5e-6add2892a486@shopzeus.com> References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> <3245beca-3d5f-aff9-3c5e-6add2892a486@shopzeus.com> Message-ID: <18d799f9-1189-bb8e-e7ba-2e60676c0bb0@shopzeus.com> > In other words: if the server starts to send the response, but then > stops sending it (without closing the connection), then this will block > forever anyway. Or maybe I misunderstood the docs and the timeout means the max. time elapsed between receiving two chunks of data from the server? From skip.montanaro at gmail.com Mon Jan 15 13:46:42 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 15 Jan 2018 12:46:42 -0600 Subject: pip --user by default In-Reply-To: <3a8f05d2-ac76-2e6a-aea2-336b06aa62f6@tjol.eu> References: <3a8f05d2-ac76-2e6a-aea2-336b06aa62f6@tjol.eu> Message-ID: >> Inside of a virtualenv, what's the difference between a --user install >> and a system one? >> > > It errors out: > > % pip install --user urllib3 > Can not perform a '--user' install. User site-packages are not visible > in this virtualenv. I was able to 'pip install --user ...' a package yesterday in a Conda environment, so I think it's a YMMV sort of thing. Having only ever used the Conda environment stuff, I've generally interpreted the term "virtual environment" in a fairly generic way. Skip From tjol at tjol.eu Mon Jan 15 13:52:08 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 15 Jan 2018 19:52:08 +0100 Subject: pip --user by default In-Reply-To: References: <3a8f05d2-ac76-2e6a-aea2-336b06aa62f6@tjol.eu> Message-ID: <1307976c-ca27-3e5b-1578-4139edd9843d@tjol.eu> On 2018-01-15 19:46, Skip Montanaro wrote: >>> Inside of a virtualenv, what's the difference between a --user install >>> and a system one? >>> >> >> It errors out: >> >> % pip install --user urllib3 >> Can not perform a '--user' install. User site-packages are not visible >> in this virtualenv. > > I was able to 'pip install --user ...' a package yesterday in a Conda > environment, so I think it's a YMMV sort of thing. Having only ever > used the Conda environment stuff, I've generally interpreted the term > "virtual environment" in a fairly generic way. > > Skip > I just tried that and it turns out that that installs the package into the main anaconda/miniconda installation rather than the conda env. Definitely not what you want. -- Thomas From jon+usenet at unequivocal.eu Mon Jan 15 13:55:48 2018 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 15 Jan 2018 18:55:48 -0000 (UTC) Subject: requests / SSL blocks forever? References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> <3245beca-3d5f-aff9-3c5e-6add2892a486@shopzeus.com> <18d799f9-1189-bb8e-e7ba-2e60676c0bb0@shopzeus.com> Message-ID: On 2018-01-15, Nagy L?szl? Zsolt wrote: >> In other words: if the server starts to send the response, but then >> stops sending it (without closing the connection), then this will block >> forever anyway. > Or maybe I misunderstood the docs and the timeout means the max. time > elapsed between receiving two chunks of data from the server? Yes. It's documented better here: http://docs.python-requests.org/en/master/user/advanced/#timeouts You can't specify a "total time" within which the operation must succeed or be abandoned, but if you specify a timeout and the server stops responding, either at the start of the process or in the middle of the response, then it will time out after the specified delay. From skip.montanaro at gmail.com Mon Jan 15 14:30:05 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 15 Jan 2018 13:30:05 -0600 Subject: pip --user by default In-Reply-To: <1307976c-ca27-3e5b-1578-4139edd9843d@tjol.eu> References: <3a8f05d2-ac76-2e6a-aea2-336b06aa62f6@tjol.eu> <1307976c-ca27-3e5b-1578-4139edd9843d@tjol.eu> Message-ID: > I just tried that and it turns out that that installs the package into > the main anaconda/miniconda installation rather than the conda env. > Definitely not what you want. Yes, installing in the root environment (as it's called) is generally a bad idea. I use Conda at work, in a shared sort of environment. It took our smart people (I'm not one) awhile along with some updates from the Anaconda folks to get the whole enterprise-wide Conda thing to work reasonably well. I do one key thing to prevent myself from being the cause of problems. I never run with the root environment's bin directory in PATH. Caveat... I write this with my Linux/Unix hat on. I don't own a Windows hat. Again, YMMV. Skip From python at mrabarnett.plus.com Mon Jan 15 15:24:54 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 15 Jan 2018 20:24:54 +0000 Subject: Detecting a cycle in a graph In-Reply-To: References: Message-ID: <1f67363c-4d2a-f5ac-7fa8-b6690ddbaf66@mrabarnett.plus.com> On 2018-01-15 06:15, Frank Millman wrote: > "Christian Gollwitzer" wrote in message news:p3gh84$kfm$1 at dont-email.me... >> >> Am 14.01.18 um 22:04 schrieb Christian Gollwitzer: >> > Am 14.01.18 um 09:30 schrieb Frank Millman: >> >> I need to detect when a 'cycle' occurs - when a path loops back on >> >> itself and revisits a node previously visited. I have figured out a way >> >> to do this, but I have a problem. >> > >> > I don't know if that helps, but there is a classic graph theory >> > algorithm called "Floyd's cycle detector". The idea is to have a pointer >> > move along the graph and a second one which runs at double the speed. If >> > they meet, you found a cycle. It is not straight-forward to come up with >> > this algorithm, which even runs in constant storage. ISTR that there are >> > some variants which can give you the split point of the cycle, too. >> >> And here is an algorithm to enumerate all cycles in a directed graph: >> >> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.5999&rep=rep1&type=pdf >> >> with an implementation in C++ here: >> >> https://github.com/hellogcc/circuit-finding-algorithm >> > > I appreciate the input, Christian, but I am afraid both of those were over > my head :-( > > I think/hope that a business process graph does not require such a complex > solution. Here is my cycle-detecting algorithm. > > In BPMN terms, each node can have 0->* incoming connections, and 0->* > outgoing connections. > > Any node with 0 incoming is deemed to start the process. Normally there is > only one such node. > > Any node with 0 outgoing represents the end of an 'active path'. If there is > more than one such node, all 'active' ones must reach the end before the > process is finished. There is no formal definition of an 'active path', and > I can think of a few corner cases which could prove problematic, but > normally the meaning is clear. > > I start my cycle-detection with a node with 0 incoming connections. > > def find_cycle(node, path): > for output in node.outputs: > if output in path: > print('Cycle found in', path) > else: > new_path = path[:] > new_path.append(output) > find_cycle(output, new_path) > > find_cycle(node, [node]) > > This traverses every possible path in the graph. I think/hope that a typical > business process will not grow so large as to cause a problem. > > If anyone can see a flaw in this, please let me know. > A couple of suggestions: 1. Instead of copying the list and appending, I'd do: find_cycle(output, path + [output]) 2. Lists are ordered, but searching them is O(n), so I'd pass a set too to speed it up a bit: def find_cycle(node, path, visited): for output in node.outputs: if output in visited: print('Cycle found in', path) else: find_cycle(output, path + [output], visited | {output}) That will help as the paths get longer, although on a small graph, it won't matter. From pengyu.ut at gmail.com Mon Jan 15 16:29:36 2018 From: pengyu.ut at gmail.com (Peng Yu) Date: Mon, 15 Jan 2018 15:29:36 -0600 Subject: Can utf-8 encoded character contain a byte of TAB? In-Reply-To: <1516028981.3063620.1235899488.4E91462D@webmail.messagingengine.com> References: <1516028981.3063620.1235899488.4E91462D@webmail.messagingengine.com> Message-ID: > Just to be clear, TAB *only* appears in utf-8 as the encoding for the actual TAB character, not as a part of any other character's encoding. The only bytes that can appear in the utf-8 encoding of non-ascii characters are starting with 0xC2 through 0xF4, followed by one or more of 0x80 through 0xBF. So for utf-8 encoded input, I only need to use this code to split each line into fields? import sys for line in sys.stdin: fields=line.rstrip('\n').split('\t') print fields Is there a need to use this code to split each line into fields? import sys for line in sys.stdin: fields=line.rstrip('\n').decode('utf-8').split('\t') print [x.encode('utf-8') for x in fields] -- Regards, Peng From rosuav at gmail.com Mon Jan 15 16:41:55 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Jan 2018 08:41:55 +1100 Subject: Can utf-8 encoded character contain a byte of TAB? In-Reply-To: References: <1516028981.3063620.1235899488.4E91462D@webmail.messagingengine.com> Message-ID: On Tue, Jan 16, 2018 at 8:29 AM, Peng Yu wrote: >> Just to be clear, TAB *only* appears in utf-8 as the encoding for the actual TAB character, not as a part of any other character's encoding. The only bytes that can appear in the utf-8 encoding of non-ascii characters are starting with 0xC2 through 0xF4, followed by one or more of 0x80 through 0xBF. > > So for utf-8 encoded input, I only need to use this code to split each > line into fields? > > import sys > for line in sys.stdin: > fields=line.rstrip('\n').split('\t') > print fields > > Is there a need to use this code to split each line into fields? > > import sys > for line in sys.stdin: > fields=line.rstrip('\n').decode('utf-8').split('\t') > print [x.encode('utf-8') for x in fields] > One of the deliberate design features of UTF-8 is that the ASCII byte values (those below 128) are *exclusively* used for ASCII characters. Characters >=128 are encoded using multiple bytes in the 128-255 range. But what you should ideally do is decode everything as UTF-8, then manipulate it as text. That's the default way to do things in Py3 anyway. The reason for this is that it's entirely possible for an arbitrary byte stream to NOT follow the rules of UTF-8, which could break your code. The way to be confident is to do the decode, and if it fails, reject the input. ChrisA From frank at chagford.com Tue Jan 16 00:00:49 2018 From: frank at chagford.com (Frank Millman) Date: Tue, 16 Jan 2018 07:00:49 +0200 Subject: Detecting a cycle in a graph In-Reply-To: <1f67363c-4d2a-f5ac-7fa8-b6690ddbaf66@mrabarnett.plus.com> References: <1f67363c-4d2a-f5ac-7fa8-b6690ddbaf66@mrabarnett.plus.com> Message-ID: "MRAB" wrote in message news:1f67363c-4d2a-f5ac-7fa8-b6690ddbaf66 at mrabarnett.plus.com... > On 2018-01-15 06:15, Frank Millman wrote: > > > I start my cycle-detection with a node with 0 incoming connections. > > > > def find_cycle(node, path): > > for output in node.outputs: > > if output in path: > > print('Cycle found in', path) > > else: > > new_path = path[:] > > new_path.append(output) > > find_cycle(output, new_path) > > > > find_cycle(node, [node]) > > > > This traverses every possible path in the graph. I think/hope that a > > typical > > business process will not grow so large as to cause a problem. > > > > If anyone can see a flaw in this, please let me know. > > > A couple of suggestions: > > 1. Instead of copying the list and appending, I'd do: > > find_cycle(output, path + [output]) > > 2. Lists are ordered, but searching them is O(n), so I'd pass a set too to > speed it up a bit: > > def find_cycle(node, path, visited): > for output in node.outputs: > if output in visited: > print('Cycle found in', path) > else: > find_cycle(output, path + [output], visited | {output}) > > That will help as the paths get longer, although on a small graph, it > won't matter. Both suggestions are much appreciated - so simple, and yet they improve my code enormously. I have never seen the use of '|' to update a set before, though now I check the docs, it is there. It is very neat. Many thanks Frank From arj.python at gmail.com Tue Jan 16 01:57:27 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 16 Jan 2018 10:57:27 +0400 Subject: pip --user by default In-Reply-To: References: Message-ID: another thing which amazed me with pip is that you can write library1 == 1.2.7 library2 == 3.6.1 in requirements.txt and pip install requirements.txt will install those libs On 13 Jan 2018 17:16, "Thomas Jollans" wrote: > Hi, > > I recently discovered the wonders of pip.conf: if I create a file > ~/.config/pip/pip.conf* with: > > [install] > user = true > > then pip will install to the --user site-packages by default, rather > than trying to install packages into system directories. > > The trouble is that this fails when you also use virtualenvs. In a > virtualenv, --user doesn't work, so pip fails when trying to install > anything in a virtualenv as long as the user pip.conf contains those lines. > > Short of adding a pip.conf to every single virtualenv, is there any way > to work around this, and configure pip to install packages > > - into the user directories if possible > - into the environment when in an environment > > by default? > > Thanks, > Thomas > > > > * the path obviously depends on the OS: > https://pip.pypa.io/en/stable/user_guide/#config-file > -- > https://mail.python.org/mailman/listinfo/python-list > From arj.python at gmail.com Tue Jan 16 01:58:43 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 16 Jan 2018 10:58:43 +0400 Subject: Generating SVG from turtle graphics In-Reply-To: References: Message-ID: https://image.online-convert.com/convert-to-svg On 15 Jan 2018 02:55, "Niles Rogoff" wrote: > On Sun, 14 Jan 2018 16:32:53 +0400, Abdur-Rahmaan Janhangeer wrote: > > > maybe save to .png then use another tool to svg > > PNG is a bitmap format[1], so you can't covert it to an SVG (a vector > format) without guessing things like the start/end points of the lines, > their slopes, etc... not to mention things like arcs. > > [1] RFC2083 > -- > https://mail.python.org/mailman/listinfo/python-list > From rustompmody at gmail.com Tue Jan 16 06:39:48 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 16 Jan 2018 03:39:48 -0800 (PST) Subject: Pandas printing in jupyter In-Reply-To: <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> <130ddd25-f45a-4f23-a333-e5d1bb0b204a@googlegroups.com> <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> Message-ID: On Sunday, January 14, 2018 at 3:28:02 AM UTC+5:30, bo... at questa.la.so wrote: > Rustom Mody writes: > > > Specifically and for starters, I want a numpy array ? lets say 2D to > > start with ? to be displayed(displayable) as elegantly as sympy does > > to (its) matrices > ######################################################################## > import numpy as np > from IPython.display import Latex > > def prmat(mat): > return (r'\begin{bmatrix}' + > r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + > r'\end{bmatrix}' > > a = np.arange(12).reshape((3, 4))+1 > display(Latex(prmat(a))) > ######################################################################## > you could add optional arguments to modify the type of brackets and the > formatting string Thanks Well I had to tiny-tweak the code (import the display) --------------- import numpy as np from IPython.display import Latex, display def prmat(mat): return (r'\begin{bmatrix}' + r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + r'\end{bmatrix}' ) a = np.arange(12).reshape((3, 4))+1 display(Latex(prmat(a))) --------------- After that it works? for 5 seconds!! ie it shows a nice centered matrix like a math-display in latex Then it goes away and I see a left aligned bunch of unicode boxes! From rustompmody at gmail.com Tue Jan 16 07:33:39 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 16 Jan 2018 04:33:39 -0800 (PST) Subject: Pandas printing in jupyter In-Reply-To: References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> <130ddd25-f45a-4f23-a333-e5d1bb0b204a@googlegroups.com> <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> Message-ID: On Tuesday, January 16, 2018 at 5:10:14 PM UTC+5:30, Rustom Mody wrote: > On Sunday, January 14, 2018 at 3:28:02 AM UTC+5:30, bo... at questa.la.so wrote: > > Rustom Mody writes: > > > > > Specifically and for starters, I want a numpy array ? lets say 2D to > > > start with ? to be displayed(displayable) as elegantly as sympy does > > > to (its) matrices > > ######################################################################## > > import numpy as np > > from IPython.display import Latex > > > > def prmat(mat): > > return (r'\begin{bmatrix}' + > > r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + > > r'\end{bmatrix}' > > > > a = np.arange(12).reshape((3, 4))+1 > > display(Latex(prmat(a))) > > ######################################################################## > > you could add optional arguments to modify the type of brackets and the > > formatting string > > Thanks > > Well I had to tiny-tweak the code (import the display) > --------------- > > import numpy as np > from IPython.display import Latex, display > > def prmat(mat): > return (r'\begin{bmatrix}' + > r'\\'.join('&'.join('%f'%x for x in row) for row in mat) + > r'\end{bmatrix}' ) > > a = np.arange(12).reshape((3, 4))+1 > display(Latex(prmat(a))) > --------------- > > After that it works? for 5 seconds!! > > ie it shows a nice centered matrix like a math-display in latex > Then it goes away and I see a left aligned bunch of unicode boxes! Inspired by this I tried this It works? kinda? but the matrix columns dont align def prmat(mat): return (r'\begin{bmatrix}' + r'\\'.join('&'.join('%d'%x for x in row) for row in mat) + r'\end{bmatrix}' ) matprefix = """ [ """ rowprefix = """ """ elemfmt = """ %d """ rowsuffix = """ """ matsuffix = """ ] """ def prmht(mat): return (matprefix + "".join(prmhtrow(r) for r in mat) + matsuffix) def prmhtrow(row): return rowprefix + "".join(elemfmt%x for x in row) + rowsuffix display(HTML(prmht(a))) From rustompmody at gmail.com Tue Jan 16 09:03:06 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 16 Jan 2018 06:03:06 -0800 (PST) Subject: Pandas printing in jupyter In-Reply-To: References: <46cb0b0e-6996-4245-b544-cd7fefc1b31e@googlegroups.com> <130ddd25-f45a-4f23-a333-e5d1bb0b204a@googlegroups.com> <87d12dbewg.fsf@debian.i-did-not-set--mail-host-address--so-tickle-me> Message-ID: <629dfa2e-be26-4b79-827d-6575451fc6e5@googlegroups.com> On Tuesday, January 16, 2018 at 6:04:06 PM UTC+5:30, Rustom Mody wrote: Had missed the mtd element ie changing elemfmt = """ %d """ to elemfmt = """ %d """ From skip.montanaro at gmail.com Tue Jan 16 11:00:16 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 16 Jan 2018 10:00:16 -0600 Subject: Right way to io.open(...) an existing file object in Python 2.7? Message-ID: I'd like to take advantage of the seekable() method of io.IOBase with existing open file objects, especially the standard in/out/err file objects. I know on Linux I can just do this: fp = io.open("/dev/stderr") but that's kind of cheating. File objects in Python 2.7 aren't created using the io module's machinery. Assume somewhere else a plain old open file object, for example: fp = open("/some/path/to/nowhere.txt", "w") and wanted to treat fp as if it had been opened with the io module machinery, I don't see any sort of "fdopen" or "freopen" equivalent mentioned in the io module documentation. Is this possible in a clean way? Thx, Skip From gandalf at shopzeus.com Tue Jan 16 11:11:33 2018 From: gandalf at shopzeus.com (=?UTF-8?Q?Nagy_L=c3=a1szl=c3=b3_Zsolt?=) Date: Tue, 16 Jan 2018 17:11:33 +0100 Subject: requests / SSL blocks forever? In-Reply-To: References: <89e38df7-0159-b44b-951c-a94adb2f90b1@shopzeus.com> <3245beca-3d5f-aff9-3c5e-6add2892a486@shopzeus.com> <18d799f9-1189-bb8e-e7ba-2e60676c0bb0@shopzeus.com> Message-ID: <1fa76715-61ba-74d9-76d5-d58248777670@shopzeus.com> >> Or maybe I misunderstood the docs and the timeout means the max. time >> elapsed between receiving two chunks of data from the server? > Yes. It's documented better here: > http://docs.python-requests.org/en/master/user/advanced/#timeouts > > You can't specify a "total time" within which the operation must > succeed or be abandoned, but if you specify a timeout and the > server stops responding, either at the start of the process or > in the middle of the response, then it will time out after the > specified delay. Very good, thank you! Setting the timeout solved the problem for now. :-) From lele at metapensiero.it Tue Jan 16 11:59:00 2018 From: lele at metapensiero.it (Lele Gaifax) Date: Tue, 16 Jan 2018 17:59:00 +0100 Subject: Right way to io.open(...) an existing file object in Python 2.7? References: Message-ID: <871sipkaej.fsf@metapensiero.it> Skip Montanaro writes: > I don't see any sort of "fdopen" or "freopen" equivalent mentioned in the io > module documentation. Is this possible in a clean way? > There is an os.fdopen(), so maybe newf = os.fdopen(fp.fileno()) ciao, lele. -- nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia. lele at metapensiero.it | -- Fortunato Depero, 1929. From larry.martell at gmail.com Tue Jan 16 12:00:09 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 16 Jan 2018 12:00:09 -0500 Subject: documentation on read.encode Message-ID: Looking for 2.7 docs on read.encode - googling did not turn up anything. Specifically, looking for the supported options for base64, and how to specify them, e.g. Base64.NO_WRAP From jon+usenet at unequivocal.eu Tue Jan 16 12:42:49 2018 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 16 Jan 2018 17:42:49 -0000 (UTC) Subject: Right way to io.open(...) an existing file object in Python 2.7? References: Message-ID: On 2018-01-16, Skip Montanaro wrote: > I'd like to take advantage of the seekable() method of io.IOBase with > existing open file objects, especially the standard in/out/err file > objects. If it's difficult to imagine a circumstance in which you would want to seek on stdio/out/err where you were not making some considerable error. But they are already io objects so you can just call sys.stdin.seekable or whatever. From skip.montanaro at gmail.com Tue Jan 16 13:02:30 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 16 Jan 2018 12:02:30 -0600 Subject: Right way to io.open(...) an existing file object in Python 2.7? In-Reply-To: <871sipkaej.fsf@metapensiero.it> References: <871sipkaej.fsf@metapensiero.it> Message-ID: >> I don't see any sort of "fdopen" or "freopen" equivalent mentioned in the io >> module documentation. Is this possible in a clean way? >> > > There is an os.fdopen(), so maybe > > newf = os.fdopen(fp.fileno()) Correct, but that just returns another File object which will still lack the API provided by io.IOBase. I'm talking Python 2.7 here. In Python 3.x, all I/O is mediated by that API. Skip From skip.montanaro at gmail.com Tue Jan 16 13:08:29 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 16 Jan 2018 12:08:29 -0600 Subject: Right way to io.open(...) an existing file object in Python 2.7? In-Reply-To: References: Message-ID: > If it's difficult to imagine a circumstance in which you would want to > seek on stdio/out/err where you were not making some considerable error. > But they are already io objects so you can just call sys.stdin.seekable > or whatever. Alas, in Python 2.7 sys.stdin/stdout/stderr are traditional (legacy?) file objects created using the open() built-in function. I was hoping there would be a clean way to transmogrify such objects into objects which support the io.IOBase APIs. I already know that I can attempt to seek(), then catch the IOError, but I was hoping to start using the more modern io module's seekable() API in a specific module I maintain without having to worry about what type of file-ish object I was passed. Data flow might look like this: File or io.IOBase object -> passed into my code -> transmogrified into io.IOBase object at the boundary, if necessary -> resulting in io nirvana in the internals of my module. :-) Skip From larry.martell at gmail.com Tue Jan 16 14:19:38 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 16 Jan 2018 14:19:38 -0500 Subject: documentation on read.encode In-Reply-To: References: Message-ID: On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell wrote: > Looking for 2.7 docs on read.encode - googling did not turn up anything. > > Specifically, looking for the supported options for base64, and how to > specify them, e.g. Base64.NO_WRAP So I just realized that encode() is not a method of read() it's a string method. But I still have the same question - can I pass in any flags? My issue is that I am base64 encoding PNG images on linux and it's putting a LF at the end of each line. If I do the same on Windows it's putting CR/LF. I want the files to be encoded with no platform dependences. Googling I found mention of Base64.NO_WRAP and I want to pass that into encode() - can I do that? From gheskett at shentel.net Tue Jan 16 14:35:10 2018 From: gheskett at shentel.net (Gene Heskett) Date: Tue, 16 Jan 2018 14:35:10 -0500 Subject: documentation on read.encode In-Reply-To: References: Message-ID: <201801161435.10062.gheskett@shentel.net> On Tuesday 16 January 2018 14:19:38 Larry Martell wrote: > On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell wrote: > > Looking for 2.7 docs on read.encode - googling did not turn up > > anything. > > > > Specifically, looking for the supported options for base64, and how > > to specify them, e.g. Base64.NO_WRAP > > So I just realized that encode() is not a method of read() it's a > string method. But I still have the same question - can I pass in any > flags? > > My issue is that I am base64 encoding PNG images on linux and it's > putting a LF at the end of each line. If I do the same on Windows it's > putting CR/LF. I want the files to be encoded with no platform > dependences. Googling I found mention of Base64.NO_WRAP and I want to > pass that into encode() - can I do that? Di you not have the manpages installed? In my copy of the manpage: base64 [OPTION]... [FILE] where option is: -w, --wrap=COLS wrap encoded lines after COLS character (default 76). Use 0 to disable line wrapping. Seems pretty simple. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From eryksun at gmail.com Tue Jan 16 14:40:23 2018 From: eryksun at gmail.com (eryk sun) Date: Tue, 16 Jan 2018 19:40:23 +0000 Subject: Right way to io.open(...) an existing file object in Python 2.7? In-Reply-To: References: Message-ID: On Tue, Jan 16, 2018 at 4:00 PM, Skip Montanaro wrote: > I'd like to take advantage of the seekable() method of io.IOBase with > existing open file objects, especially the standard in/out/err file > objects. io.open can open a file descriptor. If you don't use a duplicated FD (os.dup), then you probably also want the option `closefd=False`. For example: $ cat test.py import sys import io stdin = io.open(sys.stdin.fileno(), closefd=False) print 'stdin seekable: %s' % stdin.seekable() $ python test.py stdin seekable: False $ echo spam | python test.py stdin seekable: False $ python test.py < test.py stdin seekable: True From larry.martell at gmail.com Tue Jan 16 14:52:21 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 16 Jan 2018 14:52:21 -0500 Subject: documentation on read.encode In-Reply-To: <201801161435.10062.gheskett@shentel.net> References: <201801161435.10062.gheskett@shentel.net> Message-ID: On Tue, Jan 16, 2018 at 2:35 PM, Gene Heskett wrote: > On Tuesday 16 January 2018 14:19:38 Larry Martell wrote: > >> On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell > wrote: >> > Looking for 2.7 docs on read.encode - googling did not turn up >> > anything. >> > >> > Specifically, looking for the supported options for base64, and how >> > to specify them, e.g. Base64.NO_WRAP >> >> So I just realized that encode() is not a method of read() it's a >> string method. But I still have the same question - can I pass in any >> flags? >> >> My issue is that I am base64 encoding PNG images on linux and it's >> putting a LF at the end of each line. If I do the same on Windows it's >> putting CR/LF. I want the files to be encoded with no platform >> dependences. Googling I found mention of Base64.NO_WRAP and I want to >> pass that into encode() - can I do that? > > Di you not have the manpages installed? > > In my copy of the manpage: > base64 [OPTION]... [FILE] > where option is: > -w, --wrap=COLS > wrap encoded lines after COLS character (default 76). Use > 0 to disable line wrapping. > > Seems pretty simple. But how do I use that in read().encode('base64')? From ned at nedbatchelder.com Tue Jan 16 15:17:59 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 16 Jan 2018 15:17:59 -0500 Subject: documentation on read.encode In-Reply-To: References: Message-ID: On 1/16/18 2:19 PM, Larry Martell wrote: > On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell wrote: >> Looking for 2.7 docs on read.encode - googling did not turn up anything. >> >> Specifically, looking for the supported options for base64, and how to >> specify them, e.g. Base64.NO_WRAP > So I just realized that encode() is not a method of read() it's a > string method. But I still have the same question - can I pass in any > flags? > > My issue is that I am base64 encoding PNG images on linux and it's > putting a LF at the end of each line. If I do the same on Windows it's > putting CR/LF. I want the files to be encoded with no platform > dependences. Googling I found mention of Base64.NO_WRAP and I want to > pass that into encode() - can I do that? Base64.NO_WRAP seems to be a Java thing. --Ned. From larry.martell at gmail.com Tue Jan 16 15:21:59 2018 From: larry.martell at gmail.com (Larry Martell) Date: Tue, 16 Jan 2018 15:21:59 -0500 Subject: documentation on read.encode In-Reply-To: References: Message-ID: On Tue, Jan 16, 2018 at 3:17 PM, Ned Batchelder wrote: > On 1/16/18 2:19 PM, Larry Martell wrote: >> >> On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell >> wrote: >>> >>> Looking for 2.7 docs on read.encode - googling did not turn up anything. >>> >>> Specifically, looking for the supported options for base64, and how to >>> specify them, e.g. Base64.NO_WRAP >> >> So I just realized that encode() is not a method of read() it's a >> string method. But I still have the same question - can I pass in any >> flags? >> >> My issue is that I am base64 encoding PNG images on linux and it's >> putting a LF at the end of each line. If I do the same on Windows it's >> putting CR/LF. I want the files to be encoded with no platform >> dependences. Googling I found mention of Base64.NO_WRAP and I want to >> pass that into encode() - can I do that? > > > Base64.NO_WRAP seems to be a Java thing. Yeah I saw it mentioned in a SO post that was java related. Wondering if there is some way to do the same in python. From jon+usenet at unequivocal.eu Tue Jan 16 15:45:51 2018 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Tue, 16 Jan 2018 20:45:51 -0000 (UTC) Subject: documentation on read.encode References: Message-ID: On 2018-01-16, Larry Martell wrote: > Yeah I saw it mentioned in a SO post that was java related. Wondering > if there is some way to do the same in python. base64.b64encode(foo) From python at mrabarnett.plus.com Tue Jan 16 15:58:04 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 16 Jan 2018 20:58:04 +0000 Subject: documentation on read.encode In-Reply-To: References: <201801161435.10062.gheskett@shentel.net> Message-ID: <6a4e6e00-f388-316c-c089-f53ee4b148cb@mrabarnett.plus.com> On 2018-01-16 19:52, Larry Martell wrote: > On Tue, Jan 16, 2018 at 2:35 PM, Gene Heskett wrote: >> On Tuesday 16 January 2018 14:19:38 Larry Martell wrote: >> >>> On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell >> wrote: >>> > Looking for 2.7 docs on read.encode - googling did not turn up >>> > anything. >>> > >>> > Specifically, looking for the supported options for base64, and how >>> > to specify them, e.g. Base64.NO_WRAP >>> >>> So I just realized that encode() is not a method of read() it's a >>> string method. But I still have the same question - can I pass in any >>> flags? >>> >>> My issue is that I am base64 encoding PNG images on linux and it's >>> putting a LF at the end of each line. If I do the same on Windows it's >>> putting CR/LF. I want the files to be encoded with no platform >>> dependences. Googling I found mention of Base64.NO_WRAP and I want to >>> pass that into encode() - can I do that? >> >> Di you not have the manpages installed? >> >> In my copy of the manpage: >> base64 [OPTION]... [FILE] >> where option is: >> -w, --wrap=COLS >> wrap encoded lines after COLS character (default 76). Use >> 0 to disable line wrapping. >> >> Seems pretty simple. > > But how do I use that in read().encode('base64')? > Use the base64 module instead, which is also how you would do it in Python 3. If you're getting CR/LF on Windows, that's because you're opening the file in text mode. In both Python 2 and Python 3 the base64 string will be a bytestring, which you'd write out to a file opened in binary mode. That's an extra bit of future-proofing! :-) --- This email has been checked for viruses by AVG. http://www.avg.com From h.goebel at crazy-compilers.com Tue Jan 16 17:40:17 2018 From: h.goebel at crazy-compilers.com (Hartmut Goebel) Date: Tue, 16 Jan 2018 23:40:17 +0100 Subject: [ANN] python-ghostscript 0.6 Message-ID: <45d9ca3d-d24e-b17e-71e4-cc4945f2b1db@crazy-compilers.com> I'm pleased to announce the new release vor python-ghostscript: ???????????????? python-ghostscript 0.6 A Python-Interface to the Ghostscript C-API using ctypes :License: GNU Public License v3 (GPLv3) :Author:? Hartmut Goebel :Homepage: https://gitlab.com/pdftools/python-ghostscript :Download: https://pypi.python.org/pypi/ghostscript `Ghostscript`__, is a well known interpreter for the PostScript language and for PDF. This package implements a interface to the Ghostscript C-API using `ctypes`__. Both a low-level and a pythonic, high-level interface are provided. __ http://www.ghostscript.com/ __ http://docs.python.org/library/ctypes.html This package is currently tested only under GNU/Linux. Please report whether it works in your environment, too. Thanks. Latest Changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Please note: Version 0.5 has been skipped to avoid version conflicts with the fork `python3-ghostscript`. All changes of that fork are integrated into this release. ? * Add support for Python 3.x (tested with Python 3.4). ??? Minimum required Python version is now 2.7. ? * Add support for display callback and an example program ??? using it. Thanks to Lasse Fister. ? * Add context-interface (for the ``with``-statement). ? * ``GhostscriptError`` now has an attribute ``code`` holding ??? the numeric error-code. ? * ``Ghostscript()`` now accepts keyword-arguments ``stdin``, ??? ``stdout``, ``stderr`` for setting the respective stream. This was ??? already implementd in version 0.4.1, but not documented. ? * Add unittest suite (using pytest). ? * Several bug fixes and smaller changes. ? * Switch version control to git and move project to gitlab. See ??? Readme-file for the new URL. ? * Set up continuous integration tests. Example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is an example for how to use the high-level interface of `python-ghostscript`. This implements a very basic ps2pdf-tool:: ? import sys ? import ghostscript ? args = [ ????? "ps2pdf",??? # actual value doesn't matter ????? "-dNOPAUSE", "-dBATCH", "-dSAFER", ????? "-sDEVICE=pdfwrite", ????? "-sOutputFile=" + sys.argv[1], ????? "-c", ".setpdfwrite", ????? "-f",? sys.argv[2] ????? ] ? ghostscript.Ghostscript(*args) -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel at crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From skip.montanaro at gmail.com Tue Jan 16 19:48:46 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Tue, 16 Jan 2018 18:48:46 -0600 Subject: Right way to io.open(...) an existing file object in Python 2.7? In-Reply-To: References: Message-ID: io.open can open a file descriptor. Ah, thanks. I'd missed that in my initial reading of the io.open documentation. Skip From bhattacharya.kushal4 at gmail.com Wed Jan 17 06:03:15 2018 From: bhattacharya.kushal4 at gmail.com (bhattacharya.kushal4 at gmail.com) Date: Wed, 17 Jan 2018 03:03:15 -0800 (PST) Subject: c code generator from python Message-ID: <24315cbc-86ff-4a0d-a0d0-017b658a3373@googlegroups.com> Hi, Is there any python framework or any tool as which can generate C code from python code as it is . Thanks, Kushal From bhattacharya.kushal4 at gmail.com Wed Jan 17 06:04:00 2018 From: bhattacharya.kushal4 at gmail.com (kushal bhattacharya) Date: Wed, 17 Jan 2018 03:04:00 -0800 (PST) Subject: python to C code generator Message-ID: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Hi, Is there any python framework or any tool as which can generate C code from python code as it is . Thanks, Kushal From dpalao.python at gmail.com Wed Jan 17 06:41:17 2018 From: dpalao.python at gmail.com (David Palao) Date: Wed, 17 Jan 2018 12:41:17 +0100 Subject: python to C code generator In-Reply-To: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: Hi, Have a look at Cython. Best 2018-01-17 12:04 GMT+01:00 kushal bhattacharya : > Hi, > Is there any python framework or any tool as which can generate C code from python code as it is . > > Thanks, > Kushal > -- > https://mail.python.org/mailman/listinfo/python-list From bc at freeuk.com Wed Jan 17 07:37:08 2018 From: bc at freeuk.com (bartc) Date: Wed, 17 Jan 2018 12:37:08 +0000 Subject: python to C code generator In-Reply-To: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On 17/01/2018 11:04, kushal bhattacharya wrote: > Hi, > Is there any python framework or any tool as which can generate C code from python code as it is . What C code would you expect to see from this line of Python: a = b + c ? From leutrim.kaleci at gmail.com Wed Jan 17 09:29:45 2018 From: leutrim.kaleci at gmail.com (leutrim.kaleci at gmail.com) Date: Wed, 17 Jan 2018 06:29:45 -0800 (PST) Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python Message-ID: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> Hello everyone, I am implementing a time-dependent Recommender System which applies BPR (Bayesian Personalized Ranking), where Stochastic Gradient Ascent is used to learn the parameters of the model. Such that, one iteration involves sampling randomly the quadruple (i.e. userID, positive_item, negative_item, epoch_index) for n times, where n is the total number of positive feedbacks (i.e. the number of ratings given to all items). But, as my implementation takes too much time for learning the parameters (since it requires 100 iterations to learn the parameters), I was wondering if there is a way to improve my code and speed up the learning process of the parameters. Please find the code of my implementation (the update function named as updateFactors is the one that learns the parameters, and such that I guess is the one that should be improved in order to speed up the process of learning the parameter values) in the following link: https://codereview.stackexchange.com/questions/183707/speeding-up-the-implementation-of-stochastic-gradient-ascent-in-python From breamoreboy at gmail.com Wed Jan 17 13:57:40 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 17 Jan 2018 10:57:40 -0800 (PST) Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> Message-ID: <4d6dcc3d-3cc3-491e-85e0-72f9837378a1@googlegroups.com> On Wednesday, January 17, 2018 at 2:30:13 PM UTC, Leo wrote: > Hello everyone, > > I am implementing a time-dependent Recommender System which applies BPR (Bayesian Personalized Ranking), where Stochastic Gradient Ascent is used to learn the parameters of the model. Such that, one iteration involves sampling randomly the quadruple (i.e. userID, positive_item, negative_item, epoch_index) for n times, where n is the total number of positive feedbacks (i.e. the number of ratings given to all items). But, as my implementation takes too much time for learning the parameters (since it requires 100 iterations to learn the parameters), I was wondering if there is a way to improve my code and speed up the learning process of the parameters. > > Please find the code of my implementation (the update function named as updateFactors is the one that learns the parameters, and such that I guess is the one that should be improved in order to speed up the process of learning the parameter values) in the following link: > https://codereview.stackexchange.com/questions/183707/speeding-up-the-implementation-of-stochastic-gradient-ascent-in-python You have a fair chunk of code so I've only taken a quick glance so this could just be the tip of the iceberg. You're clearly using Python 2 as you have print statements, not functions. If you can upgrade to Python 3 is it is superior, see e.g. https://www.youtube.com/watch?v=f_6vDi7ywuA for i in range(1, len(self.userItems[userID])): is a code smell, consider using for userItem in self.userItems[userID]: if len(pos_per_user[userID]) == 0: is written if pos_per_user[userID]: if userID == None: should be if userID is None: Profile your code with https://docs.python.org/3/library/profile.html to find where the hot spots are. Read https://wiki.python.org/moin/PythonSpeed/PerformanceTips for tips on how to up your code performance. -- Kindest regards. Mark Lawrence. From ned at nedbatchelder.com Wed Jan 17 14:28:02 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 17 Jan 2018 14:28:02 -0500 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> Message-ID: <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> On 1/17/18 9:29 AM, leutrim.kaleci at gmail.com wrote: > Hello everyone, > > I am implementing a time-dependent Recommender System which applies BPR (Bayesian Personalized Ranking), where Stochastic Gradient Ascent is used to learn the parameters of the model. Such that, one iteration involves sampling randomly the quadruple (i.e. userID, positive_item, negative_item, epoch_index) for n times, where n is the total number of positive feedbacks (i.e. the number of ratings given to all items). But, as my implementation takes too much time for learning the parameters (since it requires 100 iterations to learn the parameters), I was wondering if there is a way to improve my code and speed up the learning process of the parameters. > > Please find the code of my implementation (the update function named as updateFactors is the one that learns the parameters, and such that I guess is the one that should be improved in order to speed up the process of learning the parameter values) in the following link: > https://codereview.stackexchange.com/questions/183707/speeding-up-the-implementation-of-stochastic-gradient-ascent-in-python It looks like you have lists that you are doing "in" and "remove" operations on.? I don't know how long these lists are, but sets will be much faster in general.? You'll have to replace random.choice() with random.choice(list(...)), since you can't random.choice from a set.? That might negate the benefits, but it's worth a try. --Ned. From rosuav at gmail.com Wed Jan 17 14:45:48 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Jan 2018 06:45:48 +1100 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> Message-ID: On Thu, Jan 18, 2018 at 6:28 AM, Ned Batchelder wrote: > You'll have to replace random.choice() with > random.choice(list(...)), since you can't random.choice from a set. > Side point: why can't you? You can random.sample from a set, but random.choice requires a sequence. It seems perfectly sane to ask for a random element from a set. ChrisA From ned at nedbatchelder.com Wed Jan 17 15:19:04 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 17 Jan 2018 15:19:04 -0500 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> Message-ID: <38405213-0c8b-cde2-39e1-59c296e1b5c0@nedbatchelder.com> On 1/17/18 2:45 PM, Chris Angelico wrote: > On Thu, Jan 18, 2018 at 6:28 AM, Ned Batchelder wrote: >> You'll have to replace random.choice() with >> random.choice(list(...)), since you can't random.choice from a set. >> > Side point: why can't you? You can random.sample from a set, but > random.choice requires a sequence. It seems perfectly sane to ask for > a random element from a set. > > You'd have to ask Raymond Hettinger I guess.? random.sample works on a set because it starts with: ??????? if isinstance(population, _Set): ??????????? population = tuple(population) So sample() is willing to listify (tuplify?) your set for you, but choice() is not. --Ned. From __peter__ at web.de Wed Jan 17 15:23:26 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 17 Jan 2018 21:23:26 +0100 Subject: Why does random.choice() reject sets? was Re: Speeding up the implementation of Stochastic Gradient Ascent in Python References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> Message-ID: Chris Angelico wrote: > On Thu, Jan 18, 2018 at 6:28 AM, Ned Batchelder > wrote: >> You'll have to replace random.choice() with >> random.choice(list(...)), since you can't random.choice from a set. >> > > Side point: why can't you? You can random.sample from a set, I'm not sure this was a good idea. > but > random.choice requires a sequence. It seems perfectly sane to ask for > a random element from a set. $ python3 -m timeit -s 'from random import sample; items = list(range(100000))' 'sample(items, 5)' 100000 loops, best of 3: 18 usec per loop $ python3 -m timeit -s 'from random import sample; items = set(range(100000))' 'sample(items, 5)' 100 loops, best of 3: 7.27 msec per loop You would need access to the set implementation to get reasonable performance. As random.choice() is more likely to be called repeatedly with the same argument I'd rather not hide something like if isinstance(seq, set): seq = tuple(set) in its function body. From rosuav at gmail.com Wed Jan 17 15:24:34 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Jan 2018 07:24:34 +1100 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <38405213-0c8b-cde2-39e1-59c296e1b5c0@nedbatchelder.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> <011dcdab-06c1-f46a-8774-fdca61c3f44a@nedbatchelder.com> <38405213-0c8b-cde2-39e1-59c296e1b5c0@nedbatchelder.com> Message-ID: On Thu, Jan 18, 2018 at 7:19 AM, Ned Batchelder wrote: > On 1/17/18 2:45 PM, Chris Angelico wrote: >> >> On Thu, Jan 18, 2018 at 6:28 AM, Ned Batchelder >> wrote: >>> >>> You'll have to replace random.choice() with >>> random.choice(list(...)), since you can't random.choice from a set. >>> >> Side point: why can't you? You can random.sample from a set, but >> random.choice requires a sequence. It seems perfectly sane to ask for >> a random element from a set. >> >> > > You'd have to ask Raymond Hettinger I guess. random.sample works on a set > because it starts with: > > if isinstance(population, _Set): > population = tuple(population) > > So sample() is willing to listify (tuplify?) your set for you, but choice() > is not. > Ah. So it's just as inefficient. (I should have looked.) So I guess what we really need is some lower-level support. There's no way in Python to pick a random element from a set in O(1) time. It could be done in O(n) time and O(1) space by iterating through the set until you hit the chosen random index, but you can't just say "grab me one of those". ChrisA From larry.martell at gmail.com Wed Jan 17 16:54:37 2018 From: larry.martell at gmail.com (Larry Martell) Date: Wed, 17 Jan 2018 16:54:37 -0500 Subject: documentation on read.encode In-Reply-To: <6a4e6e00-f388-316c-c089-f53ee4b148cb@mrabarnett.plus.com> References: <201801161435.10062.gheskett@shentel.net> <6a4e6e00-f388-316c-c089-f53ee4b148cb@mrabarnett.plus.com> Message-ID: On Tue, Jan 16, 2018 at 3:58 PM, MRAB wrote: > On 2018-01-16 19:52, Larry Martell wrote: >> >> On Tue, Jan 16, 2018 at 2:35 PM, Gene Heskett >> wrote: >>> >>> On Tuesday 16 January 2018 14:19:38 Larry Martell wrote: >>> >>>> On Tue, Jan 16, 2018 at 12:00 PM, Larry Martell >>> >>> wrote: >>>> >>>> > Looking for 2.7 docs on read.encode - googling did not turn up >>>> > anything. >>>> > >>>> > Specifically, looking for the supported options for base64, and how >>>> > to specify them, e.g. Base64.NO_WRAP >>>> >>>> So I just realized that encode() is not a method of read() it's a >>>> string method. But I still have the same question - can I pass in any >>>> flags? >>>> >>>> My issue is that I am base64 encoding PNG images on linux and it's >>>> putting a LF at the end of each line. If I do the same on Windows it's >>>> putting CR/LF. I want the files to be encoded with no platform >>>> dependences. Googling I found mention of Base64.NO_WRAP and I want to >>>> pass that into encode() - can I do that? >>> >>> >>> Di you not have the manpages installed? >>> >>> In my copy of the manpage: >>> base64 [OPTION]... [FILE] >>> where option is: >>> -w, --wrap=COLS >>> wrap encoded lines after COLS character (default 76). Use >>> 0 to disable line wrapping. >>> >>> Seems pretty simple. >> >> >> But how do I use that in read().encode('base64')? >> > Use the base64 module instead, which is also how you would do it in Python > 3. > > If you're getting CR/LF on Windows, that's because you're opening the file > in text mode. In both Python 2 and Python 3 the base64 string will be a > bytestring, which you'd write out to a file opened in binary mode. > > That's an extra bit of future-proofing! :-) Thanks. That's what it ended up being. The code that was receiving the PNG was not reading and writing the file as binary. Strangely that worked on Linux but not on Windows. From steve+comp.lang.python at pearwood.info Thu Jan 18 00:47:38 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 18 Jan 2018 05:47:38 +0000 (UTC) Subject: documentation on read.encode References: <201801161435.10062.gheskett@shentel.net> <6a4e6e00-f388-316c-c089-f53ee4b148cb@mrabarnett.plus.com> Message-ID: On Wed, 17 Jan 2018 16:54:37 -0500, Larry Martell wrote: > The code that was receiving the > PNG was not reading and writing the file as binary. Strangely that > worked on Linux but not on Windows. Nothing strange about it -- on Unix and Linux systems (with the possible exception of Mac OS?) in Python 2 there's no difference between text and binary mode for ASCII-only files. In Python 2, strings are byte strings, not Unicode, and reading from files returns such sequences of bytes. On Windows, reading from files in text mode treats \r\n as the end of line, and converts[1] such \r\n pairs to \n; it also treats ^Z byte as the end of file[2], or at least it used to back in the 2.5 or so days. I haven't tested it in more recent versions. [1] Technically this is a build-time option, but as far as I know it is not just the default but pretty much universal. [2] https://blogs.msdn.microsoft.com/oldnewthing/20040316-00/?p=40233/ -- Steve From larry.martell at gmail.com Thu Jan 18 06:13:25 2018 From: larry.martell at gmail.com (Larry Martell) Date: Thu, 18 Jan 2018 06:13:25 -0500 Subject: documentation on read.encode In-Reply-To: References: <201801161435.10062.gheskett@shentel.net> <6a4e6e00-f388-316c-c089-f53ee4b148cb@mrabarnett.plus.com> Message-ID: On Thu, Jan 18, 2018 at 12:47 AM, Steven D'Aprano wrote: > On Wed, 17 Jan 2018 16:54:37 -0500, Larry Martell wrote: > >> The code that was receiving the >> PNG was not reading and writing the file as binary. Strangely that >> worked on Linux but not on Windows. > > Nothing strange about it -- on Unix and Linux systems (with the possible > exception of Mac OS?) in Python 2 there's no difference between text and > binary mode for ASCII-only files. In Python 2, strings are byte strings, > not Unicode, and reading from files returns such sequences of bytes. > > On Windows, reading from files in text mode treats \r\n as the end of > line, and converts[1] such \r\n pairs to \n; it also treats ^Z byte as > the end of file[2], or at least it used to back in the 2.5 or so days. I > haven't tested it in more recent versions. > > > [1] Technically this is a build-time option, but as far as I know it is > not just the default but pretty much universal. > > [2] https://blogs.msdn.microsoft.com/oldnewthing/20040316-00/?p=40233/ Thanks for the clarification. I have been programming since 1975 and thankfully have had very little exposure to Windows. From jason.swails at gmail.com Thu Jan 18 16:37:18 2018 From: jason.swails at gmail.com (Jason Swails) Date: Thu, 18 Jan 2018 16:37:18 -0500 Subject: Very strange issues with collections.Mapping Message-ID: Hello! I am running into a very perplexing issue that is very rare, but creeps up and is crashing my app. The root cause of the issue comes down to the following check returning true: isinstance([], collections.Mapping) Obviously you can get this behavior if you register `list` as a subclass of the Mapping ABC, but I'm not doing that. Because the issue is so rare (but still common enough that I need to address it), it's hard to reproduce in a bench test. What I am going to try is to essentially monkey-patch collections.Mapping.register with a method that dumps a stack trace whenever it's called at the time of initial import so I can get an idea of where this method could *possibly* be getting called with a list as its argument. The annoying thing here is that wherever the bug is happening, the crash happens *way* far away (in a third-party library). I've also determined it as the root cause of two crashes that seem completely unrelated (except that the crash is caused by a list not behaving like a dict shortly after isinstance(obj, collections.Mapping) returns True). These are the libraries I'm using: amqp billiard celery dj-database-url Django django-redis-cache enum34 gunicorn kombu newrelic psycopg2 pyasn1 pytz redis requests rsa six vine voluptuous It's a web application, as you can probably tell. The main reason I ask here is because I'm wondering if anybody has encountered this before and managed to hunt down which of these libraries is doing something naughty? Thanks! Jason -- Jason M. Swails From rosuav at gmail.com Thu Jan 18 16:42:11 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Jan 2018 08:42:11 +1100 Subject: Very strange issues with collections.Mapping In-Reply-To: References: Message-ID: On Fri, Jan 19, 2018 at 8:37 AM, Jason Swails wrote: > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) > > Obviously you can get this behavior if you register `list` as a subclass of > the Mapping ABC, but I'm not doing that. Because the issue is so rare (but > still common enough that I need to address it), it's hard to reproduce in a > bench test. Just a quickie: Have you confirmed for sure that it is a vanilla list - that "type(x) is type([])" - not some subclass thereof? Other than that, I don't have any specific tips; have fun instrumenting your code to try to figure this out! (And I mean that sincerely - this kind of thing CAN be fun.) ChrisA From kyosohma at gmail.com Thu Jan 18 17:38:04 2018 From: kyosohma at gmail.com (Mike Driscoll) Date: Thu, 18 Jan 2018 14:38:04 -0800 (PST) Subject: Where are the moderators? Message-ID: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Hi, What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? Thanks, Mike From p.f.moore at gmail.com Thu Jan 18 17:48:04 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 18 Jan 2018 22:48:04 +0000 Subject: Where are the moderators? In-Reply-To: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Message-ID: On 18 January 2018 at 22:38, Mike Driscoll wrote: > Hi, > > What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? > > Thanks, > Mike You seem to be using the Google Groups interface. The moderators can't control what appears on that list, and it is as you say essentially unusable (IMO) because of spam. For a reasonably spam-free experience, I suggest subscribing to the mailing list itself (the web page to do that is attached to mails posted here - https://mail.python.org/mailman/listinfo/python-list) Paul From skip.montanaro at gmail.com Thu Jan 18 18:13:46 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 18 Jan 2018 17:13:46 -0600 Subject: Where are the moderators? In-Reply-To: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Message-ID: > What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? Just to emphasize what Paul said, the only moderators I'm aware of are those who moderate python-list at python.org. A bidirectional gateway runs on mail.python.org which ships messages back and forth. I take care of the SpamBayes setup on mail.python.org, whose primary task is to divert spam which arrives from comp.lang.python. Skip From duncan at invalid.invalid Thu Jan 18 18:17:27 2018 From: duncan at invalid.invalid (duncan smith) Date: Thu, 18 Jan 2018 23:17:27 +0000 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> Message-ID: On 17/01/18 14:29, leutrim.kaleci at gmail.com wrote: > > Hello everyone, > > I am implementing a time-dependent Recommender System which applies BPR (Bayesian Personalized Ranking), where Stochastic Gradient Ascent is used to learn the parameters of the model. Such that, one iteration involves sampling randomly the quadruple (i.e. userID, positive_item, negative_item, epoch_index) for n times, where n is the total number of positive feedbacks (i.e. the number of ratings given to all items). But, as my implementation takes too much time for learning the parameters (since it requires 100 iterations to learn the parameters), I was wondering if there is a way to improve my code and speed up the learning process of the parameters. > > Please find the code of my implementation (the update function named as updateFactors is the one that learns the parameters, and such that I guess is the one that should be improved in order to speed up the process of learning the parameter values) in the following link: > https://codereview.stackexchange.com/questions/183707/speeding-up-the-implementation-of-stochastic-gradient-ascent-in-python > dim0 = [] dim1 = [] ... dim19 = [] dim0.append((asin, self.theta_item_per_bin[bin][itemID][0])) dim1.append((asin,self.theta_item_per_bin[bin][itemID][1])) ... dim19.append((asin,self.theta_item_per_bin[bin][itemID][19])) for d in range(self.K2): if d == 0: max_value = max(dim0, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ if d == 1: max_value = max(dim1, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ .... if d == 19: max_value = max(dim19, key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:',d,', itemID: ', asin_, ', value = ', value_ How about something like, dims = [[] for _ in range(20)] for i in range(20): dims[i].append((asin, self.theta_item_per_bin[bin][itemID][i])) for j in range(self.K2): max_value = max(dims[j], key=operator.itemgetter(1)) asin_ = max_value[0] value_ = max_value[1] print 'dim:', j,', itemID: ', asin_, ', value = ', value_ I haven't looked at it in detail, and I'm not sure why you have exactly 20 lists or what happens if self.K2 is not equal to 20. But you can make the code simpler and shorter, and marginally more efficient by not testing all those if statements. Duncan From grant.b.edwards at gmail.com Thu Jan 18 18:23:14 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 18 Jan 2018 23:23:14 +0000 (UTC) Subject: How to use asyncore with SSL? Message-ID: I've been trying to use the secure smtpd module from https://github.com/bcoe/secure-smtpd, but the SSL support seems to be fundamentally broken. That module simply wraps a socket and then expects to use it in the normal way via asyncore. Of course that fails the first time an ssl-wrapped-socket's send or recv method raises SSLWantReadError or SSLWantWriteError. Those exceptions aren't handled and it crashes. That makes the SSL support pretty much useless. I'm trying to fix that, but I can't find any information or documentation about using asyncore with SSL. Alternatively, a pointer to a simpler smtp server library that supports SSL would be great. The use of asyncore and multiprocessing process pools by this module is _way_ overkill for my needs and results in something that 1) doesn't work, and 2) can't be debugged. -- Grant Edwards grant.b.edwards Yow! Pardon me, but do you at know what it means to be gmail.com TRULY ONE with your BOOTH! From breamoreboy at gmail.com Thu Jan 18 18:28:37 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Thu, 18 Jan 2018 15:28:37 -0800 (PST) Subject: Where are the moderators? In-Reply-To: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Message-ID: <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> On Thursday, January 18, 2018 at 10:38:18 PM UTC, Mike Driscoll wrote: > Hi, > > What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? > > Thanks, > Mike Simply point your email client at news.gmane.org and take your pick from hundreds of Python lists and thousands of other technical lists that are all spam free. -- Kindest regards. Mark Lawrence. From python at mrabarnett.plus.com Thu Jan 18 19:43:36 2018 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 19 Jan 2018 00:43:36 +0000 Subject: Where are the moderators? In-Reply-To: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Message-ID: <6886b0f2-295e-6ce6-20da-5118f4aa83e1@mrabarnett.plus.com> On 2018-01-18 22:38, Mike Driscoll wrote: > Hi, > > What happened to the moderators? I have always liked this forum, but there's so much spam now. Is there a way to become a moderator so this can be cleaned up? > How are you viewing the list? If you're viewing via Google Groups, then complain about the spam to Google. I'm subscribed to the list proper, and I'm not having a problem with spam; it's filtered out. See: https://mail.python.org/mailman/listinfo/python-list From drsalists at gmail.com Thu Jan 18 19:50:18 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 18 Jan 2018 16:50:18 -0800 Subject: Very strange issues with collections.Mapping In-Reply-To: References: Message-ID: Probably not what you want to hear, but this might be a good place for an SSCCE. On Thu, Jan 18, 2018 at 1:37 PM, Jason Swails wrote: > Hello! > > I am running into a very perplexing issue that is very rare, but creeps up > and is crashing my app. > > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) > > Obviously you can get this behavior if you register `list` as a subclass of > the Mapping ABC, but I'm not doing that. Because the issue is so rare (but > still common enough that I need to address it), it's hard to reproduce in a > bench test. > > What I am going to try is to essentially monkey-patch > collections.Mapping.register with a method that dumps a stack trace > whenever it's called at the time of initial import so I can get an idea of > where this method could *possibly* be getting called with a list as its > argument. > > The annoying thing here is that wherever the bug is happening, the crash > happens *way* far away (in a third-party library). I've also determined it > as the root cause of two crashes that seem completely unrelated (except > that the crash is caused by a list not behaving like a dict shortly after > isinstance(obj, collections.Mapping) returns True). These are the > libraries I'm using: > > amqp > billiard > celery > dj-database-url > Django > django-redis-cache > enum34 > gunicorn > kombu > newrelic > psycopg2 > pyasn1 > pytz > redis > requests > rsa > six > vine > voluptuous > > It's a web application, as you can probably tell. The main reason I ask > here is because I'm wondering if anybody has encountered this before and > managed to hunt down which of these libraries is doing something naughty? > > Thanks! > Jason > > -- > Jason M. Swails > -- > https://mail.python.org/mailman/listinfo/python-list From bob at mellowood.ca Thu Jan 18 20:42:35 2018 From: bob at mellowood.ca (Bob van der Poel) Date: Thu, 18 Jan 2018 18:42:35 -0700 Subject: Where are the moderators? In-Reply-To: <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> Message-ID: On Thu, Jan 18, 2018 at 4:28 PM, wrote: > On Thursday, January 18, 2018 at 10:38:18 PM UTC, Mike Driscoll wrote: > > Hi, > > > > What happened to the moderators? I have always liked this forum, but > there's so much spam now. Is there a way to become a moderator so this can > be cleaned up? > > > > Thanks, > > Mike > > Simply point your email client at news.gmane.org and take your pick from > hundreds of Python lists and thousands of other technical lists that are > all spam free. > > Actually, this link is broken. But, gmane.org (without the news prefix) does work. -- **** Listen to my FREE CD at http://www.mellowood.ca/music/cedars **** Bob van der Poel ** Wynndel, British Columbia, CANADA ** EMAIL: bob at mellowood.ca WWW: http://www.mellowood.ca From skip.montanaro at gmail.com Thu Jan 18 22:17:06 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Thu, 18 Jan 2018 21:17:06 -0600 Subject: Where are the moderators? In-Reply-To: References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> Message-ID: >> Simply point your email client at news.gmane.org and take your pick from >> hundreds of Python lists and thousands of other technical lists that are >> all spam free. >> >> > Actually, this link is broken. But, gmane.org (without the news prefix) > does work. Neither works for me. The news link clearly fails, but the gmane.org link has no search functionality either. Do you have a direct URL to comp.lang.python? Skip From steve+comp.lang.python at pearwood.info Thu Jan 18 23:03:39 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 19 Jan 2018 04:03:39 +0000 (UTC) Subject: Where are the moderators? References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <6886b0f2-295e-6ce6-20da-5118f4aa83e1@mrabarnett.plus.com> Message-ID: On Fri, 19 Jan 2018 00:43:36 +0000, MRAB wrote: > If you're viewing via Google Groups, then complain about the spam to > Google. Alternatively, and just as effectively, you could repeatedly hit yourself on the head with a ball-peen hammer. That will be just as effective at filtering the spam, but with the added benefit that it will feel so much better when you stop. *wink* -- Steve From steve+comp.lang.python at pearwood.info Thu Jan 18 23:16:51 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 19 Jan 2018 04:16:51 +0000 (UTC) Subject: Very strange issues with collections.Mapping References: Message-ID: On Thu, 18 Jan 2018 16:37:18 -0500, Jason Swails wrote: > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) I re-iterate Chris' suggestion that you check that the instance is an actual list, not a subclass. Can you grep the offending files for something like class .*\(.*list.*\): to see if anything subclasses list? If so, does it look like the subclass is intended to offer a mapping interface? > Obviously you can get this behavior if you register `list` as a subclass > of the Mapping ABC, but I'm not doing that. Because the issue is so > rare (but still common enough that I need to address it), it's hard to > reproduce in a bench test. Ah lovely, a heisenbug. > What I am going to try is to essentially monkey-patch > collections.Mapping.register with a method that dumps a stack trace > whenever it's called at the time of initial import so I can get an idea > of where this method could *possibly* be getting called with a list as > its argument. Seems like a reasonable step to me. Also you could try identifying the offending library by interleaving assertions between the imports: import collections assert not isinstance([], collections.Mapping) import amqp assert not isinstance([], collections.Mapping) import billiard assert not isinstance([], collections.Mapping) import celery etc. When (if) the assertion fails, you know which library is to blame. Not exactly the nicest code, but it is only there until you locate (and fix) the problem. -- Steve From jingshen at micron.com Thu Jan 18 23:47:23 2018 From: jingshen at micron.com (Jingshen Tai (jingshen)) Date: Fri, 19 Jan 2018 04:47:23 +0000 Subject: Anaconda installation problem In-Reply-To: References: Message-ID: <5d1943dce7a44eb39cf76e59d3a2f650@SIWEX4C.sing.micron.com> Hi, When I was installing windows Anaconda 3 64 bit, there is a pop up window saying that the 'python program is closing'. I ignored it and continued the installation. When the Anaconda Prompt is launched, I have this error popped at the top (refer to attached doc.) Does that mean that the batch file is missing? I have tried re-installing and restarted my PC, however, the problem still persists. Also, I further checked that the python version installed is python 2.3.4 instead ( refer to Python 2.3.4 existance). Kindly advise. Regards, JS (Jing Shen) From steve+comp.lang.python at pearwood.info Thu Jan 18 23:48:35 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 19 Jan 2018 04:48:35 +0000 (UTC) Subject: Context manager able to write to the caller's namespace Message-ID: I'm looking for a solution, or at least hints, to this problem. I want to define a context manager in one module: # a.py def CM: def __enter__(self): return self def __exit__(self, *args): pass Then call it from another module: # b.py import a with a.CM() as spam: x = 1 y = 2 in such a way that the spam context manager can, on exit, see the callers namespace and write to it. E.g. as a toy example (this isn't what I actually want to do!) we might have this: with a.CM() as spam: x = 1 print(x) # prints 2, not 1 I stress that's not the intended functionality, it just demonstrates the requirement. Don't assume that the content manager is going to be called in the global scope. In fact, the most common use I'm expecting is to call it from inside a class: class Aardvark: with a.CM() as spam: x = 1 It's okay if it doesn't work inside a function: def foo(): with a.CM() as spam: x = 1 assert x == 2 since function scopes in CPython are weird. Any suggestions? -- Steve From marko at pacujo.net Fri Jan 19 00:34:45 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Fri, 19 Jan 2018 07:34:45 +0200 Subject: How to use asyncore with SSL? References: Message-ID: <8737328l8q.fsf@elektro.pacujo.net> Grant Edwards : > I've been trying to use the secure smtpd module from > https://github.com/bcoe/secure-smtpd, but the SSL support seems to be > fundamentally broken. That module simply wraps a socket and then > expects to use it in the normal way via asyncore. > > Of course that fails the first time an ssl-wrapped-socket's send or > recv method raises SSLWantReadError or SSLWantWriteError. Those > exceptions aren't handled and it crashes. > > That makes the SSL support pretty much useless. > > I'm trying to fix that, but I can't find any information or > documentation about using asyncore with SSL. I'm all in for asynchronous programming, but asyncore is a bit too naive of an approach and shouldn't be used for anything serious. Python3, of course, has the asyncio framework. Additionally, I seem to recall Python's TLS support really supported synchronous processing only (based on some experimentation of my own). I hope I'm wrong on that. > Alternatively, a pointer to a simpler smtp server library that > supports SSL would be great. The use of asyncore and multiprocessing > process pools by this module is _way_ overkill for my needs and > results in something that 1) doesn't work, and 2) can't be debugged. Haven't tried it myself, but I supposed Twisted might be what you're looking for. Myself, I've written several "asyncore" replacements in Python as well as an SMTP server for my personal email needs. You could also consider writing your own implementation. For async, there's select.epoll and the like (assuming Linux), and SMTP is rather a simple protocol. Marko From rosuav at gmail.com Fri Jan 19 00:49:49 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Jan 2018 16:49:49 +1100 Subject: Context manager able to write to the caller's namespace In-Reply-To: References: Message-ID: On Fri, Jan 19, 2018 at 3:48 PM, Steven D'Aprano wrote: > I want to define a context manager in one module: > > # a.py > def CM: > def __enter__(self): > return self > def __exit__(self, *args): > pass > > > Then call it from another module: > > # b.py > import a > with a.CM() as spam: > x = 1 > y = 2 > > > in such a way that the spam context manager can, on exit, see the callers > namespace and write to it. E.g. as a toy example (this isn't what I > actually want to do!) we might have this: > > with a.CM() as spam: > x = 1 > print(x) > # prints 2, not 1 > > I stress that's not the intended functionality, it just demonstrates the > requirement. I'm assuming that you don't have to magically know that x was assigned to, as that's its own separate problem. AIUI, you have three separate cases: 1) Context manager was called from global scope, and needs access to globals() or locals() as returned in the caller 2) Ditto ditto class scope, and needs access to the phantom environment that's going to get put into the new class's dict 3) Ditto ditto function scope, which you specifically said is okay to fail, but which also wants to be locals(). So basically, you want to call locals() in the caller's scope. I don't think there's any standard way to do that, so you're going to be stuck with sys._getframe() and the cross-implementation incompatibilities that entails. But in CPython (tested in 3.7, should be fine in older versions), this ought to work: >>> def mutate(): ... locals = sys._getframe(1).f_locals ... locals["x"] = 4 ... >>> x 1 >>> mutate() >>> x 4 >>> class Foo: ... x = 2 ... print(x) ... mutate() ... print(x) ... 2 4 Written as a context manager: >>> import contextlib >>> @contextlib.contextmanager ... def mutate(): ... yield ... locals = sys._getframe(2).f_locals ... locals["x"] = 4 ... >>> class Foo: ... x = 2 ... print(x) ... with mutate(): ... x = 3 ... print(x) ... print(x) ... 2 3 4 >>> There's basically zero guarantees about this though. Have fun. :) ChrisA From steve+comp.lang.python at pearwood.info Fri Jan 19 00:56:41 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Fri, 19 Jan 2018 05:56:41 +0000 (UTC) Subject: Context manager able to write to the caller's namespace References: Message-ID: On Fri, 19 Jan 2018 16:49:49 +1100, Chris Angelico wrote: [...] > 1) Context manager was called from global scope, and needs access to > globals() or locals() as returned in the caller Ayyyyeeee! /facepalm Of course the caller can just pass locals() to the context manager. Why didn't I think of that? I think I'm okay with that as a solution, but... >>>> def mutate(): > ... locals = sys._getframe(1).f_locals ... locals["x"] = 4 > ... I'll play around with that too. Just for kicks :-) -- Steve From rosuav at gmail.com Fri Jan 19 01:03:52 2018 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Jan 2018 17:03:52 +1100 Subject: Context manager able to write to the caller's namespace In-Reply-To: <151634160331.28469.14923020799689694228@bankmail.host> References: <151634160331.28469.14923020799689694228@bankmail.host> Message-ID: On Fri, Jan 19, 2018 at 5:00 PM, wrote: > Hello, > > Thank you for your mail. I will answer as fast as possible. If you're going to subscribe to a mailing list, PLEASE disable your autoresponder. Otherwise, messages like this will get marked as Spam, and your legit mail will end up getting tarred with the same metaphorical brush. ChrisA From tjreedy at udel.edu Fri Jan 19 03:20:31 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 19 Jan 2018 03:20:31 -0500 Subject: Where are the moderators? In-Reply-To: References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> Message-ID: On 1/18/2018 11:05 PM, Dennis Lee Bieber wrote: > On Thu, 18 Jan 2018 18:42:35 -0700, Bob van der Poel > declaimed the following: > >> On Thu, Jan 18, 2018 at 4:28 PM, wrote: >> >>> On Thursday, January 18, 2018 at 10:38:18 PM UTC, Mike Driscoll wrote: >>>> Hi, >>>> >>>> What happened to the moderators? I have always liked this forum, but >>> there's so much spam now. Is there a way to become a moderator so this can >>> be cleaned up? >>>> >>>> Thanks, >>>> Mike >>> >>> Simply point your email client at news.gmane.org Mark meant, or should have meant, news or combined mail/news client. Thunderbird (and Outlook Express, for instance) are the latter. I read and post to this list and others through my Thunderbird news account >>> and take your pick from >>> hundreds of Python lists and thousands of other technical lists that are >>> all spam free. >>> >>> >> Actually, this link is broken. news.gmane.org is a news server, and the server I am sending this through. It is not an html web page server. >> But, gmane.org (without the news prefix) >> does work. In a browser, but not as a news server. > While I can't speak for /email client/, Agent has been running well for > some years now using news.gmane.org. Ditto for Thunderbird news (as opposed to mail) account. -- Terry Jan Reedy From pjh at nanoworks.com Fri Jan 19 03:23:04 2018 From: pjh at nanoworks.com (Percival John Hackworth) Date: Fri, 19 Jan 2018 00:23:04 -0800 Subject: Where are the moderators? References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> Message-ID: <0001HW.2011E268002D7A6E1593582CF@news.individual.net> On 18-Jan-2018, Mike Driscoll wrote (in article<73f42561-d7bf-4cba-97c7-8e63b87a9281 at googlegroups.com>): > Hi, > > What happened to the moderators? I have always liked this forum, but there's > so much spam now. Is there a way to become a moderator so this can be cleaned > up? > > Thanks, > Mike Google Groups is unmoderated, as has already been said here. I use a commercial news service but individual.net offers a year of the non-binary groups for ?10 but there are free accounts out there. The individual.net account isn't filtered by my news reader takes care of most of the spam. YMMV From tjreedy at udel.edu Fri Jan 19 03:47:52 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 19 Jan 2018 03:47:52 -0500 Subject: Very strange issues with collections.Mapping In-Reply-To: References: Message-ID: On 1/18/2018 4:37 PM, Jason Swails wrote: > Hello! > > I am running into a very perplexing issue that is very rare, but creeps up > and is crashing my app. > > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) I confirmed that one *can* register 'list' as a Mapping so that the above returns True. I also discovered that doing so does not change Mapping._abc_registry. Have you grepped the dependencies (snipped) for '.register'? Alternatively, the abc module doc says """ Changed in version 3.4: To detect calls to register(), you can use the get_cache_token() function. ... abc.get_cache_token() Returns the current abstract base class cache token. The token is an opaque object (that supports equality testing) identifying the current version of the abstract base class cache for virtual subclasses. The token changes with every call to ABCMeta.register() on any ABC. """ You could call this before, between, and after all your imports, and add such calls to the guilty import. -- Terry Jan Reedy From as at sci.fi Fri Jan 19 04:43:34 2018 From: as at sci.fi (Anssi Saari) Date: Fri, 19 Jan 2018 11:43:34 +0200 Subject: Where are the moderators? References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> Message-ID: Skip Montanaro writes: > Neither works for me. The news link clearly fails, but the gmane.org > link has no search functionality either. Do you have a direct URL to > comp.lang.python? Gmane seems to be in the middle of a massive rebuild at the moment. Comp.lang.python's webpage is http://dir.gmane.org/gmane.comp.python.general but none of the links to read or search the list seem to be working at the moment. NNTP access to news.gmane.org and group gmane.comp.python.general works for me. For reading at least, I don't know if this message goes anywhere... From hemla21 at gmail.com Fri Jan 19 06:42:30 2018 From: hemla21 at gmail.com (Heli) Date: Fri, 19 Jan 2018 03:42:30 -0800 (PST) Subject: pyinstaller not finding external fortran executable Message-ID: <1fef497f-c328-4a7f-a14e-878fb4a36a15@googlegroups.com> Dear all, My code consists of a graphical user interface written in PySide. This GUI will then call a FORTRAN executable. I run the FORTRAN executable using the following lines in my script: curPath = os.path.dirname(os.path.realpath(__file__)) execPath = os.path.join(curPath, "myFORTRAN.out") returnValue = subprocess.call([execPath, self.runConfig] The script works fine and finds executable. The problem is when I use pyinstaller. I use the following command to create standalone executable that bundles my script, the external FORTRAN executable and all necessary python modules/libraries in to one standalone executable. pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5pyl_proxy main_script.py --onefile I get the following error running this standalone executable: File "subprocess.py", line 534, in call File "subprocess.py", line 856, in __init__ File "subprocess.py", line 1464, in _execute_child FileNotFoundError: [Errno 2] No such file or directory: ..../myFORTRAN.out' If I manually copy myFORTRAN.out in to the same directory as the standalone executable created by pyinstaller, then it works. Does anybody know how to fix this? Thanks in Advance for your help, From arj.python at gmail.com Fri Jan 19 07:03:50 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Fri, 19 Jan 2018 16:03:50 +0400 Subject: Anaconda installation problem In-Reply-To: <5d1943dce7a44eb39cf76e59d3a2f650@SIWEX4C.sing.micron.com> References: <5d1943dce7a44eb39cf76e59d3a2f650@SIWEX4C.sing.micron.com> Message-ID: i suggest using version of py 3.6 On 19 Jan 2018 08:54, "Jingshen Tai (jingshen)" wrote: > > Hi, > > When I was installing windows Anaconda 3 64 bit, there is a pop up window > saying that the 'python program is closing'. I ignored it and continued the > installation. > > When the Anaconda Prompt is launched, I have this error popped at the top > (refer to attached doc.) Does that mean that the batch file is missing? > I have tried re-installing and restarted my PC, however, the problem still > persists. > > Also, I further checked that the python version installed is python 2.3.4 > instead ( refer to Python 2.3.4 existance). > Kindly advise. > > Regards, > JS (Jing Shen) > > -- > https://mail.python.org/mailman/listinfo/python-list > From tjol at tjol.eu Fri Jan 19 07:41:31 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 19 Jan 2018 13:41:31 +0100 Subject: Anaconda installation problem In-Reply-To: <5d1943dce7a44eb39cf76e59d3a2f650@SIWEX4C.sing.micron.com> References: <5d1943dce7a44eb39cf76e59d3a2f650@SIWEX4C.sing.micron.com> Message-ID: <3f93f027-59aa-f180-39b3-417a1d53baac@tjol.eu> On 2018-01-19 05:47, Jingshen Tai (jingshen) wrote: > > Hi, > > When I was installing windows Anaconda 3 64 bit, there is a pop up window saying that the 'python program is closing'. I ignored it and continued the installation. > > When the Anaconda Prompt is launched, I have this error popped at the top > (refer to attached doc.) This mailing list does not support attachments. > Does that mean that the batch file is missing? > I have tried re-installing and restarted my PC, however, the problem still persists. > > Also, I further checked that the python version installed is python 2.3.4 instead ( refer to Python 2.3.4 existance). > Kindly advise. Are you sure? Python 2.3 was released in July 2003, 14? years ago. > > Regards, > JS (Jing Shen) > From grant.b.edwards at gmail.com Fri Jan 19 11:13:45 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 19 Jan 2018 16:13:45 +0000 (UTC) Subject: How to use asyncore with SSL? References: <8737328l8q.fsf@elektro.pacujo.net> Message-ID: On 2018-01-19, Marko Rauhamaa wrote: > Grant Edwards : > >> I've been trying to use the secure smtpd module from >> https://github.com/bcoe/secure-smtpd, but the SSL support seems to be >> fundamentally broken. [...] >> I'm trying to fix that, but I can't find any information or >> documentation about using asyncore with SSL. > > I'm all in for asynchronous programming, but asyncore is a bit too naive > of an approach and shouldn't be used for anything serious. Python3, of > course, has the asyncio framework. I would definitely not use it were I writing something from scratch. But it's what's used by the only secure (SSL+AUTH) smtpd implementation I can find. > Additionally, I seem to recall Python's TLS support really supported > synchronous processing only (based on some experimentation of my own). I > hope I'm wrong on that. That depends on what you mean by "support". You can use ssl-wrapped sockets in non-blocking mode to build an asynchronous application, but the API and semantics for ssl-sockets in non-blocking mode are not identical to plain TCP sockets, so the event-handling or dispatcher needs to be SSL-aware (which asyn{core,chat} definitely are not). >> Alternatively, a pointer to a simpler smtp server library that >> supports SSL would be great. The use of asyncore and multiprocessing >> process pools by this module is _way_ overkill for my needs and >> results in something that 1) doesn't work, and 2) can't be debugged. > > Haven't tried it myself, but I supposed Twisted might be what you're > looking for. It is certianly more SSL-aware than asyncore: http://twistedmatrix.com/documents/current/core/howto/ssl.html And it has smtp server-side support examples: https://twistedmatrix.com/documents/current/mail/examples/#smtp-servers > Myself, I've written several "asyncore" replacements in Python as > well as an SMTP server for my personal email needs. You could also > consider writing your own implementation. For async, there's > select.epoll and the like (assuming Linux), and SMTP is rather a > simple protocol. I don't think a simple, low-volume SMTP server needs to be asynchronous. The protocol is completely half-duplex command/response so jumping through hoops to use an async framework seems pointless. Adding in multiprocessing the way secure-smtpd is really over-the-top unless you're designing for really high message volumes and connection counts. I plan on handling several messages per week and am fine with supporting only one connection at a time. So twisted may be overkill also, but at least it looks like it supports SSL. -- Grant Edwards grant.b.edwards Yow! I'm sitting on my at SPEED QUEEN ... To me, gmail.com it's ENJOYABLE ... I'm WARM ... I'm VIBRATORY ... From grant.b.edwards at gmail.com Fri Jan 19 11:18:20 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Fri, 19 Jan 2018 16:18:20 +0000 (UTC) Subject: Where are the moderators? References: <73f42561-d7bf-4cba-97c7-8e63b87a9281@googlegroups.com> <55cbb112-b9a8-4fd0-b87d-7ee27efcf023@googlegroups.com> Message-ID: On 2018-01-19, Anssi Saari wrote: > Skip Montanaro writes: > >> Neither works for me. The news link clearly fails, but the gmane.org >> link has no search functionality either. Do you have a direct URL to >> comp.lang.python? > > Gmane seems to be in the middle of a massive rebuild at the > moment. It's a massive "rebuild" that has been stalled for years. I do not expect Gmanes web UI to be revived. > Comp.lang.python's webpage is > http://dir.gmane.org/gmane.comp.python.general but none of the links > to read or search the list seem to be working at the moment. They've been that way for years. > NNTP access to news.gmane.org and group gmane.comp.python.general > works for me. For reading at least, I don't know if this message > goes anywhere... For most groups/lists, gmane's mail<->nntp service seems to be working. There are some "groups" where I've noticed that posting works, but traffic in the other direction mail->nntp is intermittent. -- Grant Edwards grant.b.edwards Yow! Did something bad at happen or am I in a gmail.com drive-in movie?? From john.krukoff at ubnt.com Fri Jan 19 13:49:55 2018 From: john.krukoff at ubnt.com (John Krukoff) Date: Fri, 19 Jan 2018 11:49:55 -0700 Subject: Very strange issues with collections.Mapping In-Reply-To: References: Message-ID: Have you ruled out the possibility that collections.Mapping has been (perhaps temporarily) assigned to something else? On Thu, Jan 18, 2018 at 2:37 PM, Jason Swails wrote: > Hello! > > I am running into a very perplexing issue that is very rare, but creeps up > and is crashing my app. > > The root cause of the issue comes down to the following check returning > true: > > isinstance([], collections.Mapping) > > Obviously you can get this behavior if you register `list` as a subclass of > the Mapping ABC, but I'm not doing that. Because the issue is so rare (but > still common enough that I need to address it), it's hard to reproduce in a > bench test. > > What I am going to try is to essentially monkey-patch > collections.Mapping.register with a method that dumps a stack trace > whenever it's called at the time of initial import so I can get an idea of > where this method could *possibly* be getting called with a list as its > argument. > > The annoying thing here is that wherever the bug is happening, the crash > happens *way* far away (in a third-party library). I've also determined it > as the root cause of two crashes that seem completely unrelated (except > that the crash is caused by a list not behaving like a dict shortly after > isinstance(obj, collections.Mapping) returns True). These are the > libraries I'm using: > > amqp > billiard > celery > dj-database-url > Django > django-redis-cache > enum34 > gunicorn > kombu > newrelic > psycopg2 > pyasn1 > pytz > redis > requests > rsa > six > vine > voluptuous > > It's a web application, as you can probably tell. The main reason I ask > here is because I'm wondering if anybody has encountered this before and > managed to hunt down which of these libraries is doing something naughty? > > Thanks! > Jason > > -- > Jason M. Swails > -- > https://mail.python.org/mailman/listinfo/python-list > From i.nasr91 at yahoo.com Fri Jan 19 15:44:20 2018 From: i.nasr91 at yahoo.com (=?UTF-8?Q?=E2=80=AAIbrahim_Nasr=E2=80=AC_=E2=80=AA?=) Date: Fri, 19 Jan 2018 20:44:20 +0000 (UTC) Subject: error message by installation In-Reply-To: <1529982597.1586827.1516389086954@mail.yahoo.com> References: <1529982597.1586827.1516389086954.ref@mail.yahoo.com> <1529982597.1586827.1516389086954@mail.yahoo.com> Message-ID: <432428030.1643436.1516394660180@mail.yahoo.com> kindly inform me what to do. From leo at superlel.me Fri Jan 19 16:50:22 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Fri, 19 Jan 2018 22:50:22 +0100 Subject: Fourth example from PEP 342 Message-ID: <9856943b-8c5b-0126-f0b6-a084f6398f7f@superlel.me> Hello list, I am currently trying to learn co-routine/asynchronous mechanisms in Python. I read the PEP 342, but I stumble on the fourth example. I don't understand what the lines "data = yield nonblocking_read(sock)" in echo_handler() and "connected_socket = yield nonblocking_accept(sock)" in listen_on() are trying to do. For example, can someone explain me how the returned value in the line "connected_socket = yield nonblocking_accept(sock)" can be used on the next line ("trampoline.add(handler(connected_socket))") ? To me, it looks like the returned value is lost in the Trampoline, when resume() gets the returned value of the yield expression. L?o From breamoreboy at gmail.com Fri Jan 19 19:02:26 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Fri, 19 Jan 2018 16:02:26 -0800 (PST) Subject: error message by installation In-Reply-To: References: <1529982597.1586827.1516389086954.ref@mail.yahoo.com> <1529982597.1586827.1516389086954@mail.yahoo.com> <432428030.1643436.1516394660180@mail.yahoo.com> Message-ID: <975bc7bb-53f6-4bd1-9007-45517f6e9afd@googlegroups.com> On Friday, January 19, 2018 at 8:47:52 PM UTC, i.na... at yahoo.com wrote: > kindly inform me what to do. Please read this http://www.catb.org/esr/faqs/smart-questions.html and then try asking again. -- Kindest regards. Mark Lawrence. From ben+python at benfinney.id.au Fri Jan 19 23:05:22 2018 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 20 Jan 2018 15:05:22 +1100 Subject: error message by installation References: <1529982597.1586827.1516389086954.ref@mail.yahoo.com> <1529982597.1586827.1516389086954@mail.yahoo.com> <432428030.1643436.1516394660180@mail.yahoo.com> Message-ID: <85po65jhtp.fsf@benfinney.id.au> ?Ibrahim Nasr? ? via Python-list writes: > kindly inform me what to do. Welcome to this discussion forum, and congratulations on trying Python. Can you write a new message that describes exactly what you've tried, exactly what happened (please copy-and-paste any error messages), and what you expected to happen? -- \ ?Pinky, are you pondering what I'm pondering?? ?Um, I think so, | `\ Brainie, but why would anyone want to Pierce Brosnan?? ?_Pinky | _o__) and The Brain_ | Ben Finney From steve+comp.lang.python at pearwood.info Sat Jan 20 03:32:43 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sat, 20 Jan 2018 08:32:43 +0000 (UTC) Subject: Is this the right way to write a codec error handler? Message-ID: I want an error handler that falls back on Latin-1 for anything which cannot be decoded. Is this the right way to write it? def latin1_fallback(exception): assert isinstance(exception, UnicodeError) start, end = exception.start, exception.end obj = exception.object if isinstance(exception, UnicodeDecodeError): return (obj[start:end].decode('latin1'), end+1) elif isinstance(exception, UnicodeEncodeError): return (obj[start:end].encode('latin1'), end+1) else: raise Comments, corrections and criticisms welcome. Thanks. -- Steve From tjol at tjol.eu Sat Jan 20 05:55:07 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 20 Jan 2018 11:55:07 +0100 Subject: Fourth example from PEP 342 In-Reply-To: <9856943b-8c5b-0126-f0b6-a084f6398f7f@superlel.me> References: <9856943b-8c5b-0126-f0b6-a084f6398f7f@superlel.me> Message-ID: <7ca9b0b6-824c-340a-ae81-6a981875f3a0@tjol.eu> On 19/01/18 22:50, L?o El Amri wrote: > Hello list, > > I am currently trying to learn co-routine/asynchronous mechanisms in > Python. I read the PEP 342, but I stumble on the fourth example. > I don't understand what the lines "data = yield nonblocking_read(sock)" > in echo_handler() and "connected_socket = yield > nonblocking_accept(sock)" in listen_on() are trying to do. > > For example, can someone explain me how the returned value in the line > "connected_socket = yield nonblocking_accept(sock)" can be used on the > next line ("trampoline.add(handler(connected_socket))") ? To me, it > looks like the returned value is lost in the Trampoline, when resume() > gets the returned value of the yield expression. > > L?o > Let's see. Upon the line connected_socket = yield nonblocking_accept(sock) control is returned to t.resume. nonblocking_accept is supposed to be a coroutine, so t.resume does this: if isinstance(value, types.GeneratorType): # Yielded to a specific coroutine, push the # current one on the stack, and call the new # one with no args self.schedule(value, (coroutine,stack)) Ergo, it schedules the nonblocking_accept coroutine (?value?) to be called on the next iteration, and keeps the running listen_on coroutine (?coroutine?) on the stack for safekeeping. On a subsequent iteration (jump?) of the trampoline, nonblocking_accept will yield a socket. Then, resume will elif stack: # Yielded a result, pop the stack and send the # value to the caller self.schedule(stack[0], stack[1], value) plan to return control to listen_on (?stack[0]?), and (the next time around) send the value back in. if exc: value = coroutine.throw(value,*exc) else: value = coroutine.send(value) Now, listen on is running again and has the value yielded by nonblocking_accept. I wish I knew how exactly nonblocking_accept was supposed to work here, but that's beside the point as the world of Python coroutines and async I/O has moved on since 2005. Hope this helps Thomas From storchaka at gmail.com Sat Jan 20 05:57:45 2018 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 20 Jan 2018 12:57:45 +0200 Subject: Is this the right way to write a codec error handler? In-Reply-To: References: Message-ID: 20.01.18 10:32, Steven D'Aprano ????: > I want an error handler that falls back on Latin-1 for anything which > cannot be decoded. Is this the right way to write it? > > > def latin1_fallback(exception): > assert isinstance(exception, UnicodeError) > start, end = exception.start, exception.end > obj = exception.object > if isinstance(exception, UnicodeDecodeError): > return (obj[start:end].decode('latin1'), end+1) > elif isinstance(exception, UnicodeEncodeError): > return (obj[start:end].encode('latin1'), end+1) > else: > raise Just `end` instead of `end+1`. And it is safer to use `bytes.decode(obj[start:end], 'latin1')` or `str(obj[start:end], 'latin1')` instead of `obj[start:end].decode('latin1')`. Just for the case if obj has overridden decode() method. Otherwise LGTM. From leo at superlel.me Sat Jan 20 07:36:00 2018 From: leo at superlel.me (=?UTF-8?Q?L=c3=a9o_El_Amri?=) Date: Sat, 20 Jan 2018 13:36:00 +0100 Subject: Fourth example from PEP 342 In-Reply-To: <7ca9b0b6-824c-340a-ae81-6a981875f3a0@tjol.eu> References: <9856943b-8c5b-0126-f0b6-a084f6398f7f@superlel.me> <7ca9b0b6-824c-340a-ae81-6a981875f3a0@tjol.eu> Message-ID: <14088906-42fa-8ca3-6539-719a14ea7f79@superlel.me> On 20/01/2018 11:55, Thomas Jollans wrote: > control is returned to t.resume. nonblocking_accept is supposed to be a > coroutine > Ergo, it schedules the nonblocking_accept coroutine (?value?) to be > called on the next iteration, and keeps the running listen_on coroutine > (?coroutine?) on the stack for safekeeping. Hello Thomas, I missed THE point of the examples. nonblocking_accept is a coroutine. I was stuck in the C way of doing non blocking, and I was wondering what was the magic behind the examples. Perfect explanation, now I understand. Still, I also wonder how theses "nonblocking_" functions are meant to be implemented. But it's another topic. Thanks. From grant.b.edwards at gmail.com Sat Jan 20 10:53:31 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 20 Jan 2018 15:53:31 +0000 (UTC) Subject: How to use asyncore with SSL? References: Message-ID: On 2018-01-18, Grant Edwards wrote: [regarding secure-smtpd -- a module based on smtpd and asyncore] > That makes the SSL support pretty much useless. > > I'm trying to fix that, but I can't find any information or > documentation about using asyncore with SSL. Asyncore seems to be based on fundamental assumptions that aren't true for non-blocking ssl sockets. After looking into it for a couple hours, the way you use ssl with asyncore is like this: pid = subprocess.Pipe(["stunnel","stunnel.conf"]).pid # insert asyncore app here os.kill(pid,signal.SIGTERM) -- Grant From rustompmody at gmail.com Sat Jan 20 11:06:12 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Sat, 20 Jan 2018 08:06:12 -0800 (PST) Subject: Why is there no functional xml? Message-ID: Looking around for how to create (l)xml one sees typical tutorials like this: https://www.blog.pythonlibrary.org/2013/04/30/python-101-intro-to-xml-parsing-with-elementtree/ Given the requirement to build up this xml: 1181251680 040000008200E000 1181572063 1800 Bring pizza home the way I would rather do it is thus: [Note in actual practice the 'contents' such as 1181251680 etc would come from suitable program variables/function-calls ] ex = Ea("zAppointments", {'reminder':'15'}, E("appointment", En("begin", 1181251680), Et("uid", "040000008200E000"), En("alarmTime", 1181572063), E("state"), E("location"), En("duration",1800), Et("subject", "Bring pizza home"))) with the following obvious definitions: [The function names are short so that the above becomes correspondingly readable] from lxml.etree import Element def Ea(tag, attrib=None, *subnodes): "xml node constructor" root = Element(tag, attrib) for n in subnodes: root.append(n) return root def E(tag, *subnodes): "Like E but without attributes" root = Element(tag) for n in subnodes: root.append(n) return root def Et(tag, text): "A pure text node" root = E(tag) root.text = text return root def En(tag, text): "A node containing a integer" root = E(tag) root.text = str(text) return root This approach seems so obvious that I find it hard to believe its not there somewhere? Am I missing something?? From jugurtha.hadjar at gmail.com Sat Jan 20 11:40:28 2018 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Sat, 20 Jan 2018 17:40:28 +0100 Subject: Speeding up the implementation of Stochastic Gradient Ascent in Python In-Reply-To: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> References: <916c68d5-0d6e-4bf9-b29d-5610bc8abd96@googlegroups.com> Message-ID: <5e8cac99-b8d9-6539-e289-1596aa042740@gmail.com> Aside from the obvious imports I've added (import numpy as np, etc), there are still undefined objects (`names`, `times_`, `feat`, `bind_ind`) and indentation errors. Can you post a *working* code to be sped up and a benchmarking test? As you're using Python 2, you can improve the code with the following: ?- Use `xrange` in place of `range`. ?- Printing in loops dramatically slows things down. ?- Loop directly over an iterable (and maybe use `collections.defaultdict`): ??????? # This: ??????? def pos_per_user_(self): ??????????? pos_per_user = {} ??????????? for k, v in self.userItems.items(): ??????????????? for i in range(len(self.userItems[k])): ??????????????????? item__ = self.userItems[k][i][0] ??????????????????? if k not in pos_per_user: ??????????????????????? pos_per_user[k] = [item__] ??????????????????? else: ??????????????????????? pos_per_user[k].append(item__) ??????????? return pos_per_user ??????? # Becomes like this: ??????? from collections import defaultdict ??????? def pos_per_user(self): ??????????? pos_per_user = defaultdict(list) ??????????? for k, v in self.userItems.iteritems(): ??????????????? for item in items: ??????????????????? pos_per_user[k].append(item[0]) ??????????? return pos_per_user ? - You also have too many list creations inside loops, which is also slow: ??????? # If you do nothing, use xrange here for it will not construct ??????? # a list. Avoid materializing a list if all you do is iterate. ??????? for bin in range(10): ??????????? for i in range(len(self.list_of_items)): ??????????????? for k in range(self.K2): ??????????????????? for f in range(4096): -- ~ Jugurtha Hadjar, From marko at pacujo.net Sat Jan 20 12:10:07 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Jan 2018 19:10:07 +0200 Subject: How to use asyncore with SSL? References: Message-ID: <87po6478y8.fsf@elektro.pacujo.net> Grant Edwards : > Asyncore seems to be based on fundamental assumptions that aren't true > for non-blocking ssl sockets. Pot calling kettle black. OpenSSL isn't the easiest beast to deal with, but I have been able to abstract it (in C) so it behaves very close to TCP. The one blemish is in the fact that the TLS protocol does not support a half-duplex connection. Shame. The WANT_READ/WANT_WRITE silliness should be abstracted out of the non-blocking TLS library so the application doesn't need to know anything about it. Marko From blue4comet at gmail.com Sat Jan 20 12:16:55 2018 From: blue4comet at gmail.com (Jim Sadler) Date: Sat, 20 Jan 2018 12:16:55 -0500 Subject: Can't get python running Message-ID: I downloaded python 3.6.4 and although everything about the installation seems correct (path, file size, checking on cmd to see if file installed correctly-it is) the only window that will pop up when I attempt to run it is the set-up window offering to modify, restore, or uninstall. I have uninstalled and re-installed the program several times, run the repair function many times and nothing changes. Would someone please help me or direct me to someone who can? Thank you. From grant.b.edwards at gmail.com Sat Jan 20 13:22:49 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 20 Jan 2018 18:22:49 +0000 (UTC) Subject: How to use asyncore with SSL? References: <87po6478y8.fsf@elektro.pacujo.net> Message-ID: On 2018-01-20, Marko Rauhamaa wrote: > Grant Edwards : > >> Asyncore seems to be based on fundamental assumptions that aren't true >> for non-blocking ssl sockets. > > Pot calling kettle black. > > OpenSSL isn't the easiest beast to deal with, but I have been able to > abstract it (in C) so it behaves very close to TCP. The one blemish is > in the fact that the TLS protocol does not support a half-duplex > connection. Shame. > > The WANT_READ/WANT_WRITE silliness should be abstracted out of the > non-blocking TLS library so the application doesn't need to know > anything about it. I won't argue with that. I think that non-blocking ssl-wrapped sockets _should_ have the same select/poll/send/recv API/semantics that normal sockets do. I thought about writing my own wrapped-ssl-socket class that does that, but using stunnel was just so much easier. If you _did_ want to wrap sockets like that, I think you'd need to actually run a thread to deal with the SSL socket and provide a "proxy" socket or pipe for use with select/poll. Basically you'd be doing what stunnel does only doing it in-process. -- Grant From marko at pacujo.net Sat Jan 20 16:10:08 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sat, 20 Jan 2018 23:10:08 +0200 Subject: How to use asyncore with SSL? References: <87po6478y8.fsf@elektro.pacujo.net> Message-ID: <87h8rg6xu7.fsf@elektro.pacujo.net> Grant Edwards : > On 2018-01-20, Marko Rauhamaa wrote: >> OpenSSL isn't the easiest beast to deal with, but I have been able to >> abstract it (in C) so it behaves very close to TCP. The one blemish >> is in the fact that the TLS protocol does not support a half-duplex >> connection. Shame. >> >> The WANT_READ/WANT_WRITE silliness should be abstracted out of the >> non-blocking TLS library so the application doesn't need to know >> anything about it. > > I won't argue with that. I think that non-blocking ssl-wrapped > sockets _should_ have the same select/poll/send/recv API/semantics > that normal sockets do. I thought about writing my own > wrapped-ssl-socket class that does that, but using stunnel was just so > much easier. If you _did_ want to wrap sockets like that, I think > you'd need to actually run a thread to deal with the SSL socket and > provide a "proxy" socket or pipe for use with select/poll. > > Basically you'd be doing what stunnel does only doing it in-process. Stunnel is fine for many applications but not for our needs. Also, a subsidiary thread is not necessary. Everything can be done within an async framework (in C anyway). Marko From tjreedy at udel.edu Sat Jan 20 16:49:29 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 20 Jan 2018 16:49:29 -0500 Subject: Can't get python running In-Reply-To: References: Message-ID: On 1/20/2018 12:16 PM, Jim Sadler wrote: > I downloaded python 3.6.4 and although everything about the installation > seems correct (path, file size, checking on cmd to see if file installed > correctly-it is) the only window that will pop up when I attempt to run it > is the set-up window offering to modify, restore, or uninstall. The only time that should pop up is when you rerun the installer instead of the installed binary. (Or when you invoke it from the installed programs control panel.) *Exactly* how are you trying to run the installed Python. What version of Windows? -- Terry Jan Reedy From ikorot01 at gmail.com Sat Jan 20 16:53:21 2018 From: ikorot01 at gmail.com (Igor Korot) Date: Sat, 20 Jan 2018 15:53:21 -0600 Subject: Can't get python running In-Reply-To: References: Message-ID: Hi, Jim, On Sat, Jan 20, 2018 at 11:16 AM, Jim Sadler wrote: > I downloaded python 3.6.4 and although everything about the installation > seems correct (path, file size, checking on cmd to see if file installed > correctly-it is) the only window that will pop up when I attempt to run it > is the set-up window offering to modify, restore, or uninstall. I have > uninstalled and re-installed the program several times, run the repair > function many times and nothing changes. Would someone please help me or > direct me to someone who can? How exactly are attempting to run python? Thank you. > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list From grant.b.edwards at gmail.com Sat Jan 20 19:26:46 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 21 Jan 2018 00:26:46 +0000 (UTC) Subject: How to use asyncore with SSL? References: <87po6478y8.fsf@elektro.pacujo.net> <87h8rg6xu7.fsf@elektro.pacujo.net> Message-ID: On 2018-01-20, Marko Rauhamaa wrote: > Grant Edwards : [...] >> I won't argue with that. I think that non-blocking ssl-wrapped >> sockets _should_ have the same select/poll/send/recv API/semantics >> that normal sockets do. I thought about writing my own >> wrapped-ssl-socket class that does that, but using stunnel was just so >> much easier. If you _did_ want to wrap sockets like that, I think >> you'd need to actually run a thread to deal with the SSL socket and >> provide a "proxy" socket or pipe for use with select/poll. >> >> Basically you'd be doing what stunnel does only doing it in-process. > > Stunnel is fine for many applications but not for our needs. Also, a > subsidiary thread is not necessary. Everything can be done within an > async framework (in C anyway). Of course it can. But (ISTM) either the async framework has to be SSL-aware, or the sockets have to be wrapped and "proxied" using another socket or pipe -- I don't see how else you can provide something that has the same select/poll and send/recv semantacs that an unwrapped, non-blocking socket provides. -- Grant From bc at freeuk.com Sat Jan 20 20:08:01 2018 From: bc at freeuk.com (bartc) Date: Sun, 21 Jan 2018 01:08:01 +0000 Subject: Can't get python running In-Reply-To: References: Message-ID: On 20/01/2018 17:16, Jim Sadler wrote: > I downloaded python 3.6.4 and although everything about the installation > seems correct (path, file size, checking on cmd to see if file installed > correctly-it is) What do you mean by 'checking on cmd'? I would install it somewhere like c:\python364, then it is easier to find than having it buried in its default installation path (I assume this is Windows). If it's installed properly, then python.exe should be present in c:\python364, and you can start it with: c:\python364\python or: c: cd \python364 python I assume you can open a command prompt. If not in python364, then substitute the actual installation path. Or click Start, and type python.exe into the search box. If it finds it, click it. -- bartc From rosuav at gmail.com Sat Jan 20 20:21:40 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jan 2018 12:21:40 +1100 Subject: Can't get python running In-Reply-To: References: Message-ID: On Sun, Jan 21, 2018 at 12:08 PM, bartc wrote: > On 20/01/2018 17:16, Jim Sadler wrote: >> >> I downloaded python 3.6.4 and although everything about the installation >> seems correct (path, file size, checking on cmd to see if file installed >> correctly-it is) > > > What do you mean by 'checking on cmd'? > > I would install it somewhere like c:\python364, then it is easier to find > than having it buried in its default installation path (I assume this is > Windows). python36 is a better choice of name, as it doesn't cause problems when you upgrade to a new bugfix release. But there are permissions issues with dropping stuff straight into the root directory, which is why the default installers now put Python into Program Files. Jim, let the installer put it where it wants to, and make sure you've added it to PATH. Then you should be able to type "py" to start Python. ChrisA From bc at freeuk.com Sat Jan 20 20:40:12 2018 From: bc at freeuk.com (bartc) Date: Sun, 21 Jan 2018 01:40:12 +0000 Subject: Can't get python running In-Reply-To: References: Message-ID: <0cS8C.437200$QX.195480@fx10.am4> On 21/01/2018 01:21, Chris Angelico wrote: > On Sun, Jan 21, 2018 at 12:08 PM, bartc wrote: >> On 20/01/2018 17:16, Jim Sadler wrote: >>> >>> I downloaded python 3.6.4 and although everything about the installation >>> seems correct (path, file size, checking on cmd to see if file installed >>> correctly-it is) >> >> >> What do you mean by 'checking on cmd'? >> >> I would install it somewhere like c:\python364, then it is easier to find >> than having it buried in its default installation path (I assume this is >> Windows). > > python36 is a better choice of name That's what I use; I assumed everyone else used the 3-digit version in the path. , as it doesn't cause problems when > you upgrade to a new bugfix release. But there are permissions issues > with dropping stuff straight into the root directory, which is why the > default installers now put Python into Program Files. It's not in the root, the executable will be: c:\python36\python.exe Not c:\python.exe. Yes, Windows allows you to have your own directories within /. It's not Unix. > Jim, let the installer put it where it wants to, and make sure you've > added it to PATH. Then you should be able to type "py" to start > Python. If I try to install 3.7 (as I already have 3.6) it suggests putting it in: c:\Users\users\AppData\Local\Programs\Python\Python37-32 Not actually a snappy path if you need to get there in a hurry to sort out problems. That is, in a location 7 levels deep. Get it working anywhere first to find out what the problem is. -- bartc From steve+comp.lang.python at pearwood.info Sat Jan 20 20:46:27 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 21 Jan 2018 01:46:27 +0000 (UTC) Subject: Can't get python running References: Message-ID: On Sun, 21 Jan 2018 12:21:40 +1100, Chris Angelico wrote: > Jim, let the installer put it where it wants to, and make sure you've > added it to PATH. Then you should be able to type "py" to start Python. Shouldn't the installer ensure that it puts "py" somewhere on the path? -- Steve From steve+comp.lang.python at pearwood.info Sat Jan 20 20:47:26 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 21 Jan 2018 01:47:26 +0000 (UTC) Subject: Is this the right way to write a codec error handler? References: Message-ID: On Sat, 20 Jan 2018 12:57:45 +0200, Serhiy Storchaka wrote: > Just `end` instead of `end+1`. Oops! > And it is safer to use `bytes.decode(obj[start:end], 'latin1')` or > `str(obj[start:end], 'latin1')` instead of > `obj[start:end].decode('latin1')`. Just for the case if obj has > overridden decode() method. > > Otherwise LGTM. Thanks. -- Steve From rosuav at gmail.com Sat Jan 20 20:52:10 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jan 2018 12:52:10 +1100 Subject: Can't get python running In-Reply-To: <0cS8C.437200$QX.195480@fx10.am4> References: <0cS8C.437200$QX.195480@fx10.am4> Message-ID: On Sun, Jan 21, 2018 at 12:40 PM, bartc wrote: > On 21/01/2018 01:21, Chris Angelico wrote: >> >> On Sun, Jan 21, 2018 at 12:08 PM, bartc wrote: >>> >>> On 20/01/2018 17:16, Jim Sadler wrote: >>>> >>>> >>>> I downloaded python 3.6.4 and although everything about the installation >>>> seems correct (path, file size, checking on cmd to see if file installed >>>> correctly-it is) >>> >>> >>> >>> What do you mean by 'checking on cmd'? >>> >>> I would install it somewhere like c:\python364, then it is easier to find >>> than having it buried in its default installation path (I assume this is >>> Windows). >> >> >> python36 is a better choice of name > > > That's what I use; I assumed everyone else used the 3-digit version in the > path. Terrible assumption, given that it's not the default NOR a good idea :) > , as it doesn't cause problems when >> >> you upgrade to a new bugfix release. But there are permissions issues >> with dropping stuff straight into the root directory, which is why the >> default installers now put Python into Program Files. > > > It's not in the root, the executable will be: > > c:\python36\python.exe > > Not c:\python.exe. Yes, Windows allows you to have your own directories > within /. It's not Unix. I meant putting the pythonXY directory straight into the root. Yes, Windows allows it... but only if you are administrator. I think. Depends on the Windows version. And just FYI, Unix allows you to have your own directories within / too, so I don't know what your point is. Both OSes - recent versions, at least - restrict the creation of directories and files straight in the root. >> Jim, let the installer put it where it wants to, and make sure you've >> added it to PATH. Then you should be able to type "py" to start >> Python. > > > If I try to install 3.7 (as I already have 3.6) it suggests putting it in: > > c:\Users\users\AppData\Local\Programs\Python\Python37-32 > > Not actually a snappy path if you need to get there in a hurry to sort out > problems. That is, in a location 7 levels deep. > > Get it working anywhere first to find out what the problem is. > Get it working in the default location before you change things. The %PATH% environment variable exists to save you from typing seven levels of directory names. ChrisA From rosuav at gmail.com Sat Jan 20 20:53:48 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jan 2018 12:53:48 +1100 Subject: Can't get python running In-Reply-To: References: Message-ID: On Sun, Jan 21, 2018 at 12:46 PM, Steven D'Aprano wrote: > On Sun, 21 Jan 2018 12:21:40 +1100, Chris Angelico wrote: > >> Jim, let the installer put it where it wants to, and make sure you've >> added it to PATH. Then you should be able to type "py" to start Python. > > Shouldn't the installer ensure that it puts "py" somewhere on the path? > Only if you tick the box to do that (or at least, that was the case a few versions ago). You don't have to do it manually, but you do have to make sure the box is ticked. (Which might just mean making sure you don't UNtick it; I don't know what the default is.) ChrisA From marko at pacujo.net Sun Jan 21 05:27:11 2018 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 21 Jan 2018 12:27:11 +0200 Subject: How to use asyncore with SSL? References: <87po6478y8.fsf@elektro.pacujo.net> <87h8rg6xu7.fsf@elektro.pacujo.net> Message-ID: <87607v7bi8.fsf@elektro.pacujo.net> Grant Edwards : > On 2018-01-20, Marko Rauhamaa wrote: >> Also, a subsidiary thread is not necessary. Everything can be done >> within an async framework (in C anyway). > > Of course it can. But (ISTM) either the async framework has to be > SSL-aware, or the sockets have to be wrapped and "proxied" using > another socket or pipe -- I don't see how else you can provide > something that has the same select/poll and send/recv semantacs that > an unwrapped, non-blocking socket provides. If you mean that user-space software can't create new file descriptor types, you would be painfully correct. The read(2) and write(2) (as well as recv(2) and send(2)) system calls necessarily cross into the kernel-space. User-space file descriptors can be emulated but generally that requires the use of auxiliary processes or threads. If you relax your requirement a bit and let the async framework define its own I/O methods, the magic becomes possible: you can make TLS an asynchronous drop-in replacement for TCP. (I suppose every async framework defines its own I/O methods.) If all you need is a file-descriptor-based notification scheme, Linux can manage that through epollfd, eventfd and timerfd. My async framework does not offer such a notification interface but instead uses callback functions. Marko From p.f.moore at gmail.com Sun Jan 21 05:57:51 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 21 Jan 2018 10:57:51 +0000 Subject: Can't get python running In-Reply-To: References: Message-ID: On 21 January 2018 at 01:53, Chris Angelico wrote: > On Sun, Jan 21, 2018 at 12:46 PM, Steven D'Aprano > wrote: >> On Sun, 21 Jan 2018 12:21:40 +1100, Chris Angelico wrote: >> >>> Jim, let the installer put it where it wants to, and make sure you've >>> added it to PATH. Then you should be able to type "py" to start Python. >> >> Shouldn't the installer ensure that it puts "py" somewhere on the path? >> > > Only if you tick the box to do that (or at least, that was the case a > few versions ago). You don't have to do it manually, but you do have > to make sure the box is ticked. (Which might just mean making sure you > don't UNtick it; I don't know what the default is.) The "py" launcher is always added to PATH. It's installed by default (and always has been I believe) although there is a checkbox you can untick if you don't want to install it. What's less reliable is whether the "python.exe" executable is on PATH. That's been on and off by default depending on the Python version, and is currently (3.6) off, for reasons to do with the precedence of the user and system parts of the PATH variable (which are important, but that's not much help to people who wish that after you installed Python, typing "python" at the command prompt would just work...) Paul From __peter__ at web.de Sun Jan 21 06:20:36 2018 From: __peter__ at web.de (Peter Otten) Date: Sun, 21 Jan 2018 12:20:36 +0100 Subject: Why is there no functional xml? References: Message-ID: Rustom Mody wrote: > Looking around for how to create (l)xml one sees typical tutorials like > this: > > https://www.blog.pythonlibrary.org/2013/04/30/python-101-intro-to-xml-parsing-with-elementtree/ > > > > Given the requirement to build up this xml: > > > 1181251680 > 040000008200E000 > 1181572063 > > > 1800 > Bring pizza home > > > > > > the way I would rather do it is thus: > > [Note in actual practice the 'contents' such as 1181251680 etc would come > from suitable program variables/function-calls > ] > > > ex = Ea("zAppointments", {'reminder':'15'}, > E("appointment", > En("begin", 1181251680), > Et("uid", "040000008200E000"), > En("alarmTime", 1181572063), > E("state"), > E("location"), > En("duration",1800), > Et("subject", "Bring pizza home"))) > > > with the following obvious definitions: > > [The function names are short so that the above becomes correspondingly > [readable] > > > from lxml.etree import Element > > def Ea(tag, attrib=None, *subnodes): > "xml node constructor" > root = Element(tag, attrib) > for n in subnodes: > root.append(n) > return root > > def E(tag, *subnodes): > "Like E but without attributes" > root = Element(tag) > for n in subnodes: > root.append(n) > return root > > def Et(tag, text): > "A pure text node" > root = E(tag) > root.text = text > return root > > def En(tag, text): > "A node containing a integer" > root = E(tag) > root.text = str(text) > return root > > > This approach seems so obvious that I find it hard to believe its not > there somewhere? Am I missing something?? lxml.objectify? >>> from lxml import etree >>> from lxml.objectify import E >>> appointments = E.appointments( ... E.appointment( ... E.begin(1181251680), ... E.uid("040000008200E000"), ... E.alarmTime(1181572063), ... E.state(), ... E.location(), ... E.duration(1800), ... E.subject("Bring pizza home") ... ), ... reminder="15" ... ) >>> print(etree.tostring(appointments, pretty_print=True, encoding=str)) 1181251680 040000008200E000 1181572063 1800 Bring pizza home >>> Personally I'd probably avoid the extra layer and write a function that directly maps dataclasses or database records to xml using the conventional elementtree API. From tjol at tjol.eu Sun Jan 21 06:55:55 2018 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 21 Jan 2018 12:55:55 +0100 Subject: Can't get python running In-Reply-To: References: <0cS8C.437200$QX.195480@fx10.am4> Message-ID: <1f46bb89-c555-d0f1-5323-0fb170bfd464@tjol.eu> On 21/01/18 02:52, Chris Angelico wrote: > On Sun, Jan 21, 2018 at 12:40 PM, bartc wrote: >> On 21/01/2018 01:21, Chris Angelico wrote: >>> >>> On Sun, Jan 21, 2018 at 12:08 PM, bartc wrote: >>>> >>>> On 20/01/2018 17:16, Jim Sadler wrote: >>>>> >>>>> >>>>> I downloaded python 3.6.4 and although everything about the installation >>>>> seems correct (path, file size, checking on cmd to see if file installed >>>>> correctly-it is) >>>> >>>> >>>> >>>> What do you mean by 'checking on cmd'? >>>> >>>> I would install it somewhere like c:\python364, then it is easier to find >>>> than having it buried in its default installation path (I assume this is >>>> Windows). >>> >>> >>> python36 is a better choice of name >> >> >> That's what I use; I assumed everyone else used the 3-digit version in the >> path. > > Terrible assumption, given that it's not the default NOR a good idea :) > >> , as it doesn't cause problems when >>> >>> you upgrade to a new bugfix release. But there are permissions issues >>> with dropping stuff straight into the root directory, which is why the >>> default installers now put Python into Program Files. >> >> >> It's not in the root, the executable will be: >> >> c:\python36\python.exe >> >> Not c:\python.exe. Yes, Windows allows you to have your own directories >> within /. It's not Unix. > > I meant putting the pythonXY directory straight into the root. Yes, > Windows allows it... but only if you are administrator. I think. AFAIK you also have to be administrator to install things in r"C:\Program Files". If you install something user-only, (which more software supports than many people think) it ends to somewhere in the user directory. I always thought that installing things in "Program Files" rather than the root directory was just good manners (just like installing things in /usr/local or /opt on GNU/Linux) > Depends on the Windows version. And just FYI, Unix allows you to have > your own directories within / too, so I don't know what your point is. > Both OSes - recent versions, at least - restrict the creation of > directories and files straight in the root. > >>> Jim, let the installer put it where it wants to, and make sure you've >>> added it to PATH. Then you should be able to type "py" to start >>> Python. >> >> >> If I try to install 3.7 (as I already have 3.6) it suggests putting it in: >> >> c:\Users\users\AppData\Local\Programs\Python\Python37-32 >> >> Not actually a snappy path if you need to get there in a hurry to sort out >> problems. That is, in a location 7 levels deep. >> >> Get it working anywhere first to find out what the problem is. >> > > Get it working in the default location before you change things. The > %PATH% environment variable exists to save you from typing seven > levels of directory names. > > ChrisA > From bc at freeuk.com Sun Jan 21 07:06:07 2018 From: bc at freeuk.com (bartc) Date: Sun, 21 Jan 2018 12:06:07 +0000 Subject: Can't get python running In-Reply-To: References: <0cS8C.437200$QX.195480@fx10.am4> Message-ID: On 21/01/2018 01:52, Chris Angelico wrote: > On Sun, Jan 21, 2018 at 12:40 PM, bartc wrote: >> Get it working anywhere first to find out what the problem is. >> > > Get it working in the default location before you change things. The > %PATH% environment variable exists to save you from typing seven > levels of directory names. That's fine when everything works as it should. It's less ideal depending on %PATH% when trying to figure out what's gone wrong, as you don't know if it's set up properly, or whether the new path is appended or prepended to %PATH%, or whether there is some other py.exe or python.exe lurking in a search path that is checked first. Also, if you have a command prompt already open, then its %PATH% will not be updated. -- bart From rustompmody at gmail.com Sun Jan 21 10:24:31 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Sun, 21 Jan 2018 07:24:31 -0800 (PST) Subject: Why is there no functional xml? In-Reply-To: References: Message-ID: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> On Sunday, January 21, 2018 at 4:51:34 PM UTC+5:30, Peter Otten wrote: > Rustom Mody wrote: > > > Looking around for how to create (l)xml one sees typical tutorials like > > this: > > > > https://www.blog.pythonlibrary.org/2013/04/30/python-101-intro-to-xml-parsing-with-elementtree/ > > > > > > > > Given the requirement to build up this xml: > > > > > > 1181251680 > > 040000008200E000 > > 1181572063 > > > > > > 1800 > > Bring pizza home > > > > > > > > > > > > the way I would rather do it is thus: > > > > [Note in actual practice the 'contents' such as 1181251680 etc would come > > from suitable program variables/function-calls > > ] > > > > > > ex = Ea("zAppointments", {'reminder':'15'}, > > E("appointment", > > En("begin", 1181251680), > > Et("uid", "040000008200E000"), > > En("alarmTime", 1181572063), > > E("state"), > > E("location"), > > En("duration",1800), > > Et("subject", "Bring pizza home"))) > > > > > > with the following obvious definitions: > > > > [The function names are short so that the above becomes correspondingly > > [readable] > > > > > > from lxml.etree import Element > > > > def Ea(tag, attrib=None, *subnodes): > > "xml node constructor" > > root = Element(tag, attrib) > > for n in subnodes: > > root.append(n) > > return root > > > > def E(tag, *subnodes): > > "Like E but without attributes" > > root = Element(tag) > > for n in subnodes: > > root.append(n) > > return root > > > > def Et(tag, text): > > "A pure text node" > > root = E(tag) > > root.text = text > > return root > > > > def En(tag, text): > > "A node containing a integer" > > root = E(tag) > > root.text = str(text) > > return root > > > > > > This approach seems so obvious that I find it hard to believe its not > > there somewhere? Am I missing something?? > > lxml.objectify? > > >>> from lxml import etree > >>> from lxml.objectify import E > >>> appointments = E.appointments( > ... E.appointment( > ... E.begin(1181251680), > ... E.uid("040000008200E000"), > ... E.alarmTime(1181572063), > ... E.state(), > ... E.location(), > ... E.duration(1800), > ... E.subject("Bring pizza home") > ... ), > ... reminder="15" > ... ) > >>> print(etree.tostring(appointments, pretty_print=True, encoding=str)) > xmlns:xsd="http://www.w3.org/2001/XMLSchema" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" reminder="15"> > > 1181251680 > 040000008200E000 > 1181572063 > > > 1800 > Bring pizza home > > > > >>> Nice! I almost liked it? Then noticed that the attribute dict is in the wrong place Here's the same without any namespace malarky: >>> E = objectify.ElementMaker(annotate=False,namespace=None,nsmap=None) >>> appointments = E.appointments( E.appointment( E.begin(1181251680), E.uid("040000008200E000"), E.alarmTime(1181572063), E.state(), E.location(), E.duration(1800), E.subject("Bring pizza home") ), reminder="15" ) >>> print(et.tostring(appointments,pretty_print=1).decode('ascii')) 1181251680 040000008200E000 1181572063 1800 Bring pizza home >>> Its obviously easier in python to put optional/vararg parameters on the right side rather than on the left of a parameter list. But its not impossible to get it in the desired order ? one just has to 'hand-parse' the parameter list received as a *param Thusly: appointments = E.appointments( {"reminder":"15"}, E.appointment( E.begin(1181251680), E.uid("040000008200E000"), E.alarmTime(1181572063), E.state(), E.location(), E.duration(1800), E.subject("Bring pizza home") ) ) > > Personally I'd probably avoid the extra layer and write a function that > directly maps dataclasses or database records to xml using the conventional > elementtree API. I dont understand? [I find the OO/imperative style of making a half-done node and then throwing piece-by-piece of contents in/at it highly aggravating] From eryksun at gmail.com Mon Jan 22 03:11:19 2018 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Jan 2018 08:11:19 +0000 Subject: Can't get python running In-Reply-To: References: Message-ID: On Sun, Jan 21, 2018 at 10:57 AM, Paul Moore wrote: > > The "py" launcher is always added to PATH. It's installed by default > (and always has been I believe) although there is a checkbox you can > untick if you don't want to install it. Yes, we can choose to not install the launcher. However, if we select to install it, it seems we don't get to choose the install path or whether it's added to PATH. When installed for the current user, the launcher installs in "%LocalAppData%\Python\Launcher". This directory is added to PATH even if we don't select "add Python to environment variables". When installed for all users it goes in "%SystemRoot%", which is already in the default system PATH and also implicitly searched by WinAPI CreateProcess even if PATH isn't defined. The launcher's primary purpose is to handle the .py[w] file association and support shebangs in scripts, and to that end it does not have to be in PATH. But it has also become a convenient way to manage multiple Python installations from the command line, especially for `py -X[.Y[-32]] -m pip` and (for 3.5+) `py -X[.Y[-32]] -m venv` As to installing Windows applications in the root of the system drive, that's fine for a single-user or personal system, if it's what you prefer. The default DACL of the system drive's root directory allows authenticated users to create directories that all authenticated users have the right to modify. This is too permissive in general, so starting with 3.5 the installer stopped using C:\PythonXY as the default target. Now, all-users installs default to "%ProgramFiles%\PythonXY" or "%ProgramFiles(x86)%\PythonXY-32", and current-user installs default to "%LocalAppData%\Programs\Python\PythonXY[-32]". %LocalAppData% expands to "%UserProfile%\AppData\Local". By default, a user's profile directory is accessible only to the user, administrators, and SYSTEM. From ken.py at gameofy.com Mon Jan 22 03:22:16 2018 From: ken.py at gameofy.com (ken.py at gameofy.com) Date: Mon, 22 Jan 2018 00:22:16 -0800 Subject: exec and traceback Message-ID: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> I'm using exec() to run a (multi-line) string of python code. If an exception occurs, I get a traceback containing a stack frame for the string. I've labeled the code object with a "file name" so I can identify it easily, and when I debug, I find that I can interact with the context of that stack frame, which is pretty handy. What I would like to also be able to do is make the code string visible to the debugger so I can look at and step through the code in the string as if it were from a python file. Lest this topic forks into a security discussion, I'll just add that for my purposes the data source is trusted. If you really want to talk about the security of using exec and eval, fine, but start another thread (BTW, I've written a simple secure eval()).... Thanks in advance, Ken From jkn_gg at nicorp.f9.co.uk Mon Jan 22 05:22:11 2018 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Mon, 22 Jan 2018 02:22:11 -0800 (PST) Subject: 'QMessageBox' object has no attribute 'setCheckBox' on recent pull Message-ID: Hi Edward Seen after a recent pull: D:\winapps\leo-editor>python launchLeo.py can not import leo.plugins.importers.wikimarkup can not import leo.plugins.importers.wikimarkup reading settings in D:\winapps\leo-editor\leo\config\leoSettings.leo reading settings in C:\Users\jnicoll\.leo\myLeoSettings.leo Note: sys.stdout.encoding is not UTF-8 See: https://stackoverflow.com/questions/14109024 Leo 5.6, build 20180121141749, Sun Jan 21 14:17:49 CST 2018 Git repo info: branch = master, commit = 0a36546f5664 Python 2.7.2, PyQt version 4.8.6 Windows 7 x86 (build 6.1.7601) SP1 ** isPython3: False ** caching enabled reading settings in C:\Users\jnicoll\.leo\workbook.leo Traceback (most recent call last): File "launchLeo.py", line 8, in leo.core.runLeo.run() File "D:\winapps\leo-editor\leo\core\runLeo.py", line 74, in run g.app.loadManager.load(fileName, pymacs) File "D:\winapps\leo-editor\leo\core\leoApp.py", line 2216, in load g.app.gui.runMainLoop() File "D:\winapps\leo-editor\leo\plugins\qt_gui.py", line 1073, in runMainLoop g.app.gui.show_tips() File "D:\winapps\leo-editor\leo\plugins\qt_gui.py", line 1141, in show_tips m.setCheckBox(cb) AttributeError: 'QMessageBox' object has no attribute 'setCheckBox' QMessageBox::setCheckBox() appears to have been introduced in Qt v5.2. Is this a dependancy now? http://leoeditor.com/installing.html#dependencies still says Qt4 or Qt5 Thanks, Jon N From jkn_gg at nicorp.f9.co.uk Mon Jan 22 07:55:19 2018 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Mon, 22 Jan 2018 04:55:19 -0800 (PST) Subject: 'QMessageBox' object has no attribute 'setCheckBox' on recent pull In-Reply-To: References: Message-ID: <984c7725-7142-4f8f-8241-e85b1b04e31a@googlegroups.com> On Monday, January 22, 2018 at 10:22:36 AM UTC, jkn wrote: [...] oops, wrong group, sorry! From rosuav at gmail.com Mon Jan 22 08:38:11 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Jan 2018 00:38:11 +1100 Subject: exec and traceback In-Reply-To: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> References: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> Message-ID: On Mon, Jan 22, 2018 at 7:22 PM, wrote: > > > I'm using exec() to run a (multi-line) string of python code. If an > exception occurs, I get a traceback containing a stack frame for the string. > I've labeled the code object with a "file name" so I can identify it easily, > and when I debug, I find that I can interact with the context of that stack > frame, which is pretty handy. > > What I would like to also be able to do is make the code string visible to > the debugger so I can look at and step through the code in the string as if > it were from a python file. If you're interacting with a stack frame, you should have access to its locals and globals, right? Worst case, all you need is this: _exec = exec def exec(source): return _exec(source) There, now you guarantee that you have a stack frame with the source code visible in it. If you control the code which calls exec, you could just do the same thing there: source = ... exec(source) Either way, it should be accessible from the frame's f_locals. > Lest this topic forks into a security discussion, I'll just add that for my > purposes the data source is trusted. If you really want to talk about the > security of using exec and eval, fine, but start another thread (BTW, I've > written a simple secure eval()).... That would indeed be another thread, but I can guarantee you that your "simple secure eval" is either *extremely* simple (with restricted data types and operations) or not secure. ChrisA From ned at nedbatchelder.com Mon Jan 22 08:40:52 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Mon, 22 Jan 2018 08:40:52 -0500 Subject: exec and traceback In-Reply-To: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> References: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> Message-ID: On 1/22/18 3:22 AM, ken.py at gameofy.com wrote: > (BTW, I've written a simple secure eval()) You have accurately guessed our interest!? Would you mind starting a new thread to show us your simple secure eval? --Ned. From jorge.conrado at cptec.inpe.br Mon Jan 22 10:17:54 2018 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Mon, 22 Jan 2018 13:17:54 -0200 Subject: close figure after plot Message-ID: <25e55e6cc948b038f3a4ebae5cc54c1e@cptec.inpe.br> Hi, I have several satellite data (500 images). I read it and plot using plt.show(). I would like know how can I delete the window after I save the image. I use plt.close(), plt.close('all') but these options didn't work. Thanks, Conrado From codydaviestv at gmail.com Mon Jan 22 10:37:14 2018 From: codydaviestv at gmail.com (codydaviestv at gmail.com) Date: Mon, 22 Jan 2018 07:37:14 -0800 (PST) Subject: Installing "kitchen" module Message-ID: So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server I have everything working up to the point where I run it and it tells me this: "Please install the Kitchen module. Please install or update the Requests module." One suggestion was that I try "import kitchen", but that gives me this error: "Traceback : File "", line 1, in ImportError: No module named kitchen" So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? Any help would be appreciated. P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 From breamoreboy at gmail.com Mon Jan 22 11:10:33 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 22 Jan 2018 08:10:33 -0800 (PST) Subject: Installing "kitchen" module In-Reply-To: References: Message-ID: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: > > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server > > I have everything working up to the point where I run it and it tells me this: > > "Please install the Kitchen module. > Please install or update the Requests module." > > One suggestion was that I try "import kitchen", but that gives me this error: > > "Traceback : > File "", line 1, in > ImportError: No module named kitchen" > > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? > > Any help would be appreciated. > > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 Forget sudo as that's a *nix command. From the command line you should be able to run:- pip install kitchen pip install requests -- Kindest regards. Mark Lawrence. From codydaviestv at gmail.com Mon Jan 22 11:19:51 2018 From: codydaviestv at gmail.com (codydaviestv at gmail.com) Date: Mon, 22 Jan 2018 08:19:51 -0800 (PST) Subject: Installing "kitchen" module In-Reply-To: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> Message-ID: <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: > On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: > > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: > > > > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server > > > > I have everything working up to the point where I run it and it tells me this: > > > > "Please install the Kitchen module. > > Please install or update the Requests module." > > > > One suggestion was that I try "import kitchen", but that gives me this error: > > > > "Traceback : > > File "", line 1, in > > ImportError: No module named kitchen" > > > > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? > > > > Any help would be appreciated. > > > > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 > > Forget sudo as that's a *nix command. From the command line you should be able to run:- > > pip install kitchen > pip install requests > > -- > Kindest regards. > > Mark Lawrence. Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png From p.f.moore at gmail.com Mon Jan 22 11:26:25 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 22 Jan 2018 16:26:25 +0000 Subject: Installing "kitchen" module In-Reply-To: <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> Message-ID: You need to run that command from a CMD prompt, not from inside the Python interpreter. On 22 January 2018 at 16:19, wrote: > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: >> > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: >> > >> > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server >> > >> > I have everything working up to the point where I run it and it tells me this: >> > >> > "Please install the Kitchen module. >> > Please install or update the Requests module." >> > >> > One suggestion was that I try "import kitchen", but that gives me this error: >> > >> > "Traceback : >> > File "", line 1, in >> > ImportError: No module named kitchen" >> > >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? >> > >> > Any help would be appreciated. >> > >> > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 >> >> Forget sudo as that's a *nix command. From the command line you should be able to run:- >> >> pip install kitchen >> pip install requests >> >> -- >> Kindest regards. >> >> Mark Lawrence. > > Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png > -- > https://mail.python.org/mailman/listinfo/python-list From alister.ware at ntlworld.com Mon Jan 22 11:28:42 2018 From: alister.ware at ntlworld.com (alister) Date: Mon, 22 Jan 2018 16:28:42 GMT Subject: Installing "kitchen" module References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> Message-ID: <_io9C.434987$Jg1.317354@fx33.am4> On Mon, 22 Jan 2018 08:19:51 -0800, codydaviestv wrote: > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com > wrote: >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com >> wrote: >> > So here's the situation. I am unfamiliar with Python but need it to >> > export a wiki, so I have been following this tutorial, using the >> > latest version of Python 2 on Windows 7: >> > >> > https://github.com/WikiTeam/wikiteam/wiki/ Tutorial#I_have_no_shell_access_to_server >> > >> > I have everything working up to the point where I run it and it tells >> > me this: >> > >> > "Please install the Kitchen module. >> > Please install or update the Requests module." >> > >> > One suggestion was that I try "import kitchen", but that gives me >> > this error: >> > >> > "Traceback : >> > File "", line 1, in >> > ImportError: No module named kitchen" >> > >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, >> > but that hasn't helped. Maybe it needs to be in a specific folder to >> > work? >> > >> > Any help would be appreciated. >> > >> > P.S. Here is someone else running into the same problem but they >> > seemed to have fixed it through a solution that didn't work for me >> > (it doesn't recognise a command called sudo in the first place when I >> > type it): https://github.com/WikiTeam/wikiteam/issues/252 >> >> Forget sudo as that's a *nix command. From the command line you should >> be able to run:- >> >> pip install kitchen pip install requests >> >> -- >> Kindest regards. >> >> Mark Lawrence. > > Here's what I see when I try that. Maybe I'm missing some kind of > initial setup? https://i.imgur.com/XQHO19W.png run it from the windows command line not the Python prompt -- Scientists will study your brain to learn more about your distant cousin, Man. From codydaviestv at gmail.com Mon Jan 22 11:31:59 2018 From: codydaviestv at gmail.com (codydaviestv at gmail.com) Date: Mon, 22 Jan 2018 08:31:59 -0800 (PST) Subject: Installing "kitchen" module In-Reply-To: References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> Message-ID: On Tuesday, 23 January 2018 02:56:56 UTC+10:30, Paul Moore wrote: > You need to run that command from a CMD prompt, not from inside the > Python interpreter. > > On 22 January 2018 at 16:19, cody wrote: > > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: > >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: > >> > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: > >> > > >> > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server > >> > > >> > I have everything working up to the point where I run it and it tells me this: > >> > > >> > "Please install the Kitchen module. > >> > Please install or update the Requests module." > >> > > >> > One suggestion was that I try "import kitchen", but that gives me this error: > >> > > >> > "Traceback : > >> > File "", line 1, in > >> > ImportError: No module named kitchen" > >> > > >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? > >> > > >> > Any help would be appreciated. > >> > > >> > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 > >> > >> Forget sudo as that's a *nix command. From the command line you should be able to run:- > >> > >> pip install kitchen > >> pip install requests > >> > >> -- > >> Kindest regards. > >> > >> Mark Lawrence. > > > > Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png > > -- > > https://mail.python.org/mailman/listinfo/python-list https://imgur.com/a/NfMJJ <- Still not much luck, unless I'm still at the wrong place From p.f.moore at gmail.com Mon Jan 22 12:10:54 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 22 Jan 2018 17:10:54 +0000 Subject: Installing "kitchen" module In-Reply-To: References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> Message-ID: "python -m pip install kitchen" is probably your best approach (from the CMD prompt). On 22 January 2018 at 16:31, wrote: > On Tuesday, 23 January 2018 02:56:56 UTC+10:30, Paul Moore wrote: >> You need to run that command from a CMD prompt, not from inside the >> Python interpreter. >> >> On 22 January 2018 at 16:19, cody wrote: >> > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: >> >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: >> >> > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: >> >> > >> >> > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server >> >> > >> >> > I have everything working up to the point where I run it and it tells me this: >> >> > >> >> > "Please install the Kitchen module. >> >> > Please install or update the Requests module." >> >> > >> >> > One suggestion was that I try "import kitchen", but that gives me this error: >> >> > >> >> > "Traceback : >> >> > File "", line 1, in >> >> > ImportError: No module named kitchen" >> >> > >> >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? >> >> > >> >> > Any help would be appreciated. >> >> > >> >> > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 >> >> >> >> Forget sudo as that's a *nix command. From the command line you should be able to run:- >> >> >> >> pip install kitchen >> >> pip install requests >> >> >> >> -- >> >> Kindest regards. >> >> >> >> Mark Lawrence. >> > >> > Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png >> > -- >> > https://mail.python.org/mailman/listinfo/python-list > > https://imgur.com/a/NfMJJ <- Still not much luck, unless I'm still at the wrong place > -- > https://mail.python.org/mailman/listinfo/python-list From codydaviestv at gmail.com Mon Jan 22 12:20:02 2018 From: codydaviestv at gmail.com (codydaviestv at gmail.com) Date: Mon, 22 Jan 2018 09:20:02 -0800 (PST) Subject: Installing "kitchen" module In-Reply-To: References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> Message-ID: <420092a8-0c4c-4447-9a5f-0761c3b99b4b@googlegroups.com> On Tuesday, 23 January 2018 03:41:28 UTC+10:30, Paul Moore wrote: > "python -m pip install kitchen" is probably your best approach (from > the CMD prompt). > > On 22 January 2018 at 16:31, wrote: > > On Tuesday, 23 January 2018 02:56:56 UTC+10:30, Paul Moore wrote: > >> You need to run that command from a CMD prompt, not from inside the > >> Python interpreter. > >> > >> On 22 January 2018 at 16:19, cody wrote: > >> > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: > >> >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: > >> >> > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: > >> >> > > >> >> > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server > >> >> > > >> >> > I have everything working up to the point where I run it and it tells me this: > >> >> > > >> >> > "Please install the Kitchen module. > >> >> > Please install or update the Requests module." > >> >> > > >> >> > One suggestion was that I try "import kitchen", but that gives me this error: > >> >> > > >> >> > "Traceback : > >> >> > File "", line 1, in > >> >> > ImportError: No module named kitchen" > >> >> > > >> >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? > >> >> > > >> >> > Any help would be appreciated. > >> >> > > >> >> > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 > >> >> > >> >> Forget sudo as that's a *nix command. From the command line you should be able to run:- > >> >> > >> >> pip install kitchen > >> >> pip install requests > >> >> > >> >> -- > >> >> Kindest regards. > >> >> > >> >> Mark Lawrence. > >> > > >> > Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png > >> > -- > >> > https://mail.python.org/mailman/listinfo/python-list > > > > https://imgur.com/a/NfMJJ <- Still not much luck, unless I'm still at the wrong place > > -- > > https://mail.python.org/mailman/listinfo/python-list Thanks, that seems to have worked. Out of curiosity, what did that change? From jqian at tibco.com Mon Jan 22 16:00:01 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 22 Jan 2018 16:00:01 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 Message-ID: Hello! I am using ctypes on Windows to interface with a dll and it works fine on Linux and windows 32-bit python. But, when using 64-bit python, we got error "exception: access violation writing 0xFFFFFFFF99222A60". Checking our server, it seems work without any problem. but the python gives an error and stop the application. -- c -- class myPythonAPI { public: myPythonAPI(); int createService(const char* serviceName){ /* run our application*/}; } extern "C" { __declspec(dllexport) myPythonAPI* loadInstance(){ return new myPythonAPI(); } __declspec(dllexport) int createService(myPythonAPI* obj, const char* serviceName) { eturn obj->createService(serviceName); }; -- python -- from ctypes import * lib = cdll.LoadLibrary('xxxxxxx.dll') lib.createService.argtypes=[c_int,ctypes.c_char_p] lib.createService.restype=ctypes.c_int class myDriver(object): def init(self): self.obj = lib.loadInstance() def create_services(self,servicename): result=lib.createService(self.obj,servicename) return result driver=myDriver() driver.create_services(b"myExample") Thanks for the help Jason From breamoreboy at gmail.com Mon Jan 22 16:03:41 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Mon, 22 Jan 2018 13:03:41 -0800 (PST) Subject: How to use asyncore with SSL? In-Reply-To: References: Message-ID: <78137699-f807-4570-986e-62a190015612@googlegroups.com> On Thursday, January 18, 2018 at 11:25:58 PM UTC, Grant Edwards wrote: > I've been trying to use the secure smtpd module from > https://github.com/bcoe/secure-smtpd, but the SSL support seems to be > fundamentally broken. That module simply wraps a socket and then > expects to use it in the normal way via asyncore. > > Of course that fails the first time an ssl-wrapped-socket's send or > recv method raises SSLWantReadError or SSLWantWriteError. Those > exceptions aren't handled and it crashes. > > That makes the SSL support pretty much useless. > > I'm trying to fix that, but I can't find any information or > documentation about using asyncore with SSL. > > Alternatively, a pointer to a simpler smtp server library that > supports SSL would be great. The use of asyncore and multiprocessing > process pools by this module is _way_ overkill for my needs and > results in something that 1) doesn't work, and 2) can't be debugged. > > -- > Grant Edwards grant.b.edwards Yow! Pardon me, but do you > at know what it means to be > gmail.com TRULY ONE with your BOOTH! I haven't tried it myself but I've just stumbled across this https://github.com/aio-libs/aiosmtpd. -- Kindest regards. Mark Lawrence. From random832 at fastmail.com Mon Jan 22 16:28:03 2018 From: random832 at fastmail.com (Random832) Date: Mon, 22 Jan 2018 16:28:03 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: References: Message-ID: <1516656483.3769193.1244377472.7B5E9523@webmail.messagingengine.com> On Mon, Jan 22, 2018, at 16:00, Jason Qian via Python-list wrote: > Hello! > > I am using ctypes on Windows to interface with a dll and it works fine > on Linux and windows 32-bit python. But, when using 64-bit python, we got > error "exception: access violation writing 0xFFFFFFFF99222A60". You are treating the obj type (myPythonAPI *) as c_int, which is only 32 bits. You should be using a pointer type instead (ideally you should be using void * and c_void_p, so Python doesn't need the class definition.) Don't forget to set lib.loadInstance.restype as well. > __declspec(dllexport) myPythonAPI* loadInstance(){ return new > myPythonAPI(); } > __declspec(dllexport) int createService(myPythonAPI* obj, const char* > serviceName) { eturn obj->createService(serviceName); > lib = cdll.LoadLibrary('xxxxxxx.dll') > > lib.createService.argtypes=[c_int,ctypes.c_char_p] > lib.createService.restype=ctypes.c_int > > class myDriver(object): > def init(self): > self.obj = lib.loadInstance() From eryksun at gmail.com Mon Jan 22 17:41:38 2018 From: eryksun at gmail.com (eryk sun) Date: Mon, 22 Jan 2018 22:41:38 +0000 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: References: Message-ID: On Mon, Jan 22, 2018 at 9:00 PM, Jason Qian via Python-list wrote: > > I am using ctypes on Windows to interface with a dll and it works fine > on Linux and windows 32-bit python. But, when using 64-bit python, we got > error "exception: access violation writing 0xFFFFFFFF99222A60". > > from ctypes import * Try to avoid * imports if it's a public module. > lib = cdll.LoadLibrary('xxxxxxx.dll') Just use CDLL directly. cdll.LoadLibrary is pointless, and `cdll.xxxxxxx` is problematic in some cases because it caches CDLL instances, which cache function pointers. Also, the ".dll" filename extension isn't required in Windows. > lib.createService.argtypes=[c_int,ctypes.c_char_p] > lib.createService.restype=ctypes.c_int > > class myDriver(object): > def init(self): > self.obj = lib.loadInstance() Since you didn't set loadInstance.restype, the result gets truncated as a C int, the default result type. I recommend defining an opaque ctypes struct (i.e. no defined fields) for the C++ class. This provides type safety. Staying strict and literal on types is more work than using a `void *` everywhere, but it pays off in terms of easier debugging and more resilient code. A simple example is that ctypes returns a `void *` result (or gets a struct field or array item) as a Python integer. Then for any FFI call that you may have forgotten to define argtypes, ctypes will default to truncating this integer value as a C int. At best that causes an immediate crash. At worst it's a subtle bug that corrupts data. Here's an example implementation with an opaque struct: import ctypes lib = ctypes.CDLL('xxxxxxx') class myPythonAPI(ctypes.Structure): pass PmyPythonAPI = ctypes.POINTER(myPythonAPI) lib.loadInstance.restype = PmyPythonAPI lib.loadInstance.argtypes = () lib.createService.restype = ctypes.c_int lib.createService.argtypes = (PmyPythonAPI, ctypes.c_char_p) class myDriver(object): def init(self): self.obj = lib.loadInstance() def create_services(self, servicename): return lib.createService(self.obj, servicename) From p.f.moore at gmail.com Mon Jan 22 17:54:18 2018 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 22 Jan 2018 22:54:18 +0000 Subject: Installing "kitchen" module In-Reply-To: <420092a8-0c4c-4447-9a5f-0761c3b99b4b@googlegroups.com> References: <1d944af6-125f-474a-b4cb-d07e9b4487b4@googlegroups.com> <97360e51-29d5-4a99-bdba-e271b702cd6a@googlegroups.com> <420092a8-0c4c-4447-9a5f-0761c3b99b4b@googlegroups.com> Message-ID: On 22 January 2018 at 17:20, wrote: > On Tuesday, 23 January 2018 03:41:28 UTC+10:30, Paul Moore wrote: >> "python -m pip install kitchen" is probably your best approach (from >> the CMD prompt). >> >> On 22 January 2018 at 16:31, wrote: >> > On Tuesday, 23 January 2018 02:56:56 UTC+10:30, Paul Moore wrote: >> >> You need to run that command from a CMD prompt, not from inside the >> >> Python interpreter. >> >> >> >> On 22 January 2018 at 16:19, cody wrote: >> >> > On Tuesday, 23 January 2018 02:41:04 UTC+10:30, bream... at gmail.com wrote: >> >> >> On Monday, January 22, 2018 at 3:37:44 PM UTC, codyda... at gmail.com wrote: >> >> >> > So here's the situation. I am unfamiliar with Python but need it to export a wiki, so I have been following this tutorial, using the latest version of Python 2 on Windows 7: >> >> >> > >> >> >> > https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server >> >> >> > >> >> >> > I have everything working up to the point where I run it and it tells me this: >> >> >> > >> >> >> > "Please install the Kitchen module. >> >> >> > Please install or update the Requests module." >> >> >> > >> >> >> > One suggestion was that I try "import kitchen", but that gives me this error: >> >> >> > >> >> >> > "Traceback : >> >> >> > File "", line 1, in >> >> >> > ImportError: No module named kitchen" >> >> >> > >> >> >> > So I went to https://pypi.python.org/pypi/kitchen/ to download it, but that hasn't helped. Maybe it needs to be in a specific folder to work? >> >> >> > >> >> >> > Any help would be appreciated. >> >> >> > >> >> >> > P.S. Here is someone else running into the same problem but they seemed to have fixed it through a solution that didn't work for me (it doesn't recognise a command called sudo in the first place when I type it): https://github.com/WikiTeam/wikiteam/issues/252 >> >> >> >> >> >> Forget sudo as that's a *nix command. From the command line you should be able to run:- >> >> >> >> >> >> pip install kitchen >> >> >> pip install requests >> >> >> >> >> >> -- >> >> >> Kindest regards. >> >> >> >> >> >> Mark Lawrence. >> >> > >> >> > Here's what I see when I try that. Maybe I'm missing some kind of initial setup? https://i.imgur.com/XQHO19W.png >> >> > -- >> >> > https://mail.python.org/mailman/listinfo/python-list >> > >> > https://imgur.com/a/NfMJJ <- Still not much luck, unless I'm still at the wrong place >> > -- >> > https://mail.python.org/mailman/listinfo/python-list > > Thanks, that seems to have worked. Out of curiosity, what did that change? The key is that you have to run the command at the CMD prompt, not at the Python interpreter prompt. The two commands "pip" and "python -m pip" are basically equivalent, but the executables "python.exe" and "pip.exe" are installed in different places, and it's possible that one might be on your PATH but not the other. I knew you were able to run "python", so I recommended "python -m pip" to avoid the risk that "pip" wouldn't work because of PATH issues. Hope that clarifies. Paul From jqian at tibco.com Mon Jan 22 17:58:50 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 22 Jan 2018 17:58:50 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: <1516656483.3769193.1244377472.7B5E9523@webmail.messagingengine.com> References: <1516656483.3769193.1244377472.7B5E9523@webmail.messagingengine.com> Message-ID: Thanks you very much, fixed the problem :) On Mon, Jan 22, 2018 at 4:28 PM, Random832 wrote: > On Mon, Jan 22, 2018, at 16:00, Jason Qian via Python-list wrote: > > Hello! > > > > I am using ctypes on Windows to interface with a dll and it works fine > > on Linux and windows 32-bit python. But, when using 64-bit python, we > got > > error "exception: access violation writing 0xFFFFFFFF99222A60". > > You are treating the obj type (myPythonAPI *) as c_int, which is only 32 > bits. You should be using a pointer type instead (ideally you should be > using void * and c_void_p, so Python doesn't need the class definition.) > Don't forget to set lib.loadInstance.restype as well. > > > __declspec(dllexport) myPythonAPI* loadInstance(){ return new > > myPythonAPI(); } > > __declspec(dllexport) int createService(myPythonAPI* obj, const char* > > serviceName) { eturn obj->createService(serviceName); > > > lib = cdll.LoadLibrary('xxxxxxx.dll') > > > > lib.createService.argtypes=[c_int,ctypes.c_char_p] > > lib.createService.restype=ctypes.c_int > > > > class myDriver(object): > > def init(self): > > self.obj = lib.loadInstance() > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Mon Jan 22 20:50:11 2018 From: jqian at tibco.com (Jason Qian) Date: Mon, 22 Jan 2018 20:50:11 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: References: Message-ID: Thanks for the help, Jason On Mon, Jan 22, 2018 at 5:41 PM, eryk sun wrote: > On Mon, Jan 22, 2018 at 9:00 PM, Jason Qian via Python-list > wrote: > > > > I am using ctypes on Windows to interface with a dll and it works fine > > on Linux and windows 32-bit python. But, when using 64-bit python, we > got > > error "exception: access violation writing 0xFFFFFFFF99222A60". > > > > from ctypes import * > > Try to avoid * imports if it's a public module. > > > lib = cdll.LoadLibrary('xxxxxxx.dll') > > Just use CDLL directly. cdll.LoadLibrary is pointless, and > `cdll.xxxxxxx` is problematic in some cases because it caches CDLL > instances, which cache function pointers. Also, the ".dll" filename > extension isn't required in Windows. > > > lib.createService.argtypes=[c_int,ctypes.c_char_p] > > lib.createService.restype=ctypes.c_int > > > > class myDriver(object): > > def init(self): > > self.obj = lib.loadInstance() > > Since you didn't set loadInstance.restype, the result gets truncated > as a C int, the default result type. > > I recommend defining an opaque ctypes struct (i.e. no defined fields) > for the C++ class. This provides type safety. Staying strict and > literal on types is more work than using a `void *` everywhere, but it > pays off in terms of easier debugging and more resilient code. A > simple example is that ctypes returns a `void *` result (or gets a > struct field or array item) as a Python integer. Then for any FFI call > that you may have forgotten to define argtypes, ctypes will default to > truncating this integer value as a C int. At best that causes an > immediate crash. At worst it's a subtle bug that corrupts data. > > Here's an example implementation with an opaque struct: > > import ctypes > > lib = ctypes.CDLL('xxxxxxx') > > class myPythonAPI(ctypes.Structure): > pass > > PmyPythonAPI = ctypes.POINTER(myPythonAPI) > > lib.loadInstance.restype = PmyPythonAPI > lib.loadInstance.argtypes = () > > lib.createService.restype = ctypes.c_int > lib.createService.argtypes = (PmyPythonAPI, ctypes.c_char_p) > > class myDriver(object): > > def init(self): > self.obj = lib.loadInstance() > > def create_services(self, servicename): > return lib.createService(self.obj, servicename) > From rustompmody at gmail.com Mon Jan 22 22:53:21 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 22 Jan 2018 19:53:21 -0800 (PST) Subject: Why is there no functional xml? In-Reply-To: References: Message-ID: On Sunday, January 21, 2018 at 4:51:34 PM UTC+5:30, Peter Otten wrote: > Personally I'd probably avoid the extra layer and write a function that > directly maps dataclasses or database records to xml using the conventional > elementtree API. Would appreciate your thoughts/comments Peter! I find that you can get 'E' from lxml.objectify as well as lxml.builder builder seems better in that its at least sparsely documented objectify seems to have almost nothing beyond the original David Mertz' docs builder.E seems to do what objectify.E does modulo namespaces builder.E and objectify.E produce types that are different and look backwards (at least to me ? Elementbase is less base than _Element) You seem to have some reservation against objectify, preferring the default Element ? I'd like to know what Insofar as builder seems to produce the same type as Element unlike objectify which seems to be producing a grandchild type, do you have the same reservations against builder.E? -------------- $ python3 Python 3.5.3 (default, Nov 23 2017, 11:34:05) [GCC 6.3.0 20170406] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from lxml.etree import Element, tostring >>> from lxml.builder import E as Eb >>> from lxml.objectify import E as Eo >>> e = Element("tag") >>> tostring(e) b'' >>> o = Eb.tag() >>> o >>> tostring(o) b'' >>> o = Eo.tag() >>> tostring(o) b'' >>> b = Eb.tag() >>> tostring(b) b'' >>> type(b) >>> type(b).__bases__ (,) >>> type(e) >>> type(o) >>> type(o).__bases__ (,) >>> type(o).__bases__[0].__bases__ (,) >>> -------------- From dieter at handshake.de Tue Jan 23 03:12:25 2018 From: dieter at handshake.de (dieter) Date: Tue, 23 Jan 2018 09:12:25 +0100 Subject: exec and traceback References: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> Message-ID: <87k1w9uh7a.fsf@handshake.de> ken.py at gameofy.com writes: > I'm using exec() to run a (multi-line) string of python code. If an > exception occurs, I get a traceback containing a stack frame for the > string. I've labeled the code object with a "file name" so I can > identify it easily, and when I debug, I find that I can interact with > the context of that stack frame, which is pretty handy. > > What I would like to also be able to do is make the code string > visible to the debugger so I can look at and step through the code in > the string as if it were from a python file. The debugger may not be prepared for this kind of source; thus, you might need a much more intelligent debugger. The "exec" itself works with arbitary strings, not only with string constants coming from a source file. It has no way to reliably associate a filename and a line number in the respective file with the code lines. The debugger (at least currently) expects to locate the source code in a file (maybe an archive) given a line number in that file. Based on this, I expect that your wish will be difficult to fulfill. From anussdn00 at gmail.com Tue Jan 23 04:00:15 2018 From: anussdn00 at gmail.com (Anu. SSDN) Date: Tue, 23 Jan 2018 01:00:15 -0800 (PST) Subject: Java Programming in Delhi | Java Training Institute in Delhi Message-ID: <17d7a324-57ef-4648-8914-b6c702adfbad@googlegroups.com> Learn All The Essential Java Keywords, Operators, Statements, And Expressions. SSDN Technologies is the premier Java training in Delhi, India. Highly Qualified Trainers are available to assist you. Enroll Now! Read More:- http://www.ssdntech.com/Java-training.aspx From bhattacharya.kushal4 at gmail.com Tue Jan 23 07:25:59 2018 From: bhattacharya.kushal4 at gmail.com (kushal bhattacharya) Date: Tue, 23 Jan 2018 04:25:59 -0800 (PST) Subject: python to C code generator In-Reply-To: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: <9f4081db-4b28-4f7a-9ffe-aeab5b2a813a@googlegroups.com> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: > Hi, > Is there any python framework or any tool as which can generate C code from python code as it is . > > Thanks, > Kushal hi, I have found nuitka as asuitable candidate but it seems that nuitka doesnt generate a simple C code which could be included as a C file in another program.Is there any alternative easier way regarding this? Thanks From paulina.zuniga at financecentre.club Tue Jan 23 07:36:55 2018 From: paulina.zuniga at financecentre.club (paulina.zuniga at financecentre.club) Date: Tue, 23 Jan 2018 13:36:55 +0100 Subject: python to C code generator In-Reply-To: <9f4081db-4b28-4f7a-9ffe-aeab5b2a813a@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> <9f4081db-4b28-4f7a-9ffe-aeab5b2a813a@googlegroups.com> Message-ID: <35b1d717-cf6d-6e06-e1fe-ba4218284d49@financecentre.club> What about Cython? On 01/23/2018 01:25 PM, kushal bhattacharya wrote: > On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: >> Hi, >> Is there any python framework or any tool as which can generate C code from python code as it is . >> >> Thanks, >> Kushal > > hi, > I have found nuitka as asuitable candidate but it seems that nuitka doesnt generate a simple C code which could be included as a C file in another program.Is there any alternative easier way regarding this? > > Thanks > From bhattacharya.kushal4 at gmail.com Tue Jan 23 08:23:13 2018 From: bhattacharya.kushal4 at gmail.com (kushal bhattacharya) Date: Tue, 23 Jan 2018 05:23:13 -0800 (PST) Subject: python to C code generator In-Reply-To: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: > Hi, > Is there any python framework or any tool as which can generate C code from python code as it is . > > Thanks, > Kushal yes i have but it generates a complex C code with python dependencies.I want to call the generated function from another C code but i Cant figure out how to do that From bc at freeuk.com Tue Jan 23 08:34:50 2018 From: bc at freeuk.com (bartc) Date: Tue, 23 Jan 2018 13:34:50 +0000 Subject: python to C code generator In-Reply-To: References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On 23/01/2018 13:23, kushal bhattacharya wrote: > On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: >> Hi, >> Is there any python framework or any tool as which can generate C code from python code as it is . >> >> Thanks, >> Kushal > > yes i have but it generates a complex C code with python dependencies.I want to call the generated function from another C code but i Cant figure out how to do that Because the translation isn't simply defined. I've just tried nuitka on the Python code 'a=b+c', and it generates 2400 lines of C. The main purpose seems to be to generate a self-contained executable corresponding to the Python, but generating first a C equivalent then using a C compiler and linker. This equivalent code may just contain all the bits in CPython needed to do the job, but bypassing all the stuff to do with executing actual byte-code. But it also seems to do some optimisations (in the generated C before it uses C compiler optimisations), so that if static types can be inferred it might make use of that info. Perhaps you simply want to use Python syntax to write C code? That would be a different kind of translator. And a simpler one, as 'a=b+c' translates to 'a+b+c;' in C. -- bartc From bhattacharya.kushal4 at gmail.com Tue Jan 23 08:48:03 2018 From: bhattacharya.kushal4 at gmail.com (kushal bhattacharya) Date: Tue, 23 Jan 2018 05:48:03 -0800 (PST) Subject: python to C code generator In-Reply-To: References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On Tuesday, January 23, 2018 at 7:05:02 PM UTC+5:30, bartc wrote: > On 23/01/2018 13:23, kushal bhattacharya wrote: > > On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: > >> Hi, > >> Is there any python framework or any tool as which can generate C code from python code as it is . > >> > >> Thanks, > >> Kushal > > > > yes i have but it generates a complex C code with python dependencies.I want to call the generated function from another C code but i Cant figure out how to do that > > Because the translation isn't simply defined. > > I've just tried nuitka on the Python code 'a=b+c', and it generates 2400 > lines of C. The main purpose seems to be to generate a self-contained > executable corresponding to the Python, but generating first a C > equivalent then using a C compiler and linker. > > This equivalent code may just contain all the bits in CPython needed to > do the job, but bypassing all the stuff to do with executing actual > byte-code. But it also seems to do some optimisations (in the generated > C before it uses C compiler optimisations), so that if static types can > be inferred it might make use of that info. > > Perhaps you simply want to use Python syntax to write C code? That would > be a different kind of translator. And a simpler one, as 'a=b+c' > translates to 'a+b+c;' in C. > > -- > bartc This is exactly what i meant to say.My goal is to translate the python code into its C equivalent with function name as it is. From ned at nedbatchelder.com Tue Jan 23 09:00:52 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 23 Jan 2018 09:00:52 -0500 Subject: python to C code generator In-Reply-To: References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: <26304288-5f7f-bd2e-31ea-dd07d52b2a33@nedbatchelder.com> On 1/23/18 8:48 AM, kushal bhattacharya wrote: > On Tuesday, January 23, 2018 at 7:05:02 PM UTC+5:30, bartc wrote: >> On 23/01/2018 13:23, kushal bhattacharya wrote: >>> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: >>>> Hi, >>>> Is there any python framework or any tool as which can generate C code from python code as it is . >>>> >>>> Thanks, >>>> Kushal >>> yes i have but it generates a complex C code with python dependencies.I want to call the generated function from another C code but i Cant figure out how to do that >> Because the translation isn't simply defined. >> >> I've just tried nuitka on the Python code 'a=b+c', and it generates 2400 >> lines of C. The main purpose seems to be to generate a self-contained >> executable corresponding to the Python, but generating first a C >> equivalent then using a C compiler and linker. >> >> This equivalent code may just contain all the bits in CPython needed to >> do the job, but bypassing all the stuff to do with executing actual >> byte-code. But it also seems to do some optimisations (in the generated >> C before it uses C compiler optimisations), so that if static types can >> be inferred it might make use of that info. >> >> Perhaps you simply want to use Python syntax to write C code? That would >> be a different kind of translator. And a simpler one, as 'a=b+c' >> translates to 'a+b+c;' in C. >> >> -- >> bartc > > This is exactly what i meant to say.My goal is to translate the python code into its C equivalent with function name as it is. The best way to do that is to read the Python code, understand what it does, and re-write it in C.? You won't find an automatic tool that can do the job you want.? The semantics of Python and C are too different. --Ned. From kirillbalunov at gmail.com Tue Jan 23 09:11:03 2018 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 23 Jan 2018 17:11:03 +0300 Subject: python to C code generator In-Reply-To: <26304288-5f7f-bd2e-31ea-dd07d52b2a33@nedbatchelder.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> <26304288-5f7f-bd2e-31ea-dd07d52b2a33@nedbatchelder.com> Message-ID: You can look at SymPy code generator http://docs.sympy.org/latest/modules/utilities/codegen.html Perhaps this is exactly what you need. With kind regards, -gdg 2018-01-23 17:00 GMT+03:00 Ned Batchelder : > On 1/23/18 8:48 AM, kushal bhattacharya wrote: > >> On Tuesday, January 23, 2018 at 7:05:02 PM UTC+5:30, bartc wrote: >> >>> On 23/01/2018 13:23, kushal bhattacharya wrote: >>> >>>> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal >>>> bhattacharya wrote: >>>> >>>>> Hi, >>>>> Is there any python framework or any tool as which can generate C >>>>> code from python code as it is . >>>>> >>>>> Thanks, >>>>> Kushal >>>>> >>>> yes i have but it generates a complex C code with python dependencies.I >>>> want to call the generated function from another C code but i Cant figure >>>> out how to do that >>>> >>> Because the translation isn't simply defined. >>> >>> I've just tried nuitka on the Python code 'a=b+c', and it generates 2400 >>> lines of C. The main purpose seems to be to generate a self-contained >>> executable corresponding to the Python, but generating first a C >>> equivalent then using a C compiler and linker. >>> >>> This equivalent code may just contain all the bits in CPython needed to >>> do the job, but bypassing all the stuff to do with executing actual >>> byte-code. But it also seems to do some optimisations (in the generated >>> C before it uses C compiler optimisations), so that if static types can >>> be inferred it might make use of that info. >>> >>> Perhaps you simply want to use Python syntax to write C code? That would >>> be a different kind of translator. And a simpler one, as 'a=b+c' >>> translates to 'a+b+c;' in C. >>> >>> -- >>> bartc >>> >> >> This is exactly what i meant to say.My goal is to translate the python >> code into its C equivalent with function name as it is. >> > > The best way to do that is to read the Python code, understand what it > does, and re-write it in C. You won't find an automatic tool that can do > the job you want. The semantics of Python and C are too different. > > --Ned. > -- > https://mail.python.org/mailman/listinfo/python-list > From formisc at gmail.com Tue Jan 23 09:25:27 2018 From: formisc at gmail.com (Andrew Z) Date: Tue, 23 Jan 2018 09:25:27 -0500 Subject: python to C code generator In-Reply-To: <26304288-5f7f-bd2e-31ea-dd07d52b2a33@nedbatchelder.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> <26304288-5f7f-bd2e-31ea-dd07d52b2a33@nedbatchelder.com> Message-ID: Id go this way too. Basic C is straightforward. I usually consider learning a new "thing " if the time to support potwntially combersome solution using existing methods justifies the effort. On Jan 23, 2018 09:01, "Ned Batchelder" wrote: > On 1/23/18 8:48 AM, kushal bhattacharya wrote: > >> On Tuesday, January 23, 2018 at 7:05:02 PM UTC+5:30, bartc wrote: >> >>> On 23/01/2018 13:23, kushal bhattacharya wrote: >>> >>>> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal >>>> bhattacharya wrote: >>>> >>>>> Hi, >>>>> Is there any python framework or any tool as which can generate C >>>>> code from python code as it is . >>>>> >>>>> Thanks, >>>>> Kushal >>>>> >>>> yes i have but it generates a complex C code with python dependencies.I >>>> want to call the generated function from another C code but i Cant figure >>>> out how to do that >>>> >>> Because the translation isn't simply defined. >>> >>> I've just tried nuitka on the Python code 'a=b+c', and it generates 2400 >>> lines of C. The main purpose seems to be to generate a self-contained >>> executable corresponding to the Python, but generating first a C >>> equivalent then using a C compiler and linker. >>> >>> This equivalent code may just contain all the bits in CPython needed to >>> do the job, but bypassing all the stuff to do with executing actual >>> byte-code. But it also seems to do some optimisations (in the generated >>> C before it uses C compiler optimisations), so that if static types can >>> be inferred it might make use of that info. >>> >>> Perhaps you simply want to use Python syntax to write C code? That would >>> be a different kind of translator. And a simpler one, as 'a=b+c' >>> translates to 'a+b+c;' in C. >>> >>> -- >>> bartc >>> >> >> This is exactly what i meant to say.My goal is to translate the python >> code into its C equivalent with function name as it is. >> > > The best way to do that is to read the Python code, understand what it > does, and re-write it in C. You won't find an automatic tool that can do > the job you want. The semantics of Python and C are too different. > > --Ned. > -- > https://mail.python.org/mailman/listinfo/python-list > From theodore.leblanc at financecentre.club Tue Jan 23 09:45:35 2018 From: theodore.leblanc at financecentre.club (theodore.leblanc at financecentre.club) Date: Tue, 23 Jan 2018 15:45:35 +0100 Subject: python to C code generator In-Reply-To: <25153760-9102-5961-7c2f-d853c28c837f@bankmail.host> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> <9f4081db-4b28-4f7a-9ffe-aeab5b2a813a@googlegroups.com> <25153760-9102-5961-7c2f-d853c28c837f@bankmail.host> Message-ID: Hey Ally, Cython adds a big chunk of complexity to simple things. That's the problem. Greetings. On 01/23/2018 01:54 PM, ally.mata at bankmail.host wrote: > Have you tried cython ? > > On 01/23/2018 01:25 PM, kushal bhattacharya wrote: >> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal >> bhattacharya wrote: >>> Hi, >>> Is there any python framework or any tool as? which can generate C >>> code from python code as it is . >>> >>> Thanks, >>> Kushal >> >> hi, >> I have found nuitka as asuitable candidate but it seems that nuitka >> doesnt generate a simple C code which could be included as a C file in >> another program.Is there any alternative easier way regarding this? >> >> Thanks >> From bhattacharya.kushal4 at gmail.com Tue Jan 23 09:49:06 2018 From: bhattacharya.kushal4 at gmail.com (kushal bhattacharya) Date: Tue, 23 Jan 2018 06:49:06 -0800 (PST) Subject: python to C code generator In-Reply-To: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: <13efa155-1f7e-4098-b9ff-8b0479a16193@googlegroups.com> On Wednesday, January 17, 2018 at 4:34:23 PM UTC+5:30, kushal bhattacharya wrote: > Hi, > Is there any python framework or any tool as which can generate C code from python code as it is . > > Thanks, > Kushal ok so which python tool would be the best one which can be included and parameters can be passed to from another C code file From larry at hastings.org Tue Jan 23 09:52:12 2018 From: larry at hastings.org (Larry Hastings) Date: Tue, 23 Jan 2018 06:52:12 -0800 Subject: [RELEASED] Python 3.4.8rc1 and Python 3.5.5rc1 are now available Message-ID: <91bc5411-3a98-ef11-fc25-ee3fa75941ba@hastings.org> On behalf of the Python development community, I'm pleased to announce the availability of Python 3.4.8rc1 and Python 3.5.5rc1. Both Python 3.4 and 3.5 are in "security fixes only" mode. Both versions only accept security fixes, not conventional bug fixes, and both releases are source-only. You can find Python 3.4.8rc1 here: https://www.python.org/downloads/release/python-348rc1/ And you can find Python 3.5.5rc1 here: https://www.python.org/downloads/release/python-355rc1/ Happy Pythoning, //arry/ From __peter__ at web.de Tue Jan 23 09:52:58 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 23 Jan 2018 15:52:58 +0100 Subject: Why is there no functional xml? References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> Message-ID: Rustom Mody wrote: > Its obviously easier in python to put optional/vararg parameters on the > right side rather than on the left of a parameter list. > But its not impossible to get it in the desired order ? one just has to > 'hand-parse' the parameter list received as a *param > Thusly: > appointments = E.appointments( > {"reminder":"15"}, > E.appointment( > E.begin(1181251680), > E.uid("040000008200E000"), > E.alarmTime(1181572063), > E.state(), > E.location(), > E.duration(1800), > E.subject("Bring pizza home") > ) > ) Let's not waste too much effort on minor aesthic improvements ;) >> Personally I'd probably avoid the extra layer and write a function that >> directly maps dataclasses or database records to xml using the >> conventional elementtree API. > > I dont understand? > [I find the OO/imperative style of making a half-done node and then > [throwing > piece-by-piece of contents in/at it highly aggravating] What I meant is that once you throw a bit of introspection at it much of the tedium vanishes. Here's what might become of the DOM-creation as part of an actual script: import xml.etree.ElementTree as xml from collections import namedtuple Appointment = namedtuple( "Appointment", "begin uid alarmTime state location duration subject" ) appointments = [ Appointment( begin=1181251680, uid="040000008200E000", alarmTime=1181572063, state=None, location=None, duration=1800, subject="Bring pizza home" ) ] def create_dom(appointments, reminder): node = xml.Element("zAppointments", reminder=str(reminder)) for appointment in appointments: for name, value in zip(appointment._fields, appointment): child = xml.SubElement(node, name) if value is not None: child.text = str(value) return node with open("appt.xml", "wb") as outstream: root = create_dom(appointments, 15) xml.ElementTree(root).write(outstream) To generalize that to handle arbitrarily nested lists and namedtuples a bit more effort is needed, but I can't see where lxml.objectify could make that much easier. From __peter__ at web.de Tue Jan 23 10:03:31 2018 From: __peter__ at web.de (Peter Otten) Date: Tue, 23 Jan 2018 16:03:31 +0100 Subject: Why is there no functional xml? References: Message-ID: Rustom Mody wrote: > On Sunday, January 21, 2018 at 4:51:34 PM UTC+5:30, Peter Otten wrote: >> Personally I'd probably avoid the extra layer and write a function that >> directly maps dataclasses or database records to xml using the >> conventional elementtree API. > > Would appreciate your thoughts/comments Peter! > > I find that you can get 'E' from lxml.objectify as well as lxml.builder > builder seems better in that its at least sparsely documented > objectify seems to have almost nothing beyond the original David Mertz' > docs > > builder.E seems to do what objectify.E does modulo namespaces > > builder.E and objectify.E produce types that are different and look > backwards (at least to me ? Elementbase is less base than _Element) > > You seem to have some reservation against objectify, preferring the > default Element ? I'd like to know what While I don't have any actual experience with it, my gut feeling is that it simplifies something that is superfluous to begin with. > Insofar as builder seems to produce the same type as Element unlike > objectify which seems to be producing a grandchild type, do you have the > same reservations against builder.E? If I understand you correctly you are talking about implementation details. Unfortunately I cannot comment on these -- I really just remembered objectify because of the catchy name... From rosuav at gmail.com Tue Jan 23 10:14:48 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Jan 2018 02:14:48 +1100 Subject: python to C code generator In-Reply-To: References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> <9f4081db-4b28-4f7a-9ffe-aeab5b2a813a@googlegroups.com> <25153760-9102-5961-7c2f-d853c28c837f@bankmail.host> Message-ID: On Wed, Jan 24, 2018 at 1:45 AM, wrote: > Hey Ally, > > Cython adds a big chunk of complexity to simple things. That's the problem. That's like saying "Unicode adds a big chunk of complexity to the simple task of translating a word from Japanese into Russian". No, it doesn't; the complexity is inherent in the problem. You cannot translate Python code into C code without either (a) reimplementing all of Python's semantics, as Cython does; or (b) drastically changing the semantics, such that even the very simplest of code might behave quite differently; or (c) manually reading through the code and writing equivalent C, which is what you might call "porting" or "rewriting". (Or possibly "prototyping", if the intention was always to transform it into C.) There is fundamentally NO easy way to translate code from one language into another and get readable, idiomatic code at the other end. ChrisA From bc at freeuk.com Tue Jan 23 12:43:18 2018 From: bc at freeuk.com (bartc) Date: Tue, 23 Jan 2018 17:43:18 +0000 Subject: python to C code generator In-Reply-To: References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On 23/01/2018 13:34, bartc wrote: > Perhaps you simply want to use Python syntax to write C code? That would > be a different kind of translator. And a simpler one, as 'a=b+c' > translates to 'a+b+c;' in C. Or rather, 'a=b+c;' (I've written source to source translators, some of which could target C, but not Python to C. It would be feasible to write C code in a syntax that looks rather like Python, but it won't be real Python, and you can't run it as Python. It wouldn't be a satisfactory way of writing C programs. So, although I'm not that big a fan of C syntax, it might be better to write C as C, and Python as Python, to avoid confusion.) -- bartc From vs at it.uu.se Tue Jan 23 13:50:57 2018 From: vs at it.uu.se (Virgil Stokes) Date: Tue, 23 Jan 2018 19:50:57 +0100 Subject: Processing a key pressed in Python 3.6 Message-ID: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> I would appreciate help on finding a solution to following problem. This is the general structure of the "problem" code: > while True > ??? # Get some data from the web and process it > ??? ... > ??? ... > ??? # Write these data to a file > ?? ?... > ?? ?... > ?? ?# Check if a_key has been pressed in the command window > ?? ?... > ?? ?... > ??? if a_key_pressed: > ??????? # Perform some pre-termination tasks > ??????? ... > ??????? ... > ?? ??? ?# Close the output data file > ?? ??? ?... > ?? ??? ?... > ?? ??? ?raise SystemExit('Exit') I am running the code with Python 3.6 on a windows 10 platform. I have tried many approaches (mainly those posted on stackoverflow) but I have yet to find an approach that works for this structure. Note: ? 1) The code is executed in the windows 10 command window ? 2) I am not using wxPython, IDLE, or pyGame in this application. ? 3) The time to get the data, process it and write it to a file can ???? take from 0.5 sec to 1.5 sec ? 4) The key hit need not be echoed to the command window From rosuav at gmail.com Tue Jan 23 14:15:41 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 24 Jan 2018 06:15:41 +1100 Subject: Processing a key pressed in Python 3.6 In-Reply-To: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> Message-ID: On Wed, Jan 24, 2018 at 5:50 AM, Virgil Stokes wrote: > I would appreciate help on finding a solution to following problem. This is > the general structure of the "problem" code: > >> while True >> # Get some data from the web and process it >> ... >> ... >> # Write these data to a file >> ... >> ... >> # Check if a_key has been pressed in the command window >> ... >> ... >> if a_key_pressed: >> # Perform some pre-termination tasks >> ... >> ... >> # Close the output data file >> ... >> ... >> raise SystemExit('Exit') > > > I am running the code with Python 3.6 on a windows 10 platform. I have tried > many approaches (mainly those posted on stackoverflow) but I have yet to > find an approach that works for this structure. > > Note: > 1) The code is executed in the windows 10 command window > 2) I am not using wxPython, IDLE, or pyGame in this application. > 3) The time to get the data, process it and write it to a file can > take from 0.5 sec to 1.5 sec > 4) The key hit need not be echoed to the command window > Are you okay with demanding a specific key, rather than simply "press any key"? Even better, key combination? Handle Ctrl-C by catching KeyboardInterrupt and you can take advantage of Python's existing cross-platform handling of the standard interrupt signal. If Ctrl-C won't work for you, how about stipulating that it be Enter? "Press Enter to quit" isn't too much worse than "Press any key to quit" (plus you have less chance of accidentally terminating the program when you don't want to). Spin off a thread to wait for enter. I've tested this only on Linux, but it ought to work: import threading import time shutdown = False def wait_for_enter(): print("Hit Enter to quit.") input() global shutdown; shutdown = True threading.Thread(target=wait_for_enter).start() while "more work to do": print("Getting data...") time.sleep(1) print("Saving data to file...") time.sleep(1) if shutdown: print("Pre-termination...") time.sleep(1) raise SystemExit("exit") If it doesn't, try switching around which is the secondary thread and which is the primary - spin off a thread to do the work, then call input() in the main thread. ChrisA From vs at it.uu.se Tue Jan 23 15:04:04 2018 From: vs at it.uu.se (Virgil Stokes) Date: Tue, 23 Jan 2018 21:04:04 +0100 Subject: =?UTF-8?Q?Re:_Processing_a_key_pressed_in_Python_3.6_=f0=9f=a6=89?= In-Reply-To: References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> Message-ID: Thanks very much Chris, This code worked perfectly for "Enter". Your knowledge of? Python and more specifically this elegant solution are greatly appreciated. I now know that I need to learn more about threads. :-) On 2018-01-23 20:15, Chris Angelico wrote: > On Wed, Jan 24, 2018 at 5:50 AM, Virgil Stokes wrote: >> I would appreciate help on finding a solution to following problem. This is >> the general structure of the "problem" code: >> >>> while True >>> # Get some data from the web and process it >>> ... >>> ... >>> # Write these data to a file >>> ... >>> ... >>> # Check if a_key has been pressed in the command window >>> ... >>> ... >>> if a_key_pressed: >>> # Perform some pre-termination tasks >>> ... >>> ... >>> # Close the output data file >>> ... >>> ... >>> raise SystemExit('Exit') >> >> I am running the code with Python 3.6 on a windows 10 platform. I have tried >> many approaches (mainly those posted on stackoverflow) but I have yet to >> find an approach that works for this structure. >> >> Note: >> 1) The code is executed in the windows 10 command window >> 2) I am not using wxPython, IDLE, or pyGame in this application. >> 3) The time to get the data, process it and write it to a file can >> take from 0.5 sec to 1.5 sec >> 4) The key hit need not be echoed to the command window >> > Are you okay with demanding a specific key, rather than simply "press > any key"? Even better, key combination? Handle Ctrl-C by catching > KeyboardInterrupt and you can take advantage of Python's existing > cross-platform handling of the standard interrupt signal. > > If Ctrl-C won't work for you, how about stipulating that it be Enter? > "Press Enter to quit" isn't too much worse than "Press any key to > quit" (plus you have less chance of accidentally terminating the > program when you don't want to). Spin off a thread to wait for enter. > I've tested this only on Linux, but it ought to work: > > import threading > import time > > shutdown = False > def wait_for_enter(): > print("Hit Enter to quit.") > input() > global shutdown; shutdown = True > > threading.Thread(target=wait_for_enter).start() > > while "more work to do": > print("Getting data...") > time.sleep(1) > print("Saving data to file...") > time.sleep(1) > if shutdown: > print("Pre-termination...") > time.sleep(1) > raise SystemExit("exit") > > If it doesn't, try switching around which is the secondary thread and > which is the primary - spin off a thread to do the work, then call > input() in the main thread. > > ChrisA From vincent at vincentdavis.net Tue Jan 23 15:51:55 2018 From: vincent at vincentdavis.net (Vincent Davis) Date: Tue, 23 Jan 2018 13:51:55 -0700 Subject: plot / graph connecting re ordered lists Message-ID: Looking for suggestions. I have an ordered list of names these names will be reordered. I am looking to make a plot, graph, with the two origins of the names in separate columns and a line connecting them to visually represent how much they have moved in the reordering. Surely there is some great example code for this on the net an am not finding a clean example. Thanks Vincent Davis From vs at it.uu.se Tue Jan 23 18:29:31 2018 From: vs at it.uu.se (Virgil Stokes) Date: Wed, 24 Jan 2018 00:29:31 +0100 Subject: Processing a key pressed in Python 3.6 In-Reply-To: References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> Message-ID: <30596a1c-c917-cdfd-d910-ee450d2e29cf@it.uu.se> Another follow-up question: How would this code be modified to handle using the "Esc" key instead of the "Enter" key? On 2018-01-23 20:15, Chris Angelico wrote: > On Wed, Jan 24, 2018 at 5:50 AM, Virgil Stokes wrote: >> I would appreciate help on finding a solution to following problem. This is >> the general structure of the "problem" code: >> >>> while True >>> # Get some data from the web and process it >>> ... >>> ... >>> # Write these data to a file >>> ... >>> ... >>> # Check if a_key has been pressed in the command window >>> ... >>> ... >>> if a_key_pressed: >>> # Perform some pre-termination tasks >>> ... >>> ... >>> # Close the output data file >>> ... >>> ... >>> raise SystemExit('Exit') >> >> I am running the code with Python 3.6 on a windows 10 platform. I have tried >> many approaches (mainly those posted on stackoverflow) but I have yet to >> find an approach that works for this structure. >> >> Note: >> 1) The code is executed in the windows 10 command window >> 2) I am not using wxPython, IDLE, or pyGame in this application. >> 3) The time to get the data, process it and write it to a file can >> take from 0.5 sec to 1.5 sec >> 4) The key hit need not be echoed to the command window >> > Are you okay with demanding a specific key, rather than simply "press > any key"? Even better, key combination? Handle Ctrl-C by catching > KeyboardInterrupt and you can take advantage of Python's existing > cross-platform handling of the standard interrupt signal. > > If Ctrl-C won't work for you, how about stipulating that it be Enter? > "Press Enter to quit" isn't too much worse than "Press any key to > quit" (plus you have less chance of accidentally terminating the > program when you don't want to). Spin off a thread to wait for enter. > I've tested this only on Linux, but it ought to work: > > import threading > import time > > shutdown = False > def wait_for_enter(): > print("Hit Enter to quit.") > input() > global shutdown; shutdown = True > > threading.Thread(target=wait_for_enter).start() > > while "more work to do": > print("Getting data...") > time.sleep(1) > print("Saving data to file...") > time.sleep(1) > if shutdown: > print("Pre-termination...") > time.sleep(1) > raise SystemExit("exit") > > If it doesn't, try switching around which is the secondary thread and > which is the primary - spin off a thread to do the work, then call > input() in the main thread. > > ChrisA From vincent at vincentdavis.net Tue Jan 23 18:42:59 2018 From: vincent at vincentdavis.net (Vincent Davis) Date: Tue, 23 Jan 2018 23:42:59 +0000 Subject: plot / graph connecting re ordered lists In-Reply-To: References: Message-ID: On Tue, Jan 23, 2018 at 4:15 PM Dennis Lee Bieber wrote: > On Tue, 23 Jan 2018 13:51:55 -0700, Vincent Davis > declaimed the following: > > >Looking for suggestions. I have an ordered list of names these names will > >be reordered. I am looking to make a plot, graph, with the two origins of > > IE: you have two lists with the same items in different orders... > > >the names in separate columns and a line connecting them to visually > >represent how much they have moved in the reordering. > >Surely there is some great example code for this on the net an am not > >finding a clean example. > > > > Determine positions: > > pos = [] > for p, name in enumerate(first_list): > np = second_list.index(name) > pos.append( (name, p, np) ) > > for (name, p, np) in pos: > draw_line((1,p) , (2, np)) > label( (1, p), name) > > Exact details of graphics package and scaling left as an exercise Actualy, it?s recomendations for a graphing package And an example using it for such a graph that I am most interested in. I know how to relate the names on the 2 lists. > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Sent from mobile app. Vincent Davis 720-301-3003 From duncan at invalid.invalid Tue Jan 23 19:08:35 2018 From: duncan at invalid.invalid (duncan smith) Date: Wed, 24 Jan 2018 00:08:35 +0000 Subject: plot / graph connecting re ordered lists In-Reply-To: References: Message-ID: <88Q9C.120639$kr1.93696@fx01.iad> On 23/01/18 23:42, Vincent Davis wrote: > On Tue, Jan 23, 2018 at 4:15 PM Dennis Lee Bieber > wrote: > >> On Tue, 23 Jan 2018 13:51:55 -0700, Vincent Davis >> declaimed the following: >> >>> Looking for suggestions. I have an ordered list of names these names will >>> be reordered. I am looking to make a plot, graph, with the two origins of >> >> IE: you have two lists with the same items in different orders... >> >>> the names in separate columns and a line connecting them to visually >>> represent how much they have moved in the reordering. >>> Surely there is some great example code for this on the net an am not >>> finding a clean example. >>> >> >> Determine positions: >> >> pos = [] >> for p, name in enumerate(first_list): >> np = second_list.index(name) >> pos.append( (name, p, np) ) >> >> for (name, p, np) in pos: >> draw_line((1,p) , (2, np)) >> label( (1, p), name) >> >> Exact details of graphics package and scaling left as an exercise > > > Actualy, it?s recomendations for a graphing package And an example using it > for such a graph that I am most interested in. I know how to relate the > names on the 2 lists. > > >> -- >> Wulfraed Dennis Lee Bieber AF6VN >> wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> Maybe http://graphviz.org/ and http://matthiaseisen.com/articles/graphviz/ Duncan From python at mrabarnett.plus.com Tue Jan 23 20:00:38 2018 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 24 Jan 2018 01:00:38 +0000 Subject: Processing a key pressed in Python 3.6 In-Reply-To: <30596a1c-c917-cdfd-d910-ee450d2e29cf@it.uu.se> References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> <30596a1c-c917-cdfd-d910-ee450d2e29cf@it.uu.se> Message-ID: On 2018-01-23 23:29, Virgil Stokes wrote: > Another follow-up question: > > How would this code be modified to handle using the "Esc" key instead of > the "Enter" key? > This version uses msvcrt on Windows: import msvcrt import threading import time shutdown = False def wait_for_enter(): global shutdown print("Hit Enter to quit.") # b'\x0D' is the bytestring produced by the Enter key. # b'\x1B' is the bytestring produced by the Esc key. if msvcrt.getch() == b'\x1B': shutdown = True threading.Thread(target=wait_for_enter).start() while "more work to do": print("Getting data...") time.sleep(1) print("Saving data to file...") time.sleep(1) if shutdown: print("Pre-termination...") time.sleep(1) raise SystemExit("exit") Another function of note is "msvcrt.kbhit()", which will tell you if there's a key waiting to be read. This is all in the docs for the msvcrt module. > > On 2018-01-23 20:15, Chris Angelico wrote: >> On Wed, Jan 24, 2018 at 5:50 AM, Virgil Stokes wrote: >>> I would appreciate help on finding a solution to following problem. This is >>> the general structure of the "problem" code: >>> >>>> while True >>>> # Get some data from the web and process it >>>> ... >>>> ... >>>> # Write these data to a file >>>> ... >>>> ... >>>> # Check if a_key has been pressed in the command window >>>> ... >>>> ... >>>> if a_key_pressed: >>>> # Perform some pre-termination tasks >>>> ... >>>> ... >>>> # Close the output data file >>>> ... >>>> ... >>>> raise SystemExit('Exit') >>> >>> I am running the code with Python 3.6 on a windows 10 platform. I have tried >>> many approaches (mainly those posted on stackoverflow) but I have yet to >>> find an approach that works for this structure. >>> >>> Note: >>> 1) The code is executed in the windows 10 command window >>> 2) I am not using wxPython, IDLE, or pyGame in this application. >>> 3) The time to get the data, process it and write it to a file can >>> take from 0.5 sec to 1.5 sec >>> 4) The key hit need not be echoed to the command window >>> >> Are you okay with demanding a specific key, rather than simply "press >> any key"? Even better, key combination? Handle Ctrl-C by catching >> KeyboardInterrupt and you can take advantage of Python's existing >> cross-platform handling of the standard interrupt signal. >> >> If Ctrl-C won't work for you, how about stipulating that it be Enter? >> "Press Enter to quit" isn't too much worse than "Press any key to >> quit" (plus you have less chance of accidentally terminating the >> program when you don't want to). Spin off a thread to wait for enter. >> I've tested this only on Linux, but it ought to work: >> >> import threading >> import time >> >> shutdown = False >> def wait_for_enter(): >> print("Hit Enter to quit.") >> input() >> global shutdown; shutdown = True >> >> threading.Thread(target=wait_for_enter).start() >> >> while "more work to do": >> print("Getting data...") >> time.sleep(1) >> print("Saving data to file...") >> time.sleep(1) >> if shutdown: >> print("Pre-termination...") >> time.sleep(1) >> raise SystemExit("exit") >> >> If it doesn't, try switching around which is the secondary thread and >> which is the primary - spin off a thread to do the work, then call >> input() in the main thread. >> >> ChrisA > From eryksun at gmail.com Tue Jan 23 20:09:23 2018 From: eryksun at gmail.com (eryk sun) Date: Wed, 24 Jan 2018 01:09:23 +0000 Subject: Processing a key pressed in Python 3.6 In-Reply-To: <30596a1c-c917-cdfd-d910-ee450d2e29cf@it.uu.se> References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> <30596a1c-c917-cdfd-d910-ee450d2e29cf@it.uu.se> Message-ID: On Tue, Jan 23, 2018 at 11:29 PM, Virgil Stokes wrote: > > How would this code be modified to handle using the "Esc" key instead of the > "Enter" key? The input() function depends on the console or terminal to read a line of input. If you're using the Windows console, it calls the high-level ReadConsole (or ReadFile) function, which performs a 'cooked' read. In this case some keys are reserved. Escape is consumed to clear/flush the input buffer. Function keys and arrows are consumed for line editing and history navigation. And for some reason line-feed (^J) is always ignored. Additionally, normal operation requires the following input modes to be enabled: ENABLE_PROCESSED_INPUT process Ctrl+C as CTRL_C_EVENT and carriage return (^M) as carriage return plus line feed (CRLF). consume backspace (^H) to delete the previous character. ENABLE_LINE_INPUT return only when a carriage return is read. ENABLE_ECHO_INPUT echo read characters to the active screen. Reading the escape key from the console requires the low-level ReadConsoleInput, GetNumberOfConsoleInputEvents, and FlushConsoleInputBuffer functions. These access the console's raw stream of input event records, which includes key events, mouse events, and window/buffer resize events. It's a bit complicated to use this API, but fortunately the C runtime wraps it with convenient functions such as _kbhit, _getwch, and _getwche (w/ echo). On Windows only, you'll find these functions in the msvcrt module, but named without the leading underscore. Here's an example of reading the escape character without echo: >>> msvcrt.getwch() '\x1b' Simple, no? From vs at it.uu.se Tue Jan 23 20:13:35 2018 From: vs at it.uu.se (Virgil Stokes) Date: Wed, 24 Jan 2018 02:13:35 +0100 Subject: Processing a key pressed in Python 3.6 In-Reply-To: <0f3f6dt8dn795act52l1fhmdsnoici923o@4ax.com> References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> <0f3f6dt8dn795act52l1fhmdsnoici923o@4ax.com> Message-ID: Ok Dennis, You were correct. The following also works in the Windows 10 command window. > import time > import msvcrt > > while "more work to do": > ??? print("Getting data...") > ??? time.sleep(1) > ??? print("Saving data to file...") > ??? time.sleep(1) > ??? key = msvcrt.getwch() > ??? #print('key: %s'%key)? # just to show the result > ??? if key == chr(27): > ??????? print("Pre-termination...") > ??????? time.sleep(1) > ??????? raise SystemExit("exit") Note, I am using the "Esc" key to exit, which is one answer to my last posting on this topic. On 2018-01-23 20:37, Dennis Lee Bieber wrote: > On Tue, 23 Jan 2018 19:50:57 +0100, Virgil Stokes declaimed > the following: > >> I am running the code with Python 3.6 on a windows 10 platform. I have >> tried many approaches (mainly those posted on stackoverflow) but I have >> yet to find an approach that works for this structure. >> >> Note: >> ? 1) The code is executed in the windows 10 command window >> ? 2) I am not using wxPython, IDLE, or pyGame in this application. >> ? 3) The time to get the data, process it and write it to a file can >> ???? take from 0.5 sec to 1.5 sec >> ? 4) The key hit need not be echoed to the command window > And none of your searching found > https://docs.python.org/3/library/msvcrt.html > which is part of the standard library (and documented in the library > manual). The module IS Windows specific, you'd have to rewrite the code to > run on Linux. > > Now, if your requirement was to detect a keypress WHILE the data > fetch/processing was happening, the solution will be different. From rustompmody at gmail.com Tue Jan 23 22:50:45 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 23 Jan 2018 19:50:45 -0800 (PST) Subject: Why is there no functional xml? In-Reply-To: References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> Message-ID: <3f25867b-f444-4727-afb9-a91b1b77f916@googlegroups.com> On Tuesday, January 23, 2018 at 8:23:43 PM UTC+5:30, Peter Otten wrote: > Rustom Mody wrote: > > [I find the OO/imperative style of making a half-done node and then > > [throwing > > piece-by-piece of contents in/at it highly aggravating] > > What I meant is that once you throw a bit of introspection at it much of the > tedium vanishes. Here's what might become of the DOM-creation as part of an > actual script: ?snipped named-tuple magic? > To generalize that to handle arbitrarily nested lists and namedtuples a bit > more effort is needed, but I can't see where lxml.objectify could make that > much easier. You really mean that?? Well sure in the programming world and even more so in the python world ?Flat is better than nested? is a maxim But equally programmers need to satisfy requirements? And right now I am seeing things like this ----------------------------------------------- AGGREGATION_ANALYSIS ?Big base64 blob? ----------------------------------------------- Thats 7 levels of nesting (assuming I can count right!) Speaking of which another followup question: With # Read above xml >>> with open('soap_response.xml') as f: inp = etree.parse(f) # namespace dict >>> nsd = {'soap': "http://schemas.xmlsoap.org/soap/envelope/", 'locns': "http://example.com/"} The following behavior is observed ? actual responses elided in the interest of brevity >>> inp.xpath('//soap:Body', namespaces = nsd) finds/reaches the node >>> inp.xpath('//locns:blobRetrieveResponse', namespaces = nsd) finds >>> inp.xpath('//locns:dtCreationDate', namespaces = nsd) does not find >>> inp.xpath('//dtCreationDate', namespaces = nsd) finds >>> inp.xpath('//dtCreationDate') also finds Doesnt this contradict the fact that dtCreationDate is under the locns namespace?? Any explanations?? From steve+comp.lang.python at pearwood.info Wed Jan 24 02:04:11 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jan 2018 07:04:11 +0000 (UTC) Subject: python to C code generator References: <947d2b90-d126-44b7-a209-baf65a7ef39d@googlegroups.com> Message-ID: On Tue, 23 Jan 2018 17:43:18 +0000, bartc wrote: > It wouldn't be a satisfactory way of writing C programs. So, although > I'm not that big a fan of C syntax, it might be better to write C as C, > and Python as Python, to avoid confusion.) This. The fundamental reality is that `a + b` means different things in C and Python. Even if you limit yourself to integers and not arbitrary values (fractions, lists, strings, etc) the semantics are different: - in C, ints have a fixed number of bits and any addition which ends up out of range is undefined behaviour[1]; - while Python uses BigInts, overflow is impossible, and the only possible error is that you run out of memory and an exception is raised (although the addition can take an indefinite long amount of time). Often the difference doesn't matter... but when it does matter, it *really* matters. [1] If anyone thinks that it is addition with overflow, you are wrong. Some C compilers *may* use overflow, but the language strictly defines it as undefined behaviour, so the compiler can equally choose to set your computer on fire[2] if it prefers. https://blog.regehr.org/archives/213 [2] http://www.catb.org/jargon/html/H/HCF.html -- Steve From __peter__ at web.de Wed Jan 24 03:42:27 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Jan 2018 09:42:27 +0100 Subject: Why is there no functional xml? References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> <3f25867b-f444-4727-afb9-a91b1b77f916@googlegroups.com> Message-ID: Rustom Mody wrote: >> To generalize that to handle arbitrarily nested lists and namedtuples a >> bit more effort is needed, but I can't see where lxml.objectify could >> make that much easier. > > You really mean that?? > Well sure in the programming world and even more so in the python world > ?Flat is better than nested? is a maxim > > But equally programmers need to satisfy requirements? > > And right now I am seeing things like this > ----------------------------------------------- > Hm, what happens if you throw a dedicated library at the problem? Google found zeep http://docs.python-zeep.org/en/master/datastructures.html From __peter__ at web.de Wed Jan 24 04:00:10 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Jan 2018 10:00:10 +0100 Subject: xpath prob, was Re: Why is there no functional xml? References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> <3f25867b-f444-4727-afb9-a91b1b77f916@googlegroups.com> Message-ID: Rustom Mody wrote: > With > # Read above xml >>>> with open('soap_response.xml') as f: inp = etree.parse(f) > # namespace dict >>>> nsd = {'soap': "http://schemas.xmlsoap.org/soap/envelope/", 'locns': >>>> "http://example.com/"} > > The following behavior is observed ? actual responses elided in the > interest of brevity > >>>> inp.xpath('//soap:Body', namespaces = nsd) > finds/reaches the node > >>>> inp.xpath('//locns:blobRetrieveResponse', namespaces = nsd) > finds > >>>> inp.xpath('//locns:dtCreationDate', namespaces = nsd) > does not find > >>>> inp.xpath('//dtCreationDate', namespaces = nsd) > finds > >>>> inp.xpath('//dtCreationDate') > also finds > > > Doesnt this contradict the fact that dtCreationDate is under the locns > namespace?? > > Any explanations?? Can you rewrite that question as a simple self-contained demo, similar to the snippet shown under http://lxml.de/xpathxslt.html#namespaces-and-prefixes ? From jorge.conrado at cptec.inpe.br Wed Jan 24 08:06:20 2018 From: jorge.conrado at cptec.inpe.br (jorge.conrado at cptec.inpe.br) Date: Wed, 24 Jan 2018 11:06:20 -0200 Subject: Read satellite images Message-ID: Hi, I have some gridded 4Km satellite images. I don't have experience with Python. The size of my image is (9896,3298) and I use this to read f = open('merg_2018011100_4km-pixel', "r") # reopen the file x = f.read() print (x[0]) I have this value for x = ? This is the information for this data from where I did the ftp: the data is 1-byte. Please, what can I do to get the value in ascii. Thanks, Conrado From jugurtha.hadjar at gmail.com Wed Jan 24 11:41:26 2018 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Wed, 24 Jan 2018 17:41:26 +0100 Subject: Read satellite images In-Reply-To: References: Message-ID: <6070deac-aeea-202f-1946-2fce149ad401@gmail.com> On 01/24/2018 02:06 PM, jorge.conrado at cptec.inpe.br wrote: > > Hi, > > I have some gridded 4Km satellite images. I don't have experience with > Python. The size of my image is (9896,3298) and I use this to read > > f = open('merg_2018011100_4km-pixel', "r")? # reopen the file > > x = f.read() > > print (x[0]) > Hello, What are you trying to accomplish? Do you want the first byte? Do you want data in a specific format (maybe a hex string like "FFABE6", etc)? Also, when dealing with "binary" files, you may want to append a "b" to "r". Assuming you are using Python 2 (from your print): ??? import binascii ??? with open('merg_2018011100_4km-pixel', 'rb') as f: ??????? data = f.read() ??? hex_data = binascii.hexlify(data) ??? first_byte = hex_data[0:2] ??? print first_byte -- ~ Jugurtha Hadjar, From cl at isbd.net Wed Jan 24 14:54:09 2018 From: cl at isbd.net (Chris Green) Date: Wed, 24 Jan 2018 19:54:09 +0000 Subject: How to diagnose this, fails on 3.6.3, works on 3.5.2? Message-ID: <1s4nje-arm.ln1@esprimo.zbmc.eu> I have a fairly simple little python program to automate starting an editor on a wiki page. It works fine on the system where I wrote it (xubuntu 16.04, python 3 version 3.5.2) but it comes up with the following error on a newer system (xubuntu 17.10, python 3 version 3.6.3). Here is the error:- chris$ no Traceback (most recent call last): File "/home/chris/bin/no", line 59, in os.execvp("vi", ("", monthFile,)) File "/usr/lib/python3.6/os.py", line 559, in execvp _execvpe(file, args) File "/usr/lib/python3.6/os.py", line 594, in _execvpe exec_func(fullname, *argrest) ValueError: execv() arg 2 first element cannot be empty Has execvp() become stricter in 3.6.3 or what? ... and here is the program:- #!/usr/bin/python3 # # # Create Dokuwiki journal month pages # import sys import os import time import calendar jdir = "/home/chris/wiki/data/pages/journal" # # # Default month and year is 'now' # month = time.localtime().tm_mon year = time.localtime().tm_year # # # If one parameter is given then it's the month # if len(sys.argv) == 2: month = int(sys.argv[1]) # # # If two parameters are given they are month and year # if len(sys.argv) == 3: year = int(sys.argv[2]) month = int(sys.argv[1]) # # # # # # # Check if the year directory exists and create it if it doesn't # yearDir = os.path.join(jdir, str(year)) if not os.path.exists(yearDir): os.mkdir(yearDir) # # # Check if month file exists, create it if it doesn't and write heading and links # if month < 10: monthFile = os.path.join(yearDir, '0' + str(month) + '.txt') else: monthFile = os.path.join(yearDir, str(month) + '.txt') if not os.path.exists(monthFile): monthName = calendar.month_name[month] f = open(monthFile, 'w') f.write(monthName + " " + str(year) + "\n") for i in range(len(monthName) + 5): f.write("=") f.write("\n") f.close() os.execvp("vi", ("", monthFile,)) -- Chris Green ? From vs at it.uu.se Wed Jan 24 15:02:39 2018 From: vs at it.uu.se (Virgil Stokes) Date: Wed, 24 Jan 2018 21:02:39 +0100 Subject: Processing a key pressed in Python 3.6 In-Reply-To: <3efh6ddi4duvlnt4jamsh84ker3tv7cgvi@4ax.com> References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> <0f3f6dt8dn795act52l1fhmdsnoici923o@4ax.com> <3efh6ddi4duvlnt4jamsh84ker3tv7cgvi@4ax.com> Message-ID: Yes, I am aware of this Dennis. However, but, on my system I actually had it running without the msvcrt.kbhit() This occurred during testing while trying different options. And I was able to reproduce this several times. Why? I do not have an answer. This is one reason why I posted the code. It would be interesting to know if anyone else has obtained the same results. Note: 1) 3.6.2 (v3.6.2:5fd33b5, Jul? 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] ?????????? 2) msvcrt is built-in (at least on my system) On 2018-01-24 18:28, Dennis Lee Bieber wrote: > On Wed, 24 Jan 2018 02:13:35 +0100, Virgil Stokes *declaimed* ? > the following: > > > >>> ??? key = msvcrt.getwch() > NOTE: per documentation, that is a blocking read... It won't return > unless a key (any key) has been pressed. That means your "more work to do" > loop requires a key press for each loop. > > Recommendation would be to use kbhit() first. > > > if msvcrt.kbhit(): > key = msvcrt.getwch() > ... > > > Consider > > -=-=-=-=- > > import msvcrt as ms > import time > > ESCAPE = chr(27) > > workPending = True > cycle = 0 > > while workPending: > print("Getting imaginary data %s..." % cycle) > time.sleep(1.5) > print("\tSaving imaginary data %s..." % cycle) > time.sleep(1.5) > > if ms.kbhit(): > key = ms.getwch() > if key == ESCAPE: > print("Pre-termination on cycle %s..." % cycle) > break > else: > print("Random keypress %r found on cycle %s..." % (key, cycle)) > > cycle += 1 > > time.sleep(1.5) > print("Termination") > -=-=-=-=- > C:\Users\Wulfraed\Documents\Python Progs>kbhit > Getting imaginary data 0... > Saving imaginary data 0... > Getting imaginary data 1... > Saving imaginary data 1... > Random keypress u'd' found on cycle 1... > Getting imaginary data 2... > Saving imaginary data 2... > Getting imaginary data 3... > Saving imaginary data 3... > Random keypress u'a' found on cycle 3... > Getting imaginary data 4... > Saving imaginary data 4... > Pre-termination on cycle 4... > Termination > > C:\Users\Wulfraed\Documents\Python Progs> From __peter__ at web.de Wed Jan 24 15:56:22 2018 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Jan 2018 21:56:22 +0100 Subject: How to diagnose this, fails on 3.6.3, works on 3.5.2? References: <1s4nje-arm.ln1@esprimo.zbmc.eu> Message-ID: Chris Green wrote: > I have a fairly simple little python program to automate starting an > editor on a wiki page. It works fine on the system where I wrote it > (xubuntu 16.04, python 3 version 3.5.2) but it comes up with the > following error on a newer system (xubuntu 17.10, python 3 version > 3.6.3). > > Here is the error:- > > chris$ no > Traceback (most recent call last): > File "/home/chris/bin/no", line 59, in > os.execvp("vi", ("", monthFile,)) > File "/usr/lib/python3.6/os.py", line 559, in execvp > _execvpe(file, args) > File "/usr/lib/python3.6/os.py", line 594, in _execvpe > exec_func(fullname, *argrest) > ValueError: execv() arg 2 first element cannot be empty > > Has execvp() become stricter in 3.6.3 or what? Yes; the relevant issue on the bug tracker seems to be https://bugs.python.org/issue28732 > ... and here is the program:- [snip] A smaller demo is $ cat demo.py import os os.execvp("ls", ("",)) $ python3.5 demo.py demo.py $ python3.6 demo.py Traceback (most recent call last): File "demo.py", line 2, in os.execvp("ls", ("",)) File "/usr/local/lib/python3.6/os.py", line 559, in execvp _execvpe(file, args) File "/usr/local/lib/python3.6/os.py", line 594, in _execvpe exec_func(fullname, *argrest) ValueError: execv() arg 2 first element cannot be empty From jqian at tibco.com Wed Jan 24 16:25:24 2018 From: jqian at tibco.com (Jason Qian) Date: Wed, 24 Jan 2018 16:25:24 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: References: <1516656483.3769193.1244377472.7B5E9523@webmail.messagingengine.com> Message-ID: Again, thanks for the help. Everything is working fine after the changes. Here is one more new issue needs some help. On c side, The createService function can pass a callback handler as second parameter. Without callback handler, it works fine. But if we add the callback handler, the application will give a exception due to the pointer of callback handler = NULL; Not sure, why the callback handler missed up, when the app calling from python. Thanks -- python lib.createService.argtypes=[ctypes.c_void_p,ctypes.c_char_p] lib.createService.restype=ctypes.c_int def create_services(self,servicename): result=lib.createService(self.obj,servicename) return result --c -- __declspec(dllexport) int createService(void* obj, const char* serviceName) { return ((myPythonAPI*)obj)->createService(serviceName); } int myPythonAPI::createService(const char* serviceName) { //case 1 : //This works fine createService(methodname); //case 2 //This will not working, InvocationCallback serviceCallback; createService(methodname, &serviceCallback); } On Mon, Jan 22, 2018 at 5:58 PM, Jason Qian wrote: > Thanks you very much, fixed the problem :) > > On Mon, Jan 22, 2018 at 4:28 PM, Random832 wrote: > >> On Mon, Jan 22, 2018, at 16:00, Jason Qian via Python-list wrote: >> > Hello! >> > >> > I am using ctypes on Windows to interface with a dll and it works >> fine >> > on Linux and windows 32-bit python. But, when using 64-bit python, we >> got >> > error "exception: access violation writing 0xFFFFFFFF99222A60". >> >> You are treating the obj type (myPythonAPI *) as c_int, which is only 32 >> bits. You should be using a pointer type instead (ideally you should be >> using void * and c_void_p, so Python doesn't need the class definition.) >> Don't forget to set lib.loadInstance.restype as well. >> >> > __declspec(dllexport) myPythonAPI* loadInstance(){ return new >> > myPythonAPI(); } >> > __declspec(dllexport) int createService(myPythonAPI* obj, const char* >> > serviceName) { eturn obj->createService(serviceName); >> >> > lib = cdll.LoadLibrary('xxxxxxx.dll') >> > >> > lib.createService.argtypes=[c_int,ctypes.c_char_p] >> > lib.createService.restype=ctypes.c_int >> > >> > class myDriver(object): >> > def init(self): >> > self.obj = lib.loadInstance() >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > From eryksun at gmail.com Wed Jan 24 17:07:54 2018 From: eryksun at gmail.com (eryk sun) Date: Wed, 24 Jan 2018 22:07:54 +0000 Subject: Processing a key pressed in Python 3.6 In-Reply-To: References: <2ecb7f39-c967-b5ea-6dc8-0b8fcf3aa66a@it.uu.se> <0f3f6dt8dn795act52l1fhmdsnoici923o@4ax.com> <3efh6ddi4duvlnt4jamsh84ker3tv7cgvi@4ax.com> Message-ID: On Wed, Jan 24, 2018 at 8:02 PM, Virgil Stokes wrote: > Yes, I am aware of this Dennis. However, but, on my system I actually had it > running without the msvcrt.kbhit() _getwch loops calling ReadConsoleInput until a key event is read. The key can be typed manually in the console, or posted (i.e. PostMessage, not SendMessage) to the console window as a WM_KEYDOWN message, or the key event can be written directly to the input buffer via WriteConsoleInput. _kbhit calls PeekConsoleInput to scan for key events without removing them from the input buffer. This call does not block. From jqian at tibco.com Wed Jan 24 17:16:22 2018 From: jqian at tibco.com (Jason Qian) Date: Wed, 24 Jan 2018 17:16:22 -0500 Subject: Python call c pass a callback function on Linux Message-ID: Hi, I have following code that works fine on windows. InvocationCB=WINFUNCTYPE(None, c_char_p, c_int) submit = lib.submit submit.argtypes = [ctypes.c_void_p, c_void_p,InvocationCB] submit.restype = ctypes.c_int def handleResponse(message, code): print('--- handleResponse ---') print(message) print(code) class GSPythonDriver(object): def submif(self,methodname) invm_fn = InvocationCB(handleResponse) lib.submit(self.obj,methodname,invm_fn) How can I do this on the Linux ? Thanks for the help Jason From rosuav at gmail.com Wed Jan 24 17:23:53 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 25 Jan 2018 09:23:53 +1100 Subject: Python call c pass a callback function on Linux In-Reply-To: References: Message-ID: On Thu, Jan 25, 2018 at 9:16 AM, Jason Qian via Python-list wrote: > Hi, > > I have following code that works fine on windows. > > InvocationCB=WINFUNCTYPE(None, c_char_p, c_int) > submit = lib.submit > submit.argtypes = [ctypes.c_void_p, c_void_p,InvocationCB] > submit.restype = ctypes.c_int > > def handleResponse(message, code): > print('--- handleResponse ---') > print(message) > print(code) > > class GSPythonDriver(object): > def submif(self,methodname) > invm_fn = InvocationCB(handleResponse) > lib.submit(self.obj,methodname,invm_fn) > > How can I do this on the Linux ? If you're doing a lot of "got this C code, wanna call it from Python", I recommend looking into Cython and building a wrapper. Should be a lot less fiddly - and a lot less fragile - than playing with ctypes for every call you want to make. ChrisA From jqian at tibco.com Wed Jan 24 22:19:21 2018 From: jqian at tibco.com (Jason Qian) Date: Wed, 24 Jan 2018 22:19:21 -0500 Subject: Python call c pass a callback function on Linux In-Reply-To: References: Message-ID: HI Dennis, Thanks for the help, After changing WINFUNCTYPE to CFUNCTYPE, the call back function works on the Linux :) Thanks again, Jason On Wed, Jan 24, 2018 at 6:15 PM, Dennis Lee Bieber wrote: > On Wed, 24 Jan 2018 17:16:22 -0500, Jason Qian via Python-list > declaimed the following: > > >Hi, > > > > I have following code that works fine on windows. > > > You have not provided a minimal functional example... > > >InvocationCB=WINFUNCTYPE(None, c_char_p, c_int) > >submit = lib.submit > > Where did lib.submit come from? No import statements are shown. > > >submit.argtypes = [ctypes.c_void_p, c_void_p,InvocationCB] > >submit.restype = ctypes.c_int > > > > You are setting things on the name submit yet... > > >def handleResponse(message, code): > > print('--- handleResponse ---') > > print(message) > > print(code) > > > >class GSPythonDriver(object): > > def submif(self,methodname) > > Is that a typo for submit? > > > invm_fn = InvocationCB(handleResponse) > > lib.submit(self.obj,methodname,invm_fn) > > ... down here you are referring back to the full lib.submit (which may be > the same object) > > > No example instance of GSPythonDriver is created, and thereby > nothing > defined within it is invoked... > > However, the one thing that stands out is that WINFUNCTYPE is > Windows > "stdcall" specific, and ctypes defines CFUNCTYPE for the more global C > calling conventions. But from there? You have to specify the proper library > containing the functions you are invoking... Is that library (or > equivalent) even available on your proposed target OS? > > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Wed Jan 24 23:04:22 2018 From: jqian at tibco.com (Jason Qian) Date: Wed, 24 Jan 2018 23:04:22 -0500 Subject: Help: 64bit python call c and got OSError: exception: access violation writing 0xFFFFFFFF99222A60 In-Reply-To: References: <1516656483.3769193.1244377472.7B5E9523@webmail.messagingengine.com> Message-ID: Figured it out, Thanks On Wed, Jan 24, 2018 at 4:25 PM, Jason Qian wrote: > Again, thanks for the help. Everything is working fine after the changes. > > Here is one more new issue needs some help. > > On c side, > > The createService function can pass a callback handler as second > parameter. > Without callback handler, it works fine. But if we add the callback > handler, the application will give a exception due to the pointer of > callback handler = NULL; > > Not sure, why the callback handler missed up, when the app calling from > python. > > Thanks > > -- python > > lib.createService.argtypes=[ctypes.c_void_p,ctypes.c_char_p] > lib.createService.restype=ctypes.c_int > > def create_services(self,servicename): > result=lib.createService(self.obj,servicename) > return result > > --c -- > > __declspec(dllexport) int createService(void* obj, const char* > serviceName) > { > return ((myPythonAPI*)obj)->createService(serviceName); > } > > int myPythonAPI::createService(const char* serviceName) > { > //case 1 : > //This works fine > createService(methodname); > > //case 2 > //This will not working, > InvocationCallback serviceCallback; > createService(methodname, &serviceCallback); > } > > > > > > > > > > > > On Mon, Jan 22, 2018 at 5:58 PM, Jason Qian wrote: > >> Thanks you very much, fixed the problem :) >> >> On Mon, Jan 22, 2018 at 4:28 PM, Random832 >> wrote: >> >>> On Mon, Jan 22, 2018, at 16:00, Jason Qian via Python-list wrote: >>> > Hello! >>> > >>> > I am using ctypes on Windows to interface with a dll and it works >>> fine >>> > on Linux and windows 32-bit python. But, when using 64-bit python, we >>> got >>> > error "exception: access violation writing 0xFFFFFFFF99222A60". >>> >>> You are treating the obj type (myPythonAPI *) as c_int, which is only 32 >>> bits. You should be using a pointer type instead (ideally you should be >>> using void * and c_void_p, so Python doesn't need the class definition.) >>> Don't forget to set lib.loadInstance.restype as well. >>> >>> > __declspec(dllexport) myPythonAPI* loadInstance(){ return new >>> > myPythonAPI(); } >>> > __declspec(dllexport) int createService(myPythonAPI* obj, const char* >>> > serviceName) { eturn obj->createService(serviceName); >>> >>> > lib = cdll.LoadLibrary('xxxxxxx.dll') >>> > >>> > lib.createService.argtypes=[c_int,ctypes.c_char_p] >>> > lib.createService.restype=ctypes.c_int >>> > >>> > class myDriver(object): >>> > def init(self): >>> > self.obj = lib.loadInstance() >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> >> >> > From dieter at handshake.de Thu Jan 25 02:12:53 2018 From: dieter at handshake.de (dieter) Date: Thu, 25 Jan 2018 08:12:53 +0100 Subject: xpath prob, was Re: Why is there no functional xml? References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> <3f25867b-f444-4727-afb9-a91b1b77f916@googlegroups.com> Message-ID: <87d11y5s3u.fsf@handshake.de> > Rustom Mody wrote: > >> With >> # Read above xml >>>>> with open('soap_response.xml') as f: inp = etree.parse(f) >> # namespace dict >>>>> nsd = {'soap': "http://schemas.xmlsoap.org/soap/envelope/", 'locns': >>>>> "http://example.com/"} >> >> The following behavior is observed $(G!7(B actual responses elided in the >> interest of brevity >> >>>>> inp.xpath('//soap:Body', namespaces = nsd) >> finds/reaches the node >> >>>>> inp.xpath('//locns:blobRetrieveResponse', namespaces = nsd) >> finds >> >>>>> inp.xpath('//locns:dtCreationDate', namespaces = nsd) >> does not find >> >>>>> inp.xpath('//dtCreationDate', namespaces = nsd) >> finds >> >>>>> inp.xpath('//dtCreationDate') >> also finds >> >> >> Doesnt this contradict the fact that dtCreationDate is under the locns >> namespace?? Apparently, "dtCreationDate" is not associated with the namespace corresponding to the "locns" namespace. Note, that the namespace association is not by default inherited by child elements -- as least not with stand alone XML. Thus, if you have e.g. then the element "child" does not belong to the namespace indicated by "nspref" but to the "default namespace". An XML schema can change this default. However, to get such an XML schema effective, you must specify this wish when you are parsing your XML document. Otherwise, your XML document is parsed as a "stand alone" XML and the rules of the XML-namespace standard apply -- which means, that the namespace association is not inherited to child elements. From rustompmody at gmail.com Thu Jan 25 06:40:16 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Thu, 25 Jan 2018 03:40:16 -0800 (PST) Subject: xpath prob, was Re: Why is there no functional xml? In-Reply-To: References: <94eba3b9-9545-480a-a007-b84f6910c8f7@googlegroups.com> <3f25867b-f444-4727-afb9-a91b1b77f916@googlegroups.com> Message-ID: <0cc71919-dc40-4682-a3f5-0b9363448258@googlegroups.com> On Wednesday, January 24, 2018 at 2:31:22 PM UTC+5:30, Peter Otten wrote: > Rustom Mody wrote: > > > With > > # Read above xml > >>>> with open('soap_response.xml') as f: inp = etree.parse(f) > > # namespace dict > >>>> nsd = {'soap': "http://schemas.xmlsoap.org/soap/envelope/", 'locns': > >>>> "http://example.com/"} > > > > The following behavior is observed ? actual responses elided in the > > interest of brevity > > > >>>> inp.xpath('//soap:Body', namespaces = nsd) > > finds/reaches the node > > > >>>> inp.xpath('//locns:blobRetrieveResponse', namespaces = nsd) > > finds > > > >>>> inp.xpath('//locns:dtCreationDate', namespaces = nsd) > > does not find > > > >>>> inp.xpath('//dtCreationDate', namespaces = nsd) > > finds > > > >>>> inp.xpath('//dtCreationDate') > > also finds > > > > > > Doesnt this contradict the fact that dtCreationDate is under the locns > > namespace?? > > > > Any explanations?? > > Can you rewrite that question as a simple self-contained demo, similar to > the snippet shown under > > http://lxml.de/xpathxslt.html#namespaces-and-prefixes > > ? I guess Dieter has cleared [thanks Dieter] that namespaces dont inherit to child tags. I need to wrap my head around the concepts and the syntax From ned at nedbatchelder.com Thu Jan 25 06:42:34 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Thu, 25 Jan 2018 06:42:34 -0500 Subject: exec and traceback In-Reply-To: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> References: <20180122002216.Horde.F4kekgJA7h_QsYI9h_bOAFI@vps32166.inmotionhosting.com> Message-ID: <452c83e2-7af7-aea7-8045-615c65a32517@nedbatchelder.com> On 1/22/18 3:22 AM, ken.py at gameofy.com wrote: > > > I'm using exec() to run a (multi-line) string of python code. If an > exception occurs, I get a traceback containing a stack frame for the > string. I've labeled the code object with a "file name" so I can > identify it easily, and when I debug, I find that I can interact with > the context of that stack frame, which is pretty handy. > > What I would like to also be able to do is make the code string > visible to the debugger so I can look at and step through the code in > the string as if it were from a python file. > > Lest this topic forks into a security discussion, I'll just add that > for my purposes the data source is trusted. If you really want to talk > about the security of using exec and eval, fine, but start another > thread (BTW, I've written a simple secure eval()).... > I haven't tried this, but what if you write the string to a temporary file, and then claim that the code came from that file? Code objects are immutable, so you'll need to recompile a new code object, but then you can poke that object into the frame (I think?).? It will be fiddly, and may not work at all, as is typical with hacking at this level of the interpreter. I'm still interested in your simple secure eval if you have the time to tell us about it. --Ned. From cl at isbd.net Fri Jan 26 06:30:51 2018 From: cl at isbd.net (Chris Green) Date: Fri, 26 Jan 2018 11:30:51 +0000 Subject: How to diagnose this, fails on 3.6.3, works on 3.5.2? References: <1s4nje-arm.ln1@esprimo.zbmc.eu> Message-ID: Peter Otten <__peter__ at web.de> wrote: > Chris Green wrote: > > > I have a fairly simple little python program to automate starting an > > editor on a wiki page. It works fine on the system where I wrote it > > (xubuntu 16.04, python 3 version 3.5.2) but it comes up with the > > following error on a newer system (xubuntu 17.10, python 3 version > > 3.6.3). > > > > Here is the error:- > > > > chris$ no > > Traceback (most recent call last): > > File "/home/chris/bin/no", line 59, in > > os.execvp("vi", ("", monthFile,)) > > File "/usr/lib/python3.6/os.py", line 559, in execvp > > _execvpe(file, args) > > File "/usr/lib/python3.6/os.py", line 594, in _execvpe > > exec_func(fullname, *argrest) > > ValueError: execv() arg 2 first element cannot be empty > > > > Has execvp() become stricter in 3.6.3 or what? > > Yes; the relevant issue on the bug tracker seems to be > > https://bugs.python.org/issue28732 > OK, thanks all, I just need to put 'vi' in that empty parameter. -- Chris Green ? From chris at withers.org Fri Jan 26 09:07:24 2018 From: chris at withers.org (Chris Withers) Date: Fri, 26 Jan 2018 14:07:24 +0000 Subject: MailingLogger 4.0.0 Released! Message-ID: Only 4 years later than planned, I'm pleased to announce a new release of MailingLogger... Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested This release has a bunch of changes: - Drop Zope and Plone support - Drop ZConfig support - Removed the deprecated "ignore" parameter to MailingLogger and SummarisingLogger. - Move from zope.testrunner to pytest for running tests. - Switch from manuel to sybil for checking examples in documentation. - Moved from buildout to virtualenv for development. - Gracefully handle bugs elsewhere that call SummarisingLogger.close() more than once in a multi-threaded or multi-process environment. Full docs can be found here: http://mailinglogger.readthedocs.io/en/latest/ For more information, please see: https://github.com/Simplistix/mailinglogger/ cheers, Chris From abhi.darkness at gmail.com Fri Jan 26 12:47:12 2018 From: abhi.darkness at gmail.com (Abhiram R) Date: Fri, 26 Jan 2018 23:17:12 +0530 Subject: Doubt in line_profiler documentation Message-ID: ?I'm having trouble understanding something in the ?documentation of https://github.com/rkern/line_profiler The definition for the time column says - "Time: The total amount of time spent executing the line in the timer's units. In the header information before the tables, you will see a line 'Timer unit:' giving the conversion factor to seconds. It may be different on different systems." I don't really understand the conversion factor. For example, if the timer unit is :* 3.20802e-07 s* and a particular instruction's time column says its value is 83.0, is the time taken 83.0*3.20802e-07 s? Or is there more to it? If my understanding is correct however, why would there be a need for this? What could be the cause of this - " It may be different on different systems "? Can someone help me out? Thanks Abhiram R ? From mohammedfarazali at gmail.com Fri Jan 26 23:33:22 2018 From: mohammedfarazali at gmail.com (mohammedfarazali at gmail.com) Date: Fri, 26 Jan 2018 20:33:22 -0800 (PST) Subject: Please Help Message-ID: import numpy as np x=np.unit8([250) print(x) y=np.unit8([10]) print(y) z=x+y print(z) output [250] [10] [4] My question how is z [4] From jladasky at itu.edu Sat Jan 27 00:31:24 2018 From: jladasky at itu.edu (jladasky at itu.edu) Date: Fri, 26 Jan 2018 21:31:24 -0800 (PST) Subject: Please Help In-Reply-To: References: Message-ID: Please copy and paste the exact code you are running. The code you show has several syntax errors, and would not execute at all. Now, I think that I can read through your errors anyway, so let me ask you a question: what is the largest possible value that can be represented with an unsigned 8-bit integer? From gherron at digipen.edu Sat Jan 27 01:36:01 2018 From: gherron at digipen.edu (Gary Herron) Date: Fri, 26 Jan 2018 22:36:01 -0800 Subject: Please Help In-Reply-To: References: Message-ID: <9154d084-c0e0-fd19-5977-cad5ceece660@digipen.edu> On 01/26/2018 08:33 PM, mohammedfarazali at gmail.com wrote: > import numpy as np > x=np.unit8([250) > print(x) > y=np.unit8([10]) > print(y) > z=x+y > print(z) > > > output > > [250] > [10] > [4] > > My question how is z [4] Despite all the typos in your post, you appear to be doing 8 bit unsigned arithmetic.? Do you know what that means?? The answer you might have expected (i.e. 260) does not fit in the 0 ... 255 range of 8 bits, and so the result has overflowed and "wrapped around" to produce 4. Try this for a simpler example of the same: >>> np.uint8(260) 4 Gary Herron -- Dr. Gary Herron Professor of Computer Science DigiPen Institute of Technology (425) 895-4418 From xuanwu348 at 163.com Sat Jan 27 03:10:06 2018 From: xuanwu348 at 163.com (xuanwu348) Date: Sat, 27 Jan 2018 16:10:06 +0800 (CST) Subject: Please Help In-Reply-To: References: Message-ID: <7507153d.2943.16136aa9ede.Coremail.xuanwu348@163.com> np.uint8 is unsigned int, range 0~255 >>> np.uint8(256) 0 At 2018-01-27 12:33:22, "" wrote: >import numpy as np >x=np.unit8([250) >print(x) >y=np.unit8([10]) >print(y) >z=x+y >print(z) > > >output > >[250] >[10] >[4] > >My question how is z [4] >-- >https://mail.python.org/mailman/listinfo/python-list From alain at universite-de-strasbourg.fr.invalid Sat Jan 27 03:25:20 2018 From: alain at universite-de-strasbourg.fr.invalid (Alain Ketterlin) Date: Sat, 27 Jan 2018 09:25:20 +0100 Subject: Doubt in line_profiler documentation References: Message-ID: <87y3kjzp1r.fsf@universite-de-strasbourg.fr.invalid> Abhiram R writes: [...] > https://github.com/rkern/line_profiler > > The definition for the time column says - > > "Time: The total amount of time spent executing the line in the timer's > units. In the header information before the tables, you will see a line > 'Timer unit:' giving the conversion factor to seconds. It may be different > on different systems." > For example, if the timer unit is :* 3.20802e-07 s* > and a particular instruction's time column says its value is 83.0, is the > time taken 83.0*3.20802e-07 s? Or is there more to it? That's it. > If my understanding is correct however, why would there be a need for > this? What could be the cause of this - " It may be different on > different systems "? Time is a complicated thing on a computer, and is only measured with a certain precision (or "resolution"). This precision may vary from system to system. It is customary to mention the resolution when profiling, because the resolution is usually coarse wrt processor frequency (typically 1 microsecond, around 3000 processor cycles at 3Ghz). So profiling very short running pieces of code is highly inaccurate. You can look at the code used in rkern, at https://github.com/rkern/line_profiler/blob/master/timers.c You'll see that on Windows it uses QueryPerformanceCounter() [*] and QueryPerformanceFrequency(). On Unix it uses gettimeofday(), which has a fixed/conventional resolution. By the way, the use of gettimeofday() is strange since this function is now deprecated... clock_gettime() should be used instead. It has an associated clock_getres() as well. -- Alain. [*] WTF is wrong with these microsoft developpers? Clocks and performance counters are totally different things. what's the need for confusing terms in the API? From larry.martell at gmail.com Sat Jan 27 10:58:36 2018 From: larry.martell at gmail.com (Larry Martell) Date: Sat, 27 Jan 2018 10:58:36 -0500 Subject: error from Popen only when run from cron Message-ID: I have a script that does this: subprocess.Popen(['service', 'some_service', 'status'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) When I run it from the command line it works fine. When I run it from cron I get: subprocess.Popen(['service', 'some_service', 'status'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory Anyone have any clue as to what file it's complaining about? Or how I can debug this further? From grant.b.edwards at gmail.com Sat Jan 27 11:04:45 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sat, 27 Jan 2018 16:04:45 +0000 (UTC) Subject: error from Popen only when run from cron References: Message-ID: On 2018-01-27, Larry Martell wrote: > I have a script that does this: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > > When I run it from the command line it works fine. When I run it from > cron I get: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ > errread, errwrite) > File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory > > Anyone have any clue as to what file it's complaining about? Or how I > can debug this further? Try using the complete path of the executable. Cron jobs run with a pretty limited set of environment variables and may not have the PATH value you expect. -- Grant From rosuav at gmail.com Sat Jan 27 11:09:08 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Jan 2018 03:09:08 +1100 Subject: error from Popen only when run from cron In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 2:58 AM, Larry Martell wrote: > I have a script that does this: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > > When I run it from the command line it works fine. When I run it from > cron I get: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ > errread, errwrite) > File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory > > Anyone have any clue as to what file it's complaining about? Or how I > can debug this further? Looks like you're trying to invoke a process without a full path? It could be because cron jobs execute in a restricted environment. Two things to try: 1) Dump out os.environ to a file somewhere (maybe in /tmp if you don't have write permission). See if $PATH is set to some really short value, or at least to something different from what you see when you run it outside of cron. 2) Replace the word "service" with whatever you get from running "which service" on your system. On mine, it's "/usr/sbin/service". See what that does. Might not solve your problem, but it's worth a try. ChrisA From drsalists at gmail.com Sat Jan 27 11:30:26 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sat, 27 Jan 2018 08:30:26 -0800 Subject: error from Popen only when run from cron In-Reply-To: References: Message-ID: If you have your script set $PATH and $HOME, you can test it with: env - ./my-script The difference between running a script from the command line, and running a script from cron, is often environment variables. env - clears the environment. On Sat, Jan 27, 2018 at 7:58 AM, Larry Martell wrote: > I have a script that does this: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > > When I run it from the command line it works fine. When I run it from > cron I get: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ > errread, errwrite) > File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory > > Anyone have any clue as to what file it's complaining about? Or how I > can debug this further? > -- > https://mail.python.org/mailman/listinfo/python-list From larry.martell at gmail.com Sat Jan 27 11:42:00 2018 From: larry.martell at gmail.com (Larry Martell) Date: Sat, 27 Jan 2018 11:42:00 -0500 Subject: error from Popen only when run from cron In-Reply-To: References: Message-ID: On Sat, Jan 27, 2018 at 11:09 AM, Chris Angelico wrote: > On Sun, Jan 28, 2018 at 2:58 AM, Larry Martell wrote: >> I have a script that does this: >> >> subprocess.Popen(['service', 'some_service', 'status'], >> stdout=subprocess.PIPE, stderr=subprocess.STDOUT) >> >> When I run it from the command line it works fine. When I run it from >> cron I get: >> >> subprocess.Popen(['service', 'some_service', 'status'], >> stdout=subprocess.PIPE, stderr=subprocess.STDOUT) >> File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ >> errread, errwrite) >> File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child >> raise child_exception >> OSError: [Errno 2] No such file or directory >> >> Anyone have any clue as to what file it's complaining about? Or how I >> can debug this further? > > Looks like you're trying to invoke a process without a full path? It > could be because cron jobs execute in a restricted environment. Two > things to try: > > 1) Dump out os.environ to a file somewhere (maybe in /tmp if you don't > have write permission). See if $PATH is set to some really short > value, or at least to something different from what you see when you > run it outside of cron. > > 2) Replace the word "service" with whatever you get from running > "which service" on your system. On mine, it's "/usr/sbin/service". See > what that does. > > Might not solve your problem, but it's worth a try. Thanks! Using the full path fixed the issue. From best_lay at yahoo.com Sat Jan 27 11:52:23 2018 From: best_lay at yahoo.com (Wildman) Date: Sat, 27 Jan 2018 10:52:23 -0600 Subject: error from Popen only when run from cron References: Message-ID: <5PednRhwYefaM_HHnZ2dnUU7-TWdnZ2d@giganews.com> On Sat, 27 Jan 2018 10:58:36 -0500, Larry Martell wrote: > I have a script that does this: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > > When I run it from the command line it works fine. When I run it from > cron I get: > > subprocess.Popen(['service', 'some_service', 'status'], > stdout=subprocess.PIPE, stderr=subprocess.STDOUT) > File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__ > errread, errwrite) > File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child > raise child_exception > OSError: [Errno 2] No such file or directory > > Anyone have any clue as to what file it's complaining about? Or how I > can debug this further? Cron provides this as $PATH: /usr/bin;/usr/sbin >From within a terminal enter: whereis service If service is not in Cron's $PATH, that is your problem. Adding the complete path to 'service' in the script should fix things. If service is in Cron's $PATH, I have no further ideas. -- GNU/Linux user #557453 The cow died so I don't need your bull! From gjshen at gmail.com Sat Jan 27 12:25:35 2018 From: gjshen at gmail.com (George Shen) Date: Sat, 27 Jan 2018 11:25:35 -0600 Subject: Fwd: Text Strip() now working constantly. In-Reply-To: References: Message-ID: Hi Python Group, I am not sure if I am doing this correctly however, I believe I found a bug that involves the striping of a text string. I have attached a JPG that clearly illustrate the issue. I am currently using 2.7.13 In short: Given a string. 'cm_text.data' if you try and strip '.data' the return string will be 'cm_tex' which drops the last 't' however, given other text. 'om_ol.data' the same strip '.data' will return 'om_ol' which is correct! As for a work around right now I am doing the following. string_abc = 'some_text.data' string_next = string_abc.strip('data') string_final = string_next.strip('.') Please see the JPG. Sorry if this has been filed before, if I have filed this incorrectly could you please provide me a better avenue for future reference. Regards, -George J Shen From mittra at juno.com Sat Jan 27 13:01:47 2018 From: mittra at juno.com (qrious) Date: Sat, 27 Jan 2018 10:01:47 -0800 (PST) Subject: Data-structure for multiway associativity in Python Message-ID: I need a data structure and a corresponding (hopefully fast) mechanism associated with it to do the following. While I am looking for the concept first, my preference for implementation of this will be in Python. [c1, c2,..., cn] is a list of strings (for my own implementation, but could be any other type for the generic problem). There will be many hundreds, if not thousands, of such lists with no shared member. The method getAssocList(e) will return lists of the lists for which e is an element. Here a hash may be a way to go, but need help in figuring it out. Also, can there be a faster and more memory efficient solution other than hashes? From rosuav at gmail.com Sat Jan 27 13:25:42 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Jan 2018 05:25:42 +1100 Subject: Text Strip() now working constantly. In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 4:25 AM, George Shen wrote: > Hi Python Group, > > I am not sure if I am doing this correctly however, I believe I found a bug > that involves the striping of a text string. If you're implying that it's a bug in Python, no it isn't. If you're trying to understand the bug in your own code, you've come to the right place :) > I have attached a JPG that clearly illustrate the issue. Attachments aren't carried on this list; text is better anyway. > I am currently using 2.7.13 > > In short: > Given a string. > 'cm_text.data' > if you try and strip '.data' > the return string will be > 'cm_tex' Correct. In fact, the strip function does not remove a substring; it removes a set of characters. It's most commonly used for removing any whitespace from the ends of a string. It faithfully removed every ".", "d", "a", and "t" from your string - including the one at the end of "text". If you know for sure that the name WILL end with ".data", you can simply ask Python to remove the last five characters: >>> 'cm_text.data'[:-5] 'cm_text' That may better suit what you're doing. By the way, you may wish to consider migrating to Python 3, as Python 2.7 is a decade old and not getting any further enhancements. ChrisA From nasirahamed40719 at gmail.com Sat Jan 27 13:46:19 2018 From: nasirahamed40719 at gmail.com (nasirahamed40719 at gmail.com) Date: Sat, 27 Jan 2018 10:46:19 -0800 (PST) Subject: Text Strip() now working constantly. In-Reply-To: References: Message-ID: Hi, I'm a begginer in Python, i have a question... can we use replace method to do it. E.g. a='cm_text.data' a.replace('.data', ''), this will return output as 'cm_text'. From drsalists at gmail.com Sat Jan 27 13:49:24 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sat, 27 Jan 2018 10:49:24 -0800 Subject: Data-structure for multiway associativity in Python In-Reply-To: <14hp6d5rcg6ucmg81hsf7ehm908idsjj2l@4ax.com> References: <14hp6d5rcg6ucmg81hsf7ehm908idsjj2l@4ax.com> Message-ID: It's possible, but not common, to do association lists in Python. They're pretty inefficient in just about any language. I'm not totally clear on what you need, but it might be a good thing to do a list of sets - if you're looking for an in-memory solution. On Sat, Jan 27, 2018 at 10:33 AM, Dennis Lee Bieber wrote: > On Sat, 27 Jan 2018 10:01:47 -0800 (PST), qrious > declaimed the following: > >> >> >>I need a data structure and a corresponding (hopefully fast) mechanism associated with it to do the following. While I am looking for the concept first, my preference for implementation of this will be in Python. >> >>[c1, c2,..., cn] is a list of strings (for my own implementation, but could be any other type for the generic problem). There will be many hundreds, if not thousands, of such lists with no shared member. >> >>The method getAssocList(e) will return lists of the lists for which e is an element. >> >>Here a hash may be a way to go, but need help in figuring it out. Also, can there be a faster and more memory efficient solution other than hashes? > > > Don't know about speed or memory but... > > SQLite3 (*n* is primary key/auto increment; _n_ is foreign key) > > LoL(*ID*, description) > > anL(*ID*, _LoL_ID_, cx) > > select LoL.description, anL.cx from LoL > inner join anL on anL.LoL_ID = LoL.ID > where anL.cx like "%e%" > > {note: using like and wildcards means e is anywhere in the cx field; > otherwise just use > > anL.cx = "e" > } > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list From python at mrabarnett.plus.com Sat Jan 27 13:59:58 2018 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 27 Jan 2018 18:59:58 +0000 Subject: Data-structure for multiway associativity in Python In-Reply-To: References: Message-ID: <65845de7-f3bc-8543-05e3-9f8db0aefd05@mrabarnett.plus.com> On 2018-01-27 18:01, qrious wrote: > > > I need a data structure and a corresponding (hopefully fast) mechanism associated with it to do the following. While I am looking for the concept first, my preference for implementation of this will be in Python. > > [c1, c2,..., cn] is a list of strings (for my own implementation, but could be any other type for the generic problem). There will be many hundreds, if not thousands, of such lists with no shared member. > > The method getAssocList(e) will return lists of the lists for which e is an element. > > Here a hash may be a way to go, but need help in figuring it out. Also, can there be a faster and more memory efficient solution other than hashes? > You could build a dict where the key is the element and the value is a list of those lists that contain that element. The 'defaultdict' class is useful for that. from collections import defaultdict assoc_dict = defaultdict(list) for lst in list_of_lists: for elem in lst: assoc_dict[elem].append(lst) # Optionally convert to a plain dict once it's built. assoc_dict = dict(assoc_dict) From rosuav at gmail.com Sat Jan 27 14:10:06 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Jan 2018 06:10:06 +1100 Subject: Text Strip() now working constantly. In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 5:46 AM, wrote: > Hi, I'm a begginer in Python, i have a question... > > can we use replace method to do it. > > E.g. a='cm_text.data' > a.replace('.data', ''), this will return output as 'cm_text'. Yep! That works too. Be aware, though, that it might replace ".data" in the middle of the string, too. If you know for sure that that won't happen, then go for it! BTW, when you reply to someone else's message, it's best to include a bit of the original text and then type your reply below it - like I do in this message. That way, people get enough context to understand what's going on. ChrisA From jqian at tibco.com Sat Jan 27 15:15:41 2018 From: jqian at tibco.com (Jason Qian) Date: Sat, 27 Jan 2018 15:15:41 -0500 Subject: Please help on print string that contains 'tab' and 'newline' Message-ID: HI I am a string that contains \r\n\t [Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet I would like it print as : [Ljava.lang.Object; does not exist tat com.livecluster.core.tasklet From jqian at tibco.com Sat Jan 27 15:17:45 2018 From: jqian at tibco.com (Jason Qian) Date: Sat, 27 Jan 2018 15:17:45 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: HI I have a string that contains \r\n\t [Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet I would like to print it as : [Ljava.lang.Object; does not exist tat com.livecluster.core.tasklet How can I do this in python print ? Thanks On Sat, Jan 27, 2018 at 3:15 PM, Jason Qian wrote: > HI > > I am a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at > com.livecluster.core.tasklet > > > I would like it print as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet > > > From breamoreboy at gmail.com Sat Jan 27 15:55:45 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Sat, 27 Jan 2018 12:55:45 -0800 (PST) Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: <7086999e-d90e-4448-ab95-7f6d9af19b45@googlegroups.com> On Saturday, January 27, 2018 at 8:16:58 PM UTC, Jason Qian wrote: > HI > > I am a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet > > I would like it print as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet Unless I've missed something just call print on the string. >>> print('[Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet') [Ljava.lang.Object; does not exist* *at com.livecluster.core.tasklet -- Kindest regards. Mark Lawrence. From barry at python.org Sat Jan 27 16:02:08 2018 From: barry at python.org (Barry Warsaw) Date: Sat, 27 Jan 2018 16:02:08 -0500 Subject: =?utf-8?Q?Welcome_the_3=2E8_and_3=2E9_Release_Manager_-_=C5=81uka?= =?utf-8?Q?sz_Langa!?= Message-ID: As Ned just announced, Python 3.7 is very soon to enter beta 1 and thus feature freeze. I think we can all give Ned a huge round of applause for his amazing work as Release Manager for Python 3.6 and 3.7. Let?s also give him all the support he needs to make 3.7 the best version yet. As is tradition, Python release managers serve for two consecutive releases, and so with the 3.7 release branch about to be made, it?s time to announce our release manager for Python 3.8 and 3.9. By unanimous and enthusiastic consent from the Python Secret Underground (PSU, which emphatically does not exist), the Python Cabal of Former and Current Release Managers, Cardinal Xim?nez, and of course the BDFL, please welcome your next release manager? ?ukasz Langa! And also, happy 24th anniversary to Guido?s Python 1.0.0 announcement[1]. It?s been a fun and incredible ride, and I firmly believe that Python?s best days are ahead of us. Enjoy, -Barry [1] https://groups.google.com/forum/?hl=en#!original/comp.lang.misc/_QUzdEGFwCo/KIFdu0-Dv7sJ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From tkadm30 at yandex.com Sat Jan 27 16:04:42 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sat, 27 Jan 2018 16:04:42 -0500 Subject: How to embed a native JIT compiler to a django app? Message-ID: Hi, I want to compile a Django application into a C source file and embed a JIT compiler into the binary. Is there any way of doing this with llvm/clang? Regards, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From mittra at juno.com Sat Jan 27 16:05:53 2018 From: mittra at juno.com (qrious) Date: Sat, 27 Jan 2018 13:05:53 -0800 (PST) Subject: Sentiment analysis using sklearn Message-ID: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> I am attempting to understand how scikit learn works for sentiment analysis and came across this blog post: https://marcobonzanini.wordpress.com/2015/01/19/sentiment-analysis-with-python-and-scikit-learn The corresponding code is at this location: https://gist.github.com/bonzanini/c9248a239bbab0e0d42e My question is while trying to predict, why does the curr_class in Line 44 of the code need a classification (pos or neg) for the test data? After all, am I not trying to predict it? Without any initial value of curr_class, the program has a run time error. Any help will be greatly appreciated. From ericsnowcurrently at gmail.com Sat Jan 27 16:14:35 2018 From: ericsnowcurrently at gmail.com (Eric Snow) Date: Sat, 27 Jan 2018 14:14:35 -0700 Subject: =?UTF-8?Q?Re=3A_=5Bpython=2Dcommitters=5D_Welcome_the_3=2E8_and_3=2E9_Rele?= =?UTF-8?Q?ase_Manager_=2D_=C5=81ukasz_Langa=21?= In-Reply-To: References: Message-ID: On Sat, Jan 27, 2018 at 2:02 PM, Barry Warsaw wrote: > please welcome your next release manager? > > ?ukasz Langa! Congrats, ?ukasz! (or condolences? ) -eric From ned at nedbatchelder.com Sat Jan 27 16:20:32 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Sat, 27 Jan 2018 16:20:32 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> On 1/27/18 3:15 PM, Jason Qian via Python-list wrote: > HI > > I am a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet > > > I would like it print as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet It looks like you are doing something that is a little bit, and perhaps a lot, more complicated than printing a string.? Can you share the code that is trying to produce that output? --Ned. From tjreedy at udel.edu Sat Jan 27 17:39:06 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Jan 2018 17:39:06 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: On 1/27/2018 3:15 PM, Jason Qian via Python-list wrote: > HI > > I am a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet > > > I would like it print as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet Your output does not match the input. Don't add, or else remove, the *s if you don't want to see them. Having '\t' print as ' t' makes no sense. In IDLE >>> print('[Ljava.lang.Object; does not exist*\r\n\t*at com.livecluster.core.tasklet') [Ljava.lang.Object; does not exist* *at com.livecluster.core.tasklet IDLE ignores \r, other display mechanisms may not. You generally should not use it. Pasting the text, with the literal newline embedded, does not work in Windows interactive interpreter, so not testing this there. -- Terry Jan Reedy From tjreedy at udel.edu Sat Jan 27 17:44:47 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Jan 2018 17:44:47 -0500 Subject: Sentiment analysis using sklearn In-Reply-To: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> References: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> Message-ID: On 1/27/2018 4:05 PM, qrious wrote: > I am attempting to understand how scikit learn works for sentiment analysis and came across this blog post: > > https://marcobonzanini.wordpress.com/2015/01/19/sentiment-analysis-with-python-and-scikit-learn > > The corresponding code is at this location: > > https://gist.github.com/bonzanini/c9248a239bbab0e0d42e > > My question is while trying to predict, why does the curr_class in Line 44 of the code need a classification (pos or neg) for the test data? After all, am I not trying to predict it? Without any initial value of curr_class, the program has a run time error. In order for the 'bot' to classify new samples, by learning the difference between positive and negative samples, it needs to be trained on existing samples that are 'correctly' classified. -- Terry Jan Reedy From drsalists at gmail.com Sat Jan 27 20:20:46 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sat, 27 Jan 2018 17:20:46 -0800 Subject: Sentiment analysis using sklearn In-Reply-To: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> References: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> Message-ID: On Sat, Jan 27, 2018 at 1:05 PM, qrious wrote: > I am attempting to understand how scikit learn works for sentiment analysis and came across this blog post: > > https://marcobonzanini.wordpress.com/2015/01/19/sentiment-analysis-with-python-and-scikit-learn > > The corresponding code is at this location: > > https://gist.github.com/bonzanini/c9248a239bbab0e0d42e > > My question is while trying to predict, why does the curr_class in Line 44 of the code need a classification (pos or neg) for the test data? After all, am I not trying to predict it? Without any initial value of curr_class, the program has a run time error. I'm a real neophyte when it comes to modern AI, but I believe the intent is to divide your inputs into "training data" and "test data" and "real world data". So you create your models using training data including correct classifications as part of the input. And you check how well your models are doing on inputs they haven't seen before with test data, which also is classified in advance, to verify how well things are working. And then you use real world, as-yet-unclassified data in production, after you've selected your best model, to derive a classification from what your model has seen in the past. So both the training data and test data need accurate labels in advance, but the real world data trusts the model to do pretty well without further labeling. From jqian at tibco.com Sat Jan 27 20:33:58 2018 From: jqian at tibco.com (Jason Qian) Date: Sat, 27 Jan 2018 20:33:58 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: Thanks for taking look this. 1. Python pass a function to c side as callback, and print out the message. def handleError(message, code): print('** handleError **') * print('exception ' + str(message))* 2. On c side : send stack trace back to python by calling the callback function Callback::Callback(InvocationER rcb) : _rcb(rcb) { } void Callback::handleError(Exception &e, int taskId) { *(_rcb)((char*)e.getStackTrace().c_str(), taskId);* } So, the source of the string is std::string. On the python side is byte array. Ljava.lang.Object; does not exist*\r\n\t*at com Thanks On Sat, Jan 27, 2018 at 4:20 PM, Ned Batchelder wrote: > On 1/27/18 3:15 PM, Jason Qian via Python-list wrote: > >> HI >> >> I am a string that contains \r\n\t >> >> [Ljava.lang.Object; does not exist*\r\n\t*at >> com.livecluster.core.tasklet >> >> >> I would like it print as : >> >> [Ljava.lang.Object; does not exist >> tat com.livecluster.core.tasklet >> > > It looks like you are doing something that is a little bit, and perhaps a > lot, more complicated than printing a string. Can you share the code that > is trying to produce that output? > > --Ned. > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Sat Jan 27 20:39:38 2018 From: jqian at tibco.com (Jason Qian) Date: Sat, 27 Jan 2018 20:39:38 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: Thanks for taking look this. The source of the string is std::string from our c code as callback . On the python side is shows as bytes. Is there way we can reformat the string that replace \r\n with newline, so python can correctly print it ? Thanks On Sat, Jan 27, 2018 at 5:39 PM, Terry Reedy wrote: > On 1/27/2018 3:15 PM, Jason Qian via Python-list wrote: > >> HI >> >> I am a string that contains \r\n\t >> >> [Ljava.lang.Object; does not exist*\r\n\t*at >> com.livecluster.core.tasklet >> >> >> I would like it print as : >> >> [Ljava.lang.Object; does not exist >> tat com.livecluster.core.tasklet >> > > Your output does not match the input. Don't add, or else remove, the *s > if you don't want to see them. Having '\t' print as ' t' makes no sense. > > In IDLE > > >>> print('[Ljava.lang.Object; does not exist*\r\n\t*at > com.livecluster.core.tasklet') > [Ljava.lang.Object; does not exist* > *at com.livecluster.core.tasklet > > IDLE ignores \r, other display mechanisms may not. You generally should > not use it. > > Pasting the text, with the literal newline embedded, does not work in > Windows interactive interpreter, so not testing this there. > > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list > From acharbly at gmail.com Sat Jan 27 21:18:18 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Sun, 28 Jan 2018 07:48:18 +0530 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: In python 3. X Use. Decode and. Encode On 28 Jan 2018 1:47 am, "Jason Qian via Python-list" wrote: > HI > > I am a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at > com.livecluster.core.tasklet > > > I would like it print as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet > -- > https://mail.python.org/mailman/listinfo/python-list > From jqian at tibco.com Sat Jan 27 21:23:02 2018 From: jqian at tibco.com (Jason Qian) Date: Sat, 27 Jan 2018 21:23:02 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: there are 0D 0A 09 %c %d 116 *%c %d 13%c %d 10%c %d 9* %c %d 97 On Sat, Jan 27, 2018 at 9:05 PM, Dennis Lee Bieber wrote: > On Sat, 27 Jan 2018 20:33:58 -0500, Jason Qian via Python-list > declaimed the following: > > > Ljava.lang.Object; does not exist*\r\n\t*at com > > > > Does that source contain > > 0x0d 0x0a 0x09 > > > or is it really > > 0x5c 0x72 0x5c 0x61 0x5c 0x74 > \ r \ n \ t > > as individual characters? > > > >>> bks = chr(0x5c) > >>> ar = "r" > >>> en = "n" > >>> te = "t" > >>> > >>> strn = "".join([bks, ar, bks, en, bks, te]) > >>> strn > '\\r\\n\\t' > >>> print strn > \r\n\t > >>> cstr = "\r\n\t" > >>> cstr > '\r\n\t' > >>> print cstr > > > >>> > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com HTTP://wlfraed.home.netcom.com/ > > -- > https://mail.python.org/mailman/listinfo/python-list > From gg.galvez at gmail.com Sat Jan 27 21:23:53 2018 From: gg.galvez at gmail.com (Gg Galvez) Date: Sat, 27 Jan 2018 18:23:53 -0800 (PST) Subject: Sending Email using examples From Tutorials Message-ID: <4582fe34-1c8c-4cb0-9760-3891c0fb7fe2@googlegroups.com> I am having difficulty getting the python script to send an email to work. Here is the code I use from among a number of other examples which I used. The only changes I made were the email addresses, so I can see the result if it works. If you have any suggestions, please email your reply also to gg.galvez at gmail.com. Python Script used: import smtplib server = smtplib.SMTP('localhost') server.sendmail('gg77galvez at yahoo.com', """To: gg.galvez at gmail.com From: gg77galvez at yahoo.com Beware the Ides of March. """) server.quit() when running this I get the following message. Please help: Traceback (most recent call last): File "D:\ProgramDev\PythonDev\sendemail.py", line 3, in server = smtplib.SMTP('localhost') File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", line 251, in __init__ (code, msg) = self.connect(host, port) File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", line 335, in connect self.sock = self._get_socket(host, port, self.timeout) File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", line 306, in _get_socket self.source_address) File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 722, in create_connection raise err File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\socket.py", line 713, in create_connection sock.connect(sa) ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it From jsf80238 at gmail.com Sat Jan 27 23:02:35 2018 From: jsf80238 at gmail.com (Jason Friedman) Date: Sat, 27 Jan 2018 21:02:35 -0700 Subject: Sending Email using examples From Tutorials In-Reply-To: <4582fe34-1c8c-4cb0-9760-3891c0fb7fe2@googlegroups.com> References: <4582fe34-1c8c-4cb0-9760-3891c0fb7fe2@googlegroups.com> Message-ID: > > import smtplib > server = smtplib.SMTP('localhost') > server.sendmail('gg77galvez at yahoo.com', > """To: gg.galvez at gmail.com > From: gg77galvez at yahoo.com > > Beware the Ides of March. > """) > server.quit() > > when running this I get the following message. Please help: > > Traceback (most recent call last): > File "D:\ProgramDev\PythonDev\sendemail.py", line 3, in > server = smtplib.SMTP('localhost') > File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", > line 251, in __init__ > (code, msg) = self.connect(host, port) > File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", > line 335, in connect > self.sock = self._get_socket(host, port, self.timeout) > File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\smtplib.py", > line 306, in _get_socket > self.source_address) > File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\socket.py", > line 722, in create_connection > raise err > File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\socket.py", > line 713, in create_connection > sock.connect(sa) > ConnectionRefusedError: [WinError 10061] No connection could be made > because the target machine actively refused it Your code is fine, but to send mail you need a SMTP server that accepts connections. With: server = smtplib.SMTP('localhost') your code is saying that you have an SMTP server listening on your localhost. I don't think you do. From pendrysammuel at gmail.com Sun Jan 28 00:26:06 2018 From: pendrysammuel at gmail.com (pendrysammuel at gmail.com) Date: Sat, 27 Jan 2018 21:26:06 -0800 (PST) Subject: Compression of random binary data In-Reply-To: References: Message-ID: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> If it is then show him this 387,420,489 = 00110011 00111000 00110111 00101100 00110100 00110010 00110000 00101100 00110100 00111000 00111001 9^9 = ?? (^ = to the power of) = 387,420,489 But 9^9 = 00111001 01011110 00111001 From rosuav at gmail.com Sun Jan 28 00:37:37 2018 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Jan 2018 16:37:37 +1100 Subject: Compression of random binary data In-Reply-To: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> Message-ID: On Sun, Jan 28, 2018 at 4:26 PM, wrote: > If it is then show him this > > 387,420,489 > = > 00110011 00111000 00110111 00101100 00110100 00110010 00110000 00101100 00110100 00111000 00111001 > > 9^9 = ?? (^ = to the power of) > = 387,420,489 > > But > > 9^9 > = > 00111001 01011110 00111001 I have no idea what you're responding to or how this has anything to do with the thread you're replying to, but I would advise you to look into the difference between exponentiation and exclusive-or. Even so, I don't know what result you got. ChrisA From pendrysammuel at gmail.com Sun Jan 28 00:50:24 2018 From: pendrysammuel at gmail.com (pendrysammuel at gmail.com) Date: Sat, 27 Jan 2018 21:50:24 -0800 (PST) Subject: Compression of random binary data In-Reply-To: References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> Message-ID: 387,420,489 is a number with only 2 repeating binary sequences In binary 387,420,489 is expressed as 00110011 00111000 00110111 00101100 00110100 00110010 00110000 00101100 00110100 00111000 00111001 387,420,489 can be simplified to 9*9 or nine to the power of nine In binary 9*9 is represented by 00111001 00101010 00111001 9*9 and 387,420,489 are the same thing they are both a representation of numbers or data, yet in binary 9*9 is 64 bits shorter than 387,420,489 Not a solution just help. From pendrysammuel at gmail.com Sun Jan 28 01:14:46 2018 From: pendrysammuel at gmail.com (pendrysammuel at gmail.com) Date: Sat, 27 Jan 2018 22:14:46 -0800 (PST) Subject: Compression of random binary data (Posting On Python-List Prohibited) In-Reply-To: <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> Message-ID: <62f9eaac-317c-4844-ae15-574ae4fe97cb@googlegroups.com> I have it in my head, just need someone to write the program for me, I know nothing about data compression or binary data other than 1s and 0s and that you can not take 2 number without a possible value more or less than them selves and compress them, I have been working for 1 1/2 years on a solution, just need help with programming. If someone could get ahold of me when I am sober I could probably explain it a lot better, but I said fuck it I?ll show everyone a possibility instead of trying to do it on my own. From pendrysammuel at gmail.com Sun Jan 28 01:15:46 2018 From: pendrysammuel at gmail.com (pendrysammuel at gmail.com) Date: Sat, 27 Jan 2018 22:15:46 -0800 (PST) Subject: Compression of random binary data (Posting On Python-List Prohibited) In-Reply-To: <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> Message-ID: Lawrence D?Oliveiro In other words yes, I just need to be sober first. From mittra at juno.com Sun Jan 28 01:53:24 2018 From: mittra at juno.com (qrious) Date: Sat, 27 Jan 2018 22:53:24 -0800 (PST) Subject: Sentiment analysis using sklearn In-Reply-To: References: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> Message-ID: On Saturday, January 27, 2018 at 2:45:30 PM UTC-8, Terry Reedy wrote: > On 1/27/2018 4:05 PM, qrious wrote: > > I am attempting to understand how scikit learn works for sentiment analysis and came across this blog post: > > > > https://marcobonzanini.wordpress.com/2015/01/19/sentiment-analysis-with-python-and-scikit-learn > > > > The corresponding code is at this location: > > > > https://gist.github.com/bonzanini/c9248a239bbab0e0d42e > > > > My question is while trying to predict, why does the curr_class in Line 44 of the code need a classification (pos or neg) for the test data? After all, am I not trying to predict it? Without any initial value of curr_class, the program has a run time error. > > In order for the 'bot' to classify new samples, by learning the > difference between positive and negative samples, it needs to be trained > on existing samples that are 'correctly' classified. > > > -- > Terry Jan Reedy The training samples already do that. I think Dan's reply below makes sense. From mittra at juno.com Sun Jan 28 01:59:00 2018 From: mittra at juno.com (qrious) Date: Sat, 27 Jan 2018 22:59:00 -0800 (PST) Subject: Sentiment analysis using sklearn In-Reply-To: References: <5dd8ff96-702f-4f47-9388-84e52cc08eb6@googlegroups.com> Message-ID: <52f46c61-bfc8-4276-af7f-f070f87aea6d@googlegroups.com> On Saturday, January 27, 2018 at 5:21:15 PM UTC-8, Dan Stromberg wrote: > On Sat, Jan 27, 2018 at 1:05 PM, qrious wrote: > > I am attempting to understand how scikit learn works for sentiment analysis and came across this blog post: > > > > https://marcobonzanini.wordpress.com/2015/01/19/sentiment-analysis-with-python-and-scikit-learn > > > > The corresponding code is at this location: > > > > https://gist.github.com/bonzanini/c9248a239bbab0e0d42e > > > > My question is while trying to predict, why does the curr_class in Line 44 of the code need a classification (pos or neg) for the test data? After all, am I not trying to predict it? Without any initial value of curr_class, the program has a run time error. > > I'm a real neophyte when it comes to modern AI, but I believe the > intent is to divide your inputs into "training data" and "test data" > and "real world data". > > So you create your models using training data including correct > classifications as part of the input. > > And you check how well your models are doing on inputs they haven't > seen before with test data, which also is classified in advance, to > verify how well things are working. > > And then you use real world, as-yet-unclassified data in production, > after you've selected your best model, to derive a classification from > what your model has seen in the past. > > So both the training data and test data need accurate labels in > advance, but the real world data trusts the model to do pretty well > without further labeling. Dan, Thanks and I was also thinking along this line: 'So both the training data and test data need accurate labels in advance'. It makes sense to me. For this part: 'the real world data trusts the model to do pretty well without further labeling', the question is: how do I do this using sklearn library functions? Is there some code example for using the actual data that needs prediction? From __peter__ at web.de Sun Jan 28 03:57:01 2018 From: __peter__ at web.de (Peter Otten) Date: Sun, 28 Jan 2018 09:57:01 +0100 Subject: Please help on print string that contains 'tab' and 'newline' References: Message-ID: Jason Qian via Python-list wrote: > HI > > I have a string that contains \r\n\t > > [Ljava.lang.Object; does not exist*\r\n\t*at > [com.livecluster.core.tasklet > > > I would like to print it as : > > [Ljava.lang.Object; does not exist > tat com.livecluster.core.tasklet > > How can I do this in python print ? Assuming the string contains the escape sequences rather than an actual TAB, CR or NL you can apply codecs.decode(): >>> s = r"[Ljava.lang.Object; does not exist\r\n\tat com.livecluster.core.tasklet" >>> print(s) [Ljava.lang.Object; does not exist\r\n\tat com.livecluster.core.tasklet >>> import codecs >>> print(codecs.decode(s, "unicode-escape")) [Ljava.lang.Object; does not exist at com.livecluster.core.tasklet Note that this will decode all escape sequences that may occur in a string literal in Python, e. g. >>> codecs.decode(r"\x09 \u03c0 \N{soft ice cream}", "unicode-escape") '\t ? ?' and will complain when the string is not a valid Python string literal: >>> codecs.decode(r"\x", "unicode-escape") Traceback (most recent call last): File "", line 1, in UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 0-1: truncated \xXX escape If you need more control you can build your own conversion routine: import re lookup = { "r": "\r", "n": "\n", "t": "\t", } def substitute(match): group = match.group(1) return lookup.get(group, group) def decode(s): return re.sub(r"\\(.)", substitute, s) s = decode("alpha\\n \\xomega\\\\") print(s) print(repr(s)) From stephane at wirtel.be Sun Jan 28 08:32:28 2018 From: stephane at wirtel.be (Stephane Wirtel) Date: Sun, 28 Jan 2018 14:32:28 +0100 Subject: How to embed a native JIT compiler to a django app? In-Reply-To: References: Message-ID: <20180128133228.GA25126@xps> On 01/27, Etienne Robillard wrote: >Hi, > >I want to compile a Django application into a C source file and embed >a JIT compiler into the binary. Is there any way of doing this with >llvm/clang? Hi Etienne, I think no, Django will use Python and this one is interpreted. Answer, no... Now, you could try to create a wheel file with your django project and try that, but it is really complex. Why do you want to package a Django app into a C file with a compiler? -- St?phane Wirtel - http://wirtel.be - @matrixise From steve+comp.lang.python at pearwood.info Sun Jan 28 09:07:45 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 14:07:45 +0000 (UTC) Subject: Fwd: Text Strip() now working constantly. References: Message-ID: On Sat, 27 Jan 2018 11:25:35 -0600, George Shen wrote: > I have attached a JPG that clearly illustrate the issue. [...] > Please see the JPG. And what of us who are blind or visually impaired and use a screen reader to read text? Unless you use Photoshop to edit your source code, why are you using JPGs to show us code? Code is text. Copy and paste the text into your message. Thank you. -- Steve From rosuav at gmail.com Sun Jan 28 09:12:33 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jan 2018 01:12:33 +1100 Subject: Fwd: Text Strip() now working constantly. In-Reply-To: References: Message-ID: On Mon, Jan 29, 2018 at 1:07 AM, Steven D'Aprano wrote: > On Sat, 27 Jan 2018 11:25:35 -0600, George Shen wrote: > >> I have attached a JPG that clearly illustrate the issue. > [...] >> Please see the JPG. > > And what of us who are blind or visually impaired and use a screen reader > to read text? > > Unless you use Photoshop to edit your source code, why are you using JPGs > to show us code? Code is text. Copy and paste the text into your message. ... obligatory XKCD: https://xkcd.com/1685/ ChrisA From steve+comp.lang.python at pearwood.info Sun Jan 28 09:42:43 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 14:42:43 +0000 (UTC) Subject: Data-structure for multiway associativity in Python References: Message-ID: On Sat, 27 Jan 2018 10:01:47 -0800, qrious wrote: > I need a data structure and a corresponding (hopefully fast) mechanism > associated with it to do the following. While I am looking for the > concept first, my preference for implementation of this will be in > Python. > > [c1, c2,..., cn] is a list of strings (for my own implementation, but > could be any other type for the generic problem). There will be many > hundreds, if not thousands, of such lists with no shared member. > > The method getAssocList(e) will return lists of the lists for which e is > an element. Since you specified that there are no lists with shared members, why bother returning a list of lists? There will only ever be a single matching list. data = [ [c1, c2, ..., cn], [d1, d2, ..., dn], # hundreds more... [z1, z2, ..., zn], ] def getAssocList(element): for L in data: if element in L: return L raise ValueError('not found') For faster results, use sets {c1, c2, ..., cn} rather than lists. The outer list still has to be a list though. To speed it up more, we'd need to know more information about how you are using this. For example, if the values c1, ... d1, ... etc have some sort of relationship, you might be able to generate some kind of multiway tree that avoids having to search all of the thousands of lists before giving up. Are searches going to typically hit the same set c1... over and over again? If so, then after matching, bring it to the front of the master list. (You might want to use a deque for that.) > Here a hash may be a way to go, but need help in figuring it out. Also, > can there be a faster and more memory efficient solution other than > hashes? Probably not, not unless you have some sort of internal structure you can take advantage of. For example, if all the strings in any group start with the same letter, then you can dispatch directly to the right list: data = {'c': {c1, c2, c3, ..., cn}, 'd': {d1, ... } # and so on... } def getAssocList(element): if element in data[element[0]]: return L raise ValueError('not found') But if there is no structure at all, and the elements in each list are completely arbitrary, then there is nothing better than a linear search through the entire collection of lists, looking at every single element until you've matched what you're after. But your description is pretty vague and for all I know what you actually want is already a solved problem. Can you give more detail and cut-down example of your data set? Say, two or three values per list, and two or three lists. -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 09:50:31 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 14:50:31 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sat, 27 Jan 2018 21:23:02 -0500, Jason Qian via Python-list wrote: > there are 0D 0A 09 If your string actually contains CARRIAGE RETURN (OD) NEWLINE (OA), and TAB (09) characters, then you don't need to do anything. Just call print, and they will be printed correctly. If that doesn't work, then your input string doesn't contain what you think it contains. Please call this: print(repr(the_string)) and COPY AND PASTE the output here so we can see it. -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 10:04:26 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 15:04:26 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? Message-ID: I'm seeing this annoying practice more and more often. Even for trivial pieces of text, a few lines, people post screenshots instead of copying the code. Where has this meme come from? It seems to be one which inconveniences *everyone* involved: - for the sender, instead of a simple copy and paste, they have to take a screen shot, possibly trim the image to remove any bits of the screen they don't want to show, attach it to their email or upload it to an image hosting site; - for the receiver, you are reliant on a forum which doesn't strip attachments, or displays externally hosted images; the visually impaired are excluded from using a screen reader; and nobody can copy or edit the given text. It is as if people are deliberately inconveniencing themselves in order to inconvenience the people they are asking to help them. With the exception of one *exceedingly* overrated advantage, namely the ability to annotate the image with coloured lines and circles and squiggles or other graphics (which most people don't bother to do), this seems to me to be 100% counter-productive for everyone involved. Why has it spread and why do people keep doing it? I don't want to be the old man yelling "Get Of My Lawn!" to the cool kids, but is this just another sign of the downward spiral of programming talent? Convince me that there is *some* justification for this practice. Even a tiny one. (The day a programmer posts a WAV file of themselves reading their code out aloud, is the day I turn my modem off and leave the internet forever.) -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 10:08:57 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 15:08:57 +0000 (UTC) Subject: Compression of random binary data References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> <62f9eaac-317c-4844-ae15-574ae4fe97cb@googlegroups.com> Message-ID: On Sat, 27 Jan 2018 22:14:46 -0800, pendrysammuel wrote: > I have it in my head, just need someone to write the program for me, Sure, my rate is $150 an hour. > I > know nothing about data compression or binary data other than 1s and 0s > and that you can not take 2 number without a possible value more or less > than them selves and compress them, I have been working for 1 1/2 years > on a solution, just need help with programming. Sounds like you should have spend two hours on learning a bit about data compression (start with Wikipedia) instead of wasting 1.5 years trying to guess a solution for something you know nothing about. > If someone could get ahold of me when I am sober O_o Make that $550 an hour. Payable in advance. -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 10:10:09 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 15:10:09 +0000 (UTC) Subject: Compression of random binary data References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> Message-ID: On Sat, 27 Jan 2018 21:50:24 -0800, pendrysammuel wrote: > 387,420,489 is a number with only 2 repeating binary sequences Okay. Now try these two numbers: 387420479 387420499 -- Steve From best_lay at yahoo.com Sun Jan 28 10:32:18 2018 From: best_lay at yahoo.com (Wildman) Date: Sun, 28 Jan 2018 09:32:18 -0600 Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> On Sun, 28 Jan 2018 15:04:26 +0000, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: > > - for the sender, instead of a simple copy and paste, they have to take a > screen shot, possibly trim the image to remove any bits of the screen > they don't want to show, attach it to their email or upload it to an > image hosting site; > > - for the receiver, you are reliant on a forum which doesn't strip > attachments, or displays externally hosted images; the visually impaired > are excluded from using a screen reader; and nobody can copy or edit the > given text. > > It is as if people are deliberately inconveniencing themselves in order > to inconvenience the people they are asking to help them. > > With the exception of one *exceedingly* overrated advantage, namely the > ability to annotate the image with coloured lines and circles and > squiggles or other graphics (which most people don't bother to do), this > seems to me to be 100% counter-productive for everyone involved. Why has > it spread and why do people keep doing it? > > I don't want to be the old man yelling "Get Of My Lawn!" to the cool > kids, but is this just another sign of the downward spiral of programming > talent? Convince me that there is *some* justification for this practice. > Even a tiny one. > > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) I can think of no justification for it. -- GNU/Linux user #557453 May the Source be with you. From skip.montanaro at gmail.com Sun Jan 28 10:35:35 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 28 Jan 2018 09:35:35 -0600 Subject: Sending Email using examples From Tutorials In-Reply-To: References: <4582fe34-1c8c-4cb0-9760-3891c0fb7fe2@googlegroups.com> Message-ID: Your code is fine, ... In addition to what Jason wrote, note that the way you need to authenticate to most email servers has changed substantially since this tutorial example was written. The OP has a yahoo.com email address. Even assuming you used something like smtp.yahoo.com as the SMTP server, it's unlikely you could just send mail through that host without some sort of authentication. Skip From acharbly at gmail.com Sun Jan 28 10:39:52 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Sun, 28 Jan 2018 21:09:52 +0530 Subject: IoT automation In-Reply-To: References: Message-ID: Hello team, Could you please help me out in automation of IoT product end to end Regards Prahallad From Karsten.Hilbert at gmx.net Sun Jan 28 10:41:32 2018 From: Karsten.Hilbert at gmx.net (Karsten Hilbert) Date: Sun, 28 Jan 2018 16:41:32 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <20180128154132.GJ8707@hermes.hilbert.loc> On Sun, Jan 28, 2018 at 03:04:26PM +0000, Steven D'Aprano wrote: > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) And the clever hack will be to send a WAV that tricks your modem into surprising things by whistling just the right way ... :-) Best, Karsten -- GPG key ID E4071346 @ eu.pool.sks-keyservers.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 From skip.montanaro at gmail.com Sun Jan 28 10:43:37 2018 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Sun, 28 Jan 2018 09:43:37 -0600 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> Message-ID: I've noticed it as well. I suspect it's from the Windows universe where it's common to snip a bit of the screen which isn't pure text when asking about some problematic GUI thing which is causing problems. I've never been a Windows user, but at my current job, Windows is core to just about everything, so I am forced to use it for a lot of stuff (Outlook, SQL Server, Excel, etc). I see lots of screen snips flow around, and even use that technique at times when asking our help desk for some mystifying Windows thing. Skip From grant.b.edwards at gmail.com Sun Jan 28 10:49:32 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 28 Jan 2018 15:49:32 +0000 (UTC) Subject: Compression of random binary data (Posting On Python-List Prohibited) References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> <21e2ad40-1111-495c-85e4-84b4c179cb15@googlegroups.com> <62f9eaac-317c-4844-ae15-574ae4fe97cb@googlegroups.com> Message-ID: On 2018-01-28, pendrysammuel at gmail.com wrote: > I have it in my head, just need someone to write the program for me, > I know nothing about data compression or binary data other than 1s > and 0s and that you can not take 2 number without a possible value > more or less than them selves and compress them, I have been working > for 1 1/2 years on a solution, just need help with programming. > > If someone could get ahold of me when I am sober I could probably > explain it a lot better, but I said fuck it I?ll show everyone a > possibility instead of trying to do it on my own. Wow... Just... wow. -- Grant From mail at timgolden.me.uk Sun Jan 28 10:54:31 2018 From: mail at timgolden.me.uk (Tim Golden) Date: Sun, 28 Jan 2018 15:54:31 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> On 28/01/2018 15:04, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: > > - for the sender, instead of a simple copy and paste, they have to take a > screen shot, possibly trim the image to remove any bits of the screen > they don't want to show, attach it to their email or upload it to an > image hosting site; At least for Windows users, grabbing a partial screenshot (eg of text) has been very easy since Windows 7 when the "Snipping Tool" was added to the builtins. Certainly easier for the average user than trying to do a slightly tricky rectangle selection within the Windows console. Likewise, including it in an email isn't hard; there's a command to do it right there from within that tool. And some at least of the disadvantages you cite for the receiver are rarely known or well understood by the senders. They regularly send and receive emails with embedded images; why should the mailing list they use be any different? FWIW I agree with you; and I even see this at work in different forms: someone sends a screenshot of a spreadsheet to illustrate a problem rather than the sheet itself. (When there's no especial sensitivity which might otherwise be a good reason). But most people don't interact with text-only forums these days, so it's only natural that the don't consider that aspect of things. TJG From hjp-python at hjp.at Sun Jan 28 10:55:30 2018 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 28 Jan 2018 16:55:30 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <20180128155530.vkkltnko5m4ko6hu@hjp.at> On 2018-01-28 15:04:26 +0000, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? Twitter? You can't send more than 140 characters[1], but you can send an image, so just put your text in an image to get around pesky size restrictions. But no, our users have done that for much longer than twitter exists. The typical mail to support doesn't contain an error message in plain text, not even a screenshot, it contains a word (or excel) file with a screenshot of the error message (typically scaled down so that the error message isn't readable any more). It reminds me about the old joke about the mathematician making coffee: He finds an empty cup in the sink, rinses it, puts some ground coffee and water into the coffee maker, waits for the water to run through and pours the coffee into the cup. The next day he wants some coffee again. But there is no cup in the sink. Instead there is a cup in the cupboard. So he takes the cup from the cupboard and puts it into the sink. Now he has reduced the problem to a previously solved problem and proceeds as before. Similarly the user sending a wort attachment instead of a plain text message knows how to take a screenshot, knows how to paste that into word and knows how to attach a word file to an email. So they combine those steps. They may or may not know how to copy some text into the email (to be fair, Windows error messages often cannot be copied), but it simply doesn't occur to them. I used to think that programmers (or techies in general) ought to be able to write emails in a fashion that makes it easy to extract the necessary information. I have since been disabused of the notion. Programmers are just as thoughtless and unable to put themselves into the recipient's shoes as the general population. Oh, and finally there is tools: I switched to Outlook for in-company mails a year ago (because my boss wants me to top-post and I simply can't do that if I have a decent editor, but with a crappy program like Outlook I can) and it is just amazing how time consuming it is to format a mail containing code examples to be readable. Taking a screenshot and pasting it into the mail is faster (even though Outlooks inline image handling is also atrocious). > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) When the first MIME RFCs came out, a co-worker predicted that we would soon get audio-clips as signatures. Thank god he was wrong about that. hp [1] 280 now. -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From steve+comp.lang.python at pearwood.info Sun Jan 28 11:36:20 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jan 2018 16:36:20 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On Sun, 28 Jan 2018 15:54:31 +0000, Tim Golden wrote: > At least for Windows users, grabbing a partial screenshot (eg of text) > has been very easy since Windows 7 when the "Snipping Tool" was added to > the builtins. Thanks, I didn't know that. > Certainly easier for the average user than trying to do a > slightly tricky rectangle selection within the Windows console. But I'm not seeing that it could possibly be easier than selecting text and hitting copy and paste. Not even in the Windows console, which I admit is a bit clunky, let alone a modern IDE. More *familiar*, maybe, but easier? [...] > FWIW I agree with you; and I even see this at work in different forms: > someone sends a screenshot of a spreadsheet to illustrate a problem > rather than the sheet itself. (When there's no especial sensitivity > which might otherwise be a good reason). Ah, this is the 21st century equivalent of somebody printing a document, scanning it, and emailing the scan. (Bonus points if they somehow manage to get the pages out of order or upside down when scanning.) > But most people don't interact with text-only forums these days, so it's > only natural that the don't consider that aspect of things. No, sorry, I don't agree with that. I'm not talking about "most people", I'm talking about programmers, nor about "text-only" forums. Even if attachments came through, it would still be a bad idea to send screenshots to ask questions about your code. Unless your sole programming language is Scratch or another "visual programming language", you're writing text and your question is about text. And the output is almost certainly text. Especially in the case of an exception, say. (If your question is about, say, the layout of graphical elements in your GUI, then a screenshot is fine -- I'm not a troglodyte, I understand that sometimes a picture really is worth a thousand words.) And writing an email requires writing text. And when people ask "fix my code for me", they are expecting to receive text they can copy and paste back. So text is the natural media for this sort of question, and there's nothing natural about thinking "I know, I'll take a virtual photo of this text and send that". I didn't even say *plain text*. I would completely understand it -- hate it, but understand it -- if people posted HTML and marked up their text with comments and colour. Hell, I'm even willing to consider that /maybe/ programming source code should be some form of rich text. But at least rich text is text, not a bunch of pixels. I'm sorry to the OP of the other thread if he feels I'm picking on him, I'm not intending to single him out. I'm just seeing this habit more and more often in many different forums, and I had to ask where it was coming from. It's obviously *learned* behaviour: there's nothing natural about taking a screen shot to ask a question. I'm not surprised that yet again Microsoft has made the world a little bit worse by trying to make things easier for ordinary (l)users, and their bad habits are spreading into the programming community. -- Steve From gheskett at shentel.net Sun Jan 28 11:41:16 2018 From: gheskett at shentel.net (Gene Heskett) Date: Sun, 28 Jan 2018 11:41:16 -0500 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: <20180128155530.vkkltnko5m4ko6hu@hjp.at> References: <20180128155530.vkkltnko5m4ko6hu@hjp.at> Message-ID: <201801281141.16982.gheskett@shentel.net> On Sunday 28 January 2018 10:55:30 Peter J. Holzer wrote: > On 2018-01-28 15:04:26 +0000, Steven D'Aprano wrote: > > I'm seeing this annoying practice more and more often. Even for > > trivial pieces of text, a few lines, people post screenshots instead > > of copying the code. > > > > Where has this meme come from? > > Twitter? You can't send more than 140 characters[1], but you can send > an image, so just put your text in an image to get around pesky size > restrictions. > > But no, our users have done that for much longer than twitter exists. > The typical mail to support doesn't contain an error message in plain > text, not even a screenshot, it contains a word (or excel) file with a > screenshot of the error message (typically scaled down so that the > error message isn't readable any more). > > It reminds me about the old joke about the mathematician making > coffee: He finds an empty cup in the sink, rinses it, puts some ground > coffee and water into the coffee maker, waits for the water to run > through and pours the coffee into the cup. > The next day he wants some coffee again. But there is no cup in the > sink. Instead there is a cup in the cupboard. So he takes the cup from > the cupboard and puts it into the sink. Now he has reduced the problem > to a previously solved problem and proceeds as before. > > Similarly the user sending a wort attachment instead of a plain text > message knows how to take a screenshot, knows how to paste that into > word and knows how to attach a word file to an email. So they combine > those steps. They may or may not know how to copy some text into the > email (to be fair, Windows error messages often cannot be copied), but > it simply doesn't occur to them. > > I used to think that programmers (or techies in general) ought to be > able to write emails in a fashion that makes it easy to extract the > necessary information. I have since been disabused of the notion. > Programmers are just as thoughtless and unable to put themselves into > the recipient's shoes as the general population. > > Oh, and finally there is tools: I switched to Outlook for in-company > mails a year ago (because my boss wants me to top-post and I simply > can't do that if I have a decent editor, but with a crappy program > like Outlook I can) and it is just amazing how time consuming it is to > format a mail containing code examples to be readable. Taking a > screenshot and pasting it into the mail is faster (even though > Outlooks inline image handling is also atrocious). > > > (The day a programmer posts a WAV file of themselves reading their > > code out aloud, is the day I turn my modem off and leave the > > internet forever.) > > When the first MIME RFCs came out, a co-worker predicted that we would > soon get audio-clips as signatures. Thank god he was wrong about that. > > hp > > [1] 280 now. But by mentioning it, somebody will now do it. The problem will be what the hell do you play it with... -- Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page From pkpearson at nowhere.invalid Sun Jan 28 12:53:33 2018 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 28 Jan 2018 17:53:33 GMT Subject: Compression of random binary data References: <67752e1d-bbe1-4152-9651-5cb5d4341b4e@googlegroups.com> Message-ID: On Sat, 27 Jan 2018 21:26:06 -0800 (PST), pendrysammuel at gmail.com wrote: > If it is then show him this > > 387,420,489 >= > 00110011 00111000 00110111 00101100 00110100 00110010 00110000 0 ... To save the casual reader a moment of disorientation, the above binary string is just the ASCII representation of the text string "387,420,489". > 9^9 = ?? (^ = to the power of) >= 387,420,489 > > But > > 9^9 >= > 00111001 01011110 00111001 Similarly, this is the ASCII representation of "9^9". Our self-confessedly intermittently sober correspondent appears to have discovered the backside of the principle that a short text expression can generate a large number. -- To email me, substitute nowhere->runbox, invalid->com. From rosuav at gmail.com Sun Jan 28 14:19:51 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jan 2018 06:19:51 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On Mon, Jan 29, 2018 at 3:36 AM, Steven D'Aprano wrote: >> Certainly easier for the average user than trying to do a >> slightly tricky rectangle selection within the Windows console. > > But I'm not seeing that it could possibly be easier than selecting text > and hitting copy and paste. Not even in the Windows console, which I > admit is a bit clunky, let alone a modern IDE. More *familiar*, maybe, > but easier? The vanilla Windows console (conhost.exe IIRC) is far from ideal for copying and pasting from, and by and large, Windows error popups are *impossible* to copy text from. So people get into the habit of either transcribing by hand (tedious, error-prone, will inevitably abbreviate) or taking a screenshot (100% reliable, nice and easy, gives all the info). In a forum where attachments are acceptable, which one are they going to be encouraged to use? > Unless your sole programming language is Scratch or another "visual > programming language", you're writing text and your question is about > text. And the output is almost certainly text. Especially in the case of > an exception, say. I know that, and you know that, but not everyone does. > I didn't even say *plain text*. I would completely understand it -- hate > it, but understand it -- if people posted HTML and marked up their text > with comments and colour. Hell, I'm even willing to consider that /maybe/ > programming source code should be some form of rich text. But at least > rich text is text, not a bunch of pixels. Yeah, I've occasionally seen HTML emails with full syntax highlighting. It's rare though. > I'm sorry to the OP of the other thread if he feels I'm picking on him, > I'm not intending to single him out. I'm just seeing this habit more and > more often in many different forums, and I had to ask where it was coming > from. It's obviously *learned* behaviour: there's nothing natural about > taking a screen shot to ask a question. I'm not surprised that yet again > Microsoft has made the world a little bit worse by trying to make things > easier for ordinary (l)users, and their bad habits are spreading into the > programming community. Agreed, this is not about any single person. There is a huge problem, and most of it (IMO) derives from a general habit of the Windows (and maybe Mac) GUIs of giving critical information in pop-up windows that have *no way* to get a text dump from. The "culture of the screen shot" has been around for way too long, and it's not going anywhere any time soon, so all we can do is continue to tell people a better way, and... well... hope for the text. Okay, that pun was bad even in my own head. Sorry. Hope for the best. ChrisA From grant.b.edwards at gmail.com Sun Jan 28 15:06:11 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Sun, 28 Jan 2018 20:06:11 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> Message-ID: On 2018-01-28, Skip Montanaro wrote: > I've noticed it as well. I suspect it's from the Windows universe where > it's common to snip a bit of the screen which isn't pure text when asking > about some problematic GUI thing which is causing problems. It's definitely a Windows thing. Most Windows uses don't even understand the concept of plain text vs. a bitmap picture of some text being displayed by an app. Just be glad that they're using a screen capture utilitly and not taking a picture of thier screen with their phone. [Yes, I've dealt with problem reports from customers where that's what they send.] -- Grant From tkadm30 at yandex.com Sun Jan 28 15:09:35 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Sun, 28 Jan 2018 15:09:35 -0500 Subject: How to embed a native JIT compiler to a django app? In-Reply-To: <20180128133228.GA25126@xps> References: <20180128133228.GA25126@xps> Message-ID: Hi Stephane, Thank you for your reply. As you suggested, I believe my approach was too complex. So I decided to replace Cython with PyPy in order to enable experimental support of JIT compilation for my Python/Django web applications. Best regards, Etienne Le 2018-01-28 ? 08:32, Stephane Wirtel via Python-list a ?crit?: > On 01/27, Etienne Robillard wrote: >> Hi, >> >> I want to compile a Django application into a C source file and embed >> a JIT compiler into the binary. Is there any way of doing this with >> llvm/clang? > > Hi Etienne, > > I think no, Django will use Python and this one is interpreted. > Answer, no... > > Now, you could try to create a wheel file with your django project and > try that, but it is really complex. > > Why do you want to package a Django app into a C file with a compiler? > > > > -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From tjreedy at udel.edu Sun Jan 28 15:11:59 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 28 Jan 2018 15:11:59 -0500 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On 1/28/2018 10:54 AM, Tim Golden wrote: > On 28/01/2018 15:04, Steven D'Aprano wrote: >> I'm seeing this annoying practice more and more often. Even for trivial >> pieces of text, a few lines, people post screenshots instead of copying >> the code. This happens on Stackoverflow too. There, one can vote to close for lack of code or error message in the text. I generally do not follow code image links and would encourage other to abstain also. Hypothesis: as the number of beginning programming classes increases, the average quality, in some sense, of instructors and students go down. On the other hand, I am sometimes impressed by what people attempt after too little preparation, in an attempt to improve their lives. >> Where has this meme come from? It seems to be one which inconveniences >> *everyone* involved: > At least for Windows users, grabbing a partial screenshot (eg of text) > has been very easy since Windows 7 when the "Snipping Tool" was added to > the builtins. Certainly easier for the average user than trying to do a > slightly tricky rectangle selection within the Windows console. I somehow missed the Snipping Tool. Thanks for the information. I won't use it for this list ;-), but an image of the MS Error box might be helpful occasionally on the tracker. In Win10 Command Prompt, selection now works more of less normally, so that excuse is gone. -- Terry Jan Reedy From kwpolska at gmail.com Sun Jan 28 15:13:57 2018 From: kwpolska at gmail.com (Chris Warrick) Date: Sun, 28 Jan 2018 21:13:57 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On 28 January 2018 at 20:19, Chris Angelico wrote: > The vanilla Windows console (conhost.exe IIRC) is far from ideal for > copying and pasting from It?s been fixed in recent Windows 10 releases (select and Ctrl+C works now). > Windows error popups are *impossible* to copy text from. Most standard error popups support pressing Ctrl+C to copy the text displayed in them. -- Chris Warrick PGP: 5EAAEA16 From rosuav at gmail.com Sun Jan 28 15:26:39 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jan 2018 07:26:39 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On Mon, Jan 29, 2018 at 7:13 AM, Chris Warrick wrote: > On 28 January 2018 at 20:19, Chris Angelico wrote: >> The vanilla Windows console (conhost.exe IIRC) is far from ideal for >> copying and pasting from > > It?s been fixed in recent Windows 10 releases (select and Ctrl+C works now). Haven't used it, but that's good news at least. >> Windows error popups are *impossible* to copy text from. > > Most standard error popups support pressing Ctrl+C to copy the text > displayed in them. Really? Most? That would be a HUGE improvement. Historically, only a handful have actually had selectable text. And really, it has to be not just the core Windows error popups, but application ones as well; so it has to be the underlying message-box API that supports it. ChrisA From codewizard at gmail.com Sun Jan 28 15:41:17 2018 From: codewizard at gmail.com (codewizard at gmail.com) Date: Sun, 28 Jan 2018 12:41:17 -0800 (PST) Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: <6cff43f5-9f3c-4ce5-b1c0-95fa68ed487f@googlegroups.com> On Sunday, January 28, 2018 at 3:27:06 PM UTC-5, Chris Angelico wrote: > On Mon, Jan 29, 2018 at 7:13 AM, Chris Warrick wrote: > > On 28 January 2018 at 20:19, Chris Angelico wrote: > >> The vanilla Windows console (conhost.exe IIRC) is far from ideal for > >> copying and pasting from > > > > It?s been fixed in recent Windows 10 releases (select and Ctrl+C works now). > > Haven't used it, but that's good news at least. > > >> Windows error popups are *impossible* to copy text from. > > > > Most standard error popups support pressing Ctrl+C to copy the text > > displayed in them. > > Really? Most? That would be a HUGE improvement. Historically, only a > handful have actually had selectable text. And really, it has to be > not just the core Windows error popups, but application ones as well; > so it has to be the underlying message-box API that supports it. > > ChrisA Most popups in applications using the standard Windows dialogs can still be copied from, even if the text doesn't look selectable: - give the dialog a focus - press Ctrl-A (this invisibly selects all text) - press Ctrl-C to copy the text - paste (Ctrl-V) into your favorite text editor. Regards, Igor. From eryksun at gmail.com Sun Jan 28 15:43:35 2018 From: eryksun at gmail.com (eryk sun) Date: Sun, 28 Jan 2018 20:43:35 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> Message-ID: On Sun, Jan 28, 2018 at 4:36 PM, Steven D'Aprano wrote: > On Sun, 28 Jan 2018 15:54:31 +0000, Tim Golden wrote: > >> At least for Windows users, grabbing a partial screenshot (eg of text) >> has been very easy since Windows 7 when the "Snipping Tool" was added to >> the builtins. > > Thanks, I didn't know that. It's a common feature in desktop environments. In Ubuntu Linux I can hit Shift+PrintScreen to select and save part of the screen as a PNG file, or copy it to the clipboard. But it's no excuse for sending screen shots of source code. Fortunately on Stack Overflow I usually see this for command prompt screenshots of errors, not source code. >> Certainly easier for the average user than trying to do a >> slightly tricky rectangle selection within the Windows console. > > But I'm not seeing that it could possibly be easier than selecting text > and hitting copy and paste. Not even in the Windows console, which I > admit is a bit clunky, let alone a modern IDE. More *familiar*, maybe, > but easier? The new console in Windows 10 defaults to line-wrapping selection instead of rectangle selection. Holding the ALT key temporarily toggles the selection mode. From rosuav at gmail.com Sun Jan 28 15:58:54 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jan 2018 07:58:54 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: <6cff43f5-9f3c-4ce5-b1c0-95fa68ed487f@googlegroups.com> References: <53b614fb-43a3-bf51-9a1d-c7992596b0da@timgolden.me.uk> <6cff43f5-9f3c-4ce5-b1c0-95fa68ed487f@googlegroups.com> Message-ID: On Mon, Jan 29, 2018 at 7:41 AM, wrote: > On Sunday, January 28, 2018 at 3:27:06 PM UTC-5, Chris Angelico wrote: >> On Mon, Jan 29, 2018 at 7:13 AM, Chris Warrick wrote: >> > On 28 January 2018 at 20:19, Chris Angelico wrote: >> >> The vanilla Windows console (conhost.exe IIRC) is far from ideal for >> >> copying and pasting from >> > >> > It?s been fixed in recent Windows 10 releases (select and Ctrl+C works now). >> >> Haven't used it, but that's good news at least. >> >> >> Windows error popups are *impossible* to copy text from. >> > >> > Most standard error popups support pressing Ctrl+C to copy the text >> > displayed in them. >> >> Really? Most? That would be a HUGE improvement. Historically, only a >> handful have actually had selectable text. And really, it has to be >> not just the core Windows error popups, but application ones as well; >> so it has to be the underlying message-box API that supports it. >> >> ChrisA > > Most popups in applications using the standard Windows dialogs > can still be copied from, even if the text doesn't look selectable: > > - give the dialog a focus > - press Ctrl-A (this invisibly selects all text) > - press Ctrl-C to copy the text > - paste (Ctrl-V) into your favorite text editor. > Ah. Good that it's possible... but... a novice programmer is expected to know this... how? ChrisA From timothy.c.delaney at gmail.com Sun Jan 28 16:55:54 2018 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 29 Jan 2018 08:55:54 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On 29 January 2018 at 02:04, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. I don't tend to see this from programmers I work with, but I'm constantly having to deal with support tickets where the person raising the ticket put a screenshot of something like a console or grid output of an SQL tool or even a logfile opened in a text editor ... Even worse, usually they'll paste the screenshot into a Word document first (which then causes difficulties to view the screenshot due to page width, etc). I had one case the other day where they'd taken a screenshot of some of the columns of the output of an SQL query and pasted it into a Word document. I specifically asked them not to do this, explained that the tool they were using could export to CSV and that would be much more useful as I could search it, etc. I offered to walk them through how to do the CSV export. And I requested that they send me the entire output (all columns) of the SQL output. I got back a Word document containing about 10 screenshots where they'd apparently taken a screenshot, moved the horizontal scrollbar one screen, taken another screenshot, etc. These are support people who are employed by the company I'm contracted to. Doesn't matter how often I try to train them otherwise, this type of thing keeps happening. BTW: I have nothing to do with the final persistence format of the data, but in practice I've had to learn the DB schema and stored procedures for everything I support. Strangely the DB team don't have to learn my parts ... Tim Delaney From mittra at juno.com Sun Jan 28 17:48:02 2018 From: mittra at juno.com (qrious) Date: Sun, 28 Jan 2018 14:48:02 -0800 (PST) Subject: Data-structure for multiway associativity in Python In-Reply-To: References: Message-ID: <702ef270-5212-4af8-a98a-e613e9f367a6@googlegroups.com> On Sunday, January 28, 2018 at 7:00:38 AM UTC-8, Steven D'Aprano wrote: > > Since you specified that there are no lists with shared members, why > bother returning a list of lists? There will only ever be a single > matching list. > That's correct. It will be a single list. My mistake in typing. > > To speed it up more, we'd need to know more information about how you are > using this. For example, if the values c1, ... d1, ... etc have some sort > of relationship, No relationship. > you might be able to generate some kind of multiway tree > that avoids having to search all of the thousands of lists before giving > up. Which brings us to the Subject line of this thread. Without any relationship among the members, could we solve this using clever data structure? > > Are searches going to typically hit the same set c1... over and over > again? If so, then after matching, bring it to the front of the master > list. (You might want to use a deque for that.) > > > > > Here a hash may be a way to go, but need help in figuring it out. Also, > > can there be a faster and more memory efficient solution other than > > hashes? > > Probably not, not unless you have some sort of internal structure you can > take advantage of. For example, if all the strings in any group start > with the same letter, then you can dispatch directly to the right list: > > data = {'c': {c1, c2, c3, ..., cn}, > 'd': {d1, ... } # and so on... > } > > def getAssocList(element): > if element in data[element[0]]: > return L > raise ValueError('not found') > > > But if there is no structure at all, and the elements in each list are > completely arbitrary, then there is nothing better than a linear search > through the entire collection of lists, looking at every single element > until you've matched what you're after. The motivation of posting this thread is to explore if the above is true. > > But your description is pretty vague and for all I know what you actually > want is already a solved problem. Can you give more detail and cut-down > example of your data set? Say, two or three values per list, and two or > three lists. > > First list = { 1, 2, 3} Second list = { 4, 5, 6} Third list = { 7, 8, 9} If I pass 9 as the argument, the return value of the function would be {7, 8}. > -- > Steve Thanks very much for your help and thoughts. From nelsonjonkane6 at live.com Sun Jan 28 18:05:04 2018 From: nelsonjonkane6 at live.com (nelson jon kane) Date: Sun, 28 Jan 2018 23:05:04 +0000 Subject: Welcome to the "Python-list" mailing list (Digest mode) In-Reply-To: References: Message-ID: When I enter the word python in the search box on my Chrome Windows 10, this is what comes up. Can you tell me what each of these "types" of Python mean? Thank you. [cid:aa3fd74d-d71d-42c0-b063-4f20c463987b] ________________________________ From: Python-list on behalf of python-list-request at python.org Sent: Sunday, January 28, 2018 1:28:19 PM To: nelsonjonkane6 at live.com Subject: Welcome to the "Python-list" mailing list (Digest mode) Welcome to the Python-list at python.org mailing list! The purpose of this mailing list is to support general discussions about the Python programming language. Please remember that this list is mirrored to the Usenet newsgroup comp.lang.python. For more information on the Python programming language see To post to this list, send your message to: python-list at python.org General information about the mailing list is at: https://mail.python.org/mailman/listinfo/python-list If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: https://mail.python.org/mailman/options/python-list/nelsonjonkane6%40live.com You can also make such adjustments via email by sending a message to: Python-list-request at python.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe without confirmation. It is: Parkhill40 Normally, Mailman will remind you of your python.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you. From greg.ewing at canterbury.ac.nz Sun Jan 28 18:17:39 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Mon, 29 Jan 2018 12:17:39 +1300 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: Steven D'Aprano wrote: > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) Shh! Don't give them ideas! -- Greg From nelsonjonkane6 at live.com Sun Jan 28 18:21:04 2018 From: nelsonjonkane6 at live.com (nelson jon kane) Date: Sun, 28 Jan 2018 23:21:04 +0000 Subject: Welcome to the "Python-list" mailing list (Digest mode) In-Reply-To: References: , Message-ID: Where did this guy in the video below get the version of Python that he's using? I don't have it in my computer. His version has this across the top: *for.py - C:/Users/Madhur/Desktop/for.py* https://www.youtube.com/watch?v=I5_UxvF_svw [https://www.bing.com/th?id=OVP.24cLPPXEaXtBRV4AsjxkNwEsCo&pid=Api] Python Programming Tutorial - 18: The FOR Loop www.youtube.com In this tutorial we'll check out the FOR Loop in python and see how it is different from the WHILE Loop. ________________________________ From: nelson jon kane Sent: Sunday, January 28, 2018 6:05:04 PM To: python-list at python.org Subject: Re: Welcome to the "Python-list" mailing list (Digest mode) When I enter the word python in the search box on my Chrome Windows 10, this is what comes up. Can you tell me what each of these "types" of Python mean? Thank you. [cid:aa3fd74d-d71d-42c0-b063-4f20c463987b] ________________________________ From: Python-list on behalf of python-list-request at python.org Sent: Sunday, January 28, 2018 1:28:19 PM To: nelsonjonkane6 at live.com Subject: Welcome to the "Python-list" mailing list (Digest mode) Welcome to the Python-list at python.org mailing list! The purpose of this mailing list is to support general discussions about the Python programming language. Please remember that this list is mirrored to the Usenet newsgroup comp.lang.python. For more information on the Python programming language see To post to this list, send your message to: python-list at python.org General information about the mailing list is at: https://mail.python.org/mailman/listinfo/python-list If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: https://mail.python.org/mailman/options/python-list/nelsonjonkane6%40live.com You can also make such adjustments via email by sending a message to: Python-list-request at python.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe without confirmation. It is: Parkhill40 Normally, Mailman will remind you of your python.org mailing list passwords once every month, although you can disable this if you prefer. This reminder will also include instructions on how to unsubscribe or change your account options. There is also a button on your options page that will email your current password to you. From steve+comp.lang.python at pearwood.info Sun Jan 28 19:22:14 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 00:22:14 +0000 (UTC) Subject: Data-structure for multiway associativity in Python References: <702ef270-5212-4af8-a98a-e613e9f367a6@googlegroups.com> Message-ID: On Sun, 28 Jan 2018 14:48:02 -0800, qrious wrote: > First list = { 1, 2, 3} > Second list = { 4, 5, 6} > Third list = { 7, 8, 9} > > If I pass 9 as the argument, the return value of the function would be > {7, 8}. subsets = [{1, 2, 3}, {4, 5, 6}, {7, 8, 9}] data = {} for subset in subsets: for key in subset: data[key] = subset # At this point, your master data looks like this: # # {1: {1, 2, 3}, 2: {1, 2, 3}, 3: {1, 2, 3}, # 4: {4, 5, 6}, 5: {4, 5, 6}, 6: {4, 5, 6}, # 7: {7, 8, 9}, 8: {7, 8, 9}, 9: {7, 8, 9}} # # but don't worry, that's not three COPIES of each set, just three # references to the same one: each reference costs only a single # pointer. # Find a set associated with a key subset = data[9] # this is a lightning-fast lookup # Get the items in the set, excluding the key itself result = subset - {9} assert result == {7, 8} # Add a new data point to that set: subset.add(10) data[10] = subset # Delete a data point: del subset[9] del data[9] If you had started with a concrete example from the start, this would not have seemed like such a mysterious and difficult problem. Describing it as "multiway associativity" doesn't really help: as far as I can see, it's not multiway at all, it is just a straightforward case of one way associativity between each key and a set of values that contains that key. Concrete examples are *really* useful for clarifying the requirements. -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 19:27:07 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 00:27:07 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 08:55:54 +1100, Tim Delaney wrote: > I got back a Word document containing about 10 screenshots where they'd > apparently taken a screenshot, moved the horizontal scrollbar one > screen, taken another screenshot, etc. You're lucky they didn't just take a single screen shot, thinking that you can scroll past the edges to see what is off-screen. -- Steve From jqian at tibco.com Sun Jan 28 19:51:06 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 28 Jan 2018 19:51:06 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: print(repr(message)) out : *does not exist\r\n\tat com.* for ch in message: printf("%d %c",ch, chr(ch)) %d %c 110 n %d %c 111 o %d %c 116 t %d %c 32 %d %c 101 e %d %c 120 x %d %c 105 i %d %c 115 s %d %c 116 t *%d %c 13%d %c 10* *%d %c 9* %d %c 97 a %d %c 116 t %d %c 32 %d %c 99 c %d %c 111 o %d %c 109 m On Sun, Jan 28, 2018 at 9:50 AM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Sat, 27 Jan 2018 21:23:02 -0500, Jason Qian via Python-list wrote: > > > there are 0D 0A 09 > > If your string actually contains CARRIAGE RETURN (OD) NEWLINE (OA), and > TAB (09) characters, then you don't need to do anything. Just call print, > and they will be printed correctly. > > If that doesn't work, then your input string doesn't contain what you > think it contains. Please call this: > > print(repr(the_string)) > > and COPY AND PASTE the output here so we can see it. > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From drsalists at gmail.com Sun Jan 28 20:04:56 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 17:04:56 -0800 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: How about: >>> os.write(1, message) On Sun, Jan 28, 2018 at 4:51 PM, Jason Qian via Python-list wrote: > print(repr(message)) out : > > *does not exist\r\n\tat com.* > > > for ch in message: > printf("%d %c",ch, chr(ch)) > > > %d %c 110 n > %d %c 111 o > %d %c 116 t > %d %c 32 > %d %c 101 e > %d %c 120 x > %d %c 105 i > %d %c 115 s > %d %c 116 t > > *%d %c 13%d %c 10* > *%d %c 9* > %d %c 97 a > %d %c 116 t > %d %c 32 > %d %c 99 c > %d %c 111 o > %d %c 109 m From timothy.c.delaney at gmail.com Sun Jan 28 20:07:20 2018 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Mon, 29 Jan 2018 12:07:20 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On 29 January 2018 at 11:27, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Mon, 29 Jan 2018 08:55:54 +1100, Tim Delaney wrote: > > > I got back a Word document containing about 10 screenshots where they'd > > apparently taken a screenshot, moved the horizontal scrollbar one > > screen, taken another screenshot, etc. > > You're lucky they didn't just take a single screen shot, thinking that > you can scroll past the edges to see what is off-screen. I suspect that was the case on the original screenshot in Word document ... Tim Delaney From drsalists at gmail.com Sun Jan 28 20:13:05 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 17:13:05 -0800 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: I'm afraid the perspective may be: Text == primitive GUI == modern It feel like it'd be possible to train a neural network to translate text in a screenshot to plain text though. On Sun, Jan 28, 2018 at 7:04 AM, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. From dan at tombstonezero.net Sun Jan 28 20:18:53 2018 From: dan at tombstonezero.net (Dan Sommers) Date: Mon, 29 Jan 2018 01:18:53 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 00:27:07 +0000, Steven D'Aprano wrote: > On Mon, 29 Jan 2018 08:55:54 +1100, Tim Delaney wrote: > >> I got back a Word document containing about 10 screenshots where they'd >> apparently taken a screenshot, moved the horizontal scrollbar one >> screen, taken another screenshot, etc. > > You're lucky they didn't just take a single screen shot, thinking that > you can scroll past the edges to see what is off-screen. No! Not another pass by reference vs. pass by value thread! From jqian at tibco.com Sun Jan 28 20:31:39 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 28 Jan 2018 20:31:39 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: Thanks a lot :) os.write(1, message) works ! On Sun, Jan 28, 2018 at 8:04 PM, Dan Stromberg wrote: > How about: > >>> os.write(1, message) > > On Sun, Jan 28, 2018 at 4:51 PM, Jason Qian via Python-list > wrote: > > print(repr(message)) out : > > > > *does not exist\r\n\tat com.* > > > > > > for ch in message: > > printf("%d %c",ch, chr(ch)) > > > > > > %d %c 110 n > > %d %c 111 o > > %d %c 116 t > > %d %c 32 > > %d %c 101 e > > %d %c 120 x > > %d %c 105 i > > %d %c 115 s > > %d %c 116 t > > > > *%d %c 13%d %c 10* > > *%d %c 9* > > %d %c 97 a > > %d %c 116 t > > %d %c 32 > > %d %c 99 c > > %d %c 111 o > > %d %c 109 m > From steve+comp.lang.python at pearwood.info Sun Jan 28 20:34:28 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 01:34:28 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: Jason, your Python output and the C output are not the same. Also, it looks like your email client is adding formatting codes to the email, or something. Please look at your post here: https://mail.python.org/pipermail/python-list/2018-January/730384.html Do you notice the extra asterisks added? I think they are added by your email program. Please do not use any formatting, they make the output confusing and make it difficult to understand what you are doing. In Python, the output of print(repr(message)) looks like this: does not exist\r\n\tat com. (removing the asterisks at the start and end, which I believe are formatting added by to your email). That is, spelling it out character by character: DE OH EE ES space EN OH TE space EE EX I ES TE backslash AR backslash EN backslash TE AY TE space CE OH EM dot That is what I expected. So if you print(message) without the repr, you should get exactly this: does not exist at com. Here is the input and output of my Python session: py> s = 'does not exist\r\n\tat com.' py> print(s) does not exist at com. That looks like exactly what you are expecting. If you are getting something different, please COPY AND PASTE as PLAIN TEXT (no formatting, no bold, no italics) the exact output from your Python session. And tell us what IDE you are using, or which interpreter. Are you using IDLE or PyCharm or Spyder or something else? I repeat: do not add bold, or italics, do not use formatting of any kind. If you don't know how to turn the formatting off in your email program, you will need to find out. You also print the individual characters of message from C: for ch in message: printf("%d %c",ch, chr(ch)) and that outputs something completely different: the messages are not the same in your C code and your Python code. Your Python code message is: does not exist\r\n\tat com. but your C code message appears to be: not exist\r\n\tat com (missing word "does" and final dot). I do not understand why you are showing us C code. This is a Python forum, and you are having trouble printing from Python -- C is irrelevant. I think you should just call print(message). If that doesn't do what you want, you need to show us what it actually does. -- Steve From jqian at tibco.com Sun Jan 28 20:35:06 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 28 Jan 2018 20:35:06 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: Message-ID: Thanks Peter, replace print with os.write fixed the problem. On Sun, Jan 28, 2018 at 3:57 AM, Peter Otten <__peter__ at web.de> wrote: > Jason Qian via Python-list wrote: > > > HI > > > > I have a string that contains \r\n\t > > > > [Ljava.lang.Object; does not exist*\r\n\t*at > > [com.livecluster.core.tasklet > > > > > > I would like to print it as : > > > > [Ljava.lang.Object; does not exist > > tat com.livecluster.core.tasklet > > > > How can I do this in python print ? > > Assuming the string contains the escape sequences rather than an actual > TAB, CR or NL you can apply codecs.decode(): > > >>> s = r"[Ljava.lang.Object; does not exist\r\n\tat > com.livecluster.core.tasklet" > >>> print(s) > [Ljava.lang.Object; does not exist\r\n\tat com.livecluster.core.tasklet > >>> import codecs > >>> print(codecs.decode(s, "unicode-escape")) > [Ljava.lang.Object; does not exist > at com.livecluster.core.tasklet > > Note that this will decode all escape sequences that may occur in a string > literal > in Python, e. g. > > >>> codecs.decode(r"\x09 \u03c0 \N{soft ice cream}", "unicode-escape") > '\t ? ?' > > and will complain when the string is not a valid Python string literal: > > >>> codecs.decode(r"\x", "unicode-escape") > Traceback (most recent call last): > File "", line 1, in > UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position > 0-1: truncated \xXX escape > > If you need more control you can build your own conversion routine: > > import re > > lookup = { > "r": "\r", > "n": "\n", > "t": "\t", > } > > def substitute(match): > group = match.group(1) > return lookup.get(group, group) > > def decode(s): > return re.sub(r"\\(.)", substitute, s) > > s = decode("alpha\\n \\xomega\\\\") > print(s) > print(repr(s)) > > > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+comp.lang.python at pearwood.info Sun Jan 28 20:35:30 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 01:35:30 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, 28 Jan 2018 17:04:56 -0800, Dan Stromberg wrote: > How about: >>>> os.write(1, message) What do you think that will do that print() doesn't do? -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 20:41:25 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 01:41:25 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, 28 Jan 2018 20:31:39 -0500, Jason Qian via Python-list wrote: > Thanks a lot :) > > os.write(1, message) works ! I still do not believe that print(message) doesn't work. I can see no reason why you need to use os.write(). -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 20:46:46 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 01:46:46 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Sun, 28 Jan 2018 17:13:05 -0800, Dan Stromberg wrote: > I'm afraid the perspective may be: > Text == primitive > GUI == modern > > It feel like it'd be possible to train a neural network to translate > text in a screenshot to plain text though. That would be OCR, which has been around long before neural networks. Neither OCR nor neural networks can magically enhance low-res pixellated images and *accurately* turn them into text. https://en.wikipedia.org/wiki/Optical_character_recognition -- Steve From drsalists at gmail.com Sun Jan 28 20:49:44 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 17:49:44 -0800 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, Jan 28, 2018 at 5:35 PM, Steven D'Aprano wrote: > On Sun, 28 Jan 2018 17:04:56 -0800, Dan Stromberg wrote: >> How about: >>>>> os.write(1, message) > > What do you think that will do that print() doesn't do? >>> message = b'*does not exist\r\n\tat com.*' >>> os.write(1, message) *does not exist at com.*26 >>> print(message) b'*does not exist\r\n\tat com.*' >>> From steve+comp.lang.python at pearwood.info Sun Jan 28 21:03:19 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 02:03:19 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, 28 Jan 2018 19:51:06 -0500, Jason Qian via Python-list wrote: > print(repr(message)) out : > > *does not exist\r\n\tat com.* For the record, I'd just like to say that this is not the output of Jason's call to print(). It has been edited to remove the b' ' delimiters which would have solved the problem: he isn't printing a string at all, he is printing bytes. Jason, you have wasted our time by editing your output to hide what it actually shows. Don't do that again. When we ask to see the output of something, we need to see the ACTUAL output, not an edited version. -- Steve From steve+comp.lang.python at pearwood.info Sun Jan 28 21:04:18 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 02:04:18 +0000 (UTC) Subject: Please help on print string that contains 'tab' and 'newline' References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, 28 Jan 2018 17:49:44 -0800, Dan Stromberg wrote: > On Sun, Jan 28, 2018 at 5:35 PM, Steven D'Aprano > wrote: >> On Sun, 28 Jan 2018 17:04:56 -0800, Dan Stromberg wrote: >>> How about: >>>>>> os.write(1, message) >> >> What do you think that will do that print() doesn't do? > >>>> message = b'*does not exist\r\n\tat com.*' >>>> os.write(1, message) > *does not exist > at com.*26 >>>> print(message) > b'*does not exist\r\n\tat com.*' What was the clue that made you think that Jason was using bytes? In his posts, he always said it was a string, and when he printed the repr() of the message, it was a string, not bytes. If it were bytes, we would have seen this: py> message = b'*does not exist\r\n\tat com.*' py> print(repr(message)) b'*does not exist\r\n\tat com.*' That is NOT the output that Jason showed. His output was: py> string_message = '*does not exist\r\n\tat com.*' py> print(repr(string_message)) '*does not exist\r\n\tat com.*' Ah wait no it wasn't even that. The output he showed had no quotation marks. I didn't pick up on that: what he sent us as supposed output was actually impossible. He has been editing the output. So what was the clue that it was bytes that you saw that (nearly) everyone else missed? Especially me. -- Steve From drsalists at gmail.com Sun Jan 28 21:14:35 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 18:14:35 -0800 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: On Sun, Jan 28, 2018 at 6:04 PM, Steven D'Aprano wrote: > On Sun, 28 Jan 2018 17:49:44 -0800, Dan Stromberg wrote: > So what was the clue that it was bytes that you saw that (nearly) > everyone else missed? Especially me. Can I get away with saying "it was just a good guess"? I've been using byte strings a bunch in one of my projects. That didn't hurt. From larry.martell at gmail.com Sun Jan 28 21:41:00 2018 From: larry.martell at gmail.com (Larry Martell) Date: Sun, 28 Jan 2018 21:41:00 -0500 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 10:04 AM, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: > > - for the sender, instead of a simple copy and paste, they have to take a > screen shot, possibly trim the image to remove any bits of the screen > they don't want to show, attach it to their email or upload it to an > image hosting site; > > - for the receiver, you are reliant on a forum which doesn't strip > attachments, or displays externally hosted images; the visually impaired > are excluded from using a screen reader; and nobody can copy or edit the > given text. > > It is as if people are deliberately inconveniencing themselves in order > to inconvenience the people they are asking to help them. > > With the exception of one *exceedingly* overrated advantage, namely the > ability to annotate the image with coloured lines and circles and > squiggles or other graphics (which most people don't bother to do), this > seems to me to be 100% counter-productive for everyone involved. Why has > it spread and why do people keep doing it? > > I don't want to be the old man yelling "Get Of My Lawn!" to the cool > kids, but is this just another sign of the downward spiral of programming > talent? Convince me that there is *some* justification for this practice. > Even a tiny one. > > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) I work remotely and have for over 20 years. At first I communicated with my colleagues via phone and email. Then it was skype for a while but then it went back to email. Then IRC had a moment, then it was slack for a while, then back to email. Now everyone seems to be switching to google hangouts, both chat and video. Recently I have seen that some people are sending screen shots of their code and/or error messages instead of copy/pasting. It's mostly with the younger ones, and I do not care for it at all, but I did not want to be the old fogy, so I did not say anything. From jqian at tibco.com Sun Jan 28 21:57:11 2018 From: jqian at tibco.com (Jason Qian) Date: Sun, 28 Jan 2018 21:57:11 -0500 Subject: Please help on print string that contains 'tab' and 'newline' In-Reply-To: References: <3b25ab51-051f-6eec-e5ee-774d3cdca820@nedbatchelder.com> Message-ID: The message type is bytes, this may make different ? print(type(message)) On Sun, Jan 28, 2018 at 8:41 PM, Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > On Sun, 28 Jan 2018 20:31:39 -0500, Jason Qian via Python-list wrote: > > > Thanks a lot :) > > > > os.write(1, message) works ! > > I still do not believe that print(message) doesn't work. I can see no > reason why you need to use os.write(). > > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From drsalists at gmail.com Sun Jan 28 23:24:55 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 20:24:55 -0800 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 5:46 PM, Steven D'Aprano wrote: > On Sun, 28 Jan 2018 17:13:05 -0800, Dan Stromberg wrote: >> It feel like it'd be possible to train a neural network to translate >> text in a screenshot to plain text though. > > That would be OCR, which has been around long before neural networks. > Neither OCR nor neural networks can magically enhance low-res pixellated > images and *accurately* turn them into text. Yes, I'm familiar with OCR, but last I heard, it was still requiring tuning with a specific font. Is it really true that OCR appeared long before Neural Networks (NN's)? I first heard of NN's in the 80's, but OCR more like the 90's. NN's have been around a long time, but it's only recently that they've become highly useful because of the increase in computer power, an explosion of digital data availability, and algorithm improvements. If an NN can translate from English to German well, and can tell a cat from a dog, or play go on a level that can beat the best human in the world, an NN might be able to do the job OCR was intended for given adequate training data. And we could probably write a little gobject introspection app to generate copious training data. From drsalists at gmail.com Sun Jan 28 23:29:31 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Sun, 28 Jan 2018 20:29:31 -0800 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Sun, Jan 28, 2018 at 8:24 PM, Dan Stromberg wrote: > If an NN can ... play go on a level that can beat the best human in the > world Correcting myself: I think Google's AlphaGo used more than one NN, plus perhaps a little traditional reading algorithm. So I probably should have said "If NN's can ...". From dmarv at dop.com Mon Jan 29 00:46:15 2018 From: dmarv at dop.com (Dale Marvin) Date: Sun, 28 Jan 2018 21:46:15 -0800 Subject: IoT automation In-Reply-To: References: Message-ID: <6e6c5a2e-d7a7-48c1-0996-d4e9f2727d71@dop.com> On 1/28/18 7:39 AM, Prahallad Achar wrote: > Hello team, > Could you please help me out in automation of IoT product end to end > > Regards > Prahallad > ? --Dale From acharbly at gmail.com Mon Jan 29 02:02:24 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Mon, 29 Jan 2018 12:32:24 +0530 Subject: IoT automation In-Reply-To: <6e6c5a2e-d7a7-48c1-0996-d4e9f2727d71@dop.com> References: <6e6c5a2e-d7a7-48c1-0996-d4e9f2727d71@dop.com> Message-ID: Thank you Mr. Marvin On 29 Jan 2018 12:02 pm, "Dale Marvin via Python-list" < python-list at python.org> wrote: > On 1/28/18 7:39 AM, Prahallad Achar wrote: > >> Hello team, >> Could you please help me out in automation of IoT product end to end >> >> Regards >> Prahallad >> >> > ? > > --Dale > > -- > https://mail.python.org/mailman/listinfo/python-list > From aounabbas272 at gmail.com Mon Jan 29 02:07:19 2018 From: aounabbas272 at gmail.com (Abbas Hans) Date: Sun, 28 Jan 2018 23:07:19 -0800 Subject: python not working Message-ID: it shows the pythonw.exe system error: The program can't start because api-ms-win-crt-runtime-|1-1-0.dll is missing from your computer. try reinstalling the program to fix this problem. I reinstall it many times try to repair it is not working From mark.luther0987 at protonmail.com Mon Jan 29 05:52:19 2018 From: mark.luther0987 at protonmail.com (Mark.luther0987) Date: Mon, 29 Jan 2018 05:52:19 -0500 Subject: Issues while using CallerLookup Package Message-ID: I accessed the CallerLookup package from below link https://pypi.python.org/pypi/CallerLookup and faced the following issue- >>> RESTART: C:\Users\hp\Desktop\CallerLookup-1.2.94\CallerLookup-1.2.94\CallerLookup\Main.py Traceback (most recent call last): File "C:\Users\hp\Desktop\CallerLookup-1.2.94\CallerLookup-1.2.94\CallerLookup\Main.py", line 6, in from CallerLookup.Responses import * File "C:\Python27\lib\site-packages\CallerLookup\__init__.py", line 6, in from CallerLookup.Main import lookup_number File "C:\Python27\lib\site-packages\CallerLookup\Main.py", line 6, in from CallerLookup.Responses import * File "C:\Python27\lib\site-packages\CallerLookup\Responses.py", line 7, in from CallerLookup.Utils.Logs import format_exception ImportError: No module named Utils.Logs I tried on python 2.7.14. Please help to resolve this issue. From acharbly at gmail.com Mon Jan 29 05:53:23 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Mon, 29 Jan 2018 16:23:23 +0530 Subject: Automation query... Plz help In-Reply-To: References: Message-ID: Hello friends, There is an desktop application which runs on Windows and written in java There is a requirement to automate that application. Am trying with pyautogui but it is very slow and lengthy code to compete. Is there a way to run this automation without launching the application (headless) Plz help me From rosuav at gmail.com Mon Jan 29 06:37:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jan 2018 22:37:36 +1100 Subject: Issues while using CallerLookup Package In-Reply-To: References: Message-ID: On Mon, Jan 29, 2018 at 9:52 PM, Mark.luther0987 via Python-list wrote: > I accessed the CallerLookup package from below link > https://pypi.python.org/pypi/CallerLookup > > and faced the following issue- >>>> > RESTART: C:\Users\hp\Desktop\CallerLookup-1.2.94\CallerLookup-1.2.94\CallerLookup\Main.py > > Traceback (most recent call last): > File "C:\Users\hp\Desktop\CallerLookup-1.2.94\CallerLookup-1.2.94\CallerLookup\Main.py", line 6, in > from CallerLookup.Responses import * > File "C:\Python27\lib\site-packages\CallerLookup\__init__.py", line 6, in > from CallerLookup.Main import lookup_number > File "C:\Python27\lib\site-packages\CallerLookup\Main.py", line 6, in > from CallerLookup.Responses import * > File "C:\Python27\lib\site-packages\CallerLookup\Responses.py", line 7, in > from CallerLookup.Utils.Logs import format_exception > ImportError: No module named Utils.Logs > > I tried on python 2.7.14. Please help to resolve this issue. Did you follow the installation and usage instructions in the PyPI landing page? ChrisA From steve+comp.lang.python at pearwood.info Mon Jan 29 07:04:14 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 12:04:14 +0000 (UTC) Subject: Automation query... Plz help References: Message-ID: On Mon, 29 Jan 2018 16:23:23 +0530, Prahallad Achar wrote: > Hello friends, > > There is an desktop application which runs on Windows and written in > java [...] > Is there a way to run this automation without launching the application > (headless) Is the name of the application a secret? -- Steve From acharbly at gmail.com Mon Jan 29 07:20:46 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Mon, 29 Jan 2018 17:50:46 +0530 Subject: Automation query... Plz help In-Reply-To: References: Message-ID: No.. Not at all. Its CTP application.. Which is basically transport planner for networks On 29 Jan 2018 5:38 pm, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > On Mon, 29 Jan 2018 16:23:23 +0530, Prahallad Achar wrote: > > > Hello friends, > > > > There is an desktop application which runs on Windows and written in > > java > [...] > > Is there a way to run this automation without launching the application > > (headless) > > Is the name of the application a secret? > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From steve+comp.lang.python at pearwood.info Mon Jan 29 09:14:50 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 14:14:50 +0000 (UTC) Subject: Automation query... Plz help References: Message-ID: On Mon, 29 Jan 2018 17:50:46 +0530, Prahallad Achar wrote: > No.. Not at all. > > Its CTP application.. Which is basically transport planner for networks If you want to know whether CTP can be run headless, you should ask the CTP support team or software maintainer, not Python forums. Do you have a support contract for this software? You could ask the help desk. -- Steve From rhodri at kynesim.co.uk Mon Jan 29 09:17:51 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 29 Jan 2018 14:17:51 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On 29/01/18 04:29, Dan Stromberg wrote: > On Sun, Jan 28, 2018 at 8:24 PM, Dan Stromberg wrote: >> If an NN can ... play go on a level that can beat the best human in the >> world > > Correcting myself: I think Google's AlphaGo used more than one NN, > plus perhaps a little traditional reading algorithm. So I probably > should have said "If NN's can ...". No, you should have said "If NNs can..." without the grocer's apostrophe :-) (Well, it seems to be that sort of thread.) -- Rhodri James *-* Kynesim Ltd From jugurtha.hadjar at gmail.com Mon Jan 29 09:20:06 2018 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Mon, 29 Jan 2018 15:20:06 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> Message-ID: On 01/28/2018 04:43 PM, Skip Montanaro wrote: > I've never been a Windows user, but at my current job, Windows is core to > just about everything, so I am forced to use it for a lot of stuff > (Outlook, SQL Server, Excel, etc). I was hired at a startup which made a good impression on me and I was eager to start working. I checked in for my first day, signed the paperwork, then the CTO showed me around and told me "Here's your laptop, you can install Windows and I'll check in with you later". Life started draining out of my body and my mind was racing in all directions before he even finished his sentence. I felt tricked: I was hired based on a test and the file I've received was a *.tar.gz* and it made me happy because I assumed they were a NIX shop.. I was considering how I should go about quitting gracefully. I have stopped using Windows in 2008 for psychological reasons for it made me really anxious, irritable, angry, and tense to the point of abusing hardware with low kicks and punches which was not very healthy or sane. It was only when I switched OS that this behavior stopped. There was no way I would be going back to my bully. The CTO then said "Sorry.. I meant Ubuntu." and seeing my pale face, he said in a reassuring tone "Don't be afraid, there are no Windows machines here." which brought me back to life. I hope to be as brave as you some day. -- ~ Jugurtha Hadjar, From alister.ware at ntlworld.com Mon Jan 29 09:46:01 2018 From: alister.ware at ntlworld.com (alister) Date: Mon, 29 Jan 2018 14:46:01 GMT Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Sun, 28 Jan 2018 22:11:12 +0000, Stefan Ram wrote: > Tim Delaney writes: >>These are support people who are employed by the company I'm contracted >>to. >>Doesn't matter how often I try to train them otherwise, this type of >>thing keeps happening. > > That might be more a problem of power. I would simply repeatedly reject the fault report until the user provided the information in the requested format -- Work smarter, not harder, and be careful of your speling. From alister.ware at ntlworld.com Mon Jan 29 09:48:46 2018 From: alister.ware at ntlworld.com (alister) Date: Mon, 29 Jan 2018 14:48:46 GMT Subject: Where has the practice of sending screen shots as source code come from? References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> Message-ID: On Mon, 29 Jan 2018 15:20:06 +0100, Jugurtha Hadjar wrote: > On 01/28/2018 04:43 PM, Skip Montanaro wrote: >> I've never been a Windows user, but at my current job, Windows is core >> to just about everything, so I am forced to use it for a lot of stuff >> (Outlook, SQL Server, Excel, etc). > > I was hired at a startup which made a good impression on me and I was > eager to start working. I checked in for my first day, signed the > paperwork, then the CTO showed me around and told me "Here's your > laptop, you can install Windows and I'll check in with you later". Life > started draining out of my body and my mind was racing in all directions > before he even finished his sentence. I felt tricked: I was hired based > on a test and the file I've received was a *.tar.gz* and it made me > happy because I assumed they were a NIX shop.. > > I was considering how I should go about quitting gracefully. I have > stopped using Windows in 2008 for psychological reasons for it made me > really anxious, irritable, angry, and tense to the point of abusing > hardware with low kicks and punches which was not very healthy or sane. > It was only when I switched OS that this behavior stopped. There was no > way I would be going back to my bully. > > The CTO then said "Sorry.. I meant Ubuntu." and seeing my pale face, he > said in a reassuring tone "Don't be afraid, there are no Windows > machines here." which brought me back to life. > > I hope to be as brave as you some day. Any Vacancies whatever they do I am sure I can learn :-) -- The ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is generally understood. Indeed, the world is ruled by little else. -- John Maynard Keyes From alister.ware at ntlworld.com Mon Jan 29 09:54:24 2018 From: alister.ware at ntlworld.com (alister) Date: Mon, 29 Jan 2018 14:54:24 GMT Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 12:17:39 +1300, Gregory Ewing wrote: > Steven D'Aprano wrote: >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet >> forever.) > > Shh! Don't give them ideas! just wait, once they realise you cant send pictures to a text only news group they take the screen shot, print it place it on a wooden table*, photograph it & then convert the JPeg to base64 encoding so they can paste it in! * credit to thedailywtf.com -- You look tired. From drsalists at gmail.com Mon Jan 29 10:35:26 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 29 Jan 2018 07:35:26 -0800 Subject: python not working In-Reply-To: References: Message-ID: Please google for "pythonw.exe system error: The program can't start because api-ms-win-crt-runtime-|1-1-0.dll is missing from your computer. try reinstalling the program to fix this problem.". I believe the Windows binaries for Python are built with a proprietary compiler that requires a separate download. On Sun, Jan 28, 2018 at 11:07 PM, Abbas Hans wrote: > it shows the pythonw.exe system error: > The program can't start because api-ms-win-crt-runtime-|1-1-0.dll is > missing from your computer. try reinstalling the program to fix this > problem. > > > I reinstall it many times try to repair it is not working > -- > https://mail.python.org/mailman/listinfo/python-list From linux4michelle at tamay-dogan.net Mon Jan 29 12:01:29 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Mon, 29 Jan 2018 18:01:29 +0100 Subject: New to Python and understanding problem Message-ID: Hello *, because I am runing into problems with SOME python based programs, I the this as opportunity to learn python (after ASM, C, BaSH, CP/M, COBOL, JS, PHP and perl). OK, I tried to install "blueman" (Bluetooth Manager) on my Debian 9.2 (Stretch system and discovered a problem: ----[ c 'blueman-applet' ]---------------------------------------------- Traceback (most recent call last): File "./blueman-applet", line 15, in from blueman.Functions import create_logger, create_parser, set_proc_title ImportError: No module named 'blueman' ------------------------------------------------------------------------ So ist does not find the module and this are the fist 16 lines of the python script: ----[ '/usr/bin/blueman-applet' ]--------------------------------------- #!/usr/bin/env python3 # coding=utf-8 import sys import os import signal import logging # support running uninstalled _dirname = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) if 'BLUEMAN_SOURCE' in os.environ: sys.path = [_dirname, os.path.join(_dirname, 'module', '.libs')] + sys.path os.environ["GSETTINGS_SCHEMA_DIR"] = os.path.join(_dirname, "data") from blueman.Functions import create_logger, create_parser, set_proc_title from blueman.main.Applet import BluemanApplet ------------------------------------------------------------------------ I think, that I have found the error here: sys.path = [_dirname, os.path.join(_dirname, 'module', '.libs')] + sys.path because there is written in ----[ '/usr/lib/python-3.5/os.py' ]------------------------------------- To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name). ------------------------------------------------------------------------ Hence, os.path.join() has only 2 parameters and not 3. The module "blueman" is a subdirectory and the full path is /usr/lib/python-3.5/site-packages/blueman So, how can I correct this problem? And then here is anoter thing which I do not understand becasue I have not found it in the Tutorial: What do the [ ... ] mean or what function is it? Thanks in avance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From acharbly at gmail.com Mon Jan 29 12:15:38 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Mon, 29 Jan 2018 22:45:38 +0530 Subject: Automation query... Plz help In-Reply-To: References: Message-ID: Thanks for the kind response. Sure.. Definitely I shall ask development team for the same. Regards Prahallad On 29 Jan 2018 7:48 pm, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > On Mon, 29 Jan 2018 17:50:46 +0530, Prahallad Achar wrote: > > > No.. Not at all. > > > > Its CTP application.. Which is basically transport planner for networks > > If you want to know whether CTP can be run headless, you should ask the > CTP support team or software maintainer, not Python forums. > > Do you have a support contract for this software? You could ask the help > desk. > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From pkpearson at nowhere.invalid Mon Jan 29 12:26:32 2018 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 29 Jan 2018 17:26:32 GMT Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Sun, 28 Jan 2018 20:24:55 -0800, Dan Stromberg wrote: [snip] > > Is it really true that OCR appeared long before Neural Networks > (NN's)? I first heard of NN's in the 80's, but OCR more like the > 90's. In 1964, the IBM exhibit at the World's Fair in New York demonstrated a system that read dates that visitors wrote by hand. (You were supposed to write your date of birth, and the system then printed the New York Times's headline for that date.) -- To email me, substitute nowhere->runbox, invalid->com. From gordon at panix.com Mon Jan 29 12:34:34 2018 From: gordon at panix.com (John Gordon) Date: Mon, 29 Jan 2018 17:34:34 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: In Steven D'Aprano writes: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. In some (perhaps rare) cases, a screenshot can provide useful independent verification. Such a screenshot recently helped me diagnose an issue in a stackoverflow post. A user was having trouble with some code that opens a file, and he posted a screenshot of the code and a screenshot of a Windows File Explorer window containing the desired file. The File Explorer screenshot clearly contained the desired file (named input.txt), and he was mystified as to why the code was claiming the file did not exist. The displayed filename in File Explorer was input.txt -- meaning that the real filename was actually input.txt.txt, because File Explorer shows file extensions as a separate column. Without this screenshot, we would have had only the user's (incorrect) assertion that the file existed, and no way to diagnose the true issue. Granted, this was an environment issue and not a code issue, but I can imagine situations where the same sort of thing could apply to code. -- John Gordon A is for Amy, who fell down the stairs gordon at panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" From drsalists at gmail.com Mon Jan 29 12:42:55 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Mon, 29 Jan 2018 09:42:55 -0800 Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: On Mon, Jan 29, 2018 at 9:01 AM, Michelle Konzack wrote: > OK, I tried to install "blueman" (Bluetooth Manager) on my Debian 9.2 > (Stretch system and discovered a problem: > > ----[ c 'blueman-applet' ]---------------------------------------------- > Traceback (most recent call last): > File "./blueman-applet", line 15, in > from blueman.Functions import create_logger, create_parser, > set_proc_title > ImportError: No module named 'blueman' > ------------------------------------------------------------------------ I don't see blueman on pypi, so this is probably part of the package you downloaded, and not something you need to "pip3 install". Sometimes this sort of problem surfaces when someone tries to run a package with a different python than the python the package is installed into. Do you have more than one python on your machine? From linux4michelle at tamay-dogan.net Mon Jan 29 13:14:57 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Mon, 29 Jan 2018 19:14:57 +0100 Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: Hello Dan, Am 2018-01-29 hackte Dan Stromberg in die Tasten: > I don't see blueman on pypi, so this is probably part of the package > you downloaded, and not something you need to "pip3 install". I have Python 2.7 and 3.5 from the Debian GNU/Linux repository installed I use the Stable (Stretch) version. Installing blueman 2.0.4 (for Python 2.7) ended with a SEGFAULT. Hence I downloaded the sources 2.1~alpha2 from the Experimantal Mirror, comiled and packed it as Backport. Anything went fine, except, if I execute /usr/bin/blueman-applet it does not find the path to blueman which is in /usr/lib/python-3.5/site-packages as subdirectory "bluman". OK, I can hardcode this pig for myself, but I would prefer to make it the correct way and submit a bugreport with patch. > Sometimes this sort of problem surfaces when someone tries to run a > package with a different python than the python the package is > installed into. Do you have more than one python on your machine? 2.7 and 3.5 But since the Python script /usr/bin/blueman-applet has the shebang #!/usr/bin/env python3 The module path should be ok. Thanks in avance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From python at mrabarnett.plus.com Mon Jan 29 13:15:05 2018 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Jan 2018 18:15:05 +0000 Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: <50266223-682f-a06d-ffe8-7d323665f63a@mrabarnett.plus.com> On 2018-01-29 17:01, Michelle Konzack wrote: [snip] > I think, that I have found the error here: > > sys.path = [_dirname, os.path.join(_dirname, 'module', '.libs')] + sys.path > > because there is written in > > ----[ '/usr/lib/python-3.5/os.py' ]------------------------------------- > To get a full path (which begins with top) to a file or directory in > dirpath, do os.path.join(dirpath, name). > ------------------------------------------------------------------------ > > Hence, os.path.join() has only 2 parameters and not 3. > os.path.join() will accept any number of parameters, except zero. [snip] From __peter__ at web.de Mon Jan 29 13:29:20 2018 From: __peter__ at web.de (Peter Otten) Date: Mon, 29 Jan 2018 19:29:20 +0100 Subject: New to Python and understanding problem References: Message-ID: Michelle Konzack wrote: > Hello *, > > because I am runing into problems with SOME python based programs, I the > this as opportunity to learn python (after ASM, C, BaSH, CP/M, COBOL, > JS, PHP and perl). > > > OK, I tried to install "blueman" (Bluetooth Manager) on my Debian 9.2 > (Stretch system and discovered a problem: > > ----[ c 'blueman-applet' ]---------------------------------------------- > Traceback (most recent call last): > File "./blueman-applet", line 15, in > from blueman.Functions import create_logger, create_parser, > set_proc_title > ImportError: No module named 'blueman' > ------------------------------------------------------------------------ > > So ist does not find the module and this are the fist 16 lines of the > python script: > > ----[ '/usr/bin/blueman-applet' ]--------------------------------------- > #!/usr/bin/env python3 > # coding=utf-8 > > import sys > import os > import signal > import logging > > # support running uninstalled > _dirname = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) > if 'BLUEMAN_SOURCE' in os.environ: > sys.path = [_dirname, os.path.join(_dirname, 'module', '.libs')] + > sys.path > os.environ["GSETTINGS_SCHEMA_DIR"] = os.path.join(_dirname, "data") > > from blueman.Functions import create_logger, create_parser, set_proc_title > from blueman.main.Applet import BluemanApplet > ------------------------------------------------------------------------ > > I think, that I have found the error here: > > sys.path = [_dirname, os.path.join(_dirname, 'module', '.libs')] + > sys.path > > because there is written in > > ----[ '/usr/lib/python-3.5/os.py' ]------------------------------------- > To get a full path (which begins with top) to a file or directory in > dirpath, do os.path.join(dirpath, name). > ------------------------------------------------------------------------ > > Hence, os.path.join() has only 2 parameters and not 3. os.path.join() accepts one or more parameters as you can verify in the interactive interpreter: $ python3 Python 3.4.3 (default, Nov 28 2017, 16:41:13) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.path.join() Traceback (most recent call last): File "", line 1, in TypeError: join() missing 1 required positional argument: 'a' >>> os.path.join("foo") 'foo' >>> os.path.join("foo", "bar") 'foo/bar' >>> os.path.join("foo", "bar", "baz") 'foo/bar/baz' > > The module "blueman" is a subdirectory and the full path is > > /usr/lib/python-3.5/site-packages/blueman > > So, how can I correct this problem? Random idea: Did you set the BLUEMAN_SOURCE environment variable? If so, try to unset it. If that doesn't help insert the line print(sys.path) into the blueman-applet script right after the line import sys This should print a list of paths that Python searches for modules. Does this list contain the /usr/lib/python-3.5/site-packages directory? If it doesn't, what's actually in that list? > And then here is anoter thing which I do not understand becasue I have > not found it in the Tutorial: > > What do the [ ... ] mean or what function is it? If it occurs in Python source code it is a list literal >>> a = [1, 10, 100] >>> type(a) or a list comprehension >>> b = [x*x for x in a] >>> b [1, 100, 10000] Both should be mentioned in every Python tutorial. From john_ladasky at sbcglobal.net Mon Jan 29 14:43:36 2018 From: john_ladasky at sbcglobal.net (John Ladasky) Date: Mon, 29 Jan 2018 11:43:36 -0800 (PST) Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: > > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) What's a... modem? From rosuav at gmail.com Mon Jan 29 14:57:30 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jan 2018 06:57:30 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 6:43 AM, John Ladasky wrote: > On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >> >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet forever.) > > What's a... modem? A device for turning good code into line noise. Synonym: maintenance-programmer. ChrisA From eryksun at gmail.com Mon Jan 29 15:01:11 2018 From: eryksun at gmail.com (eryk sun) Date: Mon, 29 Jan 2018 20:01:11 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Mon, Jan 29, 2018 at 5:34 PM, John Gordon wrote: > > The displayed filename in File Explorer was input.txt -- meaning that the > real filename was actually input.txt.txt, because File Explorer shows file > extensions as a separate column. One of the first things I do after creating an account is to modify Explorer to show all files and not hide extensions. Windows 10 makes this more discoverable. In the settings page that enables developer mode, you can configure the system to show file extensions, hidden and system files, empty drives, the full path in Explorer's title bar, and the start-menu option to run a program as a different user. From darcy at VybeNetworks.com Mon Jan 29 15:13:33 2018 From: darcy at VybeNetworks.com (D'Arcy Cain) Date: Mon, 29 Jan 2018 14:13:33 -0600 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <96722c8f-43ce-af1d-be24-bf5a0550e3f6@VybeNetworks.com> On 01/29/2018 01:43 PM, John Ladasky wrote: > On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >> >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet forever.) > > What's a... modem? That would be a MOdulator-DEModulator device. It's what you plug your DSL phone line or cable coax into. At one time it simply plugged into a regular phone line but there aren't many of those around any more. You may be connecting directly to a provider's Wifi AP but 90% of the people still use modems. And even if you connect directly to an AP that device still needs to talk to a modem so you are still connected to a modem at some point. Modems are still around. They have simply evolved from the 300 baud acoustic coupler. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:darcy at Vex.Net VoIP: sip:darcy at VybeNetworks.com From eryksun at gmail.com Mon Jan 29 15:27:04 2018 From: eryksun at gmail.com (eryk sun) Date: Mon, 29 Jan 2018 20:27:04 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Mon, Jan 29, 2018 at 7:43 PM, John Ladasky wrote: > On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >> >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet forever.) > > What's a... modem? A modem is a MOulator-DEMmodulator. The idea is to transmit a baseband signal by modulating the amplitude, frequency, or phase of a carrier signal. At the receiving end, the signal is demodulated back to baseband. It's a common technique in communication systems (e.g. DSL, cable, wireless modems -- even optical modems). In popular culture, people tend to think of telephone-system modems that were common in the 1980s and 90s. From tjreedy at udel.edu Mon Jan 29 15:59:02 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 29 Jan 2018 15:59:02 -0500 Subject: python not working In-Reply-To: References: Message-ID: On 1/29/2018 10:35 AM, Dan Stromberg wrote: > Please google for "pythonw.exe system error: > The program can't start because api-ms-win-crt-runtime-|1-1-0.dll is > missing from your computer. try reinstalling the program to fix this > problem.". > > I believe the Windows binaries for Python are built with a proprietary > compiler that requires a separate download. They are build with the standard VS2017 compiler. The new .dll is an update for pre-Win10 and probably early Win10, but some people block updates and don't manually request needed one's like this. -- Terry Jan Reedy From linux4michelle at tamay-dogan.net Mon Jan 29 16:08:12 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Mon, 29 Jan 2018 22:08:12 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: Am 2018-01-29 hackte John Ladasky in die Tasten: > On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >> >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet >> forever.) > > What's a... modem? Something I have in my ThinkPad T400 and let me access my Homenetwork without going over the Internet... Hehehe, ThinkPads are the more advanced Laptops... PS/2 Mouse/Keyboard, LPT with Centronics Cabel, ... All with a Dualcore 2,27GHz and 4 GByte of memory (8 GByte upgrade in sight) because of memory hungry Debian Stretch! Oh, I can even exchange my DVD-RW against a second HDD on the fly without shuting down the Laptop... ROTFL! All for only 114? on german eBay. Thanks in avance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From grant.b.edwards at gmail.com Mon Jan 29 16:16:40 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 29 Jan 2018 21:16:40 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: <96722c8f-43ce-af1d-be24-bf5a0550e3f6@VybeNetworks.com> Message-ID: On 2018-01-29, D'Arcy Cain wrote: > Modems are still around. They have simply evolved from the 300 baud > acoustic coupler. Those did _not_ work well with "trimline" style phones, but you could get by if you wrapped it in a couple towels, stuffed it in a small cooler, and kept the TV volume low. Really. -- Grant Edwards grant.b.edwards Yow! The FALAFEL SANDWICH at lands on my HEAD and I gmail.com become a VEGETARIAN ... From israel at ravnalaska.net Mon Jan 29 17:28:32 2018 From: israel at ravnalaska.net (Israel Brewster) Date: Mon, 29 Jan 2018 13:28:32 -0900 Subject: String matching based on sound? Message-ID: I am working on a python program that, at one step, takes an input (string), and matches it to songs/artists in a users library. I'm having some difficulty, however, figuring out how to match when the input/library contains numbers/special characters. For example, take the group "All-4-One". In my library it might be listed exactly like that. I need to match this to ANY of the following inputs: ? all-4-one (of course) ? all 4 one (no dashes) ? all 4 1 (all numbers) ? all four one (all spelled out) ? all for one Or, really, any other combination that sounds the same. The reasoning for this is that the input comes from a speech recognition system, so the user speaking, for example, "4", could be recognized as "for", "four" or "4". I'd imagine that Alexa/Siri/Google all do things like this (since you can ask them to play songs/artists), but I want to implement this in Python. In initial searching, I did find the "fuzzy" library, which at first glance appeared to be what I was looking for, but it, apparently, ignores numbers, with the result that "all 4 one" gave the same output as "all in", but NOT the same output as "all 4 1" - even though "all 4 1" sounds EXACTLY the same, while "all in" is only similar if you ignore the 4. So is there something similar that works with strings containing numbers? And that would only give me a match if the two strings sound identical? That is, even ignoring the numbers, I should NOT get a match between "all one" and "all in" - they are similar, but not identical, while "all one" and "all 1" would be identical. From steve+comp.lang.python at pearwood.info Mon Jan 29 17:41:36 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 22:41:36 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 11:43:36 -0800, John Ladasky wrote: > On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >> >> (The day a programmer posts a WAV file of themselves reading their code >> out aloud, is the day I turn my modem off and leave the internet >> forever.) > > What's a... modem? Its the component of the router that actually handles the telecommunications side of things. Legend has it that once upon a time they were a stand alone device. -- Steve From steve+comp.lang.python at pearwood.info Mon Jan 29 17:58:10 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 22:58:10 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 17:34:34 +0000, John Gordon wrote: [...] > The displayed filename in File Explorer was input.txt -- meaning that > the real filename was actually input.txt.txt, because File Explorer > shows file extensions as a separate column. > > Without this screenshot, we would have had only the user's (incorrect) > assertion that the file existed, and no way to diagnose the true issue. No way to diagnose it -- apart from asking the programmer to run the DOS command `dir` in the directory and copy and paste the file listing. Or os.listdir() in Python. > Granted, this was an environment issue and not a code issue, but I can > imagine situations where the same sort of thing could apply to code. The only time a programmer MUST include a screen shot is when they are trying to diagnose a problem with graphical output that can't easily and accurately be described in words. And that's a screen shot of the *output*, not the code. It MAY be useful to include a screen shot of system crashes where the error message is impossible to copy and too long and complex to transcribe. Or possibly when trying to ask for help with your GUI IDE or editor, although even then most questions can be described in words. So I don't quite rule out the possibility of programmers needing to take screen shots at all. I'm not a troglodyte :-) But what I do rule out is the necessity and usefulness of programmers taking screen shots of their *code* to ask for help with program logic or solving errors (with the very rare possible exception of especially difficult to solve syntax errors). -- Steve From steve+comp.lang.python at pearwood.info Mon Jan 29 18:04:34 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Mon, 29 Jan 2018 23:04:34 +0000 (UTC) Subject: String matching based on sound? References: Message-ID: On Mon, 29 Jan 2018 13:28:32 -0900, Israel Brewster wrote: > In initial searching, I did find the "fuzzy" library, which at first > glance appeared to be what I was looking for, but it, apparently, > ignores numbers, with the result that "all 4 one" gave the same output > as "all in", but NOT the same output as "all 4 1" - even though "all 4 > 1" sounds EXACTLY the same, while "all in" is only similar if you ignore > the 4. Before passing the string to the fuzzy matcher, do a simple text replacement of numbers to their spelled out version: "4" -> "four". You may want to do other text replacements too, based on sound or visual design, for example to deal with Kei$ha a.k.a. Keisha, etc. -- Steve From ethan at stoneleaf.us Mon Jan 29 19:32:07 2018 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 29 Jan 2018 16:32:07 -0800 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <5A6FBD07.4020908@stoneleaf.us> On 01/29/2018 02:41 PM, Steven D'Aprano wrote: > On Mon, 29 Jan 2018 11:43:36 -0800, John Ladasky wrote: >> On Sunday, January 28, 2018 at 7:07:11 AM UTC-8, Steven D'Aprano wrote: >>> (The day a programmer posts a WAV file of themselves reading their code >>> out aloud, is the day I turn my modem off and leave the internet >>> forever.) >> >> What's a... modem? > > Its the component of the router that actually handles the > telecommunications side of things. Legend has it that once upon a time > they were a stand alone device. Mine was never stand-alone. I always had to prop it up with some books. -- ~Ethan~ From arj.python at gmail.com Mon Jan 29 19:40:53 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 30 Jan 2018 04:40:53 +0400 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: well maybe screenshot of shell sessions to show varying formatted test cases might be valid (like the increasingly popular practise of coding py on android) also, sreenshot sources tend to be syntax colored which might be easier to read. overall it is a bad idea as you won't have the full code if the code is long unless you are on android with the scroll screenshot option. ease of access is the culprit ! Abdur-Rahmaan Janhangeer https://abdurrahmaanjanhangeer.wordpress.com On 28 Jan 2018 19:08, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: > > - for the sender, instead of a simple copy and paste, they have to take a > screen shot, possibly trim the image to remove any bits of the screen > they don't want to show, attach it to their email or upload it to an > image hosting site; > > - for the receiver, you are reliant on a forum which doesn't strip > attachments, or displays externally hosted images; the visually impaired > are excluded from using a screen reader; and nobody can copy or edit the > given text. > > It is as if people are deliberately inconveniencing themselves in order > to inconvenience the people they are asking to help them. > > With the exception of one *exceedingly* overrated advantage, namely the > ability to annotate the image with coloured lines and circles and > squiggles or other graphics (which most people don't bother to do), this > seems to me to be 100% counter-productive for everyone involved. Why has > it spread and why do people keep doing it? > > I don't want to be the old man yelling "Get Of My Lawn!" to the cool > kids, but is this just another sign of the downward spiral of programming > talent? Convince me that there is *some* justification for this practice. > Even a tiny one. > > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Mon Jan 29 20:13:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jan 2018 12:13:58 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 11:40 AM, Abdur-Rahmaan Janhangeer wrote: > well maybe screenshot of shell sessions to show varying formatted test > cases might be valid (like the increasingly popular practise of coding py > on android) Maybe, but that isn't code. > also, sreenshot sources tend to be syntax colored which might be easier to > read. No. The syntax highlighting comes from the code, so I should be able to paste your code into my editor and see it in colour. No benefit to seeing it in YOUR colours, which may not be helpful to me. > ease of access is the culprit ! I find that hard to believe... you're saying it's actually easier to screenshot a text editor than to copy and paste? ChrisA From arj.python at gmail.com Mon Jan 29 20:56:40 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 30 Jan 2018 05:56:40 +0400 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: XD since we were elaborating on the reasons why users use screenshot, well i was elaborating why users use screenshot, not me. those are some reasons i came across being admin in some whatsapp groups and python lists (where the user complains of his attachments not showing). Garanti sans virus. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Tue, Jan 30, 2018 at 5:13 AM, Chris Angelico wrote: > On Tue, Jan 30, 2018 at 11:40 AM, Abdur-Rahmaan Janhangeer > wrote: > > well maybe screenshot of shell sessions to show varying formatted test > > cases might be valid (like the increasingly popular practise of coding py > > on android) > > Maybe, but that isn't code. > > > also, sreenshot sources tend to be syntax colored which might be easier > to > > read. > > No. The syntax highlighting comes from the code, so I should be able > to paste your code into my editor and see it in colour. No benefit to > seeing it in YOUR colours, which may not be helpful to me. > > > ease of access is the culprit ! > > I find that hard to believe... you're saying it's actually easier to > screenshot a text editor than to copy and paste? > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From acharbly at gmail.com Mon Jan 29 21:48:35 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Tue, 30 Jan 2018 08:18:35 +0530 Subject: Variable scope in nested functions In-Reply-To: References: Message-ID: def a() : Print (value) def b() : Value = 100 Return b Its a nested function. How can I use variable value just one function above the parent function. This is possible in tcl.. Is it possible in Python too? From rosuav at gmail.com Mon Jan 29 21:58:53 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jan 2018 13:58:53 +1100 Subject: Variable scope in nested functions In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 1:48 PM, Prahallad Achar wrote: > def a() : > Print (value) > def b() : > Value = 100 > Return b > > Its a nested function. How can I use variable value just one function > above the parent function. > This is possible in tcl.. Is it possible in Python too? It is. What you have is a "nonlocal" variable. You will need to assign to the variable in the outer function though. def a(): value = None def b(): nonlocal value value = 100 return b You can do this through any number of levels of nested functions. ChrisA From rosuav at gmail.com Mon Jan 29 22:00:17 2018 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jan 2018 14:00:17 +1100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 12:56 PM, Abdur-Rahmaan Janhangeer wrote: > > XD since we were elaborating on the reasons why users use screenshot, well i was elaborating why users use screenshot, not me. those are some reasons i came across being admin in some whatsapp groups and python lists (where the user complains of his attachments not showing). > Sorry, I was using the generic "you", rather than pointing the finger specifically at you. That was a bit unclear in my post - mea culpa. ChrisA From acharbly at gmail.com Mon Jan 29 22:02:43 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Tue, 30 Jan 2018 08:32:43 +0530 Subject: Variable scope in nested functions In-Reply-To: References: Message-ID: Thanks Chris, Without using nonlocal any other options available? On 30 Jan 2018 8:30 am, "Chris Angelico" wrote: > On Tue, Jan 30, 2018 at 1:48 PM, Prahallad Achar > wrote: > > def a() : > > Print (value) > > def b() : > > Value = 100 > > Return b > > > > Its a nested function. How can I use variable value just one function > > above the parent function. > > This is possible in tcl.. Is it possible in Python too? > > It is. What you have is a "nonlocal" variable. You will need to assign > to the variable in the outer function though. > > def a(): > value = None > def b(): > nonlocal value > value = 100 > return b > > You can do this through any number of levels of nested functions. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From wikihassni at gmail.com Tue Jan 30 00:19:39 2018 From: wikihassni at gmail.com (Muhammad Waqas) Date: Mon, 29 Jan 2018 21:19:39 -0800 (PST) Subject: [UDEMY] complete python course 13 hour long course Message-ID: link as follow: https://www.udemy.com/complete-package-of-python-course-mastery-in-python-course/?couponCode=PYTHONFORUM From rustompmody at gmail.com Tue Jan 30 00:32:11 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Mon, 29 Jan 2018 21:32:11 -0800 (PST) Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> On Sunday, January 28, 2018 at 8:37:11 PM UTC+5:30, Steven D'Aprano wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: Have you heard of the ?Dutch Reach?? http://www.telegraph.co.uk/travel/news/the-dutch-reach-how-opening-car-door-like-the-dutch-could-save-lives-cycling/ Presumably it goes beyond the 'inconvenience' of images-instead-of-text to the saving-of-lives? From careenjoseph36 at gmail.com Tue Jan 30 01:53:24 2018 From: careenjoseph36 at gmail.com (careenjoseph36 at gmail.com) Date: Mon, 29 Jan 2018 22:53:24 -0800 (PST) Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: One contributing factor to this problem is the artificial environment that online courses provide to students. Students are usually typing code into a web page that contains instructions and hints. This is not how real programming gets done. So when the course is over and it?s time to use a real programming environment, students feel lost without the environment that they are accustomed to. https://www.besanttechnologies.com/training-courses/salesforce-training-institute-in-chennai From linux4michelle at tamay-dogan.net Tue Jan 30 02:07:51 2018 From: linux4michelle at tamay-dogan.net (Michelle Konzack) Date: Tue, 30 Jan 2018 08:07:51 +0100 Subject: New to Python and understanding problem In-Reply-To: References: Message-ID: <5215a8e94c6e6d370b746cf9b292c04a.squirrel@webmail.tamay-dogan.net> Good morning, Am 2018-01-30 hackte careenjoseph36 at gmail.com in die Tasten: > One contributing factor to this problem is the artificial environment that > online courses provide to students. Students are usually typing code into > a web page that contains instructions and hints. This is not how real > programming gets done. So when the course is over and it???s time to use a > real programming environment, students feel lost without the environment > that they are accustomed to. I started coding in 1984 with ASM on a 8049 Microcontroller and since then, I was always looking into other sourcecodes to understand how something is working. I have no clue about Python, but I feel very well/comfortable with it. I think, it is not very complicate to learn, because python is very logic structured. This is why I just go the idea of bug-hunting in the blueman app. Thanks in avance -- Michelle Konzack Miila ITSystems @ TDnet GNU/Linux Developer 00372-54541400 From steve+comp.lang.python at pearwood.info Tue Jan 30 02:28:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jan 2018 07:28:58 +0000 (UTC) Subject: Where has the practice of sending screen shots as source code come from? References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> Message-ID: On Mon, 29 Jan 2018 21:32:11 -0800, Rustom Mody wrote: > On Sunday, January 28, 2018 at 8:37:11 PM UTC+5:30, Steven D'Aprano > wrote: >> I'm seeing this annoying practice more and more often. Even for trivial >> pieces of text, a few lines, people post screenshots instead of copying >> the code. >> >> Where has this meme come from? It seems to be one which inconveniences >> *everyone* involved: > > Have you heard of the ?Dutch Reach?? > http://www.telegraph.co.uk/travel/news/the-dutch-reach-how-opening-car- door-like-the-dutch-could-save-lives-cycling/ Ah, yes, the Dutch Reach. That would be like the French Pox (which isn't French), the Spanish Flu (that didn't start in Spain), the Jerusalem artichoke (which is neither an artichoke nor from Jerusalem), and the turkey (the bird, which has nothing to do with Turkey, the country). This technique is neither taught nor commonly used used by the Dutch: apparently some Americans decided that because the Netherlands has a lot of cyclists, they'll say its Dutch. https://en.wikipedia.org/wiki/Dooring So let me see if I understand the logic... Rather than teach people to *explicitly* check their mirror to make sure it is safe before opening the car door, teach them a difficult, awkward maneuver which they're guaranteed to stop using the second the driving test is over, that merely points their head more-or-less vaguely in the right direction to *maybe* notice on-coming cyclists *if and only if* they're actually paying attention. I can see this falls under the problem solving technique, "We must do something, this is something, therefore we must do it!" The sorts of people who can't remember to check their mirror before opening the car door aren't the sort who will remember to use this awkward technique. And for those who can remember to do so, it is simpler and more effective to explicitly check your mirror (as the Dutch actually do). > Presumably it goes beyond the 'inconvenience' of images-instead-of-text > to the saving-of-lives? I have no idea what connection you think is between this and emailing pictures of source code in place of source code. -- Steve From acharbly at gmail.com Tue Jan 30 05:30:43 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Tue, 30 Jan 2018 16:00:43 +0530 Subject: Automation query... Plz help In-Reply-To: References: Message-ID: Luckily application supports headless automation now question is how to > invoke those jar using python. On 29 Jan 2018 10:45 pm, "Prahallad Achar" wrote: Thanks for the kind response. Sure.. Definitely I shall ask development team for the same. Regards Prahallad On 29 Jan 2018 7:48 pm, "Steven D'Aprano" wrote: > On Mon, 29 Jan 2018 17:50:46 +0530, Prahallad Achar wrote: > > > No.. Not at all. > > > > Its CTP application.. Which is basically transport planner for networks > > If you want to know whether CTP can be run headless, you should ask the > CTP support team or software maintainer, not Python forums. > > Do you have a support contract for this software? You could ask the help > desk. > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From antoon.pardon at vub.be Tue Jan 30 07:05:31 2018 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 30 Jan 2018 13:05:31 +0100 Subject: 2.6.7: Does socket.gethostbyaddr truncate? Message-ID: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> I am using python 2.6.7 to do a little network programming, but it seems I don't get all the results. When I call socket.gethostbyaddr(IP) entry [1] of the result is a list of 34 addresses. However when I use: dig -x IP I get a list of 46 addresses. Am I using the wrong function? Is this a bug? If the latter, is there a work around? -- Antoon Pardon. From rustompmody at gmail.com Tue Jan 30 08:48:15 2018 From: rustompmody at gmail.com (Rustom Mody) Date: Tue, 30 Jan 2018 05:48:15 -0800 (PST) Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> Message-ID: <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> On Tuesday, January 30, 2018 at 1:02:12 PM UTC+5:30, Steven D'Aprano wrote: > On Mon, 29 Jan 2018 21:32:11 -0800, Rustom Mody wrote: > > > On Sunday, January 28, 2018 at 8:37:11 PM UTC+5:30, Steven D'Aprano > > wrote: > >> I'm seeing this annoying practice more and more often. Even for trivial > >> pieces of text, a few lines, people post screenshots instead of copying > >> the code. > >> > >> Where has this meme come from? It seems to be one which inconveniences > >> *everyone* involved: > > > > Have you heard of the ?Dutch Reach?? > > http://www.telegraph.co.uk/travel/news/the-dutch-reach-how-opening-car- > door-like-the-dutch-could-save-lives-cycling/ > > Ah, yes, the Dutch Reach. That would be like the French Pox (which isn't > French), the Spanish Flu (that didn't start in Spain), the Jerusalem > artichoke (which is neither an artichoke nor from Jerusalem), and the > turkey (the bird, which has nothing to do with Turkey, the country). > > This technique is neither taught nor commonly used used by the Dutch: > apparently some Americans decided that because the Netherlands has a lot > of cyclists, they'll say its Dutch. reference please But before we wander far afield OT please also read below > > https://en.wikipedia.org/wiki/Dooring > > So let me see if I understand the logic... > > Rather than teach people to *explicitly* check their mirror to make sure > it is safe before opening the car door, teach them a difficult, awkward > maneuver which they're guaranteed to stop using the second the driving > test is over, that merely points their head more-or-less vaguely in the > right direction to *maybe* notice on-coming cyclists *if and only if* > they're actually paying attention. You have interpreted my example/analogy/allegory in the opposite sense from the intention. The example is really far OT for this list so let me back up the abc hierarchy: > > I can see this falls under the problem solving technique, "We must do > something, this is something, therefore we must do it!" ?to wit: Small changes of convenience expecially when habitualized can yield large benefits In this (OT) case using the 'wrong hand' can save lives If this example does not work for you lets not labour it but find others that do! > > The sorts of people who can't remember to check their mirror before > opening the car door aren't the sort who will remember to use this > awkward technique. And for those who can remember to do so, it is simpler > and more effective to explicitly check your mirror (as the Dutch actually > do). > > > > Presumably it goes beyond the 'inconvenience' of images-instead-of-text > > to the saving-of-lives? > > I have no idea what connection you think is between this and emailing > pictures of source code in place of source code. There is this slippery slope: 1. Action A is suboptimal (only makes sense wrt action B) 2. Action A is silly 3. You are doing a silly action A 4. You are silly for doing action A 5. You are silly 6. You are ____ How close you are to 6 is for you to say What I want to say is I'd like to be closer to 1 Text is a highly stylized unnatural medium Its an extreme digitization of something (do you remember?) called handwriting? At least I remember? being an unwilling school boy? using something called fountain pens? filled with something called ink? that for some reason had greater affinity for our school uniforms than for our notebooks That people who have not been cultured in a certain way can do aggravating things like talking with pics instead of text ? I wont dispute That people who have gone from the nature to nurture process conveniently forget this process is more aggravating From steve+comp.lang.python at pearwood.info Tue Jan 30 10:39:06 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jan 2018 15:39:06 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 05:48:15 -0800, Rustom Mody wrote: [...] >> Ah, yes, the Dutch Reach. That would be like the French Pox (which >> isn't French), the Spanish Flu (that didn't start in Spain), the >> Jerusalem artichoke (which is neither an artichoke nor from Jerusalem), >> and the turkey (the bird, which has nothing to do with Turkey, the >> country). >> >> This technique is neither taught nor commonly used used by the Dutch: >> apparently some Americans decided that because the Netherlands has a >> lot of cyclists, they'll say its Dutch. > > reference please The onus should be on those who claim that the technique actually is used by the Dutch. Anecdotally, the Dutch people I've spoken to on the Internet had no idea that this technique even existed. (I happened to know about this because this week's "QI" on the BBC happened to mention it, and that show is very popular among the Dutch. There's been quite a number of comments on Reddit from Dutch people complaining about the QI Elves' lack of fact checking regarding this, as well as machine vision.) Even the organisation who invented the term acknowledge that lots of people, *especially* Dutch people, take issue with their claim that the technique is required by Dutch law, is taught in Dutch schools, and is routinely practiced by the Dutch. Their answer? To paraphrase their rambling explanation here: https://www.dutchreach.org/is-the-dutch-reach-really-dutch/ Trust us, it was required by NL law decades ago but isn't any longer now, it was never given a name and so even though it is really commonplace and everyone in NL does it, nobody remembers doing it. And besides, its only young Dutch people who say that nobody uses this technique there, what do they know? Old people use it all the time. (Not a direct quote.) Yeah, colour me skeptical that this actually ever was a commonplace practice in the NL, or anywhere else for that matter. There's an easy way to find out whether this is commonplace in the NLs. Look at Dutch movies and television. How do the characters open their car doors? By the way, anyone who actually has a car: I encourage you to sit in driver's seat of your car and try out this "Dutch Reach" maneuver for yourself. If you do, you will probably have the same experience that I did when I tried it: It is awkward and clumsy to reach across and open the door with your opposite hand, but most importantly, doing so does NOT cause your upper body to twist around enough to automatically look behind the car for on- coming cyclists. I estimate that my head and upper torso rotated no more than about 30 degree from "straight ahead". This effectively and completely undermines the supposed claim that this technique makes it *automatic* to look behind you for on-coming cyclists. That simply isn't the case. Whether you use the arm closest or furthest from the door, you still need to make a conscious effort to either look over your shoulder (where your view will be obstructed by the car pillar by the door), or consciously check your side mirror. Guess which one is both easier and more effective? -- Steve From m at funkyhat.org Tue Jan 30 10:48:29 2018 From: m at funkyhat.org (Matt Wheeler) Date: Tue, 30 Jan 2018 15:48:29 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 at 15:39 Steven D'Aprano < steve+comp.lang.python at pearwood.info> wrote: > This effectively and completely undermines the supposed claim that this > technique makes it *automatic* to look behind you for on-coming cyclists. > That simply isn't the case. Whether you use the arm closest or furthest > from the door, you still need to make a conscious effort to either look > over your shoulder (where your view will be obstructed by the car pillar > by the door), or consciously check your side mirror. > Checking the side mirrors isn't particularly helpful advice if you're sitting in any seat other than the driver's seat, however. -- -- Matt Wheeler http://funkyh.at From nicholas.cole at gmail.com Tue Jan 30 10:54:30 2018 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Tue, 30 Jan 2018 15:54:30 +0000 Subject: Import statements and multiprocessing Message-ID: Dear List, I have a strange problem on python 3.6.1 I am using the multiprocessing function to parallelize an expensive operation, using the multiprocessing.Pool() and Pool.map() functions. The function I am passing to map calls a function in another file within the same model. And that file has a from .some_file_in_the_package import * line at the top. However, in each function called in that file, I seem to need to put an explicit important statement buried within the function in order for the code to work: def some_function(): from .some_file_in_the_package import ThisObject It is as if in the worker processes created by Pool.map() the from . import * directive is being completely ignored. With the explicit import statements everything works as expected. What could be going wrong? Best wishes, Nicholas From steve+comp.lang.python at pearwood.info Tue Jan 30 11:09:08 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jan 2018 16:09:08 +0000 (UTC) Subject: [OT] Text as digitization [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 05:48:15 -0800, Rustom Mody wrote: > Text is a highly stylized unnatural medium Hmmm. I think it is no more "unnatural" than whale songs or the extremely formalised dancing rituals of birds or any other animal communication. Our species just takes this communication to a higher level of information content -- but that's a matter of degree, not kind. But okay, for the sake of the argument, it's highly stylised (like formal logic, mathematical reasoning, filling in your tax return, and Kabuki theatre). How is that relevant? We're talking about *programmers* here -- if they can't cope with the highly stylised textual medium in which they work, they're going to really struggle to, you know, actually program. And as for being unnatural -- that would be like electric lights, antibiotics, eyeglasses, and dentistry. Still not seeing the downside here. The natural state of humanity is to spend the majority of your life being cold and hungry, hot and hungry, or terrified, before dying young. > Its an extreme digitization > of something (do you remember?) called handwriting? Of course I remember handwriting. I still use handwriting. Typed text has evolved from handwriting, via printing. What's your point? > That people who have not been cultured in a certain way can do > aggravating things like talking with pics instead of text ? I wont > dispute That people who have gone from the nature to nurture process > conveniently forget this process is more aggravating I'm sorry, I don't understand this paragraph. -- Steve From steve+comp.lang.python at pearwood.info Tue Jan 30 11:14:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jan 2018 16:14:58 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 15:48:29 +0000, Matt Wheeler wrote: > Checking the side mirrors isn't particularly helpful advice if you're > sitting in any seat other than the driver's seat, however. That's a fair point. But it really only applies to those sitting on the driver's side in the back seat. On the passenger's side, you open the door towards the curb, out of the way of both cyclists and on-coming traffic. For the minority of passengers sitting in the rear on the driver's side, you just have to be extra careful about opening the door and stepping out into the road, not just for the sake of cyclists, but also for the sake of not being side-swiped by a car or truck as you step into the road. -- Steve From rosuav at gmail.com Tue Jan 30 11:17:48 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Jan 2018 03:17:48 +1100 Subject: [OT] Text as digitization [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Wed, Jan 31, 2018 at 3:09 AM, Steven D'Aprano wrote: > On Tue, 30 Jan 2018 05:48:15 -0800, Rustom Mody wrote: > >> Text is a highly stylized unnatural medium > [chomp] > >> That people who have not been cultured in a certain way can do >> aggravating things like talking with pics instead of text ? I wont >> dispute That people who have gone from the nature to nurture process >> conveniently forget this process is more aggravating > > I'm sorry, I don't understand this paragraph. I think that paragraph is his demonstration that text can, if properly crafted, be as useless as an image. *ducking for cover* ChrisA From steve+comp.lang.python at pearwood.info Tue Jan 30 11:23:58 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jan 2018 16:23:58 +0000 (UTC) Subject: Import statements and multiprocessing References: Message-ID: On Tue, 30 Jan 2018 15:54:30 +0000, Nicholas Cole wrote: [...] > The function I am passing to map calls a function in another file within > the same model. And that file has a > > from .some_file_in_the_package import * > > line at the top. > > However, in each function called in that file, I seem to need to put an > explicit important statement buried within the function in order for the > code to work: > > def some_function(): > from .some_file_in_the_package import ThisObject > > It is as if in the worker processes created by Pool.map() the from . > import * directive is being completely ignored. [...] > What could be going wrong? I would say you're probably misinterpreting the nature of the problem. Import * isn't a directive that can be ignored. Can you show us a *simplified* demonstration? A minimal sample program which we can run that demonstrates the issue? Here's an expensive function for you to call: import time def process(value): time.sleep(3) return value+1 And a data set to process: data = [1, 2, 3, 4] Can you put them in a minimal package, show us how you would import them, and call worker processes with them? We ought to be able to copy the code and run it ourselves to see the issue first hand. See http://sscce.org/ for further details. -- Steve From grant.b.edwards at gmail.com Tue Jan 30 11:24:57 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 30 Jan 2018 16:24:57 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On 2018-01-30, Steven D'Aprano wrote: > On Tue, 30 Jan 2018 15:48:29 +0000, Matt Wheeler wrote: > >> Checking the side mirrors isn't particularly helpful advice if you're >> sitting in any seat other than the driver's seat, however. > > That's a fair point. > > But it really only applies to those sitting on the driver's side in the > back seat. On the passenger's side, you open the door towards the curb, > out of the way of both cyclists and on-coming traffic. Unless the bike lane is between the "parallel parking lane" and the curb[*], in which case it's the passenger side doors that are used to catch bicycles rather than the driver's side doors. [*] This seems to be increasingly common here in the Minneapolis / St. Paul area -- Grant Edwards grant.b.edwards Yow! People humiliating at a salami! gmail.com From nicholas.cole at gmail.com Tue Jan 30 11:33:34 2018 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Tue, 30 Jan 2018 16:33:34 +0000 Subject: Import statements and multiprocessing In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 4:23 PM, Steven D'Aprano wrote: > On Tue, 30 Jan 2018 15:54:30 +0000, Nicholas Cole wrote: > I would say you're probably misinterpreting the nature of the problem. > Import * isn't a directive that can be ignored. > > Can you show us a *simplified* demonstration? A minimal sample program > which we can run that demonstrates the issue? [snip] I find it extremely odd. File A: the multiprocessing code and the map function. file B: a set of library functions called by the function called in file A. file C: included in file B by use of a from .C import * statement. But none of the functions in B can see the objects defined in C unless an explicit relative import is included in their functions. In single process code this seems to work perfectly. Producing a simplified version is not trivial. But I shall see what I can do. At any rate, in our current code, the import * directive *is* ineffective in subprocesses. N. From ian.g.kelly at gmail.com Tue Jan 30 11:39:26 2018 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 30 Jan 2018 09:39:26 -0700 Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, Jan 30, 2018 at 9:24 AM, Grant Edwards wrote: > On 2018-01-30, Steven D'Aprano wrote: >> On Tue, 30 Jan 2018 15:48:29 +0000, Matt Wheeler wrote: >> >>> Checking the side mirrors isn't particularly helpful advice if you're >>> sitting in any seat other than the driver's seat, however. >> >> That's a fair point. >> >> But it really only applies to those sitting on the driver's side in the >> back seat. On the passenger's side, you open the door towards the curb, >> out of the way of both cyclists and on-coming traffic. > > Unless the bike lane is between the "parallel parking lane" and the > curb[*], in which case it's the passenger side doors that are used to > catch bicycles rather than the driver's side doors. > > [*] This seems to be increasingly common here in the Minneapolis / > St. Paul area This seems like it would tend to make the "dooring" problem worse, since people are not generally accustomed to using caution when opening their door toward the curb rather than the street. Also, I just wanted to add that if you're going to use the side mirror then you need to watch it for a couple of seconds rather than a quick glance. Most people's mirrors are not particularly well adjusted to capture the car's blind spot, which is exactly where an oncoming cyclist would be. A blind spot that can fit an entire car inside of it is enormous compared to the size of a bicycle. From nicholas.cole at gmail.com Tue Jan 30 11:45:02 2018 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Tue, 30 Jan 2018 16:45:02 +0000 Subject: Import statements and multiprocessing In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 4:33 PM, Nicholas Cole wrote: > On Tue, Jan 30, 2018 at 4:23 PM, Steven D'Aprano > wrote: >> On Tue, 30 Jan 2018 15:54:30 +0000, Nicholas Cole wrote: > >> I would say you're probably misinterpreting the nature of the problem. >> Import * isn't a directive that can be ignored. >> >> Can you show us a *simplified* demonstration? A minimal sample program >> which we can run that demonstrates the issue? > > [snip] Ah! Looking at our code, I was wrong. The from .this_package import * directive is not importing a single file but a subpackage __init__ file. That __init__ file does not have its own __all__ statement, but seems to just be relying on importing from different files in the subpackage. Could that be the problem? Even so, I'm unsure why it is showing up only when used in multiprocessing and works fine everywhere else. N. From alister.ware at ntlworld.com Tue Jan 30 11:47:13 2018 From: alister.ware at ntlworld.com (alister) Date: Tue, 30 Jan 2018 16:47:13 GMT Subject: Where has the practice of sending screen shots as source code come from? References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 07:28:58 +0000, Steven D'Aprano wrote: > On Mon, 29 Jan 2018 21:32:11 -0800, Rustom Mody wrote: > >> On Sunday, January 28, 2018 at 8:37:11 PM UTC+5:30, Steven D'Aprano >> wrote: >>> I'm seeing this annoying practice more and more often. Even for >>> trivial pieces of text, a few lines, people post screenshots instead >>> of copying the code. >>> >>> Where has this meme come from? It seems to be one which inconveniences >>> *everyone* involved: >> >> Have you heard of the ?Dutch Reach?? >> http://www.telegraph.co.uk/travel/news/the-dutch-reach-how-opening-car- > door-like-the-dutch-could-save-lives-cycling/ > > Ah, yes, the Dutch Reach. That would be like the French Pox (which isn't > French), the Spanish Flu (that didn't start in Spain), the Jerusalem > artichoke (which is neither an artichoke nor from Jerusalem), and the > turkey (the bird, which has nothing to do with Turkey, the country). > > This technique is neither taught nor commonly used used by the Dutch: > apparently some Americans decided that because the Netherlands has a lot > of cyclists, they'll say its Dutch. > The British TV show QI seemed to think this is actually part of the Dutch driving test although they have been known to make mistakes > https://en.wikipedia.org/wiki/Dooring > > So let me see if I understand the logic... > > Rather than teach people to *explicitly* check their mirror to make sure > it is safe before opening the car door, teach them a difficult, awkward > maneuver which they're guaranteed to stop using the second the driving > test is over, that merely points their head more-or-less vaguely in the > right direction to *maybe* notice on-coming cyclists *if and only if* > they're actually paying attention. > > I can see this falls under the problem solving technique, "We must do > something, this is something, therefore we must do it!" > on this I can agree with you. Personally I tend to "crack" the door a little & then look down the road Before opening fully. i am pretty sure I also (at least subconsciously) check the mirror. Then again many* cyclists tend to be in the blind spot, dressed in dark clothing & have no lights & ride with zero regard to the rules of the road. > The sorts of people who can't remember to check their mirror before > opening the car door aren't the sort who will remember to use this > awkward technique. And for those who can remember to do so, it is > simpler and more effective to explicitly check your mirror (as the Dutch > actually do). > > >> Presumably it goes beyond the 'inconvenience' of images-instead-of-text >> to the saving-of-lives? > > I have no idea what connection you think is between this and emailing > pictures of source code in place of source code. * not all, some do appear have a desire to live. -- The pollution's at that awkward stage. Too thick to navigate and too thin to cultivate. -- Doug Sneyd From grant.b.edwards at gmail.com Tue Jan 30 11:47:34 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 30 Jan 2018 16:47:34 +0000 (UTC) Subject: [OT] Text as digitization [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On 2018-01-30, Steven D'Aprano wrote: > We're talking about *programmers* here -- if they can't cope with the > highly stylised textual medium in which they work, they're going to > really struggle to, you know, actually program. Well, to be fair, many of them do (struggle to actually program, that is). Spend any time at all reading PHP forums and you'll despair for humanity. Not only are they trying to build airports and radios out of bamboo and twine, they don't even know how to split cane or tie a knot. -- Grant Edwards grant.b.edwards Yow! WHO sees a BEACH BUNNY at sobbing on a SHAG RUG?! gmail.com From drsalists at gmail.com Tue Jan 30 11:56:16 2018 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 30 Jan 2018 08:56:16 -0800 Subject: 2.6.7: Does socket.gethostbyaddr truncate? In-Reply-To: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> References: <14988d6b-5039-029c-3295-5bb48479b5cf@vub.be> Message-ID: dig -x should return a single PTR in all cases, shouldn't it? What IP are you using? 2.6 is very old. You probably should move to at Least 2.7, and plan a move to 3.x. On Tue, Jan 30, 2018 at 4:05 AM, Antoon Pardon wrote: > I am using python 2.6.7 to do a little network programming, but it seems I don't > get all the results. > > When I call socket.gethostbyaddr(IP) entry [1] of the result is a list of 34 addresses. > > However when I use: dig -x IP I get a list of 46 addresses. From rhodri at kynesim.co.uk Tue Jan 30 13:10:53 2018 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 30 Jan 2018 18:10:53 +0000 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> Message-ID: <8010e71c-5a00-5fec-ca92-7075a8d09444@kynesim.co.uk> On 30/01/18 16:47, alister via Python-list wrote: > The British TV show QI seemed to think this is actually part of the Dutch > driving test although they have been known to make mistakes It has to be noted that the QI Elves did not do particularly well in Only Connect... -- Rhodri James *-* Kynesim Ltd From python at mrabarnett.plus.com Tue Jan 30 13:22:26 2018 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 30 Jan 2018 18:22:26 +0000 Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: <8b9a503a-1a70-b4ac-00d0-68021445f715@mrabarnett.plus.com> On 2018-01-30 15:39, Steven D'Aprano wrote: > On Tue, 30 Jan 2018 05:48:15 -0800, Rustom Mody wrote: > > [...] >>> Ah, yes, the Dutch Reach. That would be like the French Pox (which >>> isn't French), the Spanish Flu (that didn't start in Spain), the >>> Jerusalem artichoke (which is neither an artichoke nor from Jerusalem), >>> and the turkey (the bird, which has nothing to do with Turkey, the >>> country). >>> >>> This technique is neither taught nor commonly used used by the Dutch: >>> apparently some Americans decided that because the Netherlands has a >>> lot of cyclists, they'll say its Dutch. >> >> reference please > > The onus should be on those who claim that the technique actually is used > by the Dutch. Anecdotally, the Dutch people I've spoken to on the > Internet had no idea that this technique even existed. > Do we know of anyone who's Dutch and frequents a Python newsgroup? :-) [snip] From pkpearson at nowhere.invalid Tue Jan 30 13:31:31 2018 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 30 Jan 2018 18:31:31 GMT Subject: Where has the practice of sending screen shots as source code come from? References: Message-ID: On Mon, 29 Jan 2018 14:46:59 -0500, Dennis Lee Bieber wrote: > On 29 Jan 2018 17:26:32 GMT, Peter Pearson > declaimed the following: > >> >>In 1964, the IBM exhibit at the World's Fair in New York demonstrated >>a system that read dates that visitors wrote by hand. (You were >>supposed to write your date of birth, and the system then printed >>the New York Times's headline for that date.) > > Was it somehow scanning dates from paper or cards, or was it using some > sort of touch sensor input (pressure tablet, or a pen in a harness with > position sensors). > > The first would be OCR proper, while the second is what many PDAs (and > some tablets) rely upon. The second provides actually stroke vector > information on how the characters are formed, which is more reliable than > just seeing pixel transitions and trying to match characters to them. We wrote our dates of birth on paper or punch cards (I forget which) with an ordinary pen. We handed the papers (or cards) to someone who fed them into a machine, which printed slips of paper that were handed to us as we exited. According to the promotional displays, our writing was examined optically; one poster showed a scan path that resembled an extremely prolate cycloid following along the handwritten line. -- To email me, substitute nowhere->runbox, invalid->com. From pkpearson at nowhere.invalid Tue Jan 30 14:21:42 2018 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 30 Jan 2018 19:21:42 GMT Subject: checksum problem References: Message-ID: On Tue, 30 Jan 2018 11:24:07 +0100, jak wrote: > Hello everybody, > I'm using python 2.7.14 and calculating the checksum with the sha1 > algorithm and this happens: the checksum is wrong until I read the whole > file in one shot. Here is a test program: > > import hashlib > > def Checksum(fname, blocks): > m = hashlib.sha1() > print "sha1 block size: " + str(m.block_size * blocks) > with open(fname, "rb") as fh: > for data in fh.read(m.block_size * blocks): > m.update(data) > return m.hexdigest() > > def main(): > for b in range(10, 260, 10): > print str(b) + ': ' + > Checksum("d:/upload_688df390ea0bd728fdbeb8972ae5f7be.zip", b) > > if __name__ == '__main__': > main() > > and this is the result output: > > sha1 block size: 640 > 10: bf09de3479b2861695fb8b7cb18133729ef00205 > sha1 block size: 1280 > 20: 71a5499e4034fdcf0eb0c5d960c8765a8b1f032d > . > . > . > sha1 block size: 12160 > 190: 956d017b7ed734a7b4bfdb02519662830dab4fbe > sha1 block size: 12800 > 200: 1b2febe05b70f58350cbb87df67024ace43b76e5 > sha1 block size: 13440 > 210: 93832713edb40cf4216bbfec3c659842fbec6ae4 > sha1 block size: 14080 > 220: 93832713edb40cf4216bbfec3c659842fbec6ae4 > . > . > . > > the file size is 13038 bytes and its checksum is > 93832713edb40cf4216bbfec3c659842fbec6ae4 > > Why do I get these results? What am I doing wrong? > > Thanks to everyone in advance. I believe your "for data in fh.read" loop just reads the first block of the file and loops over the bytes in that block (calling m.update once for each byte, probably the least efficient approach imaginable), omitting the remainder of the file. That's why you start getting the right answer when the first block is big enough to encompass the whole file. -- To email me, substitute nowhere->runbox, invalid->com. From tjreedy at udel.edu Tue Jan 30 14:26:25 2018 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 30 Jan 2018 14:26:25 -0500 Subject: Import statements and multiprocessing In-Reply-To: References: Message-ID: On 1/30/2018 10:54 AM, Nicholas Cole wrote: > I have a strange problem on python 3.6.1 [involving multiprocessing] I think the first thing you should do is upgrade to 3.6.4 to get all the bugfixes since 3.6.4. I am pretty sure there have been some for multiprocessing itself. *Then* see if you still have a problem. -- Terry Jan Reedy From tkadm30 at yandex.com Tue Jan 30 14:35:23 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 30 Jan 2018 14:35:23 -0500 Subject: PyPy support breaking CPython compatibility? Message-ID: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> Hi, I managed to patch Schevo and Durus to run under PyPy 5.9. However, I'm afraid the changes is breaking Python 2.7 compatibility. I'm not sure how I should distribute my changes to the respective projects. Since I decided to use more PyPy in my Django projects, should I drop Python 2.7 support and release the experimental code on PyPi ? What do you think? Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Tue Jan 30 14:37:09 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Jan 2018 06:37:09 +1100 Subject: checksum problem In-Reply-To: References: Message-ID: On Wed, Jan 31, 2018 at 6:21 AM, Peter Pearson wrote: > On Tue, 30 Jan 2018 11:24:07 +0100, jak wrote: >> with open(fname, "rb") as fh: >> for data in fh.read(m.block_size * blocks): >> m.update(data) >> return m.hexdigest() >> > > I believe your "for data in fh.read" loop just reads the first block of > the file and loops over the bytes in that block (calling m.update once > for each byte, probably the least efficient approach imaginable), > omitting the remainder of the file. That's why you start getting the > right answer when the first block is big enough to encompass the whole > file. Correct analysis. Generally, if you want to read a file in chunks, the easiest way is this: while "moar data": data = fh.read(block_size) if not data: break m.update(data) That should get you the correct result regardless of your block size, and then you can tweak the block size to toy with performance. ChrisA From rosuav at gmail.com Tue Jan 30 14:53:24 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Jan 2018 06:53:24 +1100 Subject: PyPy support breaking CPython compatibility? In-Reply-To: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> Message-ID: On Wed, Jan 31, 2018 at 6:35 AM, Etienne Robillard wrote: > Hi, > > I managed to patch Schevo and Durus to run under PyPy 5.9. However, I'm > afraid the changes is breaking Python 2.7 compatibility. > > I'm not sure how I should distribute my changes to the respective projects. > > Since I decided to use more PyPy in my Django projects, should I drop Python > 2.7 support and release the experimental code on PyPi ? > > > What do you think? First off, what's your status on Python 3? When you say "PyPy 5.9", are you referring to the one that's compatible with Python 3.5 or the one that's compatible with Python 2.7? If you're supporting Python 3, I don't think there's any problem with saying "Python 2.7 support ceases as of Schevo v4.0, so if you need Py 2.7 use Schevo 3.x". (It's not as if the old versions will suddenly cease working or anything.) On the other hand, if you're looking at two different Python 2.7 interpreters, you'll be narrowing your potential userbase significantly by dropping CPython support. Maybe that's not an issue for you (hey, I've "released" code that depends on features in unreleased versions of Python - the intended userbase is basically just me, even though the code's open source and published), but it's definitely a consideration. ChrisA From tkadm30 at yandex.com Tue Jan 30 15:09:16 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 30 Jan 2018 15:09:16 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> Message-ID: Hi Chris, Le 2018-01-30 ? 14:53, Chris Angelico a ?crit?: > If you're supporting Python 3, I don't think there's any problem with > saying "Python 2.7 support ceases as of Schevo v4.0, so if you need Py > 2.7 use Schevo 3.x". (It's not as if the old versions will suddenly > cease working or anything.) Fair enough. I'll gladly release libschevo and libdurus 4.0 very soon. :) Thank you, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From ned at nedbatchelder.com Tue Jan 30 15:14:28 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 30 Jan 2018 15:14:28 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> Message-ID: <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> On 1/30/18 2:35 PM, Etienne Robillard wrote: > Hi, > > I managed to patch Schevo and Durus to run under PyPy 5.9. However, > I'm afraid the changes is breaking Python 2.7 compatibility. I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 thing as Chris mentions.) > > I'm not sure how I should distribute my changes to the respective > projects. You should make pull requests to the projects so they can incorporate the changes. --Ned. > > Since I decided to use more PyPy in my Django projects, should I drop > Python 2.7 support and release the experimental code on PyPi ? > > > What do you think? > > Etienne > > From jugurtha.hadjar at gmail.com Tue Jan 30 15:22:39 2018 From: jugurtha.hadjar at gmail.com (Jugurtha Hadjar) Date: Tue, 30 Jan 2018 21:22:39 +0100 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> Message-ID: <8b1293fd-8a30-fa21-ae74-64bb2e61dbf1@gmail.com> On 01/29/2018 03:48 PM, alister via Python-list wrote: > On Mon, 29 Jan 2018 15:20:06 +0100, Jugurtha Hadjar wrote: > >> On 01/28/2018 04:43 PM, Skip Montanaro wrote: >>> I've never been a Windows user, but at my current job, Windows is core >>> to just about everything, so I am forced to use it for a lot of stuff >>> (Outlook, SQL Server, Excel, etc). >> I was hired at a startup which made a good impression on me and I was >> eager to start working. I checked in for my first day, signed the >> paperwork, then the CTO showed me around and told me "Here's your >> laptop, you can install Windows and I'll check in with you later". Life >> started draining out of my body and my mind was racing in all directions >> before he even finished his sentence. I felt tricked: I was hired based >> on a test and the file I've received was a *.tar.gz* and it made me >> happy because I assumed they were a NIX shop.. >> >> I was considering how I should go about quitting gracefully. I have >> stopped using Windows in 2008 for psychological reasons for it made me >> really anxious, irritable, angry, and tense to the point of abusing >> hardware with low kicks and punches which was not very healthy or sane. >> It was only when I switched OS that this behavior stopped. There was no >> way I would be going back to my bully. >> >> The CTO then said "Sorry.. I meant Ubuntu." and seeing my pale face, he >> said in a reassuring tone "Don't be afraid, there are no Windows >> machines here." which brought me back to life. >> >> I hope to be as brave as you some day. > Any Vacancies > whatever they do I am sure I can learn :-) > > > We're hiring all the good (technical and human side) people we can find/afford. It's a small startup that helps its clients improve business using machine learning. We're in Algiers, Algeria and Paris, France. -- ~ Jugurtha Hadjar, From tkadm30 at yandex.com Tue Jan 30 15:58:13 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Tue, 30 Jan 2018 15:58:13 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> Message-ID: Hi Ned, Le 2018-01-30 ? 15:14, Ned Batchelder a ?crit?: > I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 > thing as Chris mentions.) Please take a look at the changesets: https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 In short, it seems PyPy automagically adds a __weakref__ attribute to __slots__, causing the CPython interpreter to raise a TypeError... Cheers, Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Tue Jan 30 16:08:18 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Jan 2018 08:08:18 +1100 Subject: PyPy support breaking CPython compatibility? In-Reply-To: References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> Message-ID: On Wed, Jan 31, 2018 at 7:58 AM, Etienne Robillard wrote: > Hi Ned, > > > Le 2018-01-30 ? 15:14, Ned Batchelder a ?crit : >> >> I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 thing >> as Chris mentions.) > > > Please take a look at the changesets: > > https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 > https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 > > In short, it seems PyPy automagically adds a __weakref__ attribute to > __slots__, causing the CPython interpreter to raise a TypeError... > I'm confused by this: -if os.environ.get('SCHEVO_OPTIMIZE', '1') == '1': +if os.environ.get('SCHEVO_OPTIMIZE', '1') == True: BTW, this part you should probably consider doing differently: - except schevo.error.SchemaFileIOError: + except schevo.error.SchemaFileIOError, ex: + print ex The comma version is only needed on really old versions of Python (2.4? something like that), and won't work on Python 3. Unless support for ancient Pythons is important to you, I recommend using the "except type as name:" syntax, which works on 2.7 and on 3.x. But the weakref situation is curious. Can you show a minimal test-case for the difference? Might be something that one interpreter or the other needs to update. ChrisA From ned at nedbatchelder.com Tue Jan 30 16:38:06 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Tue, 30 Jan 2018 16:38:06 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> Message-ID: On 1/30/18 4:08 PM, Chris Angelico wrote: > On Wed, Jan 31, 2018 at 7:58 AM, Etienne Robillard wrote: >> Hi Ned, >> >> >> Le 2018-01-30 ? 15:14, Ned Batchelder a ?crit : >>> I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 thing >>> as Chris mentions.) >> Please take a look at the changesets: >> >> https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 >> https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 >> >> In short, it seems PyPy automagically adds a __weakref__ attribute to >> __slots__, causing the CPython interpreter to raise a TypeError... >> > I'm confused by this: > > -if os.environ.get('SCHEVO_OPTIMIZE', '1') == '1': > +if os.environ.get('SCHEVO_OPTIMIZE', '1') == True: > I was also curious about this: when does os.environ.get return anything but a string? --Ned. From greg.ewing at canterbury.ac.nz Tue Jan 30 16:45:54 2018 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Wed, 31 Jan 2018 10:45:54 +1300 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: > On Mon, 29 Jan 2018 22:41:36 +0000 (UTC), Steven D'Aprano > declaimed the following: > >>Its the component of the router that actually handles the >>telecommunications side of things. Legend has it that once upon a time >>they were a stand alone device. Even more distant legend suggests that modems existed before the concept of a router was even thought of. Hard to imagine, I know! -- Greg From grant.b.edwards at gmail.com Tue Jan 30 17:40:08 2018 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Tue, 30 Jan 2018 22:40:08 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On 2018-01-30, Ian Kelly wrote: > >> Unless the bike lane is between the "parallel parking lane" and the >> curb[*], in which case it's the passenger side doors that are used to >> catch bicycles rather than the driver's side doors. >> >> [*] This seems to be increasingly common here in the Minneapolis / >> St. Paul area > > This seems like it would tend to make the "dooring" problem worse, > since people are not generally accustomed to using caution when > opening their door toward the curb rather than the street. That may be mitigated by the high percentage of cars in US cities that have no passengers. -- Grant Edwards grant.b.edwards Yow! Hello. Just walk at along and try NOT to think gmail.com about your INTESTINES being almost FORTY YARDS LONG!! From steve+comp.lang.python at pearwood.info Tue Jan 30 20:23:11 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 31 Jan 2018 01:23:11 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> Message-ID: On Tue, 30 Jan 2018 09:39:26 -0700, Ian Kelly wrote: > Also, I just wanted to add that if you're going to use the side mirror > then you need to watch it for a couple of seconds rather than a quick > glance. Most people's mirrors are not particularly well adjusted to > capture the car's blind spot, which is exactly where an oncoming cyclist > would be. A blind spot that can fit an entire car inside of it is > enormous compared to the size of a bicycle. How slow is the bike travelling that it takes a couple of seconds to cross the length of a car? If they're travelling that slowly, they're not so much going to collide with the door as gently nudge it. Of course you should do more than a quick glance, it should be a proper look[1], but surely in a couple of seconds a cyclist could easily travel three or four car lengths. [1] You're not just looking out for cyclists, but traffic, since you're stepping out into traffic yourself and presumably would prefer not to have some wayward car take your door off, and you with it. -- Steve From steve+comp.lang.python at pearwood.info Tue Jan 30 20:37:54 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Wed, 31 Jan 2018 01:37:54 +0000 (UTC) Subject: Automation query... Plz help References: Message-ID: On Tue, 30 Jan 2018 16:00:43 +0530, Prahallad Achar wrote: > Luckily application supports headless automation now question is how to > invoke those jar using python. I can see two approaches: (1) Calling the jar directly from Python. I don't think you can do that from CPython, but you might be able to do it from Jython. (Jython is Python implemented in Java, and is designed to allow Java and Python code to call each other). Or there might be some third-party library that runs under CPython which allows you to do so. (2) Call the jar as an external process, in whatever way you would call a jar from an external application. Have you tried googling? https://duckduckgo.com/html/?q=call%20a%20jar%20from%20python (I think this is the limit of any help I can give -- I know nothing about jars except that they are some sort of Java code.) -- Steve From porton at narod.ru Wed Jan 31 00:12:18 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 07:12:18 +0200 Subject: Package containing C sources Message-ID: I am going to create a Python wrapper around a generally useful C library. So the wrapper needs to contain some C code to glue them together. Can I upload a package containing C sources to PyPi? If not, what is the proper way to distribute it? -- Victor Porton - http://portonvictor.org From acharbly at gmail.com Wed Jan 31 00:24:40 2018 From: acharbly at gmail.com (Prahallad Achar) Date: Wed, 31 Jan 2018 10:54:40 +0530 Subject: Automation query... Plz help In-Reply-To: References: Message-ID: Thank you. Indeed I did a search but couldn't find a right approach. Jython! Yes.. It supports to call jar file. As you said... Application support team has to modify few things on application side where object creation should be public rather protected On 31 Jan 2018 7:12 am, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > On Tue, 30 Jan 2018 16:00:43 +0530, Prahallad Achar wrote: > > > Luckily application supports headless automation now question is how to > > invoke those jar using python. > > I can see two approaches: > > (1) Calling the jar directly from Python. > > I don't think you can do that from CPython, but you might be able to do > it from Jython. (Jython is Python implemented in Java, and is designed to > allow Java and Python code to call each other). Or there might be some > third-party library that runs under CPython which allows you to do so. > > > (2) Call the jar as an external process, in whatever way you would call a > jar from an external application. > > Have you tried googling? > > https://duckduckgo.com/html/?q=call%20a%20jar%20from%20python > > > (I think this is the limit of any help I can give -- I know nothing about > jars except that they are some sort of Java code.) > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From porton at narod.ru Wed Jan 31 01:52:14 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 08:52:14 +0200 Subject: Package containing C sources (Posting On Python-List Prohibited) References: Message-ID: Lawrence D?Oliveiro wrote: > On Wednesday, January 31, 2018 at 6:13:00 PM UTC+13, Victor Porton wrote: >> I am going to create a Python wrapper around a generally useful C >> library. So the wrapper needs to contain some C code to glue them >> together. > > Not necessarily. It?s often possible to implement such a wrapper entirely > in Python, using ctypes . But if I will find that I need C code, do I need to package it separately? So I would get three packages: the C library, the C wrapper for Python, and the Python code. Can this be done with just two packages: the C library and C wrapper and Python in one package? -- Victor Porton - http://portonvictor.org From dieter at handshake.de Wed Jan 31 02:37:10 2018 From: dieter at handshake.de (dieter) Date: Wed, 31 Jan 2018 08:37:10 +0100 Subject: Package containing C sources References: Message-ID: <87bmha5vix.fsf@handshake.de> Victor Porton writes: > I am going to create a Python wrapper around a generally useful C library. > So the wrapper needs to contain some C code to glue them together. > > Can I upload a package containing C sources to PyPi? You can. This is documented in "https://docs.python.org/2/extending/building.html" (for Python 2; likely, it works in the same way for Python 3). Note: I use "setuptools" (a "distutils" extension) rather than the base "distutils" -- essentially, because it facilitates the inclusion of non Python files in a distribution. I do not know whether the base "distutils" automatically includes the sources of C extensions (likely it does); "setuptools" provides a way to include everything under control of a versioning system (which I rely upon to not miss important files). From porton at narod.ru Wed Jan 31 02:57:55 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 09:57:55 +0200 Subject: Handle SIGINT in C and Python Message-ID: I need to assign a real C signal handler to SIGINT. This handler may be called during poll() waiting for data. For this reason I cannot use Python signals because "A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction)." I want after my signal handler for SIGINT executed to raise KeyboardInterrupt (as if I didn't installed my own signal handler). Is this possible? How? -- Victor Porton - http://portonvictor.org From arj.python at gmail.com Wed Jan 31 03:33:13 2018 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Wed, 31 Jan 2018 12:33:13 +0400 Subject: Where has the practice of sending screen shots as source code come from? In-Reply-To: References: Message-ID: it seems that at the end, invariably, the subject becomes well routed to regions far away from codeland Abdur-Rahmaan Janhangeer https://abdurrahmaanjanhangeer.wordpress.com On 28 Jan 2018 19:08, "Steven D'Aprano" < steve+comp.lang.python at pearwood.info> wrote: > I'm seeing this annoying practice more and more often. Even for trivial > pieces of text, a few lines, people post screenshots instead of copying > the code. > > Where has this meme come from? It seems to be one which inconveniences > *everyone* involved: > > - for the sender, instead of a simple copy and paste, they have to take a > screen shot, possibly trim the image to remove any bits of the screen > they don't want to show, attach it to their email or upload it to an > image hosting site; > > - for the receiver, you are reliant on a forum which doesn't strip > attachments, or displays externally hosted images; the visually impaired > are excluded from using a screen reader; and nobody can copy or edit the > given text. > > It is as if people are deliberately inconveniencing themselves in order > to inconvenience the people they are asking to help them. > > With the exception of one *exceedingly* overrated advantage, namely the > ability to annotate the image with coloured lines and circles and > squiggles or other graphics (which most people don't bother to do), this > seems to me to be 100% counter-productive for everyone involved. Why has > it spread and why do people keep doing it? > > I don't want to be the old man yelling "Get Of My Lawn!" to the cool > kids, but is this just another sign of the downward spiral of programming > talent? Convince me that there is *some* justification for this practice. > Even a tiny one. > > (The day a programmer posts a WAV file of themselves reading their code > out aloud, is the day I turn my modem off and leave the internet forever.) > > > > -- > Steve > > -- > https://mail.python.org/mailman/listinfo/python-list > From vincent.vande.vyvre at telenet.be Wed Jan 31 03:48:13 2018 From: vincent.vande.vyvre at telenet.be (Vincent Vande Vyvre) Date: Wed, 31 Jan 2018 09:48:13 +0100 Subject: Package containing C sources (Posting On Python-List Prohibited) In-Reply-To: References: Message-ID: Le 31/01/18 ? 07:52, Victor Porton a ?crit?: > Lawrence D?Oliveiro wrote: > >> On Wednesday, January 31, 2018 at 6:13:00 PM UTC+13, Victor Porton wrote: >>> I am going to create a Python wrapper around a generally useful C >>> library. So the wrapper needs to contain some C code to glue them >>> together. >> Not necessarily. It?s often possible to implement such a wrapper entirely >> in Python, using ctypes . > But if I will find that I need C code, do I need to package it separately? > > So I would get three packages: the C library, the C wrapper for Python, and > the Python code. > > Can this be done with just two packages: the C library and C wrapper and > Python in one package? > Hi, You can made an all-in-one package without problems. Vincent From porton at narod.ru Wed Jan 31 03:53:36 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 10:53:36 +0200 Subject: Handle SIGINT in C and Python References: Message-ID: Victor Porton wrote: > I need to assign a real C signal handler to SIGINT. > > This handler may be called during poll() waiting for data. For this reason > I cannot use Python signals because "A Python signal handler does not get > executed inside the low-level (C) signal handler. Instead, the low-level > signal handler sets a flag which tells the virtual machine to execute the > corresponding Python signal handler at a later point(for example at the > next bytecode instruction)." > > I want after my signal handler for SIGINT executed to raise > KeyboardInterrupt (as if I didn't installed my own signal handler). > > Is this possible? How? I've found a solution: I can use PyOS_setsig() from Python implementation to get the old (Python default) OS-level signal handler, while I assign the new one. -- Victor Porton - http://portonvictor.org From porton at narod.ru Wed Jan 31 03:55:24 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 10:55:24 +0200 Subject: Handle SIGINT in C and Python (Posting On Python-List Prohibited) References: <7eaf9dd8-236d-43c1-815c-1757be126030@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > On Wednesday, January 31, 2018 at 8:58:18 PM UTC+13, Victor Porton wrote: >> For this reason I >> cannot use Python signals because "A Python signal handler does not get >> executed inside the low-level (C) signal handler. Instead, the low-level >> signal handler sets a flag which tells the virtual machine to execute the >> corresponding Python signal handler at a later point(for example at the >> next bytecode instruction)." > > Why is that a problem? As I already said, I need to handle the signal (SIGCHLD) while poll() waits for i/o. You can read the sources (and the file HACKING) of the C library which I am wrapping into Python here: https://github.com/vporton/libcomcom -- Victor Porton - http://portonvictor.org From tkadm30 at yandex.com Wed Jan 31 04:15:25 2018 From: tkadm30 at yandex.com (Etienne Robillard) Date: Wed, 31 Jan 2018 04:15:25 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> Message-ID: <0b7915c7-c967-0a79-10b5-65687db18eee@yandex.com> Le 2018-01-30 ? 16:38, Ned Batchelder a ?crit?: >> I'm confused by this: >> >> -if os.environ.get('SCHEVO_OPTIMIZE', '1') == '1': >> +if os.environ.get('SCHEVO_OPTIMIZE', '1') == True: >> > I was also curious about this: when does os.environ.get return > anything but a string? I was probably high when I coded this! ;) I think a better solution is to write : if os.environ.get('SCHEVO_OPTIMIZE', '0')? == '1': ?... Anyways, what do you think about the weakref case? Etienne -- Etienne Robillard tkadm30 at yandex.com https://www.isotopesoftware.ca/ From rosuav at gmail.com Wed Jan 31 04:36:36 2018 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 31 Jan 2018 20:36:36 +1100 Subject: PyPy support breaking CPython compatibility? In-Reply-To: <0b7915c7-c967-0a79-10b5-65687db18eee@yandex.com> References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> <0b7915c7-c967-0a79-10b5-65687db18eee@yandex.com> Message-ID: On Wed, Jan 31, 2018 at 8:15 PM, Etienne Robillard wrote: > > > Le 2018-01-30 ? 16:38, Ned Batchelder a ?crit : >>> >>> I'm confused by this: >>> >>> -if os.environ.get('SCHEVO_OPTIMIZE', '1') == '1': >>> +if os.environ.get('SCHEVO_OPTIMIZE', '1') == True: >>> >> I was also curious about this: when does os.environ.get return anything >> but a string? > > > I was probably high when I coded this! ;) > > I think a better solution is to write : > > if os.environ.get('SCHEVO_OPTIMIZE', '0') == '1': > ... > > Anyways, what do you think about the weakref case? > I think, without any real facts to justify it, that this sort of thing is *probably* an unintended compatibility break. So if you can show a minimal test-case that highlights the difference, you can probably raise an issue against PyPy and maybe get it fixed. But like I said, I don't have any facts to back that conjecture. ChrisA From nicholas.cole at gmail.com Wed Jan 31 04:47:43 2018 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Wed, 31 Jan 2018 09:47:43 +0000 Subject: Import statements and multiprocessing In-Reply-To: References: Message-ID: On Tue, Jan 30, 2018 at 7:26 PM, Terry Reedy wrote: > On 1/30/2018 10:54 AM, Nicholas Cole wrote: > >> I have a strange problem on python 3.6.1 > > [involving multiprocessing] Interestingly it seems to have been a very subtle circular import problem that was showing up only in multiprocessing, and which wasn't raising an exception at the time of the import. From ned at nedbatchelder.com Wed Jan 31 05:21:49 2018 From: ned at nedbatchelder.com (Ned Batchelder) Date: Wed, 31 Jan 2018 05:21:49 -0500 Subject: PyPy support breaking CPython compatibility? In-Reply-To: References: <5d3adcb3-a3ef-8534-0fad-2dc7f3e7993a@yandex.com> <10701171-14b1-7fef-09e7-87d5680e3385@nedbatchelder.com> Message-ID: <683c5cb5-f0b3-af46-7717-f806a4de900e@nedbatchelder.com> On 1/30/18 3:58 PM, Etienne Robillard wrote: > Hi Ned, > > > Le 2018-01-30 ? 15:14, Ned Batchelder a ?crit?: >> I'm curious what you had to change for PyPy? (Unless it's a Py2/Py3 >> thing as Chris mentions.) > > Please take a look at the changesets: > > https://bitbucket.org/tkadm30/libschevo/commits/745d1aeab5c6ee0d336790cf13d16f327e10c2f8 > > https://bitbucket.org/tkadm30/libdurus/commits/875636e9b6caa840fd50ca87d69217d87fc06f43 > > > In short, it seems PyPy automagically adds a __weakref__ attribute to > __slots__, causing the CPython interpreter to raise a TypeError... PyPy and CPython are separate implementations, that never mix: you run your code in one, or in the other.? How can PyPy do something at runtime (add a __weakref__ attribute to __slots__) that can cause CPython to do something at runtime (raise a TypeError)? A small demonstration case would be very helpful for figuring out what is going on. --Ned. From mythri.mroy at gmail.com Wed Jan 31 07:00:03 2018 From: mythri.mroy at gmail.com (user2301) Date: Wed, 31 Jan 2018 04:00:03 -0800 (PST) Subject: Install PyDev package or any other regular package during runtime in the python project Message-ID: <1511b84f-d901-4e1d-b217-174680c527a9@googlegroups.com> I am trying to dynamically install PyDev package at runtime. I have a PyDev project created in eclipse. I have a PyDev package on network location. Based on some configuration and input in my PyDev project, I want to install and load the PyDev package from network location. The classes contained in modules within this package will be used later. How can I install the regular(PyDev package) at runtime?. Is it possible to install the package at runtime? To which location I have to add this package ? project root directory or sys.path? What I had done: I had created the PyDev package inside my project root directory. Then import the module in the package dynamically. from pydoc import locate #package already created/installed within the project root def load(self): myclass = locate('package.module.classname') #importing module dynamically instance = myclass() what I need to do: from pydoc import locate #package not yet installed within the project root def load(self): # install the module first from given network path? how? myclass = locate('package.module.classname') #importing module dynamically instance = myclass() From renting at astron.nl Wed Jan 31 10:14:45 2018 From: renting at astron.nl (Adriaan Renting) Date: Wed, 31 Jan 2018 16:14:45 +0100 Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] In-Reply-To: <8b9a503a-1a70-b4ac-00d0-68021445f715@mrabarnett.plus.com> References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> <8b9a503a-1a70-b4ac-00d0-68021445f715@mrabarnett.plus.com> Message-ID: <5A71DD650200001B0001D70A@smtp1.astron.nl> Adriaan Renting | Email: renting at astron.nl Software Engineer Radio Observatory ASTRON | Phone: +31 521 595 100 (797 direct) P.O. Box 2 | GSM: +31 6 24 25 17 28 NL-7990 AA Dwingeloo | FAX: +31 521 595 101 The Netherlands | Web: http://www.astron.nl/~renting/ >>> On 30-1-2018 at 19:22, MRAB wrote: > On 2018-01-30 15:39, Steven D'Aprano wrote: >> On Tue, 30 Jan 2018 05:48:15 -0800, Rustom Mody wrote: >> >> [...] >>>> Ah, yes, the Dutch Reach. That would be like the French Pox (which >>>> isn't French), the Spanish Flu (that didn't start in Spain), the >>>> Jerusalem artichoke (which is neither an artichoke nor from Jerusalem), >>>> and the turkey (the bird, which has nothing to do with Turkey, the >>>> country). >>>> >>>> This technique is neither taught nor commonly used used by the Dutch: >>>> apparently some Americans decided that because the Netherlands has a >>>> lot of cyclists, they'll say its Dutch. >>> >>> reference please >> >> The onus should be on those who claim that the technique actually is used >> by the Dutch. Anecdotally, the Dutch people I've spoken to on the >> Internet had no idea that this technique even existed. >> > Do we know of anyone who's Dutch and frequents a Python newsgroup? :-) > I am Dutch and after googling the term, I can confirm that the "Dutch Reach" is taught in driving school here. I was taught this maneuvre when getting my licence 20 years ago. If it is actually used by a lot of people, I can't confirm. I use it most of the time, depending on what model car I'm driving. (wether the door handles are easy to reach/operate). The way I was taught, you have to check your mirrors and then use it to force you to check your blind spot, mostly to avoid cars hitting you/your door on the drivers side. I don't remember any specific mention of bicycles or use by passengers of this maneuvre. I think in general cars are a much larger danger than bicycles: - Faster: More likely have not been seen by you or not see you. - More mass: Damage much bigger. (yes also depends on the square of the velocity). - Worse perception. A Bicyclist usually has unrestricted hearing and vision around themselves, and is much more likely to have noticed the person in the car. - Bicyclists will understand they are vulnerable and be more careful. Motorbikes and mopeds are the hardest for me, as they are more likely to be missed due to their speed. And in the Netherlands, we largely solve this problem by just having everyone on a bike. ;-) (We do ride cars as well) If you want to know more about how the Dutch go about their cycling and images of Dutch infrastructure, then I suggest the BicycleDutch Youtube channel, for example this video: https://www.youtube.com/watch?v=swqaAIkGtpA The difference with a lot of other places, is that we use bicycles as a means of transport, not as an excercise/sports device. In general for distances < 15km (10 miles) and anything up to about the size of a couch. > [snip] From harnali70 at gmail.com Wed Jan 31 10:55:31 2018 From: harnali70 at gmail.com (harnali70 at gmail.com) Date: Wed, 31 Jan 2018 07:55:31 -0800 (PST) Subject: for info Message-ID: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> from where we learn python for free of cost. i am begineer in python.plzz help me From alister.ware at ntlworld.com Wed Jan 31 11:31:04 2018 From: alister.ware at ntlworld.com (alister) Date: Wed, 31 Jan 2018 16:31:04 GMT Subject: Where has the practice of sending screen shots as source code come from? References: <-LKdne7RMs6fcPDHnZ2dnUU7-aGdnZ2d@giganews.com> <8b1293fd-8a30-fa21-ae74-64bb2e61dbf1@gmail.com> Message-ID: On Tue, 30 Jan 2018 21:22:39 +0100, Jugurtha Hadjar wrote: > On 01/29/2018 03:48 PM, alister via Python-list wrote: >> On Mon, 29 Jan 2018 15:20:06 +0100, Jugurtha Hadjar wrote: >> >>> On 01/28/2018 04:43 PM, Skip Montanaro wrote: >>>> I've never been a Windows user, but at my current job, Windows is >>>> core to just about everything, so I am forced to use it for a lot of >>>> stuff (Outlook, SQL Server, Excel, etc). >>> I was hired at a startup which made a good impression on me and I was >>> eager to start working. I checked in for my first day, signed the >>> paperwork, then the CTO showed me around and told me "Here's your >>> laptop, you can install Windows and I'll check in with you later". >>> Life started draining out of my body and my mind was racing in all >>> directions before he even finished his sentence. I felt tricked: I was >>> hired based on a test and the file I've received was a *.tar.gz* and >>> it made me happy because I assumed they were a NIX shop.. >>> >>> I was considering how I should go about quitting gracefully. I have >>> stopped using Windows in 2008 for psychological reasons for it made me >>> really anxious, irritable, angry, and tense to the point of abusing >>> hardware with low kicks and punches which was not very healthy or >>> sane. It was only when I switched OS that this behavior stopped. There >>> was no way I would be going back to my bully. >>> >>> The CTO then said "Sorry.. I meant Ubuntu." and seeing my pale face, >>> he said in a reassuring tone "Don't be afraid, there are no Windows >>> machines here." which brought me back to life. >>> >>> I hope to be as brave as you some day. >> Any Vacancies whatever they do I am sure I can learn :-) >> >> >> >> > We're hiring all the good (technical and human side) people we can > find/afford. It's a small startup that helps its clients improve > business using machine learning. We're in Algiers, Algeria and Paris, > France. bit of a commute from Sevenage in UK -- A pencil with no point needs no eraser. From joel.goldstick at gmail.com Wed Jan 31 12:53:17 2018 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 31 Jan 2018 12:53:17 -0500 Subject: for info In-Reply-To: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> References: <38684a61-8145-48a5-b065-70d9aa43cdc2@googlegroups.com> Message-ID: On Wed, Jan 31, 2018 at 10:55 AM, wrote: > from where we learn python for free of cost. i am begineer in python.plzz > help me > -- > https://mail.python.org/mailman/listinfo/python-list > Start with python.org tutorial pages -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From porton at narod.ru Wed Jan 31 13:58:56 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 20:58:56 +0200 Subject: Help to debug my free library Message-ID: LibComCom is a C library which passes a string as stdin of an OS command and stores its stdout in another string. I wrote this library recently: https://github.com/vporton/libcomcom Complete docs are available at https://vporton.github.io/libcomcom-docs/ Now I am trying to make Python bindings to the library: https://github.com/vporton/libcomcom-python I use CFFI (API level). However testing my Python bindings segfaults in an unknown reason. Please help to debug the following: $ LD_LIBRARY_PATH=.:/usr/local/lib:/usr/lib:/lib \ PYTHONPATH=build/lib.linux-x86_64-2.7/ python test2.py Traceback (most recent call last): Segmentation fault (here libcomcom.so is installed in /usr/local/lib) -- Victor Porton - http://portonvictor.org From porton at narod.ru Wed Jan 31 14:09:57 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 21:09:57 +0200 Subject: Handle SIGINT in C and Python (Posting On Python-List Prohibited) References: <7eaf9dd8-236d-43c1-815c-1757be126030@googlegroups.com> <02ad997d-1ccf-4789-8f75-4bd60accc7f7@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > On Wednesday, January 31, 2018 at 9:55:45 PM UTC+13, Victor Porton wrote: >> Lawrence D?Oliveiro wrote: >> >>> On Wednesday, January 31, 2018 at 8:58:18 PM UTC+13, Victor Porton >>> wrote: >>>> For this reason I >>>> cannot use Python signals because "A Python signal handler does not get >>>> executed inside the low-level (C) signal handler. Instead, the >>>> low-level signal handler sets a flag which tells the virtual machine to >>>> execute the corresponding Python signal handler at a later point(for >>>> example at the next bytecode instruction)." >>> >>> Why is that a problem? >> >> As I already said, I need to handle the signal (SIGCHLD) while poll() >> waits for i/o. > > The usual behaviour for POSIX is that the call is aborted with EINTR after > you get the signal. That poll() is interrupted does not imply that Python will run its pythonic signal handler at the point of interruption. That is a problem. -- Victor Porton - http://portonvictor.org From rosuav at gmail.com Wed Jan 31 14:12:46 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Feb 2018 06:12:46 +1100 Subject: Help to debug my free library In-Reply-To: References: Message-ID: On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: > LibComCom is a C library which passes a string as stdin of an OS command and > stores its stdout in another string. Something like the built-in subprocess module does? ChrisA From porton at narod.ru Wed Jan 31 14:26:29 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 21:26:29 +0200 Subject: Help to debug my free library References: Message-ID: Chris Angelico wrote: > On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: >> LibComCom is a C library which passes a string as stdin of an OS command >> and stores its stdout in another string. > > Something like the built-in subprocess module does? I was going to write: "It seems that subprocess module can cause deadlocks. For example, if it first writes a long string to "cat" command input (going to read cat's stdout later), then "cat" would block because its output is not read while writing input." But after reading the docs it seems that Popen.communicate() does the job. Well, I first wrote in Java. For Java there was no ready solution. Later I switched to Python and haven't checked the standard libraries. So, please help me to make sure if Popen.communicate() is a solution for my problem (namely that it does not deadlock, as in "cat" example above). I am interested in both Python 2.7 and 3.x. > ChrisA -- Victor Porton - http://portonvictor.org From porton at narod.ru Wed Jan 31 14:41:29 2018 From: porton at narod.ru (Victor Porton) Date: Wed, 31 Jan 2018 21:41:29 +0200 Subject: Help to debug my free library References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: wxjmfauth at gmail.com wrote: > Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : >> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: >> > LibComCom is a C library which passes a string as stdin of an OS >> > command and stores its stdout in another string. >> >> Something like the built-in subprocess module does? >> >> ChrisA > > Do you mean the buggy subprocess module (coding of characters) ? Please elaborate: which bugs it has? in which versions? > Yes, there is a working workaround : QtCore.QProcess(). -- Victor Porton - http://portonvictor.org From rosuav at gmail.com Wed Jan 31 14:46:58 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Feb 2018 06:46:58 +1100 Subject: Help to debug my free library In-Reply-To: References: Message-ID: On Thu, Feb 1, 2018 at 6:26 AM, Victor Porton wrote: > Chris Angelico wrote: > >> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: >>> LibComCom is a C library which passes a string as stdin of an OS command >>> and stores its stdout in another string. >> >> Something like the built-in subprocess module does? > > I was going to write: "It seems that subprocess module can cause deadlocks. > For example, if it first writes a long string to "cat" command input (going > to read cat's stdout later), then "cat" would block because its output is > not read while writing input." > > But after reading the docs it seems that Popen.communicate() does the job. > > Well, I first wrote in Java. For Java there was no ready solution. Later I > switched to Python and haven't checked the standard libraries. > > So, please help me to make sure if Popen.communicate() is a solution for my > problem (namely that it does not deadlock, as in "cat" example above). > > I am interested in both Python 2.7 and 3.x. I believe communicate() is indeed safe for this. It does retain all data in memory, so with arbitrarily large output it may be better to use a pipe and read from it progressively (similarly if the program's going to run for a long time; communicate() will wait for the process to terminate before giving you any output), but it won't just deadlock. Even if what you want can't be done with communicate(), I would definitely recommend looking at the subprocess module. Worst case, you do in Python what your C library is doing: create pipes to communicate with the process. It's a lot easier in Python than in C, but the logic would be similar. Bonus: subprocess is available on all supported platforms (most notably Windows and POSIX, which are drastically different; there are a handful of Windows-only or Unix-only features, but the core is all identical, despite the implementation differing), and you can still support 2.7, although you have to forego the new and improved API in Python 3.5+. ChrisA From rosuav at gmail.com Wed Jan 31 14:48:03 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Feb 2018 06:48:03 +1100 Subject: Help to debug my free library In-Reply-To: References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: On Thu, Feb 1, 2018 at 6:41 AM, Victor Porton wrote: > wxjmfauth at gmail.com wrote: > >> Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : >>> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: >>> > LibComCom is a C library which passes a string as stdin of an OS >>> > command and stores its stdout in another string. >>> >>> Something like the built-in subprocess module does? >>> >>> ChrisA >> >> Do you mean the buggy subprocess module (coding of characters) ? > > Please elaborate: which bugs it has? in which versions? > >> Yes, there is a working workaround : QtCore.QProcess(). Ignore jmf, he wouldn't know a bug if he were eating it for dinner. His posts are blocked on the mailing list, and you'd do well to just plonk him in your newsreader. ChrisA From breamoreboy at gmail.com Wed Jan 31 15:01:15 2018 From: breamoreboy at gmail.com (breamoreboy at gmail.com) Date: Wed, 31 Jan 2018 12:01:15 -0800 (PST) Subject: Help to debug my free library In-Reply-To: References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: On Wednesday, January 31, 2018 at 7:41:50 PM UTC, Victor Porton wrote: > wxjmfauth at gmail.com wrote: > > > Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : > >> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton wrote: > >> > LibComCom is a C library which passes a string as stdin of an OS > >> > command and stores its stdout in another string. > >> > >> Something like the built-in subprocess module does? > >> > >> ChrisA > > > > Do you mean the buggy subprocess module (coding of characters) ? > > Please elaborate: which bugs it has? in which versions? > > > Yes, there is a working workaround : QtCore.QProcess(). > -- > Victor Porton - http://portonvictor.org You've now met the RUE, the Resident Unicode Expert. For years he's been stating that the Python 3 Flexible String Representation is buggy, to the extent that Python 3.6.2 is unusable on Windows. Strangely he cannot produce a shred of evidence to support his case so feel free to ignore him. -- Kindest regards. Mark Lawrence. From alister.ware at ntlworld.com Wed Jan 31 17:31:46 2018 From: alister.ware at ntlworld.com (alister) Date: Wed, 31 Jan 2018 22:31:46 GMT Subject: Help to debug my free library References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: On Thu, 01 Feb 2018 06:48:03 +1100, Chris Angelico wrote: > On Thu, Feb 1, 2018 at 6:41 AM, Victor Porton wrote: >> wxjmfauth at gmail.com wrote: >> >>> Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : >>>> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton >>>> wrote: >>>> > LibComCom is a C library which passes a string as stdin of an OS >>>> > command and stores its stdout in another string. >>>> >>>> Something like the built-in subprocess module does? >>>> >>>> ChrisA >>> >>> Do you mean the buggy subprocess module (coding of characters) ? >> >> Please elaborate: which bugs it has? in which versions? >> >>> Yes, there is a working workaround : QtCore.QProcess(). > > Ignore jmf, he wouldn't know a bug if he were eating it for dinner. His > posts are blocked on the mailing list, and you'd do well to just plonk > him in your newsreader. > > ChrisA I disagree jmf's posts are worth reading, if you find yourself agreeing with him you know you have got something wrong. -- Bernard Shaw is an excellent man; he has not an enemy in the world, and none of his friends like him either. -- Oscar Wilde From rosuav at gmail.com Wed Jan 31 17:39:05 2018 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 1 Feb 2018 09:39:05 +1100 Subject: Help to debug my free library In-Reply-To: References: <6fcc2db7-649f-4987-86df-6a73c21c91d7@googlegroups.com> Message-ID: On Thu, Feb 1, 2018 at 9:31 AM, alister via Python-list wrote: > On Thu, 01 Feb 2018 06:48:03 +1100, Chris Angelico wrote: > >> On Thu, Feb 1, 2018 at 6:41 AM, Victor Porton wrote: >>> wxjmfauth at gmail.com wrote: >>> >>>> Le mercredi 31 janvier 2018 20:13:06 UTC+1, Chris Angelico a ?crit : >>>>> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton >>>>> wrote: >>>>> > LibComCom is a C library which passes a string as stdin of an OS >>>>> > command and stores its stdout in another string. >>>>> >>>>> Something like the built-in subprocess module does? >>>>> >>>>> ChrisA >>>> >>>> Do you mean the buggy subprocess module (coding of characters) ? >>> >>> Please elaborate: which bugs it has? in which versions? >>> >>>> Yes, there is a working workaround : QtCore.QProcess(). >> >> Ignore jmf, he wouldn't know a bug if he were eating it for dinner. His >> posts are blocked on the mailing list, and you'd do well to just plonk >> him in your newsreader. >> >> ChrisA > > > I disagree jmf's posts are worth reading, if you find yourself agreeing > with him you know you have got something wrong. > Heh. That would be useful if he ever actually posted anything other than "Python is buggy". I'm pretty sure I've already read everything he actually has to say. ChrisA From nad at python.org Wed Jan 31 20:34:12 2018 From: nad at python.org (Ned Deily) Date: Wed, 31 Jan 2018 20:34:12 -0500 Subject: [RELEASE] Python 3.7.0b1 is now available for testing Message-ID: <9425596C-A92F-4B10-A8B7-98F4E827E8D0@python.org> On behalf of the Python development community and the Python 3.7 release team, I'm happy to announce the availability of Python 3.7.0b1. b1 is the first of four planned beta releases of Python 3.7, the next major release of Python, and marks the end of the feature development phase for 3.7. You can find Python 3.7.0b1 here: https://www.python.org/downloads/release/python-370b1/ Among the new major new features in Python 3.7 are: * PEP 538, Coercing the legacy C locale to a UTF-8 based locale * PEP 539, A New C-API for Thread-Local Storage in CPython * PEP 540, UTF-8 mode * PEP 552, Deterministic pyc * PEP 553, Built-in breakpoint() * PEP 557, Data Classes * PEP 560, Core support for typing module and generic types * PEP 562, Module __getattr__ and __dir__ * PEP 563, Postponed Evaluation of Annotations * PEP 564, Time functions with nanosecond resolution * PEP 565, Show DeprecationWarning in __main__ * PEP 567, Context Variables Please see "What?s New In Python 3.7" for more information. Additional documentation for these features and for other changes will be provided during the beta phase. https://docs.python.org/3.7/whatsnew/3.7.html Beta releases are intended to give you the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. We strongly encourage you to test your projects with 3.7 during the beta phase and report issues found to https://bugs.python.org as soon as possible. While the release is feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2018-05-21). Our goal is have no ABI changes after beta 3 and no code changes after rc1. To achieve that, it will be extremely important to get as much exposure for 3.7 as possible during the beta phase. Attention macOS users: with 3.7.0b1, we are providing a choice of two binary installers. The new variant provides a 64-bit-only version for macOS 10.9 and later systems; this variant also now includes its own built-in version of Tcl/Tk 8.6. We welcome your feedback. Please keep in mind that this is a preview release and its use is not recommended for production environments. The next planned release of Python 3.7 will be 3.7.0b2, currently scheduled for 2018-02-26. More information about the release schedule can be found here: https://www.python.org/dev/peps/pep-0537/ -- Ned Deily nad at python.org -- [] From steve+comp.lang.python at pearwood.info Wed Jan 31 22:07:49 2018 From: steve+comp.lang.python at pearwood.info (Steven D'Aprano) Date: Thu, 1 Feb 2018 03:07:49 +0000 (UTC) Subject: [OT] Dutch Reach [was Re: Where has the practice of sending screen shots as source code come from?] References: <21fe131d-d34a-4887-9648-dcf3d8b03e80@googlegroups.com> <1f87b241-6e51-45a9-a492-7c22506235cb@googlegroups.com> <8b9a503a-1a70-b4ac-00d0-68021445f715@mrabarnett.plus.com> <5A71DD650200001B0001D70A@smtp1.astron.nl> Message-ID: On Wed, 31 Jan 2018 16:14:45 +0100, Adriaan Renting wrote: > I am Dutch and after googling the term, I can confirm that the "Dutch > Reach" is taught in driving school here. I was taught this maneuvre when > getting my licence 20 years ago. Thanks for the data point. Was it a requirement of the driving test? > If it is actually used by a lot of people, I can't confirm. I use it > most of the time, depending on what model car I'm driving. (wether the > door handles are easy to reach/operate). > > The way I was taught, you have to check your mirrors and then use it to > force you to check your blind spot, But it doesn't force you to do anything - reaching over to the door handle with your opposite hand doesn't make you look over your shoulder. And given that you are stopped, any vehicle travelling in your blind spot is going to only be there for a fraction of a second. If you are taking a genuine careful look for traffic, rather than a quick careless glance, you'll see any vehicle in your blind spot because it won't be in the blind spot for long. > mostly to avoid cars hitting > you/your door on the drivers side. Yes, this is also a big problem. I used to work with somebody who came within a fraction of a centimetre of taking a woman's arm off when she suddenly opened the door to her car as he drove past in a narrow road. He took the door completely off. -- Steve From porton at narod.ru Wed Jan 31 23:54:25 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 06:54:25 +0200 Subject: Help to debug my free library References: Message-ID: Dennis Lee Bieber wrote: > On Wed, 31 Jan 2018 20:58:56 +0200, Victor Porton > declaimed the following: > >>LibComCom is a C library which passes a string as stdin of an OS command >>and stores its stdout in another string. >> >>I wrote this library recently: >>https://github.com/vporton/libcomcom >> >>Complete docs are available at >>https://vporton.github.io/libcomcom-docs/ >> >>Now I am trying to make Python bindings to the library: >>https://github.com/vporton/libcomcom-python >> > > Debug -- no help, I'm not a fluent Linux programmer... > > But based upon the description of this library, I would have to ask: > > "What does this library provide that isn't already in the Python standard > library?" > > "Why would I want to use this library instead of, say > subprocess.Popen().communicate()?" (or the older Popen* family) > > {Though .communicate() is a one-shot call -- sends one packet to > subprocess' stdin, reads to EOF, and waits for subprocess to terminate. If > one needs interactive control, one needs explicit write/read calls, > although those can deadlock if the caller and subprocess aren't written > for such interaction} If it "sends one packet", this would lead to a deadlock (even for "cat" command). Hopefully, you are wrong here. I found that Popen.communicate() seems to solve my problem. (Previously I programmed in Java and found no native solution. For this reason I created my LibComCom.) -- Victor Porton - http://portonvictor.org From porton at narod.ru Wed Jan 31 23:57:48 2018 From: porton at narod.ru (Victor Porton) Date: Thu, 01 Feb 2018 06:57:48 +0200 Subject: Handle SIGINT in C and Python (Posting On Python-List Prohibited) References: <7eaf9dd8-236d-43c1-815c-1757be126030@googlegroups.com> <02ad997d-1ccf-4789-8f75-4bd60accc7f7@googlegroups.com> <10d4b0b7-1de8-4874-b723-32d058cfc006@googlegroups.com> Message-ID: Lawrence D?Oliveiro wrote: > On Thursday, February 1, 2018 at 8:10:24 AM UTC+13, Victor Porton wrote: >> Lawrence D?Oliveiro wrote: >> >>> The usual behaviour for POSIX is that the call is aborted with EINTR >>> after you get the signal. >> >> That poll() is interrupted does not imply that Python will run its >> pythonic signal handler at the point of interruption. That is a problem. > > * Python calls poll() > * poll() aborted with EINTR > * Python runs your signal handler > > Versus native C code: > > * your code calls poll() > * poll() aborted with EINTR > * your signal handler is run > > Where is there a fundamental difference? I meant to call poll() from C code, not Python code. In this case when poll() is aborted with EINTR, the pythonic signal handler does not run. -- Victor Porton - http://portonvictor.org